No need to install this package to non-vagrant system.
Do not add this package to Sipwise-grml image - it's too heavy (86M)
and not needed on real systems.
Change-Id: I9ec9ff76d588f4ced30ba199f05bb167eec5288a
Previously the functions and the code were fixed so it was hard to
understand the flow of execution.
Move all functions to a single section at the beginning of the script
so it's easier to understand the code.
Fix shellcheck warnings:
SC2086: Double quote to prevent globbing and word splitting.
SC2154: efidev1 is referenced but not assigned.
Change-Id: Ie4bf28c166e4a9ff236eff807ee97adae6ecddd0
This variable is used in some ngcpcfg *.services file.
Specifically 'ngcp-provisioning-tools/mysql_values.cfg.services' uses
this variable to specify mysql db host and if it's necessary to reload
kamailio.
Change-Id: Ibf137cdd0ad6f6492a30cfa715c468e4ac22832f
Initially wanted to sync with https://github.com/grml/grml-network/ as
of commit 49409a5587d, though templates/scripts/includes/netcardconfig
existed only to provide the VLAN patch until we had this available from
upstream and our own ISO. Now that everything has been upstreamed and we
have up2date Grml ISOs, let's switch to the one included in upstream
Grml.
Change-Id: I685cd0ad95033ad2b50036e53c2ee9814a776e63
Instead of having to maintain this ISO in our web server off-band,
we switch to use the packaged version, which makes validation
unnecessary as apt gives us that. And it also gives us a newer
version (currently 6.0.4 with Debian buster 10.3 vs the old 5.2.26).
Change-Id: Id89280bbe7fadeb35d391b5dc46e930935017588
From the packages' deps and content POV Pro and Carrier packages are
equal to each other so get rid of this separation.
Change-Id: I587dba3147bdc9b3c0f10da820ba4cdc8a0a0f08
This reverts commit ee2398b6f8.
We've removed mcollective from our infrastructure and ready to update
puppet to the latest available version.
Change-Id: I7a860be9e051ed09bd69c165aaafc2acd1f17cd8
In the past we were associating them by MAC address and only in some cases, like
Pro/Carrier and in Virtual Machine environments.
However, it's beneficial to have this in other scenarios, so we will create udev
rules to pin them down during deployment.
Change-Id: Ic1b397e81af673b974961158a0e9a05ce5b80a69
Initially dirmngr package was installed to allow running apt-key inside
deployment.sh to install puppetlabs key. Then after moving puppetlabs
key to local file package installation was removed. But we still need it
for successful first puppet run that also uses apt-key. Otherwise there
is the error: 'Could not find a suitable provider for apt_key'. Puppet
also installs dirmngr package but in such case only the second puppet run
will be successful so we need to install the package before the very
first puppet run.
Change-Id: I28edbd8c91c841074ac1b3f1eb6df16e14daa084
In deployment.sh line 2109:
local puppet_gpg='/root/puppet.gpg'
^-- SC2168: 'local' is only valid in functions.
Change-Id: I628cbac844db6aa1913ab1747906656d7d6d739b
The puppet.gpg is located in Grml-sipwise so no need to download the
hard-coded key but use the local one.
Change-Id: Id57180de96efef4a5a13086e7807c615bd65b886
In the event of network blips, HA switchovers or failure of the servers, there
might be a problem downloading some of the hundreds or thousands of packages
needed for installation.
Set the option to retry 3 times in the case of such failures, to try to minimise
their impact.
This has to be done in several places because mmdebstrap (which uses apt) and
apt-get are invoked at different points.
Change-Id: I5acbd9895fa37452026e582c241ced945fbba3d7
The boot parameter 'ngcpstatus' is used to wait the number of seconds
before ending process so the external system can retrieve the final
status from 4242 port.
Add the value of this parameter as STATUS_WAIT_SECONDS option to
ngcp-initial-configuration config so external system can track the
configuration stage also.
Change-Id: I68d9c65a1cc96bf581305e92d8b9b6bfc34e7ed2
mmdebstrap and debootstrap install any "important" packages in the
standard installation variants, pulling in e.g. libgdbm5 (which was of
priority `important` until #890832 was fixed and it was downgraded to
priority `optional` with libgdbm6).
Same for libusb-1.0-0 vs libusb-0.1-4:amd64:
% apt-cache show libusb-1.0-0 | grep Priority
Priority: optional
% apt-cache show libusb-0.1-4:amd64 | grep Priority
Priority: important
Resulting in having libusb-0.1-4 *and* libusb-1.0-0 installed on
our NGCP systems, while libusb-0.1-4 is completely unneeded and irrelevant.
So this is unnecessary, and all the other packages that are automatically
pulled in via the standard installation should be handled via according
and explicit dependencies elsewhere, JFTR - currently being in
Debian/buster:
| apt-utils bsdmainutils cpio cron debconf-i18n dmidecode dmsetup
| e2fsprogs gdbm-l10n ifupdown init iproute2 iptables iputils-ping
| isc-dhcp-client isc-dhcp-common kmod less libapparmor1 libapt-inst2.0
| libargon2-1 libbsd0 libcap2 libcap2-bin libcom-err2 libcryptsetup12
| libdevmapper1.02.1 libdns-export1104 libelf1 libestr0 libext2fs2
| libfastjson4 libidn11 libip4tc0 libip6tc0 libiptc0 libisc-export1100
| libjson-c3 libkmod2 liblocale-gettext-perl liblognorm5 libmnl0
| libncurses6 libnetfilter-conntrack3 libnewt0.52 libnfnetlink0 libnftnl11
| libpopt0 libprocps7 libslang2 libss2 libtext-charwidth-perl
| libtext-iconv-perl libtext-wrapi18n-perl libxtables12 logrotate lsb-base
| mount nano netbase procps readline-common rsyslog sensible-utils systemd
| systemd-sysv tasksel tasksel-data tzdata udev vim-common vim-tiny
| whiptail xxd
The essential variant lacks apt(-get), while the minbase variant provides
the same base as essential, *plus* apt so we can actually install
further packages, so that's what we're using.
We pull in systemd + systemd-sysv + init explicitly to force its usage,
ngcp-installer is complaining and prompting otherwise.
We pull in isc-dhcp-client + ifupdown to have a working networking/DHCP
setup after reboot, since we don't use systemd-networkd or alike (yet).
Applying this change will result in lacking the following packages in
our Debian/stretch based trunk builds:
* apt-utils
* blends-tasks
* debconf-i18n
* iputils-ping
* isc-dhcp-common
* libapt-inst2.0
* liblocale-gettext-perl
* libtext-charwidth-perl
* libtext-iconv-perl
* libtext-wrapi18n-perl
* libusb-0.1-4
* logrotate
* tasksel
* tasksel-data
* vim-tiny
The ones we want to have available will be handled via explicit
dependencies in NGCP (meta)packages.
Change-Id: I14dac92d99172cf792a0334601a930ce6698dc83
This variable tells ngcp-installer that it is run from
deployment.sh.
In this case we need it to skip CE warning in ngcp-installer:
===================
This installation script is not intended to run in a shared system,
as it can add/delete/update existing configurations.
Please run this script only in a base install of 64 bit Debian 9 (stretch)
on a dedicated server.
Do you want to continue with the installation process? (y/N):
===================
Change-Id: I9a33ccbe07332a09a64c98a2bef844fa95498c05
By default we should not skip any confirmation messages. In automation
setup we need just change it to 'yes' before
ngcp-initial-configuration.
Change-Id: I1ca69aaa1e19f8e34b82434d99b50e41add74f6f
apt > 1.6 started creating this directory, but older apt versions do not
know how to clean it up. When using a newer host apt than the one that
will be used in the chroot, the chroot apt will be unable to remove it
properly, and will emit a warning.
mmdebstrap was supposedly fixed in version 0.4.0-1, but the code was
inserted before further «apt-get update» calls, so the directory gets
regenerated. We workaround this here for now, after having run
grml-debootstrap which might be calling mmdebstrap depending on the
release.
Change-Id: I13ad1b7ecc9f60414ab7e9222bada10d09baa2ab
Otherwise the servers with Software RAID cannot be
installed due to the error on stage 'grub-install "/dev/$disk"':
> grub-install: error: /usr/lib/grub/i386-pc/modinfo.sh doesn't exist. Please specify --target or --directory.
The package grub-pc is a NGCP dependency, so it is available
when we install NGCP but missing when we recovery server using Puppet
or install Debian plain on Software RAID.
Change-Id: I46caf893d2d6c523b6c007702d21f6b81caac291
When bootstrapping into a tmpfs (running inside a VirtualBox VM using a
Grml-Sipwise ISO), debootstrap takes 2min 17seconds, while mmdebstrap
takes only ~20 seconds. This is a notable change that's worth using it
by default.
Quoting mmdebstrap's package description:
| Downloads, unpacks and installs Debian packages to either directly create a
| directory which can be chrooted into, or a tarball of it. In contrast to
| debootstrap it uses apt, supports more than one mirror, automatically uses
| security and updates mirrors for Debian stable chroots, is 3-6 times faster,
| produces smaller output by removing unnecessary cruft, is bit-by-bit
| reproducible if $SOURCE_DATE_EPOCH is set, allows unprivileged operation using
| Linux user namespaces, fakechroot or proot and can setup foreign architecture
| chroots using qemu-user.
Further differences noted between debootstrap + mmdebstrap:
* debootstrap requires exec + dev permissions on the target,
while mmdebstrap doesn't need them (being a good thing, actually)
* mmdebstrap pulls in gnupg1/gpgv1 on the target system (stretch-only),
while debootstrap doesn't
* debootstrap leaves the Debian packages in /var/cache/apt/archives behind,
while mmdebstrap doesn't
* debootstrap leaves the Debian repository files in /var/lib/apt/lists behind,
while mmdebstrap doesn't
* mmdebstrap doesn't consider apt, debconf as Priority 'required' but as 'important',
gcc-8-base, libacl1, libattr1 + zlib1g as 'required' instead of 'optional',
libbz2-1.0 + libpcre3 as 'important' instead of 'optional';
libdb5.3 + libtasn1-6 as 'standard' instead of 'optional'
None of those issues should cause any issues for us, though.
Change-Id: I93616263c2fed45ab8063fce024b98a7c6272660
The version 5.2.18 building is very slow on the recent
Debian stretch kernel 4.9.0-8-amd64:
VBoxGuestAdditions version 5.2.18:
> 12:32:52 (netscript.grml:1854): vagrant_configuration(): grml-chroot /mnt /media/cdrom/VBoxLinuxAdditions.run --nox11
> Writing /etc/debian_chroot ...
> Verifying archive integrity... All good.
> Uncompressing VirtualBox 5.2.18 Guest Additions for Linux........
> VirtualBox Guest Additions installer
> Copying additional installer modules ...
> Installing additional modules ...
> VirtualBox Guest Additions: Building the VirtualBox Guest Additions kernel modules. This may take a while.
> VirtualBox Guest Additions: Starting.
> VirtualBox Guest Additions: Building the VirtualBox Guest Additions kernel modules. This may take a while.
> +12:39:15 (netscript.grml:1856): vagrant_configuration(): ...
VBoxGuestAdditions version 5.2.26:
> +13:35:50 (netscript.grml:1854): vagrant_configuration(): grml-chroot /mnt /media/cdrom/VBoxLinuxAdditions.run --nox11
> Writing /etc/debian_chroot ...
> Verifying archive integrity... All good.
> Uncompressing VirtualBox 5.2.26 Guest Additions for Linux........
> VirtualBox Guest Additions installer
> Copying additional installer modules ...
> Installing additional modules ...
> VirtualBox Guest Additions: Building the VirtualBox Guest Additions kernel
> modules. This may take a while.
> VirtualBox Guest Additions: To build modules for other installed kernels, run
> VirtualBox Guest Additions: /sbin/rcvboxadd quicksetup <version>
> VirtualBox Guest Additions: Building the modules for kernel 4.9.0-8-amd64.
> VirtualBox Guest Additions: Starting.
> Running in chroot, ignoring request: daemon-reload
> +13:39:12 (netscript.grml:1856): vagrant_configuration():
5.2.26 is the latest stable version from VirtualBox,
reported as such by upstream (see
https://download.virtualbox.org/virtualbox/LATEST-STABLE.TXT)
Change-Id: Ieb4b158344b3e4d0bf2719e8897cdfcdf133082b
Instead of runtime compiling during the installation compile this lib
in package building and deliver as part of the package.
Change-Id: Ic97adb0c958c57976ac5d23974b0efc306ccb326
This is the same solution for a similar problem and that it was implemented in
5ab1f5418a, but extending it to other parts of the
code that did not wait-and-retry.
Apparently commands like "blockdev --flushbufs" (flush buffers of block devices)
do not solve the situation in all cases, so this is a more foolproof --if
inelegant-- solution that should not slow down the deployment more than a few
seconds at most.
Change-Id: If74e134262475ab0b100981f94fa310536f0a7ab
In newer systems it can be under ngcp-data partition (/ngcp-data/home/sipwise),
in older systems without this partition it can be under /var/sipwise.
Also this way is more future-proof, if the location changes again.
Change-Id: If2d3a3b55ea81871071bf846c8ca981e703d3d88
HOSTNAME env variable is not set from 'ip=' but seems to be set
outside of deployment.sh script and exported to it. HOSTNAME
variable is set differently if newer grml20181230 is set in
dnsmasq dhcp.conf. We have 'ip=' option where we explicitly set
hostname for the host so let's use it (instead of uncontrolled
HOSTNAME variable) for puppet installation case.
Change-Id: I3fa2cc7ec982b270302d2d0940d6477b666eaf5c
GRML 2018.12 adds 'iface lo inet dhcp' line to /etc/network/interfaces
which is used in stretch system. This line breaks networking service on
boot so it isn't properly restarted in system_restart_network function
so network configuration is not complete.
Change-Id: I5e2ec763fea7db6f605e87b171514a985b0de621
This part is the installation of the packages in GRML system which is
testing/buster now so we need to use this debian name in source list.
Change-Id: I417065021bb08b704bf614181f68187705e09f8b
We seem to be hitting https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=918590
which causes installations to take "forever" as soon as LVM is involved, spitting
messages like:
| WARNING: Device /dev/loop0 not initialized in udev database even after waiting 10000000 microseconds.
grml-scripts >=2.8.4 + grml-debootstrap >=0.86 provide workarounds for this,
so when we're installing Debian/buster then make sure we have
recent versions of grml-scripts (providing grml-chroot tool)
and grml-debootstrap available.
Change-Id: I07409790989445a0a30b7373a37bee3bda47ae89
The option INTERNAL_NETMASK is set to either passed parameter
ngcpnetmask or to DEFAULT_INTERNAL_NETMASK value so there is no sense to
pass this variable to ngcp-initial-configuration.
Change-Id: I9d5f4aa72534286b7a2db9d28db42435eaf49fb3
This reverts commit 805bfcbabb.
Debian's virtualbox 5.2.18-dfsg-2 (which provides a working
virtualbox-guest-dkms) included a fix to build against kernel 4.18,
nowadays virtualbox 5.2.22-dfsg-2 is available in Debian/testing
(AKA upcoming buster) repository.
Change-Id: Id8ee9d5d646a46275447e0e5ea4e9f7d14588962
This is an ongoing discussion within the Debian project, see
https://wiki.debian.org/UsrMerge. As far as we know it won't give *us*
any benefits, while it's a one-way decision - so once we'd have a merged
/usr we can't back. We decided to force disabling it therefore.
While at it kill the unused $EXTRA_DEBOOTSTRAP_OPTS variable, which is
not present anymore. I tracked this down to a change in netscript.git
back in 2011 (see commit 6ec9cb274), where we used this for assigning it
to '--pre-scripts /etc/debootstrap/pre-scripts --keep_src_list'. This is
no longer existing as such, so drop this unused variable.
JFTR, debootstrap lists the `--merged-usr` option in its `--help` output
starting with debootstrap v1.0.89, though with v1.0.102 it was changed
to `--no-merged-usr`. We could either check for that or use `grep -q --
--no-merged-usr /usr/sbin/debootstrap`. But the option is available
since debootstrap v1.0.83 and in Debian/stretch we have debootstrap
v1.0.89 available, so enable the option by default.
Change-Id: I3e115afbb9095b0f36d45e8b69e7aeb89e6e7dbe
In order to detect if it is necessary to run init or join actions
during the initial configuration JOIN_CLUSTER option is used so
there is no sense to pass RETRIEVE_MGMT_CONFIG to it.
Change-Id: I89c0bf9a96511b2de5e1bf53acd0f631a32f7429
Make it enabled by default.
This option is needed only for initial configuration tools so
we can safely change it here.
Since we use the same way of installation as Carrier for Pro we
need this option in Pro config also.
Change-Id: Id72cb92c2b808143c9380dc23160574061c6c225
For Pro installation from another node we use internal network for
installation and we do not need to configure gateway on internal interface.
Change-Id: I8e95851c1b2132728e8846cf0c1d0d5149d06d74
Updating to Virtualbox 5.2.18 is a requirement but not enough
yet to build against Debian/buster, as building its vboxsf is
failing:
| /tmp/vbox.0/utils.c: In function ‘sf_init_inode’:
| /tmp/vbox.0/utils.c:165:28: error: passing argument 1 of ‘sf_ftime_from_timespec’ from incompatible pointer type [-Werror=incompatible-pointer-types]
| sf_ftime_from_timespec(&inode->i_atime, &info->AccessTime);
| ^
| /tmp/vbox.0/utils.c:53:13: note: expected ‘struct timespec *’ but argument is of type ‘struct timespec64 *’
Debian's virtualbox 5.2.18-dfsg-2 (which provides a working
virtualbox-guest-dkms) includes a fix to build against kernel 4.18,
see:
https://tracker.debian.org/news/983571/accepted-virtualbox-5218-dfsg-2-source-into-unstable/
We don't use Debian's virtualbox-guest-dkms though. Instead let's
patch the sources until upstream provides an ISO which includes the
according change for compiling vboxsf against more recent kernel
versions.
I adapted the VirtualBox-kernel-4.18.patch, based on the relevant
change from Debian::
b00c7b4d53/debian/patches/kernel-4.18.patch
NOTE: We need to patch the source after it was installed by
`/media/cdrom/VBoxLinuxAdditions.run --nox11`. If we'd invoke
`/media/cdrom/VBoxLinuxAdditions.run` again it would overwrite our
modified sources. Instead patch the source and directly invoke the
relevant steps to compile and install the module.
Change-Id: Iea86c07009838dea5b42af91cbfb0dc233179533
5.2.18 is the latest stable version from VirtualBox,
reported as such by upstream (see
https://download.virtualbox.org/virtualbox/LATEST-STABLE.TXT)
as well as present in current stretch-backports.
Change-Id: Iff5cb0e8ada2419aebbd8b26a507d09d8537ae92
By default we do assumption that we are installing to standard block devices,
this is not working for NVMe hardware name spaces.
Here we try to detect underlying device for named partition.
Change-Id: I7d2ea339a3aee2a8458a72cfc392441721d350c7
We perform all packages installation on 'root' partition.
Some packages, like faxserver for now, want to create
something on 'data' partition. They do it successfully in
Debian maintainer files (like postinst), so the 'data'
partition has to be mounted properly, otherwise the data
will be gone on reboot.
Also we need to unmount it properly at the end of installation.
P.S. we need to call mount under grml-chroot as chroot fails:
> (netscript.grml:1353): main(): chroot /mnt mount /ngcp-data
> mount: /ngcp-data: mount failed: Unknown error -1
Change-Id: Ief90d380d71ea6e0a64eba45f403465989865952
Starting with the mr6.5 release we no longer set up swap partition/LV via
deployment.sh (see deployment-iso commit 6a92f155).
Instead, the system will now use swapfiles, which are easier for operations like
changing the size dynamically or removing them completely.
This change implements the configuration for a swapfile during deployment,
enabled by default and with 1/2 the size of the main memory, limited between 4
and 16GB. The file itself will be created later, during the initial NGCP
configuration phase.
Change-Id: I09a5a77ecfed3924184fcc7c84c6b6b21dcacc98
NGCP software is fully functional without '/ngcp-fallback',
the boot process should NOT be aborted on the errors here.
'/ngcp-fallback' is necessary for better usability of the system
and during upgrade where we will check mount point in RW state anyway.
Also tune layout here to match with spaces count to '/ngcp-data'.
Change-Id: Ic63f87e38ee5e8582060e9ca515492274ba8c55b
It is necessary to test Carrier installations on real hardware,
where we have partitions/RAID configs from the previous tests.
Change-Id: Ifec5ab28ee365737775ef883d152049bc5733b2b
We need to store FS size into config.yml to be able to tune iPXE setup
to install second+ Carrier node on internal Proxmox-based test
Carrier installation (as Jenkins control web01a node there only).
Change-Id: I03db9702e540b7e90b564645e9933316c08f3b59
We need an ability to install all our products in a limited
HW environment and also be able to pack them into different boxes
(like Vagrant, VirtualBox, VmWare, Docker).
The new boot option 'fallbackfssize' gives us ability to manage
the virtual VMs 'fallback' partition size.
Also mount '/ngcp-fallback' in read-only mode by default,
it is very useful to have access to the old 'code' partition.
Rename '/ngcpdata' to '/ngcp-data' for better readability and
matching to 'ngcp-fallback' ('ngcpfallback' is hardly readable).
Change-Id: I512d65254d2f163482734d94cfc36190fb297d8e
Otherwise PRO/Carrier cannot be installed because NGCP
depends on a lot of huge packages including:
* NGCP CloudPBX firmwares ~1Gb ad for mr6.5 and growing
* GRML squashfs ~300Mb
* Sipwise prompts ~400Mb
At the moment of installing all the packages are downloaded
into apt cache (~2.9Gb in total as for mr6.5) + have to be
installed/unpacked (~3.2Gb in total as for mr6.5).
All-to-all I was not able to install Carrier web01a node
even with ROOTFS_SIZE=6Gb. I had to increase it till '7Gb'
to install Carrier successfully.
Lets use 10GB for root FS to have free space protection here.
The possible options were checked:
* move apt cache into RAM, has some disadvantages, like
we need 4GB+ RAM for cache only => hard to test on Jenkins
* move apt cache to 'data' partition is also bad idea as
it is a 'code' cache so have to be consistent with 'code'
(so it is release specific and must be stored on 'root'/'fallback')
Change-Id: Ia34dcc15230e152e894e813dfbaa8688a5145de1
Mcollective is deprecated in v6.0 and I don't see
any quick fix for that. So let's keep using v5.5
while deciding how to proceed further.
Change-Id: Ie72e5a57afc0a7235f9d7d29d52af7d796db1ea8
The users can assign disks using the full path:
'swraiddisk1=/dev/sda' instead of expected 'swraiddisk1=sda'.
As a result the installation fails with not a user friendly errors.
Let's be more user friendly here and remove initial '/dev/' is available.
Change-Id: I094814433b294ca8f084b0412cb9b67537cba8cf
Assuming we have two disks /dev/sda + /dev/sdb and boot the
system with the (new) boot options "swraiddisk1=sda
swraiddisk2=sdb", then we will get a SW-RAID (Software RAID)
setup which looks like (the /boot/efi is only present with EFI
support and only present for the active partition):
| NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
| sda 8:0 0 16G 0 disk
| |-sda1 8:1 0 1M 0 part
| |-sda2 8:2 0 486M 0 part /boot/efi
| `-sda3 8:3 0 15.5G 0 part
| `-md0 9:0 0 15.5G 0 raid1
| |-ngcp-root 253:0 0 5G 0 lvm /
| |-ngcp-fallback 253:1 0 5G 0 lvm
| `-ngcp-data 253:3 0 5G 0 lvm /ngcpdata
| sdb 8:16 0 16G 0 disk
| |-sdb1 8:17 0 1M 0 part
| |-sdb2 8:18 0 486M 0 part
| `-sdb3 8:19 0 15.5G 0 part
| `-md0 9:0 0 15.5G 0 raid1
| |-ngcp-root 253:0 0 5G 0 lvm /
| |-ngcp-fallback 253:1 0 5G 0 lvm
| `-ngcp-data 253:3 0 5G 0 lvm /ngcpdata
The /dev/md0 resides in the 3rd partition, where we usually have
plain LVM. /dev/md0 is RAID level 1, on top of the two disks.
Notes:
* we hardcode /dev/md0 as raid-device for mdadm/SW-RAID
* existing data on EFI partitions will be re-used (as grml-debootstrap
won't overwrite the partition it if contains a filesystem already)
* neither the BIOS (legacy, being first partition) nor the UEFI
partition (being the second partition) can be part of the SW-RAID
setup (see https://wiki.debian.org/UEFI -> "RAID for the EFI
System Partition"). Therefore we need to make sure, that the BIOS
+ UEFI partitions are kept in sync, or at least provide data as
needed. We will have to handle this during e.g. upgrades +
re-installations.
* we need to invoke grub-install for both disks, otherwise
GRUB is available only from the first disk
* install gdisk package to have sgdisk binary available (missing
on grml-small)
Change-Id: I6ebe2c25326c2fdf9ed8fe2bd7b9bf540c7689fd
Requested features:
* switch from msdos partition table to GPT
* (optional) support for (U)EFI
* use a new LVM partition layout, with option to
rollback/install/upgrade via second rootfs partition
(which is called "fallback" in this implementation)
* drop swap partition
Note:
* we no longer support msdos partition tables but GPT only,
as (U)EFI systems can only boot from GPT (and not from
BIOS/legacy boot), while BIOS systems can also boot from GPT
* https://wiki.archlinux.org/index.php/Partitioning +
https://wiki.archlinux.org/index.php/GRUB provide a decent
overview, if you're not familiar with BIOS/GPT/(U)EFI
New partition layout:
* 1st partition: 1M BIOS boot, for BIOS/GPT (legacy) boot
- this allows fallback to grub-pc package
(needs to be partition type GUID 21686148-6449-6E6F-744E-656564454649
AKA set to bios_grub with parted, then installing grub to disk
so it properly embeds core.img, can be done from a
rescue/live system)
* 2nd partition: ~500MB EFI System, for UEFI/GPT boot
- used as /boot/efi, iff EFI support is available
* 3rd partition: LVM with:
- /dev/mapper/ngcp-root with 5GB (rootfs target) +
/dev/mapper/ngcp-fallback with 5GB (for rollback/install/upgrade)
- 10% or >=500MB unassigned of the remaining space (whatever is bigger)
- /dev/mapper/ngcp-data data partition with rest of
available disk space, used as /ngcpdata on system
Old layout with a 16GB disk for NGCP systems:
| # lsblk
| NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
| sda 8:0 0 16G 0 disk
| └─sda1 8:1 0 16G 0 part
| ├─ngcp-root 254:0 0 14.3G 0 lvm /
| └─ngcp-swap 254:1 0 1004M 0 lvm [SWAP]
| # pvs
| PV VG Fmt Attr PSize PFree
| /dev/sda1 ngcp lvm2 a-- 16.00g 764.00m
| # vgs
| VG #PV #LV #SN Attr VSize VFree
| ngcp 1 2 0 wz--n- 16.00g 764.00m
| # lvs
| LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
| root ngcp -wi-ao---- 14.27g
| swap ngcp -wi-ao---- 1004.00m
New layout with a 16GB disk (+ EFI support) for NGCP systems:
| # lsblk
| NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
| sda 8:0 0 16G 0 disk
| |-sda1 8:1 0 1M 0 part
| |-sda2 8:2 0 486M 0 part /boot/efi
| `-sda3 8:3 0 15.5G 0 part
| |-ngcp-root 254:0 0 5G 0 lvm /
| |-ngcp-fallback 254:1 0 5G 0 lvm
| `-ngcp-data 254:2 0 5G 0 lvm /ngcpdata
| # pvs
| PV VG Fmt Attr PSize PFree
| /dev/sda3 ngcp lvm2 a-- 15.52g 564.00m
| # vgs
| VG #PV #LV #SN Attr VSize VFree
| ngcp 1 3 0 wz--n- 15.52g 564.00m
| # lvs
| LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
| data ngcp -wi-ao---- 4.97g
| fallback ngcp -wi-a----- 5.00g
| root ngcp -wi-ao---- 5.00g
Change-Id: I39206b225edb25fd1e27de49a818ad7a54532f82
We have confusing situation for an ages here.
We were writing a lot of log files:
> /var/log/deployment-installer-debug.log
> /var/log/ngcp-installer.log
> /var/log/ngcp-installer-debug.log
The first one contains all the GRML boot, Debian debootstrap
and ngcp-installer-debug.log already. Also having both
ngcp-installer.log and ngcp-installer-debug.log confuses
new users as they do not know which logs they need to
check first to see ngcp-installer logs.
There is no need to keep both files anymore.
Nowadays we have a nice way to distinguish logs between components,
so /var/log/deployment-installer-debug.log is enough here:
> +15:07:48 (netscript.grml:1403): main(): echo 'Generating ngcp-installer run script ...'
> +15:07:48 (netscript.grml:1404): main(): cat
> +15:07:48 (netscript.grml:1417): main(): echo 'Execute ngcp-installer inside deployment chroot environment ...'
> ...
> Running ngcp-installer via grml-chroot.
> ...
> +17:07:48 (ngcp-installer:27): main(): . /usr/share/ngcp-installer//system_pro.inc
> +17:07:48 (ngcp-installer:28): main(): . /usr/share/ngcp-installer//packages.inc
> +17:07:48 (ngcp-installer:31): main(): . /usr/share/ngcp-installer//cfg.inc
> ...
> +17:07:57 (packages.inc:58): packages_ppa_sourceslist(): echo '(system installed using NGCP PPA: gerrit_gtid_imp)'
> +17:07:57 (ngcp-installer:54): main(): packages_aptconfig
> +17:07:57 (packages.inc:90): packages_aptconfig(): mkdir -p /etc/apt/preferences.d
> +17:07:57 (packages.inc:92): packages_aptconfig(): cat
The commit here should minimize amount of log files we
produce and should simplify the debug process.
Change-Id: I26a2ca8e21a3385f7d57d830feb207bbb1039b39
We do not use /tmp/ngcp-installer.log since mr6.5 and
using directly /var/log/ngcp-installer.log inside chroot.
Change-Id: I35b098fb09046a0fbc0a9bfc56bd125362c34fd0
We're executing `apt-get update` even when nothing needs to be
installed. This adds unnecessary overhead and can fill ram/disk
on live system on consecutive runs, so let's avoid that.
Change-Id: Ic1a40d82988caf454f317b4803e03f4c9de3feea
Implement suggestions from review I16e4f19f3b9270ccfef6c7c1274bb8b8d95695bd
1. In some cases there can be multiple gateways. Return only the 1st one.
2. Remove configuration of the management (IMM2/IPMI) interface of
lenovo servers.
Change-Id: Ia1bb6e9f0e44bcb0c6be87407a8c11899163c03a
All non-zero returned values are caught by '-e' option and passed
to function wait_exit by 'trap'.
So if it is an error we can set 'error' status in this function.
Change-Id: I762e5470d94a537b4f8f0e32fe6d0c1575bbb075
We need it to be able to connect from mgmt node to installed one
to run 'ngcp-initial-configuration'.
Change-Id: Id907094ba2eae32c05ad8d42d8a504e498a57ad4
wget in Debian/buster:
| Package: wget
| Version: 1.19.5-1
| [...]
| Priority: standard
wget in Debian/stretch and earlier:
| Package: wget
| Version: 1.18-5+deb9u2
| [...]
| Priority: important
To ensure we still have wget available inside the
Debian system, e.g. for retrieving the ngcp-installer*.deb
file, we need to add it to the package selection.
The ngcp-nth-* packages also include wget in their dependencies,
ngcp-installer also checks and adds it, though it's too late
there for bootstrapping our systems, as we're e.g. retrieving the
installer deb + checking for PPAs using wget.
While at it move the slightly misleading comment to the
appropriate place.
Change-Id: I971b2d06528a33b1dc4c310feef83c32e10ae557
Switch from:
> +10:43:04 (/usr/sbin/ngcp-status:10): main(): export LC_ALL=C.UTF-8
to:
> +10:43:04 (ngcp-status:10): main(): export LC_ALL=C.UTF-8
Since we have unique names for all the script we use,
the full path is not so important as the space we spent of the console.
Change-Id: Ie6fe2ce1ca409033189e1c033a4aebf9e037eaba
1706:9 shellcheck:warning When used with -p, -m only applies to the deepest directory. [SC2174]
1726:9 shellcheck:warning When used with -p, -m only applies to the deepest directory. [SC2174]
Change-Id: I98efba1253c0aefd9e97ac34500476936b860ca5
257:9 shellcheck:error Arrays implicitly concatenate in [[ ]]. Use a loop (or explicit * instead of @). [SC2199]
Change-Id: I49504d5b1a8d8ae021d4d6c6465639d58a2670e3
If the installation device is virtual it is necessary to redefine
DEFAULT_INSTALL_DEV variable to the name of real interface but
save the origin name to stop it.
Change-Id: I9e59ca8411e713235a0e0e5407ef0c3acc30e299
This list is used to add all network interfaces to network.yml but
we do not need to add virtual interfaces there - they will be added
automatically from installation's configuration.
Change-Id: I3f21abc066b4f0c714b1780dcfdf4335e7506d1d
The deployment.sh script as well as installer should not configure system
or services.
Remove network's configuration code.
Remove services' configuration code.
Remove unused variables and parameters.
Copy existing /e/n/i into chroot env so the server starts with the same
network configuration.
Move gateway detection to the top - we need this information to generate
proper /e/n/i
Change-Id: I16e4f19f3b9270ccfef6c7c1274bb8b8d95695bd
1. 'localhost' has to be resolved to ipv4 only not ipv6.
2. There should be 'hostname.' with dot record.
Change-Id: I9f34f45b00453746b254f41494d7652a10de0d04
Without explicit return function returns the exit code of the last function
which in not 0 in this case so it causes amiss exit of the script.
Change-Id: Ic7f1f534048cb051fe116cb76ee386bf4ca07ff4
If 'systemctl reboot|halt' is called within script 'trap' function
is not called so deployment.sh does not wait and jenkins can miss
the status.
Move waiting to separate function and call it directly so there is
waiting in the end of deployment.sh in any case.
Change-Id: Ib93bf6fd4e6f133c9342d02d412ea1f18cdceba6
For now if there is an error in deployment.sh it is ended immediately
so jenkins node can miss deploy status.
Add wait_exit function which is called on any exit and waits
STATUS_WAIT seconds.
Change-Id: I715c2484a25222221d61a01bd220973e89547929
We are NOT installing non-free firmware packages on NGCP (since TT#42672),
as a result we should not purge them. See commit 9161d0f57 in
metapackages.git for more information.
Change-Id: I84ff3486c0b03bbc228e5332955f6654b3519137
The hardcoded list of accepted drives is never really up2date
and was meant as a safety net in the beginnings of deployment.sh.
Let's drop those checks to avoid failures in automated installations,
interactive installer/deployment mode is supposed to prompt for
before actually executing anything anyway.
Change-Id: Ib6dcaec8a221dc268638a98055219a5ae3c92a00
deployment.sh can install plain Debian systems (without NGCP),
in commit d2afaf9 such functionality has been lost. Fixing it here.
Change-Id: I883ef3c69c67cc04ae796c373f3b52d75780b4de
Mykola Malkov
Michael Prokop
Guillem Jover
Sergii Kipot
Manuel Montecelo
Alexander Lutay
Volodymyr Fedorov