As we started to use hieradata also from puppet code, hiera.yaml now
points to /etc/puppetlabs/code/environments/ and so we should put puppet
code with hieradata there
Change-Id: I4a5e00124ed6bd1ebd1288d35dc4070dea0f195e
We have to always provide Sipwise Hiera rescue ISO
if we are using recovery from git repository as
it doesn't have any passwords and SSH keys.
In the same time we search drive by label,
so there is no point to have an options here anymore.
Change-Id: Ib47f147cbb3a154f96d37ef31d43acad3a68c9e5
He have spent 2 hours trying to figure out why puppet1.mgm
installation has failed. And it was human mistake with
forgotten boot option. In the same time we must provide
puppetrescuedrive boot option and we should execute Hiera
initialisation in case if puppetrescuedrive is provided.
So, lets reuse puppetrescuedrive for both cases.
Change-Id: I31cd80522fbce720be67090398322b3426701ec9
Otherwise deployment.sh abort disaster recovery installation if we have 1 drive only:
> Error: /dev/sdb does not look like a VirtIO, ServeRAID, LSILOGIC or PowerEdge disk/controller.
> Exiting to avoid possible data damage.
Change-Id: I4de4d0de75964fc6f09b8f0b1ae3104a6bdfc6ee
Apt modules should be also executed during puppet apply core. It's not
reasonable to tag all hierarchy of apt modules with 'core'. Instead
let's add 'apt' to --tags.
Change-Id: I6a2a6feb5f6b82a5894682b07958e4084c705273
Previously we were installing old puppet from jessie
and then upgraded to new puppet-agent on puppet.mgm role only.
So, it is a time to start installing puppet-agent by default
as a first step to switch all our server to puppet-agent
which will allows us to shutdown mcollective.mgm host
as Puppet 4 has integrated mcollective already.
There is no point to have support for squeezy here,
as puppetlabs-squeezy doesn't exist, so puppet-agent
cannot be installed.
Change-Id: I6fe26da35a883a498d1c13014ff210d5744231fc
We have no apt source lists configured at that moment,
so we have to install it through temporary apt database
Change-Id: Ia8dc8c9a9fa37ac92f84d78739ad7f8bea7e34a4
Noticed the following errors during Carrier mr4.3 installation:
> Installing apt-transport-https
> ...
> Err http://web01:9998 wheezy/main amd64 Packages
> 404 Not Found
> Err http://web01:9998 wheezy/contrib amd64 Packages
> 404 Not Found
> Err http://web01:9998 wheezy/non-free amd64 Packages
> 404 Not Found
> WReading package lists...: Failed to fetch http://web01:9998/debian/dists/wheezy/main/binary-amd64/Packages 404 Not Found
> W: Failed to fetch http://web01:9998/debian/dists/wheezy/contrib/binary-amd64/Packages 404 Not Found
> W: Failed to fetch http://web01:9998/debian/dists/wheezy/non-free/binary-amd64/Packages 404 Not Found
Which is correct as there is no wheezy repository in Approx.
We should use appropriate ${DEBIAN_RELEASE} repository.
Notes:
* apt-transport-https from jessie repo is good enough,
switch from wheezy to jessie
* fai-setup-storage liblinux-lvm-perl from jessie repo is good enough,
cleanup the update code here
* virtualbox-guest-additions-iso from jessie repo is good enough,
switch from wheezy-backports to jessie
* augeas-tools from jessie is good enough,
switch from wheezy to jessie
Change-Id: I0de23b673da61b332af9fd9cfbcb36fb9b99d0d6
We're hitting "Error: File not found" GRUB errors during wheezy->jessie
upgrades. While this doesn't cause any hard problems since the boot
still continues automatically we should ensure that the grub-pc
package knows where to install the bootloader.
grml-debootstrap >=0.74 provides the according grub-pc/install_devices
debconf support.
Change-Id: I6062888a132389b279c0ce4987334866da98a689
Label is limited to 11 symbols for FAT (type 'vfat'),
default for USB drives nowadays to mount it easily.
So we need to change the defaults here to support both
CDROM and USB flash cards.
Also it is not recommended using spaces here.
So, SIPWRESCUE0 - test drive with fake rescue data
SIPWRESCUE[1-x] - real rescue drives, where x is a version.
Change-Id: I21abd08b01023f9ac6c63258027d515dd093def5
New mode 'auto' is introduced, we are trying to find any device with
label "Sipwise Hiera Rescue *" and mount it using appropriate device type.
Also switch to function die() on error to set set_deploy_status=error
properly (otherwise jenkins doesn't detect error on autotests).
Change-Id: I0a9900b0e8edcb1e12f09caaf66e94a49b8ac863
In some cases puppet should be restarted in order to catch up with
system changes it did itself. For instance if ldap is installed in
puppet session new users won't be available for puppet in the same
session. With this change we have flexibility to run puppet first just
to insall core utils and then do a second normal run.
Change-Id: I7f8d9a7cb45ddcf6c32fd836a4cc488f572092c8
parted supports the "-a optimal" option to get better alignment during partition table setup.
We should use that by default, since it:
a) shouldn't cause any harm on VMs anyway and
b) might improve performance especially on SSDs with non-standard
optimal_io_size settings (as observed on a Samsung EVA 850
with a setting of 33553920, so the partition should start
at 33.6MB and not 1MiB/2048s)
NOTE: fai-setup-storage doesn't support setting the parted options
like "-a optimal" yet, reported as #812712 in Debian's BTS.
Change-Id: Ib1c49d4f2b405697f0dcac5f164473ea9f87557c
NOTE: puppet success exit codes are different for different actions:
- 'puppet agent -t' exit code is '2' if no errors happen,
- 'puppet agent --enable' exit code is '0' if no errors happen.
Jenkins should detect puppet error exit codes and fail the test job.
Change-Id: I8df33897bd647e53a4f895c79c82749b97c7313c
The "--yes" workaround for LVM2 is needed for LVM versions
newer than >=2.02.106 to avoid a hanging prompt during
partition setup. Starting with FAI 5.0 the workaround
for this is included in FAI's setup-storage and therefore
we don't need our workaround any longer, see
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=750212
So ensure that fai-setup-storage is new enough, and only
if not then check the lvm2 package version and apply the
workaround as needed.
Fixes:
| Executing: lvcreate --yes --yes -n swap -L 1002.755859375 vg0
| (STDERR) Option -y/--yes may not be repeated.
Change-Id: Ic4445e0add5aab8da0ef87b77b08e23fc0d1c9ab
The latter is a transitional dummy package that will be going away in
the near future. Switch to it now, instead of waiting for the inevitable.
Also this way we get less cruft on the installed system.
Change-Id: I6679d3eafed07548dfebf0dc8a5b2fe75a5ef58d
Sipwise systems/VMs works well because we rebuild /etc/ssh/sshd_config
from templates and enable supporting /root/.ssh/sipwise_vagrant_key
while for plain VMs we do nothing, so we have to handle it here.
Change-Id: Ib7eccda1a8b9692da6e581aef47c3ccddeb229f6
When we install a fresh squeeze system we want to avoid the
occasional "Hash Sum mismatch" errors during apt-get runs,
so let's not only cover that via upgrades but also in the
initial installation.
Also cover it for Debian lenny, since it very probably might
be affected as well (unverified though).
Change-Id: I01dbeec5ef2a727e5749be5db3ebea47818ed3b5
With the according SSL setup in place nothing should prevent
us to enable HTTPS by default.
The only issues left here are the usage of https for
our internal management web service on port 3000 and
the usage of HTTPS with approx.
Change-Id: I7c9c3d947c6b880b9145ca83a31c832049c6573a
puppet in jessie depends on ca-certificates/openssl which
cant' be removed therefore. We actually want to have
ca-certificates/openssl on all our systems anyway and
nowadays debootstrap doesn't install tcpd and xauth any longer,
so we can safely skip those package removals for jessie and newer.
The templatedir doesn't need to be set in recent puppet versions
and older versions use a sane default anyway, therefore dropping it.
puppet as present in jessie (and newer) expects 'puppet agent --enable'
to be executed to actually be enabled.
Change-Id: Iae70eaeec6a56bef0e67171aab74482aebb24181
Since lvm2 v2.02.106 its udev rule creates a
/dev/disk/by-id/lvm-pv-uuid-<PV_UUID> symlink for PVs:
https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=417e52c13a8156b11c25c411d44bda8b32bf87e4
When debootstrapping Debian/wheezy to a virtio disk from a system
using lvm2 >=v2.02.106 (as present in Grml >=2014.11) this causes
the resulting GRUB installation to look like that
(`/media/ngcp-root` is the mounted logical volume with the rootfs
which also includes /boot):
| root@grml ~ # cat /media/ngcp-root/boot/grub/device.map
| (hd0) /dev/disk/by-id/lvm-pv-uuid-RNZHr8-hLlc-nles-X9Gs-RYSp-Mqo7-uZTk6N
| (hd1) /dev/vda
and failing to boot with:
> error: no such disk
> Entering rescue mode...
> grub rescue>
When manually removing the lvm-pv-uuid* symlink from the
Debian/jessie system before installing GRUB it works fine:
| root@grml ~ # rm /dev/disk/by-id/lvm-pv-uuid-RNZHr8-hLlc-nles-X9Gs-RYSp-Mqo7-uZTk6N
| root@grml ~ # ls /dev/disk/by-id
| ata-QEMU_DVD-ROM_QM00003@ dm-name-ngcp-swap@ dm-uuid-LVM-lQhnvLyYmhESObLrZcnJs4FTZq9usVDzRex3ImRtrmqhpDH9xKBj22St82lHDADh@
| dm-name-ngcp-root@ dm-uuid-LVM-lQhnvLyYmhESObLrZcnJs4FTZq9usVDzrcMaDfk8O0RionEYRxdjnYxsY6eb0Esi@
| root@grml ~ # grml-chroot /media/ngcp-root/ grub-install --recheck /dev/vda # `grml-chroot` makes sure /proc, /sysfs and /dev are properly mounted in the chroot
| Writing /etc/debian_chroot ...
| Installation finished. No error reported.
| root@grml ~ # cat /media/ngcp-root/boot/grub/device.map
| (hd0) /dev/vda
NOTE: This problem seems to exist only for virtio disks, when
using a disk on the SATA bus the resulting device.map of the
Debian/wheezy system looks like that and boots fine:
| root@grml ~ # cat /media/vg0-root/boot/grub/device.map
| (hd0) /dev/disk/by-id/ata-QEMU_HARDDISK_QM00005
| (hd1) /dev/disk/by-id/lvm-pv-uuid-lJJr4E-M96R-3eIS-b6YO-5usn-bH5o-osQV2B
As a result check for Debian wheezy (or older) and if the disk is
a virtio one (AKA /dev/vda), if so then remove the symlink of the
live system before installing Debian incl. GRUB via
grml-debootstrap.
Reported as https://bugs.debian.org/782998 in Debian's BTS.
Change-Id: I4b2cf9e39c88521ab7eac7799531508e658cc30d
During the refactoring in 43c9f159 we missed the point that
/etc/debootstrap/pre-scripts/install-sipwise-key.sh should be
executed on CE install CD too even if we do NOT download key.
Otherwise we are getting:
> Reading package lists...
> W: GPG error: http://debian.sipwise.com wheezy Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 4EB7765DA42C4F2A
> W: GPG error: http://debian.sipwise.com wheezy-security Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 4EB7765DA42C4F2A
> W: GPG error: http://debian.sipwise.com wheezy-updates Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 4EB7765DA42C4F2A
and CE installation failed as unauthorized packages cannot be installed.
Change-Id: I3ccc8b8801ba2be829c4eea84507f11c841b7658
It allows to test scenarios where ngcp-installer is comming from one PPA
while other packages (like ngcp-templates) are comming from another PPA.
By default NGCP_PPA_INSTALLER is equal to NGCP_PPA,
so we still can define one boot option only:
> ngcpppa=gerrit_MT12345_review777
while for some cases it is useful to define:
> ngcpppa=gerrit_MT12345_review777 ngcpppainstaller=gerrit_MT54321_review778
Change-Id: Icf2fee70db1beaae08e9d9c3eba87c5a836661b7
Also removed sipwise user from 'plain' VM boxes,
it is not necessary there as we use user root in Vagrant.
Change-Id: I84c8ab6b1cd6e9dd7ba94f6871b79bafb153bd4e
Also, we did s/RETRIEVE_MGMT_CONFIG/CARRIER_EDITION/g because /etc/hosts
and /etc/network/interfaces are fully configured for Carrier by ngcp-installer.
So, no need to touch them for Carrier at all now
(see commits to installer.git through this ticket).
Change-Id: Ie16038d1f6c194564bb91c52faab75e86e3a0c11
1) Previously we were installing keys everywhere,
this produces a mess, so lets install them once on start.
2) GRML bypass /etc/apt/trusted.gpg.d/sipwise.gpg to NGCP CHROOT via install-sipwise-key.sh.
So we have /etc/apt/trusted.gpg.d/sipwise.gpg on every installed VM for wheezy/squeezy/jessie.
3) /var/log/deployment-installer-debug.log on all plain and NGCP VMs contains:
> echo "Sipwise Debian mirror key is already present."
So, we can safetely clean it from vagrant_configuration, IMHO.
Change-Id: I873f87faed24df4a93474cf17f41eb19e8a14ab1
We already have a function which take care about the keys,
we should use it everywhere!
Moreover the public keys contains both release and autobuild keys already:
>> http://deb.sipwise.com/spce/sipwise.gpg
>> http://deb.sipwise.com/sppro/sipwise.gpg
> gpg sipwise.gpg
>> pub 1024D/A42C4F2A 2010-11-29 Sipwise GmbH (Sipwise repository key) <support@sipwise.com>
>> sub 2048g/680FBA8A 2010-11-29
>> pub 4096R/EE5E097D 2011-06-06 Sipwise autobuilder (Used to sign packages for autobuild) <development@sipwise.com>
>> sub 4096R/3A5C488C 2011-06-06
Change-Id: I8b729ff624954de0d133b803e4d220c29a4fd799
It's PITA to install an internal system which depends on vlan
tools being present to get network acess but not having vlan
tools installed. So far we installed the vlan plus related tools
only when the vlan boot option was present during deployment
time. This might be relevant also for our ngcp users and it
should be safe to always include those tools. So we decided to
always include vlan (+ related) tools on all our installations.
Change-Id: I17bc5b9aba967d13bebe2db631e9167e693b09bc
We have migrated to ngcp.inc with modern list of services to be stopped
at the end of ngcp-installer.
See commit e75b01441c for more details.
Unfortunately deployment.sh is performing 'magic' with network.yml and depends on glusterfs
for pulling/pushing configs from/to shared storage. I am planning to move this 'magic' to
ngcp-installer, but while it is here, we have to start stop glusterfs-server manually.
Change-Id: Ia0aac183df52a1e2bd51ff6f2326f8438cad5b79
It was a temparary option to check systemd on wheezy while we
were waiting for Jessie. Systemd doesn't work well in wheezy,
we had a problems wil kamailio-config-tests and other components.
So, currently we have Jessie VMs available and no reason to keep the
code here as we are not going to use systemd on wheezy anyway.
Change-Id: I1c35b4e3bdd9718107812e7ac2112e796c2e83b0
We are not using config.yml->{networking}->{hb_device} since ~2.7
We are using ha_int from networking.yml, so, it is a time to clean
deployment.sh and config.yml
Change-Id: Ifb9f9a78d267120db4fa0423ceaacc5309f74eae
This is needed to resolve web01 to the proper ip after restarting.
Before it was the boot_int IP (192.168.1.1) now it should be the
ha_int IP (X.X.X.X)
Change-Id: I78969321a09798084955d4d045676dbb83732865
For some reason 'Packages' doesn't work for Carrier 3.x
We had to switch to 'Packages.gz' which works perfectly
on approx cache and should be fine for CE/PRO.
Change-Id: I15ae526aca33f3d819dba11b248882f88b37a1ee
At the moment we use wheezy-backports from repo 'wheezy-backports':
> http://debian.sipwise.com/wheezy-backports wheezy-backports main
which is actually not a Debain repository 'wheezy-backports',
but Sipwise repository 'wheezy-backports':
> http://deb.sipwise.com/wheezy-backports wheezy-backports main
It works currently because debian.sipwise.com and deb.sipwise.com are
the same host. So, we have to switch it to sync codebase with bootenv.git
Change-Id: I735402f3e077b5217f7bfa566b6fa7c0a3240a78
We have to sync deployment.sh in netscript.git and bootenv.git
At the moment netscript.git uses /spce/sipwise.gpg and
bootenv.git /sppro/sipwise.gpg (spce repo is missed in approx).
So, lets use the same code here: /spce/ on CE and /sppro/ on PRO.
Change-Id: Ida01fb110ecc90777b4adc5fe8f4bc2c684814ec
Required for usage with recent kernel versions like 3.16 from
wheezy-backports.
Note: this needs actual testing for all our different builds.
Change-Id: I53a4c276ce265dde18f4d27cba2c6810ac502746
We cannot test web services avialability if they listen eth0 only,
so we have to continue listening localhost (which is fine from
security point of view) to be able to connect nginx/ngcp-panel.
In this case ngcp-system-tests will be fine.
Change-Id: Ia4097461d03be51221c1f622018a7974f3984b44
Alexander Lutay
Sergii Kipot
Michael Prokop
Guillem Jover
Victor Seva