We cannot test web services avialability if they listen eth0 only,
so we have to continue listening localhost (which is fine from
security point of view) to be able to connect nginx/ngcp-panel.
In this case ngcp-system-tests will be fine.
Change-Id: Ia4097461d03be51221c1f622018a7974f3984b44
We're customizing /e/n/i in our ngcp installations anyway
after [grml-]deboostrapping the installation via deployment.sh.
But when installing plain Debian systems we depend on a working
/e/n/i file, which isn't true yet because recently we get the
/e/n/i file copied from the host to the installed system which
doesn't work there then as-is.
Depend on grml-deboostrap v0.67 even though the
--defaultinterfaces option was introduced in v0.64,
but v0.67 is what's targeted for Debian/jessie as well
as the upcoming Grml stable release and we're using that
version already anyway because it's the one present in
the grml-testing repository.
Change-Id: Id48636a16cce25fc6a52b6fcaa3db828b9459484
I have tried to add web_* types using deployment.sh, but we need to
rebuild configs later, so the proper way here is to move it installer
(like we did for performance options).
So, reverting web_* part (last two commits) and stops using function
enable_vm_services in deployment.sh since mr3.6
P.S. vagrant_configuration is the last Vagrant part in deployment.sh
I hope to move it in installer also one day.
Change-Id: I57fcb68b71f51cd39b821fb1f30da658fc68d152
When we use b0 as device name for the bonding device, then
we get a bond0 *and* b0 device, while bond0 isn't used then.
This is slightly confusing and there's no need to do that,
since using "bond0" as device name leads to one configured
bond0 device, which operates as needed.
This also follows best practice as documented at
https://wiki.debian.org/Bonding and
https://www.kernel.org/doc/Documentation/networking/bonding.txt
Change-Id: Icfef7d34991200025821a67569b0708d24a43eef
When executing deployment.sh from outside the Sipwise network
it fails due to:
| --2014-09-16 16:49:04-- http://deb.sipwise.com/autobuild/680FBA8A.asc
| Resolving deb.sipwise.com (deb.sipwise.com)... 77.244.249.93, 2a00:4600:2::c0f:fee
| Connecting to deb.sipwise.com (deb.sipwise.com)|77.244.249.93|:80... connected.
| HTTP request sent, awaiting response... 403 Forbidden
Our existing sipwise.gpg file is now available at
http://deb.sipwise.com/spce/sipwise.gpg for public usage.
Change-Id: I08ca2e8c87b7133f9dc0bab59cb85875c57f065a
We've the fai-setup-storage Debian package in our wheezy-backports
repository (taken from http://jenkins.grml.org/job/fai-binaries/architecture=amd64/184/)
which includes the bugfix 5160db7031
to address the failing execution on empty disk(s) with parted >=3.*.
Change-Id: I3400aa1cd80dc356dbd92767756dc43c596ed138
Copy relevant sp2 commands from modify network.yml
PRO section that will be skipped later.
Don't try to connect to db, on CARRIER nodes the
config is already pointing to db1a and that node
could be not even created. This will reduce,
installation time due to ngcpcfg timeouts.
Change-Id: I54aa56277d2d61344a7e77e794da850b364d2e40
We can select the ha_int IP with ngcpip[1|2] but we should be able
to select the netmask too instead of using the default one
Change-Id: I221df9749107662b0a551800a24830fb577f3678
Add an option to control the reconfig of the network.
On our dev proxmox we don't want to touch the network
on the install phase.
Change-Id: I6a05df7c63389b0c2e2d1bb99da8bbd1e976dc23
On sp1 node, get the files after installer execution.
ngcp-update-[cfg|db]-schema does not deal with initial
files on first execution; it truncates them.
Change-Id: I1276ff5ee5f05e950ef32638e48ec7ebf7376f45
As noticed by Victor the trunk installations using the
development.sh script retrieved the latest version of the
installer, no matter to which repository this binary belonged to
(so even if it's not listed in release-trunk-${debian_release}).
This causes problems when you are using trunk installations and
someone is working on installer on branches, reviewing changes,
etc.
To avoid this problem we take a look at the Packages file of the
according repository instead. This isn't limited only to trunk
builds, but also for the release builds, to avoid any possible
problems with multiple deb files inside the pool directory (which
isn't the case yet, but we *could* do that now).
Change-Id: Ie215e70e9317133750cdf0e4b264a3b2c6f083fa
Remove installing the extra packages on carrier, now the carrier
installer is the one in charge of that.
Add CROLE parameter to installer, this should be empty on
normal PRO installations
Change-Id: Ieb7925e11387999b556f03a25f05d3db58260998
We configure ssh_ext for all present network devices (except for
some known to be irrelevant ones like the ones from VMware,
VirtualBox and Docker), instead of just enabling it for
$EXTERNAL_DEV (being eth0 by default). Retrieving the list of
available network devices is done on each of the according PRO
systems separately, so we don't enable it on sp1 for network
devices that possibly aren't available on sp2.
We also explicitly enable ssh_ext for sp2 on sp1 for loopback and
$INTERNAL_DEV (eth1 by default), so we can make sure we can
access ssh at any given time.
1) we don't have /etc/default and/or config.yml options for old releases
2) we need to decreate some option for low performance mode
3) services should never start with default options
(otherwise oom-killer does the job and kill services)
So, we have migrated all the options to installer under special
install option and are going to decrese config _and_ template
for old releases in deployment.sh
In this case VM after first reboot will have all the options in
low performance mode and we do not need apply changes.
In case of DHCP server is being used, and system was rebooted
with new IP, network.yml still has old IP and SSH will listen old IP only.
Vagrant cannot connect VM using old IP to change it to new one.
In case of CE with one network interface (common case)
ngcp-eadresss will update network.yml and apply new config.
PRO should not be installed using DHCP (at least in production).
Vagrant VMs has two interfaces, so ngcp-eadresss does nothing,
so we add option to config.yml which overwrites ssh_ext types in
network.yml.
We have an error "reset: standard error: Invalid argument" in header of
CE install CD, so replated reset with clear which worked well for long time
during the nightly builds.
Also removed all the additional calls for logo() function.
(I have no ideas what can be the reason of broken layout MT#4697,
spent a lot of time trying to catch it, it comes time-2-time,
so currently minimised amount of any functions calls during printing logo).
I hope to catch this one day and fix it in prtoper way.
Debian's BTS has all the details in #750212 which I just reported.
lvm2 version 2.02.106-1 includes a check for present signatures,
and without "--yes" option fai-setup-storage is hanging when
executing lvcreate on a device which includes such a (swap) signature.
We don't want to use the sources.list generation which
grml-debootstrap does, as we want to control 100% of
what we use/deploy, so provide sources.list as needed.
I think we finally tracked down the problem to its roots (oh boy… ٩(͡๏̯͡๏)۶)):
The problem exists only for PRO systems, where we ship the
heartbeat-2 package, which still uses and provides the /usr/lib64
directory (that's what you get for running antique software
*cough*). As soon as this directory exists the
VBoxLinuxAdditions.run script of virtualbox-guest-additions-iso
detects and uses this directory as library path. It installs the
mount.vboxsf symlink using this library path.
The symlink /sbin/mount.vboxsf points to the non-existing
/usr/lib64/VBoxGuestAdditions/mount.vboxsf file instead of
pointing to
/usr/lib/x86_64-linux-gnu/VBoxGuestAdditions/mount.vboxsf.
It works for CE systems, because we don't ship the heartbeat-2
package there. Without heartbeat-2 /usr/lib64 doesn't exist and
VBoxLinuxAdditions.run does the right™ thing by detecting
and using /usr/lib/x86_64-linux-gnu instead.
If that's finally working as expected™ I'll be able to sleep much
better again… ☻
We're still facing:
| root@sp1:~# mount -t vboxsf -o uid=`id -u sipwise`,gid=`getent group sipwise | cut -d: -f3` /vagrant /vagrant
| mount: Protocol error
on some VMs (noticed on e.g. 3.1 PRO system with 3.13-0.bpo.1-amd64).
The /sbin/mount.vboxsf file is missing for some reason we're not
aware of yet.
Let's get rid of unneeded AllowUnauthenticated usages with apt.
It would be even better if we could ship our own key with the
official Grml-Sipwise ISO (which would technically be no problem,
but we need a way for PXE boot for builder.mgm anyway).
Always writing to /etc/apt/sources.list is stupid
because it makes tracking changes harder, so let's
try to move debian specifica into debian.list and
sipwise specifica to sipwise.list instead.
Even if our own key is already installed the package list might
not be up2date yet (e.g. for wheezy-backports):
| The following NEW packages will be installed:
| linux-compiler-gcc-4.6-x86 linux-headers-3.13-0.bpo.1-amd64
| linux-headers-3.13-0.bpo.1-common linux-kbuild-3.13
| The following packages will be upgraded:
| linux-headers-amd64
| 1 upgraded, 4 newly installed, 0 to remove and 1 not upgraded.
| Need to get 5187 kB of archives.
| After this operation, 32.9 MB of additional disk space will be used.
| WARNING: The following packages cannot be authenticated!
| linux-compiler-gcc-4.6-x86 linux-headers-3.13-0.bpo.1-common
| linux-kbuild-3.13 linux-headers-3.13-0.bpo.1-amd64 linux-headers-amd64
This is an urgency bugfix to address the failing
libssl1.0.0 1.0.1e-2+deb7u6 upgrade which is prompting
via debconf and causing our builds to fail because of that.
On plain Debian installations we don't have the ngcp-keyring
package present, so we run into:
| WARNING: The following packages cannot be authenticated!
Using debian.sipwise.com which is a CNAME record pointing to
deb.sipwise.com prevents us from getting higher apt-pinning
for official Debian packages which are mirrored on our own
server.
Related commit history: 8832daa7ea1b3eb0fd8400101b6
The "Install sip:provider CE" boot entry in the Grml-Sipwise ISO
which doesn't provide a ngcpvers kernel cmdline option doesn't
work, this change is supposed to address that.
Otherwise files like
ngcp-installer-ce_0.13.0~20140117133338.395+wheezy_all.deb
as being the result of an UNRELEASED entry in the debian/changelog
are ignored.
dpkg-scanpackages is supposed to recognize the
0.10.2+0~1368529812.299+wheezy~1.gbp1691a0 being newer
than 0.10.2 anyway.
This is a re-design of the ngcp-installer version selection.
We want to avoid having to put every new mrX.Y release/build into
deployment.sh just to point a specific release installation to a
specific installer version.
Also this turned out to be a pitfall when releasing a new
ngcp-installer package and forgetting to update deployment.sh
accordingly.
So instead lets try a different approach:
We provide only *one* specific Debian package version of each
package inside each ngcp release repository already. This means
we can just check what's inside a specific ngcp-installer
directory, like:
* http://deb.sipwise.com/spce/2.8/pool/main/n/ngcp-installer/
* http://deb.sipwise.com/sppro/2.8/pool/main/n/ngcp-installer/
* http://deb.sipwise.com/spce/mr3.2/pool/main/n/ngcp-installer/
* http://deb.sipwise.com/sppro/mr3.2/pool/main/n/ngcp-installer/
* ...
and then assume that's the installer version we want to use for
installing the according ngcp release.
This is 100% UNTESTED yet!
The mediaproxy-ng kernel module can be installed successfully on
3.0 systems for kernel 3.2.0-4-rt-amd64 and therefore is reported
as "ngcp-mediaproxy-ng. kernel package already installed,
skipping". This is wrong for our needs, so let's ignore this
"-rt-amd64" in the dkms status output.
/home/ is an absolute symlink in PRO setups, therefore can't be
resolved when accessed from outside the installed system/chroot,
resulting in error message:
| + mkdir -p /mnt/home/sipwise/.ssh/
| mkdir: cannot create directory `/mnt/home': File exists
We don't have any other architectures besides the amd64 one on
our own repositories and nowadays with MultiArch people might have
e.g. i386 enabled (via 'dpkg --add-architecture i386'), resulting
in errors like:
| W: Failed to fetch http://deb.sipwise.com/spce/3.0/dists/wheezy/main/binary-i386/Packages [^] 404 Not Found
| W: Failed to fetch http://deb.sipwise.com/wheezy-backports/dists/wheezy-backports/main/binary-i386/Packages [^] 404 Not Found
| E: Some index files failed to download. They have been ignored, or old ones used instead.
Avoid retrieval of i386 Packages files by limiting the deb entry
to the amd64 architecture.
Now having MT#4463 resolved let's also get rid of "Applying
Vagrant performance optimisations for VM" steps from
daily-build-vagrant.
Implemented as separate boot option (independent from "vagrant"
boot option) to apply it only for the VM 2XX builds (to e.g. not
execute it on plain Debian systems).
gcc is already present in ngcp systems and therefore the
recommended package libc6-dev won't be pulled in anymore, the
uname fake code depends on stdio.h etc though
Virtualbox Guest Addition installation fails inside chroot
with different kernel version of live system vs installed one.
Sadly there doesn't seem to be a way to reliably control kernel
version/header location forVBoxLinuxAdditions.run (e.g. via
KERN_DIR=... as advertised) because there are several calls
to `uname -r` in use. :(
I sometimes get a failing network setup in Vagrant's ce-trunk VM
where the default route is missing then:
| # ip r
| 10.0.2.0/24 dev eth0 proto kernel scope link src 10.0.2.15
| 192.168.88.0/24 dev eth1 proto kernel scope link src 192.168.88.234
|
| # cat /etc/network/interfaces
| # This file describes the network interfaces available on your system
| # and how to activate them. For more information, see interfaces(5).
| # The loopback network interface
| auto lo
| iface lo inet loopback
|
| # The primary network interface
| allow-hotplug eth0
| iface eth0 inet dhcp
| #VAGRANT-BEGIN
| # The contents below are automatically generated by Vagrant. Do not modify.
| auto eth1
| iface eth1 inet dhcp
| post-up route del default dev $IFACE
| #VAGRANT-END
Of course the default route comes back on a manual refresh of eth0:
| # ifdown eth0
| # ifup eth0
| # ip r
| default via 10.0.2.2 dev eth0
| 10.0.2.0/24 dev eth0 proto kernel scope link src 10.0.2.15
| 192.168.88.0/24 dev eth1 proto kernel scope link src 192.168.88.234
The culprit seems to be the allow-hotplug where the device is handled
different in the startup process:
http://screeni.s3.amazonaws.com/screenshot.2013-09-17T11:43:37.png
The ordering is important because of Vagrant's
"post-up route del default dev $IFACE". While there's a workaround
available within Vagrant:
73c8299ecd
I don't think that's the proper way. Since we're using "auto"
everywhere else in our setups anyway let's make this change,
hopefully also fixing the Vagrant network issue.
ngcp-services-pro depends on linux-headers-3.2.0-4-all,
which depends on linux-headers-3.2.0-4-all-amd64,
which depends on linux-headers-3.2.0-4-amd64.
linux-headers-3.2.0-4-amd64 is installed already in version
3.2.46-1+deb7u1 on our installation, being the current version
from security.debian.org. Because this package depends on all the
other packages being in version 3.2.46-1+deb7u1 as well this
fails due to our apt-pinning. Our apt-pinning considers
security.debian.org to be at it default pinning level 500 whereas
deb.sipwise.com is pinned to 990. This results in package
dependencies that can't be resolved by using packages from
security.debian.org.
We have to think further regarding the wanted behaviour before we
actually make that MIRROR switch. :(
On Jenkins host there was this symlink present:
| cd /srv/repository
| ln -s . debian
To avoid unnecessary duplicates during repository search
I removed the symlink, let's see whether this change is
enough or if apt-get then still needs it.
dkms in Debian/wheezy fails when directly invoked via chroot(8):
| /usr/sbin/dkms: line 1868: /dev/fd/62: No such file or directory
| /usr/sbin/dkms: line 1799: /dev/fd/62: No such file or directory
and returns with exit code 0. By invoking grml-chroot we get the
according mount binds for devfs etc.
Current daily VMs are known to fail because of switching from
raw image to vmdk on proxmox. :( Abort installation immediately
if actions during setup-storage, parted and mkswap fail.
Otherwise grabbing a screenshot after deployment finished
will include to much noise and not the useful data...
From: Michael Prokop <mprokop@sipwise.com>
Make it obvious which platform releases use which underlying Debian release,
once we have wheezy support the default will be changed and we don't want
to have wrong assignments for existing releases then.
From: Michael Prokop <mprokop@sipwise.com>
The '5%-10G' sadly doesn't work as expected and 5% of the
*disk* size is a bad default, so let's see whether XX% of
RAM size makes a better choice. We might change the actual
percentage value but we need some testing anyway....
From: Michael Prokop <mprokop@sipwise.com>
Alexander Lutay
Michael Prokop
Victor Seva
Kirill Solomko
Andrew Pogrebennyk
Andreas Granig
Jon Bonilla
