cowdancer (one of the underlying components of cowbuilder) checks
whether the terminal supports colors. If the related setupterm()
invocation fails, then it outputs:
| E: Error calling setupterm: 0
This is confusing and unnecessary output in our cowbuilder Jenkins
logs, so avoid this by setting the TERM environment variable to
"dumb".
Development sponsored by Sipwise GmbH, recorded as
TT#46675 in customers' ticket system.
Starting with autopkgtest v5 the adt-run binary no longer exists
and its command line interface slighly changed.
Thanks: Axel Beckert for bug report
Thanks: Christoph Berg <myon@debian.org> for feedback
If the extend-diff-ignore option is set in a dpkg format 1.0
package we still need to set the -i option, otherwise
we again end up with .git files inside the tarball and
dpkg-source fails with
| dpkg-source: error: cannot represent change to .git/[...] binary file contents changed
| [...]
| dpkg-source: warning: the diff modifies the following upstream files:
| [...]
While at it also ensure that we skip leading spaces, which is
what Dpkg::Conf is doing:
https://anonscm.debian.org/cgit/dpkg/dpkg.git/tree/scripts/Dpkg/Conf.pm#n172
Development sponsored by Sipwise GmbH, recorded as
TT#15013 in customers' ticket system.
Fixes 8fe677820f
Thanks: Guillem Jover for feedback and code review
If a debian git tree doesn't provide debian/source/format then
the dpkg 1.0 format is assumed. The 1.0 format needs according
dpkg-buildpackage defaults (-i -I), otherwise the .git tree
gets included in the tarball.
Development sponsored by Sipwise GmbH, recorded as
TT#13353 in customers' ticket system.
Fixes 8fe677820f
Thanks: Guillem Jover for feedback and code review
* fix 42a7ea8f94
* get_arch_changes() to get file expansion for reprepro cmdline
remove copy&paste and do not use architecture var
trunk_release is copying packages from $REPOS to $TRUNK_RELEASE,
if we have SKIP_REPREPRO_WRAPPER defined the packages are not
installed in $REPOS so $TRUNK_RELEASE will not receive the new
packages
If REPOS=... is unset by a custom script and only the release
steps should be executed, then without support for skipping the
reprepro_wrapper we end up with the Debian package being included
in $package-$distribution if $distribution is set or in $package
otherwise by default. If those steps are unwanted you can now
set SKIP_REPREPRO_WRAPPER to either take care of the repository
inclusion yourself or running only through the release steps (if
$release is set accordingly).
Development sponsored by Sipwise GmbH, recorded as
TT#9830 in customers' ticket system.
For example when the MAIN_ARCHITECTURE or the host architecture
doesn't match the package architecture, we build binary only
packages by setting DEBBUILDOPTS to -B. But when looking at
packages to remove from the repository, we remove the source
package too, although we have nothing to replace it with.
Thanks: Antoine Delvaux <antoine.delvaux@gmail.com> for the bugreport + initial patch
Closes#173 @ GH
If you have a debian/source/options with specific filter regexs for
diffs and/or tarballs, using a blank '-i' or '-I' will override them and
use the defaults.
We need to set up conf/incoming accordingly with:
Permit: unused_files
Cleanup: unused_files
at least until reprepro properly understands the .buildinfo files.
Related to previous commit (see github issue #165) +
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=843402
With subversion >=1.9 it fails hard if mergeWithUpstream property isn't present:
++ svn propget mergeWithUpstream source//debian
svn: warning: W200017: Property 'mergeWithUpstream' not found on 'source/debian'
svn: E200000: A problem occurred; see other errors for details
Closes#164
Remove pbuilderrc in the shell's exit handler, so that it
will be removed in all cases, in particular error paths.
Signed-off-by: André Draszik <git@andred.net>
If running in a container where /tmp/ is shared across
containers, it can otherwise happen that two instances
interfere with each other, due to sharing the same pid
inside each container.
Signed-off-by: André Draszik <git@andred.net>
We only create the default dirs now if we're also creating
the config file. Otherwise we parse the config file to
extract the values.
Also, FREIGHT_VARLIB has some permissions tightened, as it
doesn't need to be world accessible, http is served from
FREIGHT_VARCACHE
Signed-off-by: André Draszik <git@andred.net>
When building with the branch specifier set to a commit ID,
Jenkins detaches the HEAD and sets GIT_BRANCH to detached.
The `git checkout -f ${GIT_BRANCH}` at the end of
generate-git-snapshot fails because a branch named "detached"
does not exist.
Closes#17 @ GH
Thanks: Jan Alexander Steffens <jan.steffens@gmail.com> for bugreport + patch
When using dpkg-architecture and gcc isn't installed it complains about:
| % dpkg-architecture -qDEB_HOST_ARCH
| sh: gcc: command not found
| dpkg-architecture: warning: couldn't determine gcc system type, falling back to default (native compilation)
| amd64
Usage of `dpkg --print-architecture` should be enough for our needs.
Thanks: Guillem Jover for feedback
In cases where the source package comes already with a debian
folder which is located in some subfolder, it can be needed to
specify the path to that file explicitly for git-dch to work
correctly. That's what "$DCH_CHANGELOG_FILE" can be used for.
For example in the Kamailio repository at
https://github.com/kamailio/kamailio you see that there is no
debian folder in the project root. The debian folder is located
under pkg/kamailio/deb/$distribution. When symlinking this folder
to ./debian before starting the build, then git-dch inspects
./debian/changelog but it finds no commit for the latest
changelog entry. So we have to point git-dch to the changelog
file where it is actually committed. By using
"DCH_CHANGELOG_FILE=pkg/kamailio/deb/$distribution/changelog"
this works as intended. Otherwise git-dch will start building a
changelog entry for all commits that exist in the project, but
that will run forever.
So far we set the distribution/codename to $release. Now it's
possible to point distribution/codename to a different value by
using $RELEASE_DISTRIBUTION.
Usage example:
export release=0.42-update
export RELEASE_DISTRIBUTION=jessie
will result (with default settings) in a repository under
/srv/repository/release/0.42-update/ with a setting of
"Codename: jessie" inside
/srv/repository/release/0.42-update/conf/distributions
Development sponsored by Sipwise GmbH, recorded as
MT#17701 in customers' ticket system.
To allow pushing packages to multiple release repositories
we can now use something like:
RELEASE_REPOSITORIES="/srv/repository/release/ce/${release} /srv/repository/release/pro/${release}"
iff $release is set.
Development sponsored by Sipwise GmbH, recorded as
MT#17701 in customers' ticket system.
During release rebuilds we way too often see:
| W: Failed to fetch https://[snip]/autobuild/dists/release-trunk-jessie/main/binary-amd64/Packages Hash Sum mismatch
during the `cowbuilder --update` runs.
apt versions >=1.1 are supposed to be improve that situation,
but until we have this apt version available everywhere we
need to find workarounds.
This applies only in cases where the debian/changelog is explicitly
set to "UNRELEASED" which makes the generate-git-snapshot use plain
'dch' instead of 'gbp dch' or 'git-dch' which in turn passes the
generated version string directly as the new package version.
This is implemented as an additional option to avoid breaking
existing users which are just happy with the current behaviour.
Since git-buildpackage v0.6.24 the `git-buildpackage` and
`git-dch` commands no longer exist, instead we need to use
`gbp buildpackage` and `gbp dch`.
Closes: #799183
Thanks: Daniele E. Domenichelli <daniele.domenichelli@gmail.com> for the bug report
Otherwise when e.g. building Debian/jessie on Ubuntu 14.04 causes:
| ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
The REPOSITORY variable is used to set the main/base path for the
(reprepro) repositories (defaulting to /srv/repository). This
REPOSITORY setting could be defined via /etc/jenkins/debian_glue.
Since we are sourcing /etc/jenkins/debian_glue in our scripts
this overwrites a possibly existing environment variables named
REPOSITORY, preventing us to overwrite the REPOSITORY setting on
demand as needed as soon as REPOSITORY is configured in
/etc/jenkins/debian_glue.
To fix that behaviour we hereby rename the REPOSITORY variable to
DEFAULT_REPOSITORY while being backwards compatible. If you have
configured REPOSITORY via /etc/jenkins/debian_glue please rename
it to DEFAULT_REPOSITORY instead. In an upcoming release we might
drop backwards compatibility.
Thanks: Jean Baptiste Favre for the bug report
eatmydata support gets enabled by default if it's installed on
the host system and when building for a recent Debian/Ubuntu
version (Debian/jessie + Ubuntu/vivid or newer). Usage can be
forced via USE_EATMYDATA=true and disabled via
USE_EATMYDATA=false.
ccache support can be enabled via USE_CCACHE=true
Thanks to Franco (nextime) Lanza for the initial patch in
https://github.com/mika/jenkins-debian-glue/pull/125
If multiple builds are running at the same time then the
/var/cache/pbuilder/build directory shows up as shared bindmounts
in all environments. Besides being ugly this also causes the
first build environment to fail to cleanly unmount it and
therefore failing the build.
As a workaround the pbuilderrc could include the following
snippet, as suggested by Philipp Hahn:
| mount () {
| case "$1" in
| -obind) /bin/mount --make-private "$@" ;;
| *) /bin/mount "$@" ;;
| esac
| }
Leave this decision up to the user though (to avoid any possible
side effects depending on the user's environment which we can't
control).
The only reason why /var/cache/pbuilder/build should be needed as
bindmount at all is pbuilder-hookdir/C10shell. This pbuilder hook
copies the according build directory away on failing builds (and
if requested to do so). If this feature is requested and it
fails to access /var/cache/pbuilder/build then inform the user
that /var/cache/pbuilder/build should be added to the
USER_BINDMOUNTS configuration setting.
See http://lists.alioth.debian.org/pipermail/pbuilder-maint/2015-August/005368.html
Thanks: Philipp Hahn <hahn@univention.de> for the bug report and analysis
We need variables like COWBUILDER_BASE for usage with
the lockfile variables, otherwise when PROVIDE_ONLY is set
we fail with:
| /usr/bin/build-and-provide-package: line 130: build_lockfile: unbound variable
Without root permissions we might not have write access to
/var/cache/pbuilder/, so instead use /var/run/lock/ as base
directory for writing our lockfiles.
During running cowbuilder builds we shouldn't update the
underlying cowbuilder base.cow because the hardlinks might
bring the build environment into an inconsistent state.
Also we shouldn't run multiple update operations in parallel,
since this suffers from the same problem.
Therefore we have to implement an according blocking mechanism
between a) parallel update runs and b) concurrent update and
build runs.
Thanks: Christian Hofstaedtler <christian@hofstaedtler.name> for testing + feedback and Victor Seva <linuxmaniac@torreviejawireless.org> for helping with the actual implementation
Introduce the env variable JENKINS_DEBIAN_GLUE_QUIET to turn off the
bash option 'set -x' in various scripts.
To mute them, simply set the variable to any value.
At a customer of mine when executing repository_checker with its
--validate-incoming option to ensure that the incoming directory
is in a sane state we noticed problems with different compression
defaults. With the recent switch to Debian/jessie for the build
infrastructure we noticed that *.tar.xz files are left behind:
| 12:12:59 Checking for leftover files in incoming directory /srv/repository/release/release-mr0.42/incoming/release-mr0.42:
| [...]
| 12:12:59 /srv/repository/release/release-mr0.42/incoming/release-mr0.42/ngcp-ngcpcfg_0.26.1.1+0~mr0.42.1.tar.xz
| [...]
| 12:12:59 Leftover files found. Needs investigation.
This is caused by the fact that executing generate-git-snapshot
on Debian/jessie by default generates a *.tar.xz file nowadays,
quoting dpkg-source(1) :
| -Zcompression, --compression=compression
|
| Specify the compression to use for created files
| (tarballs and diffs). Note that this option will not
| cause existing tarballs to be recompressed, it only
| affects new files. Supported values are: gzip, bzip2,
| lzma and xz. The default is xz for formats 2.0 and
| newer, and gzip for format 1.0. xz is only supported
| since dpkg 1.15.5.
During the binary package build another tarball gets created via
dpkg-buildpackage's -sa option (to include the sources). In the
binary package build for an older release (like Debian/wheezy)
this won't default to *.tar.xz though. Therefore we end up with
an additional tarball, usually being *.tar.gz. Now we have a
*.tar.gz and a *.tar.xz next to each other and when simply
copying all files from binaries/ directory to the incoming
repository we end up with one unneeded tarball which is
identified as "leftover file(s) in incoming directory".
We sadly can't just default to build with dpkg-buildpackages' -b
option because then we don't get the source files in *any* binary
package build. The source files need to be present at least once
though to end up properly in the archive.
NOTE: When uploading artifacts e.g. via dput (instead of simply
copying them around) then this isn't a problem at all since
*.changes refers to the files that should be uploaded (and unused
files aren't included).
Since we execute this *only* when running through the release
repository steps (see release_repos()) it should be safe to
enable it by default. To disable this behaviour
DROP_UNUSED_DEBFILES=false still can be used.
autopkgtest >=3.16 no longer provides the --tmp-dir option, therefore
builds are failing with:
| + adt-run --tmp-dir /tmp/adt-15150//out --summary /tmp/adt-15150//summary [...]
| adt-run: error: unrecognized arguments: --tmp-dir /tmp/adt-15150//out
The --tmp-dir option was provided for backwards compatibility
reasons in adt-run. Now that this option is gone in recent
versions of autopkgtest let's use the --output-dir option
instead. To avoid breaking any setups with very old autopkgtest
where this option *might* not be present yet let's check for its
presence and fall back to --tmp-dir otherwise.
Thanks: Axel Beckert for the hint
Michael Prokop
Michael Prokop
klemens
Victor Seva
Will DeBerry
André Draszik
Alexis Bezverkhyy
Sebastian Damm
Antoine Delvaux
Kane
Michael Prokop
Antoine Musso
Walter Kleynscheldt