The ArtifactDeployer isn't the prefered choice and there's
no reason to make BASE_PATH handling more confusing than necessary,
so just drop the userContent magic.
If we have a package named ngcp-templates-ce-hylafaxplus then
this will match within the string ngcp-templates-ce-hylafaxplus-iax
and therefore will be skipped as in the missing_packages list.
So make the grep match only leading and trailing white space characters.
In the dcmd(1) listing there are also dsc, tar.XX and changes files
included, no need to run the binary removal for them as they either
don't exist or are duplicates.
I do not want to support anything other than reprepro for
repository handling and if someone needs a simpler solution than
reprepro it should be a no-brainer anyways.
include-in-repos on the other side was the idea to split
build-and-provide-package into a build and repository only step
for scaling the build pipeline. Nowadays we also do support
BUILD_ONLY and PROVIDE_ONLY inside build-and-provide-package, so
there is no reason to duplicate that code any longer.
Stuff from include-in-repos which turned out to be useful and
if we should ever need it again it's available inside git log to
easier stumble upon it:
* reprepro includedsc
* iff section/priority is empty then reprepro will complain
about "No section and no priority for" and error out in the
cmdline above, therefore we retry with -S and -P being set
${SUDO_CMD:-} reprepro -v -S unstable -P extra -b "${REPOSITORY}" …
Not only git-dch, but (now?) also git-buildpackage refuses to operate
on a single git commit, too, so remove the temporary branch later in
the workflow, after git-buildpackage has been called.
There's no need to report what settings would be chosen for
executing cowbuilder if we're interested only in the
repository step, so just skip the code then.
We we can't do just a grep for "remotes/origin/debian when"
trying to check out the "debian" branch, instead we need to
check for a trailing '$' at the end of the grep string to
avoid matching e.g. remotes/origin/debian-lenny then.
Thanks: Axel Beckert
identify_build_type expects identify_sourcefile to always set
SOURCE_PACKAGE, set it to "*" in this code path. As it is used as a glob
pattern later, this works good enough for now.
We can't use git-dch's auto mode if we've to deal with
a distribution being set to UNRELEASED because that fails
with:
| + git-dch --auto -S --multimaint-merge --ignore-branch --new-version=1.16.9~1343298526.109
| Changelog last touched at '6293f12440f48c1e245033801f151e4c99114db7'
| Continuing from commit '6293f12440f48c1e245033801f151e4c99114db7'
| No changes detected from 6293f12440f48c1e245033801f151e4c99114db7 to HEAD.
| dch: fatal error at line 984:
| New version specified (1.16.9~1343298526.109) is less than
| the current version number (1.16.9)! Use -b to force.
| dch --no-auto-nmu --distribution=UNRELEASED --newversion=1.16.9~1343298526.109 --multimaint-merge -- "UNRELEASED" returned 25
| Couldn't run ' dch --no-auto-nmu --distribution=UNRELEASED --newversion=1.16.9~1343298526.109 --multimaint-merge -- "UNRELEASED"'
So instead run dch with custom command line to get a
version number being smaller than the one that will be
released later on.
The checkout of the branch might be behind, so just use the
$GIT_COMMIT we get. This addresses situations like:
| + git checkout master
| Previous HEAD position was 6293f12... Bump version to 1.16.9
| Switched to branch 'master'
| Your branch is behind 'origin/master' by 426 commits, and can be fast-forwarded.
Use our "manual" way of identifying latest merge/tag (known
as identify_latest_change in the implementation) only if
invoked with cmdline option "identify", otherwise default
to git-dch's auto mode which does a better job in most situations.
While at it slightly refactor the identify_latest_change code
and also drop the "j" from git rev-list's --tag option.
On e.g. Ubuntu "sid" isn't supported, so we can't just hardcode it.
Instead support setting COWBUILDER_DIST via /etc/jenkins/debian_glue
and if it's not set then use "$distribution" if it's set, otherwise
default to the currently running distribution and just fallback to
"sid" if none of all of that worked.
Thanks: Daniel Thomas for the detailed bugreport
Quoting Brian:
| The other change I made in build-and-provide-package, was to
| automatically do a binary-only build if the arch specified does not
| match the main architecture and/or the architecture of the host:
|
| http://pastebin.com/mhJGHpV5
|
| Otherwise, both my amd64 and i386 configurations build with the
| default "-sa" build option, each generating a changes file that
| references a .dsc file with a different checksum. I'm thinking the
| best approach long term might be to have the -source job actually
| upload the source package to reprepro leaving
| build-and-provide-package to always just build with -B. Your
| thoughts? In the meantime, I've just made the change above.
|
| It looks like you were addressing the problem on your end as well with
| (I can see it executed in your dpkg jobs on http://jenkins.grml.org):
|
| if grep -q '^Architecture: all' "$control_file" ; then
| if grep -q '^Architecture: any' "$control_file" ; then
| echo "*** Package provides arch 'all' + 'any', enabling -B
| buildoption for this architecture. ***"
| # -B -> binary-only build, limited to architecture dependent packages
| DEBBUILDOPTS="-B"
|
| However, this code won't work as intended if, for instance, a control
| file contains:
|
| Architecture: i386 amd64
Thanks: Brian Thomason <brian.thomason@eucalyptus.com> for the initial patch
Try to check out branches upstream, debian and pristine-tar
which are known to be common branches for building git
projects.
If environment variable $branch is set check out this branch,
otherwise fall back to $GIT_COMMIT as we used to do so far.
I'm still not entirely happy with this but I'm not aware of
anything better yet...
This brings generate-git-snapshot closer to what generate-svn-snapshot
already provided, jey.
Thanks: Raphael Hertzog <raphael@ouaza.com> for the suggestion
When moving Debian package builds from one CI system to another one
then the build number (BUILD_NUMBER) won't necessarily be the same
anymore. So do the same as in generate-svn-snapshot: include the
timestamp in the version string.
'${release}' is a hidden bomb: when provided through predefined parameters
from an upstream jenkins job (like foo-binaries receiving the parameters
from foo-source) but the job (foo-binaries) gets triggered manually (without
setting the predefined parameters therefore) then ${release} is set to
'${release}' instead of being empty
If you want to have TRUNK_RELEASE enabled by default but skipping certain
builds, e.g. if you're building tagged or branch builds then you can easily
skip inclusion of such builds using something like:
case "$branch" in tags/*|branches/*) export IGNORE_RELEASE_TRUNK=true;; esac
Support setting PROVIDE_ONLY for skipping cowbuilder stage
and BUILD_ONLY for skipping reprepro repository stage.
Drop REMOTE_REPOS configuration variable, as the name actually
doesn't reflect what the script supports nor is it of any use
in the main configuration file when having PROVIDE_ONLY and
BUILD_ONLY available.
This provides the base for using $job-source, $job-binaries and
$job-repos to get the build process scaling and distributed on
specific slaves.
git-buildpackage should provide just the according magic like
building the orig.tar.gz, but dpkg-source should do the rest™.
This avoids running the 'debian/rules clean' step which might
include execution of insecure commands.
Disclaimer: So far this works fine for all the packages I tested,
let's see how well this works in the long run...
We can't remove a package from the repository in the step where
we're checking for missing binary packages (to get rid of packages
which are no longer provided but shouldn't stay in the archive)
if the package is an 'Architecture: all' one, as e.g. the i386
build might remove the package from the amd64 build.
So let's check for "-B" in DEBBUILDOPTS and then skip the removal
process in such a situation...
If we're building a package version where Debian's build tools
don't automatically consider it as new upstream version and therefore
don't include orig.tar.gz into the upload the build process fails.
We already depend on dpkg-dev but don't have libparse-debianchangelog-perl
in our depends, so use in generate-git-snapshot the same command as in
the other scripts.
If we switch the directory we might end up in the wrong directory
for further actions, so run all the commands around svn-buildpackage
in a subshell.
Also replace the --svn-move command with --svn-move-to to place
the artifact files in the original directory, otherwise the files
might be placed in the wrong directory if $branch is set and used.
Michael Prokop
Axel Beckert
Christoph Berg
Michael Prokop
Michael Prokop