Rename gpg to pgp or OpenPGP and remove crypted from filenames.
The standard is called OpenPGP, GnuPG (or GPG, gpg) is one of many
implementations (although depending on the context the prevalent one)
so we use that when referring to the specification.
For the encrypted tarball we remove the «-crypted» term which is an
odd wording use for encryted content and it is unnecessary and confusing
when using layered containers (as when the tarball is decrypted the
name becomes invalid); and make it use «.pgp» instead of «.gpg» as
extension, while trying the old filename on decryption for backwards
compatibility.
Change-Id: If5e0349ff0c3d8e3b47ced361e9e77d7d4d2defc
Usage of IP addresses like 1.2.3.4 + 2.3.4.5 is not recommended, as they
might point to actually used and non-reserved IPs.
Quoting from RFC 5737 AKA "IPv4 Address Blocks Reserved for
Documentation" (see https://datatracker.ietf.org/doc/html/rfc5737):
| The blocks 192.0.2.0/24 (TEST-NET-1), 198.51.100.0/24 (TEST-NET-2), and
| 203.0.113.0/24 (TEST-NET-3) are provided for use in documentation.
So use 203.0.113.0/24 in our docs/code/testsuite instead.
Change-Id: Ic786a12c006a1fe11b67b788e01cca377f08771f
In ngcp-initial-configuration there are a lot of multiple sequential calls
of 'ngcpcfg set' which consume significant amount of time.
So add support for setting multiple options to do it at once.
Change-Id: I8cacdbec78ecefc4681048a0fb085b4cb705d83e
The configuration files specific to the ngcpcfg tool should not be
mixed with the site specific configuration, as that's rather confusing,
more so when the actual ngcpcfg configuration contains references to the
location of the ngcp-config dir itself, which makes it self-referential.
Change-Id: I93f59e2ce3441c2c315956f68e22e4b2e9c839ac
The rtpengine project used to be called rtpengine-ng, and before that
mediaproxy-ng, which was an implementation using the same protocol as
the rtpproxy project. It's about time we rename the config.yml section
so that this is less confusing.
Change-Id: I51fc5effc515ccca74b4d2093b9665edb20286f8
This special handling was from a time when we did not have any other
generic suffix support to override based on hostname, nodename, role,
etc. It also depends on the now obsolete systems.cfg file which is
redundant and already covered by the information contained in
network.yml.
Our tt2-process tool already handles these suffixes, so there is no
point in special-casing them from within the build script. This way we
simplify our templating and overriding mechanism, and reduce duplication
of information.
Change-Id: Iabf597dbaddf755a7f85504975afc8666a58d272
Starting with mr10.0 heartbeat-2 is no longer supported, starting with
mr10.5 (the next LTS) heartbeat-2 is no longer available.
Use better terminology to avoid confusion.
Change-Id: I585d79f83e12db92df7068f4fbd1ef798ba7e1d0
The HA_NODE, HA_FILE and HA_CONFIG variables and nomenclature are
deprecated in favor of the NGCP_NODENAME, NODE_FILE and NODE_CONFIG.
Switch the old uses to the new ones. Unify PAIRNAME to NGCP_PAIRNAME.
Leave the old HA_ variables for backwards compatibility
Change-Id: I11f09f1620c2e456f10fc17229b61ec5a35a732b
This new action concatenates the various configuration files and outputs
the result as YAML. This is useful because they have a determined
ordering, and include optional host specific files that might not be
obvious how to merge.
This will also be the foundation for verifying all the current host
settings against the cfg-schema.
Change-Id: I72a61193f74caf3b2f7a58a47eefad2ed46c973a
The prevalent convention in the project is to use «-» instead of «_» to
split words in options. Switch the options to that, and keep backwards
compatibility option alias that emits a deprecation warning while people
transition to the new names. The alias will be removed on the next LTS.
Change-Id: I8bb7f26f8a766f7ea76ba2d9e49b793e5cb8d11e
we need to produce different output for each instance defined
from the same source
* helper/instances-info, produces the needed output to feed
helper/tt2-process with the proper arguments to build
instances files. We inject ``instance_name`` with the
proper value to templates and INSTANCE_NAME to hooks
environment
* ${NGCPCTL_MAIN}/ngcpcfg.instances is where we define the
supported instance templates and where the output path is
* helper/tt2-process, add -r --replace <path:path> option
in order to allow generation of different output from
the same source
Change-Id: I067266b5226485ea0d6c3bc3436275c5553a8177
It is necessary to allow bypassing '--ignore-shared-storage-check':
> ngcpcfg apply --ignore-shared-storage-check
Previously 'apply' options were not passing to 'build' script.
* docs:
- add all supported options in *apply*
- move '--force-all-services' description to *services*
and add it in the list of options
Change-Id: I091142d1f74b7b9e09ce94b963ff4bf9dfaa8db7
Normally it is necessary to press TAB multiple times, e.g:
> vim /etc/ngcp-conf<TAB>/con<TAB>f<TAB>...
For new NGCP users it is even longer.
In the same time we have a nice usability experience for ngcp-ppa,
where users are just pressing "ngcp-ppa<ENTER>1<ENTER>1<ENTER>".
Let's add the same concept here as the order of the main files is static:
> 1) /etc/ngcp-config/config.yml
> 2) /etc/ngcp-config/network.yml
> 3) /etc/ngcp-config/constants.yml
if I need to edit network.yml I will type:
> ngcpc<TAB> e<TAB><ENTER>2<ENTER>
or even:
> ngcpc<TAB> e<TAB>2<ENTER>
Change-Id: Id20fec8c931c15c73a73689a3f8b552a72ef11f7
Switch away from the old daemon/wrapper architecture, which complicates
things and requires more scaffolding code.
We turn the daemon/wrapper and build_config into a single perl process
that will load all YAML files, and then process each input/output pair
on a parallel child, to try to speed up the processing as much as
possible.
For whole rebuilds, it might speed up the generation by at least a
factor of x2, x4 or more, depending on the number of active processors.
Change-Id: I51aa2f90336e34a20983d8733f45b64d9b6fea0b
Add '--join' option which is used in init-shared command.
We should not guess the required action but it should be defined either
it's necessary to initialize git repo in shared storage or should use it
as source of ngcpcfg files.
Change-Id: I4b0924fc6b35a718e9dfd5019568c0c3a7a37a9c
It was a widely popular question from users.
As the common sense pushes them to use 'get' when they use 'set/del'.
Let's follow the common logic here.
We still support old 'ngcpcfg values' syntax.
Change-Id: I40a210b7c13d21a4bff4b6988535c39091f53280
Sometimes (e.g. on 'push-parallel') we need to call ngcpcfg on
remote host and generating summary only output about the results.
It simplifies complex 'push-parallel' codebase which is critical
for huge Carrier installation.
P.S. we cannot print summary message all the time to console output,
as a lot of code depends on the ngcpcfg output, e.g.:
> root@web01a:~# ngcp-ppa gerrit_alutay_63758_push_parallel_2
> ERROR: There are uncommitted changes in config files:
> 2019-10-02 11:59:38 web01a: Successfully executed 'diff' on 'web01a'
> ERROR: Please commit them or discard
> root@web01a:~#
Instead, let's be backward compatible with printing on console,
but always print this message to log file (as we sometimes need to know
ngcpcfg exit code and it was impossible to find it in log file previously.)
Change-Id: I9aad0912796500cddb23d1f3eaa428b0b39152d0
- Remove epub output as asciidoctor requires unpackaged support.
- Remove html and pdf output as it is not known to be used anyway.
Change-Id: I7089e01ef17dbc40df08b796f48bcd8616936145
It produces security issues when diff is stored in logs (e.g. upgrade logs)
and may contain passwords or another secrets. The special option '--diff'
is introduced for users who require old behaviour.
Change-Id: Ie4fd1ef2ee7fe1a51e87a99e36b5f71f6ecc8084
This is necessary when the HA_INT addresses are being changed, the communication
is not possible in that scenario.
Change-Id: I658a4f40e4f5b13d6ac224978999789728d8b915
Change introduced in commit bc1095e596 as part of
the task:
TT#37257 Add option 'ngcpcfg apply --force-all-services' to force all services restart
Change-Id: Iabfd6c78091db98a74a41a0d050f7d090f9443a7
We need to initialize ngcpcfg without HA repo,
to solve chicken-eggs problem about git hooks handking.
Also it will be useful to build glusterfs configs
from tt2 templates and yml configuration.
Then start glusterfs and store HA repo on glusterfs.
In this case ngcp-installer will be able to call:
> ngcpcfg initialise --without_shared
> ngcpcfg set ...
> ngcpcfg build ...
> ngcpcfg init-shared
> ngcpcfg apply "Applying everything together"
In the past it was not so critical problem until we started
taking care about the files permissions which are coming from
ngcp-template-* packages and transferred to POSIX perms via ngcpcfg.
One of the problems there is 'git clone' (which is a part of
HA setup initialisation). It resets file perms if they are not
yet stored into file '.ngcpcfg_perms'.
Change-Id: I3cdc93f309f0b899e626779afb1e71f64b2cb953
Script init-mgmt is installed on PRO installation so change
description to the proper one - script is usable only on Carrier
installation.
Change-Id: I7a7904183e30c4146267d73fbfc1075b2b066d08
We are in the process of removing the obsolete init scripts, so trying
to use them directly will make these operations not work at all.
Change-Id: I5d974ae5e744af5723e50263b767705adc390682
The new function will simplify initial customtt->patchtt
migration for end users. Some code was merged to be generic.
Also more tests were add here.
Change-Id: I7719f45275018818b2db82f6deee5b7428670a29
The 'ngcpcfg' received support for 'patchtt' files, like
> /etc/ngcp-config/templates/etc/foo/bar.patchtt.tt2
Those 'patchtt' are going to be applied on default 'tt2 template' file:
> /etc/ngcp-config/templates/etc/foo/bar.tt2
and produce 'customtt' on 'ngcpcfg patch':
> /etc/ngcp-config/templates/etc/foo/bar.customtt.tt2
Further 'customtt' will be used to overwrite 'tt2 templates'
on 'ngcpcfg build' or 'ngcpcfg apply'.
NOTE: 'ngcpcfg patch' is executed automatically on every 'ngcpcfg build'.
It should allows to update ngcp-templates easily and support
local modifications without the pain (until the patches can be applied).
Change-Id: Ice4369386313c5d33e4d498346345eade6f3d0d7
This option is essential if we're testing from within the git
repository and aren't standing on active 'master' branch.
Change-Id: I75b62d071116313317c69ad496e4425d6e98ad5a
It is hard to clean ngcpcfg framework for users with
limited git knowledge, lets introduce action 'clean'.
It should allows users easily reset to 'previous safe state'
in the case 'if something went wrong'.
Also remove old and unreliable error handling hint from manuals,
as we have switched to fast-forward rebase long time ago.
Change-Id: I961e681d55cac15ba8d772b9345c668218313bf4
If we have multiple web0X proxies but need specific
templates on just one of those pairs then we need to
support *.customtt.tt2.$pairname + *.tt2.$pairname
templates.
While at it also support *.customtt.tt2.$nodename +
foo.tt2.$nodename (where nodename is actual hostname
and not sp1 or sp2 as already supported already).
Test script for demonstration (especially for precedence):
mkdir /etc/ngcp-config/templates/etc/mika
cd /etc/ngcp-config/templates/etc/mika
hostname=$(hostname) # e.g. web01b
pairname=${hostname%[ab]} # e.g. web01
nodename=$(cat /etc/ngcp_nodename) # e.g. sp2
for file in foo.customtt.tt2.$hostname foo.customtt.tt2.$pairname foo.customtt.tt2.$nodename foo.customtt.tt2 foo.tt2.$hostname foo.tt2.$pairname foo.tt2.$nodename foo.tt2 ; do echo $file > $file ; done
for file in foo.customtt.tt2.$hostname foo.customtt.tt2.$pairname foo.customtt.tt2.$nodename foo.customtt.tt2 foo.tt2.$hostname foo.tt2.$pairname foo.tt2.$nodename foo.tt2 ; do
ngcpcfg build /etc/mika/ >/dev/null
if [ "$(cat /etc/mika/foo)" = "$file" ] ; then
echo "Content of file /etc/mika/foo matching $file"
rm "$file" ; rm /etc/mika/foo
else
echo "Problem with precedence of file $file"
return 1
fi
done
Change-Id: If7c222374bb56d7bf243200b37a414e39e050cd4
Git doesn't track file permissions (except for the executable
flag). For sensitive data (like the 'ssl' directory and file
'constants.yml' with passwords included) we've to prevent
non-root users from accessing those files.
hooks/pre-commit is inspired and based on the implementation
as present in etckeeper (and luckily we're license compatible)
and takes care of storing the file permissions inside file
/etc/ngcp-config/.ngcpcfg_perms.
The restore-permissions helper script takes care of restoring the
permissions after cloning the ngcpcfg repository via ngcpcfg
itself (being actions decrypt, pull (PRO-only) + initialise
(PRO-only)). It can be executed manually as well via
`usr/share/ngcp-ngcpcfg/helper/restore-permissions /etc/ngcp-config/`
(or wherever the according ngcpcfg repository is placed at).
Regarding the commit integration: git(1) itself doesn't track
file permissions, so we can't detect changes to file permissions
using git itself. Our new pre-commit hook records file
permissions via the .ngcpcfg_perms file. Now by just invoking it
during 'ngcpcfg commit' time we can ensure that even if there
have been any file permission changes in the working directory
the file .ngcpcfg_perms is then up2date and committed.
JFTR: The solution via the git pre-commit hook ensures that no
matter whether you're using 'ngcpcfg commit …' or 'git commit …'
you always get the file permissions handled via .ngcpcfg_perms.
Now if you want to change file permissions in a clean working
directory and commit *without* using 'ngcpcfg commit' but
directly via git itself then you've to use 'git commit
--allow-empty ...' and thanks to the pre-commit hook the file
.ngcpcfg_perms will still be up2date.
Change-Id: I84d608585c626b52112ff649893e232e441c59d8
Mainly nobody validate the schema in trunk right now
as it requires manual changes for ngcpcfg.cfg.
Also having force validation will allow us to add
validation in ngcp-upgrade and Jenkins nightly builds.
Change-Id: Ia48a778ecf7dae30b26bdff0c5dd5b9000c9d089
Guillem Jover
Michael Prokop
Mykola Malkov
ngcp-config
Alexander Lutay
Manuel Montecelo