* $NGCP_TYPE is unset on SPCE by default, add checks for $NGCP_TYPE
being unset and load it manually for such cases as it should be
present on all NGCP platforms.
Change-Id: I5a8fafd1a376fadb2e15aa96db8d473d65b212d7
* on carrier proxy nodes ngcp-sync-db-grants and ngcp-sync-db-creds
are now also invoked for 127.0.0.1:3308 instance because
table grants and user credentials are not replicated by default
* change "Synchronizing MySQL grants/credentials" log string to
"Synchronizing MariaDB grants/credentials" to better reflect that
MariaDB is used as the database
Change-Id: Ie4e77147621df34fc4e7f0e333183fb3b313f260
When doing an «ngcpcfg apply» we are required to pass a reason for it
which might be included in a «git commit», but not in the etckeeper
commit. Modify the wording for the latter, and include that reason so
that the git history makes more sense and is more traceable.
So when executing:
# ngcpcfg apply "Fix array in component B"
The git commit message looked before like:
ngcpcfg apply
And now it will look like
Fix array in component B via "ngcpcfg apply"
Change-Id: Ib7b4ce986366ca51a487a087cb327b0d0dfe1ccd
The date is already tracked as part of the commit metadata, and we use
the current day which will always match the author date for the commit,
so no information is lost.
This makes the commit message more clear and removes redundant and
distracting information from it.
For «ngcpcfg log» we include the date from the commit metadata explicitly
in the output, which will be duplicated for old entries but gets it from
where it belongs for new ones. (Even though the date is also printed in
relative terms on the front of the line.) We now get this output for old
and new entries:
* b3767f6 (10 hours ago) new [2024-02-02 14:28:11 +0100] (HEAD -> master)
* dc0a310 (3 days ago) old [2024-01-30 23:10:32.674452749+01:00] [2024-01-30 23:10:32 +0100]
Change-Id: I4eb4faed433db52e6755bc7782ec34545741d452
* sync_extra_sockets() renamed into sync_lb_and_extra_sockets()
* extra_sockets_sync() renamed into lb_and_extra_sockets_sync()
* $config now also contains merged $NETWORK_CONFIG (network.yml)
* lb_and_extra_sockets_sync does the following and then calls
generic_enum_sync() to provide with the gathered data:
* all shared ips from interfaces that
- host have 'lb' role
- host status 'online'
- interface has type '^sip_'
are added into the outbound sockets list of protos (udp, tcp, tls)
as: 'host:interface:proto' = 'proto:shared_ip:port'
(port is taken from config.kamailio.lb.port in case of ('udp', 'tcp')
and config.kamailio.tls.port in case of 'tls'
)
- if an interface has multiple shared IPs,
the key becomes 'host:interface:shared_ip:proto'
- $config->{kamailio}{lb}{extra_sockets} keys are merged
with the gathered config
* generic_enum_sync() - processes hash keys are now sorted
to have consistent output/processing
Change-Id: I218f1ae584e2dc9dfeb3a51c43d9f3fc950e829b
These commands are db-specific, and the constants one involves the
database credentials, so give both better names to make it clear these
are not general purpose commands.
Preserve backwards compatibility symlinks for external callers, or
user muscle memory, which this gets migrated away.
Change-Id: I3baae364e786ebbdc9e386dfc4f8c0bf54333cd1
We need to make sure the permissions for any config file are set
correctly, otherwise on the next build action these will be updated,
which might cause spurious commits just for these updates, that can
end up generating rebase conflicts.
Change-Id: I7c0f4cf13593da0d696ba25097b202e0b7a33674
It is necessary to allow bypassing '--ignore-shared-storage-check':
> ngcpcfg apply --ignore-shared-storage-check
Previously 'apply' options were not passing to 'build' script.
* docs:
- add all supported options in *apply*
- move '--force-all-services' description to *services*
and add it in the list of options
Change-Id: I091142d1f74b7b9e09ce94b963ff4bf9dfaa8db7
SC2259/SC2260 were added in ShellCheck v7.0.2
("Warn when redirections override pipes")
Fixes:
| In scripts/commit line 91:
| ngcp-sync-constants >/dev/null | sed "s/^/$timestamp_replacementchars/"
| ^-- SC2260: This redirection overrides the output pipe. Use 'tee' to output to both.
While at it, fix incomplete log_debug regarding no-db-sync,
thanks Alexander Lutay <alutay@sipwise.com> for spotting.
Change-Id: I92e5ce73dc000a52fa7c5c79f710d87cec292908
shellcheck v0.7.1 complains about a bunch of issues:
SC1083: This { is literal. Check expression (missing ;/\n?) or quote it.
SC1083: This } is literal. Check expression (missing ;/\n?) or quote it.
SC1090: Can't follow non-constant source. Use a directive to specify location.
SC2034: foo appears unused. Verify it or export it.
SC2059: Don't use variables in the printf format string. Use printf '..%s..' "$foo".
SC2128: Expanding an array without an index only gives the first element.
SC2148: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.
SC2155: Declare and assign separately to avoid masking return values.
SC2162: read without -r will mangle backslashes.
SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.
SC2207: Prefer mapfile or read -a to split command output (or quote to avoid splitting).
SC2237: Use [ -z .. ] instead of ! [ -n .. ].
We're still bound to support shellcheck v0.3.4 and sadly this version
fails hard when using the modern "shellcheck source=" directives to
address SC1090 and SC1091. It's failing like:
| not ok 1 source/functions/init:8:28: note: The mentioned parser error was in this then clause. [SC1009]
| not ok 2 source/functions/init:10:3: error: Couldn't parse this shellcheck annotation. [SC1073]
| not ok 3 source/functions/init:10:16: error: Unexpected "s". Fix any mentioned problems and try again. [SC1072]
So until we can get rid of supporting this ancient shellcheck version,
instead of using:
| # shellcheck source=etc/ngcp-config/ngcpcfg.cfg
| # shellcheck source=functions/init
| # shellcheck source=functions/logs
| # shellcheck source=functions/main
we need to place disable=SC1090 and disable=SC1091 accordingly.
Change-Id: I14da87a00102609bffdb60e932045180c6d13d5e
When running the test suite, in case we need to cover the code paths
that are handling the hooks, we should not assume the files are
installed in the system directories.
Change-Id: I1cd09eaf186d28093eb03ad5e9fc096fe7d63a3e
'git status' output has been changed in 2.9 which causes some troubles for us
(we didn't change all the places, we missed ngcpcfg-ha.git repo,
better to use common code in all the places).
Change-Id: Ie664af89ee6d6b31fb3011e763107c41bf4cf00f
When installing Debian/stretch we fail at this stage:
| ngcpcfg --no-db-sync commit
On first glance it looks like a mariadb issue, though actually
it's the git status check inside ngcpcfg that's failing. The
reason is that starting with git v2.9.3 there's a different
output in its "status" output:
| commit 2a0e6cdedab306eccbd297c051035c13d0266343
| Author: Lars Vogel <lars.vogel@gmail.com>
| Date: Thu Jun 9 20:19:30 2016 +0200
|
| Use "working tree" instead of "working directory" for git status
|
| Working directory can be easily confused with the current directory.
| In one of my patches I already updated the usage of working directory
| with working tree for the man page but I noticed that git status also
| uses this incorrect term.
|
| Signed-off-by: Lars Vogel <Lars.Vogel@vogella.com>
| Signed-off-by: Junio C Hamano <gitster@pobox.com>
In Debian/stretch we've 2.11.0-2 and are affected by this change.
Adjust the `git status` checks accordingly to support git in all
its versions.
NOTE: we should consider switching to `git status --porcelain`
(see git commit fc5af3bbb2 for
some history about it) once we've the testsuite in place.
Change-Id: If4223a35083d279e746dce5a80bcb86b52f64025
/var/log/ngcp/ngcpcfg.log might not contain error messages from
ngcpcfg's sync-db script. This might be confusing when comparing
output from the command line with the logged actions from
ngcpcfg.log. As sync-db doesn't log all error messages to stderr
anyway and we ignore exit code of the sync-db script as well (on
purpose) let's just redirect stderr and stdout to the same file
descriptor and then provide output to the console as well as
syslog's ngcpcfg tag.
Output before this change:
| root@sp1:~# ngcpcfg commit "before"
| 2016-10-25 22:12:20: Executing sync-db:
| DBI connect('database=provisioning;host=localhost;port=3306','sipwise',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/ngcp-ngcpcfg/helper//sync-db line 70.
| Error: Could not connect to database 'provisioning' at 'localhost:3306' as 'sipwise': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
| 2016-10-25 22:12:20: OK: nothing to commit.
| 2016-10-25 22:12:20: Synchronizing MySQL grants/credentials
| Can't connect to MySQL database 'mysql': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/sbin/ngcp-sync-grants line 66.
| root@sp1:~# tail -f /var/log/ngcp/ngcpcfg.log
| [...]
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: Executing sync-db:
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: OK: nothing to commit.
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: Checking state of /etc files
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: OK: nothing to commit.
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: Synchronizing MySQL grants/credentials
Output after this change gets applied:
| root@sp1:~# ngcpcfg commit "after"
| 2016-10-25 22:12:59: Executing sync-db:
| DBI connect('database=provisioning;host=localhost;port=3306','sipwise',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/ngcp-ngcpcfg/helper//sync-db line 70.
| Error: Could not connect to database 'provisioning' at 'localhost:3306' as 'sipwise': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
| 2016-10-25 22:12:59: OK: nothing to commit.
| 2016-10-25 22:12:59: Synchronizing MySQL grants/credentials
| Can't connect to MySQL database 'mysql': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/sbin/ngcp-sync-grants line 66.
| root@sp1:~# tail -f /var/log/ngcp/ngcpcfg.log
| [...]
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: Executing sync-db:
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: DBI connect('database=provisioning;host=localhost;port=3306','sipwise',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/ngcp-ngcpcfg/helper//sync-db line 70.
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: Error: Could not connect to database 'provisioning' at 'localhost:3306' as 'sipwise': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: OK: nothing to commit.
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: Checking state of /etc files
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: OK: nothing to commit.
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: Synchronizing MySQL grants/credentials
Change-Id: Iabb0c0ab24e212b79b0dfa85947cb274a161cc00
Git doesn't track file permissions (except for the executable
flag). For sensitive data (like the 'ssl' directory and file
'constants.yml' with passwords included) we've to prevent
non-root users from accessing those files.
hooks/pre-commit is inspired and based on the implementation
as present in etckeeper (and luckily we're license compatible)
and takes care of storing the file permissions inside file
/etc/ngcp-config/.ngcpcfg_perms.
The restore-permissions helper script takes care of restoring the
permissions after cloning the ngcpcfg repository via ngcpcfg
itself (being actions decrypt, pull (PRO-only) + initialise
(PRO-only)). It can be executed manually as well via
`usr/share/ngcp-ngcpcfg/helper/restore-permissions /etc/ngcp-config/`
(or wherever the according ngcpcfg repository is placed at).
Regarding the commit integration: git(1) itself doesn't track
file permissions, so we can't detect changes to file permissions
using git itself. Our new pre-commit hook records file
permissions via the .ngcpcfg_perms file. Now by just invoking it
during 'ngcpcfg commit' time we can ensure that even if there
have been any file permission changes in the working directory
the file .ngcpcfg_perms is then up2date and committed.
JFTR: The solution via the git pre-commit hook ensures that no
matter whether you're using 'ngcpcfg commit …' or 'git commit …'
you always get the file permissions handled via .ngcpcfg_perms.
Now if you want to change file permissions in a clean working
directory and commit *without* using 'ngcpcfg commit' but
directly via git itself then you've to use 'git commit
--allow-empty ...' and thanks to the pre-commit hook the file
.ngcpcfg_perms will still be up2date.
Change-Id: I84d608585c626b52112ff649893e232e441c59d8
- ngcp-sync-grants is responsible for ngcp mysql
grants sync from a template
- it is executed by the 'commit' trigger before ngcp-sync-constants
Change-Id: I082256e57b1394a3f056ad1ca56a5443bfb5a745
Otherwise users notice error during upgrade mr3.7->mr3.8:
> ngcpcfg commit 'snapshot during /usr/share/ngcp-upgrade-mr3.8/upgrade-2.8-to-mr3.8 - Mon Apr 13 10:29:24 CEST 2015'
> YAML Error: Couldn't open /etc/ngcp-rtpengine-daemon/interfaces.yml for input:\nNo such file or directory
> Code: YAML_LOAD_ERR_FILE_INPUT
> at /usr/share/perl5/YAML.pm line 71
> OK
> Synchronizing data from /etc/ngcp-config/constants.yml
Change-Id: I2e4b8780d2ada64854186da93b03b8817eac99bf
Kirill Solomko
Guillem Jover
ngcp-config
Mykola Malkov
Michael Prokop
Alexander Lutay
Richard Fuchs