Quoting from https://twitter.com/hackebrot/status/873083574676733953:
| pytest looks up fixtures based on the import path etc.. Importing
| then creates an internal copy, which messes with the cache etc.
Change-Id: I3ee6f5ab859218ce0cfe8681505b670952e47df6
If a template named
/etc/ngcp-config/templates/etc/apt/apt.conf.d/71_no_recommended.tt2 exists
the resulting destination file would be
/etc/apt/apt.conf.d/71_no_recommended. Though if
/etc/apt/apt.conf.d/71_no_recommended exists as directory ngcpcfg should
abort and inform the user about the existing situation with a useful error
message.
Change-Id: I2c6e1e3a4ec485183674c1fe72251631ad9867ac
The old testsuite wasn't updated for way too long and since
ngcpcfg receives more and more features we need a decent test
coverage. pytest seems to provide the right level of
abstraction, excellent fixtures and junit-xml reporting as
needed.
Inspired by Vincent Bernat's
https://github.com/vincentbernat/lldpd/tree/master/tests/integration
Thanks Victor Seva <vseva@sipwise.com>, Vincent Bernat <vincent@bernat.im>, Christian Hofstaedtler <christian@hofstaedtler.name> and Lukas Prokop <admin@lukas-prokop.at> for feedback, inspiration and help
Change-Id: Iffed87e8cc540169bed89c00967a03e80859179e
This reverts commit 8a6053a104.
We can't run autopkgtest runs inside *-binaries jobs yet
since they don't support PPAs which we depend on.
So until we've a working solution for this let's revert this.
Change-Id: I99035ca33049ca6f9c698fd78c9064504cf2409b
This let's us control the directory where files should be written to.
Otherwise we'd have to write to the system's /etc/ + /var, which is
clearly unwanted.
Change-Id: I1010afcd4637928868fc81a0adfafebd3bbce874
This option is essential if we're testing from within the git
repository and aren't standing on active 'master' branch.
Change-Id: I75b62d071116313317c69ad496e4425d6e98ad5a
Make sure ngcpcfg binary is available and installed
appropriately for execution.
Provide placeholder files for the configuration files
that are needed for ngcpcfg execution.
This serves as a starting point for further system tests
during Debian package build time.
Change-Id: I70dca5e4bed6a896f31c3a3eb909da17e496fc92
At the moment we provide no output for the end users, which
confuses them for the checks with and without the validation.
We should clearly report to users what has been checked.
Change-Id: Icc61744637345a1dc51fe8d8dd9bd3fd7f86824d
It is hard to clean ngcpcfg framework for users with
limited git knowledge, lets introduce action 'clean'.
It should allows users easily reset to 'previous safe state'
in the case 'if something went wrong'.
Also remove old and unreliable error handling hint from manuals,
as we have switched to fast-forward rebase long time ago.
Change-Id: I961e681d55cac15ba8d772b9345c668218313bf4
ticket 13069 describes in detail the additions for event-based
billing required by a customer of the hpbx platform. these
additional edr fields will allow to export a single column
showing the subscriber alias for both start and stop profile
events.
Change-Id: Ibd2f9b05ed708521b23cf3db09c68267cf085e3b
In templates' bootenv/sources.list.d/{debian,sipwise}.list we
don't want to hardcode the Debian release, but instead make sure
it matches the underlying Debian release.
Change-Id: I9c844fa2f22bdfa4008b7ff2b9afe9f6634f7d29
When installing Debian/stretch we fail at this stage:
| ngcpcfg --no-db-sync commit
On first glance it looks like a mariadb issue, though actually
it's the git status check inside ngcpcfg that's failing. The
reason is that starting with git v2.9.3 there's a different
output in its "status" output:
| commit 2a0e6cdedab306eccbd297c051035c13d0266343
| Author: Lars Vogel <lars.vogel@gmail.com>
| Date: Thu Jun 9 20:19:30 2016 +0200
|
| Use "working tree" instead of "working directory" for git status
|
| Working directory can be easily confused with the current directory.
| In one of my patches I already updated the usage of working directory
| with working tree for the man page but I noticed that git status also
| uses this incorrect term.
|
| Signed-off-by: Lars Vogel <Lars.Vogel@vogella.com>
| Signed-off-by: Junio C Hamano <gitster@pobox.com>
In Debian/stretch we've 2.11.0-2 and are affected by this change.
Adjust the `git status` checks accordingly to support git in all
its versions.
NOTE: we should consider switching to `git status --porcelain`
(see git commit fc5af3bbb2 for
some history about it) once we've the testsuite in place.
Change-Id: If4223a35083d279e746dce5a80bcb86b52f64025
For iptables firewall rules it is necessary to translate hostnames,
which may be used in config.yml into a list of IPs as returned by
"gethostbyname".
Change-Id: I149227d5031534d3826a2c91012a599f7c7c4756
* all possible grant variations should be supported now
* changes detection algorithm should support all possible
user_options now
Change-Id: Id715219948374c60fff54408037d4506c872af35
There are currently several places in templates where we do request
ngcp version, lets create a helper here, then we will be able to
switch on it after the next LTS is released (mr5.5).
Unfortunately we cannot switch on it before the next LTS,
otherwise upgrade mr4.4->mr5.5 may be corrupted by users.
If sp1 has been upgraded already and sp2 is not while
sp2 has received new templates already while ngcpcfg package
is still old -> templates cannot be built as helper is not found.
Change-Id: Icce966097335eddb7b09303ac2ef8081092bb8f3
/var/log/ngcp/ngcpcfg.log might not contain error messages from
ngcpcfg's sync-db script. This might be confusing when comparing
output from the command line with the logged actions from
ngcpcfg.log. As sync-db doesn't log all error messages to stderr
anyway and we ignore exit code of the sync-db script as well (on
purpose) let's just redirect stderr and stdout to the same file
descriptor and then provide output to the console as well as
syslog's ngcpcfg tag.
Output before this change:
| root@sp1:~# ngcpcfg commit "before"
| 2016-10-25 22:12:20: Executing sync-db:
| DBI connect('database=provisioning;host=localhost;port=3306','sipwise',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/ngcp-ngcpcfg/helper//sync-db line 70.
| Error: Could not connect to database 'provisioning' at 'localhost:3306' as 'sipwise': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
| 2016-10-25 22:12:20: OK: nothing to commit.
| 2016-10-25 22:12:20: Synchronizing MySQL grants/credentials
| Can't connect to MySQL database 'mysql': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/sbin/ngcp-sync-grants line 66.
| root@sp1:~# tail -f /var/log/ngcp/ngcpcfg.log
| [...]
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: Executing sync-db:
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: OK: nothing to commit.
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: Checking state of /etc files
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: OK: nothing to commit.
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: Synchronizing MySQL grants/credentials
Output after this change gets applied:
| root@sp1:~# ngcpcfg commit "after"
| 2016-10-25 22:12:59: Executing sync-db:
| DBI connect('database=provisioning;host=localhost;port=3306','sipwise',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/ngcp-ngcpcfg/helper//sync-db line 70.
| Error: Could not connect to database 'provisioning' at 'localhost:3306' as 'sipwise': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
| 2016-10-25 22:12:59: OK: nothing to commit.
| 2016-10-25 22:12:59: Synchronizing MySQL grants/credentials
| Can't connect to MySQL database 'mysql': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/sbin/ngcp-sync-grants line 66.
| root@sp1:~# tail -f /var/log/ngcp/ngcpcfg.log
| [...]
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: Executing sync-db:
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: DBI connect('database=provisioning;host=localhost;port=3306','sipwise',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/ngcp-ngcpcfg/helper//sync-db line 70.
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: Error: Could not connect to database 'provisioning' at 'localhost:3306' as 'sipwise': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: OK: nothing to commit.
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: Checking state of /etc files
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: OK: nothing to commit.
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: Synchronizing MySQL grants/credentials
Change-Id: Iabb0c0ab24e212b79b0dfa85947cb274a161cc00
If we have multiple web0X proxies but need specific
templates on just one of those pairs then we need to
support *.customtt.tt2.$pairname + *.tt2.$pairname
templates.
While at it also support *.customtt.tt2.$nodename +
foo.tt2.$nodename (where nodename is actual hostname
and not sp1 or sp2 as already supported already).
Test script for demonstration (especially for precedence):
mkdir /etc/ngcp-config/templates/etc/mika
cd /etc/ngcp-config/templates/etc/mika
hostname=$(hostname) # e.g. web01b
pairname=${hostname%[ab]} # e.g. web01
nodename=$(cat /etc/ngcp_nodename) # e.g. sp2
for file in foo.customtt.tt2.$hostname foo.customtt.tt2.$pairname foo.customtt.tt2.$nodename foo.customtt.tt2 foo.tt2.$hostname foo.tt2.$pairname foo.tt2.$nodename foo.tt2 ; do echo $file > $file ; done
for file in foo.customtt.tt2.$hostname foo.customtt.tt2.$pairname foo.customtt.tt2.$nodename foo.customtt.tt2 foo.tt2.$hostname foo.tt2.$pairname foo.tt2.$nodename foo.tt2 ; do
ngcpcfg build /etc/mika/ >/dev/null
if [ "$(cat /etc/mika/foo)" = "$file" ] ; then
echo "Content of file /etc/mika/foo matching $file"
rm "$file" ; rm /etc/mika/foo
else
echo "Problem with precedence of file $file"
return 1
fi
done
Change-Id: If7c222374bb56d7bf243200b37a414e39e050cd4
* MariaDB + dbd::mysql: dbh->select* returns
"fetch() without execute()" on non-table based sql
statements such as "SHOW GRANTS FOR ..." when there are
no rows to return. Such false error is avoided as
execute() is performed automatically by DBI on "non-prepare()"
methods.
Change-Id: Iac1c1c0473f39ed9f377abb5dea1cbcfbc67868a
We need to use some folder on glusterfs to ensure it is reachable.
/mnt/glusterfs/mgmt-share is good enough here as /mnt/glusterfs/shared_config
is no longer available.
Change-Id: I9f604db9604af41405cfd7b088abb332a463fe4d
During the installation we need an ability to fix perms the first time
on PRO sp2 installation. The file /etc/ngcp-config/config.yml is not
available at the moment causing function "main" failed on load:
> /usr/share/ngcp-ngcpcfg/helper/restore-permissions /tmp/tmp.Hrt0sBRYy3
> Error: Configuration file /etc/ngcp-config/config.yml does not exist (unconfigured?) - exiting.
Change-Id: Ieb196690246d7c1ea59e7f4359d72179472c278b
the way we create these output timestamps with fractional
seconds is wrong, because FROM_UNIXTIME is not truncating
its argument to int, but its rounds up/down.
"CONCAT(FROM_UNIXTIME(accounting.cdr.start_time +
accounting.cdr.duration), \".
\", SUBSTRING_INDEX(accounting.cdr.start_time +
accounting.cdr.duration, \".\",
-1))"
correct values are:
start: 2016-09-06 06:58:27.586
+duration: 0.520
end: 2016-09-06 06:58:28.120
so all of init_time, start_time, end_time are wrongn. fix
means an explicit int cast via FLOOR:
"CONCAT(FROM_UNIXTIME(FLOOR(accounting.cdr.start_time +
accounting.cdr.duration)), \".
\", SUBSTRING_INDEX(accounting.cdr.start_time +
accounting.cdr.duration, \".\",
-1))"
Change-Id: If6afaed4b1a573b36d0151c0d39eea05ba14b9ba
Git doesn't track file permissions (except for the executable
flag). For sensitive data (like the 'ssl' directory and file
'constants.yml' with passwords included) we've to prevent
non-root users from accessing those files.
hooks/pre-commit is inspired and based on the implementation
as present in etckeeper (and luckily we're license compatible)
and takes care of storing the file permissions inside file
/etc/ngcp-config/.ngcpcfg_perms.
The restore-permissions helper script takes care of restoring the
permissions after cloning the ngcpcfg repository via ngcpcfg
itself (being actions decrypt, pull (PRO-only) + initialise
(PRO-only)). It can be executed manually as well via
`usr/share/ngcp-ngcpcfg/helper/restore-permissions /etc/ngcp-config/`
(or wherever the according ngcpcfg repository is placed at).
Regarding the commit integration: git(1) itself doesn't track
file permissions, so we can't detect changes to file permissions
using git itself. Our new pre-commit hook records file
permissions via the .ngcpcfg_perms file. Now by just invoking it
during 'ngcpcfg commit' time we can ensure that even if there
have been any file permission changes in the working directory
the file .ngcpcfg_perms is then up2date and committed.
JFTR: The solution via the git pre-commit hook ensures that no
matter whether you're using 'ngcpcfg commit …' or 'git commit …'
you always get the file permissions handled via .ngcpcfg_perms.
Now if you want to change file permissions in a clean working
directory and commit *without* using 'ngcpcfg commit' but
directly via git itself then you've to use 'git commit
--allow-empty ...' and thanks to the pre-commit hook the file
.ngcpcfg_perms will still be up2date.
Change-Id: I84d608585c626b52112ff649893e232e441c59d8
The file should be accessible only to user root.
Also don't leave the unencrypted file /etc/ngcp-config-crypted.tgz around.
Change-Id: I82f7f36d8561d8d5e82435edb70e11170fa6c082
* fax gateways are no longer needed in kamailio.dispatcher
as asterisk is the only destination for fax2mail now
* error messages are prefixed with Error:
Change-Id: I865fda3ed0aab967c62cea519f3ad42716ed5878
* ngcp-sync-grants now works with the extra suffix
"with grant option"
* ngcp-sync-constants also syncs user "user" sipwise from
/etc/mysql/sipwise.cnf
Change-Id: I6159257e3c9d34cb674e003e910535807c4e841b
* when there is a situation when a user has mixed records
with correct and incorrect passowrds to detect such
scenario and trigger password sync for the user
Change-Id: I2821dafa211779b149b9c0a8763939cd52f3bb55
Michael Prokop
Alexander Lutay
Sipwise Jenkins Builder
Rene Krenn
Guillem Jover
Gernot Fuchs
Kirill Solomko
Hans-Peter Herzog