There are currently several places in templates where we do request
ngcp version, lets create a helper here, then we will be able to
switch on it after the next LTS is released (mr5.5).
Unfortunately we cannot switch on it before the next LTS,
otherwise upgrade mr4.4->mr5.5 may be corrupted by users.
If sp1 has been upgraded already and sp2 is not while
sp2 has received new templates already while ngcpcfg package
is still old -> templates cannot be built as helper is not found.
Change-Id: Icce966097335eddb7b09303ac2ef8081092bb8f3
/var/log/ngcp/ngcpcfg.log might not contain error messages from
ngcpcfg's sync-db script. This might be confusing when comparing
output from the command line with the logged actions from
ngcpcfg.log. As sync-db doesn't log all error messages to stderr
anyway and we ignore exit code of the sync-db script as well (on
purpose) let's just redirect stderr and stdout to the same file
descriptor and then provide output to the console as well as
syslog's ngcpcfg tag.
Output before this change:
| root@sp1:~# ngcpcfg commit "before"
| 2016-10-25 22:12:20: Executing sync-db:
| DBI connect('database=provisioning;host=localhost;port=3306','sipwise',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/ngcp-ngcpcfg/helper//sync-db line 70.
| Error: Could not connect to database 'provisioning' at 'localhost:3306' as 'sipwise': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
| 2016-10-25 22:12:20: OK: nothing to commit.
| 2016-10-25 22:12:20: Synchronizing MySQL grants/credentials
| Can't connect to MySQL database 'mysql': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/sbin/ngcp-sync-grants line 66.
| root@sp1:~# tail -f /var/log/ngcp/ngcpcfg.log
| [...]
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: Executing sync-db:
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: OK: nothing to commit.
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: Checking state of /etc files
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: OK: nothing to commit.
| Oct 25 22:12:20 sp1 (notice) ngcpcfg: Synchronizing MySQL grants/credentials
Output after this change gets applied:
| root@sp1:~# ngcpcfg commit "after"
| 2016-10-25 22:12:59: Executing sync-db:
| DBI connect('database=provisioning;host=localhost;port=3306','sipwise',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/ngcp-ngcpcfg/helper//sync-db line 70.
| Error: Could not connect to database 'provisioning' at 'localhost:3306' as 'sipwise': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
| 2016-10-25 22:12:59: OK: nothing to commit.
| 2016-10-25 22:12:59: Synchronizing MySQL grants/credentials
| Can't connect to MySQL database 'mysql': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/sbin/ngcp-sync-grants line 66.
| root@sp1:~# tail -f /var/log/ngcp/ngcpcfg.log
| [...]
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: Executing sync-db:
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: DBI connect('database=provisioning;host=localhost;port=3306','sipwise',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/ngcp-ngcpcfg/helper//sync-db line 70.
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: Error: Could not connect to database 'provisioning' at 'localhost:3306' as 'sipwise': Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: OK: nothing to commit.
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: Checking state of /etc files
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: OK: nothing to commit.
| Oct 25 22:12:59 sp1 (notice) ngcpcfg: Synchronizing MySQL grants/credentials
Change-Id: Iabb0c0ab24e212b79b0dfa85947cb274a161cc00
If we have multiple web0X proxies but need specific
templates on just one of those pairs then we need to
support *.customtt.tt2.$pairname + *.tt2.$pairname
templates.
While at it also support *.customtt.tt2.$nodename +
foo.tt2.$nodename (where nodename is actual hostname
and not sp1 or sp2 as already supported already).
Test script for demonstration (especially for precedence):
mkdir /etc/ngcp-config/templates/etc/mika
cd /etc/ngcp-config/templates/etc/mika
hostname=$(hostname) # e.g. web01b
pairname=${hostname%[ab]} # e.g. web01
nodename=$(cat /etc/ngcp_nodename) # e.g. sp2
for file in foo.customtt.tt2.$hostname foo.customtt.tt2.$pairname foo.customtt.tt2.$nodename foo.customtt.tt2 foo.tt2.$hostname foo.tt2.$pairname foo.tt2.$nodename foo.tt2 ; do echo $file > $file ; done
for file in foo.customtt.tt2.$hostname foo.customtt.tt2.$pairname foo.customtt.tt2.$nodename foo.customtt.tt2 foo.tt2.$hostname foo.tt2.$pairname foo.tt2.$nodename foo.tt2 ; do
ngcpcfg build /etc/mika/ >/dev/null
if [ "$(cat /etc/mika/foo)" = "$file" ] ; then
echo "Content of file /etc/mika/foo matching $file"
rm "$file" ; rm /etc/mika/foo
else
echo "Problem with precedence of file $file"
return 1
fi
done
Change-Id: If7c222374bb56d7bf243200b37a414e39e050cd4
* MariaDB + dbd::mysql: dbh->select* returns
"fetch() without execute()" on non-table based sql
statements such as "SHOW GRANTS FOR ..." when there are
no rows to return. Such false error is avoided as
execute() is performed automatically by DBI on "non-prepare()"
methods.
Change-Id: Iac1c1c0473f39ed9f377abb5dea1cbcfbc67868a
We need to use some folder on glusterfs to ensure it is reachable.
/mnt/glusterfs/mgmt-share is good enough here as /mnt/glusterfs/shared_config
is no longer available.
Change-Id: I9f604db9604af41405cfd7b088abb332a463fe4d
During the installation we need an ability to fix perms the first time
on PRO sp2 installation. The file /etc/ngcp-config/config.yml is not
available at the moment causing function "main" failed on load:
> /usr/share/ngcp-ngcpcfg/helper/restore-permissions /tmp/tmp.Hrt0sBRYy3
> Error: Configuration file /etc/ngcp-config/config.yml does not exist (unconfigured?) - exiting.
Change-Id: Ieb196690246d7c1ea59e7f4359d72179472c278b
the way we create these output timestamps with fractional
seconds is wrong, because FROM_UNIXTIME is not truncating
its argument to int, but its rounds up/down.
"CONCAT(FROM_UNIXTIME(accounting.cdr.start_time +
accounting.cdr.duration), \".
\", SUBSTRING_INDEX(accounting.cdr.start_time +
accounting.cdr.duration, \".\",
-1))"
correct values are:
start: 2016-09-06 06:58:27.586
+duration: 0.520
end: 2016-09-06 06:58:28.120
so all of init_time, start_time, end_time are wrongn. fix
means an explicit int cast via FLOOR:
"CONCAT(FROM_UNIXTIME(FLOOR(accounting.cdr.start_time +
accounting.cdr.duration)), \".
\", SUBSTRING_INDEX(accounting.cdr.start_time +
accounting.cdr.duration, \".\",
-1))"
Change-Id: If6afaed4b1a573b36d0151c0d39eea05ba14b9ba
Git doesn't track file permissions (except for the executable
flag). For sensitive data (like the 'ssl' directory and file
'constants.yml' with passwords included) we've to prevent
non-root users from accessing those files.
hooks/pre-commit is inspired and based on the implementation
as present in etckeeper (and luckily we're license compatible)
and takes care of storing the file permissions inside file
/etc/ngcp-config/.ngcpcfg_perms.
The restore-permissions helper script takes care of restoring the
permissions after cloning the ngcpcfg repository via ngcpcfg
itself (being actions decrypt, pull (PRO-only) + initialise
(PRO-only)). It can be executed manually as well via
`usr/share/ngcp-ngcpcfg/helper/restore-permissions /etc/ngcp-config/`
(or wherever the according ngcpcfg repository is placed at).
Regarding the commit integration: git(1) itself doesn't track
file permissions, so we can't detect changes to file permissions
using git itself. Our new pre-commit hook records file
permissions via the .ngcpcfg_perms file. Now by just invoking it
during 'ngcpcfg commit' time we can ensure that even if there
have been any file permission changes in the working directory
the file .ngcpcfg_perms is then up2date and committed.
JFTR: The solution via the git pre-commit hook ensures that no
matter whether you're using 'ngcpcfg commit …' or 'git commit …'
you always get the file permissions handled via .ngcpcfg_perms.
Now if you want to change file permissions in a clean working
directory and commit *without* using 'ngcpcfg commit' but
directly via git itself then you've to use 'git commit
--allow-empty ...' and thanks to the pre-commit hook the file
.ngcpcfg_perms will still be up2date.
Change-Id: I84d608585c626b52112ff649893e232e441c59d8
The file should be accessible only to user root.
Also don't leave the unencrypted file /etc/ngcp-config-crypted.tgz around.
Change-Id: I82f7f36d8561d8d5e82435edb70e11170fa6c082
* fax gateways are no longer needed in kamailio.dispatcher
as asterisk is the only destination for fax2mail now
* error messages are prefixed with Error:
Change-Id: I865fda3ed0aab967c62cea519f3ad42716ed5878
* ngcp-sync-grants now works with the extra suffix
"with grant option"
* ngcp-sync-constants also syncs user "user" sipwise from
/etc/mysql/sipwise.cnf
Change-Id: I6159257e3c9d34cb674e003e910535807c4e841b
* when there is a situation when a user has mixed records
with correct and incorrect passowrds to detect such
scenario and trigger password sync for the user
Change-Id: I2821dafa211779b149b9c0a8763939cd52f3bb55
* when in the "copy" mode and all *.localhost grants need
to be copied for some users there may be no such
key (e.g. replicator has only sp1 and sp2). Therefore,
if in the "copy" mode and there is no host for a user
- skip the user and continue instead of dropping with an error
* normalise_grant_str(): added support for more grant elements,
added a check to die if there are unsorted elements left
* recreate use before hosts processing to avoid situations when
the user is repeatedly removed
Change-Id: I706dfcbf52279abc5260b01f658ce554d53a604e
On Carrier:
local mysql is 127.0.0.1:3308
pair mysql is localhost:3306
On CE/PRO:
local mysql is localhost:3306
pair mysql is localhost:3306
ngcp-sync-grants/-constants should use pair mysql to upload grants/passwords.
local mysql is comming from db01:3306 using ngcp-sync-db.
Change-Id: I293bda36c64184acfde89a96703f1691279feba6
* generated password is represented with "!" prefix in mysql.user,
instead of the default "*" where internally it is still the valid one.
that allows to overcome the behavior when a non valid password
is assigned to a user "show grants" returns as if no grants assigned
to him causing ngcp-sync-grants recreate the user every time.
Change-Id: Id0acf626677177e4b17d17b50c78baea5a7f9955
- work with the new passwords schema in constants.yml
- all grants related code cleanup
- all is performed as a single, not replicated transaction
Change-Id: I7bfafcd6c1a9da67705fefa8beabf395447d96c1
- ngcp-sync-grants is responsible for ngcp mysql
grants sync from a template
- it is executed by the 'commit' trigger before ngcp-sync-constants
Change-Id: I082256e57b1394a3f056ad1ca56a5443bfb5a745
Mainly nobody validate the schema in trunk right now
as it requires manual changes for ngcpcfg.cfg.
Also having force validation will allow us to add
validation in ngcp-upgrade and Jenkins nightly builds.
Change-Id: Ia48a778ecf7dae30b26bdff0c5dd5b9000c9d089
tt2-daemon is stopped via killall in the build and values
action, therefore we need to add psmisc (the package providing
the killall binary) as dependency.
Found while writing the new testsuite.
Change-Id: I6dfbd23b012b588cc1562db05cf1d927da97a13c
new fields can be simply specified in config.yml, e.g.
" - source_customer_cash_balance_before"
" - source_customer_free_time_balance_before"
" - source_customer_profile_package_id"
" - source_customer_contract_balance_id"
whereby
+"source" can be replaced by "destination"
+"customer" can be replaced by "carrier", "reseller"
+"before" can be replaced by "after"
note that:
+ these field names are not case sensitive
+ those fields cannot be used in export conditions atm
finally, also melita's END_TIME named field was added,
identical like START_TIME.
Change-Id: I669259be53878058bc2ec0927e96496dcbd534bd
Configuring the identical shared IP once again shouldn't cause
duplicate IP records, since e.g. nginx receives two "listen"
lines with the same IP and fails hard then.
New behaviour with this change is to avoid duplicates:
| root@spce:~# ngcp-network --verbose --set-interface=eth0 --shared-ip=1.2.3.4
| [...]
| adding IP entry shared_ip: 1.2.3.4
| [...]
| root@spce:~# ngcp-network --verbose --set-interface=eth0 --shared-ip=1.2.3.4
| [...]
| not setting shared_ip to 1.2.3.4 to avoid duplicates
| [...]
Change-Id: Iee3ff1e7a27bc3298128835468e1e888c327d13d
As soon as we merge 2e74d71 to ngcpcfg.git we cannot install PRO/Carrier:
> /usr/share/ngcp-ngcpcfg/functions/ha_features: line 54: cd: /etc /var: No such file or directory
Change-Id: I124adc4e42d3c87d0f4fbf19b076986d757cff39
When setting up carrier environments we have to deal
with many similar nodes, often involving >10 proxy pairs.
By default we create prx01a and prx01b sections only.
Provide --clone-from=<HOST> --clone-to=<HOST> options
as convenience helpers so the operator can clone a
specific host definition and just needs to adapt IP
addresses accordingly afterwards.
Use example to clone the prox01a host definition
to prox02a, prox03a,.... up to prox10a:
for host in {02..10} ; do
ngcp-network --clone-from=prx01a --clone-to=prx${host}a
done
If the peer setting (as used in carrier environments) is
present in the source host config (--clone-from=...) then
we automatically set the peer for the destination, like:
| # ngcp-network --clone-from=prx01a --clone-to=prx01b
| Setting peer of host 'prx01b' to 'prx01a'
| Finished cloning host section 'prx01a' to 'prx01b'.
| Please do not forget to manually adjust '/etc/ngcp-config/network.yml'!
and:
| # ngcp-network --clone-from=prx01a --clone-to=prx04a
| Setting peer of host 'prx04a' to 'prx04b'
| Finished cloning host section 'prx01a' to 'prx04a'.
| Please do not forget to manually adjust '/etc/ngcp-config/network.yml'!
Change-Id: I6c30a2ce58dd8bc66247cfdd0028c18736173e99
Verify that the user is operating on branch master,
otherwise the result might be unexpected.
If branch master is checked out:
| root@spce:/etc/ngcp-config# ngcpcfg check
| root@spce:/etc/ngcp-config# ngcpcfg status
| [...]
| 2016-02-23 05:48:32: Checking currently active branch:
| 2016-02-23 05:48:32: OK: branch master active
If a branch other than 'master' is checked out:
| root@spce:/etc/ngcp-config# ngcpcfg check
| 2016-02-23 05:47:42: Error: branch 'mika' in '/etc/ngcp-config' active - please switch to branch 'master' before continuing.
| root@spce:/etc/ngcp-config# echo $?
| 1
| root@spce:/etc/ngcp-config# ngcpcfg status
| [...]
| 2016-02-23 05:47:47: Checking currently active branch:
| 2016-02-23 05:47:47: ACTION_NEEDED: branch 'mika' active - please switch to branch 'master'
| [...]
Change-Id: I5df92075905cafa3b714211581c8cfe749df04ba
Rene Krenn
Alexander Lutay
Hans-Peter Herzog
Sipwise Jenkins Builder
Michael Prokop
Kirill Solomko
ngcp-config
Guillem Jover
Victor Seva