Move the ngcp object variable into NGCP::Template::Object, and make
NGCP::Template just a customized variant of Template that we can use
instead.
This way we hide all the internal details of how to set up the
environment we need to process NGCP templates.
Change-Id: I690cf1a74551f4751380a506ddcc047b0942ba21
* ngcp.timezone table is not updated if
new timezone=old timezone
* ngcp.timezone change is not replicated to
preserve stability of the other (active) node
and to address the replication issue during upgrade
Change-Id: Id8f7b291c188792a33093ac3ed706b55d1b0a654
The perl Template::Toolkit is very rich, but its "function" support is a
bit poor. The ways to do it are either via MACRO directives, or by
simulating them with one function per file and then using PROCESS on
these. The problem is that this is very clunky, does not support
nesting, as we'd need different "argument" names for each "function",
and it's quite cumbersome to use, need to assign aguments passed
beforehand, and then assign back a designated return value from another
variable. This is also one of the reasons some of the functions are not
encapsulated, and have been inlined in various loops, because it was not
possible to cleanly PROCESS them from those call sites.
Instead we should use its native support for perl objects and perl
subroutines, which exposes these as proper methods of a designated
variable, and have none of the above mentioned problems. So we'll switch
from constructs such as:
argv.arg-a = variable;
argv.arg-b = 'value';
PROCESS 'path-to-library-dir/function'
result = out
into:
result = ngcp.function(variable, 'value');
In addition this might actually be faster, as it does not require
processing additional files, and it's all just native perl code.
This will be exposed within the NGCP templates as the ngcp object, and
new member functions will start replacing our old and clunky native
Template PROCESS-style library.
Change-Id: Id2f0d181c695a9dd074646881b7d9de3478570af
In this way, if the file fails to be generated (e.g. because of a broken .yml
file) the old file is preserved intact.
It doesn't protect from all failures (in particular, it is ineffective against
wrong content due to programming errors), but it's safer than having an empty
file or truncated as a result of the parsing/generation being stopped due to
incorrect files or external events.
This is added in the context of creating new /etc/hosts and
/etc/network/interfaces files because we found the problem at this point, and
having those files empty completely breaks the system.
Change-Id: I01e8ef9d6971506aa3f8d9da174ce890ea83393c
If the file /etc/hosts gets handled by templates, as ongoing changes
attempt to do, there's a problem because /etc/hosts gets truncated to
create a new file when it's needed to generate itself, to resolve the
"localhost" string as hostname, so the generation fails and /etc/hosts
gets empty at that point, affecting the system until restored.
The simplest way to break that cycle is to switch to Unix sockets.
Change-Id: I1e33ead6a134625346b9cf1beb09a7bdbfdfc8d2
If the output file has been replaced by a symlink then the
permissions of the symlink's target are adjusted. This is clearly
unwanted and unexpected behavior.
This can happen e.g. when masking a service via systemd's systemctl,
demonstration:
| sipwise@sp1:~$ sudo rm /etc/systemd/system/ntp.service
| sipwise@sp1:~$ sudo systemctl mask ntp
| Created symlink /etc/systemd/system/ntp.service → /dev/null.
| sipwise@sp1:~$ ls -la /dev/null
| crw-rw-rw- 1 root root 1, 3 May 12 01:54 /dev/null
| sipwise@sp1:~$ sudo ngcpcfg build /etc/systemd/system/ntp.service
| 2018-05-15 16:06:47: Checking state of local storage:
| 2018-05-15 16:06:47: OK: nothing to pull
| 2018-05-15 16:06:47: configs were checked successfuly
| 2018-05-15 16:06:47: No patchtt files found, nothing to patch.
| Loading /etc/ngcp-config/config.yml in memory: OK
| Loading /etc/ngcp-config/network.yml in memory: OK
| Loading /etc/ngcp-config/constants.yml in memory: OK
| 2018-05-15 16:06:48: Generating /etc/systemd/system/ntp.service: OK
| sipwise@sp1:~$ ls -la /dev/null
| cr--r--r-- 1 root root 1, 3 May 12 01:54 /dev/null
Instead check whether the expected output file is a symlink and
if so don't modify permissions at all, but warn instead.
Change-Id: I1c1bb0941a8a2f599652b7c089efd63ff33ca455
* sync_smsc_peers() is used to sync the 'smsc_peer' domain
preference
* generic_enum_sync() is extended and accepts boolean args
$usr_pref, $dom_pref, $peer_pref
Change-Id: I376a2ba823bef53b3548b88c38960c893a74efc8
On migration to systemd we have separated mysqld1 and mysqld2 by
separate my.cnf files. We have to use the accordingly here.
Change-Id: I6d291e3b7c9601c09b82103cd32a775ba44911ef
* sync_db_timezones use 'mysql_tzinfo_to_sql'
to load timezone info from /usr/share/zoneinfo into
MariaDB.
* tzdata package version is checked and the timezone data
sync is skipped if the version in ngcp.tzinfo_version
is already up to date.
Change-Id: I92c87fb52fea20df0366c93c2e3568c25833b9bb
* general.timezone value from config.yml
is now populated into ngcp.timezone DB table.column
* it produces an error if general.timezone field is undefined
Change-Id: I975f786a7f8ff42916d92e13f6fe9971d5b1e2f0
The latter does not support YAML 1.1, nor many parts of the
specification. Use the more compliant implementation, in addition to try
to converge to a single one, so that we do not get serialization delta
surprises.
Change-Id: Ie51f1c79859d40ef0877fc0ab75f86ee72e14ea4
This module is more compliant and it is faster than the pure perl
implementation. The latter is also deprecated in favor of the former
(see man YAML for more details).
Change-Id: I3fccca4ab57ad7c316b6cf58a81bc4baa1bdabe1
If a template named
/etc/ngcp-config/templates/etc/apt/apt.conf.d/71_no_recommended.tt2 exists
the resulting destination file would be
/etc/apt/apt.conf.d/71_no_recommended. Though if
/etc/apt/apt.conf.d/71_no_recommended exists as directory ngcpcfg should
abort and inform the user about the existing situation with a useful error
message.
Change-Id: I2c6e1e3a4ec485183674c1fe72251631ad9867ac
This let's us control the directory where files should be written to.
Otherwise we'd have to write to the system's /etc/ + /var, which is
clearly unwanted.
Change-Id: I1010afcd4637928868fc81a0adfafebd3bbce874
If we have multiple web0X proxies but need specific
templates on just one of those pairs then we need to
support *.customtt.tt2.$pairname + *.tt2.$pairname
templates.
While at it also support *.customtt.tt2.$nodename +
foo.tt2.$nodename (where nodename is actual hostname
and not sp1 or sp2 as already supported already).
Test script for demonstration (especially for precedence):
mkdir /etc/ngcp-config/templates/etc/mika
cd /etc/ngcp-config/templates/etc/mika
hostname=$(hostname) # e.g. web01b
pairname=${hostname%[ab]} # e.g. web01
nodename=$(cat /etc/ngcp_nodename) # e.g. sp2
for file in foo.customtt.tt2.$hostname foo.customtt.tt2.$pairname foo.customtt.tt2.$nodename foo.customtt.tt2 foo.tt2.$hostname foo.tt2.$pairname foo.tt2.$nodename foo.tt2 ; do echo $file > $file ; done
for file in foo.customtt.tt2.$hostname foo.customtt.tt2.$pairname foo.customtt.tt2.$nodename foo.customtt.tt2 foo.tt2.$hostname foo.tt2.$pairname foo.tt2.$nodename foo.tt2 ; do
ngcpcfg build /etc/mika/ >/dev/null
if [ "$(cat /etc/mika/foo)" = "$file" ] ; then
echo "Content of file /etc/mika/foo matching $file"
rm "$file" ; rm /etc/mika/foo
else
echo "Problem with precedence of file $file"
return 1
fi
done
Change-Id: If7c222374bb56d7bf243200b37a414e39e050cd4
During the installation we need an ability to fix perms the first time
on PRO sp2 installation. The file /etc/ngcp-config/config.yml is not
available at the moment causing function "main" failed on load:
> /usr/share/ngcp-ngcpcfg/helper/restore-permissions /tmp/tmp.Hrt0sBRYy3
> Error: Configuration file /etc/ngcp-config/config.yml does not exist (unconfigured?) - exiting.
Change-Id: Ieb196690246d7c1ea59e7f4359d72179472c278b
Git doesn't track file permissions (except for the executable
flag). For sensitive data (like the 'ssl' directory and file
'constants.yml' with passwords included) we've to prevent
non-root users from accessing those files.
hooks/pre-commit is inspired and based on the implementation
as present in etckeeper (and luckily we're license compatible)
and takes care of storing the file permissions inside file
/etc/ngcp-config/.ngcpcfg_perms.
The restore-permissions helper script takes care of restoring the
permissions after cloning the ngcpcfg repository via ngcpcfg
itself (being actions decrypt, pull (PRO-only) + initialise
(PRO-only)). It can be executed manually as well via
`usr/share/ngcp-ngcpcfg/helper/restore-permissions /etc/ngcp-config/`
(or wherever the according ngcpcfg repository is placed at).
Regarding the commit integration: git(1) itself doesn't track
file permissions, so we can't detect changes to file permissions
using git itself. Our new pre-commit hook records file
permissions via the .ngcpcfg_perms file. Now by just invoking it
during 'ngcpcfg commit' time we can ensure that even if there
have been any file permission changes in the working directory
the file .ngcpcfg_perms is then up2date and committed.
JFTR: The solution via the git pre-commit hook ensures that no
matter whether you're using 'ngcpcfg commit …' or 'git commit …'
you always get the file permissions handled via .ngcpcfg_perms.
Now if you want to change file permissions in a clean working
directory and commit *without* using 'ngcpcfg commit' but
directly via git itself then you've to use 'git commit
--allow-empty ...' and thanks to the pre-commit hook the file
.ngcpcfg_perms will still be up2date.
Change-Id: I84d608585c626b52112ff649893e232e441c59d8
* fax gateways are no longer needed in kamailio.dispatcher
as asterisk is the only destination for fax2mail now
* error messages are prefixed with Error:
Change-Id: I865fda3ed0aab967c62cea519f3ad42716ed5878
The default Hash::Merge->merge() behaviour is LEFT_PRECEDENT:
http://search.cpan.org/~rehsack/Hash-Merge-0.200/lib/Hash/Merge.pm
Which means:
> The values buried in the left hash will never be lost;
> any values that can be added from the right hash will be attempted.
So, the values from HOST_CONFIG never overwrites default NGCPCTL_CONFIG:
> NGCPCTL_CONFIG="${NGCPCTL_MAIN}/config.yml"
> HOST_CONFIG="${NGCPCTL_MAIN}/config.$(hostname).yml"
We need to use RIGHT_PRECEDENT order to be in sync with 'scripts/build':
> ${NGCPCTL_CONFIG:-} ${HOST_CONFIG:-} ${LOCAL_CONFIG:-} ${NETWORK_CONFIG:-} ${EXTRA_CONFIG_FILES:-} ${CONSTANTS_CONFIG:-}
Change-Id: I597bb082a1791c1f06072d85c4a26fbb8b8320ce
- added quiet option to disable loading output at tt2-daemon
- detect arrays and output values joined by space
Change-Id: I0b7694a366a284ca2b24ea2df55cb285b598e66d
We will use first free port instead which will be stored into
/var/run/ngcpcfg.port file while tt2-daemon is up and running.
Change-Id: I1a918f865d87863eef7d34be8abd1032282220b8
Sometimes it contains very useful information, like:
> /usr/share/ngcp-ngcpcfg/helper//validate-yml /etc/ngcp-config/network.yml
>> YAML::Tiny failed to classify line ' hwaddr:5C:F3:FC:11:67:26 '
>> Can't call method "write_string" without a package or object
>> reference at /usr/share/ngcp-ngcpcfg/helper//validate-yml line 15.
Change-Id: I762b4dc1b0e6b3446f6a431b038fa6ed674e5516
When runnning 'ngcpcfg build' (which is also executed when
running 'ngcpcfg apply') we waste quite some time with loading
the same YAML configuration files and merging them again and
again.
In a 'ngcpcfg build' run we don't have to re-load and merge those
configuration files again and again (since they won't modify
during a single run), instead just load+merge them *once* and
re-use the result on all the following execution steps in the
same run.
Original:
| # time ngcpcfg apply >/dev/null
| ngcpcfg apply > /dev/null 149.01s user 9.49s system 96% cpu 2:43.57 total
Reworked optimized runs:
| # time ngcpcfg apply >/dev/null
| ngcpcfg apply > /dev/null 81.25s user 8.62s system 95% cpu 1:34.17 total
For debugging purposes you can execute 'ngcpcfg build' under
NO_REUSE=1 to not use this feature, executing it under 'DEBUG=1'
won't remove the so called reuse file which stores the merged
configuration data.
Thanks: Alexander Lutay for the initial patch and idea
Guillem Jover
Kirill Solomko
Michael Prokop
Manuel Montecelo
Alexander Lutay
Richard Fuchs
Victor Seva