At the end of installer's 'configuration' section we need
to restart all services were touched during the installation
(otherwise we need to restart the node completely).
The option helps us in this case as etckeeper is
not reliable enough during the installation.
P.S. we pass options from 'apply' to 'services' now,
we should remove 'Unsupported option(s) given: $*' check,
otherwise it is failing as 'apply' requires commit message,
which cause error on 'services' level: 'Unsupported option(s) given'
> 2018-09-13 18:08:33: Error: Unsupported option(s) given: my commit message
> 2018-09-13 18:08:34: Did you mean '--dry-run'?
Change-Id: I5dac44cd87dc1fc90f81e4ac3a14af32c00f825e
We need to have some reproducible order of services restart,
otherwise this might lead to different results, depending on the order.
Also unifyer() looks too heavy here since the files can be easily
unified in a moment of writing them into ${TMPFILE}.
Change-Id: I0a683a7d57cd52b74cba0ee9a285f84f77f3dc20
It was initially introduced in the commit:
> commit 7d422b7878
> Author: Victor Seva <vseva@sipwise.com>
> Date: Fri Aug 22 17:03:39 2014 +0200
>
> MT#8763 check for .git before running the services
>
> On a multi directory config, the services depends on git. Check
> first if the git repository exists and warn about it
The subshell here was necessary to isolate 'cd $dir' call.
In the same time the "proper" bash way is to use pushd/popd.
For the history, 'shellcheck disable' was muting the warning:
> This only exits the subshell caused by the (..) group. [SC2106]
Change-Id: I3546fa9615858dbbc4c0f91d74de003e22837d55
NGCP doesn't use NDB anymore and also it looks like non-safe commit
as the commit has been done by user 'ngcp-config <root@4bd5de92c84d>' :-D
Removing it.
This reverts commit 9f4d913f6b.
Change-Id: Ie91442a1f33e9ddcae5589c095d19c9e46cd68cb
To be able to generate an authorized_keys file from templates we need
to fetch the public ssh keys for the root user. This function will make
this task easier.
Change-Id: If8edef0e77a5f3c1167fe8dfd0e92e4d9e468a6a
Previously 'systemd daemon-reload' and 'systemctl preset-all'
were executed only if changes detected in configs.
Otherwise the early exit happened if no .service files were triggered:
> DEBUG: No services file(s) reported - nothing to do.
Move the code into the function systed_daemon_reload_preset and
execute it on the top of the script. Also print info message
informing users about reloaded systemd and newly preset units.
It should provide better visibility here.
Change-Id: I992af9fb274ea93a37b812a51cebcd7af5c54133
Otherwise the code in between the functions can be easily missed
which cause issues as described in the ticket and will be fixed
in the following commit.
Change-Id: I57decfcbcd41691d35d085b13881e1e6b5208f6e
This partially reverts commit bc8ae9e795.
We should not match li_dist with regexes as then we would trap cases
that we should not be covering by the li_dist mapping logic. Just do
exact matches.
This also fixes the hypothetical case with other regexes that are not
the wrongly specified '*'.
Change-Id: I9d9d850969e03fa9dc587f2d47a688bc0359e63c
Corrupted YML schema is the popular way for time spending on debug
mystery behaviour on NGCP. We have validation schema since mr4.5
and it coverts all YML files nowadays. Let's enable it by default,
since in field testing shows good results.
In case of the inconsistent schema, apply is still possible with
ngcpcfg option '--no-validate':
> root@sp1:~# ngcpcfg --no-validate apply 'some changes with inconsistent schema'
> 2018-08-29 13:12:27: Error: Invalid schema detected for /etc/ngcp-config/config.yml
> /etc/ngcp-config/config.yml#0: INVALID
> - [/apps] Expected required key 'malicious_call'
>
> DANGEROUS ZONE: invalid configs detected, continue anyway due to option '--no-validate'
> Checking state of local storage:
> ...
Change-Id: Ifa51c9e0c2fd396696f73760d89eadcbe9763456
Otherwise it matches on 'li/li_dist' cases and the library get_all_ips
returns no IPs if called like:
> argv.role='*';
> argv.type='*';
> PROCESS '/usr/lib/ngcp-ngcpcfg/get_all_ips';
> all_ips = out;
This is a hotfix commit for backporting into mr6.4.1 while the "proper" fix
should be re-considered and committed for mr6.5+
P.S we have 3 tt2 templates where we have * in argv parameters.
P.P.S. The code is from Andreas, tnx for the fix!
Change-Id: I119a687389075d2ebdcd824ed457f768b5fb2123
This will collect all service actions, synthetize them into their
minimal expression and execute them in a single batch per action.
Change-Id: I950d5db32e0ec6327964faac4ce8f15449f90e90
the following additional fields can be used in
admin_export_fields/reseller_export_fields in config.yml:
- FURNISHED_CHARGE_INFO:
the fci data returned by external lnp requests
- HEADER_*:
the value of a sip header. * is the sip header name,
which is case-sensitive.
Change-Id: Ie0c95d341648fc63fff23ea2d3054b70fa2cf9e9
We need to use rename semantics when moving the built template into the
destination, so that we avoid reacing on ETXTBSY for executable files.
But we need to fallback to use copy semantics, because at least Docker
bind mounts /etc/hosts, which means we cannot rename over it.
We'll use perl's File::Copy which gives us the exact semantics we need.
Ref: https://github.com/moby/moby/issues/22281
Change-Id: I6ae6ce2050050c13c7ec9d08b0e6e01fb2801fd6
We should not just check whether these keys are defined, but whether
they contain actual IPs too.
Fixes: commit 80f70d88b2
Change-Id: I122de32629164cc46537ab0800612c8654f1ba14
If we are using DHCP we do not need either an IPv4 nor an IPv6. If we
have an IPv6 we do not need an IPv4. If we have a netmask, a shared
IP, an advertised IP, we should have the matching IP setting.
Change-Id: I13f190086a0540495b0314e58ca301aae6db0453
This adds an additional virtual li_dist role on top of the existing li
role. The difference is that li is the cluster simpli acting in central
mode, while li_dist is the cluster acting in distributed mode.
Change-Id: I930c93380a0e88649c325b7faf3c0fb86d78e5ff
We need to preserve symlinks, as was the case when we were using cat.
This way we'll not get caught in the ETXTBSY race from the kernel, and
we'll have the old semantics.
Bisected-by: Alex Lutay <alutay@sipwise.com>
Fixes: 7480ebe7c5
Change-Id: I0be7473b271cc3807da957bbef1063018d3b42b2
Otherwise ngcp-installer failed to install PRO/Carrier as
we build the file /etc/ngcp_mgmt_node the very first time on the first MGMT node:
> +08:22:48 cfg_build_templates
> +08:22:48 cfg_build_configs /etc/ngcp_mgmt_node
> +08:22:48 build_configs=($1)
> +08:22:48 declare -a build_configs
> +08:22:48 log_info 'Generating default configuration files /etc/ngcp_mgmt_node'
> +08:22:48 tee -a /tmp/ngcp-installer.log
> +08:22:48 echo 'Generating default configuration files /etc/ngcp_mgmt_node'
> Generating default configuration files /etc/ngcp_mgmt_node
> +08:22:48 ngcpcfg build /etc/ngcp_mgmt_node
> cat: /etc/ngcp_mgmt_node: No such file or directory
> +08:22:48 log_die 'Error running '\''ngcpcfg build'\'''
Checking the file availability is not enough here, the next error is:
> (sp1)root@sp1:/# ngcpcfg build
> /usr/share/ngcp-ngcpcfg/scripts//check: line 83: NGCP_IS_MGMT: unbound variable
> (sp1)root@sp1:/# cat /etc/default/ngcp-roles
> NGCP_TYPE="sppro"
> (sp1)root@sp1:/#
It happens because /etc/default/ngcp-roles is a fake one at this stage.
Handling NGCP_IS_MGMT properly is also not enough:
> (sp1)root@sp1:/# ngcpcfg build
> 2018-07-16 16:49:32: Error: Remote origin of ngcpcfg is '/mnt/glusterfs/ngcpcfg-share', expected: 'sp:/mnt/glusterfs/ngcpcfg-share'.
> 2018-07-16 16:49:32: Error: NOTE: execute `cd /etc/ngcp-config ; git remote set-url origin 'sp:/mnt/glusterfs/ngcpcfg-share'` to adjust setting.
> 2018-07-16 16:49:32: Error: NOTE: perform `ngcpcfg clean --all` to recreate local master branch from remote.
> (sp1)root@sp1:/#
Which happens because the peer is not yet configured at the moment (first node installation).
Introducing new internal option '--no-check-origin' to skip the test into installer.
Change-Id: I0265c65f45972e92ca92320871a7ef29f8904fec
We should be exhaustive here, and any unknown pattern should be caught
and reported, as we might miss information about peers and the cluster
layout otherwise.
Change-Id: I56a9a2b5b6c6f776c0c64fc2f8e01f5f76bb8e10
Files that are being executed cannot be modified in place as that
returns EBUSY. But their dentry can be replaced with a rename(2),
which is atomic and does not touch the original inode. This is the
standard procedure to replace running executables in Unix.
So, we just replace the cat(1) with a mv(1), and stop quiescing its
stderr so that we get proper errors reported.
Change-Id: If15ea1cfa749a6140ff4022200c7fc730c76aa3a
Previously it had the following behavior:
Select all records with 'Host LIKE' filtering so if requested host was '%'
this select returned all the record for this user.
Then user was delete with 'DELETE ... Host LIKE' statement so if
requested host was '%' it deleted all the records.
With using 'DROP USER' statement the same behavior can be achieved with
the same selecting but drop user not with requested host but with actual
one from select statement.
Change-Id: I72f1dd1962e139939be700794e0eb025fe1615b2
There are a lot of flushing privileges so in order not to flood the
console move it to debug level.
Change-Id: I98e3247881393d7892799cc23c2a4e5dc865185a
Replace unsafe 'DELETE FROM mysql.user' with recommended 'DROP USER'
statement to avoid problems with DROP/CREATE USER due to missing
'FLUSH PRIVILEGES' like:
Error: Cannot create grant temp user: Operation CREATE USER failed
for 'ngcp-sync-db'@'localhost' at /usr/sbin/ngcp-sync-grants line 322.
Execute 'FLUSH PRIVILEGES' before creating and after dropping of temp user
'ngcp-sync-db'.
Change-Id: I49c29b6c39353d4a47f086851a915af1469ebcdd
This makes it possible to depend on this new package while not having to
pull the huge amount of dependencies.
Change-Id: I2df3d072ecca0751d4d05d30f5b5c1ac0ec4ed25
This file should be installed only for the ngcp-ngcpcfg package, as
that's the one making sure etckeeper is installed, and we only need
one doing the setup, not all the rdepends too.
Change-Id: Ib20111ada44964a3bdfda4c50a84971cf9678eb9
The Template class accepts both a hash reference and a hash to get the
configuration arguments. And the Dancer framework passes those as a
hash.
Change-Id: Ic9ae758bd81a2bb3108900e6f92ad91ad88f3cc9
Move the ngcp object variable into NGCP::Template::Object, and make
NGCP::Template just a customized variant of Template that we can use
instead.
This way we hide all the internal details of how to set up the
environment we need to process NGCP templates.
Change-Id: I690cf1a74551f4751380a506ddcc047b0942ba21
This is another good candidate to replace with a proper ngcp method, as
the function is already implemented as a PERL block, and the most used
"function".
Change-Id: I49dc638a197a0215ea65c0a60c82c6c556aca3fa
This replaces the old has_role Template "function" with a proper method
of the ngcp object, anywhere we need to check for the host role,
including other macros that could not use our previous "function"
before. Which will make it possible to modify the has_role behavior in
a single place, and affect all code paths.
Change-Id: I73526aa77fab3dec8f33361e142164204d7c1481
* ngcp.timezone table is not updated if
new timezone=old timezone
* ngcp.timezone change is not replicated to
preserve stability of the other (active) node
and to address the replication issue during upgrade
Change-Id: Id8f7b291c188792a33093ac3ed706b55d1b0a654
The perl Template::Toolkit is very rich, but its "function" support is a
bit poor. The ways to do it are either via MACRO directives, or by
simulating them with one function per file and then using PROCESS on
these. The problem is that this is very clunky, does not support
nesting, as we'd need different "argument" names for each "function",
and it's quite cumbersome to use, need to assign aguments passed
beforehand, and then assign back a designated return value from another
variable. This is also one of the reasons some of the functions are not
encapsulated, and have been inlined in various loops, because it was not
possible to cleanly PROCESS them from those call sites.
Instead we should use its native support for perl objects and perl
subroutines, which exposes these as proper methods of a designated
variable, and have none of the above mentioned problems. So we'll switch
from constructs such as:
argv.arg-a = variable;
argv.arg-b = 'value';
PROCESS 'path-to-library-dir/function'
result = out
into:
result = ngcp.function(variable, 'value');
In addition this might actually be faster, as it does not require
processing additional files, and it's all just native perl code.
This will be exposed within the NGCP templates as the ngcp object, and
new member functions will start replacing our old and clunky native
Template PROCESS-style library.
Change-Id: Id2f0d181c695a9dd074646881b7d9de3478570af
Alexander Lutay
Guillem Jover
Rene Krenn
Sipwise Jenkins Builder
Michael Prokop
Mykola Malkov
Kirill Solomko