according to security framework spec
5.5.4 Allow r/w access /etc/ngcp-config/config.yml for users in security
domain ngcp-admin This file contains operational configuration data of
the NGCP system. Reading and editing it is part of day-to-day
operations. As such it has to be read- and writable by users in security
domain ngcp-admin. It is also readable by users of any other security
domain. This file is also read by many NGCP processes.
The current default setup of the NGCP does not implement permissions as
required by tightened user security. Required permission settings have
to be applied manually issuing command:
chgrp ngcp-admin /etc/ngcp-config/config.yml
Security domain: ngcp-admin User ownership: root Group ownership:
ngcp-admin File permissions: 664
IMPACT: Configuration data can be edited by all users in group
ngcp-admin, the sipwise user, and user root. If for some reason (e.g.
after upgrade) the proposed settings are reverted to default settings,
this does not pose a security risk. However, write access to
configuration data in this file will be denied to named users.
5.5.5 Allow r/w access to /etc/ngcp-config/network.yml for users in
security domain ngcp-admin This file contains information about the
network configuration of the NGCP cluster. This information should not
frequently change. Nevertheless, this file and its content is meant for
the operator and as such is read- and writable by users in security
domain ngcp-admin. It is also readable by users of any other security
domain. This file is also read by many NGCP processes.
The current default setup of the NGCP does not implement permissions as
required by tightened user security. Required permission settings have
to be applied manually issuing command:
chgrp ngcp-admin /etc/ngcp-config/network.yml
Security domain: ngcp-admin User ownership: root Group ownership:
ngcp-admin File permissions: 664
IMPACT: Configuration data can be edited by all users in group
ngcp-admin, the sipwise user, and user root. If for some reason (e.g.
after upgrade) the proposed settings are reverted to default settings,
this does not pose a security risk. However, write access to
configuration data in this file will be denied to named users.
5.5.6 Restrict access to /etc/ngcp-config/constants.yml to users in
security domain root This file contains values set during the
initialization of the NGCP system. It contains passwords used by
different NGCP functions to connect to other secured subsystems (e.g. DB
or lawful intercept). As such the file has a high security impact and is
read- and writeable to users of security domain root only.
The current default setup of the NGCP does not implement permissions as
required by tightened user security. Required permission settings have
to be applied manually issuing command:
chmod 600 /etc/ngcp-config/constants.yml
Security domain: root User ownership: root Group ownership: root File
permissions: 600
IMPACT: Data in this configuration file are usually entered once during
commissioning of the platform. Only users root or sipwise can edit or
read this file. The customer’s named users have no access to this data.
If for some reason (e.g. after upgrade) the proposed settings are
reverted to default settings, this poses a security risk as credentials
used internally may be leaked to unprivileged users.
Change-Id: I49a2994a227b9c296966c805c9370ae3b067de12
These services files just setup some sysctl settings and should always
be safe to be executed first. In this case we do require them to be
executed first, because many of our services are binding into the shared
IP, which will not be present on the standby node, and would otherwise
fail.
Change-Id: I0df095403461268cdff8517ee5c369355ca9228c
We already know what we have processed and what remains from the
complete list and the new partial list. Use that instead of hard-coding
the directories, which is error-prone as it requires keeping both lists
in sync.
Change-Id: Ia16ac6999beaf89f4c2fb5b74216d0bf17de8414
NGCP uses 'YAML::XS' as only the library to work with YAML files.
Meanwhile pkwalify has a long fallback list:
> @try_order = ('YAML::XS', 'YAML::Syck', 'YAML', 'JSON::XS', 'JSON');
It may hide some issues in validation schema (used by 'ngcpcfg check')
and bypass the testing on Gerrit reviews, but failed to 'ngcpcfg apply'
on the customer's system.
Example:
1) cfg_schema.git had a bug in validation schema committed long time ago
in mr6.1+ , commit b9d69e0b33d316c04d8b70536f1e166fec9a67ac.
The commit passed review and all internal testing:
> https://gerrit.mgm.sipwise.com/#/c/17825/ ->
> https://repoapi.mgm.sipwise.com/job/cfg-schema-ce-validation-docker/849/console.txt
> ...
> + ngcpcfg --validate check
> 2017-12-20 14:34:37: yml configs were validated successfully
The bug was simple, the space was missing in schema:
> "rtcp_logging_facility":{ type: str, required: yes }
The library 'YAML::XS' was not able to load the YAML file.
Meanwhile the library 'YAML' was able to load it and pkwalify happily reported us
"yml configs were validated successfully".
2) Later another developer has noticed the problem with YAML schema here
and fixed it in trunk, commit 4636eadb78e5e91dd6d93378a42a8df2ed84838a.
> - "rtcp_logging_facility":{ type: str, required: yes }
> + "rtcp_logging_facility": { type: str, required: yes }
Unfortunately the fix has been applied in trunk only.
3) Later the third developer committed new code which was valid
for 'YAML::XS' but invalid for 'YAML' library (which is fine, as NGCP uses 'YAML::XS').
Since in trunk pkwalify used 'YAML::XS' the changes passed all the testing:
> commit a79e54fb9fbdf0db61b4dfb9c52e572795e854af
> https://gerrit.mgm.sipwise.com/#/c/28486/ ->
> https://repoapi.mgm.sipwise.com/job/cfg-schema-ce-validation-docker/1987/console.txt
> ...
> 2019-04-05 17:20:08 06828d6f31f5: yml configs were validated successfully
4) Later the third developer backported the commit a79e54fb9fbdf0db
to the previous release mr6.5 (where commit 4636eadb78e was missing).
And nightly upgrade tests has failed on mr6.4->mr6.5 upgrade saying:
> 2019-04-09 04:06:27 sp1: Error: Invalid schema detected for /etc/ngcp-config/config.yml
> Cannot parse </usr/share/ngcp-cfg-schema/validate/config.yml>. Cumulated errors:
> YAML::XS::Load Error: The problem:
> did not find expected key
It happens because pkwalify was not able to load config.yml with any libraries it support.
We have to force pkwalify to use only the library NGCP is using 'YAML::XS'.
This is a commit about it.
Hopefully all the necessary changes have been done by Guillem 2 years ago,
and were uploaded upstream (and even available in default Debian buster):
> https://github.com/eserte/p5-Kwalify/pull/6
Thanks to Guillem Jover for the library fix here and for the help with tracing this issue.
Change-Id: Idbd46b4048a03b5b01a0280a6dcd50406b4222dc
This is necessary when the HA_INT addresses are being changed, the communication
is not possible in that scenario.
Change-Id: I658a4f40e4f5b13d6ac224978999789728d8b915
We will gather the actions from within policy-rc.d and then execute all
of them in a single batch.
This will make sure any required restart or reload is performed even if
there has been no template file changes.
Change-Id: I6458936860a86ddb10cd738a8f2770c109d9ed37
We should print the deferred service actions that will be performed
before executing them. To add traceability.
Change-Id: I912af91a6b609d20a312182b18f4d959a159de65
The previous logic had several issues that need to be addressed, when
doing an early exit:
- The temporary file was not getting removed.
- The services sync-state step was not being executed.
This makes the code more future-proof, and fixes the above problems,
by moving the sync-state into its own function, removing the early
exit, and making the service explicit state changes conditional on
the previous early exit conditional.
Change-Id: Ie420cbb94ea6d16a83c7ae3f546cd3eb38435dd8
We need to initialize ngcpcfg without HA repo,
to solve chicken-eggs problem about git hooks handking.
Also it will be useful to build glusterfs configs
from tt2 templates and yml configuration.
Then start glusterfs and store HA repo on glusterfs.
In this case ngcp-installer will be able to call:
> ngcpcfg initialise --without_shared
> ngcpcfg set ...
> ngcpcfg build ...
> ngcpcfg init-shared
> ngcpcfg apply "Applying everything together"
In the past it was not so critical problem until we started
taking care about the files permissions which are coming from
ngcp-template-* packages and transferred to POSIX perms via ngcpcfg.
One of the problems there is 'git clone' (which is a part of
HA setup initialisation). It resets file perms if they are not
yet stored into file '.ngcpcfg_perms'.
Change-Id: I3cdc93f309f0b899e626779afb1e71f64b2cb953
When running the test suite, in case we need to cover the code paths
that are handling the hooks, we should not assume the files are
installed in the system directories.
Change-Id: I1cd09eaf186d28093eb03ad5e9fc096fe7d63a3e
There was a fix for the similar issue in Mantis ticket #7577,
see commit fc5af3bbb2 for more details.
Unfortunately one corner case was missing there:
"git status" doesn't show untracked files in untracked folders by default.
It requires "-u" option with value "all":
>> man git status
> ...
> -u[<mode>], --untracked-files[=<mode>]
> Show untracked files.
>
> The mode parameter is used to specify the handling of untracked files.
> It is optional: it defaults to all, and if specified,
> it must be stuck to the option (e.g. -uno, but not -u no).
>
> The possible options are:
> · no - Show no untracked files.
> · normal - Shows untracked files and directories.
> · all - Also shows individual files in untracked directories.
The ngcpcfg need to see complete path and filename to untracked file
to properly detect .services files and execute them (if configs were changed):
> root@web01a:/etc# git status
> On branch master
> nothing to commit, working tree clean
> root@web01a:/etc# git status --porcelain | sed 's/^...//'
> root@web01a:/etc# touch test1
> root@web01a:/etc# git status --porcelain | sed 's/^...//'
> test1
> root@web01a:/etc# mkdir test2
> root@web01a:/etc# touch test2/test2_file
> root@web01a:/etc# git status --porcelain | sed 's/^...//'
> test1
> test2/
> root@web01a:/etc# git status -uall --porcelain | sed 's/^...//'
> test1
> test2/test2_file
> root@web01a:/etc#
Change-Id: I18286fd76c511061159f90e8aa9841a6e98c7199
Although the NGCPCFG_RUNNING environment variable might be useful for
some of the subprocesses, it is definitely not useful for anything
that gets started via systemd. In that case we have to transmit this
information via the filesystem.
In the future we should probably extend this with some proper locking,
but for our current problem with ngcp-eaddres we'll still want to
completely avoid running if such file exists, not just prevent
multiple concurrent executions.
Change-Id: I0546620d8ef4263177c5d04f7d2546345c42fbb6
This will make sure any service that should not be running is stopped,
and any that should is started.
Change-Id: I7ab181b66ed3554c8f93f2d9f46dc08ee087a87d
'git status' output has been changed in 2.9 which causes some troubles for us
(we didn't change all the places, we missed ngcpcfg-ha.git repo,
better to use common code in all the places).
Change-Id: Ie664af89ee6d6b31fb3011e763107c41bf4cf00f
It was impossible to set value like '10G' previosuly:
>> ngcpcfg set /etc/ngcp-config/config.yml bootenv.netscript.fallbackfssize=10M
> Bareword found where operator expected at /tmp/tmp.px3MpkUFZS line 28, near "10M"
> (Missing operator before M?)
> Bareword found where operator expected at /tmp/tmp.px3MpkUFZS line 30, near "10M"
> (Missing operator before M?)
> Bareword found where operator expected at /tmp/tmp.px3MpkUFZS line 31, near "10M"
> (Missing operator before M?)
> Bareword found where operator expected at /tmp/tmp.px3MpkUFZS line 33, near "10M"
> (Missing operator before M?)
> Bareword found where operator expected at /tmp/tmp.px3MpkUFZS line 34, near "10M"
> (Missing operator before M?)
> syntax error at /tmp/tmp.px3MpkUFZS line 28, near "10M"
> syntax error at /tmp/tmp.px3MpkUFZS line 30, near "10M"
> syntax error at /tmp/tmp.px3MpkUFZS line 39, near "}"
> Execution of /tmp/tmp.px3MpkUFZS aborted due to compilation errors.
The 'case' statement doesn't support regular expresions in Bash,
let's rewrite them into linear check.
Change-Id: I29ced837fca0abdc3cbe1ea18c614c4c1dd6780c
At the end of installer's 'configuration' section we need
to restart all services were touched during the installation
(otherwise we need to restart the node completely).
The option helps us in this case as etckeeper is
not reliable enough during the installation.
P.S. we pass options from 'apply' to 'services' now,
we should remove 'Unsupported option(s) given: $*' check,
otherwise it is failing as 'apply' requires commit message,
which cause error on 'services' level: 'Unsupported option(s) given'
> 2018-09-13 18:08:33: Error: Unsupported option(s) given: my commit message
> 2018-09-13 18:08:34: Did you mean '--dry-run'?
Change-Id: I5dac44cd87dc1fc90f81e4ac3a14af32c00f825e
We need to have some reproducible order of services restart,
otherwise this might lead to different results, depending on the order.
Also unifyer() looks too heavy here since the files can be easily
unified in a moment of writing them into ${TMPFILE}.
Change-Id: I0a683a7d57cd52b74cba0ee9a285f84f77f3dc20
It was initially introduced in the commit:
> commit 7d422b7878
> Author: Victor Seva <vseva@sipwise.com>
> Date: Fri Aug 22 17:03:39 2014 +0200
>
> MT#8763 check for .git before running the services
>
> On a multi directory config, the services depends on git. Check
> first if the git repository exists and warn about it
The subshell here was necessary to isolate 'cd $dir' call.
In the same time the "proper" bash way is to use pushd/popd.
For the history, 'shellcheck disable' was muting the warning:
> This only exits the subshell caused by the (..) group. [SC2106]
Change-Id: I3546fa9615858dbbc4c0f91d74de003e22837d55
Previously 'systemd daemon-reload' and 'systemctl preset-all'
were executed only if changes detected in configs.
Otherwise the early exit happened if no .service files were triggered:
> DEBUG: No services file(s) reported - nothing to do.
Move the code into the function systed_daemon_reload_preset and
execute it on the top of the script. Also print info message
informing users about reloaded systemd and newly preset units.
It should provide better visibility here.
Change-Id: I992af9fb274ea93a37b812a51cebcd7af5c54133
Otherwise the code in between the functions can be easily missed
which cause issues as described in the ticket and will be fixed
in the following commit.
Change-Id: I57decfcbcd41691d35d085b13881e1e6b5208f6e
This will collect all service actions, synthetize them into their
minimal expression and execute them in a single batch per action.
Change-Id: I950d5db32e0ec6327964faac4ce8f15449f90e90
Otherwise ngcp-installer failed to install PRO/Carrier as
we build the file /etc/ngcp_mgmt_node the very first time on the first MGMT node:
> +08:22:48 cfg_build_templates
> +08:22:48 cfg_build_configs /etc/ngcp_mgmt_node
> +08:22:48 build_configs=($1)
> +08:22:48 declare -a build_configs
> +08:22:48 log_info 'Generating default configuration files /etc/ngcp_mgmt_node'
> +08:22:48 tee -a /tmp/ngcp-installer.log
> +08:22:48 echo 'Generating default configuration files /etc/ngcp_mgmt_node'
> Generating default configuration files /etc/ngcp_mgmt_node
> +08:22:48 ngcpcfg build /etc/ngcp_mgmt_node
> cat: /etc/ngcp_mgmt_node: No such file or directory
> +08:22:48 log_die 'Error running '\''ngcpcfg build'\'''
Checking the file availability is not enough here, the next error is:
> (sp1)root@sp1:/# ngcpcfg build
> /usr/share/ngcp-ngcpcfg/scripts//check: line 83: NGCP_IS_MGMT: unbound variable
> (sp1)root@sp1:/# cat /etc/default/ngcp-roles
> NGCP_TYPE="sppro"
> (sp1)root@sp1:/#
It happens because /etc/default/ngcp-roles is a fake one at this stage.
Handling NGCP_IS_MGMT properly is also not enough:
> (sp1)root@sp1:/# ngcpcfg build
> 2018-07-16 16:49:32: Error: Remote origin of ngcpcfg is '/mnt/glusterfs/ngcpcfg-share', expected: 'sp:/mnt/glusterfs/ngcpcfg-share'.
> 2018-07-16 16:49:32: Error: NOTE: execute `cd /etc/ngcp-config ; git remote set-url origin 'sp:/mnt/glusterfs/ngcpcfg-share'` to adjust setting.
> 2018-07-16 16:49:32: Error: NOTE: perform `ngcpcfg clean --all` to recreate local master branch from remote.
> (sp1)root@sp1:/#
Which happens because the peer is not yet configured at the moment (first node installation).
Introducing new internal option '--no-check-origin' to skip the test into installer.
Change-Id: I0265c65f45972e92ca92320871a7ef29f8904fec
If the file /etc/hosts gets handled by templates, as ongoing changes
attempt to do, there's a problem because /etc/hosts gets truncated to
create a new file when it's needed to generate itself, to resolve the
"localhost" string as hostname, so the generation fails and /etc/hosts
gets empty at that point, affecting the system until restored.
The simplest way to break that cycle is to switch to Unix sockets.
Change-Id: I1e33ead6a134625346b9cf1beb09a7bdbfdfc8d2
Remove any stale .wants/ relationships as systemd does not always seem
to be able to cleanup those by itself. This makes sure we always start
from a clean slate.
Change-Id: I1b241d4530c45658b7245cdbca33e34dde5beaf6
We should be as safe as possible here (at least at the beginning)
to prevent 'patch' command patching wrong code parts.
Change-Id: Ide012169bec1f329309baae32fe186fd77cb9592
The new function will simplify initial customtt->patchtt
migration for end users. Some code was merged to be generic.
Also more tests were add here.
Change-Id: I7719f45275018818b2db82f6deee5b7428670a29
The 'ngcpcfg' received support for 'patchtt' files, like
> /etc/ngcp-config/templates/etc/foo/bar.patchtt.tt2
Those 'patchtt' are going to be applied on default 'tt2 template' file:
> /etc/ngcp-config/templates/etc/foo/bar.tt2
and produce 'customtt' on 'ngcpcfg patch':
> /etc/ngcp-config/templates/etc/foo/bar.customtt.tt2
Further 'customtt' will be used to overwrite 'tt2 templates'
on 'ngcpcfg build' or 'ngcpcfg apply'.
NOTE: 'ngcpcfg patch' is executed automatically on every 'ngcpcfg build'.
It should allows to update ngcp-templates easily and support
local modifications without the pain (until the patches can be applied).
Change-Id: Ice4369386313c5d33e4d498346345eade6f3d0d7
Under systemd we need to issue "daemon-reload" after applying
templates, to avoid spurious warnings when adding/changing/removing
systemd unit files.
Change-Id: Ifd474bec25b94ee9cb1c247b1390103a29c7b7b3
The latter does not support YAML 1.1, nor many parts of the
specification. Use the more compliant implementation, in addition to try
to converge to a single one, so that we do not get serialization delta
surprises.
Change-Id: Ie51f1c79859d40ef0877fc0ab75f86ee72e14ea4
This script will validate the network.yml based on a schema constructed
from information only available from the network.yml file itself. This
way we can do the strictest validation, which we could not do before.
Change-Id: I32714e678e901e58d70e4253bcc61a147494c225
This option is essential if we're testing from within the git
repository and aren't standing on active 'master' branch.
Change-Id: I75b62d071116313317c69ad496e4425d6e98ad5a
At the moment we provide no output for the end users, which
confuses them for the checks with and without the validation.
We should clearly report to users what has been checked.
Change-Id: Icc61744637345a1dc51fe8d8dd9bd3fd7f86824d
Richard Fuchs
Guillem Jover
Michael Prokop
Alexander Lutay
Andreas Granig
Manuel Montecelo
Andrew Pogrebennyk
Mykola Malkov