This is a script, which properly prepares hosts records for
connections towards nosql/sql DBs.
It is able to differentiate the Carrier grade from the PRO setup.
It creates the following list of records:
- at least one record towards central node of the 'db' role (db.central);
- at least one record towards central node of the 'db' role (nosql.central) ;
- records for each PRX/LB node pointing to replicated pair and replicated central ;
In the Carrier grade:
- db.central, points to -> shared IP of the central DB (only on Proxy nodes)
- nosql.central, points to -> shared IP of the central Nosql server (only on Proxy nodes)
- LB has records (for all LB nodes) of replicated pair (nosql db)
pointing to its 'ha_int' type IP address
- Proxy has records (for all Proxy nodes) of replicated pair (nosql db, sql db)
and replicated central type (db only), pointing to its 'ha_int' type IP address
In the PRO setup:
- db.central, points to -> own 'ha_int' type IP address
- nosql.central, points to -> own 'ha_int' type IP address
- records (for all nodes) of replicated pair (nosql db, sql db) and
replicated central type (sql db only), pointing to its 'ha_int' type IP address
This gives us a way for the nosql db / sql db be listening on the
internal IP addresses (not on the loopback ifaces) and not use floating IPs
to be reachable constantly by instances.
Meanwhile instances constantly remain using local nosql/sql DB
(of course excluding a case with the Carrier,
where central sql db/nosql db has a remote shared IP).
Change-Id: Ifceb9fccab48cf78856f8aa8752c3a8b769e3f63
Related to previous commit (fa645475), we shouldn't execute
restore-permissions with any actions that aren't modifying system state.
Therefore also skip restore-permissions execution for diff, log + status
actions.
While at it, sort list of actions alphabetically.
Thanks Guillem Jover for raising this.
Change-Id: I803b08eee843d31f12d9810fe33b4327490228da
Execution of `ngcpcfg check` shouldn't modify system state,
and executing the restore-permissions helper script also
takes some notable time (e.g. during ngcp-status execution
it causes ~10% of its runtime).
Change-Id: I8ad861379da3817f0bdf760ea16939c80eae2ce6
We should validate all configuration files by concatenating them, so
that any local addition or override gets schema validated, otherwise
these modifications will pass without notice, until it is probably too
late.
Change-Id: I9c7774eefdeadddb399dd68668779dffcd173696
This new action concatenates the various configuration files and outputs
the result as YAML. This is useful because they have a determined
ordering, and include optional host specific files that might not be
obvious how to merge.
This will also be the foundation for verifying all the current host
settings against the cfg-schema.
Change-Id: I72a61193f74caf3b2f7a58a47eefad2ed46c973a
The YAML::XS module handles its passed or returned data for Load() and
Dump() as raw UTF-8, irrespective of perl's UTF-8 string markers. When
we use these functions we need to use a «:raw» encoding or we will end
up with doubly encoded strings.
We also need to tell the Template Toolkit to use UTF-8 as encoding, and
then setup the file handles to use the appropriate encodings.
Change-Id: I37a6811f0680763d7177c3ad92ddf9b890869e66
We need to add a new script, which finds and grabs
a list needed advertised ip addresses for a particular instance.
Change-Id: Id33c4e2d4de071c5feb34dd9346a33ef4a61381b
Defining instances connections could be necessary (for example in SEMS)
to get both the IP of the sip_int interface of LB instance and the IP
of the sip_ext interface.
This is not currently possible because we are allowing definitions of
only one interface type per connection.
The new supported structure is:
connections:
lb:
- type: instance
name: A
interfaces:
- name: vlan1720
type: sip_int
- name: vlan1721
type: sip_ext
* pre-commit: exclude dirs from end-of-file-fixer hook
Change-Id: I832645cb916ee7f2febbea6397a439ea06fbe955
Otherwise 'ngcpcfg apply' silently aborted on stage 'build'
and error message is not visible due to long/parallel output.
Change-Id: I34b2c14fecad33526d89939e62d85dbafab31535
Returns an array of IPs from the connection list passed as argument
The ips are retrieved from the definition of other instances or hosts
@param argv.connection the connection list of the instance
@return out the array of IPs.
* ignore vscode files
Change-Id: I1e108f00d771e5af1d479ffa2cbced786d15e25f
* We were missing the scenario when only the build files for an instance
had changes. The service for that instance was not restarted
Change-Id: I85ba86bc57c7fc8b6df1ac2ef1144c2d3952a4da
This will be used in the templates for values that are scaled based on
the number of online CPUs on the system, such as process pools, or load
averages.
Change-Id: I3666873f327becc3f7a02be02dba82493b29252e
The prevalent convention in the project is to use «-» instead of «_» to
split words in options. Switch the options to that, and keep backwards
compatibility option alias that emits a deprecation warning while people
transition to the new names. The alias will be removed on the next LTS.
Change-Id: I8bb7f26f8a766f7ea76ba2d9e49b793e5cb8d11e
Instead of having to identify the parent process name (ngcpcfg vs.
ngcp-config) inside subprocesses (like "status"), let's provide the
invocation name through the main ngcpcfg binary via variable
NGCPCFG_NAME.
Thanks to Guillem Jover for the suggestion.
Change-Id: Iafbac535bedb9cc9dc73f3d95861a5bc735cc739
will provide a new instance_info variable with the all the content
defined at network.yml for the instance
removed intance_name since instance_info.name has the same value
Change-Id: I349d0b430b59280bc383d46d5ba6a61f5d1f9046
When invoking `ngcp-config status`, we reported "ngcpcfg":
| [sipwise-lab-trunk] sipwise@sp1:~$ sudo ngcpcfg status
| 2021-12-09 17:30:08 sp1: Checking state of ngcpcfg:
| 2021-12-09 17:30:08 sp1: OK: has been initialised already
| [...]
While what we should report instead is "ngcpcfg-config", like:
| [sipwise-lab-trunk] sipwise@sp1:~$ sudo ngcp-config status
| 2021-12-09 17:31:31 sp1: Checking state of ngcp-config:
| 2021-12-09 17:31:31 sp1: OK: has been initialised already
| [...]
Change-Id: If8fc8076fdf1d214baea3c7388ab208947d66a05
The previous error message was really confusing:
> root@spce:~# ngcpcfg get /asdf
> 2021-12-06 18:48:12 sp1: Error: Cannot process template '/tmp/tmp.RBRJg02PEz':
>
> file error - parse error - input file handle line 1: unexpected token (/asdf)
> [% /asdf %]
> 2021-12-06 18:48:12 sp1: Error: Generating /tmp/tmp.NKfB8BtJXv based on /tmp/tmp.RBRJg02PEz: FAILED
It happens due to the way ngcpcfg using TT2 template which
is trying to include the file it the string looks like a PATH.
The new error message is more user friendly
(and generates proper/non-zero exit code):
> root@spce:~# ngcpcfg get /asdf
> 2021-12-07 12:42:50 spce: Error: cannot process request for '/asdf'!
> root@spce:~# echo $?
> 1
> root@spce:~#
Change-Id: Ifaf56dec07adf8600bd9d7cf2e733769d6d82a4f
- NGCPCTL_BASE
- OUTPUT_DIRECTORY{CONFIG_POOL}
* add empty-git-repo.tar.gz
* fs.py: define check_output() helper
* set DEBUG by default
* ngcpcfgcli, return env and cfg instead of just outdir
* cleanup on tests:
- don't compile the regex, it's useless in a test contex
- remove unused imports and fixtures
- use check_output() from fs
Change-Id: I0eb859b6f7aad4cd3dd8e15f4e211395c5131ce7
* expects values from CONFIG_POOL
* append OUTPUT_DIRECTORY just when changing directory
added at 83003649e3
Change-Id: I9c22bff70b58d4cfb49c3dab45330bccb898011e
> # Backward compatibility config for upgrade mr3.4*->mr3.5*
> # it can be removed when the next LTS is released:
Change-Id: Ie93fc45965251dc3384a3b624339d45ad144683f
Switch all code to use a loop to parse options so that these can parse
multiple options (currently some scripts would simply parse one or
another but not several), and to make them easier to extend.
Change-Id: I5e902f2d7d8db78e66e48db1237924fa7d8fdda1
The file /etc/ngcp-config/ngcpcfg.instances can describe several instances
(like 'lb' and 'proxy'), but /etc/ngcp-config/network.yml might describe one
instance only (e.g. 'lb') in this case helper instances-info produced an error:
> Can't use an undefined value as an ARRAY reference at /usr/share/ngcp-ngcpcfg/helper//instances-info line 117.
We should not consider records from ngcpcfg.instances if no such
instances defined in network.yml.
Change-Id: I9dd6cfff79229fa464c64c565c96835cd183e308
(cherry picked from commit efc9eb8e7adfb57427051b86cc96520c7dd9cf81)
we need to produce different output for each instance defined
from the same source
* helper/instances-info, produces the needed output to feed
helper/tt2-process with the proper arguments to build
instances files. We inject ``instance_name`` with the
proper value to templates and INSTANCE_NAME to hooks
environment
* ${NGCPCTL_MAIN}/ngcpcfg.instances is where we define the
supported instance templates and where the output path is
* helper/tt2-process, add -r --replace <path:path> option
in order to allow generation of different output from
the same source
Change-Id: I067266b5226485ea0d6c3bc3436275c5553a8177
hook_setup() is called directly by some scripts
* fix test for apply: forcing changes in the repo in order to
have a commit message
Change-Id: I7da6f210826ae817b574bba793e61346ed6cac0d
gitrepo.from_archive() is suppose to be used at a ``with`` statement
so at the end the temporary directory gitrepo.localpath is removed
> with gitrepo.from_archive(path_tar) as git:
> pass
> # here gitrepo.localpath is gone already
So instead of using from_archive() let's create the temporary directory
and extract the tar file there instead.
Be aware that gitrepo.default has the git repository inside the ``testing``
directory
Change-Id: I7a50baab8c5a444e08aeabc2f1597e3a13d94c84
ngcpcfg service expects that the produced files are inside a git
repository
* ngcpcfgcli:
- set RUN_DIR by default
- set STATE_FILES_DIR by default
- use default-git-repository.tar.gz as base for outdir
- return outdir value too
- print stdout and stderr so on failure we can debug
* t/test_*:
- remove unused imports
- don't create OUTPUT_DIRECTORY unless really necessary
- remove print stdout/stderr
- black indent
Change-Id: I61e475cc9c775c30c3700a8c0e560e7c5b22f0fc
* remove support for local result
We need to force using /code volumen as read-only
to detect problems with the test environment
Change-Id: Idce04b1b00caf22562fc4161eb52317459aee03d
As of commit f607c7d and its raise of debhelper compat level to v13,
we no longer support Debian/buster, as builds are failing with:
| The following packages have unmet dependencies:
| builddeps:. : Depends: debhelper-compat (= 13)
| E: Unable to correct problems, you have held broken packages.
Replace our github action for Debian/buster (current oldstable)
with Debian/bullseye (current stable).
Change-Id: Ia4289454bd807351e6bb9389ff50bffd1587f043
shellcheck v0.8 has new check, known as SC2295 ("Warn about "${x#$y}"
treating $y as a pattern when not quoted").
Fixes:
| In scripts/build line 117:
| build_file="${build_file%%.${host}}"
| ^-----^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.
| In scripts/encrypt line 24:
| x=${file##${NGCPCTL_MAIN}/templates/} # drop leading /etc/ngcp-config
| ^-------------^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.
... as reported by our Github actions at
https://github.com/sipwise/ngcpcfg
Change-Id: I3371388ecf445905cca0bc4681276151aceb43ae
Sipwise Jenkins Builder
Donat Zenichev
ngcp-config
Marco Capetta
Michael Prokop
Guillem Jover
Alexander Lutay
Richard Fuchs
Rene Krenn