21 Commits (80ead123abb65ad98a6c240855c51484a4831452)

Author	SHA1	Message	Date
Alexander Lutay	80ead123ab	TT#65907 Convert EXTRA_CONFIG_FILES from string to array to make shellcheck happy here ... Otherwise: > warning: Brace expansions and globs are literal in assignments. Quote it or use an array. [SC2125] Change-Id: I5a209cc1b50cea37e13aef40dfbd86c6178b53dc	6 years ago
Michael Prokop	ca5730fe23	TT#58205 Fix misc typos ... s/Swtich/Switch/ s/explicitely/explicitly/ s/precendence/precedence/ s/successfull/successful/ s/successfuly/successfully/ Change-Id: I7b010d36c7b970771001a6aa66e51f0c7eba4933	7 years ago
Alexander Lutay	eff2d91d82	TT#57012 Force 'ngcpcfg check'->pkwalify to use 'YAML::XS' only ... NGCP uses 'YAML::XS' as only the library to work with YAML files. Meanwhile pkwalify has a long fallback list: > @try_order = ('YAML::XS', 'YAML::Syck', 'YAML', 'JSON::XS', 'JSON'); It may hide some issues in validation schema (used by 'ngcpcfg check') and bypass the testing on Gerrit reviews, but failed to 'ngcpcfg apply' on the customer's system. Example: 1) cfg_schema.git had a bug in validation schema committed long time ago in mr6.1+ , commit b9d69e0b33d316c04d8b70536f1e166fec9a67ac. The commit passed review and all internal testing: > https://gerrit.mgm.sipwise.com/#/c/17825/ -> > https://repoapi.mgm.sipwise.com/job/cfg-schema-ce-validation-docker/849/console.txt > ... > + ngcpcfg --validate check > 2017-12-20 14:34:37: yml configs were validated successfully The bug was simple, the space was missing in schema: > "rtcp_logging_facility":{ type: str, required: yes } The library 'YAML::XS' was not able to load the YAML file. Meanwhile the library 'YAML' was able to load it and pkwalify happily reported us "yml configs were validated successfully". 2) Later another developer has noticed the problem with YAML schema here and fixed it in trunk, commit 4636eadb78e5e91dd6d93378a42a8df2ed84838a. > - "rtcp_logging_facility":{ type: str, required: yes } > + "rtcp_logging_facility": { type: str, required: yes } Unfortunately the fix has been applied in trunk only. 3) Later the third developer committed new code which was valid for 'YAML::XS' but invalid for 'YAML' library (which is fine, as NGCP uses 'YAML::XS'). Since in trunk pkwalify used 'YAML::XS' the changes passed all the testing: > commit a79e54fb9fbdf0db61b4dfb9c52e572795e854af > https://gerrit.mgm.sipwise.com/#/c/28486/ -> > https://repoapi.mgm.sipwise.com/job/cfg-schema-ce-validation-docker/1987/console.txt > ... > 2019-04-05 17:20:08 06828d6f31f5: yml configs were validated successfully 4) Later the third developer backported the commit a79e54fb9fbdf0db to the previous release mr6.5 (where commit 4636eadb78e was missing). And nightly upgrade tests has failed on mr6.4->mr6.5 upgrade saying: > 2019-04-09 04:06:27 sp1: Error: Invalid schema detected for /etc/ngcp-config/config.yml > Cannot parse </usr/share/ngcp-cfg-schema/validate/config.yml>. Cumulated errors: > YAML::XS::Load Error: The problem: > did not find expected key It happens because pkwalify was not able to load config.yml with any libraries it support. We have to force pkwalify to use only the library NGCP is using 'YAML::XS'. This is a commit about it. Hopefully all the necessary changes have been done by Guillem 2 years ago, and were uploaded upstream (and even available in default Debian buster): > https://github.com/eserte/p5-Kwalify/pull/6 Thanks to Guillem Jover for the library fix here and for the help with tracing this issue. Change-Id: Idbd46b4048a03b5b01a0280a6dcd50406b4222dc	7 years ago
Andreas Granig	0700257c13	TT#56454 Fix cosmetic typo in log message ... Change-Id: Id81696c8fcf82db4beb879d9c30854ec6ce80208	7 years ago
Manuel Montecelo	6b833f7f31	TT#44854 Add option --ignore-shared-storage-check for "build" and "check" ... This is necessary when the HA_INT addresses are being changed, the communication is not possible in that scenario. Change-Id: I658a4f40e4f5b13d6ac224978999789728d8b915	7 years ago
Alexander Lutay	e218928a60	TT#38453 Fix f95eeb2: skip new check_origin() on installer stage ... Otherwise ngcp-installer failed to install PRO/Carrier as we build the file /etc/ngcp_mgmt_node the very first time on the first MGMT node: > +08:22:48 cfg_build_templates > +08:22:48 cfg_build_configs /etc/ngcp_mgmt_node > +08:22:48 build_configs=($1) > +08:22:48 declare -a build_configs > +08:22:48 log_info 'Generating default configuration files /etc/ngcp_mgmt_node' > +08:22:48 tee -a /tmp/ngcp-installer.log > +08:22:48 echo 'Generating default configuration files /etc/ngcp_mgmt_node' > Generating default configuration files /etc/ngcp_mgmt_node > +08:22:48 ngcpcfg build /etc/ngcp_mgmt_node > cat: /etc/ngcp_mgmt_node: No such file or directory > +08:22:48 log_die 'Error running '\''ngcpcfg build'\''' Checking the file availability is not enough here, the next error is: > (sp1)root@sp1:/# ngcpcfg build > /usr/share/ngcp-ngcpcfg/scripts//check: line 83: NGCP_IS_MGMT: unbound variable > (sp1)root@sp1:/# cat /etc/default/ngcp-roles > NGCP_TYPE="sppro" > (sp1)root@sp1:/# It happens because /etc/default/ngcp-roles is a fake one at this stage. Handling NGCP_IS_MGMT properly is also not enough: > (sp1)root@sp1:/# ngcpcfg build > 2018-07-16 16:49:32: Error: Remote origin of ngcpcfg is '/mnt/glusterfs/ngcpcfg-share', expected: 'sp:/mnt/glusterfs/ngcpcfg-share'. > 2018-07-16 16:49:32: Error: NOTE: execute `cd /etc/ngcp-config ; git remote set-url origin 'sp:/mnt/glusterfs/ngcpcfg-share'` to adjust setting. > 2018-07-16 16:49:32: Error: NOTE: perform `ngcpcfg clean --all` to recreate local master branch from remote. > (sp1)root@sp1:/# Which happens because the peer is not yet configured at the moment (first node installation). Introducing new internal option '--no-check-origin' to skip the test into installer. Change-Id: I0265c65f45972e92ca92320871a7ef29f8904fec	8 years ago
Michael Prokop	f95eeb2eff	TT#38453 Verify remote repository origin in 'check' action (PRO/Carrier) ... CE: nothing to be done, no remote present PRO (both sp1 and sp2): Fetch URL: /mnt/glusterfs/ngcpcfg-share Push URL: /mnt/glusterfs/ngcpcfg-share Carrrier web01a/web01b (mgmt node): Fetch URL: /mnt/glusterfs/ngcpcfg-share Push URL: /mnt/glusterfs/ngcpcfg-share Carrrier db01a/... (non-mgmt nodes): Fetch URL: $(cat /etc/ngcp_mgmt_node}:/mnt/glusterfs/ngcpcfg-share Push URL: $(cat /etc/ngcp_mgmt_node}:/mnt/glusterfs/ngcpcfg-share Change-Id: Ic5e9bd51580e9703eac1be49ac2559b9e90fd606	8 years ago
Michael Prokop	a42248419e	TT#22411 Fix spelling mistakes/typos ... s/Syncronizes/Synchronizes/ s/Wheras/Whereas/ s/configuation/configuration/ s/exisiting/existing/ s/explicitely/explicitly/ s/files files/files/ s/has has/has/ s/initialise initialise/initialise/ s/mulitple/multiple/ s/ouput/output/ s/propogates/propagates/ s/successfull/successful/ s/syncronizes/synchronizes/ s/the the/the/ s/uncommited/uncommitted/ Change-Id: Id13ec889c5de620b675e357d99066688cfc97617	8 years ago
Guillem Jover	a4dd0365ef	TT#17650 Switch to use YAML::XS instead of YAML::Tiny ... The latter does not support YAML 1.1, nor many parts of the specification. Use the more compliant implementation, in addition to try to converge to a single one, so that we do not get serialization delta surprises. Change-Id: Ie51f1c79859d40ef0877fc0ab75f86ee72e14ea4	9 years ago
Guillem Jover	2ab2a94830	TT#14817 Add new network.yml dynamic validator program ... This script will validate the network.yml based on a schema constructed from information only available from the network.yml file itself. This way we can do the strictest validation, which we could not do before. Change-Id: I32714e678e901e58d70e4253bcc61a147494c225	9 years ago
Michael Prokop	e06c458e22	MT#16391 Support --ignore-branch-check option in build/check actions ... This option is essential if we're testing from within the git repository and aren't standing on active 'master' branch. Change-Id: I75b62d071116313317c69ad496e4425d6e98ad5a	9 years ago
Alexander Lutay	e5a1c0d3fe	TT#16680 Add info output for successful yml check/validation ... At the moment we provide no output for the end users, which confuses them for the checks with and without the validation. We should clearly report to users what has been checked. Change-Id: Icc61744637345a1dc51fe8d8dd9bd3fd7f86824d	9 years ago
Alexander Lutay	2317a9039c	MT#19571 Add ngcpcfg option to force validation ... Mainly nobody validate the schema in trunk right now as it requires manual changes for ngcpcfg.cfg. Also having force validation will allow us to add validation in ngcp-upgrade and Jenkins nightly builds. Change-Id: Ia48a778ecf7dae30b26bdff0c5dd5b9000c9d089	10 years ago
Michael Prokop	4fd512261b	MT#16393 Check for branch 'master' being active ... Verify that the user is operating on branch master, otherwise the result might be unexpected. If branch master is checked out: | root@spce:/etc/ngcp-config# ngcpcfg check | root@spce:/etc/ngcp-config# ngcpcfg status | [...] | 2016-02-23 05:48:32: Checking currently active branch: | 2016-02-23 05:48:32: OK: branch master active If a branch other than 'master' is checked out: | root@spce:/etc/ngcp-config# ngcpcfg check | 2016-02-23 05:47:42: Error: branch 'mika' in '/etc/ngcp-config' active - please switch to branch 'master' before continuing. | root@spce:/etc/ngcp-config# echo $? | 1 | root@spce:/etc/ngcp-config# ngcpcfg status | [...] | 2016-02-23 05:47:47: Checking currently active branch: | 2016-02-23 05:47:47: ACTION_NEEDED: branch 'mika' active - please switch to branch 'master' | [...] Change-Id: I5df92075905cafa3b714211581c8cfe749df04ba	10 years ago
Alexander Lutay	550067a6a8	MT#15803 Check oustanding pull actions only (ignore outstanding push actions) ... Switch code to new common check function execute_check_shared_storage() Change-Id: I7ba86912d7b5874f9ce0778a2a87b2afb33be5ee	10 years ago
Alexander Lutay	c89f72e6b5	MT#15141 Hide debug output to keep old ngcpcfg output style ... Also show affected files with merge conflicts if available. Change-Id: I5d85e0980bb974fecb16dc45ea2426e5655be0a3	10 years ago
Michael Prokop	7adf2b23e2	MT#15803 HA: support shared storage check in 'ngcpcfg check' ... This allows us to abort in 'ngcpcfg status' whenever there are outstanding changes to pull/push from the shared storage. Using the --no-action-failure option allows the user to continue anyway. Change-Id: I4062d5bb627bb553b98705bb122575651b035849	10 years ago
Alexander Lutay	b9d07def25	MT#15261 Allow using symbolic links for YML files ... At the moment it is impossible to use symbolic links for YML files. The following error happens: > root@spce:/etc/ngcp-config# ngcpcfg apply > Error: Encoding check of /etc/ngcp-config/config.spce.yml fails: neither ASCII nor UTF-8. > Error: Please convert /etc/ngcp-config/config.spce.yml to UTF-8. > > NOTE: > * Check encoding via: > # file /etc/ngcp-config/config.spce.yml > * To convert ISO-8859/latin1 to UTF-8 execute: > # iconv -f latin1 -t utf8 < /etc/ngcp-config/config.spce.yml > /etc/ngcp-config/config.spce.yml.tmp \ > && mv /etc/ngcp-config/config.spce.yml.tmp /etc/ngcp-config/config.spce.yml As a result the following usecase ('node-type' specific file) failed: > >>> ls -la /etc/ngcp-config/* > total 72 > drwxr-xr-x 2 root root 4096 Sep 18 13:57 . > drwxr-xr-x 6 root root 4096 Sep 16 23:34 .. > lrwxrwxrwx 1 root root 15 Sep 18 13:57 config.db01a.yml -> ./config.db.yml > lrwxrwxrwx 1 root root 15 Sep 18 13:57 config.db01b.yml -> ./config.db.yml > -rw-r--r-- 1 root root 224 Sep 18 12:43 config.db.yml > -rw-r--r-- 1 root root 250 Sep 18 12:34 config.lb.yml > -rw-r--r-- 1 root root 146 Sep 18 12:44 config.mgmt.yml > -rw-r--r-- 1 root root 251 Sep 18 13:56 config.proxy.yml > lrwxrwxrwx 1 root root 18 Sep 18 13:57 config.prx01a.yml -> ./config.proxy.yml > lrwxrwxrwx 1 root root 18 Sep 18 13:57 config.prx01b.yml -> ./config.proxy.yml > lrwxrwxrwx 1 root root 18 Sep 18 13:57 config.prx02a.yml -> ./config.proxy.yml > lrwxrwxrwx 1 root root 18 Sep 18 13:57 config.prx02b.yml -> ./config.proxy.yml > lrwxrwxrwx 1 root root 18 Sep 18 13:57 config.prx03a.yml -> ./config.proxy.yml > lrwxrwxrwx 1 root root 18 Sep 18 13:57 config.prx03b.yml -> ./config.proxy.yml > lrwxrwxrwx 1 root root 18 Sep 18 13:57 config.prx04a.yml -> ./config.proxy.yml > lrwxrwxrwx 1 root root 18 Sep 18 13:57 config.prx04b.yml -> ./config.proxy.yml > lrwxrwxrwx 1 root root 18 Sep 18 13:57 config.prx05a.yml -> ./config.proxy.yml > lrwxrwxrwx 1 root root 18 Sep 18 13:57 config.prx05b.yml -> ./config.proxy.yml > lrwxrwxrwx 1 root root 15 Sep 18 13:57 config.slb01a.yml -> ./config.lb.yml > lrwxrwxrwx 1 root root 15 Sep 18 13:57 config.slb01b.yml -> ./config.lb.yml > lrwxrwxrwx 1 root root 17 Sep 18 13:57 config.web01a.yml -> ./config.mgmt.yml > lrwxrwxrwx 1 root root 17 Sep 18 13:57 config.web01b.yml -> ./config.mgmt.yml > -rw-r--r-- 1 root root 13051 Sep 18 12:54 config.yml > -rw-r--r-- 1 root root 4633 Sep 15 16:13 constants.yml > -rw-r--r-- 1 root root 16534 Sep 15 16:09 network.yml > >>> cat config.lb.yml > --- > checktools: > collcheck: > loadlong: 2 > loadmedium: 2 > loadshort: 3 > heartbeat: > pingnodes: > - 1.1.1.1 > - 2.2.2.2 Change-Id: I43045d08c715d17612617c29cf1ed04cf13e650b	11 years ago
Alexander Lutay	986030b1f2	MT#15141 Add check for merge failures ... Change-Id: Id078f26296208a0dac2112e9f2a0066dbe835984	11 years ago
Alexander Lutay	34f731bb1d	MT#13669 Add YML configs validation accordingly to kwalify schema ... Disabled by default as we need a time to check it carefuly. Also we need to solve the "tilda" problem reported upstream: https://github.com/eserte/p5-Kwalify/issues/1 Change-Id: Ia2c3d48f0ac6fc6ac9899d44f4291544373806d9	11 years ago
Alexander Lutay	0a3867a8a4	MT#10071 Add 'check' option for YAML validations ... Change-Id: I52938bfeb99a6c45d7227a3b02739cbd5f6232fa	11 years ago