ngcpcfg(8) ========== Name ---- ngcpcfg - central and template based Configuration Management Synopsis -------- ngcpcfg [] [ ] Introduction ------------ ngcpcfg is a Configuration Management System, developed for the Sipwise Next Generation Platform. It provides a central mechanism for handling configuration changes, updates and synchronisation between servers through a main configuration which is simple and easy to read and modify. tl;dr? - ngcpcfg for the impatient ---------------------------------- The main system configuration is done in the file _/etc/ngcp-config/config.yml_. After modifying the file execute 'ngcpcfg apply ""' to build the according configuration files. Taxonomy -------- *central yml files*:: *.yml files inside _/etc/ngcp-config/_ *High Availability setup*:: ngcpcfg running in a cluster setup (e.g. sip:providerPRO or sip:carrier), depends on Debian package ngcp-ngcpcfg-ha. *local repository*:: the directory _/etc/ngcp-config/_, being a Git repository *remote systems*:: the other nodes inside a High Availability setup, as defined in _/etc/ngcp-config/systems.cfg_ *shared repository*:: Git repository shared amongst nodes inside a High Availability setup, as defined by the GLUSTERFS setting in /etc/ngcp-config/ngcpcfg.d/shared_storage.cfg (requires ngcp-ngcpcfg-ha) and being _/mnt/glusterfs_ by default *templates*:: template toolkit files with .tt2 suffix, found in _/etc/ngcp-config/templates/etc/_ [[configfiles]] Configuration files ------------------- Main configuration files ~~~~~~~~~~~~~~~~~~~~~~~~ * _/etc/ngcp-config/config.yml_: central configuration file, to be configured with $EDITOR, webfrontend,... * _/etc/ngcp-config/config.$HOSTNAME.yml_: host specific configuration file, depending on the hostname (:= $HOSTNAME) of the system. * _/etc/ngcp-config/config.local.yml_: local configuration, not being host specific. * _/etc/ngcp-config/constants.yml_: configuration file that has precedence over any other .yml file _/etc/ngcp-config/_, defining important constant settings. This file is *not* supposed to be modified by the user (without having a very good reason). * _/etc/ngcp-config/ngcpcfg.cfg_: main configuration file for ngcpcfg itself, provides global variables used inside ngcpcfg and its helper scripts. This file is *not* supposed to be modified by the user (without having a very good reason). * _/etc/ngcp-config/config.d/_: optional configuration directory with additional yml configuration files. Files inside this directory aren't supposed to be added or modified by the user. The purpose of this configuration directory is that software packages can enable/disable certain features through configuration options on demand. * _/etc/ngcp-config/ngcpcfg.d/_: configuration directory for ngcpcfg itself. Files with suffix '.cfg' inside this directory are sourced after /etc/ngcp-config/ngcpcfg.cfg has been read. Files inside this directory are *not* supposed to be modified by the user (without having a very good reason). [IMPORTANT] Configuration file priority: constants.yml takes precedence over all files in _/etc/ngcp-config/config.d/_, over config.local.yml, over config.$HOSTNAME.yml, over config.yml. High Availability setup specific configuration files ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * _/etc/ngcp_ha_node_: defines node name in the High Availability setup, usually being _sp1_ for the first node and _sp2_ for the second node. * _/etc/ngcp-config/systems.cfg_: configuration file that specifies to which hosts changes should be pushed to. Supported template files ~~~~~~~~~~~~~~~~~~~~~~~~ Example for generating a configuration file named _/etc/foobar/baz_ (from lower precedence to higher precedence): * _/etc/ngcp-config/templates/etc/foobar/baz.tt2_: main and default template file, used by template-handler for generating /etc/foobar/baz. Configuration file is usually provided by a Debian package. * _/etc/ngcp-config/templates/etc/foobar/baz.tt2.$HA_NODE_: node specific template file. $HA_NODE is determined using the content of /etc/ngcp_ha_node (usually being _sp1_ for the first node and _sp2_ for the second node on the Sipwise Next Generation Platform). Whereas _*customtt.tt2_ files are used on all nodes in a High Availability setup the _*.tt2.$HA_NODE*_ file is specific for the single node only. A common use case is master vs. slave configuration of a service. The configuration file is usually provided by a Debian package. Note: Feature is available in the High Availability setup only. * _/etc/ngcp-config/templates/etc/foobar/baz.tt2.$PAIRNAME_: pair specific template file. Given a PRO pair in a CARRIER environment where two nodes are named _web01a_ and _web01b_ then $PAIRNAME corresponds to _web01_. This is useful if there are multiple pairs available but only a specific pair of them should include some specific configuration. Note: Feature is available in the High Availability setup only. * _/etc/ngcp-config/templates/etc/foobar/baz.tt2.$HOSTNAME_: hostname specific template file. While $HA_NODE is usually bound to _sp1_ for the first node and _sp2_ via /etc/ngcp_ha_node it's possible to use host specific template file by referring to its hostname ($HOSTNAME is determined via (hostname(1)). Note: Feature is available in the High Availability setup only. * _/etc/ngcp-config/templates/etc/foobar/baz.customtt.tt2_: system specific template file, but configuration usually isn't provided by a Debian package and can be modified independently from any Debian package mechanism. * _/etc/ngcp-config/templates/etc/foobar/baz.customtt.tt2.$HA_NODE_: node specific template file. Regarding $HA_NODE the same as for _baz.tt2.$HA_NODE_ applies (see the previous bullet), but the configuration file usually isn't provided by a Debian package but can be modified independently from any Debian package mechanism. Note: Feature is available in the High Availability setup only. * _/etc/ngcp-config/templates/etc/foobar/baz.customtt.tt2.$PAIRNAME_: configuration file similar to _/etc/ngcp-config/templates/etc/foobar/baz.tt2.$PAIRNAME_ but it's guaranteed that the file won't be part of any Debian package mechanism. Note: Feature is available in the High Availability setup only. * _/etc/ngcp-config/templates/etc/foobar/baz.customtt.tt2.$HOSTNAME_: configuration file similar to _/etc/ngcp-config/templates/etc/foobar/baz.tt2.$HOSTNAME_ but it's guaranteed that the file won't be part of any Debian package mechanism. Note: Feature is available in the High Availability setup only. [IMPORTANT] Configuration file precedence (highest to lowest): *.customtt.tt2.$HOSTNAME *.customtt.tt2.$PAIRNAME *.customtt.tt2.$HA_NODE *.customtt.tt2, *.tt2.$HOSTNAME *.tt2.$PAIRNAME *.tt2.$HA_NODE *.tt2. Customisation for default template files ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can overwrite the default template file (e.g. _/etc/ngcp-config/templates/etc/foobar/baz.tt2_) using a _customtt_ file, like _/etc/ngcp-config/templates/etc/foobar/baz.customtt.tt2_. Or even host a specific _customtt_ file (see above in "Supported template files"). This approach is NOT recommended as _customtt_ will become outdated very soon, hence new template files can be released by upstream any time. The better way is to handle modifications using _patchtt_ files (e.g. _/etc/ngcp-config/templates/etc/foobar/baz.patchtt.tt2_). In this case, on every "ngcpcfg patch", _patchtt_ file will be applied on top of the tt2 file and the result will be saved into the _customtt_ file, which in the future will be used in a common way. "ngcpcfg patch" is the first step on "ngcpcfg build" that guarantees the latest upstream templates with the availability of the necessary local changes on every configuration apply. [IMPORTANT] The patch to be applied to the corresponding tt2 template file is selected in the following order (highest to lowest): *.patchtt.tt2.$HOSTNAME *.patchtt.tt2.$PAIRNAME *.patchtt.tt2.$HA_NODE *.patchtt.tt2 [IMPORTANT] If a suitable _patchtt_ file is found for a template, then the **ngcpcfg patch** command will overwrite the corresponding _customtt_ file, if any exists. You can find the old version of the customtt in ngcpcfg the git repository (if necessary). Support action related files ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Example for handling a configuration file named _/etc/foobar/baz_: * _/etc/ngcp-config/templates/etc/foobar/baz.services_: service file, defining actions that have to be executed when file /etc/foobar/baz was modified. * _/etc/ngcp-config/templates/etc/foobar/ngcpcfg.services_: service file, defining actions that have to be executed whenever *any* file inside /etc/foobar was modified. * _/etc/ngcp-config/templates/etc/foobar/baz.postbuild_: script which is executed *after* a configuration file (/etc/foobar/baz) has been generated. The environment variable "$output_file" containing the file name of the generated configuration file (/etc/foobar/baz) is available within the postbuild script. A common usage case for postbuild files is adjusting file permissions. * _/etc/ngcp-config/templates/etc/foobar/baz.prebuild_: script which is executed *before* a configuration file (/etc/foobar/baz) is generated. The environment variable "$output_file" containing the file name of the generated configuration file (/etc/foobar/baz) is available within the postbuild script. Syntax and layout of configuration files ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * _/etc/ngcp-config/ngcpcfg.cfg_: plain shell syntax style "key=value" entries * _/etc/ngcp-config/systems.cfg_: one hostname or IP address per line * _central yml files_ (*.yml): YAML configuration syntax (see http://yaml.org/) * _template files_ (*.tt2): whatever the according software (Kamailio, MySQL,...) needs. Any variables that should be replaced by the configuration management system (based on the main configuration files *.yml inside /etc/ngcp-config/) have to be written in the YAML syntax (see http://yaml.org/) * _service files_ (*.services) and _build files_ ({post,pre}build): file will be executed under "bash $FILE", so you can use commands e.g. like "ngcp-service foobar restart" and any further shell scripting syntax inside the services files Global Options -------------- **--debug** []:: Run actions (see next section) in verbose mode, useful for debugging. Debug output will be sent to syslog (available as /var/log/ngcp/ngcpcfg.log on NGCP systems) as well as stderr. **--help**:: Display usage information and exit. **--no-action-failure** []:: The _check_ and _apply_ actions check for any possibly outstanding pull actions on the shared storage. If any outstanding actions are identified then the script aborts to avoid running into tricky configuration merge situations. If this option is enabled then ngcpcfg instead doesn't abort on outstanding pull actions. This option should be used with care and only if you know what you're doing. Note: This option is available in the High Availability setup only. **--no-validate** []:: Ignore schema validation results for YAML files (syntax check is still performed). **--validate** []:: Force schema validation for YAML files even if validation is disabled in config. **--version**:: Display ngcpcfg version and exit. Actions and action specific options ----------------------------------- **apply** []:: Executes the _build_, _services_ and _commit_ commands in a batch (assuming each command worked as expected). This option serves as a shortcut for the most commonly executed commands. If there are any outstanding changes that need to be committed, then the commit message needs to be provided. This is meant so the configuration change history (accessible e.g. via 'ngcpcfg log') provides useful information. **check** [--ignore-branch-check] [|]:: Check syntax of YAML files and validate schema before performing any further actions. The *--ignore-branch-check* option doesn't fail the check if the current active branch doesn't match 'master'. In the High Availability setup possibly outstanding pull actions are checked as well. **build** [--ignore-branch-check] [--modified-only] [|]:: Generate configuration files, based on values defined the central yml files and based on the templates in the configuration tree (/etc/ngcp-config/templates by default). Central yml files will be validated for integrity before building using _check_ action. The *--modified-only* option checks for _modified_ and _uncommitted_ configuration files (central yml files and templates) and builds the relevant files only. The *--ignore-branch-check* option doesn't fail the build if the current active branch doesn't match 'master'. If a central yml file is modified it builds all configuration files. If changes are in template files only just the according template files are considered for rebuild. If a file (e.g. _/etc/rsyslog.conf_) or directory (e.g. _/etc/mysql/_) is provided as argument to the _build_ option only the specified file / files inside the directory will be generated. If the argument doesn't start with the /etc prefix the argument is considered as pattern, matching all files/directories which include the specified pattern. The pattern can include shell globbing patterns - so argument 'mo..t' will match the files /etc/ngcp-monitoring-tools/collective-check.conf, /etc/monit/monitrc, /etc/ha.d/resource.d/monit-services and /etc/default/monit iff they are present. You can combine __ and __ and use multiple arguments. **clean** [--all] [--branches] [--force] [--help] [--reset-master] [--stashes] [--tracked-files] [--untracked-files]:: Clean ngcpcfg from not-yet committed/applied changes, reset git state and remove any possibly existing git stashes. It also proposes to delete unnecessary git branches if available. Note: the action (possibly) deletes some files inside /etc/ngcp-config, it should be used with care!. Option _--all_ is an alias for _--branches_ + _--reset-master_ + _--stashes_ + _--tracked-files_ + _--untracked-files_. It should restore local /etc/ngcp-config in a clean state. Note: The option _--reset-master_ is involved in the High Availability setup only. Option _--branches_ proposes to clean all the unnecessary local git branches. Active branch will be switched to 'master'. Option _--force_ answers 'yes' automatically on all the cleanup questions. Option _--help_ shows list of options for the action 'clean'. Option _--reset-master_ proposes to back up local branch 'master' and restore branch 'master' from origin. It will effectively clean all locally committed and not-yet pushed changes. Note: Already pushed changes won't be removed while they can be reverted using 'git revert '. Note: This option is available in the High Availability setup only. Option _--stashed_ proposes to clean all the changes in git stash. Option _--tracked-files_ resets cached git files and restore all tracked files to HEAD of branch master. Option _--untracked-files_ deletes all untracked files available in /etc/ngcp-config/, including files ignored by /etc/ngcp-config/.gitignore. **commit** []:: Commit all modified files in _/etc/ngcp-config_ and record changes in _/etc_ by executing the etckeeper(8) command through '/usr/share/ngcp-ngcpcfg/scripts/etckeeper'. To check whether there are any pending changes to be committed execute 'ngcpcfg status'. **decrypt**:: Decrypt /etc/ngcp-config-crypted.tgz.gpg and restore configuration files, doing the reverse operation of the _encrypt_ option. Note: This feature is only available if the ngcp-ngcpcfg-locker package is installed. **del**