4 Commits (055ec2f767e6c51138d68f61e09b0f3c239579e1)

Author	SHA1	Message	Date
Guillem Jover	a9166843c2	TT#71952 Parallelize templated output generation ... Switch away from the old daemon/wrapper architecture, which complicates things and requires more scaffolding code. We turn the daemon/wrapper and build_config into a single perl process that will load all YAML files, and then process each input/output pair on a parallel child, to try to speed up the processing as much as possible. For whole rebuilds, it might speed up the generation by at least a factor of x2, x4 or more, depending on the number of active processors. Change-Id: I51aa2f90336e34a20983d8733f45b64d9b6fea0b	5 years ago
Alexander Lutay	ead10ac2ad	TT#65907 Speedup 'ngcpcfg apply' on 1.5 seconds (re-use the same tmp file) ... In the past helper/build_config generated new tmp file for every call and we called it for hundred+ times. Let's generate it once and pass to helper/build_config to be reused every time. Command: time ngcpcfg apply "test" Old code: real 0m17.194s New code: real 0m15.477s Change-Id: Ic1c35ccb5c5f92ddee6328502194a5301123bd9d	6 years ago
Michael Prokop	8005eea3e5	TT#46601 tests: avoid DeprecationWarnings with Python >=3.6 ... There's no need to escape '.' characters within raw unicode regex strings. Starting with Python 3.6 this raises warnings: | test_ngcpcfg_build_syntax.py:30 | /build/source/t/test_ngcpcfg_build_syntax.py:30: DeprecationWarning: invalid escape sequence \. | "encoding.*etc/bad-syntax\.txt\.tt2.*or inspecting " | | test_ngcpcfg_build_syntax.py:31 | /build/source/t/test_ngcpcfg_build_syntax.py:31: DeprecationWarning: invalid escape sequence \. | "temporary /tmp/ngcpcfg\.bad-syntax\.txt\.") | | test_ngcpcfg_build_syntax.py:36 | /build/source/t/test_ngcpcfg_build_syntax.py:36: DeprecationWarning: invalid escape sequence \. | "/etc/bad-syntax\.txt\.tt2: FAILED") Verified against Debian/buster (Python 3.7.2 + pytest-3.10.1) and Debian/stretch (Python 3.5.3 + pytest-3.0.6). Also see https://bugs.python.org/issue27364 Change-Id: If95482497534283a1cd4f4d087f61d1d18216628	6 years ago
Manuel Montecelo	a529f8e0b0	TT#46601 Detect and report errors parsing tt2 templates ... Due to the code in tt2-wrapper, the only errors reported were failures to connect to the socket to use the back-end performing the parsing. But there was no attempt to detect errors in the parsing at all. Since we are only able to use the output stream and not for example exit codes of the parser at the back-end (libtemplate-perl), we can only rely on finding some kind of error string, which can change in future versions of libtemplate-perl but hopefully not very often, and at most it should only change when we move base to newer Debian releases. In the case of the original problem reported we could rely on "parse error", but since there are two such strings ("file error" and "parse error") the first is more general and also covers cases like "permission denied to read file" or other errors that might happen. So the implementation of error checking is basically grepping for "^file error", the string at the beginning of a line, to limit possible false positives. With the new checks and an example of an invalid syntax already in the first line, "ngcpcfg build" detects the error correctly and stops processing, and shows information about the source of the problem, e.g.: root@spce:~# ngcpcfg build /etc/test 2018-10-29 11:37:01 spce: yml configs were validated successfuly 2018-10-29 11:37:01 spce: configs were checked successfuly 2018-10-29 11:37:01 spce: No patchtt files found, nothing to patch. Loading /etc/ngcp-config/config.yml in memory: OK Loading /etc/ngcp-config/network.yml in memory: OK Loading /etc/ngcp-config/constants.yml in memory: OK 2018-10-29 11:37:02 spce: Error: Generating /etc/test based on /etc/ngcp-config/templates//etc/test.customtt.tt2: FAILED 2018-10-29 11:37:02 spce: Error: from generated file: 2018-10-29 11:37:02 spce: Error: file error - parse error - input file handle line 1: unexpected end of directive 2018-10-29 11:37:02 spce: NOTE: Check those files for valid syntax and encoding: 2018-10-29 11:37:02 spce: /etc/ngcp-config/templates//etc/test.customtt.tt2 2018-10-29 11:37:02 spce: /etc/ngcp-config/config.yml 2018-10-29 11:37:02 spce: /etc/ngcp-config/network.yml 2018-10-29 11:37:02 spce: /etc/ngcp-config/constants.yml 2018-10-29 11:37:02 spce: Running /usr/share/ngcp-ngcpcfg/helper/tt2-wrapper <file> 2018-10-29 11:37:02 spce: or inspecting temporary /tmp/ngcpcfg.test.PwGvShIm9G 2018-10-29 11:37:02 spce: should provide more details. Change-Id: Ic305bdab20a6ce15eca13f19586a2572a90b4e13	7 years ago