ngcpcfg

Commit Graph

Author	SHA1	Message	Date
Alexander Lutay	ead10ac2ad	TT#65907 Speedup 'ngcpcfg apply' on 1.5 seconds (re-use the same tmp file) In the past helper/build_config generated new tmp file for every call and we called it for hundred+ times. Let's generate it once and pass to helper/build_config to be reused every time. Command: time ngcpcfg apply "test" Old code: real 0m17.194s New code: real 0m15.477s Change-Id: Ic1c35ccb5c5f92ddee6328502194a5301123bd9d	7 years ago
Michael Prokop	8005eea3e5	TT#46601 tests: avoid DeprecationWarnings with Python >=3.6 There's no need to escape '.' characters within raw unicode regex strings. Starting with Python 3.6 this raises warnings: \| test_ngcpcfg_build_syntax.py:30 \| /build/source/t/test_ngcpcfg_build_syntax.py:30: DeprecationWarning: invalid escape sequence \. \| "encoding.etc/bad-syntax\.txt\.tt2.or inspecting " \| \| test_ngcpcfg_build_syntax.py:31 \| /build/source/t/test_ngcpcfg_build_syntax.py:31: DeprecationWarning: invalid escape sequence \. \| "temporary /tmp/ngcpcfg\.bad-syntax\.txt\.") \| \| test_ngcpcfg_build_syntax.py:36 \| /build/source/t/test_ngcpcfg_build_syntax.py:36: DeprecationWarning: invalid escape sequence \. \| "/etc/bad-syntax\.txt\.tt2: FAILED") Verified against Debian/buster (Python 3.7.2 + pytest-3.10.1) and Debian/stretch (Python 3.5.3 + pytest-3.0.6). Also see https://bugs.python.org/issue27364 Change-Id: If95482497534283a1cd4f4d087f61d1d18216628	7 years ago
Manuel Montecelo	a529f8e0b0	TT#46601 Detect and report errors parsing tt2 templates Due to the code in tt2-wrapper, the only errors reported were failures to connect to the socket to use the back-end performing the parsing. But there was no attempt to detect errors in the parsing at all. Since we are only able to use the output stream and not for example exit codes of the parser at the back-end (libtemplate-perl), we can only rely on finding some kind of error string, which can change in future versions of libtemplate-perl but hopefully not very often, and at most it should only change when we move base to newer Debian releases. In the case of the original problem reported we could rely on "parse error", but since there are two such strings ("file error" and "parse error") the first is more general and also covers cases like "permission denied to read file" or other errors that might happen. So the implementation of error checking is basically grepping for "^file error", the string at the beginning of a line, to limit possible false positives. With the new checks and an example of an invalid syntax already in the first line, "ngcpcfg build" detects the error correctly and stops processing, and shows information about the source of the problem, e.g.: root@spce:~# ngcpcfg build /etc/test 2018-10-29 11:37:01 spce: yml configs were validated successfuly 2018-10-29 11:37:01 spce: configs were checked successfuly 2018-10-29 11:37:01 spce: No patchtt files found, nothing to patch. Loading /etc/ngcp-config/config.yml in memory: OK Loading /etc/ngcp-config/network.yml in memory: OK Loading /etc/ngcp-config/constants.yml in memory: OK 2018-10-29 11:37:02 spce: Error: Generating /etc/test based on /etc/ngcp-config/templates//etc/test.customtt.tt2: FAILED 2018-10-29 11:37:02 spce: Error: from generated file: 2018-10-29 11:37:02 spce: Error: file error - parse error - input file handle line 1: unexpected end of directive 2018-10-29 11:37:02 spce: NOTE: Check those files for valid syntax and encoding: 2018-10-29 11:37:02 spce: /etc/ngcp-config/templates//etc/test.customtt.tt2 2018-10-29 11:37:02 spce: /etc/ngcp-config/config.yml 2018-10-29 11:37:02 spce: /etc/ngcp-config/network.yml 2018-10-29 11:37:02 spce: /etc/ngcp-config/constants.yml 2018-10-29 11:37:02 spce: Running /usr/share/ngcp-ngcpcfg/helper/tt2-wrapper <file> 2018-10-29 11:37:02 spce: or inspecting temporary /tmp/ngcpcfg.test.PwGvShIm9G 2018-10-29 11:37:02 spce: should provide more details. Change-Id: Ic305bdab20a6ce15eca13f19586a2572a90b4e13	8 years ago

Author

SHA1

Message

Date

Alexander Lutay

ead10ac2ad

TT#65907 Speedup 'ngcpcfg apply' on 1.5 seconds (re-use the same tmp file)

In the past helper/build_config generated new tmp file for every call
and we called it for hundred+ times. Let's generate it once and pass
to helper/build_config to be reused every time.

Command: time ngcpcfg apply "test"
Old code: real   0m17.194s
New code: real   0m15.477s

Change-Id: Ic1c35ccb5c5f92ddee6328502194a5301123bd9d

Michael Prokop

8005eea3e5

TT#46601 tests: avoid DeprecationWarnings with Python >=3.6

There's no need to escape '.' characters within raw unicode regex strings.
Starting with Python 3.6 this raises warnings:

| test_ngcpcfg_build_syntax.py:30
|   /build/source/t/test_ngcpcfg_build_syntax.py:30: DeprecationWarning: invalid escape sequence \.
|     "encoding.*etc/bad-syntax\.txt\.tt2.*or inspecting "
|
| test_ngcpcfg_build_syntax.py:31
|   /build/source/t/test_ngcpcfg_build_syntax.py:31: DeprecationWarning: invalid escape sequence \.
|     "temporary /tmp/ngcpcfg\.bad-syntax\.txt\.")
|
| test_ngcpcfg_build_syntax.py:36
|   /build/source/t/test_ngcpcfg_build_syntax.py:36: DeprecationWarning: invalid escape sequence \.
|     "/etc/bad-syntax\.txt\.tt2: FAILED")

Verified against Debian/buster (Python 3.7.2 + pytest-3.10.1)
and Debian/stretch (Python 3.5.3 + pytest-3.0.6).

Also see https://bugs.python.org/issue27364

Change-Id: If95482497534283a1cd4f4d087f61d1d18216628

Manuel Montecelo

a529f8e0b0

TT#46601 Detect and report errors parsing tt2 templates

Due to the code in tt2-wrapper, the only errors reported were failures to
connect to the socket to use the back-end performing the parsing.  But there was
no attempt to detect errors in the parsing at all.

Since we are only able to use the output stream and not for example exit codes
of the parser at the back-end (libtemplate-perl), we can only rely on finding
some kind of error string, which can change in future versions of
libtemplate-perl but hopefully not very often, and at most it should only change
when we move base to newer Debian releases.

In the case of the original problem reported we could rely on "parse error", but
since there are two such strings ("file error" and "parse error") the first is
more general and also covers cases like "permission denied to read file" or
other errors that might happen.  So the implementation of error checking is
basically grepping for "^file error", the string at the beginning of a line, to
limit possible false positives.

With the new checks and an example of an invalid syntax already in the first
line, "ngcpcfg build" detects the error correctly and stops processing, and
shows information about the source of the problem, e.g.:

  root@spce:~# ngcpcfg build /etc/test
  2018-10-29 11:37:01 spce: yml configs were validated successfuly
  2018-10-29 11:37:01 spce: configs were checked successfuly
  2018-10-29 11:37:01 spce: No patchtt files found, nothing to patch.
  Loading /etc/ngcp-config/config.yml in memory: OK
  Loading /etc/ngcp-config/network.yml in memory: OK
  Loading /etc/ngcp-config/constants.yml in memory: OK
  2018-10-29 11:37:02 spce: Error: Generating /etc/test based on /etc/ngcp-config/templates//etc/test.customtt.tt2: FAILED
  2018-10-29 11:37:02 spce: Error: from generated file:
  2018-10-29 11:37:02 spce: Error:   file error - parse error - input file handle line 1: unexpected end of directive
  2018-10-29 11:37:02 spce: NOTE: Check those files for valid syntax and encoding:
  2018-10-29 11:37:02 spce: /etc/ngcp-config/templates//etc/test.customtt.tt2
  2018-10-29 11:37:02 spce: /etc/ngcp-config/config.yml
  2018-10-29 11:37:02 spce: /etc/ngcp-config/network.yml
  2018-10-29 11:37:02 spce: /etc/ngcp-config/constants.yml
  2018-10-29 11:37:02 spce: Running /usr/share/ngcp-ngcpcfg/helper/tt2-wrapper <file>
  2018-10-29 11:37:02 spce: or inspecting temporary /tmp/ngcpcfg.test.PwGvShIm9G
  2018-10-29 11:37:02 spce: should provide more details.

Change-Id: Ic305bdab20a6ce15eca13f19586a2572a90b4e13

3 Commits (606f0fb3eb0af6d9056a9739bf3da04d5bd975b9)