* in general expand collection now performs only 1 sql
query to per expand field to fetch all items by the ids
instead of fetching them for every single collection item,
that should significantly increase performance in case
of large databases and reduce work for the database
* introduce $c->stash->{expand_cache} that contains
cached data for the expand fields to avoid multiple
same calculations
* expand_field() and expand_field_data() have been reworked
to support expand_cache
* new method expand_prepare_collection() is called for all
API GET collection methods before preparing resource hal
fields, to change the expand logic to only cache the data
instead of fetching it from the database
* new method expand_collection_fields() that is called in all
API GET collection methods after the @embedded data is prepared
to finalise the expand collection fields
* for expand collection there is only 1 SQL request per expand field
that fetches all items -in [ids to expand] and then the
expand_collection_fields() uses the cached items_by_ids with
O(1) fetch from the cache by id
Change-Id: Ie7c6115472878febf0d8c9b4d833f5c23b15c78b
* Role/Entities*: POST/PUT/PATCH/DELETE methods changes:
- support deadlock detection and transaction retry
(2 retry attempts at the moment)
- improve transaction control, use local $guard
instead of saving the ref to $c->stash, as in that
case it went out of scope too late and also reported
an error message into the log about abnormal $guard out
of scope interruption
- move all non transaction related code outside
of the scope
- add error handling when methods such as update_item,
and a like do not return the expected data, instead of
simply going out of scope and resulting in an uncontrolled
reply
* Role/API:
- rework transaction control:
+ get_transaction_control() is renamed to start_transaction()
to better reflect what it does
+ complete_transaction() is renamed to commit_transaction()
+ remove unused %params arg
+ pass $guard into commit_transaction() instead of having it
stored as $c->stash->{transaction_guard) that caused the $guard
ref to be destroyed much late than expected
(there was also a typo as transaction_quard,
which is not relevant anymore with the changes
+ add check_deadlock() that is invoked when an exception is caught
or an $c->errors contain an error, and if the error message
represents a transaction error, the transaction block is
re-invoked via "goto TX_START"
- rework error():
+ it now accepts args as following:
($self, $c, $code, $message, @errors)
# code -> returned as HTTP code in the reply
# message -> returned as HTTP message in the reply
# errors -> contain errors for internal logging, last element often contains a DBIx exception
+ populates all @errors into $c->error so they are available on
demend in the code via $c->error or $c->last_error
+ $c->log->error is not invoked now as the errors become printed
in log_response()
- log_response() now prints collected errors from $c->error correctly
as a separate log line, that is alike to the other api logs so that
those can be looked up by the request's tx_id, also all errors are
now printed only into api.log
* Adjust all $self->error() calls in catch($e) to include $e as the last
argument, as well as the duplicate $c->log->error is removed from
those ocassions
* Remove all $c->log->error() calls as they are replaced with either
$self->error() (that logs it correctly into api.log) or
$c->error('err') that also adds it correctly into api.log
* API::CallForwards: rework to use Entities/EntitiesItem
* API::Contracts: rework POST to use Entities
* API::PeeringGroups: rework POST to use Entities
* API::SubscriberRegistrations: rework POST to use Entities
* API::RewriteRuleSets: improve create_item() functionality
* Utils/Message: add 'api_retry' log type
* $c->session->{api_request_tx_id} is changed to
$c->stash->{api_request_tx_id} because sometimes the session
ref is different and a different tx_id becomes used
Change-Id: I633ce7a8047b1bf00a2f6889003088edf0825dcd
* new c.users.role 'lintercept', that set to when an admin user has
enabled 'lawful_intercept' flag
* only Administrator page /api/admins and /api/interceptions are available for
the role
* 'lintercept' role can only see own user and only change password
and email
Change-Id: Iadcb022a124afbd77b224e734026f380af0170e8
adds gdpr obfuscation quoting for:
+ subscriber numbers
+ subscriber ip addresses
+ subscriber usernames
+ any logmessage "DATA": query parameters, form data, response data
+ subscriber uuid's
+ call id's
+ callforward sip uri's
the quoting is centralized by $c->qs() ("quote sensitive"), using
catalyst plugin mechanism.
escape symbols are set to « (\x{ab}) and » (\x{bb}).
generate_logfile_data_inventory.pl was modified to mark loglines
with "gdpr affected" status, if $c->qs() was used in a log message.
Change-Id: I0f42d7992594232ae33e5666b0a64009211c5b76
Revert "TT#11701 check existence of voip number in InterceptionDB"
This reverts commit b803c1ae3e.
Change-Id: I56832cf412a97c4f0fc9f169498a60653d979614
* colelction_nav_links() should be used instead of the old
self/prev/next href link code
* Entities.pm fix colletion_nav_links() to pass missing
$c->request->params
* collection_nav_links() use default $c->request->path and
$c->request->params if those arguments are not specified
Change-Id: Icca5afac812f2e06e6927ba6f4e91057f9745d21
* Make sure resellers with interception rights can query both
collections and items
* Make sure we only query the reseller specific interceptions
when accessing API as reseller
Change-Id: I3010f899cbbd1a7a4f60025d95f09f013cc07c03
This patch reuses existing forms by clearing them, rather than
re-instantiating them again and again.
Also, panel start time should be better due to less package
pre-loading.
Change-Id: Ia3e64fd4b4084bb5ec35a669c5840c9fc3c58f2e
the factory Catalyst::ActionRole::HTTPMethods action role
is set for any action, and in race with our own
NGCP::Panel::Role::HTTPMethods. the testcase could have
been misleading ever since, as it uses DELETE /api/customers,
which was present when implemting the method override initially,
but dropped on the road to 4.5.
the issue is resolved by overriding
Catalyst::Controller::gather_default_action_roles.
until the class hierachy refactoring is completed, only
consistent way is to add the override to each api controller.
the action_roles config attribute can therefore be
removed.
using Plack::Middleware::MethodOverride as an alternative from
upstream has the drawback to work only with POST. but a required
usecase is to also support translation DELETE requests to PATCH.
Change-Id: I204ba59869a8327bdd5db8a867fbbb061d1c9e7c
Also call agent request in the update_item method, to make it more safe.
And added lost transaction commit
Fix 500 response for the incorret x3_port input
Change-Id: Ic538ffdd4d7fff833ba5395934a1b2bc6cf71664
which implies, among others, strict and warnings. these should never
be omitted and are one of the most frequent complaints of perlcritic
currently.
as a result TryCatch can be removed (imported by Sipwise::Base).
also, "no Moose" is removed (has no effect).
Change-Id: I901b6a9f0f6d426f62b73e68f2c5ad6365c1eeef
Switch from the discouraged base module to the light-weight parent module.
Also remove some surrounding BEGIN blocks from 'use parent' which are
completely unnecessary.
Change-Id: I3a669e8024f098819be45030ca9d1afa8756105c
by choosing "stable" (about 1m30s) or "fast" (about 30s) as second
argument of testrunner, only tests are run, which are currently
succeeding on a plain trunk vm using docker. (times are on my machine)
I hope we can in this way re-enable them on nightlies (or even in gerrit)
so that devs have some better feedback on the impact of our changes.
Current status of tests (in trunk):
| Problems:
| api-callforwards.t
| api-balanceintervals.t
| api-faxes.t
| api-journals.t
|
| Fixed:
| api-all-links.t
| api-contracts.t
| api-customercontacts.t
| api-customers.t
| api-rewriterulesets.t
| api-root.t
|
| Working:
| api-billingfees.t
| api-billingnetworks.t
| api-billingprofiles.t
| api-billingzones.t
| api-calllists.t
| api-calls.t
| api-cert-auth.t
| api-cfdestinationsets.t
| api-ncoslevels.t
| api-pbxdevicemodels.t
| api-pbxdevices.t
| api-peeringgroups.t
| api-peeringrules.t
| api-peeringservers.t
| api-preferences.t
| api-profilepackages.t
| api-resellers.t
| api-rewriterules.t
| api-soundsets.t
| api-subscriberregistrations.t
| api-subscribers.t
| api-systemcontacts.t
| api-threads.t
| api-topuplogs.t
| api-trustedsources.t
| api-valid-patch.t
| api-vouchers.t
Change-Id: I74e672d9dc6bcccfd3ace2fe233dbcf325c6fc4d
On X1 requests, properly update the configured agents on the cluster.
Currently we only throw an error if no agents are configured at all,
but let it slip through if the actual requests are failing, as they
might be down or failing.
Change-Id: I0f4e021a5cc4ba6a30e30bb197ed20d4504797d8
Kirill Solomko
Rene Krenn
Irina Peshinskaya
Gerhard Jungwirth
Andreas Granig
Guillem Jover
Richard Fuchs