* in general expand collection now performs only 1 sql
query to per expand field to fetch all items by the ids
instead of fetching them for every single collection item,
that should significantly increase performance in case
of large databases and reduce work for the database
* introduce $c->stash->{expand_cache} that contains
cached data for the expand fields to avoid multiple
same calculations
* expand_field() and expand_field_data() have been reworked
to support expand_cache
* new method expand_prepare_collection() is called for all
API GET collection methods before preparing resource hal
fields, to change the expand logic to only cache the data
instead of fetching it from the database
* new method expand_collection_fields() that is called in all
API GET collection methods after the @embedded data is prepared
to finalise the expand collection fields
* for expand collection there is only 1 SQL request per expand field
that fetches all items -in [ids to expand] and then the
expand_collection_fields() uses the cached items_by_ids with
O(1) fetch from the cache by id
Change-Id: Ie7c6115472878febf0d8c9b4d833f5c23b15c78b
* Role/Entities*: POST/PUT/PATCH/DELETE methods changes:
- support deadlock detection and transaction retry
(2 retry attempts at the moment)
- improve transaction control, use local $guard
instead of saving the ref to $c->stash, as in that
case it went out of scope too late and also reported
an error message into the log about abnormal $guard out
of scope interruption
- move all non transaction related code outside
of the scope
- add error handling when methods such as update_item,
and a like do not return the expected data, instead of
simply going out of scope and resulting in an uncontrolled
reply
* Role/API:
- rework transaction control:
+ get_transaction_control() is renamed to start_transaction()
to better reflect what it does
+ complete_transaction() is renamed to commit_transaction()
+ remove unused %params arg
+ pass $guard into commit_transaction() instead of having it
stored as $c->stash->{transaction_guard) that caused the $guard
ref to be destroyed much late than expected
(there was also a typo as transaction_quard,
which is not relevant anymore with the changes
+ add check_deadlock() that is invoked when an exception is caught
or an $c->errors contain an error, and if the error message
represents a transaction error, the transaction block is
re-invoked via "goto TX_START"
- rework error():
+ it now accepts args as following:
($self, $c, $code, $message, @errors)
# code -> returned as HTTP code in the reply
# message -> returned as HTTP message in the reply
# errors -> contain errors for internal logging, last element often contains a DBIx exception
+ populates all @errors into $c->error so they are available on
demend in the code via $c->error or $c->last_error
+ $c->log->error is not invoked now as the errors become printed
in log_response()
- log_response() now prints collected errors from $c->error correctly
as a separate log line, that is alike to the other api logs so that
those can be looked up by the request's tx_id, also all errors are
now printed only into api.log
* Adjust all $self->error() calls in catch($e) to include $e as the last
argument, as well as the duplicate $c->log->error is removed from
those ocassions
* Remove all $c->log->error() calls as they are replaced with either
$self->error() (that logs it correctly into api.log) or
$c->error('err') that also adds it correctly into api.log
* API::CallForwards: rework to use Entities/EntitiesItem
* API::Contracts: rework POST to use Entities
* API::PeeringGroups: rework POST to use Entities
* API::SubscriberRegistrations: rework POST to use Entities
* API::RewriteRuleSets: improve create_item() functionality
* Utils/Message: add 'api_retry' log type
* $c->session->{api_request_tx_id} is changed to
$c->stash->{api_request_tx_id} because sometimes the session
ref is different and a different tx_id becomes used
Change-Id: I633ce7a8047b1bf00a2f6889003088edf0825dcd
- all standard LIKE search are migrated
- will avoid LIKE unless a pattern (* wildcard) is
used as a search term. this encourage db index
usage, will be faster
- supports wildcard escape sequence \\*
- harmonize swagger UI descriptions of filters
Change-Id: Iea155871c9be6c284e6970a562d4e6af73fedc4b
* filtering by package_id with an incorrect search db query (non-existing
field) caused a 500 error
* package_id filter is renamed into profile_package_id to correspond
to the response field
* fix profile_package_id db search query
Change-Id: Ia93e0662172dd89e85572717cbb99a2ed51eb3a7
Throw descriptive error for attempt to create Customer with Template
that does not belong to Contact’s Reseller. Cover the case when the
Template belongs to System Contact (with no Reseller).
Error example:
'subscriber_email_template_id' with value '1' does not belong to
Reseller '1' that is assigned to Customer's Contact '1'
Change-Id: Iffcef0339afc4490ecba81d4667cbb9225766af4
complete the renaming of "sub rowlock_contract" into
"sub acquire_contract_rowlocks" to distinguish it from
"sub recursevly_lock_contract" (which is related to set
the subscriber lock level, not db row write locks)
Change-Id: I287ee611e20c71a90121007511c3781359968bd2
acquire the billing.contract row lock *before* any
unordered billing.voip_numbers rowlocks by
sub manage_subscriber_numbers().
- "deadlock" waittimeout errors will cease when
creating subscribers concurrently via api
- max_subscribers, is_pilot and other per-contract
constraints will be respected accurately
Change-Id: I73bb7525b327bbb09217b790be9c14cc65ddebcc
* ccareadmin and ccare roles have full access to
Customers, Subscribers and their preferences/settings,
and read-only access to BillingProfiles,InvoceTemplates,
EmailTemplates
* ccare role is restricted to the related reseller
Change-Id: I6cf7d3adf912f0fa98d1ef5c02abea2f4331ec4b
* colelction_nav_links() should be used instead of the old
self/prev/next href link code
* Entities.pm fix colletion_nav_links() to pass missing
$c->request->params
* collection_nav_links() use default $c->request->path and
$c->request->params if those arguments are not specified
Change-Id: Icca5afac812f2e06e6927ba6f4e91057f9745d21
with the previous commit, there is no need anymore to specify form
exceptions manually. validate_form will now automatically consider fields
in the correct format with and without "_id"
Change-Id: I70afae4003c6407c7a37396630110736f1fb13f8
This patch reuses existing forms by clearing them, rather than
re-instantiating them again and again.
Also, panel start time should be better due to less package
pre-loading.
Change-Id: Ia3e64fd4b4084bb5ec35a669c5840c9fc3c58f2e
the factory Catalyst::ActionRole::HTTPMethods action role
is set for any action, and in race with our own
NGCP::Panel::Role::HTTPMethods. the testcase could have
been misleading ever since, as it uses DELETE /api/customers,
which was present when implemting the method override initially,
but dropped on the road to 4.5.
the issue is resolved by overriding
Catalyst::Controller::gather_default_action_roles.
until the class hierachy refactoring is completed, only
consistent way is to add the override to each api controller.
the action_roles config attribute can therefore be
removed.
using Plack::Middleware::MethodOverride as an alternative from
upstream has the drawback to work only with POST. but a required
usecase is to also support translation DELETE requests to PATCH.
Change-Id: I204ba59869a8327bdd5db8a867fbbb061d1c9e7c
+1. terminate if terminated contracts or subscribers exists
+2. if active contracts or subscribers -> error
+3. delete otherwise
+customer contact api DELETE
+customer contact testcases
+customer api POST
+customer api PUT/PATCH
+system contact api DELETE
+system contact testcases
+contract api POST
+contract api PUT/PATCH
+panel UI contact delete
+panel UI contract create
no change panel UI contract update
+panel UI customer create
no change panel UI customer update
note contact links like customers.omm_contact or customers.tech_contact
are not considered.
Change-Id: I19ea6e2245282485cc30d3e92d17a2eb603e0c2d
which implies, among others, strict and warnings. these should never
be omitted and are one of the most frequent complaints of perlcritic
currently.
as a result TryCatch can be removed (imported by Sipwise::Base).
also, "no Moose" is removed (has no effect).
Change-Id: I901b6a9f0f6d426f62b73e68f2c5ad6365c1eeef
Switch from the discouraged base module to the light-weight parent module.
Also remove some surrounding BEGIN blocks from 'use parent' which are
completely unnecessary.
Change-Id: I3a669e8024f098819be45030ca9d1afa8756105c
any *attempt* of a top-up request should be
logged.
+create top-up log records for api
in order to record failed top-up requests as well,
the somewhat tricky thing is to have two
separate transactions here
+propery casting of topup request json field
values to numbers etc., so the transaction for
creating the log entry cannot fail and e.g.
requests with subscriber_id='blah' are recorded
correctly
+new "request_token" parameter for /api/topupvoucher
and /api/topupcash, to identify and filter for
particular requests.
+topup log api resource
+topup log api tests
+fix to correctly reject used vouchers
+topup log panel UI
+fix for balanceintervals.t threaded tests
Change-Id: I86eb845f6173803705b12cc7e5cdbac9a3153a0a
in short: whenever the contract_balances "cash_balance" is
about to decrease, it might drop below thresholds defined
by a contract's profile package. in that case, subscribers
are to be locked and/or underrun billing profile sets are
to be applied. this commit also contains some minor fixes:
+when creating/editing profile packages, all billing
profiles of initial, topup, underrun sets must show the
same interval_free_cash and interval_free_time values.
there was also a problem with adding profile sets to
packages that use profile with prepaid = 1 (see ticket
14301)
+when editing packages using the panel UI, the validation
was not properly invoked (!) +the frequency of topups has
to be limited to 1 per second.
+updating the "lock" subscriber prior to editing or viewing
it in the panel UI was added.
+topup cash and topup voucher panel UI added
+when aquiring multiple contract row locks at once, no
joined tables are used any mure. the number of row locks
is therefore minimal now (see ticket 14903)
+rework of updating prepaid subscriber preference
Change-Id: Ideb0fec2404ca10b3ac5d8735f53620d0c3a24ed
+profile package details view to overlook associations
+voucher count is properly checked before deleting a package
+package_id filters for /api/customers and /api/vouchers
+Fake::Time replaced by Time::Warp for balance_interval tests
+cleanup
Change-Id: Idc34220a9d29c115453eeee2907e70a71cf9f0cc
This is a patcht introducing a variable to disable
setting profile_package_id's in in the contracts
table. The code should therefore behave as before,
as long rate-o-mat and libswrate are ready.
Not once set to 1 and contract balances are written
to a database once, disabling this constant will
mess up contract_balances!
Change-Id: I080460395627202f3f5a386022de643577f4f3c2
+ synchronized contract balance catchup
+ balance interval resizing upon profile package transitions
+ dynamic interval length, interval start, carry-over propagation, ..
+ end-of-month 'preserve' mode correction for strictly aligned month intervals for start_mode=customer "create" timestamp
+ new api/balanceintervals resource to inspect contracts' balance interval histories
+ updated affected panel UI controllers
+ test case with time warps using Time::Fake
caveats:
- creating an invoice for a given 'period' (month) has to be refactored to select a disitnct balance interval. right now it takes the last interval in the month specified.
- generate_invoice.pl is broken and needs a major refactoring
Change-Id: I7bb54a83b76e510b1baa573a986d05400a7fec1e
-properly validate billing mapping intervals
-api/contracts and api/customers complete
+the problem with updating the "prepaid" preference remains at the moment
+billing_mappings field contains future (editable) mappings only, the full schedule is given in a separate field
+hal-links to all occurring billing profiles and networks
-extended tests: api-customer.t and api-contract.t
-api-billingnetworks.t test
-billing networks panel controller
+search by enclosed IP address
-panel UIs for billing networks, customers, contracts
+past billing mapping intervals that cannot be edited are displayed by a seperate hfh "block" part of the billing_profiles repeatable
-profile packages api, UI
+terminating billing profiles (MT#8645) will be broken
+fixed now: profile cannot be terminated unless there are:
-zero non-terminated customers, whose mappings contain the profile with future or open end
-zero profile packages, that link to the profile in either one of their initial-, underrun- or topup profile sets
+panel edit should work, termination wip
+topup_profiles w/o amounts
+contracts/customers API profile_package_id field:
+apply the package's initial profiles
+api/customer and api/contract profile definition modes (id/profiles/package)
+api OK, api-customer.t tests
+customer and contract panel UI with selector to specify actual billing mapping, schedule or profile package (same as api)
+terminate logic harmonized: profiles, networks, packages
+api tests updated
+prepare for merging: cleanup, enable tests
Change-Id: I6c0dec72f490b5731fc673cd393d6aeb08dd8143
the journal module introduces a change history of
resources modified by api invocations. the history of
the 'customer' resource demo is accessible at
/api/customers/x/journal.
Change-Id: I4d5d11bc3e35160feed587ce4c1db565991866b2
Kirill Solomko
Rene Krenn
Victor Tsvetov
Irina Peshinskaya
Gerhard Jungwirth
Andreas Granig
Guillem Jover