70 Commits (7f8d6eacd36faae303c5e5dedd66438e388c5574)

Author	SHA1	Message	Date
Rene Krenn	764ab7d2d1	MT#62283 exclude "for update no lock" in tests ... Change-Id: I12ff4d1f8ba0b150a1918de40dafa1efd6a92c10	2 months ago
Rene Krenn	82af5cf518	MT#62283 "for update no lock" logic for GET /subscribers and /contracts ... to unblock the web UI on heavy loaded systems with call rating enabled, "select .. for update no lock" will no longer wait for acquiring locks when retrieving subscriber or contract contract pages. the contract_balance record creation is will be skipped for such contracts that are locked elsewhere. for rails such as /customerbalances, an explicit contract id row lock timeout is introduced, so it does not depend on the mariadb wait_timeout. Change-Id: I6e96342f3f761279a5876c871d2477977823a39e	2 months ago
Kirill Solomko	6543743992	MT#60162 improve expand functionality for collections ... * in general expand collection now performs only 1 sql query to per expand field to fetch all items by the ids instead of fetching them for every single collection item, that should significantly increase performance in case of large databases and reduce work for the database * introduce $c->stash->{expand_cache} that contains cached data for the expand fields to avoid multiple same calculations * expand_field() and expand_field_data() have been reworked to support expand_cache * new method expand_prepare_collection() is called for all API GET collection methods before preparing resource hal fields, to change the expand logic to only cache the data instead of fetching it from the database * new method expand_collection_fields() that is called in all API GET collection methods after the @embedded data is prepared to finalise the expand collection fields * for expand collection there is only 1 SQL request per expand field that fetches all items -in [ids to expand] and then the expand_collection_fields() uses the cached items_by_ids with O(1) fetch from the cache by id Change-Id: Ie7c6115472878febf0d8c9b4d833f5c23b15c78b	11 months ago
Kirill Solomko	205b27a267	MT#59478 API better transaction and error handling ... * Role/Entities*: POST/PUT/PATCH/DELETE methods changes: - support deadlock detection and transaction retry (2 retry attempts at the moment) - improve transaction control, use local $guard instead of saving the ref to $c->stash, as in that case it went out of scope too late and also reported an error message into the log about abnormal $guard out of scope interruption - move all non transaction related code outside of the scope - add error handling when methods such as update_item, and a like do not return the expected data, instead of simply going out of scope and resulting in an uncontrolled reply * Role/API: - rework transaction control: + get_transaction_control() is renamed to start_transaction() to better reflect what it does + complete_transaction() is renamed to commit_transaction() + remove unused %params arg + pass $guard into commit_transaction() instead of having it stored as $c->stash->{transaction_guard) that caused the $guard ref to be destroyed much late than expected (there was also a typo as transaction_quard, which is not relevant anymore with the changes + add check_deadlock() that is invoked when an exception is caught or an $c->errors contain an error, and if the error message represents a transaction error, the transaction block is re-invoked via "goto TX_START" - rework error(): + it now accepts args as following: ($self, $c, $code, $message, @errors) # code -> returned as HTTP code in the reply # message -> returned as HTTP message in the reply # errors -> contain errors for internal logging, last element often contains a DBIx exception + populates all @errors into $c->error so they are available on demend in the code via $c->error or $c->last_error + $c->log->error is not invoked now as the errors become printed in log_response() - log_response() now prints collected errors from $c->error correctly as a separate log line, that is alike to the other api logs so that those can be looked up by the request's tx_id, also all errors are now printed only into api.log * Adjust all $self->error() calls in catch($e) to include $e as the last argument, as well as the duplicate $c->log->error is removed from those ocassions * Remove all $c->log->error() calls as they are replaced with either $self->error() (that logs it correctly into api.log) or $c->error('err') that also adds it correctly into api.log * API::CallForwards: rework to use Entities/EntitiesItem * API::Contracts: rework POST to use Entities * API::PeeringGroups: rework POST to use Entities * API::SubscriberRegistrations: rework POST to use Entities * API::RewriteRuleSets: improve create_item() functionality * Utils/Message: add 'api_retry' log type * $c->session->{api_request_tx_id} is changed to $c->stash->{api_request_tx_id} because sometimes the session ref is different and a different tx_id becomes used Change-Id: I633ce7a8047b1bf00a2f6889003088edf0825dcd	1 year ago
Kirill Solomko	1dabb6051d	MT#58989 /api/customers add type query param ... * add 'type' query param to filter by customer type 'sipaccount' or 'pbxaccount' Change-Id: If372cc69cf16c92551824504da7695a65dc867b3	1 year ago
Rene Krenn	308a974059	MT#58762 harmonize restapi wildcard filters ... - all standard LIKE search are migrated - will avoid LIKE unless a pattern (* wildcard) is used as a search term. this encourage db index usage, will be faster - supports wildcard escape sequence \\* - harmonize swagger UI descriptions of filters Change-Id: Iea155871c9be6c284e6970a562d4e6af73fedc4b	1 year ago
Kirill Solomko	1aef2d5d32	MT#58581 /api/customers fix filter by profile_package_id ... * filtering by package_id with an incorrect search db query (non-existing field) caused a 500 error * package_id filter is renamed into profile_package_id to correspond to the response field * fix profile_package_id db search query Change-Id: Ia93e0662172dd89e85572717cbb99a2ed51eb3a7	2 years ago
Rene Krenn	e9367e0cfe	MT#55972 support wildcard search for customer external_id ... Change-Id: I8287f7650f68dab2e806259ec00ffad2ba45fe1a	2 years ago
Victor Tsvetov	3cb71e4ceb	TT#130456 API Customers POST: show error if system template ... Throw descriptive error for attempt to create Customer with Template that does not belong to Contact’s Reseller. Cover the case when the Template belongs to System Contact (with no Reseller). Error example: 'subscriber_email_template_id' with value '1' does not belong to Reseller '1' that is assigned to Customer's Contact '1' Change-Id: Iffcef0339afc4490ecba81d4667cbb9225766af4	4 years ago
Rene Krenn	f9690f08f6	TT#77452 rename rowlock_contract ... complete the renaming of "sub rowlock_contract" into "sub acquire_contract_rowlocks" to distinguish it from "sub recursevly_lock_contract" (which is related to set the subscriber lock level, not db row write locks) Change-Id: I287ee611e20c71a90121007511c3781359968bd2	5 years ago
Rene Krenn	a11f2bed5e	TT#77452 properly synchronize subscriber create/update ... acquire the billing.contract row lock *before* any unordered billing.voip_numbers rowlocks by sub manage_subscriber_numbers(). - "deadlock" waittimeout errors will cease when creating subscribers concurrently via api - max_subscribers, is_pilot and other per-contract constraints will be respected accurately Change-Id: I73bb7525b327bbb09217b790be9c14cc65ddebcc	5 years ago
Kirill Solomko	ce664263b2	TT#65101 add ccareadmin ccare roles ... * ccareadmin and ccare roles have full access to Customers, Subscribers and their preferences/settings, and read-only access to BillingProfiles,InvoceTemplates, EmailTemplates * ccare role is restricted to the related reseller Change-Id: I6cf7d3adf912f0fa98d1ef5c02abea2f4331ec4b	6 years ago
Irina Peshinskaya	1fe6db943d	TT#48198 Use items array from paginations in all api controllers ... Change-Id: I1856cd4f073d32b4465d0778baf5f9f7c708f78c	6 years ago
Rene Krenn	7b5c9d9fa6	TT#44095 set isolation level when creating contracts ... Change-Id: Ie7944be85ea60626050896988f49917249ccc3f5	7 years ago
Rene Krenn	145167e3db	TT#38401 switch to new billing mappings tables ... Change-Id: I4e57b50a5668170c66521ed5f130bd8293b76b38	7 years ago
Rene Krenn	b35b0e1cbc	TT#35662 move product_id to billing.contracts ... Change-Id: I5381b36e226b1c0c2032fa40c3a082b85485e4ac	7 years ago
Kirill Solomko	f976fe03fb	TT#35714 use collection_nav_links in all API controllers ... * colelction_nav_links() should be used instead of the old self/prev/next href link code * Entities.pm fix colletion_nav_links() to pass missing $c->request->params * collection_nav_links() use default $c->request->path and $c->request->params if those arguments are not specified Change-Id: Icca5afac812f2e06e6927ba6f4e91057f9745d21	7 years ago
Irina Peshinskaya	05c2708832	TT#34567 Apply mandatory parameters to hal links ... Change-Id: I90dd7f7ae600ccfa589a7624681f35655ae9a098	7 years ago
Irina Peshinskaya	2b8ae95170	TT#28460 Remove unused libraries ... require Catalyst::ActionRole::ACL; require Catalyst::ActionRole::CheckTrailingSlash; require NGCP::Panel::Role::HTTPMethods; require Catalyst::ActionRole::RequireSSL; Change-Id: I3880fcb10067810d5e2de10a0c14bf6b15e34710	7 years ago
Irina Peshinskaya	0d4d8cdb23	TT#28460 Format: Remove excrescent empty lines ... Change-Id: I5ff73d92bf62451fb6dbdae85ca07428c3f5af5d	7 years ago
Irina Peshinskaya	75d6edbcd9	TT#28460 Format: add line between subs in API controllers ... Change-Id: I67f11fef716941dc965dffd4e8fbb1a211791bbf	7 years ago
Irina Peshinskaya	9fa2a19b4f	TT#28460 Replace configs - not journalized ... Change-Id: I9843a780817922f23faa91041d3fab9b7336b2a3	7 years ago
Irina Peshinskaya	92e6215b71	TT#28460 Remove gather_default_action_roles ... Change-Id: Ie8080a9ac3b5b1004de456fe0ece7b31ed1541fe	7 years ago
Irina Peshinskaya	2498c8810e	TT#28460 Remove copy-pasted sub HEAD and sub end ... Change-Id: Ia06320ead891945eebe715fa34012a14d5529f17	7 years ago
Irina Peshinskaya	05774c2338	TT#28460 Remove copy-pasted sub OPTIONS ... Change-Id: I2b00ea267edcc3caa00eaac5753638794a13398c	7 years ago
Irina Peshinskaya	a113cbf1d0	TT#28460 Remove copy-pasted sub auto ... Without config changes - all custom subs are in place Change-Id: I23e50e7d1fcb02c245bc995a26e16e97308177c8	7 years ago
Irina Peshinskaya	b14f34f008	TT#28460 Change parents in API controllers ... Use Entities and EntitiesItem respectively Change-Id: I5d32b010cc51810acee42d14c5c24ce3d4267602	7 years ago
Gerhard Jungwirth	24de4232e6	TT#30507 remove DataHal workaround layer ... (NGCP::Panel was loaded every time this was used) Change-Id: I62861e0756aaaf548c9be42b4842e9a06127119c	7 years ago
Gerhard Jungwirth	edbef3c5b8	TT#29617 remove unnecessary form exceptions ... with the previous commit, there is no need anymore to specify form exceptions manually. validate_form will now automatically consider fields in the correct format with and without "_id" Change-Id: I70afae4003c6407c7a37396630110736f1fb13f8	7 years ago
Andreas Granig	0d3bf6b1ce	TT#22827 Implement form factory ... This patch reuses existing forms by clearing them, rather than re-instantiating them again and again. Also, panel start time should be better due to less package pre-loading. Change-Id: Ia3e64fd4b4084bb5ec35a669c5840c9fc3c58f2e	8 years ago
Rene Krenn	72d9c10650	TT#13973 fix http method override ... the factory Catalyst::ActionRole::HTTPMethods action role is set for any action, and in race with our own NGCP::Panel::Role::HTTPMethods. the testcase could have been misleading ever since, as it uses DELETE /api/customers, which was present when implemting the method override initially, but dropped on the road to 4.5. the issue is resolved by overriding Catalyst::Controller::gather_default_action_roles. until the class hierachy refactoring is completed, only consistent way is to add the override to each api controller. the action_roles config attribute can therefore be removed. using Plack::Middleware::MethodOverride as an alternative from upstream has the drawback to work only with POST. but a required usecase is to also support translation DELETE requests to PATCH. Change-Id: I204ba59869a8327bdd5db8a867fbbb061d1c9e7c	8 years ago
Irina Peshinskaya	d43d51d036	TT#7453 Add filters for dates and readonly date related fields to subscriber ... Change-Id: Ic67202ce9b7b18330c978149c5170612ca1e7098	8 years ago
Irina Peshinskaya	2073b9e760	TT#9309 Introduce local version of the Data::HAL::Link ... Change-Id: Ifdf95cef32f0d118330c20316ed447e00c6dae3c	8 years ago
Irina Peshinskaya	41f32401a7	TT#9309 Introduce local version of the Data::HAL with default forcearray ... Change-Id: Ie33396a75fc6d27fd28fe8f7717caba0a476c776	8 years ago
Irina Peshinskaya	99ee7da751	TT#8272 Add necessary form_validation exceptions for the template fields ... Change-Id: I6fbccfce06d53d6cf0db58b8b84cb71a96e2057a	8 years ago
Irina Peshinskaya	a4e6c7767b	TT#8272 Collapse copy-pasted code a little ... Change-Id: If6844d2ae72d728411421faf01f905bd49c21033	8 years ago
Rene Krenn	979397cff3	MT#20639 TT#5411 contact termination ... +1. terminate if terminated contracts or subscribers exists +2. if active contracts or subscribers -> error +3. delete otherwise +customer contact api DELETE +customer contact testcases +customer api POST +customer api PUT/PATCH +system contact api DELETE +system contact testcases +contract api POST +contract api PUT/PATCH +panel UI contact delete +panel UI contract create no change panel UI contract update +panel UI customer create no change panel UI customer update note contact links like customers.omm_contact or customers.tech_contact are not considered. Change-Id: I19ea6e2245282485cc30d3e92d17a2eb603e0c2d	9 years ago
Gerhard Jungwirth	b34a7df2e1	MT#21193 use Sipwise::Base in api modules ... which implies, among others, strict and warnings. these should never be omitted and are one of the most frequent complaints of perlcritic currently. as a result TryCatch can be removed (imported by Sipwise::Base). also, "no Moose" is removed (has no effect). Change-Id: I901b6a9f0f6d426f62b73e68f2c5ad6365c1eeef	9 years ago
Rene Krenn	3e57109750	MT#19419 custom action role to support http method override ... Change-Id: I2d9fbf1f4feca76b91bdf73d4184ea707cc04e78	9 years ago
Guillem Jover	0632585eba	MT#18443 Use parent instead of base module ... Switch from the discouraged base module to the light-weight parent module. Also remove some surrounding BEGIN blocks from 'use parent' which are completely unnecessary. Change-Id: I3a669e8024f098819be45030ca9d1afa8756105c	9 years ago
Irina Peshinskaya	0272c297b8	MT#17173 Remove Moose Roles from API ... Conflicts: lib/NGCP/Panel/Controller/API/MetaConfigDefs.pm lib/NGCP/Panel/Controller/API/Root.pm Change-Id: I25597baebbec6e2436b15776da17bd6d1bfef838	9 years ago
Andreas Granig	04a64e1493	MT#16465 partly remove namespace::sweep and others ... Remove namespace::sweep Remove MooseX stuff Remove perl5i Remove Moose where possible Change-Id: I4f873afb466f4e58a04c3304ff74df2185470b03	10 years ago
Irina Peshinskaya	755ca9312a	MT#16323 Add rating_status field into item response and filter for collection ... Change-Id: Idc82840fd20cfa74ed0e6abe041125e8bb105f8f	10 years ago
Rene Krenn	7f7e3332d1	MT#15477 topup log ... any *attempt* of a top-up request should be logged. +create top-up log records for api in order to record failed top-up requests as well, the somewhat tricky thing is to have two separate transactions here +propery casting of topup request json field values to numbers etc., so the transaction for creating the log entry cannot fail and e.g. requests with subscriber_id='blah' are recorded correctly +new "request_token" parameter for /api/topupvoucher and /api/topupcash, to identify and filter for particular requests. +topup log api resource +topup log api tests +fix to correctly reject used vouchers +topup log panel UI +fix for balanceintervals.t threaded tests Change-Id: I86eb845f6173803705b12cc7e5cdbac9a3153a0a	10 years ago
Rene Krenn	33e5f4e71f	MT#14255 initial balance_intervals catchup when starting at 1st of month ... Change-Id: I9f22ffc112386da1bdac5e3fedb5b69e4a09fcfa	10 years ago
Rene Krenn	07c9d1955d	MT#14255 add cash_balance underrun logic ... in short: whenever the contract_balances "cash_balance" is about to decrease, it might drop below thresholds defined by a contract's profile package. in that case, subscribers are to be locked and/or underrun billing profile sets are to be applied. this commit also contains some minor fixes: +when creating/editing profile packages, all billing profiles of initial, topup, underrun sets must show the same interval_free_cash and interval_free_time values. there was also a problem with adding profile sets to packages that use profile with prepaid = 1 (see ticket 14301) +when editing packages using the panel UI, the validation was not properly invoked (!) +the frequency of topups has to be limited to 1 per second. +updating the "lock" subscriber prior to editing or viewing it in the panel UI was added. +topup cash and topup voucher panel UI added +when aquiring multiple contract row locks at once, no joined tables are used any mure. the number of row locks is therefore minimal now (see ticket 14903) +rework of updating prepaid subscriber preference Change-Id: Ideb0fec2404ca10b3ac5d8735f53620d0c3a24ed	10 years ago
Rene Krenn	bef06dce91	MT#14301 deleting prof packages instead of terminating ... +profile package details view to overlook associations +voucher count is properly checked before deleting a package +package_id filters for /api/customers and /api/vouchers +Fake::Time replaced by Time::Warp for balance_interval tests +cleanup Change-Id: Idc34220a9d29c115453eeee2907e70a71cf9f0cc	10 years ago
Rene Krenn	f3b711a050	MT#13903 disable applying profile package ... This is a patcht introducing a variable to disable setting profile_package_id's in in the contracts table. The code should therefore behave as before, as long rate-o-mat and libswrate are ready. Not once set to 1 and contract balances are written to a database once, disabling this constant will mess up contract_balances! Change-Id: I080460395627202f3f5a386022de643577f4f3c2	10 years ago
Rene Krenn	c378681a24	MT#13903 balance interval catchup and resize WIP ... + synchronized contract balance catchup + balance interval resizing upon profile package transitions + dynamic interval length, interval start, carry-over propagation, .. + end-of-month 'preserve' mode correction for strictly aligned month intervals for start_mode=customer "create" timestamp + new api/balanceintervals resource to inspect contracts' balance interval histories + updated affected panel UI controllers + test case with time warps using Time::Fake caveats: - creating an invoice for a given 'period' (month) has to be refactored to select a disitnct balance interval. right now it takes the last interval in the month specified. - generate_invoice.pl is broken and needs a major refactoring Change-Id: I7bb54a83b76e510b1baa573a986d05400a7fec1e	10 years ago
Rene Krenn	2537819f0d	MT#12643 billing networks and multi bill prof WIP ... -properly validate billing mapping intervals -api/contracts and api/customers complete +the problem with updating the "prepaid" preference remains at the moment +billing_mappings field contains future (editable) mappings only, the full schedule is given in a separate field +hal-links to all occurring billing profiles and networks -extended tests: api-customer.t and api-contract.t -api-billingnetworks.t test -billing networks panel controller +search by enclosed IP address -panel UIs for billing networks, customers, contracts +past billing mapping intervals that cannot be edited are displayed by a seperate hfh "block" part of the billing_profiles repeatable -profile packages api, UI +terminating billing profiles (MT#8645) will be broken +fixed now: profile cannot be terminated unless there are: -zero non-terminated customers, whose mappings contain the profile with future or open end -zero profile packages, that link to the profile in either one of their initial-, underrun- or topup profile sets +panel edit should work, termination wip +topup_profiles w/o amounts +contracts/customers API profile_package_id field: +apply the package's initial profiles +api/customer and api/contract profile definition modes (id/profiles/package) +api OK, api-customer.t tests +customer and contract panel UI with selector to specify actual billing mapping, schedule or profile package (same as api) +terminate logic harmonized: profiles, networks, packages +api tests updated +prepare for merging: cleanup, enable tests Change-Id: I6c0dec72f490b5731fc673cd393d6aeb08dd8143	10 years ago