94 Commits (764ab7d2d181277f20f31c858ea84816becc99ce)

Author	SHA1	Message	Date
Rene Krenn	764ab7d2d1	MT#62283 exclude "for update no lock" in tests ... Change-Id: I12ff4d1f8ba0b150a1918de40dafa1efd6a92c10	2 months ago
Rene Krenn	82af5cf518	MT#62283 "for update no lock" logic for GET /subscribers and /contracts ... to unblock the web UI on heavy loaded systems with call rating enabled, "select .. for update no lock" will no longer wait for acquiring locks when retrieving subscriber or contract contract pages. the contract_balance record creation is will be skipped for such contracts that are locked elsewhere. for rails such as /customerbalances, an explicit contract id row lock timeout is introduced, so it does not depend on the mariadb wait_timeout. Change-Id: I6e96342f3f761279a5876c871d2477977823a39e	2 months ago
Kirill Solomko	4eace822b5	MT#62180 /api/subscribers prevent subscribers from POST/DELETE ... * regular subscribers should not be able to use POST as they are not allowed to create other subscribers, nor other PBX groups. resource. * they should not be able to use DELETE as they cannot delete themselves nor other subscribers, nor PBX groups. Change-Id: Idaacb344b65ff8c5da9f4c3649a9b4089938a82c	2 months ago
Kirill Solomko	e2e1776090	MT#60558 move max license checks for POST /api/subscribers ... * the relevant max license checkes are moved from Controller::API::Subscribers::POST to Utils::Subscribers::prepare_resource because there we fetch customer_id from the pilot subscriber in case of pbx subscriberadmin requests that is neccessary for PBX subscribers max license check. Change-Id: I2d1c212d73fe5b9295d1595b4fffebeb67b61e5a	9 months ago
Kirill Solomko	d9f283cbc8	MT#59449 enhance password validation and handling ... * passwords are now validated based on - minlen - maxlen - min lower case chars - min uppper case chars - min digits - min special chars * Data::Password::zxcvbn is used to calculate password score and reject passwords with score < 3 as weak (this library is ported from the Dropbox password validation) * Add password journals and check last used passwords in the journals * Improve password generator javascript function to generate a password with at least 4 of each of the char group types. * Currently affected are subcriber and admin entry creation or modification via UI/API * NGCP::Utils::Auth add optional bcrypt_cost support as last argument for generate_salted_hash and get_usr_salted_pass Change-Id: I100c25107d91741d5101bc58d29a3fa558b0b017	9 months ago
Kirill Solomko	43d112bd5e	MT#60558 add max subscribers/groups license checks ... * max_subscribers, max_pbx_subscribers, max_pbx_groups license checks are added for subscriber/group creation in UI/API * new accessor $c->license_meta that returns meta license flags hashref * new accessor $c->license_max_subscribers * new accessor $c->license_max_pbx_subscribers * new accessor $c->license_max_pbx_groups * the new accessors (except license_meta) return -1 instead of 'unlimited' to ease off comprarsion * 403 Forbidden is returned by the API if a license is violated. Change-Id: I3f5a949efc84bf85b76b33404b37b362ec484d5f	9 months ago
Kirill Solomko	6543743992	MT#60162 improve expand functionality for collections ... * in general expand collection now performs only 1 sql query to per expand field to fetch all items by the ids instead of fetching them for every single collection item, that should significantly increase performance in case of large databases and reduce work for the database * introduce $c->stash->{expand_cache} that contains cached data for the expand fields to avoid multiple same calculations * expand_field() and expand_field_data() have been reworked to support expand_cache * new method expand_prepare_collection() is called for all API GET collection methods before preparing resource hal fields, to change the expand logic to only cache the data instead of fetching it from the database * new method expand_collection_fields() that is called in all API GET collection methods after the @embedded data is prepared to finalise the expand collection fields * for expand collection there is only 1 SQL request per expand field that fetches all items -in [ids to expand] and then the expand_collection_fields() uses the cached items_by_ids with O(1) fetch from the cache by id Change-Id: Ie7c6115472878febf0d8c9b4d833f5c23b15c78b	11 months ago
Kirill Solomko	c6aaaddd87	MT#56301 move lock_provisioning_voip_subscriber before underrun_lock ... * POST /api/subscribers/ invoke lock_provisioning_voip_subscriber() before underrun_lock(), so that if the subscriber is about to be locked due to customer balance costs, the lock_provisioning_lock_subscriber() invocation will not unlock it back straight after Change-Id: I55020f844c9aa76df2e2f057a88b2ae7c9ebbfcc	1 year ago
Rene Krenn	df670f501f	MT#56301 set locklevel of created subscribers only if specified ... Change-Id: I90f2d68dee50ab240de4eb5af7495ddd551c00d8	1 year ago
Marco Capetta	8945b8a5ac	MT#56301 Fix API creation of an active subscriber with lock ... Using API is possibile to edit an existing subscriber and set status: active and lock: 2. It is instead impossbile to create a new subscriber with the same configuration because the 'lock' param is not taken into account. The only way to properly setup lock level it is to set the status to 'locked' and this is in fact wrong. The fix address this issue and allow now to create a new subscriber with status: active and lock: 2. Change-Id: Id18e40bc001c5a7de30f5d148231bda93a3b1b3d	1 year ago
Kirill Solomko	205b27a267	MT#59478 API better transaction and error handling ... * Role/Entities*: POST/PUT/PATCH/DELETE methods changes: - support deadlock detection and transaction retry (2 retry attempts at the moment) - improve transaction control, use local $guard instead of saving the ref to $c->stash, as in that case it went out of scope too late and also reported an error message into the log about abnormal $guard out of scope interruption - move all non transaction related code outside of the scope - add error handling when methods such as update_item, and a like do not return the expected data, instead of simply going out of scope and resulting in an uncontrolled reply * Role/API: - rework transaction control: + get_transaction_control() is renamed to start_transaction() to better reflect what it does + complete_transaction() is renamed to commit_transaction() + remove unused %params arg + pass $guard into commit_transaction() instead of having it stored as $c->stash->{transaction_guard) that caused the $guard ref to be destroyed much late than expected (there was also a typo as transaction_quard, which is not relevant anymore with the changes + add check_deadlock() that is invoked when an exception is caught or an $c->errors contain an error, and if the error message represents a transaction error, the transaction block is re-invoked via "goto TX_START" - rework error(): + it now accepts args as following: ($self, $c, $code, $message, @errors) # code -> returned as HTTP code in the reply # message -> returned as HTTP message in the reply # errors -> contain errors for internal logging, last element often contains a DBIx exception + populates all @errors into $c->error so they are available on demend in the code via $c->error or $c->last_error + $c->log->error is not invoked now as the errors become printed in log_response() - log_response() now prints collected errors from $c->error correctly as a separate log line, that is alike to the other api logs so that those can be looked up by the request's tx_id, also all errors are now printed only into api.log * Adjust all $self->error() calls in catch($e) to include $e as the last argument, as well as the duplicate $c->log->error is removed from those ocassions * Remove all $c->log->error() calls as they are replaced with either $self->error() (that logs it correctly into api.log) or $c->error('err') that also adds it correctly into api.log * API::CallForwards: rework to use Entities/EntitiesItem * API::Contracts: rework POST to use Entities * API::PeeringGroups: rework POST to use Entities * API::SubscriberRegistrations: rework POST to use Entities * API::RewriteRuleSets: improve create_item() functionality * Utils/Message: add 'api_retry' log type * $c->session->{api_request_tx_id} is changed to $c->stash->{api_request_tx_id} because sometimes the session ref is different and a different tx_id becomes used Change-Id: I633ce7a8047b1bf00a2f6889003088edf0825dcd	1 year ago
Rene Krenn	672a3110bc	MT#59216 restore appending wildcard behaviour ... with the change to harmonize wildcard symbols % and * to * in query param filters across all restapi rails, only - /api/subscribers?alias= - /api/subscribers?pbx_extension= - /api/subscribers?primary_number= - /api/pbxdevices?pbx_extension= are supposed to include implicit leading an trailing wildcards. panel UI uses trailing wildcard by default everywhere, and so should the databales of the new Admin UI. external_id were used to support patterns, but must not include a trailing wildcard in rest API, as this can break logic that rely on unique api request results. we now therefore no longer append wildcards in particular to: - /api/voicemails?folder= - /api/voicemailgreetings?type= - /api/subscribers?subscriber_external_id= - /api/subscribers?customer_external_id= - /api/subscribers?display_name= - /api/subscribers?domain= - /api/profilepackages?profile_name= - /api/profilepackages?network_name= - /api/pbxdevices?display_name= - /api/faxserversettings?name_or_password= Change-Id: I40ef1912d7224a56c6d14619d258c70e447ab23d	1 year ago
Rene Krenn	308a974059	MT#58762 harmonize restapi wildcard filters ... - all standard LIKE search are migrated - will avoid LIKE unless a pattern (* wildcard) is used as a search term. this encourage db index usage, will be faster - supports wildcard escape sequence \\* - harmonize swagger UI descriptions of filters Change-Id: Iea155871c9be6c284e6970a562d4e6af73fedc4b	1 year ago
Kirill Solomko	d7c7391644	Revert "TT#158252 API GET subscriberpreferences - fix DBI timeout" ... This reverts commit a6b5ac98c0. Change-Id: Ie9c70a0821db61cb2bc1634d54ae0f6d3a096a92	3 years ago
Oleksandr Duts	a6b5ac98c0	TT#158252 API GET subscriberpreferences - fix DBI timeout ... * Add pref->attribute prefetching. * Remove unused contract balance calling. Change-Id: Idff3893beb42aa72bd04d6248a390ab6c89fe4a1	3 years ago
Rene Krenn	9a3c3cd42c	TT#153100 fix subscriber username queryparam search ... Change-Id: I56ca6601bb1abc38552561e0a110e875e9282a3b	3 years ago
Kirill Solomko	c50b49db96	TT#133800 fix password fields behaviour in API ... * PATCH: password fields are not removed when resource is created for apply_patch(), they are removed under the same condititions later when hal is generated, that is to ensure that admin users without the 'show_passwords' flag as well as subscribers will not run into situation when they use PATCH and cannot apply it for "path": "/password" or/and "path": "/webpassword", as they were removed before apply_patch() * rework encrypted webpassword detection. webpasword is detected as encrypted if its length is 54 or 56 and it contains at least one '$' char, there is a chance for false positive detection when a user provides with a plain-text password with the same pattern but it's very unlikely, as well as since mr8.5 webpasswords are expected to be encrypted, and moreover worth case scenario is that the plain-text password will not be returned to the user Change-Id: I8ea739cbf728b2134f3ce00cee29da42ab3fb4a3	4 years ago
Flaviu Mates	01e11a710b	TT#93753 - Enhance subscriber api filtering ... * Enable subscriber filtering by wildcard match of display_name, pbx_extension, primary_number Change-Id: Id4d356ec5abb18c82e1e369a15f665062ef69de7	5 years ago
Andreas Granig	8f8e1d5fc9	TT#78557 - Use bcrypt to hash webpassword ... * Change the way webpassword is handled accross NGCP Panel UI/API to comply with new password encryption * At login, if password is not encrypted with high cost due to the ngcp-bcrypt-webpassword script, encrypt it with proper cost * Accept old password format as well until all webpasswords are encrypted Change-Id: Iefa9584a62ab4b7d2a224d10bdd415e9cbb8dfb5	5 years ago
Rene Krenn	f9690f08f6	TT#77452 rename rowlock_contract ... complete the renaming of "sub rowlock_contract" into "sub acquire_contract_rowlocks" to distinguish it from "sub recursevly_lock_contract" (which is related to set the subscriber lock level, not db row write locks) Change-Id: I287ee611e20c71a90121007511c3781359968bd2	5 years ago
Rene Krenn	a11f2bed5e	TT#77452 properly synchronize subscriber create/update ... acquire the billing.contract row lock *before* any unordered billing.voip_numbers rowlocks by sub manage_subscriber_numbers(). - "deadlock" waittimeout errors will cease when creating subscribers concurrently via api - max_subscribers, is_pilot and other per-contract constraints will be respected accurately Change-Id: I73bb7525b327bbb09217b790be9c14cc65ddebcc	5 years ago
Flaviu Mates	28cb73e58e	TT#65651 - Display correct DBIx error on duplicate subscriber ... * Add check for provisioning subscriber webusername so the API trhows the correct error when encountering a duplicate * This implementation relies on the DBIx error to keep the database integrity Change-Id: Ica3e2673cead28759ad25f5edb3f7ca0f32e1c1e	6 years ago
Kirill Solomko	ce664263b2	TT#65101 add ccareadmin ccare roles ... * ccareadmin and ccare roles have full access to Customers, Subscribers and their preferences/settings, and read-only access to BillingProfiles,InvoceTemplates, EmailTemplates * ccare role is restricted to the related reseller Change-Id: I6cf7d3adf912f0fa98d1ef5c02abea2f4331ec4b	6 years ago
Rene Krenn	dda5ffe48f	TT#56340 fix subscribers.t not matching obfuscation chars ... Change-Id: I7023d1e23fc397e3556705fc52db50eace6c02b4	6 years ago
Rene Krenn	cae5270af5	TT#56340 panel logline obfuscation ... adds gdpr obfuscation quoting for: + subscriber numbers + subscriber ip addresses + subscriber usernames + any logmessage "DATA": query parameters, form data, response data + subscriber uuid's + call id's + callforward sip uri's the quoting is centralized by $c->qs() ("quote sensitive"), using catalyst plugin mechanism. escape symbols are set to « (\x{ab}) and » (\x{bb}). generate_logfile_data_inventory.pl was modified to mark loglines with "gdpr affected" status, if $c->qs() was used in a log message. Change-Id: I0f42d7992594232ae33e5666b0a64009211c5b76	6 years ago
Irina Peshinskaya	1fe6db943d	TT#48198 Use items array from paginations in all api controllers ... Change-Id: I1856cd4f073d32b4465d0778baf5f9f7c708f78c	6 years ago
Rene Krenn	6e1b17b26e	TT#44960 /subscribers/?alias=123 w/o group_concat() ... Change-Id: If9466bc4cd6f88cb787c0a707cc32714906dc308	7 years ago
Irina Peshinskaya	b1e4733d00	TT#43351 Check subscriberadmin access to profile and profile_set ... Unify UI and API code to make subscriberadmin behavior with profile_set and profile common Change alias_numbers input field to embedded RepeatableJS so it pass form validation Change-Id: I1b3dc6f88cc6c1f43d76acc8e339dbb664c166a3	7 years ago
Irina Peshinskaya	cc6cf3b067	TT#37375 Allow additional "order by" columns specifications ... Change-Id: Id37c305b2abeba87bd5ba25f97399888411eeab3	7 years ago
Gerhard Jungwirth	e5fccbf98e	TT#37105 fix perlcritic errors related to variables ... these are to fix sipwise TAP tests. The following policies were violated and have been fixed (or ignored where apppropriate): - Variables::ProhibitAugmentedAssignmentInDeclaration - Variables::ProhibitConditionalDeclarations - Variables::RequireLocalizedPunctuationVars Change-Id: Ifee93d537cd6e33d9e6e6ef8026be4b2b9fd0ee5	7 years ago
Kirill Solomko	f976fe03fb	TT#35714 use collection_nav_links in all API controllers ... * colelction_nav_links() should be used instead of the old self/prev/next href link code * Entities.pm fix colletion_nav_links() to pass missing $c->request->params * collection_nav_links() use default $c->request->path and $c->request->params if those arguments are not specified Change-Id: Icca5afac812f2e06e6927ba6f4e91057f9745d21	7 years ago
Irina Peshinskaya	cb1a3a4cd8	TT#34021 Allow PUT and PATCH to subscriberadmin on subscribers ... Change-Id: I3c3d5b0c5d1bfd4c3c8771ed332cbf00d3d3c1dc	7 years ago
Irina Peshinskaya	2b8ae95170	TT#28460 Remove unused libraries ... require Catalyst::ActionRole::ACL; require Catalyst::ActionRole::CheckTrailingSlash; require NGCP::Panel::Role::HTTPMethods; require Catalyst::ActionRole::RequireSSL; Change-Id: I3880fcb10067810d5e2de10a0c14bf6b15e34710	7 years ago
Irina Peshinskaya	0d4d8cdb23	TT#28460 Format: Remove excrescent empty lines ... Change-Id: I5ff73d92bf62451fb6dbdae85ca07428c3f5af5d	7 years ago
Irina Peshinskaya	75d6edbcd9	TT#28460 Format: add line between subs in API controllers ... Change-Id: I67f11fef716941dc965dffd4e8fbb1a211791bbf	7 years ago
Irina Peshinskaya	9fa2a19b4f	TT#28460 Replace configs - not journalized ... Change-Id: I9843a780817922f23faa91041d3fab9b7336b2a3	7 years ago
Irina Peshinskaya	92e6215b71	TT#28460 Remove gather_default_action_roles ... Change-Id: Ie8080a9ac3b5b1004de456fe0ece7b31ed1541fe	7 years ago
Irina Peshinskaya	2498c8810e	TT#28460 Remove copy-pasted sub HEAD and sub end ... Change-Id: Ia06320ead891945eebe715fa34012a14d5529f17	7 years ago
Irina Peshinskaya	05774c2338	TT#28460 Remove copy-pasted sub OPTIONS ... Change-Id: I2b00ea267edcc3caa00eaac5753638794a13398c	7 years ago
Irina Peshinskaya	a113cbf1d0	TT#28460 Remove copy-pasted sub auto ... Without config changes - all custom subs are in place Change-Id: I23e50e7d1fcb02c245bc995a26e16e97308177c8	7 years ago
Irina Peshinskaya	b14f34f008	TT#28460 Change parents in API controllers ... Use Entities and EntitiesItem respectively Change-Id: I5d32b010cc51810acee42d14c5c24ce3d4267602	7 years ago
Gerhard Jungwirth	24de4232e6	TT#30507 remove DataHal workaround layer ... (NGCP::Panel was loaded every time this was used) Change-Id: I62861e0756aaaf548c9be42b4842e9a06127119c	7 years ago
Gerhard Jungwirth	edbef3c5b8	TT#29617 remove unnecessary form exceptions ... with the previous commit, there is no need anymore to specify form exceptions manually. validate_form will now automatically consider fields in the correct format with and without "_id" Change-Id: I70afae4003c6407c7a37396630110736f1fb13f8	7 years ago
Irina Peshinskaya	77ae84e215	TT#23282 Replace customer field with customer_id for the API subscribers form ... Change-Id: I50eb7b11e520db1509e68f50744b5ee0dab2193c	8 years ago
Andreas Granig	0d3bf6b1ce	TT#22827 Implement form factory ... This patch reuses existing forms by clearing them, rather than re-instantiating them again and again. Also, panel start time should be better due to less package pre-loading. Change-Id: Ia3e64fd4b4084bb5ec35a669c5840c9fc3c58f2e	8 years ago
Andreas Granig	f9030b4f21	TT#4333 subscribers/customers/numbers/CF ACL ... * Implement r/o access to /api/customers for subadmin * Implement r/o access to /api/subscribers for subs * Implement create access to /api/subscribers for subadmins * Implement /api/numbers to re-assign a number * Implement CF endpoints access for subs * Implement new test framework to simplify testing * Fix updating number by using Utils to make sure allowed_clis etc are handled correctly * Install missing docker dependencies for new test framework * Add edr handling on /api/numbers/xx update Change-Id: I678da16cfb1361b2809780cba8c204ac90bb1736	8 years ago
Rene Krenn	ad3562cba8	TT#14295 timezone field for /api/subscribers ... added checking and persisting of timezone for + panel UI subscriber create + POST /api/subscribers + PUT /api/subscriber/x + PATCH /api/subscriber/x + central is_valid_timezone_name method Change-Id: I59865f5af33a71e5b5032dc35ba9c41eca74b04c	8 years ago
Rene Krenn	72d9c10650	TT#13973 fix http method override ... the factory Catalyst::ActionRole::HTTPMethods action role is set for any action, and in race with our own NGCP::Panel::Role::HTTPMethods. the testcase could have been misleading ever since, as it uses DELETE /api/customers, which was present when implemting the method override initially, but dropped on the road to 4.5. the issue is resolved by overriding Catalyst::Controller::gather_default_action_roles. until the class hierachy refactoring is completed, only consistent way is to add the override to each api controller. the action_roles config attribute can therefore be removed. using Plack::Middleware::MethodOverride as an alternative from upstream has the drawback to work only with POST. but a required usecase is to also support translation DELETE requests to PATCH. Change-Id: I204ba59869a8327bdd5db8a867fbbb061d1c9e7c	8 years ago
Irina Peshinskaya	d43d51d036	TT#7453 Add filters for dates and readonly date related fields to subscriber ... Change-Id: Ic67202ce9b7b18330c978149c5170612ca1e7098	8 years ago
Irina Peshinskaya	2073b9e760	TT#9309 Introduce local version of the Data::HAL::Link ... Change-Id: Ifdf95cef32f0d118330c20316ed447e00c6dae3c	8 years ago