154 Commits (a97490e97df609d35c5c8e8153bbec549a16266a)

Author	SHA1	Message	Date
Guillem Jover	b6e2baa6c7	MT#63479 Remove prosody support ... Change-Id: I9d3e268450396c12e4058ac9394a80bb191d7635	2 months ago
Kirill Solomko	6e2ca03a6e	MT#63997 make underrun lock on requests optional ... * a new ngcp_panel option $c->config->{api}{underrun_lock_on_request} is in place and if disabled - the underrun lock logic on GET contracts,customers,subscribers,subscriberpreferences, and POST subscribers is skipped. Change-Id: I5ea337ce42b3c979fe55b3df3516a19548b64f90	2 months ago
Rene Krenn	f501fc1dcf	MT#63706 skip_locked for POST /api/subscriber/ ... add the SELECT .. SKIP LOCK mechanism for POST /api/subscriber/, to reduce lock wait timeouts when creating suscribers for the same customer in parallel. Change-Id: I067102b3dc726e5d12fa859613b548d007ce29ee	3 months ago
Rene Krenn	82af5cf518	MT#62283 "for update no lock" logic for GET /subscribers and /contracts ... to unblock the web UI on heavy loaded systems with call rating enabled, "select .. for update no lock" will no longer wait for acquiring locks when retrieving subscriber or contract contract pages. the contract_balance record creation is will be skipped for such contracts that are locked elsewhere. for rails such as /customerbalances, an explicit contract id row lock timeout is introduced, so it does not depend on the mariadb wait_timeout. Change-Id: I6e96342f3f761279a5876c871d2477977823a39e	11 months ago
Kirill Solomko	97e457c92b	MT#62178 fix external_id, profile auto removal for subscriber roles ... * fix 'subscriber' and 'subscriberadmin' roles using PATCH to cause external_id, and profile fields removed. * remove profile_id field from the 'subscriber' and 'subscriberadmin' form Change-Id: Id8bb068ca7fcac2c0c5954f2ed79e841d18ac398	12 months ago
Kirill Solomko	bbd44f16fa	MT#60858 add /api/passwordchange endpoint ... * the new endpoint accepts new_password fields and enables for authenticated user a mechanism to quickly change their password * the global password validation rules are enabled * returns 204 No Content * if user's password is expired, the endpoint is the only accessible for the user to change the password and unlock other endpoints. * a few fixes in validate_password() to correctly fetch provisioning subscriber for password change scenarios and webpassword field Change-Id: I906fcfe5c780b850d322b46b445b54c054767673	1 year ago
Kirill Solomko	b041888807	MT#60656 add max_age password validation support ... * UI: password are now validated against $c->config->{security}{password}{web_max_age_days} (unless it's 0) and if the password is expired the user is redirected automatically to /changepassword page, and after successful password change back to the original page. * API: if password is expired all API requests will be returning 403 Forbidden "Password expired", except PUT/PATCH to /api/admins or /api/subscribers with the new password in place. * successful login on the UI now redirects to /dashboard instead of / (to prevent unintended redirect to v2) Change-Id: I075f8e17cc9b0658d6b3b3d526ca5b379d050ce4	2 years ago
Kirill Solomko	e2e1776090	MT#60558 move max license checks for POST /api/subscribers ... * the relevant max license checkes are moved from Controller::API::Subscribers::POST to Utils::Subscribers::prepare_resource because there we fetch customer_id from the pilot subscriber in case of pbx subscriberadmin requests that is neccessary for PBX subscribers max license check. Change-Id: I2d1c212d73fe5b9295d1595b4fffebeb67b61e5a	2 years ago
Kirill Solomko	d9f283cbc8	MT#59449 enhance password validation and handling ... * passwords are now validated based on - minlen - maxlen - min lower case chars - min uppper case chars - min digits - min special chars * Data::Password::zxcvbn is used to calculate password score and reject passwords with score < 3 as weak (this library is ported from the Dropbox password validation) * Add password journals and check last used passwords in the journals * Improve password generator javascript function to generate a password with at least 4 of each of the char group types. * Currently affected are subcriber and admin entry creation or modification via UI/API * NGCP::Utils::Auth add optional bcrypt_cost support as last argument for generate_salted_hash and get_usr_salted_pass Change-Id: I100c25107d91741d5101bc58d29a3fa558b0b017	2 years ago
Kirill Solomko	9d021be65a	MT#59979 add license control ... * UI and API parts are now under license control * new Util::License::get_license($c, $name) - fetches license status by name (1 if enabled, and also if /proc/ngcp/check if 'ok') * add Catalyst::Plugins::NGCP::License with license($name) to fetch valid license by name from anywhere using $c->license('pbx') or from the templates using c.license('pbx'). It internally uses Util::License::get_license($c, $name) * License::get_license_status($c) now requires $c as first argument as well logs license status check errors. * new ActionRoles::License that enables usage of :Does(License) RequiresLicense('pbx') LicenseDetachTo('/denied_page') in the Controller chains * Add license control for UI elements and return 403 Forbidden if a resource is covered by licenses and the license is not active * Hide UI elements if a license is not active * API/Entities/Entities new $c->set_config key: - per endpoint: $c->set_config({ required_licenses => [qw/pbx device_provisioning/] } - or per method: $c->set_config({ required_licenses => { POST => [qw/pbx device_provisioning/] } } } * In case if an API endpoint does not have a license: 403 Forbidden "Invalid license" reply is returned. * Add license based restrictions to API endpoints * /api documentation: - completely hide endpoints that do not have an active license - hide only methods that does not have an active license Change-Id: Iba45fc5068b02306a617fed7b5405f2210574b61	2 years ago
Kirill Solomko	205b27a267	MT#59478 API better transaction and error handling ... * Role/Entities*: POST/PUT/PATCH/DELETE methods changes: - support deadlock detection and transaction retry (2 retry attempts at the moment) - improve transaction control, use local $guard instead of saving the ref to $c->stash, as in that case it went out of scope too late and also reported an error message into the log about abnormal $guard out of scope interruption - move all non transaction related code outside of the scope - add error handling when methods such as update_item, and a like do not return the expected data, instead of simply going out of scope and resulting in an uncontrolled reply * Role/API: - rework transaction control: + get_transaction_control() is renamed to start_transaction() to better reflect what it does + complete_transaction() is renamed to commit_transaction() + remove unused %params arg + pass $guard into commit_transaction() instead of having it stored as $c->stash->{transaction_guard) that caused the $guard ref to be destroyed much late than expected (there was also a typo as transaction_quard, which is not relevant anymore with the changes + add check_deadlock() that is invoked when an exception is caught or an $c->errors contain an error, and if the error message represents a transaction error, the transaction block is re-invoked via "goto TX_START" - rework error(): + it now accepts args as following: ($self, $c, $code, $message, @errors) # code -> returned as HTTP code in the reply # message -> returned as HTTP message in the reply # errors -> contain errors for internal logging, last element often contains a DBIx exception + populates all @errors into $c->error so they are available on demend in the code via $c->error or $c->last_error + $c->log->error is not invoked now as the errors become printed in log_response() - log_response() now prints collected errors from $c->error correctly as a separate log line, that is alike to the other api logs so that those can be looked up by the request's tx_id, also all errors are now printed only into api.log * Adjust all $self->error() calls in catch($e) to include $e as the last argument, as well as the duplicate $c->log->error is removed from those ocassions * Remove all $c->log->error() calls as they are replaced with either $self->error() (that logs it correctly into api.log) or $c->error('err') that also adds it correctly into api.log * API::CallForwards: rework to use Entities/EntitiesItem * API::Contracts: rework POST to use Entities * API::PeeringGroups: rework POST to use Entities * API::SubscriberRegistrations: rework POST to use Entities * API::RewriteRuleSets: improve create_item() functionality * Utils/Message: add 'api_retry' log type * $c->session->{api_request_tx_id} is changed to $c->stash->{api_request_tx_id} because sometimes the session ref is different and a different tx_id becomes used Change-Id: I633ce7a8047b1bf00a2f6889003088edf0825dcd	2 years ago
Rene Krenn	740bb764e5	MT#59434 always take aliases from voip_numbers ... in case the DB shows discrepancy because of missing primary provisioning.voip_dbaliases records, align with legacy panel behaviour and also take the restapi aliases list from voip_numbers, instead of voip_dbaliases. .. while trying to keep the performance gain from https://gerrit.mgm.sipwise.com/c/ngcp-panel/+/65086. Change-Id: Ibd7f8c8bc6a39ae2c31b4e8818080674ab77d66c	2 years ago
Donat Zenichev	fd796a91ba	MT#56845 HG: introduce new field `Cancel Mode` ... This field controls a behavior upon cancelling unsuccessful legs: - terminate legs with CANCEL (default way) - terminate legs with BYE Change-Id: I2ff5c758c319714f0e6636db8b8ba5c0fd495e1f	3 years ago
Kirill Solomko	391aa3fbc3	MT#57564 convert deprecated timezone names ... * timezone names are converted to their links, and if a link is found (means that the current name is deprecated), it's validated and stored as the link. Change-Id: I6348659178400a96eaadd70f79b792c4fa25d7d4	3 years ago
Marco Capetta	ce6884ebab	MT#57342 /api/subscribers fix field role inconsistency ... When the subscriber preferences API is called by a subscriber admin of a normal (not PBX) customer, then some values are removed from the output. In all the other cases those values are instead returned. This fix let expose the profile_id, domain_id, status and webpassword to all the types of subscribers because necessary in the CSC context. Change-Id: I629475e7f51d747a55ebfbc44232fb94a54fed06	3 years ago
Kirill Solomko	6c179dd5c4	MT#56234 get_usr_preference changes ... * as_admin param is no longer needed as get_usr_preference() fetches actually set preference for a susbcriber and used only by the code (not exposed directly to 'subscriber', 'subscriberadmin' roles * with the aforedescribed, get_usr_preference() no longer filters by expose_to_customer, expose_to_subscriber for 'subscriber' and 'subscriberadmin' roles * refactor get_usr_preferences() to be simple and in line with the other get_*_preferences() as it's mainly responsible for fetching the preference as requested by the internal code Change-Id: Ia52d8f4ebfd854901bf446e29fb475dea1fba866	3 years ago
Kirill Solomko	6d487452bb	MT#56234 susbcriber preferences related changes ... * fix customer_view role name typo so that it's correctly limited to for 'subscriberadmin' role * revert behaviour of get_usr_preference_rs() to return undef if no preference is found (no access) * get_usr_preference_rs() now also fetches all internal preferences for internal work for 'subscriber' and 'subscriberadmin' roles but they remain invisible for them and not accesible for direct changes (only when requested by internal logic) * new get_usr_preference_rs() 'as_admin' parameter that enables for internal requests to return the preference value for 'subscriberadmin' and 'subscriber' roles, currently used to show 'lock' status and 'display_name', which are otherwise inaccessible as those preferences do not have expose_to_customer, expose_to_subscriber flags * fix api_prference_defs() correct filtering of preferences for 'subscriber' and 'subscriberadmin' roles Change-Id: I1a0e51ace1c649f9061deaccb7d6e9f8459f0ed8	3 years ago
Kirill Solomko	0c4deaaad0	MT#55871 /api/subscribers optimise GET alias_numbers ... * "alias_numbers" array when the resource is prepared is created directly from the resultset, avoiding 'foreach' loops and conditions. This improves performance on ~8000 voip_numbers from ~35 seconds to ~3 seconds. * is_devid is now always present in the "alias_numbers" object, and it's =0 if there is no related entry in provisioning.voip_dbaliases Change-Id: Ia05b26208f3fec3a9b2203aafe9b4c09b98ca44d	3 years ago
Kirill Solomko	507d088586	MT#55864 /api/subscribers correctly process administrative flag ... * administrative flag is now correctl set to false when requested by the client Change-Id: I81a1951f90ce9885bff806d39e11098475d93ac1	3 years ago
Kirill Solomko	e011960739	Revert "TT#171850 fix unintentional primary number removal" ... This reverts commit 6bdfca91129309561e72eed902a8a4b66604a7bd. Change-Id: Ie7269ac9ce0302928b234302880bfa9896e8cdf0	4 years ago
Kirill Solomko	6bdfca9112	TT#171850 fix unintentional primary number removal ... * the current condition for primary number removal for PUT/PATCH is if the primary number exists and specified in the data as primary_number => undef. The condition failed as the 'primary_number' key was explicitly created regardless of the original user data input, resulting in a false primary_number => undef. Change-Id: I17651046627f5c48696c3f1d17da5aa49452fe9a	4 years ago
Oleksandr Duts	afa9289b66	TT#167900 /api/subscribers improve number duplicate checks ... * primary and alias numbers are now validated that they do not belong to another subscriber * aliases are now validated that they are not already set as the primary number * reduce amount of related sql queries Change-Id: I4397bbdc4bc9001b7feeef22cb8f85ee0b6ce8ff	4 years ago
Rene Krenn	d82d93da9d	TT#71950 fix leftover with 35b568dde ... Change-Id: If71d07972f56cb1b7cb0bbafdcfd3b68cce6e254	4 years ago
Michael Prokop	35b568dde4	TT#71950 Fix typos ... * successfuly -> successfully * diplay -> display * recieve -> receive * mathes -> matches * cahnged -> changed * Unprocessible -> Unprocessable * The the -> The * repesent -> represents Change-Id: I123f5b1ec8c11fe01db4e5768bd640c4920d3366	4 years ago
Rene Krenn	599dd0613b	TT#151751 fix plaintext sippassword no longer returned ... this fix addresses regression reported by dominik: * $resource{_password}/{_webpassword} cannot be set before the form validation as they are effectively removed by it, causing /api/susbcribers returning no passwords at all for 'subscriber' roles * Having them after the patch makes no sense either as next resource_from_item call will effectively remove them again (in PATCH) (cherry picked from commit 5e9066c4fb) Change-Id: I88c9ec40843f1e9a6983952b96c0b0e70fbb1bb1	4 years ago
Kirill Solomko	81ebdb8dcf	Revert "TT#151751 refactor persisting subscriber webpassword" ... This reverts commit 5e9066c4fb. This implementation breaks: * $resource{_password}/{_webpassword} cannot be set before the form validation as they are effectively removed by it, causing /api/susbcribers returning no passwords at all for 'subscriber' roles * Having them after the patch makes no sense either as next resource_from_item call will effectively remove them again (in PATCH) Change-Id: I0e8389e8ab34ad72f1b87a684daba77f1030f8ba	4 years ago
Rene Krenn	5e9066c4fb	TT#151751 refactor persisting subscriber webpassword ... a multitude of issues popped after introducing bcrypted webpasswords in the database. most recently the PATCH /api/susbcribers rail was reported to reset the webpassword unintentionally. subscriber login fails afterwards, which is a severe issue. the bugs are adressed by this refactorings. the change also introduces a global variable $NGCP::Panel::Utils::Auth::ENCRYPT_SUBSCRIBER_WEBPASSWORDS to control encrypting webpasswords. it is still enabled as of now, but it's worth to consider disabling it. there other ways to have a "cost" for an authentication request, eg. adding a simple sleep(1sec). Change-Id: I2d47d54a2d83568546ffdd2b211337a5f56be3a2	4 years ago
Rene Krenn	d786af1591	TT#147151 finish refactoring of avoiding JOIN billing.product ... follow up on TT#147151 (fast loading/paging/searching panel datatables), which broke restapi tests. Change-Id: I799cb9087b9405c71dec4c690e7a7bab5dfdbdde	4 years ago
Kirill Solomko	535c38cd93	TT#133800 fix webpassword, change subadmin expose password ... * webpassword is not correctly removed based on length, and remain visible when in plain-text or empty (unset) * config->security->password_(sip|web)_expose_subadmin now only affects subscribers under the same customer that are not this subscriber admin Change-Id: I329e0f1ad97dd513a33e3652ed03b4a43a95ed04	4 years ago
Kirill Solomko	3ef167575c	TT#133901 fix administrative flag for subscriberadmin ... * 'administrative' field is read only for susbcriberadmin role and that caused it to be removed from the final update 'resource', setting it to 0 if not existed. now the 'administrative' field is only changed in the database if it's defined in the 'resource'. Change-Id: I50738a77052c2163b19b2a42293c7a00e2780bc3	5 years ago
Kirill Solomko	c50b49db96	TT#133800 fix password fields behaviour in API ... * PATCH: password fields are not removed when resource is created for apply_patch(), they are removed under the same condititions later when hal is generated, that is to ensure that admin users without the 'show_passwords' flag as well as subscribers will not run into situation when they use PATCH and cannot apply it for "path": "/password" or/and "path": "/webpassword", as they were removed before apply_patch() * rework encrypted webpassword detection. webpasword is detected as encrypted if its length is 54 or 56 and it contains at least one '$' char, there is a chance for false positive detection when a user provides with a plain-text password with the same pattern but it's very unlikely, as well as since mr8.5 webpasswords are expected to be encrypted, and moreover worth case scenario is that the plain-text password will not be returned to the user Change-Id: I8ea739cbf728b2134f3ce00cee29da42ab3fb4a3	5 years ago
Kirill Solomko	2b144caf2c	TT#128605 implement dynamic fields expand ... * add API functionality to request additional data and expand fields in GET methods * syntax: - /api/resource/?expand=all - expands all expandable fields e.g.: customer_id field is expanded and customer internally is queried and returned under "customer" => {...} (the returned data is identical to what /api/customers/id would return) - /api/resource/?expand=reseller_id,customer_id - expands only reseller_id and customer_id fields, if they are expandable - /api/resource/?expand=reseller_id,invalidfield_id - returns the data and expands only fields that are expandable (reseller_id in this case) but if it finds either unknown fields or non-expandable fields, changes HTTP status code to "409 Conflict" * adapt all API endpoints to support dynamic expand fields expanding functionality, however the actual expand for them requires modifying the form fields in the following format: has_field 'contact_id' => ( element_attr => { expand => { class => 'NGCP::Panel::Role::API::CustomerContacts', id_field => 'contact_id', alias => 'contact', fetch => 0, }, }, ); - class - represents the class that should be used by the logic to fetch the relevant data - id_field - which field from the resource needs to be expanded, it should be the "id" field (subscriber_id, domain_id, etc.) - alias - (optional), under which key the fetched data is stored. the field name is used as the key if the option is omitted. - fetch - (optional), if the returned data is under $data->{contract_id} then it will be fetched from there and stored under the key (field name or alias), otherwise the whole retreived data is stored under the key (field name or alias) * adapt /api/autoattendants to use the new approach (old one was expand=1) * currently supported endpoints with expand: - admins - autoattendants - domains - customers - customercontacts - resellers - subscribers Change-Id: Iac53409dad944ed4794039a48dc3a9f6dce25bc1	5 years ago
Flaviu Mates	67ca4d5a00	TT#114703 Use config flags password_(sip|web)_expose_subadmin in api ... Change-Id: I289a3683c09c098d52c1ddb5a489c8c2093f9c99	5 years ago
Rene Krenn	a916bd3c4f	TT#116703 prevent entering duplicate alias numbers ... Change-Id: I3d35f87738bb6d73f62b300e487a113b45223ad1	5 years ago
Rene Krenn	8238204682	TT#105100 fix update case for webpasswd db encrytion ... Change-Id: I91de91430434edef744c6b3791749ea81053d1cb	5 years ago
Flaviu Mates	c50153b1f7	TT#99561 Introduce extension range preference for Customers ... * Limit subscriber's extension to a predefined customer extension range preference (both AP and api) Change-Id: I0b6ac5c24b3838f07cc561e7ee6b7cfabe69385e	5 years ago
Kirill Solomko	96ce7b1143	TT#101300 remove devid_alias field ... * dbaliases->devid_alias field is no longer in use and available via the UI/API Change-Id: I3ec1a82ef76822595aa37e94ec002d11a0cce58c	5 years ago
Rene Krenn	65634d7014	TT#84329 encrypt/decrypt password, weppassword, pin ... Change-Id: I74ae79b798955e6e20c32a644c739d81786f90c0	6 years ago
Andreas Granig	8f8e1d5fc9	TT#78557 - Use bcrypt to hash webpassword ... * Change the way webpassword is handled accross NGCP Panel UI/API to comply with new password encryption * At login, if password is not encrypted with high cost due to the ngcp-bcrypt-webpassword script, encrypt it with proper cost * Accept old password format as well until all webpasswords are encrypted Change-Id: Iefa9584a62ab4b7d2a224d10bdd415e9cbb8dfb5	6 years ago
Rene Krenn	a11f2bed5e	TT#77452 properly synchronize subscriber create/update ... acquire the billing.contract row lock *before* any unordered billing.voip_numbers rowlocks by sub manage_subscriber_numbers(). - "deadlock" waittimeout errors will cease when creating subscribers concurrently via api - max_subscribers, is_pilot and other per-contract constraints will be respected accurately Change-Id: I73bb7525b327bbb09217b790be9c14cc65ddebcc	6 years ago
Rene Krenn	bfd2b4b5b1	TT#73364 batch provisioning ... implementation of config.yml-based "provisioning templates" for creating a subscriber setup with one click or batch-wise (.csv upload), currently comprising of: - contact - contract - contract prefs - subscriber - subscriber prefs - trusted sources - registrations Change-Id: I0c3c1ef93bbe0ca926bbe906a35346bef5094fac	6 years ago
Flaviu Mates	a4265a91e0	TT#69503 - Change subscriber's domain assignation logic ... * Only allow to assign domains for subscribers from the same reseller as subscriber's customer * Change Subscribers.yaml test to retrieve a domain for testing from the same reseller as subscriber's customer Change-Id: I6c7cc7a9874207cfcd63360a6f87e2fd6841011c	6 years ago
Flaviu Mates	42473db9d7	TT#64751 - Allow subscribers to change their password ... * Add specific check for non-pbx subscriberadmins Change-Id: I133d1f77fac54ef77182661f3d8eeee71768a4e3	7 years ago
Kirill Solomko	ce664263b2	TT#65101 add ccareadmin ccare roles ... * ccareadmin and ccare roles have full access to Customers, Subscribers and their preferences/settings, and read-only access to BillingProfiles,InvoceTemplates, EmailTemplates * ccare role is restricted to the related reseller Change-Id: I6cf7d3adf912f0fa98d1ef5c02abea2f4331ec4b	7 years ago
Flaviu Mates	6f8a1c8be0	TT#64751 - Allow subscribers to change their password ... * Change permissions to allow subscribers to change their own password Change-Id: Ieee9d1b3d16cab6ce0313491b4d65574039034f3	7 years ago
Flaviu Mates	603351c2d5	TT#59024 - Disable web_password and sip password fields ... for administrators with disabled 'show_passwords' * Add check in master data edit form to disable web_password and sip password fields. * Add check for api GET to not show passwords Change-Id: Icf95cddc07982a698c893661b529e7542002ec60	7 years ago
Rene Krenn	dda5ffe48f	TT#56340 fix subscribers.t not matching obfuscation chars ... Change-Id: I7023d1e23fc397e3556705fc52db50eace6c02b4	7 years ago
Rene Krenn	cae5270af5	TT#56340 panel logline obfuscation ... adds gdpr obfuscation quoting for: + subscriber numbers + subscriber ip addresses + subscriber usernames + any logmessage "DATA": query parameters, form data, response data + subscriber uuid's + call id's + callforward sip uri's the quoting is centralized by $c->qs() ("quote sensitive"), using catalyst plugin mechanism. escape symbols are set to « (\x{ab}) and » (\x{bb}). generate_logfile_data_inventory.pl was modified to mark loglines with "gdpr affected" status, if $c->qs() was used in a log message. Change-Id: I0f42d7992594232ae33e5666b0a64009211c5b76	7 years ago
Flaviu Mates	d59e4c97e2	TT#58570 Fix subscriberadmin PATCH ... * Store provious value for admin flag for subscribers because form field validation was deleting it because it is read-only * Fix DB query that was causing error Change-Id: Ib73b76c2b912c687da1b1c9ea346541b0b32c3a9	7 years ago
Andreas Granig	d5ba7c11cb	TT#52989 Implement device alias provisioning logic ... Change-Id: I7dc14a2f8fdcfa507a5e46872421923b44f133bc	7 years ago