928 Commits (aaaa6c1ef7ebab0cc807b4c0fb26845dbfbe3d38)

Author	SHA1	Message	Date
Marco Capetta	b72107bf7d	MT#62546 Fix permanent registration nat flags ... Due to changes in how kamailio manage the NAT contacts and the corresponding natping, the flags that the API or web interface sets while a permanent registration is created have to be updated. Before: * Default cflag is set to 0 (natping disabled) * If nat option, then cflag is set to 64 New: * Default cflag is set to 256 (netping disabled) * If nat option, then cflag is set to 128 Change-Id: I39b1d7a697ef72267fd8a05edf0552d2c1403733 (cherry picked from commit 1be3f5a257)	2 months ago
Rene Krenn	6600d7c949	MT#53706 enable_2fa option for /api/admins ... Change-Id: I7de37482fed913551fc0a1c9b18f789b51b67a05	2 months ago
Rene Krenn	82af5cf518	MT#62283 "for update no lock" logic for GET /subscribers and /contracts ... to unblock the web UI on heavy loaded systems with call rating enabled, "select .. for update no lock" will no longer wait for acquiring locks when retrieving subscriber or contract contract pages. the contract_balance record creation is will be skipped for such contracts that are locked elsewhere. for rails such as /customerbalances, an explicit contract id row lock timeout is introduced, so it does not depend on the mariadb wait_timeout. Change-Id: I6e96342f3f761279a5876c871d2477977823a39e	3 months ago
Kirill Solomko	24d28676a1	MT#62289 /api/pbxusers add username and domain fields ... * username represents provisioning.voip_subscribers.username * domain represents provisionoing.voip_domains.domain Change-Id: I98f4043f9b044a7829603f2a350d1548be935ff1	3 months ago
Kirill Solomko	e8f41aba50	MT#62223 improve error response handling when log_response is off ... * log_response: 0 is reworked, it now always logs the response but in case if it's off the body is not logged. * adjust error logging, the error in the body is correctly logged in the response even if log_response: 0 Change-Id: I6dfddeccffb5464b830133235bf63c5e82ae6073	4 months ago
Kirill Solomko	cbc4e57d45	MT#62223 API unhandled internal errors cause 500 error ... * unhandled internal errors (such as DBI error) are now properly returned as 500 Internal Server Error instead of 200 OK Change-Id: Iabc80cd200c9091dd249bc90aa586d5d7b616b69	4 months ago
Kirill Solomko	97e457c92b	MT#62178 fix external_id, profile auto removal for subscriber roles ... * fix 'subscriber' and 'subscriberadmin' roles using PATCH to cause external_id, and profile fields removed. * remove profile_id field from the 'subscriber' and 'subscriberadmin' form Change-Id: Id8bb068ca7fcac2c0c5954f2ed79e841d18ac398	4 months ago
Kirill Solomko	ef4442feeb	MT#61805 add API phonebook support for top level inclusion ... * /api/customerphonebookentries now support "include" query param that can be either "include=all", "include=shared" or "include=reseller". With "include=all" - shared and reseller entries are included and merged as customer > shared > reseller. With "include=shared" - shared entries are included and merged as customer > shared. With "include=reseller" - reseller entries are included and merged as customer > reseller. * /api/subscriberphonebookentries now support "include" query param that can be either "include=all", "include=customer" or "include=reseller" With "include=all" - customer+shared+reseller entries are included, and merged as subscriber > customer > shared > reseller. With "include=customer" - customer+shared entries are included, and merged as subscriber > customer > shared. With "include=reseller" - customer+reseller entries are included, and merged as subscriber > reseller. * Shared numbers are considered belonging to the customer so when on the customer level the same number is provided, it takes precedence. * It is not possible to edit other level included numbers (they have composide ids). * It is possible to access an included number directly by the id. Change-Id: Ice06bc0961d42a9b64db59aa93029d7505b4805e	4 months ago
Kirill Solomko	e66c568efc	MT#61803 add /api/pbxusers endpoint ... * this endpoint is designed to provide with basic subscribers info such as display_name, primary_number, pbx_extension and only available for customers with product class 'pbxaccount'. * only GET methods are allowed for this endpoint. Change-Id: I181b863e2c3e5fad34c03a291f64fd7ddb587702	4 months ago
Richard Fuchs	b2b5600c16	MT#61630 invalidate rtpengine cached files ... Add class to make RPC calls to rtpengine. Add invokations to clear rtpengine audio cache where appropriate, whenever a file stored in DB is deleted or changed. Change-Id: I3660f9f67dfb0c68c1af80a5439982046be3b6f6	5 months ago
Rene Krenn	de0595c089	MT#59447 admin user LDAP authentication #1 ... Change-Id: Ic31236d933f022b567c74998959267ef9a549b7e	7 months ago
Rene Krenn	598f7dbaf4	MT#61513 prevent duplicates in GET /api/callrecordings ... callrecording metakeys such as "uuid" for the involved call party is present multiple times by nature. the ?subscriber_id= query param will then narrow down the result. alongside an implicit role filtering (reseller/subscriber), the sql join and filtering for the uuid key is present 2 times however, and multiplied records slip through. this can be prevented with DISTINCT. to avoid performance impacts and consider other metakeys orrcuring multiple times, it will be applied only if a filter is present. Change-Id: I08047aaee265e2e0a706220e03fd4617da16d33c	7 months ago
Kirill Solomko	a7589aeba8	MT#60877 /api/preferencemetaentries accept default_val as boolean ... * default_val is now correctly accepted as either 1/0 or true/false * fix 500 error when default_val is provided as JSON true/false Change-Id: Id6db9c13ae458ddd61e5a68865249eb4e3e124bf	9 months ago
Kirill Solomko	bbd44f16fa	MT#60858 add /api/passwordchange endpoint ... * the new endpoint accepts new_password fields and enables for authenticated user a mechanism to quickly change their password * the global password validation rules are enabled * returns 204 No Content * if user's password is expired, the endpoint is the only accessible for the user to change the password and unlock other endpoints. * a few fixes in validate_password() to correctly fetch provisioning subscriber for password change scenarios and webpassword field Change-Id: I906fcfe5c780b850d322b46b445b54c054767673	9 months ago
Kirill Solomko	b97be4502d	MT#60709 add PBX expand by device_id,profile_id,config_id ... * device_id, profile_id (device related), config_id are now expandable in the respective /api/pbxdevice* endpoints. * fix expand collection logic to avoid ambiguous 'order by id'. Change-Id: I0b1f4b3da093fb04a30e5097881af6131c1afe46	10 months ago
Kirill Solomko	b041888807	MT#60656 add max_age password validation support ... * UI: password are now validated against $c->config->{security}{password}{web_max_age_days} (unless it's 0) and if the password is expired the user is redirected automatically to /changepassword page, and after successful password change back to the original page. * API: if password is expired all API requests will be returning 403 Forbidden "Password expired", except PUT/PATCH to /api/admins or /api/subscribers with the new password in place. * successful login on the UI now redirects to /dashboard instead of / (to prevent unintended redirect to v2) Change-Id: I075f8e17cc9b0658d6b3b3d526ca5b379d050ce4	10 months ago
Kirill Solomko	e2e1776090	MT#60558 move max license checks for POST /api/subscribers ... * the relevant max license checkes are moved from Controller::API::Subscribers::POST to Utils::Subscribers::prepare_resource because there we fetch customer_id from the pilot subscriber in case of pbx subscriberadmin requests that is neccessary for PBX subscribers max license check. Change-Id: I2d1c212d73fe5b9295d1595b4fffebeb67b61e5a	11 months ago
Kirill Solomko	65ae07a97b	MT#60601 obfuscate password/webpassword in API valdiation errors ... * plain text passwords are obfuscated in validation errors returned by API Change-Id: Iff989696ce09faff34692eaa129aee6981340a97	11 months ago
Kirill Solomko	60fa23cb68	MT#60585 add user ban support ... * users for admin/subscriber realms are now banned if failed to login X amount of times (UI/API). * rework Redis connection and it's now a Catalyst plugin NGCP::Redis accessed by $c->redis_get_connection({database => 19}), the connection per database, per worker process is established only once and then reused (with auto built-in reconnect support). * remove Utils::Redis.pm as it does not have any code/logic anymore. * ban values are taken from $config->{security}{login} as - ban_enable: 1 - ban_expire_time: 3600 ban expire time in seconds - max_attempts: 5 * if max_attempts set to 0, the ban functionality is disabled as it requires to be at least 1 to work. * upon successful login or ban, the failed attempts counter is removed * the failed attempts counter is also removed automatically with the expire time equals "ban_expire_time" or otherwise 3600 seconds. * user bans are logged into panel.log * banned user receives exactly the same return page/codes as per invalid logic. Change-Id: I05cc68c623ee289488fc64f1af50527004dcaae1	11 months ago
Kirill Solomko	d9f283cbc8	MT#59449 enhance password validation and handling ... * passwords are now validated based on - minlen - maxlen - min lower case chars - min uppper case chars - min digits - min special chars * Data::Password::zxcvbn is used to calculate password score and reject passwords with score < 3 as weak (this library is ported from the Dropbox password validation) * Add password journals and check last used passwords in the journals * Improve password generator javascript function to generate a password with at least 4 of each of the char group types. * Currently affected are subcriber and admin entry creation or modification via UI/API * NGCP::Utils::Auth add optional bcrypt_cost support as last argument for generate_salted_hash and get_usr_salted_pass Change-Id: I100c25107d91741d5101bc58d29a3fa558b0b017	11 months ago
Kirill Solomko	9d021be65a	MT#59979 add license control ... * UI and API parts are now under license control * new Util::License::get_license($c, $name) - fetches license status by name (1 if enabled, and also if /proc/ngcp/check if 'ok') * add Catalyst::Plugins::NGCP::License with license($name) to fetch valid license by name from anywhere using $c->license('pbx') or from the templates using c.license('pbx'). It internally uses Util::License::get_license($c, $name) * License::get_license_status($c) now requires $c as first argument as well logs license status check errors. * new ActionRoles::License that enables usage of :Does(License) RequiresLicense('pbx') LicenseDetachTo('/denied_page') in the Controller chains * Add license control for UI elements and return 403 Forbidden if a resource is covered by licenses and the license is not active * Hide UI elements if a license is not active * API/Entities/Entities new $c->set_config key: - per endpoint: $c->set_config({ required_licenses => [qw/pbx device_provisioning/] } - or per method: $c->set_config({ required_licenses => { POST => [qw/pbx device_provisioning/] } } } * In case if an API endpoint does not have a license: 403 Forbidden "Invalid license" reply is returned. * Add license based restrictions to API endpoints * /api documentation: - completely hide endpoints that do not have an active license - hide only methods that does not have an active license Change-Id: Iba45fc5068b02306a617fed7b5405f2210574b61	11 months ago
Rene Krenn	9c103302c8	MT#60575 dynamically load provisioning templates module ... Change-Id: Ibff509825f82a75c9b0221505a1673d78b401989	11 months ago
Kirill Solomko	524a264850	MT#60498 add search by not_null, change null param ... * add search by not_null for fields that ends with _id and integers, that adds "IS NOT NULL" to the SQL search query by the field. * null and not null values are expected now as $null and $not_null to avoid conflicts when user searches by null strings that may be a valid case. Change-Id: I8e8b8c9060e985dfe2b94cbfcca1587f05477fe9	11 months ago
Kirill Solomko	f72f8291a7	MT#60393 /api/timesets fix reseller_id validation ... * fix reselelr_id for 'reseller' role and it's now automatically set to the user's reseller_id instead of throwing a validation error Change-Id: Ic732366aea6c4106c37961e599cd1e40fdaba5b2	12 months ago
Kirill Solomko	18597efb87	MT#60378 accept null same way as NULL for search ... * search by value null is internally translated to undef to say search for records without a value, same way as it already works for NULL value, just null is more expected and easier to use by clients as it's how its represented in the resource Change-Id: Ia8a75bad95a34dd8167162d0f09a1ec7c4056105	12 months ago
Kirill Solomko	6543743992	MT#60162 improve expand functionality for collections ... * in general expand collection now performs only 1 sql query to per expand field to fetch all items by the ids instead of fetching them for every single collection item, that should significantly increase performance in case of large databases and reduce work for the database * introduce $c->stash->{expand_cache} that contains cached data for the expand fields to avoid multiple same calculations * expand_field() and expand_field_data() have been reworked to support expand_cache * new method expand_prepare_collection() is called for all API GET collection methods before preparing resource hal fields, to change the expand logic to only cache the data instead of fetching it from the database * new method expand_collection_fields() that is called in all API GET collection methods after the @embedded data is prepared to finalise the expand collection fields * for expand collection there is only 1 SQL request per expand field that fetches all items -in [ids to expand] and then the expand_collection_fields() uses the cached items_by_ids with O(1) fetch from the cache by id Change-Id: Ie7c6115472878febf0d8c9b4d833f5c23b15c78b	12 months ago
Kirill Solomko	9a2d66a00a	MT#60238 /api/customerpreferences allow subscriberadmin ... * subscriberadmin roles can now handle customer preferences belonging to the same customer and only those with expose_to_customer = '1' Change-Id: Iae9ab5d4a96a065b1a627d180dd523e805d954f3	1 year ago
Kirill Solomko	f81481ea66	MT#60236 disable mailtofaxsettings active state checks ... * disable active state checks for subscriber and subscriberadmin roles in mailtofaxsettings because they can now change it similar to how they could do it in the old csc. Change-Id: Ica8039e83d2acc5e162c7902afcd4b97ac1c5b6d	1 year ago
Kirill Solomko	0d8d68d9a5	MT#60236 /api/mailtofaxsettings allow subscriber roles ... * subscriberadmin and subscriber can now use /api/mailtofaxsettings * adapt item_rs query for subscriberadmin and subscriber so that subscriberadmin can have access to all 'own' subscribers Change-Id: If8e768c5c06bc4e5f0a6ef9d15e19f542d8b6a4d	1 year ago
Kirill Solomko	6926930b3a	MT#60227 handle Accept: application/json header ... * if Accept: applicaton/json header is provided for API request - the response's Content-Type header is set to application/json. The return content in this case is preserved as HAL but it's helpful for the clients that only expect aplication/json response Content-Type Change-Id: I8c60da4df7d639f45953d8ace699a59fefc88ddf	1 year ago
Kirill Solomko	f2478178f0	MT#60119 allow pbxfielddeveicepreferences to subscriberadmin role ... * subscriberadmin role can now CRUD preferences for devices that are assigned to the customer. Change-Id: I09fe3c3b2ed670f0411970dc6402486a3cf6d4be	1 year ago
Marco Capetta	42f8747a6f	MT#59749 Add use_redirection flag to Call Forwards configuration ... This new flag controls wether the CF is processed as usual (flag with value 0, default) or generates a 302 redirect message back to the caller (flag with value 1). The implementation cover both the UI and the API. Change-Id: Idf945262e17de0d77bb612101d268fd6ea7a309e	1 year ago
Michael Prokop	c8766cd567	MT#48844 Minor typo fixes ... s/paramaters/parameters/ s/wihout/without/ s/If it mathes/If it matches/ s/ in in / in / s/the the/the/ (note: fixes translation issue which was introduced with commit 2b7a1a33 and fixed in 072e897c for the original, but missed for the german translation) Change-Id: I4186bf3f42b1fac11bb7d7fe801f860d2d59adc1	1 year ago
Kirill Solomko	e4cda08bd4	MT#59835 /api/billingzones fix HAL ngcp:billingfees relation ... * billing fees relation was rendering all associated billing fees and that was causing huge data sets to be returned if a billing zones had many fees, now instead a single link is returned /api/billingfees/?billing_zone_id=:id. this is also on par with other links like /api/billingfees/:id Change-Id: Iabb7afb3bf43abf9a22c71750d45a1962a7f16ec	1 year ago
Kirill Solomko	c006e9efc1	MT#59806 add missing cfdestinationsets HAL link type ... * add missing: "_links" : { "ngcp:cftimesets" : [ .... ], in the collection representation Change-Id: I59fade61ae2679833c43fd5501d14ca45a376572	1 year ago
Kirill Solomko	b2a1bc00d1	MT#59806 adapt /api/cfdestinationsets to Entities ... * Entites logic is used for GET/POST/PUT/PATCH/DELETE * Utils::CallForwards::check_destinations() do not obfuscate destinations that are returned in the response as all logged data is obfuscated anyways Change-Id: Ia79f9e236c966410e2640d719c3a7f5784cc4c2a	1 year ago
Kirill Solomko	1872f342a0	MT#59478 Entities/EntitiesItem more $c->has_errors checks ... * additional $c->has_errors checks for post/patch/put/delete to cover all possible expected called inherited methods scenarios and returns. Change-Id: I0e09ba62bcaa51d582315d9cbb672a4cf11bacf0	1 year ago
Kirill Solomko	9e3653341a	MT#59478 adapt /api/cftimesets to global transactions, more fixes ... * /api/cftimesets now fully use Entities/EntitiesItem * EntitiesItem: delete(): fix delete_item() expression processing * Entities/EntitiesItem: post/put/patch/delete: go to TX_END in scenarios where after a method call (e.g. update_item()) there is a normal return from the function but errors in $c->error, so that they are also caught correctly Change-Id: I3bef409ded590796c2bba4f30acd28b02e99065b	1 year ago
Kirill Solomko	91f1dcf37e	MT#59478 /api/ root error handling ... * clear errors in Controller::API::Root::end so that they are not interfere with generated ones * API::error(): add $c->stash->{is_api_error_response} = 1 to track it in log_response() * API::log_response(): distinguish and correctly correct errors for cases when is_api_response == 1 and also when there were $c->error('error') calls in the code, to log them correctly in both cases Change-Id: Id922b8219832c1b99815d9d608309035b9b25cff	1 year ago
Kirill Solomko	4e2e141308	MT#59478 Entities fix non-existing $item handling in put/patch/delete ... * the error produced by item_id_is_valid() is returned instead of a new one is generated * fix incorrect expression on delete_item() in delete() Change-Id: Ic8e4e24c70bbcd99ec414d901eb2842ee39ae7de	1 year ago
Kirill Solomko	bf249f9d96	MT#59797 /api/headerruleconditions (actions) fix reseller role ... * fix resultset when user role is reseller so that reseller_id is joined correctly and a db error is not produced anymore. Change-Id: I2a1b357037d983f23770bb59519fc2cb8b68a7e4	1 year ago
Kirill Solomko	9d598d4502	MT#59478 improve MSG, LOG log string data storing ... * $c->error array now contains the $message as the first element instead, so that it's possible to obtain all the error data in the code when fetching it from @{$c->error}. the first element is not logged in the error log. * api_response $c->response_body part is now stored in MSG= and possible errors / other log data is now stored in LOG= to: - reduce amount of log lines when an API error response occurs from 2 to 1 - the message part usually contains either HTTP response message (e.g. Internal Server Error) or a validation message string, so it belongs to the MSG= part of the log line, where as the internal log data is more related to the LOG= part - both MSG= and LOG= parts are escaped for GDPR related obfuscation * Utils::Messag::info(): $msg is now also obfuscated if it's detected as a reference (also because logging is moved for the API part to $msg), as well as truncated for possible new-line char and white-spaces. Change-Id: I3b670b2251ec3060037ed6863f18d95975120b8d	1 year ago
Kirill Solomko	9dbd6b0e88	MT#59478 /api/contracts address $now and hal_from_contract ... * the following changes resolve an issue where $now in Role/Contracts resource_from_item appeared as an object and failed subsequent logic: - hal_from_contract is renamed to hal_from_item to match correctly the parent method - call hal_from_item and pass $now string there correctly as $params { now => $now } Change-Id: I5c02d5f7df5d9000550505ad120b9531f87e8d65	1 year ago
Kirill Solomko	205b27a267	MT#59478 API better transaction and error handling ... * Role/Entities*: POST/PUT/PATCH/DELETE methods changes: - support deadlock detection and transaction retry (2 retry attempts at the moment) - improve transaction control, use local $guard instead of saving the ref to $c->stash, as in that case it went out of scope too late and also reported an error message into the log about abnormal $guard out of scope interruption - move all non transaction related code outside of the scope - add error handling when methods such as update_item, and a like do not return the expected data, instead of simply going out of scope and resulting in an uncontrolled reply * Role/API: - rework transaction control: + get_transaction_control() is renamed to start_transaction() to better reflect what it does + complete_transaction() is renamed to commit_transaction() + remove unused %params arg + pass $guard into commit_transaction() instead of having it stored as $c->stash->{transaction_guard) that caused the $guard ref to be destroyed much late than expected (there was also a typo as transaction_quard, which is not relevant anymore with the changes + add check_deadlock() that is invoked when an exception is caught or an $c->errors contain an error, and if the error message represents a transaction error, the transaction block is re-invoked via "goto TX_START" - rework error(): + it now accepts args as following: ($self, $c, $code, $message, @errors) # code -> returned as HTTP code in the reply # message -> returned as HTTP message in the reply # errors -> contain errors for internal logging, last element often contains a DBIx exception + populates all @errors into $c->error so they are available on demend in the code via $c->error or $c->last_error + $c->log->error is not invoked now as the errors become printed in log_response() - log_response() now prints collected errors from $c->error correctly as a separate log line, that is alike to the other api logs so that those can be looked up by the request's tx_id, also all errors are now printed only into api.log * Adjust all $self->error() calls in catch($e) to include $e as the last argument, as well as the duplicate $c->log->error is removed from those ocassions * Remove all $c->log->error() calls as they are replaced with either $self->error() (that logs it correctly into api.log) or $c->error('err') that also adds it correctly into api.log * API::CallForwards: rework to use Entities/EntitiesItem * API::Contracts: rework POST to use Entities * API::PeeringGroups: rework POST to use Entities * API::SubscriberRegistrations: rework POST to use Entities * API::RewriteRuleSets: improve create_item() functionality * Utils/Message: add 'api_retry' log type * $c->session->{api_request_tx_id} is changed to $c->stash->{api_request_tx_id} because sometimes the session ref is different and a different tx_id becomes used Change-Id: I633ce7a8047b1bf00a2f6889003088edf0825dcd	1 year ago
Kirill Solomko	aef4c89197	MT#59662 enhance subscriber assigned header rules/sets handling ... * POST /api/headerrules now supports either 'set_id' or 'subscriber_id'. When 'subsriber_id' is specified, a subscriber assigned header rule set is automatically created if it does not exist, as well as automatically removed when its last header rule is deleted. * It is now possible to directly GET and DELETE /api/headerrules/:id where :id is a subscriber assigned header rule. * It is now possible to directly GET and DELETE /api/headerrulesets/:id where :id is a subscriber assigned header rule set. * Improve /api/headerrules data validation and duplicate header rule detection. * It is now possible to expand by /api/headerrules/?expand=set_id field Change-Id: I681bc61c2eed47a8e54847f07f31134f643930c2	1 year ago
Rene Krenn	868e2657b6	MT#59597 fix remove trailing+leading % entirely for "wildcard" params ... Change-Id: I371e2d41ff27b2801ead391778d9a2bdef914563	1 year ago
Rene Krenn	9ba93e69a5	MT#59597 remove trailing+leading % entirely for "wildcard" param types ... this code is not run by legacy panel, so the check for "/api" request can be removed. Change-Id: I29410fbbf046a8f00443138720b940de0f09f416	1 year ago
Rene Krenn	aa6760228e	MT#59363 retain trailing+leading wildcard for legacy panel only ... all api query param filters that were harmonized to "wildcard" also appended leading+trailing wildcard (this was from the original idea to make adminUI and legacy panel search behave the same). Change-Id: I017473b84f2844d14a11b4ec8b437df5335c24d0	1 year ago
Rene Krenn	740bb764e5	MT#59434 always take aliases from voip_numbers ... in case the DB shows discrepancy because of missing primary provisioning.voip_dbaliases records, align with legacy panel behaviour and also take the restapi aliases list from voip_numbers, instead of voip_dbaliases. .. while trying to keep the performance gain from https://gerrit.mgm.sipwise.com/c/ngcp-panel/+/65086. Change-Id: Ibd7f8c8bc6a39ae2c31b4e8818080674ab77d66c	1 year ago
Rene Krenn	c837dc1043	MT#56817 MT#56362 fix query param sorting for regulary chained rs ... Change-Id: I82d3a0ce67ffb61273fc716016885f6431852eff	1 year ago