764 Commits (205b27a2676efa51df1ee51a6731c7fe9942422a)

Author	SHA1	Message	Date
Kirill Solomko	205b27a267	MT#59478 API better transaction and error handling ... * Role/Entities*: POST/PUT/PATCH/DELETE methods changes: - support deadlock detection and transaction retry (2 retry attempts at the moment) - improve transaction control, use local $guard instead of saving the ref to $c->stash, as in that case it went out of scope too late and also reported an error message into the log about abnormal $guard out of scope interruption - move all non transaction related code outside of the scope - add error handling when methods such as update_item, and a like do not return the expected data, instead of simply going out of scope and resulting in an uncontrolled reply * Role/API: - rework transaction control: + get_transaction_control() is renamed to start_transaction() to better reflect what it does + complete_transaction() is renamed to commit_transaction() + remove unused %params arg + pass $guard into commit_transaction() instead of having it stored as $c->stash->{transaction_guard) that caused the $guard ref to be destroyed much late than expected (there was also a typo as transaction_quard, which is not relevant anymore with the changes + add check_deadlock() that is invoked when an exception is caught or an $c->errors contain an error, and if the error message represents a transaction error, the transaction block is re-invoked via "goto TX_START" - rework error(): + it now accepts args as following: ($self, $c, $code, $message, @errors) # code -> returned as HTTP code in the reply # message -> returned as HTTP message in the reply # errors -> contain errors for internal logging, last element often contains a DBIx exception + populates all @errors into $c->error so they are available on demend in the code via $c->error or $c->last_error + $c->log->error is not invoked now as the errors become printed in log_response() - log_response() now prints collected errors from $c->error correctly as a separate log line, that is alike to the other api logs so that those can be looked up by the request's tx_id, also all errors are now printed only into api.log * Adjust all $self->error() calls in catch($e) to include $e as the last argument, as well as the duplicate $c->log->error is removed from those ocassions * Remove all $c->log->error() calls as they are replaced with either $self->error() (that logs it correctly into api.log) or $c->error('err') that also adds it correctly into api.log * API::CallForwards: rework to use Entities/EntitiesItem * API::Contracts: rework POST to use Entities * API::PeeringGroups: rework POST to use Entities * API::SubscriberRegistrations: rework POST to use Entities * API::RewriteRuleSets: improve create_item() functionality * Utils/Message: add 'api_retry' log type * $c->session->{api_request_tx_id} is changed to $c->stash->{api_request_tx_id} because sometimes the session ref is different and a different tx_id becomes used Change-Id: I633ce7a8047b1bf00a2f6889003088edf0825dcd	1 year ago
Kirill Solomko	aef4c89197	MT#59662 enhance subscriber assigned header rules/sets handling ... * POST /api/headerrules now supports either 'set_id' or 'subscriber_id'. When 'subsriber_id' is specified, a subscriber assigned header rule set is automatically created if it does not exist, as well as automatically removed when its last header rule is deleted. * It is now possible to directly GET and DELETE /api/headerrules/:id where :id is a subscriber assigned header rule. * It is now possible to directly GET and DELETE /api/headerrulesets/:id where :id is a subscriber assigned header rule set. * Improve /api/headerrules data validation and duplicate header rule detection. * It is now possible to expand by /api/headerrules/?expand=set_id field Change-Id: I681bc61c2eed47a8e54847f07f31134f643930c2	1 year ago
Rene Krenn	48b71e68c5	MT#59363 fix /api/domains/?domain= filter pattern behaviour ... this filter is another spot, where leading/trailing wildcards were added unintentionally, during harmonization. Change-Id: Id23114f8e7ebca018ad222e35b39d8260f1da92f	1 year ago
Kirill Solomko	80f8b1da0b	MT#59238 fix rewrite rules transaction control for PATCH,DELETE ... * /api/rewriterules PATCH, DELETE methods used the generic transaction, which conflicted with the xmlrpc reload command. * own_transaction_control was only set for PUT and now also extended to PATCH, DELETE to ensure that dialplan_sip_reload() is called properly after $guard->commit. Change-Id: I7313b4f6d1f6699bf38ae34749c5383a3943f61d	1 year ago
Rene Krenn	672a3110bc	MT#59216 restore appending wildcard behaviour ... with the change to harmonize wildcard symbols % and * to * in query param filters across all restapi rails, only - /api/subscribers?alias= - /api/subscribers?pbx_extension= - /api/subscribers?primary_number= - /api/pbxdevices?pbx_extension= are supposed to include implicit leading an trailing wildcards. panel UI uses trailing wildcard by default everywhere, and so should the databales of the new Admin UI. external_id were used to support patterns, but must not include a trailing wildcard in rest API, as this can break logic that rely on unique api request results. we now therefore no longer append wildcards in particular to: - /api/voicemails?folder= - /api/voicemailgreetings?type= - /api/subscribers?subscriber_external_id= - /api/subscribers?customer_external_id= - /api/subscribers?display_name= - /api/subscribers?domain= - /api/profilepackages?profile_name= - /api/profilepackages?network_name= - /api/pbxdevices?display_name= - /api/faxserversettings?name_or_password= Change-Id: I40ef1912d7224a56c6d14619d258c70e447ab23d	1 year ago
Rene Krenn	a15d00894a	MT#56817 MT#56362 fix errors from nondetermistic query param ordering ... the /api/callists rail supports the "call_id" query parameter to match calls with a callid prefix. this filter also adds an implicity ordering ORDER BY length(call_id) ASC, 'start_time' ASC. the /api/callists api rail also requires the query parameter "susbcriber_id", which renders a (fast) compound SQL query to list matching incoming (query1) and outgoing calls (query2) using UNION ALL (instead of a slow OR query). query1 UNION ALL query2 this is supported by the https://metacpan.org/pod/DBIx::Class::Helper::ResultSet::SetOperations module, which however generates invalid SQL syntax if query1 or query2 contains an ORDER BY. this is exactly what caused the 500 error of the customer who applied both "call_id" and "susbcriber_id" parameter in the request at the same time. ... ORDER BY length(call_id) ASC, `start_time` ASC UNION ALL SELECT `me`.`id`... the error happens randomly, because query parameters are stored in a hash (which by definition has no deterministic order of the entries). when the "call_id" parameter is applied at last, it worked as expected. so the issue can be adressed by strictly ordering the UNION ALL result, and prohibit ORDER BY in query1 and query1. the latter was added already with commit b2dfe28eed, and could be hotfixed. however, the ORDER BY of "call_id" query param is lost, and should be properly handled by forcing strict ordering of how query parameters are applied. the fix will introduce paramater ordering according to their order of definition in the code. Change-Id: I165d341b5c20e9bb750bd1fba88c836b393e80bd	1 year ago
Kirill Solomko	a4d54b10e4	MT#58709 rework NGCP::Utils::Sound::get_handlers_rs ... * get_handlers_rs() is renamed into get_file_handles() and reworked from an indented 'from' resultset into 2 separate resultsets, one fetching all the handles and another one files for the sound set, it is now returned as an array with the similar data. Reason for this rework is the deprecated use of 'from', that resulted in a lot of warnings in panel-fcgi.log: DBIx::Class::SQLMaker::ClassicExtensions::_join_condition(): ResultSet {from} structures with conditions not conforming to the SQL::Abstract::Classic syntax are deprecated: you either need to stop abusing {from} altogether, or express the condition properly using the { -ident => ... } * adjust file_handles data key names to better follow the naming conventions and readability, e.g.: groupname -> group_name, fileid -> file_id Change-Id: Ia91c9104c0b4dc79ee5bc359eb0b6e035e56ef3e	1 year ago
Kirill Solomko	b6b453d69d	MT#59001 improve phonebook csv upload ... * the expected csv fields are as following: - /api/resellerphonebookentries/?reseller_id=1 name,number - /api/customerphonebookentries/?customer_id=1 name,number - /api/subscriberphonebookentries/?subscriber_id=1 name,number,shared - /api/resellerphonebookentries/ name,number,reseller_id - /api/customerphonebookentries/ name,number,contract_id - /api/customerphonebookentries/ name,number,shared,subcriber_id (note: the fields are taken from the leftmost side as the minimum required fields, so if one provides as: - /api/resellerphonebookentries/?reseller_id=1 name,number,reseller_id this line will be uploaded and columns after 'number' are ignored in this case. this is for convenience if you have a prepared upload for different customers/resellers for instance, you can upload it under one customer/reseller when needed * user access scope is now validated, e.g.: resellers cannot upload phonebook entries to another resellers, etc. * fix csv upload for /api/customerphonebookentries/?customer_id=1 Change-Id: I970ac36fdb8b366adff15515a48d296e8abeff13	1 year ago
Rene Krenn	1e9f223276	MT#59025 fix peering groups name filter ... Change-Id: I329e644c937058069c5932abf8e897dce48a8c10	1 year ago
Kirill Solomko	1dabb6051d	MT#58989 /api/customers add type query param ... * add 'type' query param to filter by customer type 'sipaccount' or 'pbxaccount' Change-Id: If372cc69cf16c92551824504da7695a65dc867b3	1 year ago
Rene Krenn	cc8fd8928d	MT#58890 force cft_ringtimeout as number ... Change-Id: I4df4fbf6522d28a30731c87a36c95ae8350830fa	1 year ago
Rene Krenn	2101e2246d	MT#58936 fix customerzonecosts resourcename ... Change-Id: I22b052bc6931e5426e1aa982396c446cb1c67883	1 year ago
Rene Krenn	853bc9f535	MT#58936 direction filter param for /api/customerzonecosts ... Change-Id: Ie831381576ffdb2a08375cf73bc66c1c5e6d2076	1 year ago
Kirill Solomko	462bfedc43	MT#58861 /api/headerruleconditions accept values as a list of strings ... * {"values:", ["value1", "value2"]} is not accepted by /api/headerruleconditions * the previous behaviour of {"values": [{"value": "value1"},{"value": "value2"}]} is preserved Change-Id: I7d9540d20ed0ecbb2712f1905a7d9e6f7af7e06d	1 year ago
Kirill Solomko	08fb96c5a6	MT#58813 adjust header rule conditions/actions filters ... * remove filter by subscriber_id as it's not possible there * add filters by rule_id, set_id Change-Id: I6dac0a24dad56564dbe2ed648f4507888e0a5c94	1 year ago
Rene Krenn	aa2d07ef29	MT#58762 adjust restpi tests to harmonized wildcard filters ... Change-Id: I68b2199a3487fedb388803012c950c3abca1a8b3	1 year ago
Rene Krenn	308a974059	MT#58762 harmonize restapi wildcard filters ... - all standard LIKE search are migrated - will avoid LIKE unless a pattern (* wildcard) is used as a search term. this encourage db index usage, will be faster - supports wildcard escape sequence \\* - harmonize swagger UI descriptions of filters Change-Id: Iea155871c9be6c284e6970a562d4e6af73fedc4b	1 year ago
Rene Krenn	71480acfa3	MT#56239 avoid ambersand symbol in apidoc descriptions ... Change-Id: Ic024d16c008648cb4d9cf14ae0e58d1ebe8febe9	2 years ago
Rene Krenn	8e82715eb8	MT#56239 api/callrecordings support filtering multiple metakeys ... filtering an Entity-Attribute-Value model cannot be done as simple conjunctions, but requires either INTERSECT set operation, or joining the same table multiple times. Change-Id: I5ce1ae1ece9406b6610487654f09d768a233b122	2 years ago
Rene Krenn	9c3a549094	MT#56239 api general caller/callee wildcard search support ... various api rails will need to support ?caller= and ?callee= url query parameters. since this involves SQL queries against potentially large database tables, special care is taken with wildcard search to prevent slow queries: - the ?wildcards=true query parameter has to be specified to accept search patterns that contain wildcard symbols, so wildcards are not accepted by default. WARNING: a search string with a leading wildcard will always force a *slow* full db table scan! - the * symbol is used as a wildcard symbol - \ (backslash) is used as escape character to search for a literal '*' Change-Id: Ie6065b0cfa883f7963e1dc8259fffea9a1edfdfe	2 years ago
Rene Krenn	ae612eb401	MT#58346 caller/callee filters for conversations ... the conversation list now supports ?caller= and ?callee= url query parameters. since this involves SQL queries against potentially large database tables, special care is taken with wildcard search to prevent slow queries: - the ?wildcard=true query parameter has to be specified to accept search patterns that contain wildcard symbols, so wilddcards are not accepted by default. WARNING: a search string with a leading wildcard will always force a *slow* full db table scan! - the * symbol is used as a wildcard symbol - \ (backslash) is used as escape character to search for a literal '*' Change-Id: I792d2ea9c649c69c4b5cc98076097cb96467d4bc	2 years ago
Marco Capetta	f8fd17bb3e	MT#58596 Enable api soundgroups and soundhandles for subscriber ... The commit extends what done already in e15e40423 allowing the subscribers to get the soundgroups and soundhandles using API. This is necessary for some CSC features like the selection of the Custom Announcement to play after Call Forward. Change-Id: I40d57d1efdc107c64fe3efafd9e87d1403c4d124	2 years ago
Kirill Solomko	1aef2d5d32	MT#58581 /api/customers fix filter by profile_package_id ... * filtering by package_id with an incorrect search db query (non-existing field) caused a 500 error * package_id filter is renamed into profile_package_id to correspond to the response field * fix profile_package_id db search query Change-Id: Ia93e0662172dd89e85572717cbb99a2ed51eb3a7	2 years ago
Kirill Solomko	cec12f7b1e	MT#58333 rework /api/phonebookentries ... * /api/phonebookentries is now split into 3 parts * /api/resellerphonebookentries * /api/customerphonebookentries * /api/subscriberphonebookentries main reason is because there are 3 different contexts (tables) that in the end return "id" which is also a resource identifier and it is not unique because the contexts handle their own ids, another reason is to simplify code and access definitions, as well as to have solid and easy representation. * /api/resllerphonebookentries * can be accessed by 'admin', 'reseller' roles * /api/customerphonebookentries * can be accessed by 'admin', 'reseller', 'subscriberadmin' roles * fix contract_id field to be correctly represented as 'customer_id' * /api/subscriberphonebookentries * can be accessed by 'admin', 'reseller', 'subscriberadmin', 'subscriber' roles * add 'shared' query filter * all the new endpoints * no more required query param fields * customer_id is now optional query parameter filter * add valid/existing resource checks for POST/PATCH/PUT ops * add correct auto filtering based on the user's role * various major code simplifications, fixes and code style improvements * csv download now also allows query param filters to be applied * fix documentation Change-Id: I735df96b7a7e18006a46b82a4fcc603e400ad9e6	2 years ago
Kirill Solomko	706b472465	MT#58430 fix allowed_ngcp_types in API GET ... * allowed_ngcp_types check is now correct in the Controller:API::Root::GET * also fix allowed roles check Change-Id: I2446d7377d2a1ef152560bfb2799bb9debd0f34b	2 years ago
Kirill Solomko	b7c22618c8	MT#57987 /api/rewriterules calculate priority on create ... * when a new rewrite rule is created and the priority attribute is not specified, only the first rewrite rule of the same rewrite rule set is set as 50, and the following are as: "MAX(priority) of the same rewrite rule set"+1 Change-Id: I1f1ae275e00918a34a625409987f21044b3b940f	2 years ago
Kirill Solomko	6953de937c	MT#33041 add reseller max subscribers support ... * Max subscribers can now be defined in the Reseller contract * Max subscribers is now shown in int Reseller's "Reseller Contract" Details (with an indictaion of how many subscribers are created out of the defined max subscribers, as well as when the limit is reached (similar to the Customer's Details page) * Add reseller subscribers count checks on the UI Create Subscriber page * Add reseller subscribers count checks in /api/contracts POST/PUT * Prevent max_subscribers definition for Peering contracts in /api/contracts POST/PUT Change-Id: I1561d4eb7da5b1a0a0c99acabd18d2a9cd98dec7	2 years ago
Rene Krenn	8fbd46e5fc	MT#57382 DELETE /api/faxes/id ... Change-Id: If578cd842d83e3471659164826d3e0305018cc29	2 years ago
Marco Capetta	967bc581fe	TT#179901 Discontinue iNew dedicated features and code (part 2) ... Remove party_call_control feature Change-Id: I6ea1c725a463ee62696de93a7ee52080515cc2b7	2 years ago
Rene Krenn	cde01e7d66	MT#53849 peer/reseller invoicing ... Change-Id: I0d869edb697779e7d02caaea70c237f0c87f4d50	2 years ago
Kirill Solomko	22a8e95c29	MT#57708 sound set/file use READ COMMITTED ... * as changes to sound sets and sound files cause internal triggers + SP to be invoked, the gap locking should be reduced to avoid possible deadlocks, hence READ COMMITTED transaction isolation level is used for all sound sets and sound files change operations. * fix a condition that caused an evaluation error when both old_parent_id and parent_id are undef. Change-Id: I7e69a0a0e5878c03b086977933315e532aaa2a29	2 years ago
Kirill Solomko	9d9c0d0669	MT#57580 add expose_to_customer support for NCOS ... * NCOS Levels and NCOS Sets now have expose_to_customer option (boolean). If enabled, the NCOS Level/Set can be seen and used by Customers in their and their subscribers' preferences * NCOS Levels and NCOS Sets with expose_to_customer = 0 (default) are not visible anymore to subscribers. * API GET /api/ncoslevels is now accessible by 'subscriberadmin' role and returns a list of NCOS Levels that belong to the same reseller and with expose_to_customer = 1 (fields are limited to id,name,description). * When NCOS Level is expose_to_customer is set from 1 to 0, all assigned ncos preferences for contracts and subscribers are that contain the NCOS Level are removed. Change-Id: Id7c444b4a0834939a128179ded6670705258519b	2 years ago
Kirill Solomko	19f5c68bce	MT#56693 fix /api/soundfiles without Content-Type ... * prevent an error if /api/soundfiles is called without the Content-Type header Change-Id: I1c0844c789839ad4dabedef216336b1387db074e	2 years ago
Kirill Solomko	391aa3fbc3	MT#57564 convert deprecated timezone names ... * timezone names are converted to their links, and if a link is found (means that the current name is deprecated), it's validated and stored as the link. Change-Id: I6348659178400a96eaadd70f79b792c4fa25d7d4	2 years ago
Marco Capetta	be7193e75b	MT#57469 Fix kamailio lcr reload on PATCH/PUT APIs ... When calling a PATCH or PUT API on a peering group or outbound rule the reload of the kamailio lcr module was triggered before the update of the database, thus the kamailio status was not updated. Change-Id: I59ed863c85219e62d6f5b5a2af80db8ef952844c	2 years ago
Kirill Solomko	89173442f3	MT#57223 improve sems audio cache invalidation ... * clear_audio_cache is now invoked also from: - when a sound file is added (to reflect use_parent/potential child sets) - when a parent is changed for a sound set (to reflect current and potential child sets) - when a sound set is removed (to reflect potential child sets) * clear_audio_cache() is reworked - clear_audio_cache() now calls virtual_child_sound_sets to fetch all potentially affected child sound sets and also clears audio cache for them. if the amount of fetched sound_sets is greater than 10000 then the the whole audio cache is invalidated * _clear_audio_cache_service() is reworked - supports/expects a list of sound sets and sends them as a string with ':' separator - if both sound handle and sound sets are provided it calls clearFiles (so that only the specific sound handle is cleared) - if only sound sets are provided it calls clearSets (so that all of the cached files beloning to the sets are removed) - if none of the above are provided it calls clearAll to invalidate the whole cache Change-Id: Ie85f208e27183e88665803b93bb16d7de8e3d7ac	2 years ago
Kirill Solomko	0370507310	MT#57155 /api/rewriterules add filter by direction,field ... * filter by direction and field (caller or callee) Change-Id: I0205ead6cf2a1d51532992f67a5add63cdeb51e0	2 years ago
Rene Krenn	5e2f599eb9	MT#57171 speedup DELETE /api/lnpnumbers/id ... Change-Id: I2b3f817b95e311b922e935453f659893004c5be4	2 years ago
Kirill Solomko	407d6f3f3b	MT#56693 /api/soundets fix DELETE subscriberadmin access ... * subscriberadmin can only delete items that belong to the same contract_id, otherwise 'read-only' forbidden error is returned Change-Id: I3ef2716039626bc1b208a5c7af06cf87f69cad73	2 years ago
Kirill Solomko	c58a8b9b1f	MT#57100 avoid where ambiguity for query_types search_eq,search_like ... * uses prefix 'me.' for query types search_eq and search_like in the where condition to avoid ambiguity if the search_rs has joins that with the same column names Change-Id: I90fef80970aa4415480b00bbed2fb9fbee1f1ccc	2 years ago
Kirill Solomko	c5b288a9da	MT#57085 fix /api/peeringroups form operations ... * API now has own GroupAPI form with the correct fields for API * db/form field name conversion is moved to the newly added resource_from_item() and process_form_resource() functions * simplify code for GET/POST/PUT/PATCH operations Change-Id: I99b76801a2894c8a4821513186796627db728189	2 years ago
Kirill Solomko	6a922c5cfb	MT#56693 parent_name, allow empty sound files, sems audio cache ... * /api/soundsets new parent_name field * /api/soundfiles now allow POST to be done without the upload, for cases when an empty record needs to be added to define "use_parent" behaviour * improve sems audio cache clear on sound files change, now sems instances that are not available (not running/request timeout) are skipped instead of raising an error Change-Id: I83af2ee2fd3672ab35cf330464ace68416ab52be	2 years ago
Rene Krenn	a9b1ba896b	MT#56917 fix rwr 'description' filter ... a 500 error results when specifying the RewriteRule "description" filter because of ambigous field names. Change-Id: I9c0a9f6c84ec62fca4c77ce904fb07c4dadef13b	2 years ago
Kirill Solomko	2a0b8dcc1e	MT#56432 add time set support to ncos levels ... * add NCOS Levels UI Time Set support * add NCOS Levels API Time Set support Change-Id: I03201c09462f0f0ceff39da271bcf8ca1fc797b8	2 years ago
Marco Capetta	36ae8cedb9	MT#55931 Add new speeddial, forward, transfer device buttons ... Change-Id: I0b9c1f3ddb0d4cd3fe28995b170e9104c668a553	2 years ago
Kirill Solomko	e90ba9d517	MT#56208 allow subscriber role access to cf sets ... * subscriber role can now GET call forward sets that does not belong to the user but used in call forwards assigned to the user * the user can only GET the sets but cannot update/delete them (403 Forbidden is raised in case of attempt) Change-Id: I391b802d962f4bbeae991e3046ac3f132a19edb6	2 years ago
Kirill Solomko	f942b07a1e	MT#32992 fix /api/resellerbrandnglogos ... * add described query_params 'subscriber_id', 'reseller_id' so they are avaialable on the doc * remove search by param from Role::ResellerBrandingLogos::_item_rs() * fix Role::ResellerBrandingLogos::_item_rs() queries * Controller::API::ResellerBrandingLogos* - now always returns the binary data containing the logo and with the Content-Type header - change NOT_FOUND message to indicate that the ResellerBrandingLogo is either not found or does not have image/image_type * do not log response queries * fix allowed_role to be: admin, reseller, subsscriberadmin Change-Id: Iaadb47fb2d72886a8d9244a523d5914500a4dd20	2 years ago
Rene Krenn	e9367e0cfe	MT#55972 support wildcard search for customer external_id ... Change-Id: I8287f7650f68dab2e806259ec00ffad2ba45fe1a	2 years ago
Rene Krenn	67f6063341	MT#55551 add ccare roles to /api/callforwards ... Change-Id: I414e10fca61bc535471a0b2525b7e684ea9d23e5	3 years ago
Kirill Solomko	a86e868a6f	MT#55538 /api/soundsets allow filtering by customer_id=null ... * when customer_id query param is specified as customer_id=null or customer_id=NULL it now correctly fetches soundsets that do not have customer_id assigned to them Change-Id: I8de3d9615c133c2abd3eb2b5f4fea8de5b652417	3 years ago