168 Commits (868e2657b6be459e40d04ed545cb42c214912d06)

Author	SHA1	Message	Date
Rene Krenn	868e2657b6	MT#59597 fix remove trailing+leading % entirely for "wildcard" params ... Change-Id: I371e2d41ff27b2801ead391778d9a2bdef914563	1 year ago
Rene Krenn	9ba93e69a5	MT#59597 remove trailing+leading % entirely for "wildcard" param types ... this code is not run by legacy panel, so the check for "/api" request can be removed. Change-Id: I29410fbbf046a8f00443138720b940de0f09f416	1 year ago
Rene Krenn	aa6760228e	MT#59363 retain trailing+leading wildcard for legacy panel only ... all api query param filters that were harmonized to "wildcard" also appended leading+trailing wildcard (this was from the original idea to make adminUI and legacy panel search behave the same). Change-Id: I017473b84f2844d14a11b4ec8b437df5335c24d0	1 year ago
Rene Krenn	c837dc1043	MT#56817 MT#56362 fix query param sorting for regulary chained rs ... Change-Id: I82d3a0ce67ffb61273fc716016885f6431852eff	1 year ago
Rene Krenn	aae0b7e071	MT#56817 MT#56362 fix query param sorting for rs chained the regular way ... Change-Id: I0dab56ae07666eb695d7f30ef633a876ff8258fb	1 year ago
Rene Krenn	a15d00894a	MT#56817 MT#56362 fix errors from nondetermistic query param ordering ... the /api/callists rail supports the "call_id" query parameter to match calls with a callid prefix. this filter also adds an implicity ordering ORDER BY length(call_id) ASC, 'start_time' ASC. the /api/callists api rail also requires the query parameter "susbcriber_id", which renders a (fast) compound SQL query to list matching incoming (query1) and outgoing calls (query2) using UNION ALL (instead of a slow OR query). query1 UNION ALL query2 this is supported by the https://metacpan.org/pod/DBIx::Class::Helper::ResultSet::SetOperations module, which however generates invalid SQL syntax if query1 or query2 contains an ORDER BY. this is exactly what caused the 500 error of the customer who applied both "call_id" and "susbcriber_id" parameter in the request at the same time. ... ORDER BY length(call_id) ASC, `start_time` ASC UNION ALL SELECT `me`.`id`... the error happens randomly, because query parameters are stored in a hash (which by definition has no deterministic order of the entries). when the "call_id" parameter is applied at last, it worked as expected. so the issue can be adressed by strictly ordering the UNION ALL result, and prohibit ORDER BY in query1 and query1. the latter was added already with commit b2dfe28eed, and could be hotfixed. however, the ORDER BY of "call_id" query param is lost, and should be properly handled by forcing strict ordering of how query parameters are applied. the fix will introduce paramater ordering according to their order of definition in the code. Change-Id: I165d341b5c20e9bb750bd1fba88c836b393e80bd	1 year ago
Rene Krenn	308a974059	MT#58762 harmonize restapi wildcard filters ... - all standard LIKE search are migrated - will avoid LIKE unless a pattern (* wildcard) is used as a search term. this encourage db index usage, will be faster - supports wildcard escape sequence \\* - harmonize swagger UI descriptions of filters Change-Id: Iea155871c9be6c284e6970a562d4e6af73fedc4b	1 year ago
Rene Krenn	8e82715eb8	MT#56239 api/callrecordings support filtering multiple metakeys ... filtering an Entity-Attribute-Value model cannot be done as simple conjunctions, but requires either INTERSECT set operation, or joining the same table multiple times. Change-Id: I5ce1ae1ece9406b6610487654f09d768a233b122	1 year ago
Rene Krenn	9c3a549094	MT#56239 api general caller/callee wildcard search support ... various api rails will need to support ?caller= and ?callee= url query parameters. since this involves SQL queries against potentially large database tables, special care is taken with wildcard search to prevent slow queries: - the ?wildcards=true query parameter has to be specified to accept search patterns that contain wildcard symbols, so wildcards are not accepted by default. WARNING: a search string with a leading wildcard will always force a *slow* full db table scan! - the * symbol is used as a wildcard symbol - \ (backslash) is used as escape character to search for a literal '*' Change-Id: Ie6065b0cfa883f7963e1dc8259fffea9a1edfdfe	2 years ago
Kirill Solomko	c58a8b9b1f	MT#57100 avoid where ambiguity for query_types search_eq,search_like ... * uses prefix 'me.' for query types search_eq and search_like in the where condition to avoid ambiguity if the search_rs has joins that with the same column names Change-Id: I90fef80970aa4415480b00bbed2fb9fbee1f1ccc	2 years ago
Kirill Solomko	0555f32585	TT#65101 expand now fills the same key only once ... * the same key is now expanded only once to use the same data ref otherwise expand like contract_id.contact_id,contract_id.billing_profile_id makes it so that 2nd contract_id would overwrite the already existing (expanded) contract_id Change-Id: I6cab67c2f549e9165dbf2cfd2f4b84daef503dd8	3 years ago
Kirill Solomko	5a18d5fee8	TT#65101 improve expand=all, /api/domains care roles support ... * expand=all now expands by the all resource keys instead of the current form fields, that is to reflect keys that are created manually in the response such as reseller_id or domain_id * ccare roles now correctly support /api/domains Change-Id: I9951bfd97b76186def4dc799c72da44425faea4a	3 years ago
Kirill Solomko	84e698e7ac	TT#176050 adjust subfield expands ... Change-Id: I9826d254cec6f03e7c7bad7122450f4a22125e18	3 years ago
Kirill Solomko	938f6ba05c	TT#176050 add expand support on array of ids ... * it is possible to expand arrays of ids now * add expand support for pbx_group_ids * add expand support for pxb_groupmember_ids * remove _password and _webpassword internally prepared fields from expands by subscriber_id, pbx_group_ids, pbx_groupmember_ids Change-Id: I7651aae4c58d98943e82d1eda6b24d260ff2480a	3 years ago
Kirill Solomko	f311177236	TT#182101 Fix check_allowed_ngcp_types detection ... * use any instead of grep for checking the allowed_ngcp_types array Change-Id: I7382a2a61764a4520cd07d2db67ffa3a5112b378	3 years ago
Kirill Solomko	09b589738d	TT#182101 Phonebook feature is hidden for CE ... * templates now rely on c.config.general.ngcp_type and hide the Phonebook feature everywhere if ngcp_type is CE * introduce "allowed_ngcp_types" config for Controller::API::* that is an array, and when specified, only the ngcp_type roles are allowed, otherwise if not specified all ngcp types allowed (default) * Controller::API::Root: - filter controllers from the documentation rendedring that have allowed_ngcp_type config specified and do not match the current ngcp type * Role::API Role::Entities* - new method check_allowed_ngcp_types() - check_allowed_ngcp_types() is now called in Entities and EntitiesItem auto() and denies to 404 Path not found if the ngcp type does not match * "Phonebook" UI is now hidden for CE * /api/phonebookentries is now hidden for CE Change-Id: I41d4b2f87121f281472be3775b862333923fe37f	3 years ago
Kirill Solomko	68af361060	TT#173700 /api/topuplogs expand related fixups ... * Role::API::Contracts add item_by_id() that calls contract_by_id() * Utils::ProfilePackages::catchup_contract_balances return if $contract object is undef * Form::Topup::Log::contract_id expands into Role::API::Customers instead of Role::API::Contracts Change-Id: Id4fc67b8ea1e91f350d0172aafc2b722f34e61f3	3 years ago
Kirill Solomko	4e309b9ef1	TT#153050 add API soft_expand query parameter ... * soft_expand=1 in API requests when coped with the expand query parameter tells the API to ignore possible expand conflicts (such as field mismatch or a permission issue), returning 200 OK instead of 409 Conflict Change-Id: Ib798aabddb1b4d66fc9708acbf713037696ad600	3 years ago
Kirill Solomko	6a0b5d7a75	TT#149459 expand consider null fields as valid ... * when field expand is requested and the field has "null" value (profile_id,profile_set_id), the field is considered as correct instead of returning 409 Conflict Change-Id: I2dfa918a0b99bff75429baa3e94a734fcb8c17e2	4 years ago
Kirill Solomko	78b80e7064	TT#149459 enhance expand functionality with allowed_roles ... * allowed_roles in the expand defintion restrict the field only to the roles in the allowed list, otherwise the field is not expanded. Change-Id: Ib6f776388457327f2fa85e71deb9591022cee2da	4 years ago
Kirill Solomko	8c556bbe36	TT#149459 improve and automate expand functionality ... * add dictionary support for fields that are expanded if encountered in all endpoints, with a possibility to override it, if defined on the endpoint's field level * move expand definitions from form fields into the Expand dictionary * simplify the expand usage, it now operates only with the <x>_id fields that are returned and visible in the response (e.g. if reseller_id is returned, then ?expand=reseller_id), the returned expand object name is <expand_field_name>_expand the, so in case of ?expand=reseller_id, the returned object will be reseller_id_expand * adapt Role/SystemContacts to work correctly with the expand functionality * expanded fields are returned as <expanded_field_name>_expand Change-Id: I4cab44ede9b40c70a95bbcedc81f58dd1f4e3b67	4 years ago
Kirill Solomko	748ac564b0	TT#148250 enhance API expand functionality ... * add recursive expand support, so it's possbile now to do nested expands as "expand=contract.reseller". * comma separated fields support is preserved and with the enhanced expand "expand=contract.reseller,customer.billing_profile" it enables expanding "contract" with nested "reseller", as well as "customer" with nested "billing_profile". Change-Id: Ie82f5118dc13e57a0397566295644452f29bccde	4 years ago
Flaviu Mates	9bb622a2bc	TT#117153 Improve /api/cfmappings performance ... * switch to 'populate' instead of using 'insert' for each destinations, sources, times, bnumbers, mappings and CF preferences * add API->check_patch_op_add_only - to check if the patch contains only "add" operations * improve /api/cfmappings, if all PATCH operations are "add" then the existing records are not fetched and not recreated, enabling very new mapping inserts Change-Id: I0b4e71565c11771026dbbc000aa57b2a613409fa	4 years ago
Rene Krenn	b6e696621b	TT#132650 /api/provisioning_templates CRUD rail ... the /api/provisionintemplates rail provides the operations to create, edit and delete "provisioning templates" know from the "batch provisioning" feature of admin panel. these templates can also be defined in config.yml, while it is however only possible to edit templates stored in the database. executing a template and/or uploading a .csv for bulk execution will be available in a separate part. Change-Id: If8627327270edfce5bca1be3b1f777c1bd44e90f	4 years ago
Kirill Solomko	cc10506b2e	TT#129162 fix subscriber webpassword field validation ... * 'webpassword' field is now also validated for invalid (non-ascii) characters * Fix multiple APP input field validation erros to comma joined. * Adjust 'webpassword' field validation errors to have better readability when there are multiple validation errors Change-Id: I21536f97a4da78cc5192a3abd8cd5adef1b819ec	4 years ago
Kirill Solomko	2b144caf2c	TT#128605 implement dynamic fields expand ... * add API functionality to request additional data and expand fields in GET methods * syntax: - /api/resource/?expand=all - expands all expandable fields e.g.: customer_id field is expanded and customer internally is queried and returned under "customer" => {...} (the returned data is identical to what /api/customers/id would return) - /api/resource/?expand=reseller_id,customer_id - expands only reseller_id and customer_id fields, if they are expandable - /api/resource/?expand=reseller_id,invalidfield_id - returns the data and expands only fields that are expandable (reseller_id in this case) but if it finds either unknown fields or non-expandable fields, changes HTTP status code to "409 Conflict" * adapt all API endpoints to support dynamic expand fields expanding functionality, however the actual expand for them requires modifying the form fields in the following format: has_field 'contact_id' => ( element_attr => { expand => { class => 'NGCP::Panel::Role::API::CustomerContacts', id_field => 'contact_id', alias => 'contact', fetch => 0, }, }, ); - class - represents the class that should be used by the logic to fetch the relevant data - id_field - which field from the resource needs to be expanded, it should be the "id" field (subscriber_id, domain_id, etc.) - alias - (optional), under which key the fetched data is stored. the field name is used as the key if the option is omitted. - fetch - (optional), if the returned data is under $data->{contract_id} then it will be fetched from there and stored under the key (field name or alias), otherwise the whole retreived data is stored under the key (field name or alias) * adapt /api/autoattendants to use the new approach (old one was expand=1) * currently supported endpoints with expand: - admins - autoattendants - domains - customers - customercontacts - resellers - subscribers Change-Id: Iac53409dad944ed4794039a48dc3a9f6dce25bc1	4 years ago
Kirill Solomko	96f0b07469	TT#116251 validate params page and rows equal 0 ... - a validate logic is added for 'page' and 'rows' query parameters and if either of them equals 0 then 400 Bad Request error is returned Change-Id: Ida850c5d942e51a6328f5df01b8e28d55c32cc56	4 years ago
Rene Krenn	6d89e74253	TT#59803 obfuscate .*password.* fields in request+response body (json) ... Change-Id: I9f457b8cd9b0ff2e94c5795fb9987f97af5ac1f9	6 years ago
Kirill Solomko	9094ec496e	TT#60502 refactor PATCH by value ARRAY ... * PATCH by value when it is an ARRAY now iterates backwards through the current values and it iterates as: for current_values for values_to_remove instead of: for values_to_remove for current_values to preserve indexes order * check and avoid situations when indexes for removal are added multiple times, by marking already added indeses for removal Change-Id: Id50c5ea08ad0f7c626a6b23b288e6d008a19696a	6 years ago
Kirill Solomko	ffdb2d0010	TT#60502 fix PATCH removal by value when multiple elements are removed ... * elements must be removed reversed so last index is removed first, otherwise the list "for removal" becomes out of sync with the "current elements" causing wrong values to be removed, or an out of bounds index array error Change-Id: I9ab9bce8205169bc7841c51f37743ab17946cc11	6 years ago
Rene Krenn	b0465b9f07	TT#59803 obfuscate logged request+response body to protect passwords ... Change-Id: I18003de5a24e39b64cf340ec0d4b758a1a223af9	6 years ago
Rene Krenn	dda5ffe48f	TT#56340 fix subscribers.t not matching obfuscation chars ... Change-Id: I7023d1e23fc397e3556705fc52db50eace6c02b4	6 years ago
Rene Krenn	cae5270af5	TT#56340 panel logline obfuscation ... adds gdpr obfuscation quoting for: + subscriber numbers + subscriber ip addresses + subscriber usernames + any logmessage "DATA": query parameters, form data, response data + subscriber uuid's + call id's + callforward sip uri's the quoting is centralized by $c->qs() ("quote sensitive"), using catalyst plugin mechanism. escape symbols are set to « (\x{ab}) and » (\x{bb}). generate_logfile_data_inventory.pl was modified to mark loglines with "gdpr affected" status, if $c->qs() was used in a log message. Change-Id: I0f42d7992594232ae33e5666b0a64009211c5b76	6 years ago
Irina Peshinskaya	345f89d407	TT#47534 Add configured defailt mime-type to Entities FW ... And enable non_json_data processing for Items PUT Change-Id: Ib87bed6aadce2f9217baa60aedc233364560876a	6 years ago
Irina Peshinskaya	858905035b	TT#50975 Replace "value" logic to "filter" in "remove-by-value" ... PATCH "remove by value" will from now remove values even if provided hash has less keys, than original value Change-Id: I2ee04fa279580d662e167c3a0b160fe29d8d79ef	6 years ago
Irina Peshinskaya	a376cc6619	TT#49027 Fix capabilities api ... Change-Id: I85d3d4e71b745115e18e1530808a3ac84c0917d5	6 years ago
Irina Peshinskaya	39bc7eb75d	TT#48198 Fix order_by functionality in api ... - All item_rs modifications should be done before we get rows, so apply order_by before pager - We can't distinguish if subscriberregistrations really has column or not if we return true for all columns. We will use has_column only in cases when it returns something really meaningful. - Subscriberregistrations can't order by nat and subscriber_id Change-Id: I04b7bb719ee058590a7705c6411cb08bcfb15387	6 years ago
Irina Peshinskaya	39918f4f90	TT#48198 Avoid extra page in no_count mode ... Change-Id: Ia0415cf708bc8d531aaf078dc74d6f81b23b60f2	6 years ago
Irina Peshinskaya	5219cdb244	TT#48198 Fix no_count behavior in API.pm ... Change-Id: Ieb665a7fc82f8442e0b2da351cf92573d433c3b6	6 years ago
Irina Peshinskaya	83d715230e	TT#47963 Fix emailtemplates put and patch ... Change-Id: I26a24c17b42e3f31390de28fd1e5098c6c167861	7 years ago
Irina Peshinskaya	7e3a1d1cd2	TT#46187 Use value in "remove" PATCH op as filter ... Change-Id: Ic04917b1e90750e4517d45082cdab7f7f2bcf534	7 years ago
Irina Peshinskaya	aeb5cd9c28	TT#46183 Add "mode" modifier to "add" PATCH operator ... Change-Id: Ic23c427ca70d3ed5d633c712468dbf3bda219563	7 years ago
Irina Peshinskaya	19e88efcab	TT#46667 Make mime-type query param higher prio than Accept header ... And make requested mime-type checking less strict Change-Id: Iba54bd640f833ac2874dba06833a506353ea3ddb	7 years ago
Irina Peshinskaya	0182e3600d	TT#45632 Use query_params and Accept header config together ... Change-Id: I0d4adf1fb0aa03557c80f8957e794d92e2781952	7 years ago
Irina Peshinskaya	af5b5d370f	TT#45632 Add checking of configured Accept values ... Change-Id: I586cc56e08de2c0ff431eaa7480e01481af90940	7 years ago
Irina Peshinskaya	2a2d116e97	TT#44635 Fix order for subscriberregistrations api ... Change-Id: I6d5cadd84db78ecb531bcdf35fd671b7b05dc7b3	7 years ago
Irina Peshinskaya	c76d300f7d	TT#40503 Remove explicit role checking for journal links ... Change-Id: Id33dc65590d87f15fa565d4a400bd059f6bd9654	7 years ago
Irina Peshinskaya	9ed0657d4c	TT#40503 Show journal links to allowed roles only ... Change-Id: I547b2db4767b1e383506bc812b6486db944fceec	7 years ago
Irina Peshinskaya	ffd1b487bf	TT#40503 Add journal actions to entities FW ... Change-Id: Ia0d51f246756f649dc440cde7b4fb0e8056d3ced	7 years ago
Gerhard Jungwirth	26435acacd	TT#38811 fix validate_fields to include Subfields of Compound ... Change-Id: I073232e4e1bd35606f4d31be9b06dd14054e4d5a	7 years ago