* /api/cftimesets now fully use Entities/EntitiesItem
* EntitiesItem: delete(): fix delete_item() expression processing
* Entities/EntitiesItem: post/put/patch/delete: go to TX_END
in scenarios where after a method call (e.g. update_item()) there
is a normal return from the function but errors in $c->error, so
that they are also caught correctly
Change-Id: I3bef409ded590796c2bba4f30acd28b02e99065b
* clear errors in Controller::API::Root::end so that they are
not interfere with generated ones
* API::error(): add $c->stash->{is_api_error_response} = 1
to track it in log_response()
* API::log_response(): distinguish and correctly correct errors
for cases when is_api_response == 1 and also when there were
$c->error('error') calls in the code, to log them correctly
in both cases
Change-Id: Id922b8219832c1b99815d9d608309035b9b25cff
* the error produced by item_id_is_valid() is returned instead of
a new one is generated
* fix incorrect expression on delete_item() in delete()
Change-Id: Ic8e4e24c70bbcd99ec414d901eb2842ee39ae7de
* fix resultset when user role is reseller so that reseller_id
is joined correctly and a db error is not produced anymore.
Change-Id: I2a1b357037d983f23770bb59519fc2cb8b68a7e4
* $c->error array now contains the $message as the first element
instead, so that it's possible to obtain all the error data in
the code when fetching it from @{$c->error}. the first element
is not logged in the error log.
* api_response $c->response_body part is now stored in MSG= and
possible errors / other log data is now stored in LOG= to:
- reduce amount of log lines when an API error response occurs
from 2 to 1
- the message part usually contains either HTTP response
message (e.g. Internal Server Error) or a validation message
string, so it belongs to the MSG= part of the log line, where
as the internal log data is more related to the LOG= part
- both MSG= and LOG= parts are escaped for GDPR related obfuscation
* Utils::Messag::info(): $msg is now also obfuscated if it's detected
as a reference (also because logging is moved for the API part to
$msg), as well as truncated for possible new-line char and
white-spaces.
Change-Id: I3b670b2251ec3060037ed6863f18d95975120b8d
* the following changes resolve an issue where $now in Role/Contracts
resource_from_item appeared as an object and failed subsequent
logic:
- hal_from_contract is renamed to hal_from_item to match correctly
the parent method
- call hal_from_item and pass $now string there correctly as $params
{ now => $now }
Change-Id: I5c02d5f7df5d9000550505ad120b9531f87e8d65
* Role/Entities*: POST/PUT/PATCH/DELETE methods changes:
- support deadlock detection and transaction retry
(2 retry attempts at the moment)
- improve transaction control, use local $guard
instead of saving the ref to $c->stash, as in that
case it went out of scope too late and also reported
an error message into the log about abnormal $guard out
of scope interruption
- move all non transaction related code outside
of the scope
- add error handling when methods such as update_item,
and a like do not return the expected data, instead of
simply going out of scope and resulting in an uncontrolled
reply
* Role/API:
- rework transaction control:
+ get_transaction_control() is renamed to start_transaction()
to better reflect what it does
+ complete_transaction() is renamed to commit_transaction()
+ remove unused %params arg
+ pass $guard into commit_transaction() instead of having it
stored as $c->stash->{transaction_guard) that caused the $guard
ref to be destroyed much late than expected
(there was also a typo as transaction_quard,
which is not relevant anymore with the changes
+ add check_deadlock() that is invoked when an exception is caught
or an $c->errors contain an error, and if the error message
represents a transaction error, the transaction block is
re-invoked via "goto TX_START"
- rework error():
+ it now accepts args as following:
($self, $c, $code, $message, @errors)
# code -> returned as HTTP code in the reply
# message -> returned as HTTP message in the reply
# errors -> contain errors for internal logging, last element often contains a DBIx exception
+ populates all @errors into $c->error so they are available on
demend in the code via $c->error or $c->last_error
+ $c->log->error is not invoked now as the errors become printed
in log_response()
- log_response() now prints collected errors from $c->error correctly
as a separate log line, that is alike to the other api logs so that
those can be looked up by the request's tx_id, also all errors are
now printed only into api.log
* Adjust all $self->error() calls in catch($e) to include $e as the last
argument, as well as the duplicate $c->log->error is removed from
those ocassions
* Remove all $c->log->error() calls as they are replaced with either
$self->error() (that logs it correctly into api.log) or
$c->error('err') that also adds it correctly into api.log
* API::CallForwards: rework to use Entities/EntitiesItem
* API::Contracts: rework POST to use Entities
* API::PeeringGroups: rework POST to use Entities
* API::SubscriberRegistrations: rework POST to use Entities
* API::RewriteRuleSets: improve create_item() functionality
* Utils/Message: add 'api_retry' log type
* $c->session->{api_request_tx_id} is changed to
$c->stash->{api_request_tx_id} because sometimes the session
ref is different and a different tx_id becomes used
Change-Id: I633ce7a8047b1bf00a2f6889003088edf0825dcd
* POST /api/headerrules now supports either 'set_id' or 'subscriber_id'.
When 'subsriber_id' is specified, a subscriber assigned header rule
set is automatically created if it does not exist, as well as
automatically removed when its last header rule is deleted.
* It is now possible to directly GET and DELETE /api/headerrules/:id
where :id is a subscriber assigned header rule.
* It is now possible to directly GET and DELETE /api/headerrulesets/:id
where :id is a subscriber assigned header rule set.
* Improve /api/headerrules data validation and duplicate header rule
detection.
* It is now possible to expand by /api/headerrules/?expand=set_id field
Change-Id: I681bc61c2eed47a8e54847f07f31134f643930c2
* if the primary numner during /api/subscribers update request is
unchanged, the primary number processing related logic is not invoked,
as otherwise it's a redundant and in some cases (pilot subscriber)
time consuming operation.
Change-Id: I4cbb0387011f5241fcbdaab1efbb0d4a5caa2143
all api query param filters that were harmonized to "wildcard"
also appended leading+trailing wildcard (this was from the original
idea to make adminUI and legacy panel search behave the same).
Change-Id: I017473b84f2844d14a11b4ec8b437df5335c24d0
this filter is another spot, where leading/trailing wildcards
were added unintentionally, during harmonization.
Change-Id: Id23114f8e7ebca018ad222e35b39d8260f1da92f
* socket field is changed to text for the API endpoint as by default
in the LocationEntry form that is used by the UI the field is
a prebuilt options list that comes from config.yml extra_sockets,
and that severely limits the usage for the API.
Change-Id: I71ee1f77dbf458c12655f840282375cee62b7534
* if no balance is found for the provided period a 422
error is returned:
showdetails => $c->loc('period start=[_1] end=[_2].', $stime, $etime),
error => 'Could not find active balance.',
httpcode => HTTP_UNPROCESSABLE_ENTITY,
instead of a 500 server because of access to undefined $balance value
Change-Id: Icccb9f253d117327641b6803ac0d2cdb09cf030f
in case the DB shows discrepancy because of missing
primary provisioning.voip_dbaliases records, align with
legacy panel behaviour and also take the restapi
aliases list from voip_numbers, instead of voip_dbaliases.
.. while trying to keep the performance gain from
https://gerrit.mgm.sipwise.com/c/ngcp-panel/+/65086.
Change-Id: Ibd7f8c8bc6a39ae2c31b4e8818080674ab77d66c
* /api/rewriterules PATCH, DELETE methods used the generic
transaction, which conflicted with the xmlrpc reload command.
* own_transaction_control was only set for PUT and now also
extended to PATCH, DELETE to ensure that dialplan_sip_reload()
is called properly after $guard->commit.
Change-Id: I7313b4f6d1f6699bf38ae34749c5383a3943f61d
with the change to harmonize wildcard symbols % and *
to * in query param filters across all restapi rails,
only
- /api/subscribers?alias=
- /api/subscribers?pbx_extension=
- /api/subscribers?primary_number=
- /api/pbxdevices?pbx_extension=
are supposed to include implicit leading an trailing
wildcards.
panel UI uses trailing wildcard by default everywhere,
and so should the databales of the new Admin UI.
external_id were used to support patterns, but must
not include a trailing wildcard in rest API, as this
can break logic that rely on unique api request results.
we now therefore no longer append wildcards in particular to:
- /api/voicemails?folder=
- /api/voicemailgreetings?type=
- /api/subscribers?subscriber_external_id=
- /api/subscribers?customer_external_id=
- /api/subscribers?display_name=
- /api/subscribers?domain=
- /api/profilepackages?profile_name=
- /api/profilepackages?network_name=
- /api/pbxdevices?display_name=
- /api/faxserversettings?name_or_password=
Change-Id: I40ef1912d7224a56c6d14619d258c70e447ab23d
the /api/callists rail supports the "call_id" query parameter to match calls with a callid prefix. this filter also adds an implicity ordering ORDER BY length(call_id) ASC, 'start_time' ASC.
the /api/callists api rail also requires the query parameter "susbcriber_id", which renders a (fast) compound SQL query to list matching incoming (query1) and outgoing calls (query2) using UNION ALL (instead of a slow OR query).
query1 UNION ALL query2
this is supported by the https://metacpan.org/pod/DBIx::Class::Helper::ResultSet::SetOperations module, which however generates invalid SQL syntax if query1 or query2 contains an ORDER BY.
this is exactly what caused the 500 error of the customer who applied both "call_id" and "susbcriber_id" parameter in the request at the same time.
... ORDER BY length(call_id) ASC, `start_time` ASC UNION ALL SELECT `me`.`id`...
the error happens randomly, because query parameters are stored in a hash (which by definition has no deterministic order of the entries). when the "call_id" parameter is applied at last, it worked as expected.
so the issue can be adressed by strictly ordering the UNION ALL result, and prohibit ORDER BY in query1 and query1. the latter was added already with commit b2dfe28eed, and could be hotfixed.
however, the ORDER BY of "call_id" query param is lost, and should be properly handled by forcing strict ordering of how query parameters are applied.
the fix will introduce paramater ordering according to their order of definition in the code.
Change-Id: I165d341b5c20e9bb750bd1fba88c836b393e80bd
* get_handlers_rs() is renamed into get_file_handles() and
reworked from an indented 'from' resultset into 2 separate
resultsets, one fetching all the handles and another one
files for the sound set, it is now returned as an array with the
similar data. Reason for this rework is the deprecated use of
'from', that resulted in a lot of warnings in panel-fcgi.log:
DBIx::Class::SQLMaker::ClassicExtensions::_join_condition():
ResultSet {from} structures with conditions not conforming to the
SQL::Abstract::Classic syntax are deprecated: you either need to
stop abusing {from} altogether, or express the condition properly
using the { -ident => ... }
* adjust file_handles data key names to better follow the
naming conventions and readability, e.g.: groupname -> group_name,
fileid -> file_id
Change-Id: Ia91c9104c0b4dc79ee5bc359eb0b6e035e56ef3e
* the expected csv fields are as following:
- /api/resellerphonebookentries/?reseller_id=1
name,number
- /api/customerphonebookentries/?customer_id=1
name,number
- /api/subscriberphonebookentries/?subscriber_id=1
name,number,shared
- /api/resellerphonebookentries/
name,number,reseller_id
- /api/customerphonebookentries/
name,number,contract_id
- /api/customerphonebookentries/
name,number,shared,subcriber_id
(note: the fields are taken from the leftmost side
as the minimum required fields, so if one provides as:
- /api/resellerphonebookentries/?reseller_id=1
name,number,reseller_id
this line will be uploaded and columns after 'number' are
ignored in this case.
this is for convenience if you have a prepared upload
for different customers/resellers for instance, you can
upload it under one customer/reseller when needed
* user access scope is now validated, e.g.:
resellers cannot upload phonebook entries to another resellers,
etc.
* fix csv upload for /api/customerphonebookentries/?customer_id=1
Change-Id: I970ac36fdb8b366adff15515a48d296e8abeff13
* {"values:", ["value1", "value2"]} is not accepted by
/api/headerruleconditions
* the previous behaviour of {"values": [{"value": "value1"},{"value":
"value2"}]} is preserved
Change-Id: I7d9540d20ed0ecbb2712f1905a7d9e6f7af7e06d
* add global expand by rwr_set_id field
* adjust rwr_set_id, rule_id in Condition and Action forms
to have correct type (as they were inherited as Hidden)
Change-Id: Id2529c67919b496303cee24da2a2c63f22df23e9
- all standard LIKE search are migrated
- will avoid LIKE unless a pattern (* wildcard) is
used as a search term. this encourage db index
usage, will be faster
- supports wildcard escape sequence \\*
- harmonize swagger UI descriptions of filters
Change-Id: Iea155871c9be6c284e6970a562d4e6af73fedc4b
filtering an Entity-Attribute-Value model cannot be done as
simple conjunctions, but requires either INTERSECT set
operation, or joining the same table multiple times.
Change-Id: I5ce1ae1ece9406b6610487654f09d768a233b122
* all reseller_id-number entries are checked for POST
* reseller_id-number pairs that != item->id are checked for PUT/PATCH
Change-Id: I93a8b7fe22ed63d2d84e2d45291981d09b4af156
various api rails will need to support ?caller= and ?callee=
url query parameters.
since this involves SQL queries against potentially large
database tables, special care is taken with wildcard search
to prevent slow queries:
- the ?wildcards=true query parameter has to be specified
to accept search patterns that contain wildcard symbols,
so wildcards are not accepted by default.
WARNING: a search string with a leading wildcard will always
force a *slow* full db table scan!
- the * symbol is used as a wildcard symbol
- \ (backslash) is used as escape character to search for
a literal '*'
Change-Id: Ie6065b0cfa883f7963e1dc8259fffea9a1edfdfe
the conversation list now supports ?caller= and ?callee=
url query parameters.
since this involves SQL queries against potentially large
database tables, special care is taken with wildcard search
to prevent slow queries:
- the ?wildcard=true query parameter has to be specified
to accept search patterns that contain wildcard symbols,
so wilddcards are not accepted by default.
WARNING: a search string with a leading wildcard will always
force a *slow* full db table scan!
- the * symbol is used as a wildcard symbol
- \ (backslash) is used as escape character to search for
a literal '*'
Change-Id: I792d2ea9c649c69c4b5cc98076097cb96467d4bc
Both the parameters 'www_admin.api_embedded_forcearray' and
'www_admin.api_links_forcearray' were created long time ago
to have API backward compatible.
But the consistency of the returned values is a priority,
so we have to get rid of this option and keep the behavior
of the current default:
* always return an array
* always return lisk as an array
Change-Id: I75dccbf3b641a7e3c7f948c020cf2ebeb8a59065
The commit extends what done already in e15e40423 allowing the
subscribers to get the soundgroups and soundhandles using API.
This is necessary for some CSC features like the selection of
the Custom Announcement to play after Call Forward.
Change-Id: I40d57d1efdc107c64fe3efafd9e87d1403c4d124
* filtering by package_id with an incorrect search db query (non-existing
field) caused a 500 error
* package_id filter is renamed into profile_package_id to correspond
to the response field
* fix profile_package_id db search query
Change-Id: Ia93e0662172dd89e85572717cbb99a2ed51eb3a7
The 'duration' field is reported in the documentation as 'number'
but in fact it is a string, for example: "0:00:16.495".
In the commit the definition of the parameter is changed from
PosInteger to Text
Change-Id: I317006f5bbb5c4b7ffea0abe29dc0b175d8f95e6
while the api is strict with expecting NULL values,
we now tolerate '' (empty string) for topup and underrun
lock level values.
Change-Id: I77357f0d1152b5af43e39389f8d4d4f3e0b81b2a
In case a susbcriber or subscriber admin get the conversation list
using the API, the caller is not properly anonymized when clir is
set to 1. This is a big security hole.
With the fix, the caller is authomatically anonymized.
Of course, administrators and resellers can still see the
original caller number.
Change-Id: I9bc676718e38ae1d462f43176e4c6611528f7e54
* /api/phonebookentries is now split into 3 parts
* /api/resellerphonebookentries
* /api/customerphonebookentries
* /api/subscriberphonebookentries
main reason is because there are 3 different contexts (tables)
that in the end return "id" which is also a resource identifier
and it is not unique because the contexts handle their own ids,
another reason is to simplify code and access definitions, as well
as to have solid and easy representation.
* /api/resllerphonebookentries
* can be accessed by 'admin', 'reseller' roles
* /api/customerphonebookentries
* can be accessed by 'admin', 'reseller', 'subscriberadmin' roles
* fix contract_id field to be correctly represented as 'customer_id'
* /api/subscriberphonebookentries
* can be accessed by 'admin', 'reseller', 'subscriberadmin',
'subscriber' roles
* add 'shared' query filter
* all the new endpoints
* no more required query param fields
* customer_id is now optional query parameter filter
* add valid/existing resource checks for POST/PATCH/PUT ops
* add correct auto filtering based on the user's role
* various major code simplifications, fixes and code style improvements
* csv download now also allows query param filters to be applied
* fix documentation
Change-Id: I735df96b7a7e18006a46b82a4fcc603e400ad9e6
* allowed_ngcp_types check is now correct in the
Controller:API::Root::GET
* also fix allowed roles check
Change-Id: I2446d7377d2a1ef152560bfb2799bb9debd0f34b
* as now all profile preferences should be visible for admin users,
the excessive _check_profile() filtering is removed, as well
as it improves the performance due to less sql queries (was 1 extra query
per preference). the preferences based on attributes are correctly
filtered for subscriber and subscriberadmin 'roles'.
Change-Id: Ifaa8c7de1f73d07fce3ef0df725a01f1c5d5d80c
invoices support call direction for the summary and zone
costs, however the detailed call list (available to customer
invoices only) always shows *source* costs and zone details.
this is fixed accordingly. in&out call directions implies a
slow OR query, and is not supported, as noted in the
documentation.
Change-Id: Ib4a8a8f535a9f170b1f3ca9a6b9be36c033d439e
* URI that comes from c.req.uri must be escaped as Catalyst
provides c.req.uri unescaped.
* new Catalyst::NGCP::Plugin::EscapeURI with escape_uri()
function
* new NGCP::Panel::Utils::Generic::escape_uri() that uses
URI::Escape::uri::escape_utf8()
* c.req.uri and c.req.path occurrences in layout/body.tt are now
escaped with c.escape_uri()
Change-Id: Id0483fa6e570a0ff8db84b1d470caf5405cc0886
* when a new rewrite rule is created and the priority attribute
is not specified, only the first rewrite rule of the same rewrite
rule set is set as 50, and the following are as:
"MAX(priority) of the same rewrite rule set"+1
Change-Id: I1f1ae275e00918a34a625409987f21044b3b940f
* Max subscribers can now be defined in the Reseller contract
* Max subscribers is now shown in int Reseller's "Reseller Contract"
Details (with an indictaion of how many subscribers are created
out of the defined max subscribers, as well as when the limit
is reached (similar to the Customer's Details page)
* Add reseller subscribers count checks on the UI Create Subscriber
page
* Add reseller subscribers count checks in /api/contracts POST/PUT
* Prevent max_subscribers definition for Peering contracts in
/api/contracts POST/PUT
Change-Id: I1561d4eb7da5b1a0a0c99acabd18d2a9cd98dec7
This field controls a behavior upon cancelling unsuccessful legs:
- terminate legs with CANCEL (default way)
- terminate legs with BYE
Change-Id: I2ff5c758c319714f0e6636db8b8ba5c0fd495e1f
* secret_key can not be correctly set/updated and unset
* last_secret_key_modify behaviour is changed, if "secret_key"
is provided in the data, the value is updated, otherwise
it points to the last time when the key was updated (or unset)
Change-Id: Ib6d7f78d4e93240ac36750fc1685f1c4c765bf55
* as changes to sound sets and sound files cause
internal triggers + SP to be invoked, the gap
locking should be reduced to avoid possible
deadlocks, hence READ COMMITTED transaction
isolation level is used for all sound sets
and sound files change operations.
* fix a condition that caused an evaluation error
when both old_parent_id and parent_id are undef.
Change-Id: I7e69a0a0e5878c03b086977933315e532aaa2a29
* address an issue where get_form() logic did not return
any $form value because of the if ($params) block was
falsely used.
Change-Id: I77f03ac7914c1b4a8f072269cf2c98b99657fa52
* NCOS Levels and NCOS Sets now have expose_to_customer
option (boolean). If enabled, the NCOS Level/Set can
be seen and used by Customers in their and their subscribers'
preferences
* NCOS Levels and NCOS Sets with expose_to_customer = 0 (default)
are not visible anymore to subscribers.
* API GET /api/ncoslevels is now accessible by 'subscriberadmin' role
and returns a list of NCOS Levels that belong to the same reseller
and with expose_to_customer = 1 (fields are limited to
id,name,description).
* When NCOS Level is expose_to_customer is set from 1 to 0, all
assigned ncos preferences for contracts and subscribers are that
contain the NCOS Level are removed.
Change-Id: Id7c444b4a0834939a128179ded6670705258519b
A reseller was not able to edit a phonebook entry unless it
specified the reseller_id. This should not be required since
the reseller could not know it.
Change-Id: I347c676e291659e9a8b2a97632531dc7b06e055a
This commit forces the creation of reseller_id preference
whenever a new subscriber is created in the same fashion of
the already existing preference account_id.
Change-Id: I8cc2b3b65e0e7a81e72452a3840fdd3540146968
subscriberadmin is now be able to get, edit and delete registrations
of the other subscribers belonging to the same customer.
This is necessary to extend the CSC features and give the PBX
administrator more flexibility in device configuration.
Change-Id: If83c1adb9af6590111c14800d83fb7d1eafaa9d3
From the very origin of the AA module the link used in panel
to create/edit the AA slots were '/preferences/speeddial/edit'.
This is wrong and most probably coming from a copy and paste of
the speeddial feature.
The commit fix the link and it is now chaged to the following
'/preferences/autoattendant/edit'.
Change-Id: I095b433cad78fb0175b0551fcfdea85815f73cdf
* timezone names are converted to their links, and if a link
is found (means that the current name is deprecated), it's
validated and stored as the link.
Change-Id: I6348659178400a96eaadd70f79b792c4fa25d7d4
When calling a PATCH or PUT API on a peering group or
outbound rule the reload of the kamailio lcr module was
triggered before the update of the database, thus the
kamailio status was not updated.
Change-Id: I59ed863c85219e62d6f5b5a2af80db8ef952844c
We have to support new preference for:
- `create_peer_registration`
- `update_peer_registration`
- `delete_peer_registration`
If there is no `peer_auth_registrar_server` value provided,
then use a realm value instead.
Change-Id: I39d3a917c9f11c1d4cfa0781fe5a3cee11d9eeea
When the subscriber preferences API is called by a subscriber
admin of a normal (not PBX) customer, then some values are
removed from the output. In all the other cases those values
are instead returned.
This fix let expose the profile_id, domain_id, status and
webpassword to all the types of subscribers because necessary
in the CSC context.
Change-Id: I629475e7f51d747a55ebfbc44232fb94a54fed06
* clear_audio_cache is now invoked also from:
- when a sound file is added (to reflect use_parent/potential child
sets)
- when a parent is changed for a sound set (to reflect current
and potential child sets)
- when a sound set is removed (to reflect potential child sets)
* clear_audio_cache() is reworked
- clear_audio_cache() now calls virtual_child_sound_sets to
fetch all potentially affected child sound sets and also
clears audio cache for them. if the amount of fetched
sound_sets is greater than 10000 then the the whole audio
cache is invalidated
* _clear_audio_cache_service() is reworked
- supports/expects a list of sound sets and sends them
as a string with ':' separator
- if both sound handle and sound sets are provided it calls
clearFiles (so that only the specific sound handle is cleared)
- if only sound sets are provided it calls clearSets (so that
all of the cached files beloning to the sets are removed)
- if none of the above are provided it calls clearAll to invalidate
the whole cache
Change-Id: Ie85f208e27183e88665803b93bb16d7de8e3d7ac
* try/catch is used for send_email() to catch MTA exception
if the delivery cannot be done
* the email delivery error cause is stored in the panel.log
file
Change-Id: I351b01309ee576b0818610d5ed927874ae2d0822
additional variables for the new_subscriber mail
template:
username
password
webusername
webpassword
cc
ac
sn
Change-Id: I8deadee11b162ef66e9f210d20bca11aa747c18f
* $c->user->id was wrongly used in the advanced cfs selection and
now changed to $prov_subscriber->id to filter it regardless of the
user editing the cfs
Change-Id: I8fa65bbce625d4b21dbb620a1f92919ed7fe2a88
* subscriberadmin can only delete items that belong to the same
contract_id, otherwise 'read-only' forbidden error is returned
Change-Id: I3ef2716039626bc1b208a5c7af06cf87f69cad73
Kirill Solomko
Rene Krenn
Sipwise Jenkins Builder
Marco Capetta
Fabricio Santolin da Silva
Donat Zenichev
Alessio Garzi