84 Commits (master)

Author	SHA1	Message	Date
Spiridonov Dmitry	7e7a603360	sorcery.c: Fixed crash error when executing "module reload" ... Fixed crash error when cli "module reload". The error appears when compiling with res_prometheus and using the sorcery memory cache for registrations	1 year ago
George Joseph	6871d1cdfc	Reduce startup/shutdown verbose logging ... When started with a verbose level of 3, asterisk can emit over 1500 verbose message that serve no real purpose other than to fill up logs. When asterisk shuts down, it emits another 1100 that are of even less use. Since the testsuite runs asterisk with a verbose level of 3, and asterisk starts and stops for every one of the 700+ tests, the number of log messages is staggering. Besides taking up resources, it also makes it hard to debug failing tests. This commit changes the log level for those verbose messages to 5 instead of 3 which reduces the number of log messages to only a handful. Of course, NOTICE, WARNING and ERROR message are unaffected. There's also one other minor change... ast_context_remove_extension_callerid2() logs a DEBUG message instead of an ERROR if the extension you're deleting doesn't exist. The pjsip_config_wizard calls that function to clean up the config and has been triggering that annoying error message for years. Resolves: #582	1 year ago
Alexander Traud	9440f6ec58	main: Fix for Doxygen. ... ASTERISK-29763 Change-Id: Ib8359e3590a9109eb04a5376559d040e5e21867e	3 years ago
Joshua C. Colp	304f8ddfb2	sorcery: Add support for more intelligent reloading. ... Some sorcery objects actually contain dynamic content that can change despite the underlying configuration itself not changing. A good example of this is the res_pjsip_endpoint_identifier_ip module which allows specifying hostnames. While the configuration may not change between reloads the DNS information of the hostnames can. This change adds the ability for a sorcery object to be marked as having dynamic contents which is then taken into account when reloading by the sorcery file based config module. If there is an object with dynamic content then a reload will be forced while if there are none then the existing behavior of not reloading occurs. ASTERISK-29321 Change-Id: I9342dc55be46cc00204533c266a68d972760a0b1	4 years ago
Joshua C. Colp	d6712790cd	pjsip: Update ACLs on named ACL changes. ... This change extends the Sorcery API to allow a wizard to be told to explicitly reload objects or a specific object type even if the wizard believes that nothing has changed. This has been leveraged by res_pjsip and res_pjsip_acl to reload endpoints and PJSIP ACLs when a named ACL changes. ASTERISK-28697 Change-Id: Ib8fee9bd9dd490db635132c479127a4114c1ca0b	5 years ago
George Joseph	7e77815ad1	sorcery.c: Sorcery enhancements for wizard management ... Added ability to specifiy a wizard is read-only when applying it to a specific object type. This allows you to specify create, update and delete callbacks for the wizard but limit which object types can use them. Added the ability to allow an object type to have multiple wizards of the same type. This is indicated when a wizard is added to a specific object type. Added 3 new sorcery wizard functions: * ast_sorcery_object_type_insert_wizard which does the same thing as the existing ast_sorcery_insert_wizard_mapping function but accepts the new read-only and allot-duplicates flags and also returns the ast_sorcery_wizard structure used and it's internal data structure. This allows immediate use of the wizard's callbacks without having to register a "wizard mapped" observer. * ast_sorcery_object_type_apply_wizard which does the same thing as the existing ast_sorcery_apply_wizard_mapping function but has the added capabilities of ast_sorcery_object_type_insert_wizard. * ast_sorcery_object_type_remove_wizard which removes a wizard matching both its name and its original argument string. * The original logic in __ast_sorcery_insert_wizard_mapping was moved to __ast_sorcery_object_type_insert_wizard and enhanced for the new capabilities, then __ast_sorcery_insert_wizard_mapping was refactored to just call __ast_sorcery_insert_wizard_mapping. * Added a unit test to test_sorcery.c to test the read-only capability. Change-Id: I40f35840252e4313d99e11dbd80e270a3aa10605	6 years ago
George Joseph	c2adeb9dc2	taskprocessor: Enable subsystems and overload by subsystem ... To prevent one subsystem's taskprocessors from causing others to stall, new capabilities have been added to taskprocessors. * Any taskprocessor name that has a '/' will have the part before the '/' saved as its "subsystem". Examples: "sorcery/acl-0000006a" and "sorcery/aor-00000019" will be grouped to subsystem "sorcery". "pjsip/distributor-00000025" and "pjsip/distributor-00000026" will bn grouped to subsystem "pjsip". Taskprocessors with no '/' have an empty subsystem. * When a taskprocessor enters high-water alert status and it has a non-empty subsystem, the subsystem alert count will be incremented. * When a taskprocessor leaves high-water alert status and it has a non-empty subsystem, the subsystem alert count will be decremented. * A new api ast_taskprocessor_get_subsystem_alert() has been added that returns the number of taskprocessors in alert for the subsystem. * A new CLI command "core show taskprocessor alerted subsystems" has been added. * A new unit test was addded. REMINDER: The taskprocessor code itself doesn't take any action based on high-water alerts or overloading. It's up to taskprocessor users to check and take action themselves. Currently only the pjsip distributor does this. * A new pjsip/global option "taskprocessor_overload_trigger" has been added that allows the user to select the trigger mechanism the distributor uses to pause accepting new requests. "none": Don't pause on any overload condition. "global": Pause on ANY taskprocessor overload (the default and current behavior) "pjsip_only": Pause only on pjsip taskprocessor overloads. * The core pjsip pool was renamed from "SIP" to "pjsip" so it can be properly grouped into the "pjsip" subsystem. * stasis taskprocessor names were changed to "stasis" as the subsystem. * Sorcery core taskprocessor names were changed to "sorcery" to match the object taskprocessors. Change-Id: I8c19068bb2fc26610a9f0b8624bdf577a04fcd56	6 years ago
Corey Farrell	687ab7aeee	astobj2: Eliminate legacy container allocation macros. ... These macros have been documented as legacy for a long time but are still used in new code because they exist. Remove all references to: * ao2_container_alloc_options * ao2_t_container_alloc_options * ao2_t_container_alloc These macro's are also removed. Only ao2_container_alloc remains due to it's use in over 100 places. Change-Id: I1a26258b5bf3deb081aaeed11a0baa175c933c7a	7 years ago
Corey Farrell	fee929c8ac	core: Remove non-critical cleanup from startup aborts. ... When built-in components of Asterisk fail to start they cause the Asterisk startup to abort. In these cases only the most critical cleanup should be performed - closing databases and terminating proceses. These cleanups are registered using ast_register_atexit, all other cleanups should not be run during startup abort. The main reason for this change is that these cleanup procedures are untestable from the partially initialized states, if they fail it could prevent us from ever running the critical cleanup with ast_run_atexits. Create separate initialization for dns_core.c to be run unconditionally during startup instead of being initialized by the first dns resolver to be registered. This ensures that 'sched' is initialized before it can be potentially used. Replace ast_register_atexit with ast_register_cleanup in media_cache.c. There is no reason for this cleanup to happen unconditionally. Change-Id: Iecc2df98008b21509925ff16740bd5fa29527db3	7 years ago
Corey Farrell	55f1d69c43	loader: Create ast_module_running_ref. ... This function returns NULL if the module in question is not running. I did not change ast_module_ref as most callers do not check the result and they always call ast_module_unref. Make use of this function when running registered items from: * app_stack API's * bridge technologies * CLI commands * File formats * Manager Actions * RTP engines * Sorcery Wizards * Timing Interfaces * Translators * AGI Commands * Fax Technologies ASTERISK-20346 #close Change-Id: Ia16fd28e188b2fc0b9d18b8a5d9cacc31df73fcc	7 years ago
Corey Farrell	bc73337e07	core: Use macros to generate ao2_container callbacks where possible. ... This uses AO2_STRING_FIELD_HASH_FN and AO2_STRING_FIELD_CMP_FN where possible in the Asterisk core. This removes CMP_STOP from the result of CMP_FN callbacks for the following structure types: * ast_bucket_metadata * ast_bucket_scheme * generic_monitor_instance_list (ccss.c) * ast_bucket_file (media_cache.c) * named_acl Change-Id: Ide4c1449a894bce70dea1fef664dade9b57578f1	7 years ago
Sean Bright	ffccce76d9	sorcery: Add ast_sorcery_retrieve_by_prefix() ... Some consumers of the sorcery API use ast_sorcery_retrieve_by_regex only so that they can anchor the potential match as a prefix and not because they truly need regular expressions. Rather than using regular expressions for simple prefix lookups, add a new operation - ast_sorcery_retrieve_by_prefix - that does them. Change-Id: I56f4e20ba1154bd52281f995c27a429a854f6a79	8 years ago
Richard Mudgett	79ddcdbc70	Stasis/ARI: Fix off-nominal path json memory leaks. ... Change-Id: Id569c624c426e3b22a99936473c730592d8b83fb	8 years ago
Corey Farrell	b8dadccbe1	sorcery: Use ao2_weakproxy to hold list of instances. ... * Store weak proxy objects in instances container. * Remove special unreference function and replace with macro that calls ao2_cleanup. * Add REF_DEBUG information to ast_sorcery_open. Change-Id: I5a150a4e13cee319d46b5a4654f95a4623a978f8	8 years ago
Richard Mudgett	8d323c74fa	sorcery.c: Speed up ast_sorcery_retrieve_by_id() ... Return early if ast_sorcery_retrieve_by_id() is not passed an id to find. Also eliminated the RAII_VAR() usage in the function. Change-Id: I871dbe162a301b5ced8b4393cec27180c7c6b218	8 years ago
Corey Farrell	a6e5bae3ef	Remove ASTERISK_REGISTER_FILE. ... ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes all traces of it. Previously exported symbols removed: * __ast_register_file * __ast_unregister_file * ast_complete_source_filename This also removes the mtx_prof static variable that was declared when MTX_PROFILE was enabled. This variable was only used in lock.c so it is now initialized in that file only. ASTERISK-26480 #close Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966	9 years ago
Corey Farrell	e875e1c12a	sorcery: Create function ast_sorcery_lockable_alloc. ... Create an alternative to ast_sorcery_generic_alloc which uses astobj2 shared locking. Use this new method for the 'struct ast_sip_aor' allocator. Change-Id: I3f62f2ada64b622571950278fbb6ad57395b5d6f	9 years ago
Richard Mudgett	d526aa5cbe	sorcery.c: Minor optimizations. ... * Remove some unused parameters from internal functions: sorcery_wizard_create() sorcery_wizard_update() sorcery_wizard_delete() * Created the struct sorcery_observer_invocation ao2 object without a lock since it is not needed in sorcery_observer_invocation_alloc(). * Cleanup generic ao2 container sorcery object id hash, sort, and cmp functions. Change-Id: Iff71d75f52bc1b8cee955456838c149faaa4f92e	9 years ago
Richard Mudgett	45e143576f	sorcery.c: Tweak some container declaration formatting. ... * Tweak sorcery_object_type_alloc() formatting. * Tweak ast_sorcery_init() formatting. Change-Id: Ib02430023f15268cd7a2ea53f2c331213e4d3944	9 years ago
Mark Michelson	b6bd97eea2	Fix Alembic upgrades. ... A non-existent constraint was being referenced in the upgrade script. This patch corrects the problem by removing the reference. In addition, the head of the alembic branch referred to a non-existent revision. This has been fixed by referring to the proper revision. This patch fixes another realtime problem as well. Our Alembic scripts store booleans as yes or no values. However, Sorcery tries to insert "true" or "false" instead. This patch introduces a new boolean type that translates to "yes" or "no" instead. ASTERISK-26128 #close Change-Id: I51574736a881189de695a824883a18d66a52dcef	9 years ago
Richard Mudgett	4879cd875c	sorcery: Add setting object type congestion levels. ... Sorcery creates taskprocessors for object types to process object observer callbacks. An API call is needed to be able to set the congestion levels of these taskprocessors for selected object types. * Updated PJSIP's contact and contact_status sorcery object type observer default congestion levels based upon stress testing. Increased the congestion levels to reduce the potential for bursty register/unregister and subscribe/unsubscribe activity from triggering the taskprocessor overload alert. ASTERISK-26088 Reported by: Richard Mudgett Change-Id: I4542e83b556f0714009bfeff89505c801f1218c6	9 years ago
George Joseph	4422905218	sorcery: Refactor create, update and delete to better deal with caches ... The ast_sorcery_create, update and delete function have been refactored to better deal with caches and errors. The action is now called on all non-caching wizards first. If ANY succeed, the action is called on all caching wizards and the observers are notified. This way we don't put something in the cache (or update or delete) before knowing the action was performed in at least 1 backend and we only call the observers once even if there were multiple writable backends. ast_sorcery_create was never adding to caches in the first place which was preventing contacts from getting added to a memory_cache when they were created. In turn this was causing memory_cache to emit errors if the contact was deleted before being retrieved (which would have populated the cache). ASTERISK-25811 #close Reported-by: Ross Beer Change-Id: Id5596ce691685a79886e57b0865888458d6e7b46	9 years ago
Richard Mudgett	f0f5fbbc01	Sorcery: Create human friendly serializer names. ... Sorcery name formats: sorcery/<type>-<seq> -- Sorcery thread pool serializer Change-Id: Idc2e5d3dbab15c825b97c38c028319a0d2315c47	9 years ago
Joshua Colp	902309fd04	res_sorcery_memory_cache: Add support for a full backend cache. ... This change introduces the configuration option 'full_backend_cache' which changes the cache to be a full mirror of the backend instead of a per-object cache. This allows all sorcery retrieval operations to be carried out against it and is useful for object types which are used in a "retrieve all" or "retrieve some" pattern. ASTERISK-25625 #close Change-Id: Ie2993487e9c19de563413ad5561c7403b48caab5	9 years ago
Alexander Traud	cf79b62778	ast_format_cap_get_names: To display all formats, the buffer was increased. ... ASTERISK-25533 #close Change-Id: Ie1a9d1a6511b3f1a56b93d04475fbf8a4e40010a	10 years ago
Scott Griepentrog	178e1adffb	CHAOS: prevent sorcery object with null id ... When allocating a sorcery object, fail if the id value was not allocated. ASTERISK-25323 Reported by: Scott Griepentrog Change-Id: I152133fb7545a4efcf7a0080ada77332d038669e	10 years ago
Matt Jordan	458715d088	main/sorcery: Don't fail object set creation from JSON if field fails ... Some individual fields may fail their conversion due to their default values being invalid for their custom handlers. In particular, configuration values that depend on others being enabled (and thus have an empty default value) are notorious for tripping this routine up. An example of this are any of the DTLS options for endpoints. Any of the DTLS options will fail to be applied (as DTLS is not enabled), causing the entire object set to be aborted. This patch makes it so that we log a debug message when skipping a field, and rumble on anyway. ASTERISK-25238 Change-Id: I0bea13de79f66bf9f9ae6ece0e94a2dc1c026a76	10 years ago
Matt Jordan	ffadb5f1de	main/sorcery: Provide log messages when a wizard does not support an operation ... If a sorcery wizard does not support one of the 'optional' CRUD operations (namely the CUD), log a WARNING message so we are aware of why the operation failed. This also removes an assert in this case, as the CUD operation may have been triggered by an external system, in which case it is not a programming error but a configuration error. Change-Id: Ifecd9df946d9deaa86235257b49c6e5e24423b53	10 years ago
Matt Jordan	b178f8701b	sorcery: Add support for object staleness ... This patch enhances the sorcery API to allow for sorcery wizards to determine if an object is stale. This includes the following: * Sorcery objects now have a timestamp that is set on creation. Since sorcery objects are immutable, this can be used by sorcery wizards to determine if an object is stale. * A new API call has been added, ast_sorcery_is_stale. This API call queries the wizards associated with the object, calling a new callback function 'is_stale'. Note that if a wizard does not support the new callback, objects are always assumed to not be stale. * Unit tests have been added that cover the new API call. Change-Id: Ica93c6a4e8a06c0376ea43e00cf702920b806064	10 years ago
Matt Jordan	ef74ccb18d	sorcery/realtime: Add a bit of debug and warning messages for bad configs ... When a mapping does not exist between a sorcery.conf defined object and a realtime mapping in extconf, currently, the user will receive a slew of ERROR messages that don't really tell what is happening. Some ERROR messages may even be misleading, as they occur after the sorcery API has already given up on the attempt to load and create the sorcery object. This patch adds a bit of debug and a useful WARNING message for when a wizard's open callback fails for a particular object type. In the bad configurations that resulted in this patch, this provided a 'root cause' WARNING message that pointed in the right direction of the configuration problem. Change-Id: I1cc7344f2b015b8b9c85a7e6ebc8cb4753a8f80b	10 years ago
Richard Mudgett	9ec8a0f3cc	sorcery: Add ast_sorcery_object_unregister() API call. ... Find and unlink the specified sorcery object type to complement ast_sorcery_object_register(). Without this function you cannot completely unload individual modules that use sorcery for configuration. ASTERISK-24907 Reported by: Kevin Harwell Change-Id: I1c04634fe9a90921bf676725c7d6bb2aeaab1c88	10 years ago
Joshua Colp	a7af6bca3c	sorcery: Fix cache creation callback. ... The cache creation callback function expects to receive a sorcery_details structure and not just a standalone object. Change-Id: I3e4a5a137cb25292eb52d7a14cbb6daa09213450	10 years ago
Joshua Colp	f2cc766d81	res_sorcery_memory_cache: Add basic module implementation. ... This change adds a basic res_sorcery_memory_cache module which implements configuration option parsing, configuration file parsing for threading, sorcery interface implementation, and unit tests. Objects can be added, updated, deleted, and retrieved from the memory cache. Automatic expiration and stale handling will be added in the future. Note that unit tests exist within the module itself in case the threading done as a result of expiration results in asynchronous actions (which it likely will). Providing access and a notification mechanism for an external test module would be complicated and not worth it. ASTERISK-25067 #close Reported by: Matt Jordan Change-Id: Id8a6a357ef5a83d466f81eee56a67d13eeb118b9	10 years ago
George Joseph	52407088f8	sorcery: Add API to insert/remove a wizard to/from an object type's list ... Currently you can 'apply' a wizard to an object type but the wizard always goes at the end of the object type's wizard list. This patch adds a new ast_sorcery_insert_wizard_mapping function that allows you to insert a wizard anyplace in the list. I.E. You could add a caching wizard to an object type and place it before all wizards. ast_sorcery_get_wizard_mapping_count and ast_sorcery_get_wizard_mapping were added to allow examination of the mapping list. ast_sorcery_remove_mapping was added to remove a mapping by name. As part of this patch, the object type's wizard list was converted from an ao2_container to an AST_VECTOR_RW. A new test was added to test_sorcery for this capability. ASTERISK-25044 #close Change-Id: I9d2469a9296b2698082c0989e25e6848dc403b57	10 years ago
Matt Jordan	4a58261694	git migration: Refactor the ASTERISK_FILE_VERSION macro ... Git does not support the ability to replace a token with a version string during check-in. While it does have support for replacing a token on clone, this is somewhat sub-optimal: the token is replaced with the object hash, which is not particularly easy for human consumption. What's more, in practice, the source file version was often not terribly useful. Generally, when triaging bugs, the overall version of Asterisk is far more useful than an individual SVN version of a file. As a result, this patch removes Asterisk's support for showing source file versions. Specifically, it does the following: * Rename ASTERISK_FILE_VERSION macro to ASTERISK_REGISTER_FILE, and remove passing the version in with the macro. Other facilities than 'core show file version' make use of the file names, such as setting a debug level only on a specific file. As such, the act of registering source files with the Asterisk core still has use. The macro rename now reflects the new macro purpose. * main/asterisk: - Refactor the file_version structure to reflect that it no longer tracks a version field. - Remove the "core show file version" CLI command. Without the file version, it is no longer useful. - Remove the ast_file_version_find function. The file version is no longer tracked. - Rename ast_register_file_version/ast_unregister_file_version to ast_register_file/ast_unregister_file, respectively. * main/manager: Remove value from the Version key of the ModuleCheck Action. The actual key itself has not been removed, as doing so would absolutely constitute a backwards incompatible change. However, since the file version is no longer tracked, there is no need to attempt to include it in the Version key. * UPGRADE: Add notes for: - Modification to the ModuleCheck AMI Action - Removal of the "core show file version" CLI command Change-Id: I6cf0ff280e1668bf4957dc21f32a5ff43444a40e	10 years ago
Corey Farrell	3ddd92902a	Replace most uses of ast_register_atexit with ast_register_cleanup. ... Since 'core stop now' and 'core restart now' do not stop modules, it is unsafe for most of the core to run cleanups. Originally all cleanups used ast_register_atexit, and were only changed when it was shown to be unsafe. ast_register_atexit is now used only when absolutely required to prevent corruption and close child processes. Exceptions that need to use ast_register_atexit: * CDR: Flush records. * res_musiconhold: Kill external applications. * AstDB: Close the DB. * canary_exit: Kill canary process. ASTERISK-24142 #close Reported by: David Brillert ASTERISK-24683 #close Reported by: Peter Katzmann ASTERISK-24805 #close Reported by: Badalian Vyacheslav ASTERISK-24881 #close Reported by: Corey Farrell Review: https://reviewboard.asterisk.org/r/4500/ Review: https://reviewboard.asterisk.org/r/4501/ ........ Merged revisions 433495 from http://svn.asterisk.org/svn/asterisk/branches/11 ........ Merged revisions 433497 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433498 65c4cc65-6c06-0410-ace0-fbb531ad65f3	10 years ago
Richard Mudgett	148e8799fe	res_pjsip_sdp_rtp,sorcery: Fix invalid access and memory leak respectively. ... Valgrind found a memory leak and invalid access. * Fix invalid access by sscanf() being fed a non-nul terminated string of digits in res/res_pjsip_sdp_rtp.c:get_codecs(). * Fix memory leak in main/sorcery.c:sorcery_object_field_destructor(). * Fix potential NULL pointer dereference in main/xmldoc.c:xmldoc_get_syntax_config_option(). Review: https://reviewboard.asterisk.org/r/4513/ ........ Merged revisions 433199 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433200 65c4cc65-6c06-0410-ace0-fbb531ad65f3	10 years ago
Joshua Colp	455a98a2f8	sorcery: Output an error message if a wizard is specified for an object type and it isn't found. ... ASTERISK-24612 #close Reported by: Joshua Colp ........ Merged revisions 431771 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@431772 65c4cc65-6c06-0410-ace0-fbb531ad65f3	10 years ago
Joshua Colp	03ce56d6c5	sorcery: Don't try to load object types which haven't been defined. ... The act of defining wizards for an object type in sorcery.conf will create a minimal object type. This can cause a problem when a module has multiple sorcery instances (which all get the wizards from sorcery.conf applied) but the sorcery instances do not all contain full information about the object types. Upon loading errors will occur stating that the objects can not be created. This is confusing and is actually perfectly fine. This change makes it so that only object types which have been fully defined will be loaded. ASTERISK-24748 #close Reported by: Joshua Colp ........ Merged revisions 431538 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@431539 65c4cc65-6c06-0410-ace0-fbb531ad65f3	10 years ago
Richard Mudgett	69e107b24e	res_pjsip_outbound_registration: Fix reload race condition. ... Performing a CLI "module reload" command when there are new pjsip.conf registration objects defined frequently failed to load them correctly. What happens is a race condition between res_pjsip pushing its reload into an asynchronous task processor task and the thread that does the rest of the reloads when it gets to reloading the res_pjsip_outbound_registration module. A similar race condition happens between a reload and the CLI/AMI show registrations commands. The reload updates the current_states container and the CLI/AMI commands call get_registrations() which builds a new current_states container. * Made res_pjsip.c reload_module() use ast_sip_push_task_synchronous() instead of ast_sip_push_task() to eliminate two threads processing config reloads at the same time. * Made get_registrations() not replace the global current_states container so the CLI/AMI show registrations command cannot interfere with reloading. You could never add/remove objects in the container without the possibility of the container being replaced out from under you by get_registrations(). * Added a registration loaded sorcery instance observer to purge any dead registration objects since get_registrations() cannot do this job anymore. The struct ast_sorcery_instance_observer callbacks must be used because the callback happens inline with the load process. The struct ast_sorcery_observer callbacks are pushed to a different thread. * Added some global current_states NULL pointer checks in case the container disappears because of unload_module(). * Made sorcery's struct ast_sorcery_instance_observer.object_type_loaded callbacks guaranteed to be called before any struct ast_sorcery_observer.loaded callbacks will be called. * Moved the check for non-reloadable objects to before the sorcery instance loading callbacks happen to short circuit unnecessary work. Previously with non-reloadable objects, the sorcery instance loading/loaded callbacks would always happen, the individual wizard loading/loaded would be prevented, and the non-reloadable type logging message would be logged for each associated wizard. ASTERISK-24729 #close Review: https://reviewboard.asterisk.org/r/4381/ ........ Merged revisions 431243 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@431251 65c4cc65-6c06-0410-ace0-fbb531ad65f3	10 years ago
George Joseph	d04445c24a	sorcery: Add additional observer capabilities. ... Add new global, instance and wizard observers. instance_created wizard_registered wizard_unregistered instance_destroying instance_loading instance_loaded wizard_mapped object_type_registered object_type_loading object_type_loaded wizard_loading wizard_loaded Tested-by: George Joseph Review: https://reviewboard.asterisk.org/r/4215/ ........ Merged revisions 428999 from http://svn.asterisk.org/svn/asterisk/branches/12 ........ Merged revisions 429000 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@429001 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
George Joseph	4394e0431c	sorcery: Make is_object_field_registered handle field names that are regexes. ... As a result of https://reviewboard.asterisk.org/r/3305, res_sorcery_realtime was tossing database fields that didn't have an exact match to a sorcery registered field. This broke the ability to use regexes as field names which manifested itself as a failure of res_pjsip_phoneprov_provider which uses this capability. It also broke handling of fields that start with '@' in realtime but I don't think anyone noticed. This patch does the following... * Modifies ast_sorcery_fields_register to pre-compile the name regex. * Modifies ast_sorcery_is_object_field_registered to test the regex if it exists instead of doing an exact strcmp. * Modifies res_pjsip_phoneprov_provider with a few tweaks to get it to work with realtime. Tested-by: George Joseph Review: https://reviewboard.asterisk.org/r/4185/ ........ Merged revisions 428543 from http://svn.asterisk.org/svn/asterisk/branches/12 ........ Merged revisions 428544 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@428545 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
George Joseph	4967478d18	sorcery: Prevent SEGV in sorcery_wizard_create when there's no create function ... When you call ast_sorcery_create() you don't necessarily know which wizard is going to be invoked. If it happens to be a wizard like 'config' that doesn't have a 'create' virtual function you get a segfault in the sorcery_wizard_create callback. This patch catches the null function pointer, does an ast_assert, and logs an error. Review: https://reviewboard.asterisk.org/r/4044/ ........ Merged revisions 424447 from http://svn.asterisk.org/svn/asterisk/branches/12 ........ Merged revisions 424448 from http://svn.asterisk.org/svn/asterisk/branches/13 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@424449 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Matthew Jordan	a2c912e997	media formats: re-architect handling of media for performance improvements ... In the old times media formats were represented using a bit field. This was fast but had a few limitations. 1. Asterisk was limited in how many formats it could handle. 2. Formats, being a bit field, could not include any attribute information. A format was strictly its type, e.g., "this is ulaw". This was changed in Asterisk 10 (see https://wiki.asterisk.org/wiki/display/AST/Media+Architecture+Proposal for notes on that work) which led to the creation of the ast_format structure. This structure allowed Asterisk to handle attributes and bundle information with a format. Additionally, ast_format_cap was created to act as a container for multiple formats that, together, formed the capability of some entity. Another mechanism was added to allow logic to be registered which performed format attribute negotiation. Everywhere throughout the codebase Asterisk was changed to use this strategy. Unfortunately, in software, there is no free lunch. These new capabilities came at a cost. Performance analysis and profiling showed that we spend an inordinate amount of time comparing, copying, and generally manipulating formats and their related structures. Basic prototyping has shown that a reasonably large performance improvement could be made in this area. This patch is the result of that project, which overhauled the media format architecture and its usage in Asterisk to improve performance. Generally, the new philosophy for handling formats is as follows: * The ast_format structure is reference counted. This removed a large amount of the memory allocations and copying that was done in prior versions. * In order to prevent race conditions while keeping things performant, the ast_format structure is immutable by convention and lock-free. Violate this tenet at your peril! * Because formats are reference counted, codecs are also reference counted. The Asterisk core generally provides built-in codecs and caches the ast_format structures created to represent them. Generally, to prevent inordinate amounts of module reference bumping, codecs and formats can be added at run-time but cannot be removed. * All compatibility with the bit field representation of codecs/formats has been moved to a compatibility API. The primary user of this representation is chan_iax2, which must continue to maintain its bit-field usage of formats for interoperability concerns. * When a format is negotiated with attributes, or when a format cannot be represented by one of the cached formats, a new format object is created or cloned from an existing format. That format may have the same codec underlying it, but is a different format than a version of the format with different attributes or without attributes. * While formats are reference counted objects, the reference count maintained on the format should be manipulated with care. Formats are generally cached and will persist for the lifetime of Asterisk and do not explicitly need to have their lifetime modified. An exception to this is when the user of a format does not know where the format came from *and* the user may outlive the provider of the format. This occurs, for example, when a format is read from a channel: the channel may have a format with attributes (hence, non-cached) and the user of the format may last longer than the channel (if the reference to the channel is released prior to the format's reference). For more information on this work, see the API design notes: https://wiki.asterisk.org/wiki/display/AST/Media+Format+Rewrite Finally, this work was the culmination of a large number of developer's efforts. Extra thanks goes to Corey Farrell, who took on a large amount of the work in the Asterisk core, chan_sip, and was an invaluable resource in peer reviews throughout this project. There were a substantial number of patches contributed during this work; the following issues/patch names simply reflect some of the work (and will cause the release scripts to give attribution to the individuals who work on them). Reviews: https://reviewboard.asterisk.org/r/3814 https://reviewboard.asterisk.org/r/3808 https://reviewboard.asterisk.org/r/3805 https://reviewboard.asterisk.org/r/3803 https://reviewboard.asterisk.org/r/3801 https://reviewboard.asterisk.org/r/3798 https://reviewboard.asterisk.org/r/3800 https://reviewboard.asterisk.org/r/3794 https://reviewboard.asterisk.org/r/3793 https://reviewboard.asterisk.org/r/3792 https://reviewboard.asterisk.org/r/3791 https://reviewboard.asterisk.org/r/3790 https://reviewboard.asterisk.org/r/3789 https://reviewboard.asterisk.org/r/3788 https://reviewboard.asterisk.org/r/3787 https://reviewboard.asterisk.org/r/3786 https://reviewboard.asterisk.org/r/3784 https://reviewboard.asterisk.org/r/3783 https://reviewboard.asterisk.org/r/3778 https://reviewboard.asterisk.org/r/3774 https://reviewboard.asterisk.org/r/3775 https://reviewboard.asterisk.org/r/3772 https://reviewboard.asterisk.org/r/3761 https://reviewboard.asterisk.org/r/3754 https://reviewboard.asterisk.org/r/3753 https://reviewboard.asterisk.org/r/3751 https://reviewboard.asterisk.org/r/3750 https://reviewboard.asterisk.org/r/3748 https://reviewboard.asterisk.org/r/3747 https://reviewboard.asterisk.org/r/3746 https://reviewboard.asterisk.org/r/3742 https://reviewboard.asterisk.org/r/3740 https://reviewboard.asterisk.org/r/3739 https://reviewboard.asterisk.org/r/3738 https://reviewboard.asterisk.org/r/3737 https://reviewboard.asterisk.org/r/3736 https://reviewboard.asterisk.org/r/3734 https://reviewboard.asterisk.org/r/3722 https://reviewboard.asterisk.org/r/3713 https://reviewboard.asterisk.org/r/3703 https://reviewboard.asterisk.org/r/3689 https://reviewboard.asterisk.org/r/3687 https://reviewboard.asterisk.org/r/3674 https://reviewboard.asterisk.org/r/3671 https://reviewboard.asterisk.org/r/3667 https://reviewboard.asterisk.org/r/3665 https://reviewboard.asterisk.org/r/3625 https://reviewboard.asterisk.org/r/3602 https://reviewboard.asterisk.org/r/3519 https://reviewboard.asterisk.org/r/3518 https://reviewboard.asterisk.org/r/3516 https://reviewboard.asterisk.org/r/3515 https://reviewboard.asterisk.org/r/3512 https://reviewboard.asterisk.org/r/3506 https://reviewboard.asterisk.org/r/3413 https://reviewboard.asterisk.org/r/3410 https://reviewboard.asterisk.org/r/3387 https://reviewboard.asterisk.org/r/3388 https://reviewboard.asterisk.org/r/3389 https://reviewboard.asterisk.org/r/3390 https://reviewboard.asterisk.org/r/3321 https://reviewboard.asterisk.org/r/3320 https://reviewboard.asterisk.org/r/3319 https://reviewboard.asterisk.org/r/3318 https://reviewboard.asterisk.org/r/3266 https://reviewboard.asterisk.org/r/3265 https://reviewboard.asterisk.org/r/3234 https://reviewboard.asterisk.org/r/3178 ASTERISK-23114 #close Reported by: mjordan media_formats_translation_core.diff uploaded by kharwell (License 6464) rb3506.diff uploaded by mjordan (License 6283) media_format_app_file.diff uploaded by kharwell (License 6464) misc-2.diff uploaded by file (License 5000) chan_mild-3.diff uploaded by file (License 5000) chan_obscure.diff uploaded by file (License 5000) jingle.diff uploaded by file (License 5000) funcs.diff uploaded by file (License 5000) formats.diff uploaded by file (License 5000) core.diff uploaded by file (License 5000) bridges.diff uploaded by file (License 5000) mf-codecs-2.diff uploaded by file (License 5000) mf-app_fax.diff uploaded by file (License 5000) mf-apps-3.diff uploaded by file (License 5000) media-formats-3.diff uploaded by file (License 5000) ASTERISK-23715 rb3713.patch uploaded by coreyfarrell (License 5909) rb3689.patch uploaded by mjordan (License 6283) ASTERISK-23957 rb3722.patch uploaded by mjordan (License 6283) mf-attributes-3.diff uploaded by file (License 5000) ASTERISK-23958 Tested by: jrose rb3822.patch uploaded by coreyfarrell (License 5909) rb3800.patch uploaded by jrose (License 6182) chan_sip.diff uploaded by mjordan (License 6283) rb3747.patch uploaded by jrose (License 6182) ASTERISK-23959 #close Tested by: sgriepentrog, mjordan, coreyfarrell sip_cleanup.diff uploaded by opticron (License 6273) chan_sip_caps.diff uploaded by mjordan (License 6283) rb3751.patch uploaded by coreyfarrell (License 5909) chan_sip-3.diff uploaded by file (License 5000) ASTERISK-23960 #close Tested by: opticron direct_media.diff uploaded by opticron (License 6273) pjsip-direct-media.diff uploaded by file (License 5000) format_cap_remove.diff uploaded by opticron (License 6273) media_format_fixes.diff uploaded by opticron (License 6273) chan_pjsip-2.diff uploaded by file (License 5000) ASTERISK-23966 #close Tested by: rmudgett rb3803.patch uploaded by rmudgetti (License 5621) chan_dahdi.diff uploaded by file (License 5000) ASTERISK-24064 #close Tested by: coreyfarrell, mjordan, opticron, file, rmudgett, sgriepentrog, jrose rb3814.patch uploaded by rmudgett (License 5621) moh_cleanup.diff uploaded by opticron (License 6273) bridge_leak.diff uploaded by opticron (License 6273) translate.diff uploaded by file (License 5000) rb3795.patch uploaded by rmudgett (License 5621) tls_fix.diff uploaded by mjordan (License 6283) fax-mf-fix-2.diff uploaded by file (License 5000) rtp_transfer_stuff uploaded by mjordan (License 6283) rb3787.patch uploaded by rmudgett (License 5621) media-formats-explicit-translate-format-3.diff uploaded by file (License 5000) format_cache_case_fix.diff uploaded by opticron (License 6273) rb3774.patch uploaded by rmudgett (License 5621) rb3775.patch uploaded by rmudgett (License 5621) rtp_engine_fix.diff uploaded by opticron (License 6273) rtp_crash_fix.diff uploaded by opticron (License 6273) rb3753.patch uploaded by mjordan (License 6283) rb3750.patch uploaded by mjordan (License 6283) rb3748.patch uploaded by rmudgett (License 5621) media_format_fixes.diff uploaded by opticron (License 6273) rb3740.patch uploaded by mjordan (License 6283) rb3739.patch uploaded by mjordan (License 6283) rb3734.patch uploaded by mjordan (License 6283) rb3689.patch uploaded by mjordan (License 6283) rb3674.patch uploaded by coreyfarrell (License 5909) rb3671.patch uploaded by coreyfarrell (License 5909) rb3667.patch uploaded by coreyfarrell (License 5909) rb3665.patch uploaded by mjordan (License 6283) rb3625.patch uploaded by coreyfarrell (License 5909) rb3602.patch uploaded by coreyfarrell (License 5909) format_compatibility-2.diff uploaded by file (License 5000) core.diff uploaded by file (License 5000) git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@419044 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Mark Michelson	eefcb79bfb	Prevent duplicate sorcery wizards from being applied to sorcery object types. ... This commit contains several changes to sorcery: 1) Application of sorcery configuration based on module name is automatically performed when sorcery is opened for a module. 2) Sorcery will not attempt to apply the same wizard to an object type more than once. 3) Sorcery gives more exact results when attempting to apply a wizard, whether as the default or based on configuration. Sorcery unit tests still pass for me after making these changes. Review: https://reviewboard.asterisk.org/r/3326 ........ Merged revisions 411159 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@411656 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Mark Michelson	a8629e53c1	Give sorcery instances a reference to their wizards. ... On graceful shutdown, sorcery wizards are all killed off, but it is possible for sorcery instances to still have dangling pointers after this, possibly causing a crash. Giving the sorcery instances a reference to their wizards ensures that the wizard reference will remain valid for the lifetime of the sorcery instance. Review: https://reviewboard.asterisk.org/r/3401 ........ Merged revisions 411295 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@411296 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Mark Michelson	eba91d2a98	Revert changes to sorcery that accidentally got committed. ... These changes were still up for review and have not been approved yet. I must have had the changes in my working copy when making a different change. ........ Merged revisions 410696 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410699 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Mark Michelson	d44aefeef4	Fix stuck channel in ARI through the introduction of synchronous bridge actions. ... Playing back a file to a channel in an ARI bridge would attempt to wait until the playback concluded before returning. The method used involved signaling the waiting thread in the ARI custom playback function. The problem with this is that there were some corner cases that were not accounted for: * If a bridge channel could not be found, then we never would attempt the playback but would still attempt to wait for the playback to complete. * If the bridge playfile action failed to queue, we would still attempt to wait for the playback to complete. * If the bridge playfile action were queued but some circumstance caused the playback not to occur (the bridge dies, the channel is removed from the bridge), then we would never be notified. The solution to this is to move the waiting logic into the bridge code. A new bridge API function is added to queue a synchronous action on a bridge. The waiting thread is notified when the queued frame has been freed, either due to an error occurring or due to successful playback. As a failsafe, the waiting thread has a 10 minute timeout just in case there is a frame leak somewhere. Review: https://reviewboard.asterisk.org/r/3338 ........ Merged revisions 410673 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410684 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
George Joseph	3ff60b75b1	pjsip_cli: Create pjsip show channel and contact, and general cli code cleanup. ... Created the 'pjsip show channel' and 'pjsip show contact' commands. Refactored out the hated ast_hashtab. Replaced with ao2_container. Cleaned up function naming. Internal only, no public name changes. Cleaned up whitespace and brace formatting in cli code. Changed some NULL checking from "if"s to ast_asserts. Fixed some register/unregister ordering to reduce deadlock potential. Fixed ast_sip_location_add_contact where the 'name' buffer was too short. Fixed some self-assignment issues in res_pjsip_outbound_registration. (closes issue ASTERISK-23276) Review: http://reviewboard.asterisk.org/r/3283/ ........ Merged revisions 410287 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410288 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Mark Michelson	c162101d69	Make res_sorcery_realtime filter unknown retrieved results. ... When retrieving data from a database or other realtime backend, it's quite possible to retrieve variables that Asterisk does not care about but that are legitimate to exist. Asterisk does not need to throw a hissy fit when these variables are encountered but rather just filter them out. Review: https://reviewboard.asterisk.org/r/3305 ........ Merged revisions 410187 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@410207 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago