61 Commits (6531f8b2ea50c0e9ead60f29f355912df907c43b)

Author	SHA1	Message	Date
George Joseph	c87158f2da	app.c: Remove deletion of pool topic on mwi state delete ... As part of an earlier voicemail refactor, ast_delete_mwi_state_full was modified to remove the pool topic for a mailbox when the state was deleted. This was an attempt to prevent stale topics from accumulating when app_voicemail was reloaded and a mailbox went away. Unfortunately because of the fact that when app_voicemail reloads, ALL mailboxes are deleted then only current ones recreated, topics were being removed from the pool that still had subscribers on them, then recreated as new topics of the same name. So now modules like res_pjsip_mwi are listening on a topic that will never receive any messages because app_voicemail is publishing on a different topic that happens to have the same name. The solutiuon to this is not easy and given that accumulating topics for deleted mailboxes is less evil that not sending NOTIFYs... * Removed the call to stasis_topic_pool_delete_topic in ast_delete_mwi_state_full. Also: * Fixed a topic reference leak in res_pjsip_mwi mwi_stasis_subscription_alloc. * Added some debugging to mwi_stasis_subscription_alloc, stasis_topic_create, and topic_dtor. * Fixed a topic reference leak in an error path in internal_stasis_subscribe. ASTERISK-28306 Reported-by: Jared Hull Change-Id: Id7da0990b3ac4be4b58491536b35f41291247b27	6 years ago
Joshua Colp	07b3253155	stasis: Improve topic/subscription names and statistics. ... Topic names now follow: <subsystem>:<functionality>[/<object>] This ensures that they are all unique, and also provides better insight in to what each topic is for. Subscriber ids now also use the main topic name they are subscribed to and an incrementing integer as their identifier to make it easier to understand what the subscription is primarily responsible for. Both the CLI commands for listing topic and subscription statistics now sort to make it a bit easier to see what is going on. Subscriptions will now show all topics that they are receiving messages from, not just the main topic they were subscribed to. ASTERISK-28335 Change-Id: I484e971a38c3640f2bd156282e532eed84bf220d	6 years ago
Joshua C. Colp	9921262c85	Merge "taskprocessor: Enable subsystems and overload by subsystem" into 13	6 years ago
Joshua C. Colp	82758295df	stasis: Store subscriber uniqueids with topic statistics. ... This change provides an easier mechanism to determine which subscribers are subscribed to a topic. Using this you can inspect the specific subscribers for further details. Change-Id: I8deea21703cd5c5357b85593b46c3eaf24e18c0c	6 years ago
George Joseph	bae3fd04c1	taskprocessor: Enable subsystems and overload by subsystem ... To prevent one subsystem's taskprocessors from causing others to stall, new capabilities have been added to taskprocessors. * Any taskprocessor name that has a '/' will have the part before the '/' saved as its "subsystem". Examples: "sorcery/acl-0000006a" and "sorcery/aor-00000019" will be grouped to subsystem "sorcery". "pjsip/distributor-00000025" and "pjsip/distributor-00000026" will bn grouped to subsystem "pjsip". Taskprocessors with no '/' have an empty subsystem. * When a taskprocessor enters high-water alert status and it has a non-empty subsystem, the subsystem alert count will be incremented. * When a taskprocessor leaves high-water alert status and it has a non-empty subsystem, the subsystem alert count will be decremented. * A new api ast_taskprocessor_get_subsystem_alert() has been added that returns the number of taskprocessors in alert for the subsystem. * A new CLI command "core show taskprocessor alerted subsystems" has been added. * A new unit test was addded. REMINDER: The taskprocessor code itself doesn't take any action based on high-water alerts or overloading. It's up to taskprocessor users to check and take action themselves. Currently only the pjsip distributor does this. * A new pjsip/global option "taskprocessor_overload_trigger" has been added that allows the user to select the trigger mechanism the distributor uses to pause accepting new requests. "none": Don't pause on any overload condition. "global": Pause on ANY taskprocessor overload (the default and current behavior) "pjsip_only": Pause only on pjsip taskprocessor overloads. * The core pjsip pool was renamed from "SIP" to "pjsip" so it can be properly grouped into the "pjsip" subsystem. * stasis taskprocessor names were changed to "stasis" as the subsystem. * Sorcery core taskprocessor names were changed to "sorcery" to match the object taskprocessors. Change-Id: I8c19068bb2fc26610a9f0b8624bdf577a04fcd56	6 years ago
George Joseph	e3439d3011	Merge "stasis: Fix ABI between DEVMODE and non-DEVMODE." into 13	7 years ago
Richard Mudgett	b26ec935ec	stasic.c: Fix printf format type mismatches with arguments. ... An int64_t is not likely the same size as a long. * Changed the int64_t values in the statistics structs to longs so casting is not necessary when generating the formatted CLI output. The offending members did not need to be int64_t anyway as they were only set by an int type variable which was already truncating bits. * Reordered the statistics structs to reduce potential padding bytes. Change-Id: Ic090a070e9dc4ca650ebdb9c01ed50a581289962	7 years ago
Corey Farrell	240f339938	stasis: Fix ABI between DEVMODE and non-DEVMODE. ... Create compatibility stubs for maximum ABI compatibility. ASTERISK-28212 #close Change-Id: I872c04842ab6b61e9dd6d37e4166bc619aa20626	7 years ago
Joshua C. Colp	e5605232df	stasis: Add statistics gathering in developer mode. ... This change adds statistics gathering to Stasis topics, subscriptions, and message types. These can be viewed using CLI commands and provide insight into how Stasis is used and how long certain operations take to execute. These are only available when Asterisk is compiled in developer mode and do not have any impact under normal operation. ASTERISK-28117 Change-Id: I94411b53767f89ee01714daaecf0c2f1666e863f	7 years ago
George Joseph	21a5ad0fd9	stasis: Allow filtering by formatter ... A subscriber can now indicate that it only wants messages that have formatters of a specific type. For instance, manager can indicate that it only wants messages that have a "to_ami" formatter. You can combine this with the existing filter for message type to get only messages with specific formatters or messages of specific types. ASTERISK-28186 Change-Id: Ifdb7a222a73b6b56c6bb9e4ee93dc8a394a5494c	7 years ago
Corey Farrell	0a9904e1c6	astobj2: Eliminate usage of legacy ao2_container_alloc routine. ... Replace usage of ao2_container_alloc with ao2_container_alloc_hash or ao2_container_alloc_list. ao2_container_alloc is now restricted to modules only and is being removed from Asterisk 17. Change-Id: I0907d78bc66efc775672df37c8faad00f2f6c088	7 years ago
Joshua C. Colp	a5a8cba8c7	stasis: Remove stringfields and lock from change message. ... When a subscribe or unsubscribe occurs a message is published containing this information. This change makes it so that the message no longer uses stringfields or a lock, as both are not really needed for the message. Change-Id: I3f4831931d79f94fd979baf48048738df5dc1632	7 years ago
Joshua Colp	d748ed4147	stasis: Add internal filtering of messages. ... This change adds the ability for subscriptions to indicate which message types they are interested in accepting. By doing so the filtering is done before being dispatched to the subscriber, reducing the amount of work that has to be done. This is optional and if a subscriber does not add message types they wish to accept and set the subscription to selective filtering the previous behavior is preserved and they receive all messages. There is also the ability to explicitly force the reception of all messages for cases such as AMI or ARI where a large number of messages are expected that are then generically converted into a different format. ASTERISK-28103 Change-Id: I99bee23895baa0a117985d51683f7963b77aa190	7 years ago
Corey Farrell	01716f70e6	taskprocessor: Warn on unused result from pushing task. ... Add attribute_warn_unused_result to ast_taskprocessor_push, ast_taskprocessor_push_local and ast_threadpool_push. This will help ensure we perform the necessary cleanup upon failure. Change-Id: I7e4079bd7b21cfe52fb431ea79e41314520c3f6d	7 years ago
George Joseph	1d141112b5	stasis: Add function to delete topic from pool ... There's been a long standing leak when using topic pools. The topics in the pool get cleaned up when the last pool reference is released but you can't remove a topic specifically. If you reloaded app_voicemail for instance, and mailboxes went away, their topics were left in the pool. * Added stasis_topic_pool_delete_topic() so modules can clean up topics from pools. * Registered the topic pool containers so it can be examined from the CLI when AO2_DEBUG is enabled. They'll be named "<topic_pool_name>-pool". Change-Id: Ib7957951ee5c9b9b4482af7b9b4349112d62bc25	7 years ago
Joshua Colp	dd1b4b38cf	devicestate: Don't create topic when change isn't cached. ... When publishing a device state the change can be marked as being cachable or not. If it is not cached the change is just published to all interested and not stored away for later query. This was not fully taken into account when publishing in stasis. The act of publishing would create a topic for the device even if it may be ephemeral. This change makes it so messages which are not cached won't create a topic for the device. If a topic does already exist it will be published to but otherwise the change will only be published to the device state all topic. ASTERISK-27591 Change-Id: I18da0e8cbb18e79602e731020c46ba4101e59f0a	7 years ago
Chris-Savinovich	ae5ba776ae	stasis: Improve message type "Use of before/init after destruction" ... Fixes issue where error msg "Use of before/init after destruction" was being printed on disabled messages in dev mode. With this fix if message is disabled a warning will print. ASTERISK-25548 Change-Id: Ie0d866d1cbc60c16dbef08bc65e99505c3c1adfa	7 years ago
Corey Farrell	608112ced3	stasis: Remove silly usage of RAII_VAR. ... Change-Id: Ib11193531e797bcb16bba560a408eab155f706d1	8 years ago
Sean Bright	ce3d56920b	Remove as much trailing whitespace as possible. ... Change-Id: I873c1c6d00f447269bd841494459efccdd2c19c0	8 years ago
Corey Farrell	501f4dcdd8	aco: Minimize use of regex. ... Remove nearly all use of regex from ACO users. Still remaining: * app_confbridge has a legitamate use of option name regex. * ast_sorcery_object_fields_register is implemented with regex, all callers use simple prefix based regex. I haven't decided the best way to fix this in both 13/15 and master. Change-Id: Ib5ed478218d8a661ace4d2eaaea98b59a897974b	8 years ago
Jenkins2	841fc3e5c2	Merge "stasis: Release object if vector append fails." into 13	8 years ago
Joshua Colp	315ccf40f4	Merge "stasis: Remove silly use of RAII_VAR in stasis_forward_all." into 13	8 years ago
Corey Farrell	3dc600ef84	stasis: Release object if vector append fails. ... Change-Id: I3e5cc669169aab6175ddfaf7486edeaeb4fdcfb1	8 years ago
Corey Farrell	803dc3dee5	stasis: Remove silly use of RAII_VAR in stasis_forward_all. ... Change-Id: I46de4c968d40144d5b049966304ff66c1469fb65	8 years ago
Richard Mudgett	9013415593	Fix ast_(v)asprintf() malloc failure usage conditions. ... When (v)asprintf() fails, the state of the allocated buffer is undefined. The library had better not leave an allocated buffer as a result or no one will know to free it. The most likely way it can return failure is for an allocation failure. If the printf conversion fails then you actually have a threading problem which is much worse because another thread modified the parameter values. * Made __ast_asprintf()/__ast_vasprintf() set the returned buffer to NULL on failure. That is much more useful than either an uninitialized pointer or a pointer that has already been freed. Many uses won't have to check for failure to ensure that the buffer won't be double freed or prevent an attempt to free an uninitialized pointer. * stasis.c: Fixed memory leak in multi_object_blob_to_ami() allocated by ast_asprintf(). * ari/resource_bridges.c:ari_bridges_play_helper(): Remove assignment to the wrong thing which is now not needed even if assigning to the right thing. Change-Id: Ib5252fb8850ecf0f78ed0ee2ca0796bda7e91c23	8 years ago
Matt Jordan	0422667d6c	manager: Add <see-also> tags to relate UserEvent actions/apps/events ... Change-Id: I80f8a981f62f50e74609c69c49edcaca6c95efa4	9 years ago
Richard Mudgett	557333ea4c	stasis: Add setting subscription congestion levels. ... Stasis subscriptions and message routers create taskprocessors to process the event messages. API calls are needed to be able to set the congestion levels of these taskprocessors for selected subscriptions and message routers. * Updated CDR, CEL, and manager's stasis subscription congestion levels based upon stress testing. Increased the congestion levels to reduce the potential for bursty call setup/teardown activity from triggering the taskprocessor overload alert. CDRs in particular need an extra high congestion level because they can take awhile to process the stasis messages. ASTERISK-26088 Reported by: Richard Mudgett Change-Id: Id0a716394b4eee746dd158acc63d703902450244	9 years ago
Richard Mudgett	f02ac1b7f9	Stasis: Create human friendly taskprocessor/serializer names. ... Stasis name formats: subm:<topic>-<seq> -- Stasis subscription mailbox task processor subp:<topic>-<seq> -- Stasis subscription thread pool serializer Change-Id: Id19234b306e3594530bb040bc95d977f18ac7bfd	10 years ago
Joshua Colp	a9d6fc571d	json: Audit ast_json_* usage for thread safety. ... The JSON library Asterisk uses, jansson, is not thread safe for us in a few ways. To help with this wrappers for JSON object reference count increasing and decreasing were added which use a global lock to ensure they don't clobber over each other. This does not extend to reference count manipulation within the jansson library itself. This means you can't safely use the object borrowing specifier (O) in ast_json_pack and you can't share JSON instances between objects. This change removes uses of the O specifier and replaces them with the o specifier and an explicit ast_json_ref. Some cases of instance sharing have also been removed. ASTERISK-25601 #close Change-Id: I06550d8b0cc1bfeb56cab580a4e608ae4f1ec7d1	10 years ago
Corey Farrell	0d266cbe02	Stasis: Fix unsafe use of stasis_unsubscribe in modules. ... Many uses of stasis_unsubscribe in modules can be reached through unload. These have been switched to stasis_unsubscribe_and_join. Some subscription callbacks do nothing, for these I've created a noop callback function in stasis.c. This is used by some modules that monitor MWI topics in order to enable cache, since the callback does not become invalid after dlclose it is safe to use stasis_unsubscribe on these, even during module unload. ASTERISK-25121 #close Change-Id: Ifc2549fbd8eef7d703c222978e8f452e2972189c	10 years ago
Corey Farrell	6adf26f14d	Replace most uses of ast_register_atexit with ast_register_cleanup. ... Since 'core stop now' and 'core restart now' do not stop modules, it is unsafe for most of the core to run cleanups. Originally all cleanups used ast_register_atexit, and were only changed when it was shown to be unsafe. ast_register_atexit is now used only when absolutely required to prevent corruption and close child processes. Exceptions that need to use ast_register_atexit: * CDR: Flush records. * res_musiconhold: Kill external applications. * AstDB: Close the DB. * canary_exit: Kill canary process. ASTERISK-24142 #close Reported by: David Brillert ASTERISK-24683 #close Reported by: Peter Katzmann ASTERISK-24805 #close Reported by: Badalian Vyacheslav ASTERISK-24881 #close Reported by: Corey Farrell Review: https://reviewboard.asterisk.org/r/4500/ Review: https://reviewboard.asterisk.org/r/4501/ ........ Merged revisions 433495 from http://svn.asterisk.org/svn/asterisk/branches/11 git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@433497 65c4cc65-6c06-0410-ace0-fbb531ad65f3	10 years ago
Matthew Jordan	d79c68d3fb	main/stasis: Allow subscriptions to use a threadpool for message delivery ... Prior to this patch, all Stasis subscriptions would receive a dedicated thread for servicing published messages. In contrast, prior to r400178 (see review https://reviewboard.asterisk.org/r/2881/), the subscriptions shared a thread pool. It was discovered during some initial work on Stasis that, for a low subscription count with high message throughput, the threadpool was not as performant as simply having a dedicated thread per subscriber. For situations where a subscriber receives a substantial number of messages and is always present, the model of having a dedicated thread per subscriber makes sense. While we still have plenty of subscriptions that would follow this model, e.g., AMI, CDRs, CEL, etc., there are plenty that also fall into the following two categories: * Large number of subscriptions, specifically those tied to endpoints/peers. * Low number of messages. Some subscriptions exist specifically to coordinate a single message - the subscription is created, a message is published, the delivery is synchronized, and the subscription is destroyed. In both of the latter two cases, creating a dedicated thread is wasteful (and in the case of a large number of peers/endpoints, harmful). In those cases, having shared delivery threads is far more performant. This patch adds the ability of a subscriber to Stasis to choose whether or not their messages are dispatched on a dedicated thread or on a threadpool. The threadpool is configurable through stasis.conf. Review: https://reviewboard.asterisk.org/r/4193 ASTERISK-24533 #close Reported by: xrobau Tested by: xrobau ........ Merged revisions 428681 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@428687 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Kinsey Moore	b32d8b5317	Stasis: Relegate log message to dev-mode ... This error message primarily applies to development tasks and will now only show up when dev-mode is enabled via configure. git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@424850 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Kinsey Moore	57a5e2ebee	Stasis: Only log errors for non-declined types ... When message type creation is declined via stasis.conf, certain operations log errors assuming that the declined type is being used before initialization or after destruction. These error messages get quite spammy for oft used message types and should not be logged in the first place since the message type is validly NULL. Reported by: Matt DiMeo git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@424769 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Kinsey Moore	f1036f40dc	Stasis: Allow message types to be blocked ... This introduces stasis.conf and a mechanism to prevent certain message types from being published. Internally, this works by preventing the chosen message types from being created which ensures that those message types can never be published. This patch also adjusts message publishers such that message payloads are not created if the related message type is not available. ASTERISK-23943 #close Review: https://reviewboard.asterisk.org/r/3823/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@420124 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Corey Farrell	fd7814ddb5	stasis: use ao2_t_alloc for certain object allocators ... Add tags to stasis objects using the name. This makes it easier to track the source of certain stasis ref leaks. Review: https://reviewboard.asterisk.org/r/3821/ ........ Merged revisions 418996 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@418997 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Scott Griepentrog	cf21644d6a	ARI: Add ability to raise arbitrary User Events ... User events can now be generated from ARI. Events can be signalled with arbitrary json variables, and include one or more of channel, bridge, or endpoint snapshots. An application must be specified which will receive the event message (other applications can subscribe to it). The message will also be delivered via AMI provided a channel is attached. Dialplan generated user event messages are still transmitted via the channel, and will only be received by a stasis application they are attached to or if the channel is subscribed to. This change also introduces the multi object blob mechanism used to send multiple snapshot types in a single message. The dialplan app UserEvent was also changed to use multi object blob, and a new stasis message type created to handle them. ASTERISK-22697 #close Review: https://reviewboard.asterisk.org/r/3494/ ........ Merged revisions 414405 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@414406 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Matthew Jordan	9cee08f502	res_corosync: Update module to work with Stasis (and compile) ... This patch fixes res_corosync such that it works with Asterisk 12. This restores the functionality that was present in previous versions of Asterisk, and ensures compatibility with those versions by restoring the binary message format needed to pass information from/to them. The following changes were made in the core to support this: * The event system has been partially restored. All event definition and event types in this patch were pulled from Asterisk 11. Previously, we had hoped that this information would live in res_corosync; however, the approach in this patch seems to be better for a few reasons: (1) Theoretically, ast_events can be used by any module as a binary representation of a Stasis message. Given the structure of an ast_event object, that information has to live in the core to be used universally. For example, defining the payload of a device state ast_event in res_corosync could result in an incompatible device state representation in another module. (2) Much of this representation already lived in the core, and was not easily extensible. (3) The code already existed. :-) * Stasis message types now have a message formatter that converts their payload to an ast_event object. * Stasis message forwarders now handle forwarding to themselves. Previously this would result in an infinite recursive call. Now, this simply creates a new forwarding object with no forwards set up (as it is the thing it is forwarding to). This is advantageous for res_corosync, as returning NULL would also imply an unrecoverable error. Returning a subscription in this case allows for easier handling of message types that are published directly to an aggregate topic that has forwarders. Review: https://reviewboard.asterisk.org/r/3486/ ASTERISK-22912 #close ASTERISK-22372 #close ........ Merged revisions 414330 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@414331 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Richard Mudgett	f9c031ec39	stasis.c: Misc code cleanups. ... * Remove some unnecessary RAII_VAR() usage. * Made the struct stasis_subscription ao2 object use the ao2 lock instead of a redundant join_lock in the struct for ast_cond_wait(). * Removed locks on some ao2 objects that don't need the lock. * Made the topic pool entries container use the ao2 template functions. * Add some missing allocation failure checks. * Add missing cleanup in off nominal path of dispatch_message(). ........ Merged revisions 409270 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@409271 65c4cc65-6c06-0410-ace0-fbb531ad65f3	11 years ago
Matthew Jordan	f8aaf585a3	stasis: Add methods to allow for synchronous publishing to subscriber ... This patch adds an API call to Stasis that allows a publisher to publish a stasis message that will not return until a specific subscriber handles the message. Since a subscriber can have their own forwarding topic which orders messages from many topics, this allows a publisher who knows of that subscriber to synchronize to that subscriber regardless of the forwarding relationships between topics. This is of particular use for dialplan applications that need to synchronize on a particular subscriber's handling of a message. (issue ASTERISK-22884) Reported by: Matt Jordan Review: https://reviewboard.asterisk.org/r/3099/ ........ Merged revisions 405311 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@405313 65c4cc65-6c06-0410-ace0-fbb531ad65f3	12 years ago
Richard Mudgett	7d2f2d6ef8	vector: Uppercase API to follow C convention. ... C does not support templates like C++. ........ Merged revisions 402438 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@402439 65c4cc65-6c06-0410-ace0-fbb531ad65f3	12 years ago
Richard Mudgett	629a5fc39b	vector: Update API to be more flexible. ... Made the vector macro API be more like linked lists. 1) Added a name parameter to ast_vector() to name the vector struct. 2) Made the API take a pointer to the vector struct instead of the struct itself. 3) Added an element cleanup macro/function parameter when removing an element from the vector for ast_vector_remove_cmp_unordered() and ast_vector_remove_elem_unordered(). 4) Added ast_vector_get_addr() in case the vector element is not a simple pointer. * Converted an inline vector usage in stasis_message_router to use the vector API. It needed the API improvements so it could be converted. * Fixed topic reference leak in router_dtor() when the stasis_message_router is destroyed. * Fixed deadlock potential in stasis_forward_all() and stasis_forward_cancel(). Locking two topics at the same time requires deadlock avoidance. * Made internal_stasis_subscribe() tolerant of a NULL topic. * Made stasis_message_router_add(), stasis_message_router_add_cache_update(), stasis_message_router_remove(), and stasis_message_router_remove_cache_update() tolerant of a NULL message_type. * Promoted a LOG_DEBUG message to LOG_ERROR as intended in dispatch_message(). Review: https://reviewboard.asterisk.org/r/2903/ ........ Merged revisions 402429 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@402430 65c4cc65-6c06-0410-ace0-fbb531ad65f3	12 years ago
Mark Michelson	addbf276f5	Multiple revisions 400318-400319 ... ........ r400318 | mmichelson | 2013-10-02 17:08:49 -0500 (Wed, 02 Oct 2013) | 12 lines Remove unnecessary waits from stasis. Since caches are updated on publisher threads, there is no need to wait for the cache updates to occur after a stasis message is published. In the case of chan_pjsip device state changes, this set of changes caused an improvement to performance. Review: https://reviewboard.asterisk.org/r/2890 ........ r400319 | mmichelson | 2013-10-02 17:10:54 -0500 (Wed, 02 Oct 2013) | 3 lines Remove svn:mergeinfo property. ........ Merged revisions 400318-400319 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@400335 65c4cc65-6c06-0410-ace0-fbb531ad65f3	12 years ago
David M. Lee	2de42c2a25	Multiple revisions 399887,400138,400178,400180-400181 ... ........ r399887 | dlee | 2013-09-26 10:41:47 -0500 (Thu, 26 Sep 2013) | 1 line Minor performance bump by not allocate manager variable struct if we don't need it ........ r400138 | dlee | 2013-09-30 10:24:00 -0500 (Mon, 30 Sep 2013) | 23 lines Stasis performance improvements This patch addresses several performance problems that were found in the initial performance testing of Asterisk 12. The Stasis dispatch object was allocated as an AO2 object, even though it has a very confined lifecycle. This was replaced with a straight ast_malloc(). The Stasis message router was spending an inordinate amount of time searching hash tables. In this case, most of our routers had 6 or fewer routes in them to begin with. This was replaced with an array that's searched linearly for the route. We more heavily rely on AO2 objects in Asterisk 12, and the memset() in ao2_ref() actually became noticeable on the profile. This was #ifdef'ed to only run when AO2_DEBUG was enabled. After being misled by an erroneous comment in taskprocessor.c during profiling, the wrong comment was removed. Review: https://reviewboard.asterisk.org/r/2873/ ........ r400178 | dlee | 2013-09-30 13:26:27 -0500 (Mon, 30 Sep 2013) | 24 lines Taskprocessor optimization; switch Stasis to use taskprocessors This patch optimizes taskprocessor to use a semaphore for signaling, which the OS can do a better job at managing contention and waiting that we can with a mutex and condition. The taskprocessor execution was also slightly optimized to reduce the number of locks taken. The only observable difference in the taskprocessor implementation is that when the final reference to the taskprocessor goes away, it will execute all tasks to completion instead of discarding the unexecuted tasks. For systems where unnamed semaphores are not supported, a really simple semaphore implementation is provided. (Which gives identical performance as the original taskprocessor implementation). The way we ended up implementing Stasis caused the threadpool to be a burden instead of a boost to performance. This was switched to just use taskprocessors directly for subscriptions. Review: https://reviewboard.asterisk.org/r/2881/ ........ r400180 | dlee | 2013-09-30 13:39:34 -0500 (Mon, 30 Sep 2013) | 28 lines Optimize how Stasis forwards are dispatched This patch optimizes how forwards are dispatched in Stasis. Originally, forwards were dispatched as subscriptions that are invoked on the publishing thread. This did not account for the vast number of forwards we would end up having in the system, and the amount of work it would take to walk though the forward subscriptions. This patch modifies Stasis so that rather than walking the tree of forwards on every dispatch, when forwards and subscriptions are changed, the subscriber list for every topic in the tree is changed. This has a couple of benefits. First, this reduces the workload of dispatching messages. It also reduces contention when dispatching to different topics that happen to forward to the same aggregation topic (as happens with all of the channel, bridge and endpoint topics). Since forwards are no longer subscriptions, the bulk of this patch is simply changing stasis_subscription objects to stasis_forward objects (which, admittedly, I should have done in the first place.) Since this required me to yet again put in a growing array, I finally abstracted that out into a set of ast_vector macros in asterisk/vector.h. Review: https://reviewboard.asterisk.org/r/2883/ ........ r400181 | dlee | 2013-09-30 13:48:57 -0500 (Mon, 30 Sep 2013) | 28 lines Remove dispatch object allocation from Stasis publishing While looking for areas for performance improvement, I realized that an unused feature in Stasis was negatively impacting performance. When a message is sent to a subscriber, a dispatch object is allocated for the dispatch, containing the topic the message was published to, the subscriber the message is being sent to, and the message itself. The topic is actually unused by any subscriber in Asterisk today. And the subscriber is associated with the taskprocessor the message is being dispatched to. First, this patch removes the unused topic parameter from Stasis subscription callbacks. Second, this patch introduces the concept of taskprocessor local data, data that may be set on a taskprocessor and provided along with the data pointer when a task is pushed using the ast_taskprocessor_push_local() call. This allows the task to have both data specific to that taskprocessor, in addition to data specific to that invocation. With those two changes, the dispatch object can be removed completely, and the message is simply refcounted and sent directly to the taskprocessor. Review: https://reviewboard.asterisk.org/r/2884/ ........ Merged revisions 399887,400138,400178,400180-400181 from http://svn.asterisk.org/svn/asterisk/branches/12 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@400186 65c4cc65-6c06-0410-ace0-fbb531ad65f3	12 years ago
David M. Lee	f29d969a79	Stasis: address refcount races; implementation comments ... Change r395954 reordered some stasis object destruction, which should have been fine. Unfortunately, it caused some hard to reproduce issues related to objects being accessed after they had been destroyed. The patch in r396329 fixed the destruction order problem; this patch addresses the underlying issue. A few other stasis-related fixes were also added. * Add ref-bumps around areas where objects may get transitively destroyed. (For example, where we lock a topic, unref a subscription, which unrefs the topic, which explodes the topic when we try to unlock it.) * Wrote an extensive doxygen page about Stasis implementation, relationships between objects, lifecycles of objects, how the refcounting works, etc. Many other comments were added, corrected, or cleaned up. * Added an assert to the topic dtor to catch extra ref decrements. * Fixed type used after destruction errors for graceful shutdown in stasis_channels.c. * I added two unit tests in an attempt to catch destruction order issues. Since the underlying cause is a race condition, though, the tests rarely failed even when the code was wrong. * Fixed a leak in stasis_cache_pattern.c. (closes issue ASTERISK-22243) Review: https://reviewboard.asterisk.org/r/2746/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396842 65c4cc65-6c06-0410-ace0-fbb531ad65f3	12 years ago
David M. Lee	e1b959ccbb	Split caching out from the stasis_caching_topic. ... In working with res_stasis, I discovered a significant limitation to the current structure of stasis_caching_topics: you cannot subscribe to cache updates for a single channel/bridge/endpoint/etc. To address this, this patch splits the cache away from the stasis_caching_topic, making it a first class object. The stasis_cache object is shared amongst individual stasis_caching_topics that are created per channel/endpoint/etc. These are still forwarded to global whatever_all_cached topics, so their use from most of the code does not change. In making these changes, I noticed that we frequently used a similar pattern for bridges, endpoints and channels: single_topic ----------------> all_topic ^ | single_topic_cached ----+----> all_topic_cached | +----> cache This pattern was extracted as the 'Stasis Caching Pattern', defined in stasis_caching_pattern.h. This avoids a lot of duplicate code between the different domain objects. Since the cache is now disassociated from its upstream caching topics, this also necessitated a change to how the 'guaranteed' flag worked for retrieving from a cache. The code for handling the caching guarantee was extracted into a 'stasis_topic_wait' function, which works for any stasis_topic. (closes issue ASTERISK-22002) Review: https://reviewboard.asterisk.org/r/2672/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395954 65c4cc65-6c06-0410-ace0-fbb531ad65f3	12 years ago
David M. Lee	c4adaf9106	Configuration for Stasis threadpool ... The appropriate settings for the Stasis threadpool is very system specific, depending upon both workload and system configuration. This patch adds a stasis.conf file which can be used to configure the key attributes of the threadpool for the Stasis message bus. (closes issue ASTERISK-21280) Review: https://reviewboard.asterisk.org/r/2651/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@393542 65c4cc65-6c06-0410-ace0-fbb531ad65f3	12 years ago
Matthew Jordan	6258bbe7bd	Update Asterisk's CDRs for the new bridging framework ... This patch is the initial push to update Asterisk's CDR engine for the new bridging framework. This patch guts the existing CDR engine and builds the new on top of messages coming across Stasis. As changes in channel state and bridge state are detected, CDRs are built and dispatched accordingly. This fundamentally changes CDRs in a few ways. (1) CDRs are now *very* reflective of the actual state of channels and bridges. This means CDRs track well with what an actual channel is doing - which is useful in transfer scenarios (which were previously difficult to pin down). It does, however, mean that CDRs cannot be 'fooled'. Previous behavior in Asterisk allowed for CDR applications, channels, and other properties to be spoofed in parts of the code - this no longer works. (2) CDRs have defined behavior in multi-party scenarios. This behavior will not be what everyone wants, but it is a defined behavior and as such, it is predictable. (3) The CDR manipulation functions and applications have been overhauled. Major changes have been made to ResetCDR and ForkCDR in particular. Many of the options for these two applications no longer made any sense with the new framework and the (slightly) more immutable nature of CDRs. There are a plethora of other changes. For a full description of CDR behavior, see the CDR specification on the Asterisk wiki. (closes issue ASTERISK-21196) Review: https://reviewboard.asterisk.org/r/2486/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@391947 65c4cc65-6c06-0410-ace0-fbb531ad65f3	12 years ago
David M. Lee	d81c846724	Avoid unnecessary cleanups during immediate shutdown ... This patch addresses issues during immediate shutdowns, where modules are not unloaded, but Asterisk atexit handlers are run. In the typical case, this usually isn't a big deal. But the introduction of the Stasis message bus makes it much more likely for asynchronous activity to be happening off in some thread during shutdown. During an immediate shutdown, Asterisk skips unloading modules. But while it is processing the atexit handlers, there is a window of time where some of the core message types have been cleaned up, but the message bus is still running. Specifically, it's still running module subscriptions that might be using the core message types. If a message is received by that subscription in that window, it will attempt to use a message type that has been cleaned up. To solve this problem, this patch introduces ast_register_cleanup(). This function operates identically to ast_register_atexit(), except that cleanup calls are not invoked on an immediate shutdown. All of the core message type and topic cleanup was moved from atexit handlers to cleanup handlers. This ensures that core type and topic cleanup only happens if the modules that used them are first unloaded. This patch also changes the ast_assert() when accessing a cleaned up or uninitialized message type to an error log message. Message type functions are actually NULL safe across the board, so the assert was a bit heavy handed. Especially for anyone with DO_CRASH enabled. Review: https://reviewboard.asterisk.org/r/2562/ git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@390122 65c4cc65-6c06-0410-ace0-fbb531ad65f3	12 years ago
David M. Lee	b97c71bb11	Fix shutdown assertions in stasis-core ... In r388005, macros were introduced to consistently define message types. This added an assert if a message type was used either before it was initialized or after it had been cleaned up. It turns out that this assertion fires during shutdown. This actually exposed a hidden shutdown ordering problem. Since unsubscribing is asynchronous, it's possible that the message types used by the subscription could be freed before the final message of the subscription was processed. This patch adds stasis_subscription_join(), which blocks until the last message has been processed by the subscription. Since joining was most commonly done right after an unsubscribe, a stasis_unsubscribe_and_join() convenience function was also added. Similar functions were also added to the stasis_caching_topic and stasis_message_router, since they wrap subscriptions and have similar problems. Other code in trunk was refactored to join() where appropriate, or at least verify that the subscription was complete before being destroyed. Review: https://reviewboard.asterisk.org/r/2540 git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@389011 65c4cc65-6c06-0410-ace0-fbb531ad65f3	12 years ago