19 Commits (77a5383957504d58889d4347ce72792aac6e51ea)

Author	SHA1	Message	Date
Joshua C. Colp	1a40a6d55a	res_sorcery_memory_cache: Disallow per-object expire with full backend. ... The AMI action and CLI command did not take into account the properties of full backend caching. This resulted in an expired object remaining removed until a full backend update occurred, instead of having the object updated when needed. This change makes it so that the AMI action and CLI command for object expire will now fail instead of putting the cache into an undesired state. If full backend caching is enabled then only operations which act on the entire cache are available. ASTERISK-28942 Change-Id: Id662d888f177ab566c8e802ad583083b742d21f4	6 years ago
Kevin Harwell	6517690168	res_sorcery_memory_cache: stale item update leak ... When a stale item was being updated the object was being retrieved, but its reference was not being decremented after the update. This patch makes it so the object is now appropriately de-referenced. ASTERISK-28523 Change-Id: I9d8173d3a0416a242f4eba92fa0853279c500ec7	7 years ago
Corey Farrell	0a9904e1c6	astobj2: Eliminate usage of legacy ao2_container_alloc routine. ... Replace usage of ao2_container_alloc with ao2_container_alloc_hash or ao2_container_alloc_list. ao2_container_alloc is now restricted to modules only and is being removed from Asterisk 17. Change-Id: I0907d78bc66efc775672df37c8faad00f2f6c088	8 years ago
Corey Farrell	54a1fbe428	astobj2: Eliminate usage of legacy container allocation macros. ... These macros have been documented as legacy for a long time but are still used in new code because they exist. Remove all references to: * ao2_container_alloc_options * ao2_t_container_alloc_options * ao2_t_container_alloc These macro's are still available for use but only in modules. Only ao2_container_alloc remains due to it's use in over 100 places. Change-Id: I1a26258b5bf3deb081aaeed11a0baa175c933c7a	8 years ago
Sean Bright	cf062303e3	sorcery: Add ast_sorcery_retrieve_by_prefix() ... Some consumers of the sorcery API use ast_sorcery_retrieve_by_regex only so that they can anchor the potential match as a prefix and not because they truly need regular expressions. Rather than using regular expressions for simple prefix lookups, add a new operation - ast_sorcery_retrieve_by_prefix - that does them. Patches against 13 and 15 have a compatibility layer needed to maintain ABI that is not needed in master. Change-Id: I56f4e20ba1154bd52281f995c27a429a854f6a79	9 years ago
Sean Bright	ea8a610776	cli: Fix various CLI documentation and completion issues ... * app_minivm: Use built-in completion facilities to complete optional arguments. * app_voicemail: Use built-in completion facilities to complete optional arguments. * app_confbridge: Add missing colons after 'Usage' text. * chan_alsa: Use built-in completion facilities to complete optional arguments. * chan_sip: Use built-in completion facilities to complete optional arguments. Add completions for 'load' for 'sip show user', 'sip show peer', and 'sip qualify peer.' * chan_skinny: Correct and extend completions for 'skinny reset' and 'skinny show line.' * func_odbc: Correct completions for 'odbc read' and 'odbc write' * main/asterisk: Correct and extend completions for 'core show file version.' * main/astmm: Use built-in completion facilities to complete arguments for 'memory' commands. * main/bridge: Correct completions for 'bridge kick.' * main/ccss: Use built-in completion facilities to complete arguments for 'cc cancel' command. * main/cli: Add 'all' completion for 'channel request hangup.' Correct completions for 'core set debug channel.' Correct completions for 'core show calls.' * main/pbx_app: Remove redundant completions for 'core show applications.' * main/pbx_hangup_handler: Remove unused completions for 'core show hanguphandlers all.' * res_sorcery_memory_cache: Add completion for 'reload' argument of 'sorcery memory cache stale' and properly implement. Change-Id: Iee58c7392f6fec34ad9d596109117af87697bbca	9 years ago
Mark Michelson	9e3150b98d	Add reload options to CLI/AMI stale object commands. ... Marking an object as stale in a memory cache is supposed to prime the cache so that the next time the item is retrieved, the stale item is deleted from the cache and a background task is run to re-populate the cache with a fresh version of the object. The problem is, there are some object types out there for which there is no natural reason that they would be retrieved from the backend with any regularity. Outbound PJSIP registrations are a good example of this. At startup, they are read, and an object-specific state is created that refers to the initially-retrieved object for all time. Adding the "reload" option to the CLI/AMI commands gives the cache the opportunity to manually re-retrieve the object from the backend, both storing the new object in the cache and applying the new object's configuration to the module that uses that object. Change-Id: Ieb1fe7270ceed491f057ec5cbf0e097bde96c5c8	10 years ago
George Joseph	91485734a4	res_sorcery_memory_cache: Change an error to a debug message ... When a sorcery user calls ast_sorcery_delete on an object that may have already expired from the cache, res_sorcery_memory_cache spits out an ERROR. Since this can happen frequently and validly when an inbound registration expires after the cache entry expired, the errors are unnecessary and misleading. Changed to a debug/1. Change-Id: Idf3a67038c16e3da814cf612ff4d6d18ad29ecd7	10 years ago
George Joseph	5aa5c49413	sorcery/res_pjsip: Refactor for realtime performance ... There were a number of places in the res_pjsip stack that were getting all endpoints or all aors, and then filtering them locally. A good example is pjsip_options which, on startup, retrieves all endpoints, then the aors for those endpoints, then tests the aors to see if the qualify_frequency is > 0. One issue was that it never did anything with the endpoints other than retrieve the aors so we probably could have skipped a step and just retrieved all aors. But nevermind. This worked reasonably well with local config files but with a realtime backend and thousands of objects, this was a nightmare. The issue really boiled down to the fact that while realtime supports predicates that are passed to the database engine, the non-realtime sorcery backends didn't. They do now. The realtime engines have a scheme for doing simple comparisons. They take in an ast_variable (or list) for matching, and the name of each variable can contain an operator. For instance, a name of "qualify_frequency >" and a value of "0" would create a SQL predicate that looks like "where qualify_frequency > '0'". If there's no operator after the name, the engines add an '=' so a simple name of "qualify_frequency" and a value of "10" would return exact matches. The non-realtime backends decide whether to include an object in a result set by calling ast_sorcery_changeset_create on every object in the internal container. However, ast_sorcery_changeset_create only does exact string matches though so a name of "qualify_frequency >" and a value of "0" returns nothing because the literal "qualify_frequency >" doesn't match any name in the objset set. So, the real task was to create a generic string matcher that can take a left value, operator and a right value and perform the match. To that end, strings.c has a new ast_strings_match(left, operator, right) function. Left and right are the strings to operate on and the operator can be a string containing any of the following: = (or NULL or ""), !=, >, >=, <, <=, like or regex. If the operator is like or regex, the right string should be a %-pattern or a regex expression. If both left and right can be converted to float, then a numeric comparison is performed, otherwise a string comparison is performed. To use this new function on ast_variables, 2 new functions were added to config.c. One that compares 2 ast_variables, and one that compares 2 ast_variable lists. The former is useful when you want to compare 2 ast_variables that happen to be in a list but don't want to traverse the list. The latter will traverse the right list and return true if all the variables in it match the left list. Now, the backends' fields_cmp functions call ast_variable_lists_match instead of ast_sorcery_changeset_create and they can now process the same syntax as the realtime engines. The realtime backend just passes the variable list unaltered to the engine. The only gotcha is that there's no common realtime engine support for regex so that's been noted in the api docs for ast_sorcery_retrieve_by_fields. Only one more change to sorcery was done... A new config flag "allow_unqualified_fetch" was added to reg_sorcery_realtime. "no": ignore fetches if no predicate fields were supplied. "error": same as no but emit an error. (good for testing) "yes": allow (the default); "warn": allow but emit a warning. (good for testing) Now on to res_pjsip... pjsip_options was modified to retrieve aors with qualify_frequency > 0 rather than all endpoints then all aors. Not only was this a big improvement in realtime retrieval but even for config files there's an improvement because we're not going through endpoints anymore. res_pjsip_mwi was modified to retieve only endpoints with something in the mailboxes field instead of all endpoints then testing mailboxes. res_pjsip_registrar_expire was completely refactored. It was retrieving all contacts then setting up scheduler entries to check for expiration. Now, it's a single thread (like keepalive) that periodically retrieves only contacts whose expiration time is < now and deletes them. A new contact_expiration_check_interval was added to global with a default of 30 seconds. Ross Beer reports that with this patch, his Asterisk startup time dropped from around an hour to under 30 seconds. There are still objects that can't be filtered at the database like identifies, transports, and registrations. These are not going to be anywhere near as numerous as endpoints, aors, auths, contacts however. Back to allow_unqualified_fetch. If this is set to yes and you have a very large number of objects in the database, the pjsip CLI commands will attempt to retrive ALL of them if not qualified with a LIKE. Worse, if you type "pjsip show endpoint <tab>" guess what's going to happen? :) Having a cache helps but all the objects will have to be retrieved at least once to fill the cache. Setting allow_unqualified_fetch=no prevents the mass retrieve and should be used on endpoints, auths, aors, and contacts. It should NOT be used for identifies, registrations and transports since these MUST be retrieved in bulk. Example sorcery.conf: [res_pjsip] endpoint=config,pjsip.conf,criteria=type=endpoint endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error ASTERISK-25826 #close Reported-by: Ross Beer Tested-by: Ross Beer Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67	10 years ago
George Joseph	6e70e8ccdb	res_sorcery_memory_cache: Fix SEGV in some CLI commands ... A few of the CLI commands weren't checking for enough arguments and were SEGVing. Change-Id: Ie6494132ad2fe54b4f014bcdc112a37c36a9b413	10 years ago
Joshua Colp	59d5bb0613	res_sorcery_memory_cache: Add support for a full backend cache. ... This change introduces the configuration option 'full_backend_cache' which changes the cache to be a full mirror of the backend instead of a per-object cache. This allows all sorcery retrieval operations to be carried out against it and is useful for object types which are used in a "retrieve all" or "retrieve some" pattern. ASTERISK-25625 #close Change-Id: Ie2993487e9c19de563413ad5561c7403b48caab5	11 years ago
Richard Mudgett	bb0b60619d	res_sorcery_memory_cache.c: Fix off nominal ref leak. ... Change-Id: If83d63cf11cbc6df9b15251848b01feb570ade49	11 years ago
Richard Mudgett	40c69e78f5	res_sorcery_memory_cache.c: Fix deadlock with scheduler. ... A deadlock can happen when a sorcery object is being expired from the memory cache when at the same time another object is being placed into the memory cache. There are a couple other variations on this theme that could cause the deadlock. Basically if an object is being expired from the sorcery memory cache at the same time as another thread tries to update the next object expiration timer the deadlock can happen. * Add a deadlock avoidance loop in expire_objects_from_cache() to check if someone is trying to remove the scheduler callback from the scheduler. ASTERISK-25441 #close Change-Id: Iec7b0bdb81a72b39477727b1535b2539ad0cf4dc	11 years ago
Richard Mudgett	dfeb513e85	res_sorcery_memory_cache.c: Replace inline code with function. ... Make sorcery_memory_cache_close() call remove_all_from_cache() instead of partially inlining it. ASTERISK-25441 Change-Id: I1aa6cb425b1a4307096f3f914d17af8ec179a74c	11 years ago
Richard Mudgett	ced0a2d71b	res_sorcery_memory_cache.c: Shutdown in a less crash potential order. ... Basically you should shutdown in the opposite order of how you setup since later setup pieces likely depend on earlier setup pieces. e.g., Registering your external API with the rest of the system should be the last thing setup and the first thing unregistered during shutdown. Change-Id: I5715765b723100c8d3c2642e9e72cc7ad5ad115e	11 years ago
Richard Mudgett	cc279eea11	res_sorcery_memory_cache.c: Misc tweaks. ... Change-Id: I8cd32dffbb4f33bb0c39518d6e4c991e73573160	11 years ago
Richard Mudgett	9af3b613f6	res_sorcery_memory_cache.c: Made use OBJ_SEARCH_MASK. ... Change-Id: Ibca6574dc3c213b29cc93486e01ccd51f5caa46c	11 years ago
Matt Jordan	bee41eec62	res/res_sorcery_memory_cache: Fix test registration issues ... Again, tests now need to not end with a newline. This patch makes it so the tests can register again, unit tests will actually pass, and we can stop wasting time trying to figure out why builds are failing when they really aren't failing. Change-Id: Ide519fbeba89f413c733446c5ff7b224fc4ce840	11 years ago
Joshua Colp	b74b071369	res_sorcery_memory_cache: Backport to 13 ... Gerrit is complaining of conflicts when trying to create a patch series of all of the cherry-picked master commits, so I have instead squashed it all into one commit. ASTERISK-25067 #close Reported by: Matt Jordan Change-Id: I6dda90343fae24a75dc5beec84980024e8d61eb9	11 years ago