189 Commits (acd1513111c30aa83aed48b2a72c3e2bc67915d1)

Author	SHA1	Message	Date
Sean Bright	7dc73c8151	loader.c: Minor module key check simplification. ... Change-Id: I65aefd4434a783096165c179b5f94f2e4810dffe	2 years ago
Naveen Albert	c209064d66	loader: Allow declined modules to be unloaded. ... Currently, if a module declines to load, dlopen is called to register the module but dlclose never gets called. Furthermore, loader.c currently doesn't allow dlclose to ever get called on the module, since it declined to load and the unload function bails early in this case. This can be problematic if a module is updated, since the new module cannot be loaded into memory since we haven't closed all references to it. To fix this, we now allow modules to be unloaded, even if they never "loaded" in Asterisk itself, so that dlclose is called and the module can be properly cleaned up, allowing the updated module to be loaded from scratch next time. ASTERISK-30345 #close Change-Id: Ifc743aadfa85ebe3284e02a63e124dafa64988d5	2 years ago
Naveen Albert	3e8629454a	loader: Prevent deadlock using tab completion. ... If tab completion using ast_module_helper is attempted during startup, deadlock will ensue because the CLI will attempt to lock the module list while it is already locked by the loader. This causes deadlock because when the loader tries to acquire the CLI lock, they are blocked on each other. Waiting for startup to complete is not feasible because the CLI lock is acquired while waiting, so deadlock will ensure regardless of whether or not a lock on the module list is attempted. To prevent deadlock, we immediately abort if tab completion is attempted on the module list before Asterisk is fully booted. ASTERISK-30039 #close Change-Id: Idd468906c512bb196631e366a8f597a0e2e9271d	3 years ago
Sean Bright	8d7819482c	loader.c: Use portable printf conversion specifier for int64. ... ASTERISK-30060 #close Change-Id: I88d47a1488be2f39017b8d562f993f081844fcb8	3 years ago
Sean Bright	5a13e95c56	loader.c: Speed up deprecation metadata lookup ... Only use an XPath query once per module, then just navigate the DOM for everything else. Change-Id: Ia0336a7185f9180ccba4b6f631a00f9a22a36e92	4 years ago
Joshua C. Colp	46ed6af9c2	loader: Output warnings for deprecated modules. ... Using the information from the MODULEINFO XML we can now output useful information at the end of module loading for deprecated modules. This includes the version it was deprecated in, the version it will be removed in, and the replacement if available. ASTERISK-29339 Change-Id: I2080dab97d2186be94c421b41dabf6d79a11611a	4 years ago
George Joseph	8e44d823c1	loader.c: Fix possible SEGV when a module fails to register ... When a module fails to register itself (usually a coding error in the module), dlerror() can return NULL. We weren't checking for that in load_dlopen() before trying to strdup the error message so a SEGV was thrown. dlerror() is now surrounded with an S_OR so we don't SEGV. Change-Id: Ie0fb9316f08a321434f3f85aecf3c7d2ede8b956	6 years ago
Sebastian Kemper	ccac55b894	loader: support for permanent dlopen() ... Asterisk assumes that dlopen() will always run the constructor of a shared library and every dlclose() will run its destructor. But dlopen() may be permanent, meaning the constructor will only be run once, as is the case with musl libc. With a permanent dlopen() the Asterisk module loader does not work correctly, because it's expectations regarding when the constructors and destructors are run are not met. In fact a segmentation fault will occur when the first module is "re-opened" that has AST_MODFLAG_GLOBAL_SYMBOLS set (the dlopen() does not call the constructor, resource_being_loaded is not set to NULL, then strlen is called with NULL instead of a string, see issue ASTERISK-28319). This commit adds code to the loader that will manually run the constructors/destructors of the (non-builtin) modules where needed. To achieve this a new ao2 container (linked list) is started and filled with objects that contain the names of the modules and the pointers to their respective info structs. This behavior can be activated when configuring Asterisk (--enable-permanent-dlopen). By default this is disabled, of course. ASTERISK-28319 #close Signed-off-by: Sebastian Kemper <sebastian_ml@gmx.net> Change-Id: I86693a0ecf25d5ba81c73773a03df4abc3426875	6 years ago
Corey Farrell	c8ee1a183f	loader: Flag module as declined in all cases where it fails to load. ... This has no effect on startup since AST_MODULE_LOAD_FAILURE aborts startup, but it's possible for this code to be returned on manual load of a module after startup. It is an error for a module to not have a load callback but this is not a fatal system error. In this case flag the module as declined, return AST_MODULE_LOAD_FAILURE only if a required module is broken. Expand doxygen documentation for AST_MODULE_LOAD_*. Change-Id: I3c030bb917f6e5a0dfd9d91491a4661b348cabf8	7 years ago
George Joseph	24dc6ee8ae	Merge "loader: Fix result of module reload error."	7 years ago
George Joseph	910f5dbd44	Merge "core: Disable astobj2 locking for some common objects."	7 years ago
Corey Farrell	cacbe32534	core: Disable astobj2 locking for some common objects. ... * ACO options * Indications * Module loader ref_debug object * Media index info and variants * xmldoc items These allocation locations were identified using reflocks.py on the master branch. Change-Id: Ie999b9941760be3d1946cdb6e30cb85fd97504d8	7 years ago
Corey Farrell	b25a261aa5	loader: Fix result of module reload error. ... When a module reload fails we never set AST_MODULE_RELOAD_ERROR. This caused reload failures to incorrectly report 'No module found'. Change-Id: I5f3953e0f7d135e53ec797f24c97ee3f73f232e7	7 years ago
Corey Farrell	e4cf513f81	loader: Improve error handling. ... * Display list of unavailable dependencies when they cause another module to fail loading. * When a module declines to load find all modules which depend on it so they can be declined and listed together. * Prevent retry of declined modules during startup. * When a module fails to dlopen try loading it with RTLD_LAZY so we can attempt to display the list of missing dependencies. These changes are meant to reduce logger spam that is caused when a module has many dependencies and declines to load. This also fixes some error paths which failed to recognize required modules. Module load/start errors are delayed until the end of loader startup. Change-Id: I046052c71331c556c09d39f47a3b92975f3e1758	7 years ago
Corey Farrell	709f4b81e7	loader: Process dependencies for built-in modules. ... With the new module loader it was missed that built-in modules never parsed dependencies from mod->info into vectors of mod. This caused manager to be initialized before acl (named_acl). If manager.conf used any named ACL's they would not be found and result in no ACL being applied to the AMI user. In addition to the manager ACL fix this adds "extconfig" to all builtin modules which support realtime configuration. This only matters if one of the builtin modules is configured with 'preload', depending on "extconfig" will cause config.c to automatically be initialize during the preload stage. Change-Id: I482ed6bca6c1064b05bb538d7861cd7a4f02d9fc	7 years ago
Corey Farrell	49f83a7490	loader: Fix startup issues. ... * Merge the preload and load stages, use load ordering to try preload's first. This fixes an issue where `preload=res_config_curl` would fail unless res_curl and func_curl were also preloaded. Now it is only required that those modules be loaded during startup: autoload or regular load is good enough. * The configuration option `require` and `preload-require` were only effective if the modules failed to load. These options will now abort Asterisk startup if required modules fail to reach the 'Running' state. * Missing or invalid 'module.conf' did not prevent startup. Asterisk doesn't do anything without modules so this a fatal error. Change-Id: Ie4176699133f0e3a823b43f90c3348677e43a5f3	7 years ago
Corey Farrell	572a508ef2	loader: Convert reload_classes to built-in modules. ... * acl (named_acl.c) * cdr * cel * ccss * dnsmgr * dsp * enum * extconfig (config.c) * features * http * indications * logger * manager * plc * sounds * udptl These modules are now loaded at appropriate time by the module loader. Unlike loadable modules these use AST_MODULE_LOAD_FAILURE on error so the module loader will abort startup on failure of these modules. Some of these modules are still initialized or shutdown from outside the module loader. logger.c is initialized very early and shutdown very late, manager.c is initialized by the module loader but is shutdown by the Asterisk core (too much uses it without holding references). Change-Id: I371a9a45064f20026c492623ea8062d02a1ab97f	7 years ago
Sungtae Kim	cb4cfb8c43	manager: Add AMI event Load/Unload ... Add an AMI events Load and Unload for notify when the module has been loaded and unloaded. ASTERISK-27661 Change-Id: Ib916c41eddd63651952998f2f49c57c42ef87a64	7 years ago
Jenkins2	093484d137	Merge "loader: Use ast_cli_completion_add for 'module load' completion."	7 years ago
Corey Farrell	84a6365164	loader: Use ast_cli_completion_add for 'module load' completion. ... This addresses all performance issues with 'module load' completion. In addition to using ast_cli_completion_add we stop using libedit's filename_completion_function, instead using ast_file_read_dir. This ensures all results are produced from a single call to opendir. Change-Id: I8bf51ffaa7ef1606f3bd1b5bb13f1905d72c6134	7 years ago
Corey Farrell	a164b7ccfb	loader: Correct overly strict startup checks. ... The code which handled loading modules had too many situations which would result in halting Asterisk startup. Treat most errors as declines instead of failures. The exception is when the module load function returns AST_MODULE_LOAD_FAILURE or an invalid code. Clear the missingdeps vector when appropriate to ensure the next loop starts clean. ASTERISK-27620 Change-Id: I45547d9641fd45bd86d80250224417625631ad84	7 years ago
Jenkins2	d4e9f7c940	Merge "loader: Add support for built-in modules."	7 years ago
Jenkins2	cec16d8e4f	Merge "loader: Rework load_resource_list."	7 years ago
Joshua Colp	908e39f186	Merge "loader: Remove global symbol only startup phase."	7 years ago
Jenkins2	2c27205e4e	Merge "loader: Process module dependencies."	7 years ago
Corey Farrell	25cb1ab05b	loader: Add support for built-in modules. ... * Add SRC_EMBEDDED variable to main/Makefile. Built-in module sources must be listed in this variable to ensure they get the correct CFLAGS. Change-Id: I920852bc17513a9c2627061a4ad40511e3a20499	7 years ago
Corey Farrell	e6142a1282	loader: Rework load_resource_list. ... Use a single loop in a loop to scan the resource list attempting to dlopen each module. The inner loop is repeated until it doesn't do any work, then it is run one more time to allow printing of error messages. Change-Id: I60c15cd57ff9680b62e2a94c7519401fa4a38e45	7 years ago
Corey Farrell	a80cbb046e	loader: Remove global symbol only startup phase. ... Dependency loader is now in place so we no longer need a separate loader phase for global symbols only. This simplifies the loader and allows us to minimize calls to dlopen. Change-Id: I33e3174d67f3b4552d3d536326dcaf0ebabb097d	7 years ago
Corey Farrell	3b73ed28c5	loader: Process module dependencies. ... * Add string vectors for requires, optional_apis and enhances. * Add reffed_deps module vector for holding references to dependencies. * Initialize string vectors after final dlopen of each module. * Free string vectors and clear references from reffed_deps in module_destroy. * Create functions necessary to process module dependencies and enforce load order. Module dependencies result in automatic references being managed by the module loader. This enforces unload order. Change-Id: I9be08d1dd331aceadc1dcba00b804d71360b2fbb	7 years ago
Corey Farrell	4fd303b630	loader: Miscellaneous fixes. ... * Remove comment about lazy load. * Improve message about module already being loaded and running. * Handle allocation error in add_to_load_order. * Dead code elimination from modules_shutdown. Change-Id: I22261599c46d0f416e568910ec9502f45143197f	7 years ago
Corey Farrell	55f1d69c43	loader: Create ast_module_running_ref. ... This function returns NULL if the module in question is not running. I did not change ast_module_ref as most callers do not check the result and they always call ast_module_unref. Make use of this function when running registered items from: * app_stack API's * bridge technologies * CLI commands * File formats * Manager Actions * RTP engines * Sorcery Wizards * Timing Interfaces * Translators * AGI Commands * Fax Technologies ASTERISK-20346 #close Change-Id: Ia16fd28e188b2fc0b9d18b8a5d9cacc31df73fcc	7 years ago
Corey Farrell	23aa20bf20	loader: Add volatile to resource_being_loaded. ... Some compiler optimizers seem to assume that dlopen will not use __attribute__((constructor)) functions to call back to the program. This was causing resource_being_loaded to be optimized away completely. ASTERISK-27531 #close Tested By: abelbeck Change-Id: If17a3b889e06811a0e7119f0539d052494d6ece9	7 years ago
Corey Farrell	9d5797616c	loader: Use vector to build apha sorted module lists. ... Change-Id: I9c519f4dec3cda98b2f34d314255a31d49a6a467	7 years ago
Corey Farrell	7b54903313	loader: Replace priority heap with vector. ... This is needed for future changes which will require being able to process the load priority out of order. Change-Id: Ia23421197f09789940510b03ebbbf3bf24d51bea	7 years ago
Corey Farrell	3505cc88e8	loader: Rework of load_dynamic_module. ... * Split off load_dlopen to perform actual dlopen, check results and log warnings when needed. * Always use RTLD_NOW. * Use flags which minimize number of calls to dlopen required. First attempt always uses RTLD_GLOBAL when global_symbols_only is enabled, RTLD_LOCAL when it is not. This patch significantly reduces the number of dlopen's performed. With 299 modules my system ran dlopen 857 times before this patch, 655 times after this patch. Change-Id: Ib2c9903cfddcc01aed3e01c1e7fe4a3fb9af0f8b	7 years ago
Corey Farrell	80bf0ee99a	loader: Minor fix to module registration. ... This protects the module loader itself against crashing if dlopen is called on a module from outside loader.c. * Expand scope of lock inside ast_module_register to include reading of resource_being_loaded. * NULL check resource_being_loaded. * Set resource_being_loaded NULL as soon as dlopen returns. This fixes some error paths where it was not NULL'ed. * Create module_destroy function to deduplicate code from ast_module_unregister and modules_shutdown. * Resolve leak that occured if a module did not successfully register. * Simplify checking for successful registration. Change-Id: I40f07a315e55b92df4fc7faf525ed6d4f396e7d2	7 years ago
Corey Farrell	d2e87b8e14	loader: Refactor resource_name_match. ... Optimize resource_name_match. This change eliminates use of ast_strdupa, instead verifying that both basename's are the same length, then using strncasecmp. Change-Id: I477275c0e954c99d74be5abfc8bb6545b04e5a3d	7 years ago
Corey Farrell	10b4b5d200	loader: Fix comments in struct ast_module. ... Make the comments follow doxygen format, move comments to the line before each field they describe. Change-Id: Ic445468398b5e88f13910f7c2f70bd15aad33a27	8 years ago
Corey Farrell	b9f457eac0	Modules: Additional improvements to CLI completion. ... Replace 'needsreload' argument with a 'type' argument to specify which type of modules you want completion. This provides more accurate CLI completion for load and unload commands. * 'module unload' now excludes modules that have active references or are not running. * 'module load' now excludes modules that are already running. * 'core set debug [atleast] <level> [module]' shows running modules only. ASTERISK-27378 Change-Id: Iea3e00054461484196c46f688f02635cc886bad1	8 years ago
Corey Farrell	e82b921c35	Modules: Fix issues with CLI completion. ... * Stop using ast_module_helper to check if a module is loaded, use ast_module_check instead (app_confbridge and app_meetme). * Stop ast_module_helper from listing reload classes when needsreload was not requested. ASTERISK-27378 Change-Id: Iaed8c1e4fcbeb242921dbac7929a0fe75ff4b239	8 years ago
George Joseph	747beb1ed1	modules: change module LOAD_FAILUREs to LOAD_DECLINES ... In all non-pbx modules, AST_MODULE_LOAD_FAILURE has been changed to AST_MODULE_LOAD_DECLINE. This prevents asterisk from exiting if a module can't be loaded. If the user wishes to retain the FAILURE behavior for a specific module, they can use the "require" or "preload-require" keyword in modules.conf. A new API was added to logger: ast_is_logger_initialized(). This allows asterisk.c/check_init() to print to the error log once the logger subsystem is ready instead of just to stdout. If something does fail before the logger is initialized, we now print to stderr instead of stdout. Change-Id: I5f4b50623d9b5a6cb7c5624a8c5c1274c13b2b25	8 years ago
Sean Bright	cf6a6226ab	core: Remove embedded module support ... This has not worked for some time and is no longer actively maintained. Change-Id: I5110b0db69c152761b58fa025cb0a53b0e544d99	8 years ago
Corey Farrell	d6ad867897	Fix shutdown crash caused by modules being left open. ... It is only safe to run ast_register_cleanup callbacks when all modules have been unloaded. Previously these callbacks were run during graceful shutdown, making it possible to crash during shutdown. ASTERISK-26513 #close Change-Id: Ibfa635bb688d1227ec54aa211d90d6bd45052e21	9 years ago
Corey Farrell	a6e5bae3ef	Remove ASTERISK_REGISTER_FILE. ... ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes all traces of it. Previously exported symbols removed: * __ast_register_file * __ast_unregister_file * ast_complete_source_filename This also removes the mtx_prof static variable that was declared when MTX_PROFILE was enabled. This variable was only used in lock.c so it is now initialized in that file only. ASTERISK-26480 #close Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966	9 years ago
Tzafrir Cohen	07b95f7c65	sd_notify (systemd status notifications) support ... sd_notify() is used to notify systemd of changes to the status of the process. This allows the systemd daemon to know when the process finished loading (and thus only start another program after Asterisk has finished loading). To use this, use a systemd unit with 'Type=notify' for Asterisk. This commit also adds the function ast_sd_notify(), a wrapper around sd_notify that does nothing if not built with systemd support. Also adds support for libsystemd detection in the configure script. Change-Id: Ied6a59dafd5ef331c5c7ae8f3ccd2dfc94be7811	9 years ago
George Joseph	195100e770	loader: Retry dlopen when loading fails ... Although we use the RTLD_LAZY flag when calling dlopen the first time on a module, this only defers resolution for function calls. Pointer references to functions are determined at link time so dlopen expects them to be there. Since we don't cross-module link, pointers to functions in other modules won't be available and dlopen will fail. Doing a "hardened" build also causes problems because it typically sets "-z now" on the ld command line which overrides RTLD_LAZY at run time. If the failing module isn't a GLOBAL_SYMBOLS module, then dlopen will be called again after all the GLOBAL_SYMBOLS modules have been loaded and they'll eventually resolve. If the calling module IS a GLOBAL_SYMBOLS module itself and a third module depends on it, then there's an issue because the second time through the dlopen loop, GLOBAL_SYMBOLS modules aren't given any special treatment and since the order in which dlopen is called isn't deterministic, the dependent may again be tried before the module it needs is loaded. Simple solution: Save modules that fail load_resource because of a dlopen error in a list and retry them immediately after the first pass. Keep retrying until the failed list is empty or we reach a #defined max retries. Error messages are suppressed until the final pass which also gets rid of those confusing error messages about module failures that are later corrected. Change-Id: Iddae1d97cd2f00b94e61662447432765755f64bb	9 years ago
Joshua Colp	d17d9a9288	json: Audit ast_json_* usage for thread safety. ... The JSON library Asterisk uses, jansson, is not thread safe for us in a few ways. To help with this wrappers for JSON object reference count increasing and decreasing were added which use a global lock to ensure they don't clobber over each other. This does not extend to reference count manipulation within the jansson library itself. This means you can't safely use the object borrowing specifier (O) in ast_json_pack and you can't share JSON instances between objects. This change removes uses of the O specifier and replaces them with the o specifier and an explicit ast_json_ref. Some cases of instance sharing have also been removed. ASTERISK-25601 #close Change-Id: I06550d8b0cc1bfeb56cab580a4e608ae4f1ec7d1	10 years ago
Benjamin Ford	6a764db370	ARI: Added new functionality to get information on a single module. ... An http request can be sent to retrieve information on a single module, including the resource name, description, use count, status, and support level. The command "curl -v -u user:pass -X GET 'http://localhost:8088/ari /asterisk/modules/{moduleName}'" (or something similar, depending on configuration) can be run in the terminal to access this new functionality. For more information, see: https://wiki.asterisk.org/wiki.display/~bford/Asterisk+ARI+Resource * Added new ARI functionality * Information on a single module can now be retrieved ASTERISK-25173 Change-Id: Ibce5a94e70ecdf4e90329cf0ba66c33a62d37463	10 years ago
Benjamin Ford	1b7760a8aa	ARI: Added new functionality to get all module information. ... An http request can be sent to retrieve a list of all existing modules, including the resource name, description, use count, status, and support level. The command "curl -v -u user:pass -X GET 'http://localhost:8088/ari/ asterisk/modules" (or something similar, depending on configuration) can be run in the terminal to access this new functionality. For more information, see: https://wiki.asterisk.org/wiki.display/~bford/Asterisk+ARI+Resource * Added new ARI functionality * Information on modules can now be retrieved Change-Id: I63cbbf0ec0c3544cc45ed2a588dceabe91c5e0b0	10 years ago
Corey Farrell	5c1d07baf0	Astobj2: Allow reference debugging to be enabled/disabled by config. ... * The REF_DEBUG compiler flag no longer has any effect on code that uses Astobj2. It is used to determine if reference debugging is enabled by default. Reference debugging can be enabled or disabled in asterisk.conf. * Caller information is provided in logger errors for ao2 bad magic numbers. * Optimizes AO2 by merging internal functions with the public counterpart. This was possible now that we no longer require a dual ABI. ASTERISK-24974 #close Reported by: Corey Farrell Change-Id: Icf3552721fe999365ba8a8cf00a965aa6b897cc1	10 years ago