This reduces the complexity of the algorithm, at the expense of more
code, which makes it faster:
Rate old new
old 57803/s -- -23%
new 74906/s 30% --
Change-Id: Ic230cddceb269e4c452f3713e75be16569908458
Unix systems default to using /bin/sh when trying to run an executable
script that has no shebang.
Change-Id: I18ef857fbc54fd72e6ad06b0f85c8c908fb0880d
We require these with perlcritic, but have not been triggered as these
scripts have no shebang, which we will be adding later.
Change-Id: I81ac98c9ee72cce9b3446a98e1aae53e4c04b578
This adds a behavior change, reflected in the unit test changes, by
always returning the lowercased versions of the array elements when
case insensitiveness is enabled. This should be fine, given that we
do not care from the callers PoV, and the result might have changed
depending on the order of the inputs.
Change-Id: I44896183b566e921cc1287f106fb4bb9332acb97
Which is also around 300% faster:
Rate had-hoc sprintf
had-hoc 3508772/s -- -75%
sprintf 14084507/s 301% --
Change-Id: I5ebce0058473f9de08c381739646a20e42308a8f
no_xfer: [cdr0]
currently cdr0->destination_user_in ok as is
bilnd_xfer: [cdr0, cdr0_xfer-1]
currently cdr0_xfer-1->destination_user_in, change to
cdr0->destination_user_in
attn_xfer_ivr: [cdr0, cdr0_pbx-1, cdr0, cdr0_xfer-1_groupcdr]
currently cdr0_pbx-1->destination_user_in, change to
cdr0_pbx-1->groupcdr->destination_user_in
Change-Id: If7099e344f303412958c902e91f2e7d5ae7bff35
- any non-numeric numbers will be sorted out as failed in general
- fix chown/chmod after renaming
Change-Id: Ib1409e16e803498b108df9b9f6e9f0959fb24c56
+ control command to reset fsn
+ control command to reset a export stream's export_status
+ produces intact .ama files now
Change-Id: I3b9456416851d2caa3273b54d9e835097663c98b
tool to switch contracts with some billing profile
to a desired profile package, as specified in mappings.yml.
run like:
perl process.pl --task=cleanup --task=set_profile_package --skip-errors
Change-Id: Ie7c07642ed29d91166aad95dfb62a0e3e40a5e3f
- not that obvious, since a "sip:" prefix is neither automatically
prepended nor enforced by panel UI.
Change-Id: I8b48996b12f184d92450dd5babe7321b2571b674
+ prepare/merge/clean callforward+voicemail Y/N data
+ add voip_cf_destinations dao
+ add voip_cf_destination_sets dao
+ voip_cf_mappings, destination_sets and destinations
insert dao method
+ write callforwards
+ write callforward related preference values
+ not well-formed enough: fixed issues popped up when
trying to view/edit created callfowards in panel
+ fix "database locked" error - properly handle SQLite's
serialized transactions (single transaction at a time).
+ write report file for debugging/review
- json graph of merged data of a subscriber
- including fields info/warning/error messages
for a subscriber
- write it snychronized now
+ get rid of JSON::XS
+ extend NGCPRestApi connector to support file transfers
+ heuristic for missing "channels" (by subscriber number
count)
+ prevent mysql deadlocks when writing to ngcp
+ strictly consider record order from imports
+ task end result stats polished
+ cleanup code a bit
+ add kmailio.location dao
+ add kmailio.location insert dao method
+ writing "permanent registrations" to kamailio.location
+ generate location "ruid" and "partition" according to kamailio
Change-Id: Ief9a7634b4930e51d79ac5e963ba48769d3708ea
+ introduce support to manipulate contract, domain, etc.
voip_preferences
+ write concurrent_max_total preference
+ error in invalid "channels = 0"
+ try to find non-empty sip-password of a subscriber, if any
+ try to find nonempty web username+password, if any
+ set emtpy web_password, if web_username is empty
+ auto-generate web_username, if duplicate
+ auto-generate web_password, if web_username is not emtpy but
web_password is empty
+ parse export_CLIR.csv
+ write clir preference
+ write allowed_ips_grp preference
+ use maximum of concurrent_max_total for a subscriber,
when inconsisten
+ use combined list of allowed ips, when inconsistent
+ write adm_ncos, derive barrings combination if
inconsistent for a subscriber
+ log created items, cleanup code a bit
+ range unfolding for numbers in allowedcli files
Change-Id: I26bab2cb9b4ef349777ebef2e883c9e41af3eeba
+ "allowed_cli" import gets a different meaning,
it are additional "numbers" (alias+allowed_cli) to import
+ unique cc+ac+sn in subscriber file
+ unique cc+ac+sn in allowed_cli file
+ logging of cc+ac+sn of allowed_cli records
already found in subscriber file
+ logging of sip_usernames of allowed_cli records
not found in subscriber file
+ reseller mapping
Change-Id: I620db415f5dab91e8d5c6336b7a45b3a0ffeacb9
+ allow to specify multiple subscriber files to import
+ allow to specify multiple allowed_cli files to import
Change-Id: I80f83542f541a0d8857d477158dc043cfabb93ba
- CSV importer: escaped field enclosing, mutliline values,
comment char, constant memory
- SQLite table to hold flat file content
- multithread import for large files
- concept of single flatfile abandonend as teletek also wants
to import AllowedIPs, MaxConcurrentCalls, NCOS levels,
CallForwards
- load teletek's main subscriber flatfile
- single insert/upsert txn import to identify duplicate
cc+ac+sn
- alias range unfolding
- alias range unfolding fix, analysis of duplicate cc+ac+sn
in export_kundinfo_leica2.csv
- fixed one-off bug in parser
- aliases sorting
- aliases dupe check
- check for sip account used by different contacts
- reseller cache/check for missing reseller
- domain cache/check for missing domain
- billing profile cache/check for missing billing profile
- rest api cert-authentification, multi-user, thread safe, tested
- contact create, no update
- contract create, no update
- billing mapping creaet, no update
- contract balance create, no update
- billing subscriber, prov subscriber, primary voip_number and voip_dbalias
- prefs: account_id, cc, ac
- prefs: allowed_cli, cli
- load allowed clis file, identify dupes
- todo: remove partly adjusted old code parts, depending on this will be extended or not.
- add aliases, tested, panel can display 1000+ aliases.
- disable multithreading
Change-Id: I4f5ae61610bab744e3d0644f7a56afa8874a2c3c
The latter does not support YAML 1.1, nor many parts of the
specification. Use the more compliant implementation, in addition to try
to converge to a single one, so that we do not get serialization delta
surprises.
Change-Id: Ia3f6808b9bb84afb224cc29040964eefe07bf118
fix tool to update acc_trash and re-enqueue records accumulated
during 3.x->4.x updates due to introduction of cdr lnp prefix
fields.
Change-Id: Iea6140edccc1aeb456f6584b7d8335e18ba1b69d
+initial process.pl
+contract balance gap fix thread
+contract iteration
+context init
+context teardown
+detectig gaps and overlaps
+insert missing contract balances:
no packages or free time considered for now
+calendar implementation
+drop URI::Encode dependency
+for now, adopt to run with mr3.8 tables only.
Change-Id: I48e0de59bb703adc0d28092c1a93a5b47432bc03
+importing lnp numbers now requires unique numbers,
it required unique prefix+number before
+ngcp billing.lnp_number records are considered as
unique per number (before it was unique per
prefix+number)
Change-Id: I5a6f9ed4235a1ee0edecd58ec3b04ad393839dbb
+tolerant line endings for the username+password import
+same for batch files
+only process batch records with updated or added delta
+convenience: no need to clear the batch import before
loading a new batch file.
+if records in a batch file overlap with ones from previous
batch files, they are "updated" and hece processed again.
+subscriber password update instead of recreating subs
(aka know as reprovisioning).
+if there's a peer_auth_pass present, its also updated.
Change-Id: I22ea6485dd2d48e3633ebb262037c8d3fafc923c
+subscriber provisioning via db:
+delta handling
+create billing subscriber
+create provisioning subscriber
+create/update billing voip_number
+create cli preference
+update billing subscriber
+create voip_dbalias
+create allowed clis preference
+release other subscriber's voip numbers
+delete obsolete dbalieses
+delete obsolete allowed_clis
+create kamailio voicemail user
+create account_id preference
+create cc preference
+check config parameters (billing profile of same reseller..)
+subscriber and contract termination via api
+reprovisioning use-case (upon changed password):
-terminate subscriber
-terminate contract
-create contact, contract, subscriber as normal
+log successful provisions
+limit to imported batch with --batch
+bulk write subscriber preferences directly to db
+barring profile to adm_ncos_ID preference
+peer_aut_* preferences: init, switchover, clear modes
+bulk write via rest-api:
+call forwards from features_define.cfg
+bulk create lnp_numbers
+checks, delta detection for lnp carriers
+changed from txn per record (42min) to txn per block (3min)
+tuning sqlite params to prevent db lock error
with multithreading
+workaround for
https://rt.cpan.org/Public/Bug/Display.html?id=102791
+include .pl and .cfg file types to be installed by the
package
+fixup to skip '0' usernames with --skip-errors
+check for minimum password length
+block rollback issue
+import txns only, stmt was not "finished"
+fix db_do "E0E" return value
+disable multithreading when perldebug
+warning counter (--skip-errors only)
+fixed locking issue
+make the --run=<db-prefix> option work to use
different import slite db's
Change-Id: I82d27b593b1d5194331c1559b022cf677e232550
+provisioning db connectivity
+kamailio db connectivity
+default db connectivity for distributed txn's (XA)
+provisioning db dao's
+kamailio db dao's
+subscriber provisioning via db:
+create contact
+create contract
+create billing_mapping
+create contract_balance
Change-Id: I11de85fa5473e59bd6a8c05e1541a9fbc96c1a9d
+possibility to load ngcp conf files
+db connectivity must be set up explicitly for now..
+start implementing subscriber creation
+main record iteration loop, only reading the dump:
single thread: ~10min
2 threads: ~5 min
4 threads: ~3.5 min
+"check" task
+rest-api requests
+collection processing
+multithreaded processing
+rest item object prototype
Change-Id: I8ee12037b863f878101e53d69b6ae2b44a96780b
+delta calculation for lnp,features,featureoptions
+skip prepaid subscribers
+import username+password.txt
-a full file with plaintext instead of hashes
is needed to test..
+import order forced
+import "batch.txt"
-prepared for a single column containing the
numbers to migrate
Change-Id: Icfedda64b635fe872f6f24c005228773d9869b21
+allow #* symbols in setoptionitems in
Features_Define.cfg
+prevent double execution+log properly
+make sure to close all db connections before
forking threads, not only project-specific ones
+per-db dao folder for ipgallery migration project
+start with ngcp db dao's:
+contract_balances
+contracts
+refactor dao/sql processing class
+params hash for method with long arg list
+take out obselete methods
+dedicated unique field option for "insert_record"
+split SqlRecord class into
SqlRecord&SqlProcessor
+dependencies in Build.pl and debian/control
+took out comments from debian/control
+calcualte deltas of Subscriber_Define.cfg imports
+"truncate subscriber import" task
Change-Id: I8eab88319e6a4ba3d5b6ba935e50a56cc1763ad9
+import LNP_Define.cfg
+workingdir folder for rollback logs
+cleanup task
+chained task execution
+dry mode for import
+refactoring txn demaraction for rowblock
db operations
+take out "lock table" impl completely,
too dangerous
+take out DateTime::Format::Excel dependency
for now
+allow '#' comments in Subscriber_Define.cfg
and Lnp_Define.cfg
+check for import sequence
+referential integrity checks for Subscriber_Define.cfg
+task summary messages / rowcounts
+exclude and exclude exception number patterns
for Subscriber_Define.cfg
+ignore_unique options
Change-Id: If1f5094e15828633e212b1a8d651c97816c388b3
+setting to drop duplicate setoptionitems in
Feature_Define.cfg
+adjust Feature_Define.cfg threads vs block size
settings to arrange with a low sqlite busy_timeout
+Subscriber_Define.cfg importer
+resolve garbage collector problems with
log4perl
+refactoring to conform with perl module
lib/folder structure for installation
-runs again
Change-Id: I1821cc0085086684c3c1415be4c262453509045a
Rene Krenn
Guillem Jover