SipCapture Module

Alexandr Dubovikov

   <alexandr.dubovikov@gmail.com>

Edited by

Alexandr Dubovikov

   <alexandr.dubovikov@gmail.com>

   Copyright © 2011-15 SIPCAPTURE ORG

   Copyright © 2011 QSC AG

   Copyright © 2011 http://www.qsc.de
     __________________________________________________________________

   Table of Contents

   1. Admin Guide

        1. Overview
        2. Dependencies

              2.1. Kamailio Modules
              2.2. External Libraries or Applications

        3. Parameters

              3.1. db_url (str)
              3.2. table_name (str)
              3.3. mt_mode (str)
              3.4. hash_source (str)
              3.5. db_insert_mode (integer)
              3.6. capture_on (integer)
              3.7. capture_mode (str)
              3.8. hep_capture_on (integer)
              3.9. raw_ipip_capture_on (integer)
              3.10. raw_moni_capture_on (integer)
              3.11. raw_socket_listen (string)
              3.12. raw_interface (string)
              3.13. raw_sock_children (integer)
              3.14. promiscuous_on (integer)
              3.15. raw_moni_bpf_on (integer)
              3.16. capture_node (str)
              3.17. insert_retries (integer)
              3.18. insert_retry_timeout (integer)
              3.19. callid_aleg_header (str)
              3.20. topoh_unmask (int)
              3.21. nonsip_hook (int)
              3.22. event_callback (str)
              3.23. capture_bad_msgs (int)

        4. Functions

              4.1. sip_capture([table], [cmode])
              4.2. report_capture([table], [cid], [data])
              4.3. float2int(fval, ival)
              4.4. sip_capture_forward(uri)

        5. Event Routes

              5.1. event_route[sipcapture:request]

        6. RPC Commands

              6.1. sipcapture.status param

        7. Database setup
        8. Limitations

   List of Examples

   1.1. Set db_url parameter
   1.2. Set sip_capture parameter
   1.3. Set mt_mode parameter
   1.4. Set mt_mode parameter
   1.5. db_insert_mode example
   1.6. Set capture_on parameter
   1.7. capture_mode example
   1.8. Set hep_capture_on parameter
   1.9. Set raw_ipip_capture_on parameter
   1.10. Set raw_moni_capture_on parameter
   1.11. Set raw_socket_listen parameter
   1.12. Set raw_interface parameter
   1.13. Set raw_sock_children parameter
   1.14. Set promiscuous_on parameter
   1.15. Set raw_moni_bpf_on parameter
   1.16. Set capture_node parameter
   1.17. Set insert_retries parameter
   1.18. Set insert_retry_timeout parameter
   1.19. Set callid_aleg_header parameter
   1.20. Set topoh_unmask parameter
   1.21. Set nonsip_hook parameter
   1.22. Set event_callback parameter
   1.23. Set capture_bad_msgs parameter
   1.24. sip_capture() usage
   1.25. report_capture() usage
   1.26. report_capture() usage
   1.27. sip_capture_forward() usage
   1.28. event_route[sipcapture:request] usage

Chapter 1. Admin Guide

   Table of Contents

   1. Overview
   2. Dependencies

        2.1. Kamailio Modules
        2.2. External Libraries or Applications

   3. Parameters

        3.1. db_url (str)
        3.2. table_name (str)
        3.3. mt_mode (str)
        3.4. hash_source (str)
        3.5. db_insert_mode (integer)
        3.6. capture_on (integer)
        3.7. capture_mode (str)
        3.8. hep_capture_on (integer)
        3.9. raw_ipip_capture_on (integer)
        3.10. raw_moni_capture_on (integer)
        3.11. raw_socket_listen (string)
        3.12. raw_interface (string)
        3.13. raw_sock_children (integer)
        3.14. promiscuous_on (integer)
        3.15. raw_moni_bpf_on (integer)
        3.16. capture_node (str)
        3.17. insert_retries (integer)
        3.18. insert_retry_timeout (integer)
        3.19. callid_aleg_header (str)
        3.20. topoh_unmask (int)
        3.21. nonsip_hook (int)
        3.22. event_callback (str)
        3.23. capture_bad_msgs (int)

   4. Functions

        4.1. sip_capture([table], [cmode])
        4.2. report_capture([table], [cid], [data])
        4.3. float2int(fval, ival)
        4.4. sip_capture_forward(uri)

   5. Event Routes

        5.1. event_route[sipcapture:request]

   6. RPC Commands

        6.1. sipcapture.status param

   7. Database setup
   8. Limitations

1. Overview

   The sipcapture module stores incoming/outgoing SIP messages in a
   database.

   Kamailio can capture SIP messages in three modes
     * IPIP encapsulation. (ETHHDR+IPHDR+IPHDR+UDPHDR).
     * Monitoring/mirroring port.
     * Homer encapsulation protocol mode (HEP v1, v2, v3).

   The capturing can be turned on/off using rpc commands. Example:

   kamctl rpc sipcapture.status on

   kamctl rpc sipcapture.status off

2. Dependencies

   2.1. Kamailio Modules
   2.2. External Libraries or Applications

2.1. Kamailio Modules

   The following modules must be loaded before this module:
     * database module - mysql, postgres, dbtext, unixodbc...

2.2. External Libraries or Applications

   The following libraries or applications must be installed before
   running Kamailio with this module loaded:
     * None.

3. Parameters

   3.1. db_url (str)
   3.2. table_name (str)
   3.3. mt_mode (str)
   3.4. hash_source (str)
   3.5. db_insert_mode (integer)
   3.6. capture_on (integer)
   3.7. capture_mode (str)
   3.8. hep_capture_on (integer)
   3.9. raw_ipip_capture_on (integer)
   3.10. raw_moni_capture_on (integer)
   3.11. raw_socket_listen (string)
   3.12. raw_interface (string)
   3.13. raw_sock_children (integer)
   3.14. promiscuous_on (integer)
   3.15. raw_moni_bpf_on (integer)
   3.16. capture_node (str)
   3.17. insert_retries (integer)
   3.18. insert_retry_timeout (integer)
   3.19. callid_aleg_header (str)
   3.20. topoh_unmask (int)
   3.21. nonsip_hook (int)
   3.22. event_callback (str)
   3.23. capture_bad_msgs (int)

3.1. db_url (str)

   Database URL.

   Default value is “mysql://kamailio:kamailiorw@localhost/kamailio”.

   Example 1.1. Set db_url parameter
...
modparam("sipcapture", "db_url", "mysql://kamailio:kamailiorw@localhost/kamailio
")
...

3.2. table_name (str)

   Name of the table's name used to store the SIP messages. Can contain
   multiple tables, separated by "|".

   Default value is "sip_capture". Only for Homer 3. For Homer 5, please
   use an argument for the sip_capture function.

   Example 1.2. Set sip_capture parameter
...
modparam("sipcapture", "table_name", "homer_capture")
...
modparam("sipcapture", "table_name", "homer_capture1|homer_capture2");
...

3.3. mt_mode (str)

   Name of the mode used for storing data in multiple tables. Modes can be
   "rand" (random), "round_robin" (use a round_robin algorithm) or "hash"
   (use hashing to determine the table to store). These modes are only
   triggered if there is more than one table specified in table_name
   parameter, separated by "|".

   Default value is "rand".

   Example 1.3. Set mt_mode parameter
...
modparam("sipcapture", "mt_mode", "hash")
...

3.4. hash_source (str)

   The field of the SIP message used for hashing, when mt_mode is set to
   "hash". The value can be "call_id", "to_user" or "from_user".

   Default value is "call_id".

   Example 1.4. Set mt_mode parameter
...
modparam("sipcapture", "hash_source", "to_user")
...

3.5. db_insert_mode (integer)

   If set to 1, use INSERT DELAYED to store sip message into capture table
   when the DB driver has support for it. If no INSERT DELAYED support is
   offered by DB driver, then standard INSERT is used.

   If set to 2, use ASYNC INSERT to store sip message into capture table
   when the DB driver has support for it. If no ASYNC INSERT support is
   offered by DB driver, then standard INSERT is used.

   Default value is 0 (no INSERT DELAYED).

   Example 1.5. db_insert_mode example
modparam("sipcapture", "db_insert_mode", 1)

3.6. capture_on (integer)

   Parameter to enable/disable capture globally (on(1)/off(0))

   Default value is "0".

   Example 1.6. Set capture_on parameter
...
modparam("sipcapture", "capture_on", 1)
...

3.7. capture_mode (str)

   This parameter can be used for defining a capture mode which can be
   used in the sip_capture calls as a parameter. A capture mode has a name
   and some parameters. It must be defined in the format:
   name=>param1=val1;param2=val2;... The parameters are db_url,
   table_name, mt_mode and hash_source (optional). Multiple capture modes
   can be defined by using this parameter multiple times. After this, the
   capture modes can be used like: sip_capture ("", "CAPTURE_MODE");

   Example 1.7. capture_mode example
modparam("sipcapture", "capture_mode", "mode1=>db_url=mysql://user:passwd@host/d
bname1;table_name=homer_capture1|homer_capture2;mt_mode=hash;hash_source=call_id
;")
modparam("sipcapture", "capture_mode", "mode2=>db_url=mysql://user:passwd@host/d
bname2;table_name=homer_capture3|homer_capture4;mt_mode=rand;")

3.8. hep_capture_on (integer)

   Parameter to enable/disable capture of HEP (on(1)/off(0))

   Default value is "0".

   Example 1.8. Set hep_capture_on parameter
...
modparam("sipcapture", "hep_capture_on", 1)
...

3.9. raw_ipip_capture_on (integer)

   Parameter to enable/disable IPIP capturing (on(1)/off(0))

   Default value is "0".

   Example 1.9. Set raw_ipip_capture_on parameter
...
modparam("sipcapture", "raw_ipip_capture_on", 1)
...

3.10. raw_moni_capture_on (integer)

   Parameter to enable/disable monitoring/mirroring port capturing
   (on(1)/off(0)) Only one mode on raw socket can be enabled! Monitoring
   port capturing currently supported only on Linux.

   Default value is "0".

   Example 1.10. Set raw_moni_capture_on parameter
...
modparam("sipcapture", "raw_moni_capture_on", 1)
...

3.11. raw_socket_listen (string)

   Parameter indicate a listen IP address of RAW socket for IPIP
   capturing. You can also define a port/portrange for IPIP/Mirroring
   mode, to capture SIP messages in specific ports:

   "10.0.0.1:5060" - the source/destination port of the SIP message must
   be equal 5060

   "10.0.0.1:5060-5090" - the source/destination port of the SIP message
   must be equal or be between 5060 and 5090.

   The port/portrange must be defined if you are planning to use mirroring
   capture! In this case, the part with IP address will be ignored, but to
   make parser happy, use i.e. 10.0.0.0

   Default value is "".

   Example 1.11. Set raw_socket_listen parameter
...
modparam("sipcapture", "raw_socket_listen", "10.0.0.1:5060-5090")
...
modparam("sipcapture", "raw_socket_listen", "10.0.0.1:5060")
...

3.12. raw_interface (string)

   Name of the interface to bind on the raw socket.

   Default value is "".

   Example 1.12. Set raw_interface parameter
...
modparam("sipcapture", "raw_interface", "eth0")
...

3.13. raw_sock_children (integer)

   Parameter define how many children that must be created to listen the
   raw socket.

   Default value is "1".

   Example 1.13. Set raw_sock_children parameter
...
modparam("sipcapture", "raw_sock_children", 6)
...

3.14. promiscuous_on (integer)

   Parameter to enable/disable promiscuous mode on the raw socket. Linux
   only.

   Default value is "0".

   Example 1.14. Set promiscuous_on parameter
...
modparam("sipcapture", "promiscuous_on", 1)
...

3.15. raw_moni_bpf_on (integer)

   Activate Linux Socket Filter (LSF based on BPF) on the mirroring
   interface. The structure is defined in linux/filter.h. The default LSF
   accept a port/portrange from the raw_socket_listen param. Currently LSF
   supported only on Linux.

   Default value is "0".

   Example 1.15. Set raw_moni_bpf_on parameter
...
modparam("sipcapture", "raw_moni_bpf_on", 1)
...

3.16. capture_node (str)

   Name of the capture node.

   Default value is "homer01".

   Example 1.16. Set capture_node parameter
...
modparam("sipcapture", "capture_node", "homer03")
...

3.17. insert_retries (integer)

   The number of times Kamailio should retry to write to the Homer
   database in case the first attempt failed. The retry is also limited
   timewise by the insert_retry_timeout parameter. Values allowed range
   from 0 to 500.

   Default value is 0 (no retries).

   Example 1.17. Set insert_retries parameter
...
modparam("sipcapture", "insert_retries", 5)
...

3.18. insert_retry_timeout (integer)

   The time limit in seconds Kamailio retries to write to the Homer
   database in case the first attempt failed. This parameter is only used
   together with the insert_retries parameter. Values allowed range from 0
   to 300.

   Default value is 60 seconds.

   Example 1.18. Set insert_retry_timeout parameter
...
modparam("sipcapture", "insert_retry_timeout", 10)
...

3.19. callid_aleg_header (str)

   Header name used to correlate A-leg with B-leg. It can take a list of
   headers, separated by semicolon, e.g. "X-CID0;X-CID1". First match
   wins.

   Default value is "X-CID".

   Example 1.19. Set callid_aleg_header parameter
...
modparam("sipcapture", "callid_aleg_header", "X-CallIDALeg")
...

3.20. topoh_unmask (int)

   If set to 1, call-id will be unmasked using topoh module api (topoh
   module must be loaded in this case).

   Default value is 0.

   Example 1.20. Set topoh_unmask parameter
...
modparam("sipcapture", "topoh_unmask", 1)
...

3.21. nonsip_hook (int)

   If set to 1, event route sipcapture:siprequest is run when HEP message
   is received.

   Default value is 0.

   Example 1.21. Set nonsip_hook parameter
...
modparam("sipcapture", "nonsip_hook", 1)
...

3.22. event_callback (str)

   The name of the function in the kemi configuration file (embedded
   scripting language such as Lua, Python, ...) to be executed instead of
   event_route[...] blocks.

   The function receives a string parameter with the name of the event,
   the values can be: 'sipcapture:request'.

   Default value is 'empty' (no function is executed for events).

   Example 1.22. Set event_callback parameter
...
modparam("sipcapture", "event_callback", "ksr_sipcapture_event")
...
-- event callback function implemented in Lua
function ksr_sipcapture_event(evname)
        KSR.info("===== sipcapture module triggered event: " .. evname .. "\n");
        return 1;
end
...

3.23. capture_bad_msgs (int)

   If set to something different than 0, tries to capture also the broken
   SIP messages.

   Default value is 0.

   Example 1.23. Set capture_bad_msgs parameter
...
modparam("sipcapture", "capture_bad_msgs", 1)
...

4. Functions

   4.1. sip_capture([table], [cmode])
   4.2. report_capture([table], [cid], [data])
   4.3. float2int(fval, ival)
   4.4. sip_capture_forward(uri)

4.1.  sip_capture([table], [cmode])

   Store the current processed HEP/IPIP SIP message in a database. It is
   stored in the form prior applying changes made to it.

   Meaning of the parameters is as follows:
     * table - The table where HEP SIP message will be stored. Homer 5 use
       now tables with datestamp. To generate an automatic table's name
       please use strftime parameters. I.e. $var(table) =
       "sip_capture_call_%Y%m%d" and set the variable as an argument of
       the sip_capture function.
     * cmode - The reference to the capture_mode parameter.

   This function can be used from ANY_ROUTE.
   Default value is "NULL".

   Example 1.24. sip_capture() usage
...
sip_capture();
...
sip_capture("sip_capture_call_20160124");
...
sip_capture("", "cmode");
...

4.2.  report_capture([table], [cid], [data])

   Store the current processed HEP REPORT message in a database.

   Meaning of the parameters is as follows:
     * table - The table where REPORT message will be stored.
     * cid - The correlation id.
     * data - The custom report data in JSON format.

   This function can be used from ANY_ROUTE.
   Default value is "NULL".

   Example 1.25. report_capture() usage
...
report_capture();
...
report_capture("report_data", "$ci");
...
report_capture("report_data", "$ci", "{\"MOS\":4.1,\"PACKET_LOST\":100"});
...

4.3.  float2int(fval, ival)

   Return the value of atof(fval) * atoi(ival). On case the result is 0,
   then -1 is returned.

   This function can be used from ANY_ROUTE.

   Example 1.26. report_capture() usage
...
$var(res) = float2int("10.5", "1");
...

4.4.  sip_capture_forward(uri)

   Forward the HEP packet to an address specified by the parameter uri (it
   has to be a sip uri format). The function should be used inside
   event_route[sipcapture:request]. After using this function, add a
   return 0 in order to stop processing further the packet in the local
   Kamailio instance.

   This function can be used from ANY_ROUTE.

   Example 1.27. sip_capture_forward() usage
...
    event_route[sipcapture:request] {
        ...
        if(src_ip==1.2.3.4) {
            sip_capture_forward("sip:2.3.4.5:5090");
            return 0;
        }
        ...
    }
...

5. Event Routes

   5.1. event_route[sipcapture:request]

5.1. event_route[sipcapture:request]

   Event route block to be executed when HEP packet is received. It
   requires module parameter 'nonsip_hook' to be set to 1.

   Example 1.28. event_route[sipcapture:request] usage
...
# new event sipcapture socket
modparam("sipcapture", "nonsip_hook", 1)
...

event_route[sipcapture:request] {

        xlog("HEP Request!\n");
        xlog("received sipcapture request from $si:$sp\n");
        xlog("HEP VERSION $hep(version) request from $si:$sp\n");
        xlog("HEP CHUNK Source IP $hep(0x002) request from $si:$sp\n");

        # is it SIP ?
        if($hep(0x00b) == 1) {

                # do parsing internal
                return 1;
        } else {
                # if report lets proceed here with payload
                xlog("HEP CHUNK PAYLOAD $hep(0x00f) request from $si:$sp\n");
                return 0;
        }
}
...

6. RPC Commands

   6.1. sipcapture.status param

6.1.  sipcapture.status param

   Name: sipcapture.status

   Parameters:
     * on or off: turns on/off SIP message capturing. Possible values are:
          + on
          + off
     * “check” does not change sipcapture status, just reports the current
       status.

   RPC Command Format:
...
kamctl rpc sipcapture.status on
kamctl rpc sipcapture.status off
kamctl rpc sipcapture.status check
...

7. Database setup

   Before running Kamailio with the sipcapture module, you have to set up
   the database tables where the module will store the data. For that, if
   the table were not created by the installation script or you choose to
   install everything by yourself you can use the homer_databases.sql, SQL
   script in the sql folder of sipcapture module as template. You can also
   find the complete database documentation on the project webpage,
   https://www.kamailio.org/docs/db-tables/kamailio-db-devel.html.

8. Limitations

     * 1. Only one capturing mode on RAW socket is supported: IPIP or
       monitoring/mirroring port. Don't activate both at the same time.
       Obsolete. Please use HEP mirroring now.
     * 2. Mirroring port capturing works only on Linux.
