pipelimit Module

Daniel-Constantin Mierla

Hendrik Scholz

Edited by

Ovidiu Sas

Daniel-Constantin Mierla

   Copyright © 2013 VoIPEmbedded Inc.

   Copyright © 2010 Asipto.com

   Copyright © 2006 Freenet Cityline GmbH
     __________________________________________________________________

   Table of Contents

   1. Admin Guide

        1. Overview
        2. Algorithms
        3. Dependencies

              3.1. Kamailio Modules
              3.2. External Libraries or Applications

        4. Parameters

              4.1. hash_size (int)
              4.2. db_url (string)
              4.3. plp_table_name (string)
              4.4. plp_pipeid_column (string)
              4.5. plp_limit_column (string)
              4.6. plp_algorithm_column (string)
              4.7. timer_interval (integer)
              4.8. timer_mode (integer)
              4.9. load_fetch (integer)
              4.10. reply_code (integer)
              4.11. reply_reason (string)
              4.12. clean_unused (int)

        5. Functions

              5.1. pl_check(name [, algorithm, limit])
              5.2. pl_active(name)
              5.3. pl_drop([ [min ], max ])

        6. RPC Commands

              6.1. pl.list
              6.2. pl.stats
              6.3. pl.set_pipe
              6.4. pl.reset_pipe
              6.5. pl.rm_pipe
              6.6. pl.get_pipes
              6.7. pl.set_pid
              6.8. pl.get_pid
              6.9. pl.push_load

   List of Examples

   1.1. Set hash_size parameter
   1.2. Set db_url parameter
   1.3. Set plp_table_name parameter
   1.4. Set plp_pipeid_column parameter
   1.5. Set plp_limit_column parameter
   1.6. Set plp_algorithm_column parameter
   1.7. Set timer_interval parameter
   1.8. Set timer_mode parameter
   1.9. Set load_fetch parameter
   1.10. Set reply_code parameter
   1.11. Set reply_code parameter at runtime
   1.12. Set reply_reason parameter
   1.13. Set reply_reason parameter at runtime
   1.14. Set clean_unused parameter
   1.15. Set clean_unused parameter at runtime
   1.16. pl_check usage
   1.17. pl_active usage
   1.18. pl_drop usage

Chapter 1. Admin Guide

   Table of Contents

   1. Overview
   2. Algorithms
   3. Dependencies

        3.1. Kamailio Modules
        3.2. External Libraries or Applications

   4. Parameters

        4.1. hash_size (int)
        4.2. db_url (string)
        4.3. plp_table_name (string)
        4.4. plp_pipeid_column (string)
        4.5. plp_limit_column (string)
        4.6. plp_algorithm_column (string)
        4.7. timer_interval (integer)
        4.8. timer_mode (integer)
        4.9. load_fetch (integer)
        4.10. reply_code (integer)
        4.11. reply_reason (string)
        4.12. clean_unused (int)

   5. Functions

        5.1. pl_check(name [, algorithm, limit])
        5.2. pl_active(name)
        5.3. pl_drop([ [min ], max ])

   6. RPC Commands

        6.1. pl.list
        6.2. pl.stats
        6.3. pl.set_pipe
        6.4. pl.reset_pipe
        6.5. pl.rm_pipe
        6.6. pl.get_pipes
        6.7. pl.set_pid
        6.8. pl.get_pid
        6.9. pl.push_load

1. Overview

   This module implements traffic limiting for SIP requests.

   The module defines in an abstract mode the notion of 'pipe', which can
   be a reference to an IP address, to a network or a trunk. The
   association of traffic to a pipe is done in the config file, therefore,
   a pipe could represent SIP traffic coming from a user or the flow of
   specific SIP requests such as INVITE or REGISTER.

   Pipelimit started from ratelimit module, adding support for definition
   of pipes limits in database and dynamic names. Complexity of keeping
   everything in a module and make it dual mode functional resulted in a
   new module which is focused on just traffic shaping policies, the
   implementation of queues was discarded.

2. Algorithms

   Algorithms are based from the ratelimit module, which describes the
   algorithms in more detail. The algorithms are used by the pipelimit
   module to determine if a message should be blocked.

   Tail Drop Algorithm (TAILDROP)

   This is a trivial algorithm that imposes some risks when used in
   conjunction with long timer intervals. At the start of each interval an
   internal counter is reset and incremented for each incoming message.
   Once the counter hits the configured limit pl_check() returns false
   (negative value).

   Random Early Detection Algorithm (RED)

   The Random Early Detection Algorithm tries to circumvent the
   synchronization problem imposed by the tail drop algorithm by measuring
   the average load and adapting the drop rate dynamically. When running
   with the RED algorithm (enabled by default) Kamailio will return errors
   to the Kamailio routing engine every n'th packet trying to evenly
   spread the measured load of the last timer interval onto the current
   interval. As a negative side effect Kamailio might drop messages
   although the limit might not be reached within the interval. Decrease
   the timer interval if you encounter this.

   Network Algorithm (NETWORK)

   This algorithm relies on information provided by network interfaces.
   The total amount of bytes waiting to be consumed on all the network
   interfaces is retrieved once every timer_interval seconds. If the
   returned amount exceeds the limit specified in the modparam, pl_check()
   returns false (negative value).

   Feedback Algorithm (FEEDBACK)

   Using the PID Controller model (see Wikipedia page), the drop rate is
   adjusted dynamically based on the load factor so that the load factor
   always drifts towards the specified limit (or setpoint, in PID terms).

   As reading the CPU load average is relatively expensive (opening
   /proc/stat, parsing it, etc), this only happens once every
   timer_interval seconds and consequently the FEEDBACK value is only at
   these intervals recomputed. This in turn makes it difficult for the
   drop rate to adjust quickly. Worst case scenarios are request rates
   going up/down instantly by thousands - it takes up to 20 seconds for
   the controller to adapt to the new request rate.

3. Dependencies

   3.1. Kamailio Modules
   3.2. External Libraries or Applications

3.1. Kamailio Modules

   The following modules must be loaded before this module:
     * database connection module.
     * sl: Stateless Request Handling.

3.2. External Libraries or Applications

   The following libraries or applications must be installed before
   running Kamailio with this module loaded:
     * None.

4. Parameters

   4.1. hash_size (int)
   4.2. db_url (string)
   4.3. plp_table_name (string)
   4.4. plp_pipeid_column (string)
   4.5. plp_limit_column (string)
   4.6. plp_algorithm_column (string)
   4.7. timer_interval (integer)
   4.8. timer_mode (integer)
   4.9. load_fetch (integer)
   4.10. reply_code (integer)
   4.11. reply_reason (string)
   4.12. clean_unused (int)

4.1. hash_size (int)

   Used to compute the number of slots for the internal hash table, as
   power of 2 (number of slots = 2^hash_size, aka 1<<hash_size). If you
   have many pipes, increase this value for proper performances, but don't
   go too high (hash_size=10 means 1024 slots).

   Default value is “6” (64 slots).

   Example 1.1. Set hash_size parameter
...
modparam("pipelimit", "hash_size", 10)
...

4.2. db_url (string)

   URL of the database server to be used.

   Default value is “mysql://kamailio:kamailiorw@localhost/kamailio”.

   Example 1.2. Set db_url parameter
...
modparam("pipelimit", "db_url", "dbdriver://username:password@dbhost/dbname")
...

4.3. plp_table_name (string)

   Name of DB table where data definition for pipes is stored.

   Default value is “pl_pipes”.

   Example 1.3. Set plp_table_name parameter
...
modparam("pipelimit", "plp_table_name", "mypipes")
...

4.4. plp_pipeid_column (string)

   Name of 'pipeid' column.

   Default value is “pipeid”.

   Example 1.4. Set plp_pipeid_column parameter
...
modparam("pipelimit", "plp_pipeid_column", "name")
...

4.5. plp_limit_column (string)

   Name of 'limit' column.

   Default value is “plimit”.

   Example 1.5. Set plp_limit_column parameter
...
modparam("pipelimit", "plp_limit_column", "name")
...

4.6. plp_algorithm_column (string)

   Name of 'algorithm' column.

   Default value is “algorithm”.

   Example 1.6. Set plp_algorithm_column parameter
...
modparam("pipelimit", "plp_algorithm_column", "name")
...

4.7. timer_interval (integer)

   The length of the timer interval in seconds. Counted messages are reset
   each timer_interval, therefore the amounts of messages have to be
   divided by this timer_interval value to get the messages per second
   value. For example, if you want to allow an average of 10 messages per
   second with a timer interval of 10 seconds, then the limit value has to
   be 100.

   Note: A too small value may lead to performance penalties due to timer
   process overloading.

   Default value is 10.

   Example 1.7. Set timer_interval parameter
...
modparam("pipelimit", "timer_interval", 5)
...

4.8. timer_mode (integer)

   Control what timer process to be used: 0 - use main fast timer; 1 - use
   secondary wheel timer.

   Default value is 0.

   Example 1.8. Set timer_mode parameter
...
modparam("pipelimit", "timer_mode", 1)
...

4.9. load_fetch (integer)

   Control if the module should fetch the load for CPU and network
   traffic. If set to 0, no load is fetched; if set to 1, the load is
   fetched every timer interval.

   Default value is 1.

   Example 1.9. Set load_fetch parameter
...
modparam("pipelimit", "load_fetch", 0)
...

4.10. reply_code (integer)

   The code of the reply sent by Kamailio while limiting.

   Default value is 503.

   Example 1.10. Set reply_code parameter
...
modparam("pipelimit", "reply_code", 505)
...

   This value can be modified at runtime using kamcmd

   Example 1.11.  Set reply_code parameter at runtime
kamcmd cfg.set_now_int pipelimit reply_code 505

4.11. reply_reason (string)

   The reason of the reply sent by Kamailio while limiting.

   Default value is "Server Unavailable".

   Example 1.12. Set reply_reason parameter
...
modparam("pipelimit", "reply_reason", "Limiting")
...

   This value can be modified at runtime using kamcmd

   Example 1.13.  Set reply_reason parameter at runtime
kamcmd cfg.set_now_string pipelimit reply_reason "Limiting"

4.12. clean_unused (int)

   Clean unused pipes after this number of timer intervals.

   Default value is 0 (cleanup disabled).

   Example 1.14. Set clean_unused parameter
...
modparam("pipelimit", "clean_unused", 10)
...

   This value can be modified at runtime using kamcmd

   Example 1.15.  Set clean_unused parameter at runtime
kamcmd cfg.set_now_int pipelimit clean_unused 10

5. Functions

   5.1. pl_check(name [, algorithm, limit])
   5.2. pl_active(name)
   5.3. pl_drop([ [min ], max ])

5.1.  pl_check(name [, algorithm, limit])

   Check the current request against the 'name' pipe.

   If algorithm and limit are provided, the function attempts to create a
   new pipe if one with that name doesn't exit. If it exists, no changes
   to algorithm is done and the pipe limit is set to the 'limit' parameter
   value, if this is greater than 0. Algorithm is case sensitive.

   The pipe name can be provided via a pseudo variable.

   The method will return:
     * -2 if no pipe was found
     * -1 if pipe limit was reached
     * 1 if pipe limit was NOT reached
     * 2 if pipe has NOP algorithm

   Meaning of the parameters is as follows:
     * name - the string or pseudovariable with the pipe name.
     * algorithm - the string or pseudovariable with the algorithm. The
       values can be: TAILDROP, RED, NETWORK, or FEEDBACK - see readme of
       ratelimit module for details on each algorithm.
     * limit - the integer or pseudovariable with the limit value per
       timer_interval.

   This function can be used from ANY_ROUTE.

   Example 1.16. pl_check usage
...
        # perform pipe match for current method
        if (!pl_check("one")) {
                pl_drop();
                exit;
        }
...
        # use pipe 'one' for the current method via PV
        $var(p) = "one";
        $var(check_result) = pl_check("$var(p)");
        switch($var(check_result)) {
        case -2:
                xlog("L_ALERT","pl_check(\"$var(p)\") drop -pipe NOT found\n");
                pl_drop();
                exit;
                break;
        case -1:
                xlog("L_ALERT","pl_check(\"$var(p)\") drop\n");
                pl_drop();
                exit;
                break;
        case 1:
                xlog("L_INFO", "pl_check(\"$var(p)\") pass\n");
                break;
        case 2:
                xlog("L_ALERT","pl_check(\"$var(p)\") pass -NOP algorithm\n");
                break;
        default:
                xlog("L_ERR","pl_check(\"$var(p)\") dropping \
with unexpected retcode=$var(check_result)\n");
                pl_drop();
                exit;
        }
...
        # perform pipe match for authenticated user
        $var(limit) = 20;
        if (!pl_check("$au", "TAILDROP", "$var(limit)")) {
                pl_drop();
                exit;
        }
...
        # perform pipe match for INVITE
        if (is_method("INVITE")) {
                $var(invlimit) = 10;
                if (!pl_check("$si", "TAILDROP", "$var(invlimit)")) {
                        pl_drop();
                        exit;
                }
        }
...

5.2.  pl_active(name)

   Check the pipe 'name' was already created. Return 1 (true) if the pipe
   is found, -1 (false) if the pipe is not found.

   This function can be used from ANY_ROUTE.

   Example 1.17. pl_active usage
...
        if (!pl_active("one")) {
                # pipe does not exist
                exit;
        }
...

5.3.  pl_drop([ [min ], max ])

   For the current request, a "503 - Server Unavailable" reply is sent
   back. The reply may or may not have a "Retry-After" header. If no
   parameter is given, there will be no "Retry-After" header. If only the
   max parameter is given, the reply will contain a "Retry-After: max"
   header. If both min and max params are given, the reply will contain a
   "Retry-After: random" header with random being a random value between
   the given min and max.

   Meaning of the parameters is as follows:
     * min - the minimum value of "Retry-After" header.
     * max - the maximum value of "Retry-After" header.

   This function can be used from
   REQUEST_ROUTE|BRANCH_ROUTE|FAILURE_ROUTE|ONSEND_ROUTE.

   Example 1.18. pl_drop usage
...
        if (!pl_check("one")) {
                # send back a "503 - Server Unavailable"
                # with a "Retry-After: 5"
                pl_drop("5");
                exit;
        }
...

6. RPC Commands

   6.1. pl.list
   6.2. pl.stats
   6.3. pl.set_pipe
   6.4. pl.reset_pipe
   6.5. pl.rm_pipe
   6.6. pl.get_pipes
   6.7. pl.set_pid
   6.8. pl.get_pid
   6.9. pl.push_load

6.1.  pl.list

   Lists the details of one or all pipes, respectively the attributes pipe
   name (id), algorithm, limit and counter.

   Name: pl.list

   Parameters: name - (optional) pipe name

   RPC Command Format:
...
kamctl rpc pl.list
kamctl rpc pl.list testid
...

6.2.  pl.stats

   Lists the parameters and variables in the pipelimit module: pipe id,
   pipe load and pipe counter.

   Name: pl.stats

   Parameters: none

   RPC Command Format:
...
kamctl rpc pl.stats
...

6.3.  pl.set_pipe

   Sets the pipe parameters for the given pipe id.

   Name: pl.set_pipe

   Parameters:
     * pipe_id - pipe id.
     * pipe_algorithm - the algorithm assigned to the given pipe id.
     * pipe_limit - the limit assigned to the given pipe id.

   RPC Command Format:
...
kamctl rpc pl.set_pipe 2 RED 10
...

6.4.  pl.reset_pipe

   Reset the values associated with the pipe.

   Name: pl.reset_pipe

   Parameters:
     * pipe_id - pipe id.

   RPC Command Format:
...
kamctl rpc pl.reset_pipe p01
...

6.5.  pl.rm_pipe

   Remove the pipe structure matching the given pipe id.

   Name: pl.rm_pipe

   Parameters:
     * pipe_id - pipe id.

   RPC Command Format:
...
kamctl rpc pl.rm_pipe p01
...

6.6.  pl.get_pipes

   Gets the list of in use pipes.

   Name: pl.get_pipes

   Parameters: none

   RPC Command Format:
...
kamctl rpc pl.get_pipes
...

6.7.  pl.set_pid

   Sets the PID Controller parameters for the Feedback Algorithm.

   Name: pl.set_pid

   Parameters:
     * ki - the integral parameter.
     * kp - the proportional parameter.
     * kd - the derivative parameter.

   RPC Command Format:
...
kamctl rpc pl.set_pid 0.5 0.5 0.5
...

6.8.  pl.get_pid

   Gets the list of in use PID Controller parameters.

   Name: pl.get_pid

   Parameters: none

   RPC Command Format:
...
kamctl rpc pl.get_pid
...

6.9.  pl.push_load

   Force the value of the load parameter. This command is useful for
   testing the Feedback algorithm.

   Name: pl.push_load

   Parameters:
     * load - the forced value of load (it must be greater than 0.0 and
       smaller than 1.0).

   RPC Command Format:
...
kamctl rpc pl.push_load 0.85
...
