# &develguide;

## Internals

The SEAS module runs within the Open Sip Express Router aka. &kamailio;. &kamailio; uses a pool of processes to execute the script logic on every new message received. These are called the worker processes. One of these processes will be selected to process the script, and at some point it will find a function invoking the relay of the SIP message to one of the Application Servers registered. This function has been called as_relay_t, which stands for Application Server relay (the _t stands for TransactionStatefully), and receives as the only parameter the name of the application server to be invoked.

The process will execute the as_relay_t function, which looks up in a table if there is a registered Application Server with that name. If there is one, the process will craft the SEAS header for the SIP message being handled, put it in a shared memory segment, and write the address of that segment to a pipe (4 bytes pointer in IA32).

This way, we will have all the &kamailio; processes composing the SEAS header along with the SIP message, and putting its shared memory address into that pipe. This technique of inter-process communication avoids race conditions because writing to a pipe is granted to be an atomic operation if the data to write is less than _POSIX_PIPE_BUF, which usually is 512 bytes.

At the initialization of &kamailio;, the SEAS module creates the discussed pipe, so that all the &kamailio; worker processes inherit the file descriptor associated to the pipe. Then it spawns a new process, which will be the one to open two server sockets, and wait for the Application Servers to connect and register.

Each Application Server wishing to receive events from &kamailio;, will have to open a socket to the module (the port and IP of the socket are defined at start time in the script). After connection, it has to print its identification name. The SEAS process (from now on, called event dispatcher) will then register it in its internal structures, so that the &kamailio; processes can push events for it. The following picture, shows the internals of the SEAS Event dispatcher process:

### Overview of Seas Event Dispatcher process operation

Within the SER server, the flowing of SIP Messages and control flow, is depicted in the following diagram:

### SIP Messages and control flow within SER

## SEAS Protocol

SIP is a very flexible protocol. It can be very easily extended with new features, and SIP entities have a high level of freedom in composing the SIP messages, for example setting IPs or hostnames in URIs, reordering header fields, folding headers, aggregating/scattering headers, etc.

This flexibility, though, makes it difficult to implement efficiently, because parsing of text headers requires a lot of state.

&kamailio; implements a very efficient parsing mechanism and SIP-transaction machinery. The goal of the SEAS protocol is to keep all this information that has been already extracted at &kamailio;, so that it can be reused at the Application Server.

### The SEAS protocol

The SEAS protocol is a layer of information regarding the internal structure of a SIP message that is added whenever SEAS sends a SIP event to the Application Servers. The protocol is used for communication between &kamailio; and the Application Servers.

Once an incoming SIP message has reached the worker process within &kamailio;, it copies its content into a private memory area (which is, a memory chunk not shared across processes). In this point, the message first line is parsed to know whether it is a SIP request or response.

&kamailio; uses a technique called lazy-parsing, which consists in delaying the parse of headers until some piece of the code requires it.

As the SIP message goes traversing functions and the script code, a function called parse_msg() gets called again and again, and the SIP message gets parsed further and further. Each call to parse_msg passes an integer value argument (32 bits) in which every bit signals a header to be parsed, if they are already parsed (because a previous invocation of parse_msg), the function returns immediately, otherwise, the SIP message is scanned and parsed until all the headers requested get parsed.

In each call to parse_msg, different parts of the message are analyzed, and different SIP header-specific structures get filled. Each one of this structures, give quick access to each of the parts of a SIP message header.

For example, a Via header struct is called via_body, and has these members: name, version, transport, host, proto, port, port_str, params, comment, received, rport, etc. each of these members gives quick access to each of the parts of the header. For example, a via header like this: “Via: SIP/2.0/UDP 192.168.1.64:5070;branch=z9hG4bK-c02c60cc” would have the member proto pointing to the “U” of “UDP”, and a length of 3, the host member would be pointing to “192.168.1.64” and have a length of 12, the branch member would be pointing to “z9hG4bK-c02c60cc” and a length of 16, and so on.

This structure is the result of the parsing. All this meta-information regarding the SIP message structure, is stored in a sip_msg structure, using dynamically-allocated memory segments.

&kamailio; defines different structure types describing different SIP headers, such as via_body, to_body, cseq_body, via_param, and so on. These structures are generally composed of another kind of structure called str.

The str structure is a key component of &kamailio;'s high performance. In the C programming language, a string's length is known because a '0' (null-character) is found at the end of it. This forces each of the string manipulation functions to keep looking for a '0' in the byte stream, which is quite processor consuming. Instead of this, &kamailio; defines a structure composed of a char pointer and an integer. The char points to the start of a string, and the integer gives its length, thus avoiding the '0' lookup problem, and giving a significant performance boost.

This structure has been quite useful to the design of the SEAS protocol, because it enables the description of the SIP message anatomy by giving pointers to each of its fields, and integers describing each of its lengths.

Knowing that a SIP header does not usually occupy more than a few characters (always less than 256), the pointer in the structure has been relativized to the beginning of the SIP message or the beginning of the SIP header, and the integer giving the length, has been casted to an unsigned byte (256 values, so 256 characters maximum length).

When messages get transferred from &kamailio; to the Application Server, it is optimum to keep this worthy meta-information regarding the SIP message, so that it can be used at the AS part. For this to be possible, it is needed to store the pointers to each of the syntactic structures and their length.

In general, pointers are variables that point to a region in the memory of a computer. The region of the memory is counted from the 0x00000000 address in IA32 architectures (from the beginning).

C provides functionality to do any kind of arithmetic operations over pointers (add, subtract, multiply and divide), so that the euclidean distance over the one-dimension address space can be calculated just by subtracting a base address from another pointer.

These pointers will have to be transmitted through the network, along with the SIP message, so for the pointers to keep their meaning, they need to be relativized to a known point, and the most meaningful known point in a SIP message is its start.

So making the pointers relative to the message start, gives two important features: first, it makes the pointers still valid when they arrive at another computer (because they are relative to the beginning of the message), and they occupy far less memory, because from a 4-byte pointer (in IA32) it gets translated to a 1 or 2 byte index, because an important amount of redundant information is elicited (we already know that each of the parts of the message belong to the message, so why carry the message begin address in each of the pointers ?).

The SIP messages are composed of protocol headers and a payload. The headers section don't usually surpass the 1500 byte limit, amongst other reasons, because the usual Maximum Transmission Unit in Ethernet networks is 1500 bytes and the protocol was initially designed to work on UDP. For that reason, 11 bits should be enough to address a particular region within the SIP message, because it yields 2048 positions. The closest greater value to 11 bits multiple of a byte (the basic TCP network transport unit) is 16 bits, or 2 bytes, which makes it possible to address 65536 positions from the beginning.

For the SEAS protocol to be extensible and platform-independent, all the 2-byte pointers or indexes to each of the message regions are sent in network-byte-order, or big endian. This is also useful in the JAVA part to retrieve the indexes, because the JAVA natively uses a big-endian representation of integers, regardless the architecture on which it runs.

For each kind of standard SIP header (this is, the headers referred to in the SIP specification) there is a code specification, regarding the composition of the header. Each one of its parts points to one the several components of the header. For example, a From header always has a SipURI and may have several parameters, amongst others, a tag. Then, the From header code has a field indicating where the URI starts, a codification of the URI, and several pointers that point to each one of the parameter names and values. This is the codification of the From header. All the other headers have a similar codification.

### General codification of a header

Every header codification, regardless it is known to the server or not, begins with a 2-byte unsigned integer, which points to the beginning of that header counted from the SIP message begin (a SIP message start based pointer to the header). Following these two bytes is another byte giving the length of the name, and another byte giving the length of the entire header (including name and value).

#### General codification of a SIP header in SEAS protocol

For example:

#### Example of a from header SEAS-protocol codification

#### Codification of a generic URI

As the SIP URI is one of the most used types in a SIP message, a special structure has been defined to describe the contents of it. A URI is always included inside a SIP header, or may be in the first line of a SIP Request (as the request URI).

The codification of any URI is as follows:

##### SEAS-codification of a SIP URI (byte meanings are shown)

What follows is an example of a SIP URI codification with the SEAS protocol.

##### Example of a SEAS SIP URI codification

The first byte in the encoded-URI structure, gives the index where the URI starts, counting from the beginning of the SIP header where it appears. The next two bytes are flags indicating known fields present in the URI (such as port, host, user, etc.).

All the following bytes are uri-start based pointers to the fields that are present in the URI, as specified by the flags. They must appear in the same order shown in the flags, and only appear if the flag was set to 1.

The end of the field, will be the place where the following pointer points to, minus one (note that all the fields present in a URI are preceded by 1 character, ie sip[:user][:password][@host][:port][;param1=x][;param2=y][?hdr1=a][&hdr2=b]) it will also be necessary to have a pointer at the end, pointing two past the end of the URI, so that the length of the last header can be computed.

The reason to have the “other parameters” and headers flags at the beginning (just after the strictly URI stuff), is that it will be necessary to know the length of the parameters section and the headers section. The parameters can appear in an arbitrary order, they won't be following the convention of transport-ttl-user-method-maddr-lr, so we can't rely on the next pointer to compute the length of the previous pointer field, as the ttl parameter can appear before the transport parameter. So the parameter pointers must have 2 bytes: pointer+length.

#### Codification of To and From headers

To and From headers follow the same structure, so the same codification structure has been used to describe both. The structure is depicted in the drawing:

##### SEAS codification of From and To headers

#### Codification of Contact

The contact header is one of those SIP headers that can be combined, which means that if several headers of the same type are present in the message, they can be aggregated in a single header, having the header values separated by a comma. Thus, a single Contact header can contain more than one contact-value. For this reason, the Contact codification is composed of a several Contact codifications concatenated, and a byte at the beginning telling how much Contact codifications are present. The code is depicted in the following drawing:

##### SEAS codification of a Contact header

#### Codification of Route and Record Route headers

Both Route and Record-Route headers follow an identical structure, and it is also permitted to combine several headers into one, with their bodies (or header values) separated by commas. In this case, both kinds of headers follow the same structure, defined as follows:

##### SEAS codification of a Route Header

#### Codification of Accept and Content-Type headers

These two kinds of headers carry mime type and subtype definitions in the form “type/subtype” (ie. text/xml, application/sdp or whatever). For internal handling of this headers, SER codifies the known types and subtypes into a single 32 bit integer, with the highest two bytes giving the mime type, and the lowest two bytes giving the subtype.

The difference is that Accept header can also be combined, carrying more than one header value in a single header row. Thus the Accept header has a leading byte giving the number of mime type/subtype integers present, while the Content-Type only uses 4 bytes (a 32-bit integer) giving the type/subtype.

#### Codification of Authorization headers

SIP has inherited the authentication scheme from HTTP, which is based on a digest scheme. There are several headers regarding these authorization scheme, namely Proxy-Authenticate, WWW-Authenticate, Authorization and Proxy-Authorization. All of them can be codified using the same schema, which is as follows:

##### SEAS codification of Authentication/Authorization headers

For each field present, there are 2 bytes, one pointing the place where it starts, the next giving how long this field is. The URI is a special case, and is composed of 1 byte telling how long is the URI structure, and then the encoded URI structure.

#### Codification of Allow headers

Allow headers carry request methods that a user agent or proxy understands or is willing to accept. In SER, request methods are codified into a 32-bit integer, each of its bits signals a different kind of header. The Allow header is codified copying that integer into the payload of the header.

#### Codification of Content-Disposition headers

The content-disposition is encoded within 2 bytes: the first is a header-start based pointer to where the content-disposition value starts, and the second is its length. If there are parameters present, each of them uses 1 byte pointing to where the parameter name starts, and 1 byte pointing to where the parameter value starts. From these two values, the parameter name and value lengths can be inferred.

#### Codification of Content-Length header

The content length header is codified as a 4-byte unsigned integer, in network byte order.

#### Codification of Cseq header

The Cseq header is codified using 9 bytes. The first one is a number corresponding to the internal value that SER assigns to that request method (the method ID). The following 4 bytes are an unsigned 32-bit integer according to the Cseq number. The next two bytes are the header based pointer to the beginning of the Cseq number and its length, and two more bytes pointing to the beginning of the method name and its length.

#### Codification of Expires header

The expires header is composed of 6 bytes. The first four bytes are an unsigned 32-bit integer with the parsed value of the header (which is the number of seconds before a request expires). Then follows 1 byte pointing to the beginning of the header value (the expires value as a string) and a byte giving the length of the value.

#### Codification of a SIP message

##### The general message information section

In SER, not only the headers are parsed with a high degree of optimization, but also the first line is. So for the SEAS protocol to realize this improvement, a codification for the first line of every SIP messages has also been defined.

The first two bytes of the codification are a 2-byte unsigned integer. If its value is equal or greater than 100, then this is a response, and the integer represents its status code. If its value is smaller than 100, then it is a request, and the integer represents the method of the request being transported.

###### SEAS codification of a SIP First Line

The next two bytes are an unsigned integer which is a pointer to where the actual SIP message starts, beginning from the start of the codified payload.

The next two bytes are also an unsigned integer giving the SIP message length.

The next bytes differ on the meaning depending on whether the message is a SIP Request or Response.

In case it is a Request:

The next two bytes, are a SIP-message-start based pointer to where the method begins, and the method length.

The next two bytes, are a SIP-message-start based pointer to where the Request URI begins, and the request URI length.

The next two bytes, are a SIP-message-start based pointer to where the version identifier begins, and the version identifier length.

In case it was a Response:

The next two bytes, are a SIP-message-start based pointer to where the response code begins, and the response code length.

The next two bytes, are a SIP-message-start based pointer to where the reason phrase begins, and the reason phrase length.

The next two bytes, are a SIP-message-start based pointer to where the version identifier begins, and the version identifier length.

In case the message is a SIP response, the following bytes correspond to the Request URI codification. The first byte is the length of the URI codification, followed by the URI code.

The last byte in this set, is the number of headers present in the SIP message. After this byte, goes a section, called the Message Headers Index, which gives quick access to each of the headers and their codifications present in the message.

##### The headers index section

As it has been already discussed, the aim of SEAS project is to achieve as high a performance as possible. One of the techniques enabling high performance in text-based servers is the so called lazy parsing. To enable the laziest possible parsing at the Application Server endpoint, a mechanism has been used so that access to a requested SIP header can be delayed until the application requests it, and the access can be direct to that header, without parsing the former headers present in the SIP message. Recall that one of the performance drawbacks of the SIP protocol is that headers of any type can be spread all along the header section, not having the constraint of putting the most critical sip-specific headers at the beginning and ordered (which would be, in fact, very desirable).

For this to be possible, there is a section right after the beginning of the payload (the general message information section) which is a kind of hash table, giving quick access to the codes (as explained in the previous sections) of each of the headers present in the message.

This sort of hash table, is composed of triplets of bytes. The first byte of each three is a code indicating which kind of header it points to (whether it is a From, To, Call-ID, Route header, etc). Then follows a 2 byte network-byte-order integer that points to a section in the codified-header where the body of this header is more specifically described.

This gives really fast access to any of the headers. For example, if all the Route Headers were requested by the application, then a lookup in this table would be necessary, looking for the value '9' (corresponding to the Route header) in each of the positions multiple of 3 (0,3,6,9,12, etc). This can be done in an extremely fast and easy way, as this snipped of pseudo code explains:

for(int j=0,int i=0;i<table_length;i+=3){

if(payload[i]==9)

results[j++]=i;

}

this would let in the “results” array all the indexes in the headers table that refer to a Route header. Then, the Route codification for each of the headers could be reached thanks to the two-byte unsigned integer that follows each of the header identifiers.

###### SEAS Headers Index section overview

So a SIP message codified by the SEAS protocol, has the following layout:

###### SEAS SIP-Message codification

SIP Messages are a fundamental part of the protocol, but they are not the only one. Transaction play a very important role in the SIP protocol, within SER and in any JAIN-SIP implementation. For this reason, the SEAS protocol also needs to define and implement some semantics regarding transaction handling. The events related to a transaction are: Incoming Request, Outgoing Request, Incoming Response, Outgoing Response, Timeout and Transport Error.

So the SEAS protocol defines a specific format for each one of these events. Internally, SER stores the transactions in a hash table. This hash table generates an integer for each transaction applying a hash function to its Via branch parameter, this integer is the hash index, and it identifies in which slot within the hash table the transaction is stored. The transaction table usually uses 65536 entries, so the hash collision is pretty unlikely. Anyway, every hash entry is in reality a linked list of transactions, so in the case a hash collision (two transactions being assigned to the same hash slot) the transactions are added to the same slot, each one being identified by another integer called the label. The label within a hash slot, is initially generated randomly, and then increased by one each time a transaction falls in the same slot. So every transaction is identified by a hash index and a label.

For incoming SIP requests, a transaction is generated at SER, and the SEAS module gets that transaction identifier (hash index + label), then grabs the source and destination IP, port and transport from every message, and crafts a SEAS RequestIn event. This kind of event carries all this information within it.

In order to send Responses out for the Server Transactions, JAIN can send a type of Action messages, that order SER to send them to the network. These messages follow a structure very similar to that of RequestIn events: they start with the Action length in bytes, then follows a byte giving the type of action, then follows the Hash Index and the Label associated with the transaction that is being replied, and finally the SIP Message in raw format. It doesn’t use the SEAS codification described above, because SER can easily parse the JAIN provided Response to process it and send it out, so the pre-parsing is not needed in that direction.

In order to generate Client Transactions, that is, sending SIP Requests out, JAIN utilizes another kind of action called Seas Request Action. In this case, when JAIN generates the Request to be sent out, it doesn’t have any means to know the transaction identifier (hash index and label) that will be assigned to it by SER, so a new mechanism has bee implemented to correlate JAIN requests to SER transactions. Basically, JAIN-SIP assigns a unique identifier (an integer) that is incremented by one for each new Client Transaction generated. This identifier is passed to SER along with the SIP Request, so when a SIP Response arrives to SER regarding that transaction, SER sends a ResponseIn event to the JAIN stack, containing both the initial integer identifying the transaction at JAIN and the hash index and label that have been assigned to the transaction. This way, JAIN can correlate its own identifiers with the identifiers used within SER.

###### Different kinds of SEAS codified Events and Actions

In case there is a Transaction Timeout, it is notified to the JAIN SIP Stack by passing it a Seas Incoming Response with a flag called Faked Reply, and a Response code number 408 (Request Timeout).