Closes#2213
With the TLS options, the Sentinel thread still spoke plain TCP to the
Sentinels and for the role check of the master they name, so a Sentinel
set with tls-port and port 0 could not be used.
sentinel_connect() now starts TLS with the same context and options. A
Sentinel's certificate must match its host as given in
--redis-sentinel; a master's must match the address the Sentinels gave,
or --redis-tls-server-name. The handshake happens in the thread, never
under r->lock.
Tested by auto-daemon-tests-redis-tls.pl.
Change-Id: I84daf340c5f464765f234b35255cc76238bc8d13
Closes#2213
There is no way to reach a Redis server that only accepts TLS (managed
services, or tls-port with port 0); asked on the mailing list in 2022:
https://groups.google.com/g/rtpengine/c/gO2hWMKxCNA/m/DNH37Z54BQAJ
With --redis-cafile or --redis-capath, every Redis connection uses TLS,
and --redis-certfile with --redis-keyfile adds a client certificate. As
for MQTT, the options exist only in a build with hiredis_ssl. hiredis
checks only the chain, so the name is checked here, in the
subjectAltName: the host as configured, the IP address, or
--redis-tls-server-name. The handshake is bounded by the command
timeout, or by the connect timeout when there is none.
Tested by auto-daemon-tests-redis-tls.pl (libio-socket-ssl-perl added to
Build-Depends).
Change-Id: I74a78be0d40c3755fb1071536deb41443c890817
Closes#2212
Replication is asynchronous, so a master that is replaced takes with it
the writes it acknowledged but had not replicated: a call created then
is missing on the new master, and a call deleted then comes back.
--redis-resync-on-master-change (off by default) keeps the
master_replid of the INFO that redis_connect() already sends. When the
write connection reconnects to another ID, every own call stored at
least once, and the own DELs of the last --redis-resync-delete-window
(at most --redis-resync-delete-max), are queued for the repair thread
of --redis-repair-writes; the calls through the call iterator, one at a
time. No key is scanned. New statistics: redisresyncs, redisresynccalls
and redisresyncdeletes.
Tested by auto-daemon-tests-redis-resync.pl.
Change-Id: I0daac4d2b9ada45a843d27a6de073d8822067a8b
Closes#2212
After a failover the new master refuses writes with NOREPLICAS until it
has a replica. Those writes are counted and logged, but never made
again: a call created then is missing at the next restore, and a call
deleted then comes back.
With --redis-repair-writes (off by default), an own call whose write was
not stored (an error reply, a connection lost before the reply, a
skipped write) is marked in a table keyed by call ID: the call itself,
held, while it is in the call table; its ID and database once it is
deleted, as a deleted call keeps its memory until its last reference
goes. A stored write clears the mark. A marked SET supersedes a marked
DEL of the same ID; if its call was never written, it takes that key
over, so that its own deletion deletes it. A "redis repair" thread
writes the marked calls again, one at a time, through
redis_write_call(), now shared with redis_update_onekey(): a live call
gets its current state, the key of a deleted call a DEL, unless a new
call uses its ID. Signalling writes waiting for r->lock go first, for
100 ms at most. A failed write is retried after a second; at most 100000
calls are kept. New statistic: redisrepairedwrites. A repaired DEL
assumes one writer per database.
Tested by auto-daemon-tests-redis-repair.pl and
auto-daemon-tests-redis-repair-ids.pl.
Change-Id: Ia56a36bba0dd7eed3754694128e2ba6a74827c42
Closes#2212
A subscription made through "sentinel" stayed on the old master after a
failover, and missed the notifications of the new one.
With Sentinel, the notification event loop gets a one-second timer:
when the master changed, it closes the subscription, and
redis_notify_loop() subscribes again on the new master. The
subscription's context also gets a connect callback, since hiredis
frees a context that fails to connect without calling the disconnect
callback. Notifications missed during the switch are not replayed.
Tested by auto-daemon-tests-redis-sentinel-subscribe.pl.
Change-Id: Ia14d48e220bbf98cc48964e76f33ff2a497953e9
Closes#2212
Behind Redis Sentinel, rtpengine stays on the address it was given:
after a failover its writes go to the old master, which refuses them
until it is demoted, then serves them as a replica.
New options --redis-sentinel, --redis-sentinel-master and
--redis-sentinel-interval; the keyword "sentinel" replaces IP:PORT in
--redis, --redis-write or --redis-subscribe. A "redis sentinel" thread
asks every Sentinel for the master, takes the highest config epoch
(never an older one than in use), checks the node with ROLE and
publishes it with a generation number. It also runs at once when a
connection meets a replica. redis_check_conn() compares the generation
and reconnects on a change; these connections require the master role.
Without "sentinel", none of this runs.
Tested by auto-daemon-tests-redis-sentinel.pl and -sentinel-start.pl.
Change-Id: I7b31b50968fd6a255b9933fbf2c95fc95a8b8408
Closes#2212
--redis-write accepts any role, so after a failover that demotes its
node, rtpengine reconnects to the same address, accepts the replica,
and every later write is refused with READONLY. --redis alone requires
the master at connect, but keeps the connection when its writes are
refused.
The write connection now requires the master role, as --redis alone
does, and a connection that requires it is dropped on a write refused
with READONLY or MASTERDOWN, so that the next operation reconnects and
checks the role. Other refusals keep the connection. The read database
of --redis, when --redis-write is given, still accepts any role.
Tested by auto-daemon-tests-redis-master-only.pl; the READONLY cases
of auto-daemon-tests-redis-errors.pl, which expected the connection to be
kept, now use NOREPLICAS.
Change-Id: Ie164c2c3495c6c8b77553caf9a449e40f8e13699
Expose locally measured RTP loss and jitter in NG query/delete output
even without RTCP, merge kernel-measured per-SSRC loss/jitter from shm
into userspace consumers (NG, MQTT, RTCP RRs, call summary), start the
kernel loss tracker at -1 like userspace, and skip jitter metrics when
no RTP clock rate is known.
Closes#2201Closes#934
Change-Id: Id7a6b1b87ae04def15fd81968d3be07fdf385395
Add a codec module for Codec 2, using libcodec2 for decoding and
encoding.
There is no RFC payload format for Codec 2, so this follows the
convention Asterisk uses: encoding name "codec2" at 8000 Hz, mono, with
one or more complete frames per RTP packet, each padded to whole bytes.
The mode is selected with the format parameter "mode=" (3200, 2400,
1600, 1400, 1300, 1200 or 700C) and defaults to 2400, the only mode
Asterisk supports. With the transcode option it can also be given as
the bit rate, e.g. "codec2/8000/1/3200".
Different modes are different bit streams, so payload types only match
when they use the same mode. Modes 1600 and below use 40 ms frames, so
their ptime is rounded up to a multiple of 40 ms. Modes not known to
rtpengine (e.g. 450) are passed through unchanged when both sides
agree, but can't be transcoded.
libcodec2 is optional. It is detected with pkg-config (or from
/usr/include/codec2) by utils/gen-codec2-flags. Without it, Codec 2 is
still recognised for SDP handling and passthrough. In a Debian build
the dependency can be dropped with the pkg.ngcp-rtpengine.nocodec2
build profile.
test-transcode gains Codec 2 tests for both directions in modes 2400,
3200 and 1300, plus passthrough. check_encoder() is no longer limited
to the extended AMR tests, as these use it too.
Closes#2226
Change-Id: I2cad021f48e2043f36c30f5e6290cb7d6eba8c1f
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ref #2211
A reconnect attempt runs under r->lock, which every ng request that
writes a call waits for (#1865). Against a server that drops packets it
takes the full connect timeout, so signalling stalls for up to that long
at each attempt, for as long as the outage lasts.
Make the attempt without the lock: the thread that reconnects sets
r->reconnecting and owns r->ctx until it clears the flag under the lock.
Other threads wait for it for up to --redis-reconnect-slow, then skip
their operation, as during the pause. A DEL, also the one after a failed
restore, waits for the attempt to end, as it waited for the lock before:
skipped, the key would stay. After the lock was released,
redis_update_onekey() checks again that the call is live.
redis_do_delete() leaves the key to a new call with the same ID, checked
on every DEL, and hands it over to that call if it was never written, so
that its own deletion deletes the key. The CLI and the notification
thread's own connection leave a connection being made alone. A command
on an established connection still holds the lock.
Tested by auto-daemon-tests-redis-reconnect.pl.
Change-Id: Ia98e01c57d4825633e473e3174947daa17afbf13
Ref #2211
On a role mismatch, the reconnect in redis_check_conn() sleeps one
second, asks INFO again and sleeps once more: two seconds under
r->lock, which every Redis update holds before the ng reply (#1865).
After a failover, with --redis on the old master's address, every
update did this, and ng requests timed out.
Make one attempt, with no sleep. After a role failure, or a failure
that took --redis-reconnect-slow (100 ms) or more, skip Redis
operations on that connection for --redis-reconnect-pause (1000 ms),
through restore_tick_us as --redis-allowed-errors does. A quick failure
(connection refused) is retried at once, as before. As with a failed
attempt, a skipped operation is not written; skips are logged once per
pause. The startup connection keeps its eleven attempts.
Tested by auto-daemon-tests-redis-errors.pl.
Change-Id: Ia84288f1c6fe97c4317689fef75d0ff5fc9061f8
Ref #2211
Call updates (SET) and deletions (DEL) are pipelined, and
redis_consume() freed their replies unread: a write refused with
READONLY or NOREPLICAS, as during a Sentinel failover, left no trace in
rtpengine.
Check every reply. An error reply is counted in a new statistic,
rediswriteerrors, and logged at most once a second per connection; the
connection is kept, since a reconnect would get the same answer. A read
error or timeout on the replies drops the connection at once instead of
at the next PING. An encoding failure no longer drops the connection.
Tested by auto-daemon-tests-redis-errors.pl, with a new fake Redis
module, NGCP::Rtpengine::FakeRedis.
Change-Id: I4c49460d7894de0c5eaf5e5c27f5a6460e4115a7
Document the new `stop-on-ingress` flag in the list of generic flags and
in the `play media` NG command.
Describe what stops the playback, that `block egress` is implied,
which packets are not forwarded, and the kernel forwarding caveat.
Change-Id: If61f7ed0d644737768d559e7e08d116a5d109a48
Ports normally come from the port pool, so the caller only learns the
port from rtpengine's reply. A controller that manages the media ports
itself, and has already announced them to the remote side, needs
rtpengine to open exactly those ports.
`local-port` sets the port facing the recipient of the message (the one
in the returned SDP) and `from-local-port` the one facing its sender.
Each takes an integer for the first media section or a list, one per
media section. RTCP takes the next port when not multiplexed. The ports
may be outside of the pool, they are only used when the media's ports
are first opened, and the request fails if they are taken.
`local-port` works in every message that opens ports: offer, answer,
publish, subscribe request and create. `from-local-port` only has a
meaning in offer and answer.
The frequencies list parser becomes a generic integer list parser.
Closes#2216
Change-Id: Ie82338becb8347757c7cc19d90675b927f1edfee
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A controller that allocates the SRTP keys itself needs rtpengine to
encrypt with them, instead of generating keys or copying the ones from
the received SDP.
`SDES-key-SUITE:KEY` (or `key-SUITE:KEY` in the `SDES` list) sets the
master key and salt, base64 encoded as in an a=crypto inline: key, for
that crypto suite on the side facing the recipient of the message. It
works in offers and answers, and also for SRTP to SRTP calls, where
rtpengine then re-encrypts instead of passing the packets through.
Closes#2217
Change-Id: I1a12e4488617cd5daaa49741773a14e330abbfdf
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SSRC only appears in the response to the offer or answer that picked it.
The `egress SSRCs` list fills in after the first forwarded packet, so a query
at signalling time returns nothing.
Report the value as its own per-media key. It is set when the offer is
processed, so it can be read before media flows.
Closes#2171
Change-Id: I738d702ab13fd71b56269960bce16994ddb3c8a0
A controlling agent feeding media into a WebRTC SFU needs to know the SSRC
of the incoming stream before it can bind a receiver to it, and the SFU
needs that SSRC to stay put for the life of the call.
With `fixed egress SSRC` set, rtpengine picks an SSRC for each media at
signalling time and returns the chosen values in the offer or answer
response, one entry per m= section along with its index and media type.
The value is recorded as the ingress entry's `ssrc_map_out`, which is
already the mapping every consumer reads: the egress entry is keyed off it,
RTCP rewriting uses it, and the kernel module is handed it as `ssrc_out`.
Because one egress entry then stands in for every ingress SSRC, it outlives
a change of source, so `seq_diff` carries across on its own. All that is
left is re-basing it once at the changeover, taking the last sequence
number from the counter shared with the kernel module so that it is still
right while the stream is offloaded.
The substitution itself goes in the plain passthrough handler rather than
the SSRC passthrough one, so that it applies to builds without transcoding
support as well. Those builds compile the kernel side regardless, so
leaving the userspace half out would have meant the original SSRC being
forwarded until the stream was offloaded and the fixed one afterwards.
Change-Id: Iad5508c53d29d1e7d54c825da1ab56069cdd89b7
RTP header extensions are only processed, and unknown ones stripped, if
at least one side of the call negotiated header extensions through
`a=extmap` attributes. Otherwise RTP is forwarded untouched, including
any header extensions an endpoint may include without having announced
them.
Add a `force strip extmap` flag which selects header extension
processing for all streams of a call regardless of whether any
extensions were negotiated, so that unannounced header extensions are
removed. This is useful when forwarding media from sources that always
include header extensions towards endpoints that do not cope with them.
Change-Id: I57e14389c3610550a6859ac08e65ad866283554f
Add an opt-in rollback message to the NG protocol. It lets a signalling
application undo an SDP offer that rtpengine has applied but the remote
endpoint subsequently rejects, without deleting the established call.
A client enables checkpointing with track-state on an offer. rtpengine
snapshots the affected dialogue before applying it; an answer commits the
exchange and discards the snapshot, and rollback restores and consumes it.
A dialogue holds at most one outstanding checkpoint, so offers arriving
before an exchange completes keep the existing snapshot and a rollback
returns to the last completed offer/answer.
A checkpoint holds a snapshot in the call record format the Redis encoder
already produces, so there is one definition of what a call's state looks
like and no second encoder to keep in step. Decoding a snapshot needs the
field readers the Redis restore path already had, so those are shared
rather than duplicated; json_restore_call() reads the same fields, in the
same order, with the same strict-return behaviour.
Applying a snapshot copies negotiated state back onto the live objects,
matched by unique id. The objects are never recreated, so local sockets
and ports survive. State a rejected offer introduced is removed as well as
overwritten: a field the encoder writes only when set is absent from a
snapshot taken before it existed, and that absence clears it.
ICE and DTLS reconverge rather than being rewound, since applying the
offer has already reset the agent and shut down the DTLS association.
Restoring the accepted credentials and candidates lets connectivity checks
rebuild ICE state, and restoring the fingerprint, TLS id and setup permits
a fresh handshake.
Outstanding checkpoints are stored in the Redis call record so they
survive takeover. Checkpoint data is auxiliary: one that cannot be read is
discarded in full while the call itself is restored without rollback
capability.
The preload shims no longer inherit the sanitizer options from CFLAGS.
They are loaded by the harness into the test scripts' own interpreter,
where a sanitizer runtime crashed the daemon tests on aarch64 during
library initialisation. Everything else in CFLAGS, the Debian hardening
flags included, still applies to them.
The Redis tests no longer assume the record length is four digits, and
read the record across as many reads as it takes.
Squashed:
commit ad49459152
Author: Daniel Donoghue <oneofsomany@hotmail.com>
Date: Fri Aug 28 13:33:17 2026 +0200
MT#55283 address review comments
Snapshots are now bencode, GLib macros are used throughout, and keys and
offsets are built with the existing helpers rather than by hand.
New tests cover the snapshot format and the state that exists only inside
a snapshot.
commit ff47944ad5
Author: Daniel Donoghue <oneofsomany@hotmail.com>
Date: Fri Aug 28 15:24:56 2026 +0200
MT#55283 scope snapshots to the dialogue
A snapshot now holds only the two monologues being checkpointed, and only
the entries the decoder reads: the socket and endpoint map pools, sinks,
subscriptions and aliases are left out.
The from-tag check in the rollback handler is dropped, since
call_get_monologue() is keyed on the tag.
commit 3bb19f2804
Author: Daniel Donoghue <oneofsomany@hotmail.com>
Date: Mon Aug 31 08:41:08 2026 +0200
MT#55283 give each monologue its own checkpoint
The checkpoint moves from a list on the call to a pointer on the monologue,
and its snapshot covers only that monologue. Checkpoints are stored in the
call record as checkpoint-<monologue id>; num_checkpoints and the offerer
and answerer ids are gone.
A monologue is shared between the branches of a forked call, so rolling one
branch back no longer reinstates what rolling another back had undone.
The call-level json dict is no longer written into snapshots, since nothing
reads it back.
commit 0c986a81e2
Author: Daniel Donoghue <oneofsomany@hotmail.com>
Date: Mon Aug 31 10:01:54 2026 +0200
MT#55283 do not report a rollback that did not happen
call_merge() renumbers every unique id, and a snapshot is keyed on them, so
a checkpoint taken before a merge no longer describes anything. It is
dropped with the ids it refers to, and rollback reports none outstanding
rather than a success that restored nothing.
Both sides of a dialogue are checkpointed together. A monologue is shared
between the branches of a forked call, so one side could already hold a
checkpoint while the other had never been tracked, leaving a rollback to
restore half a dialogue and still report success.
Each fix has a test that fails without it.
commit dfe7a72aff
Author: Daniel Donoghue <oneofsomany@hotmail.com>
Date: Mon Aug 31 10:41:50 2026 +0200
MT#55283 document that merging a call drops its checkpoints
Merging renumbers the state a snapshot refers to, so an outstanding
checkpoint is discarded and a later rollback reports none.
Closes#2159
Change-Id: Ie773e2cd4a39fc1c8f39fafe959784e269c0c80e
Forcing the DTLS role to passive only makes sense for the backwards
(torwards the offerer) direction anyway, as towards the answerer (in an
offer) it's not allowed by the RFC, and in an answer it's not possible
as the active DTLS connection would already be in progress.
Convert DTLS= processing into a list as multiple flags might be
required.
Change-Id: I54239280460e1bbce1966bc17a384489eb978618
Amends: 78ee0451 / Ib1d07831
Related: #2156
Related: #2145
Add the 'all' keyword for the codec accept option in publish mode:
- 'all': keep all codecs supported for transcoding and their
supplemental codecs, removing only unsupported ones
- 'all' + 'any': keep all codecs from the offer without removing any
Closes#2105
Change-Id: Ic271a5e461703e3b9dba3c0717bfbb9da89a03ca
- Add pyproject.toml and tox.ini with consistent black and pycodestyle
settings.
- Run black for consistent coding style.
- Manually fix remaining issues:
+ Split overlong strings.
+ Reflow comments.
Change-Id: I731497f3a08bd70eb32ce452d10c00be6dd096f7
New NG protocol commands that mix audio from a source participant
into a destination participant's stream without a full offer/answer
exchange.
- inject start: creates one-way subscription with audio_player mixing
- inject stop: removes injection and restores original codec routing
- Supports cross-call injection via source-call-id (merges calls internally)
- inject attribute persisted through Redis for HA failover
- Audio-only; matches media by media-id, index, then type fallback
Closes#2082
Change-Id: Ib88113b021a4b76bf98892eb1af60c0b36c0d6d3
Instead of having to manually disable legacy iptables support, auto
detect presence of libiptc and only include the feature if present.
Closes#2045
Change-Id: I649320727e9e9a565eabd369c683dfe6b6bdaec1