Ref #2211
cli_handle() takes a str, but most "set" handlers parsed their value
with strtol(), strtod() or atoi() on instr->s and printed it with "%s",
reading up to the next NUL. The ng "cli" command passes a slice of the
request: "set maxsessions 1000" set 10007, "set rediscmdtimeout 0" set
7 ms, and error messages quoted the rest of the request.
Parse with the str_to_X helpers, plus a new str_to_d for maxcpu and
maxload, and print with STR_FORMAT. As with the str_to_X family, a
value with trailing characters ("10x") is refused, and so is a
non-number in the seven handlers that read it as 0. "terminate" now
matches all, own and foreign exactly (str_cmp()), not as a prefix.
Tested by auto-daemon-tests-redis-errors.pl and the new t/test-str.c.
Change-Id: I92ed33a4229688aba513c3d0410b9d77a8bcd516
Ref #2211
On a role mismatch, the reconnect in redis_check_conn() sleeps one
second, asks INFO again and sleeps once more: two seconds under
r->lock, which every Redis update holds before the ng reply (#1865).
After a failover, with --redis on the old master's address, every
update did this, and ng requests timed out.
Make one attempt, with no sleep. After a role failure, or a failure
that took --redis-reconnect-slow (100 ms) or more, skip Redis
operations on that connection for --redis-reconnect-pause (1000 ms),
through restore_tick_us as --redis-allowed-errors does. A quick failure
(connection refused) is retried at once, as before. As with a failed
attempt, a skipped operation is not written; skips are logged once per
pause. The startup connection keeps its eleven attempts.
Tested by auto-daemon-tests-redis-errors.pl.
Change-Id: Ia84288f1c6fe97c4317689fef75d0ff5fc9061f8
Ref #2211
Call updates (SET) and deletions (DEL) are pipelined, and
redis_consume() freed their replies unread: a write refused with
READONLY or NOREPLICAS, as during a Sentinel failover, left no trace in
rtpengine.
Check every reply. An error reply is counted in a new statistic,
rediswriteerrors, and logged at most once a second per connection; the
connection is kept, since a reconnect would get the same answer. A read
error or timeout on the replies drops the connection at once instead of
at the next PING. An encoding failure no longer drops the connection.
Tested by auto-daemon-tests-redis-errors.pl, with a new fake Redis
module, NGCP::Rtpengine::FakeRedis.
Change-Id: I4c49460d7894de0c5eaf5e5c27f5a6460e4115a7
Firstly, play a long media playback towards A with `stop-on-ingress`
and check that the first RTP packet from B ends it. That means,
the player goes silent, the triggering packet itself is not forwarded
and the following packets from B are forwarded to A.
A later `stop media` from the controller is accepted and doesn't disturb the forwarding.
Then, as a counter-check, the same playback with just `block egress`
keeps playing while B's media stays blocked.
Change-Id: I13427c95bb4358549ad52a91b12b715b036c51be
Document the new `stop-on-ingress` flag in the list of generic flags and
in the `play media` NG command.
Describe what stops the playback, that `block egress` is implied,
which packets are not forwarded, and the kernel forwarding caveat.
Change-Id: If61f7ed0d644737768d559e7e08d116a5d109a48
For the case when the media playback is only meant
to fill a gap between the start of playing media with
the `play_media()` and the time a real remote end-point
starts sending rtp stream.
For example, the ringbacktone is emulated towards the caller
(triggered by previously seen 183 with SDP), while the callee
not actually sending anything (no media coming). In case such
a peer stays silent, keep playing the file, blob or db-id given
file until the peer actually sends at least one rtp packet from
the previously advertised source.
(The same way for the `stop media`, media forwarding resumes).
Important, the flag implies `block-egress`, so the other
parties media stays blocked while the playback is ongoing.
The check runs in the packet path (with the call locked on R),
so it only marks the player and schedules its timer object immediately.
The stop itself then runs from the media player thread
(but this time with the call locked on W).
Unfortunately, those blocked destinations are excluded from kernel forwarding,
because we need to let the daemon see the packets coming. But that's not a big
issue, because as soon as the real end-point exchange comes (final 200OK with SDP),
kernel space processing will be used again.
As the NG caller still sends its own `stop media` afterwards,
a second shutdown of the player becomes common.
Don't flush the send queue then, because without a coder handler, the flush would match
and drop packets that don't belong to the player.
Change-Id: I41c02c1278f380ccd46dd1254f140e306afd4c67
Ports normally come from the port pool, so the caller only learns the
port from rtpengine's reply. A controller that manages the media ports
itself, and has already announced them to the remote side, needs
rtpengine to open exactly those ports.
`local-port` sets the port facing the recipient of the message (the one
in the returned SDP) and `from-local-port` the one facing its sender.
Each takes an integer for the first media section or a list, one per
media section. RTCP takes the next port when not multiplexed. The ports
may be outside of the pool, they are only used when the media's ports
are first opened, and the request fails if they are taken.
`local-port` works in every message that opens ports: offer, answer,
publish, subscribe request and create. `from-local-port` only has a
meaning in offer and answer.
The frequencies list parser becomes a generic integer list parser.
Closes#2216
Change-Id: Ie82338becb8347757c7cc19d90675b927f1edfee
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A controller that allocates the SRTP keys itself needs rtpengine to
encrypt with them, instead of generating keys or copying the ones from
the received SDP.
`SDES-key-SUITE:KEY` (or `key-SUITE:KEY` in the `SDES` list) sets the
master key and salt, base64 encoded as in an a=crypto inline: key, for
that crypto suite on the side facing the recipient of the message. It
works in offers and answers, and also for SRTP to SRTP calls, where
rtpengine then re-encrypts instead of passing the packets through.
Closes#2217
Change-Id: I1a12e4488617cd5daaa49741773a14e330abbfdf
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ref #2211
When json_restore_call() fails, it sends "DEL <call-id>" twice: on the
write connection, in its own DB, and on the notification connection, in
whatever DB the last keyspace notification selected. For a foreign call
(a keyspace notification, or the restore of a subscribed keyspace at
startup) that key belongs to the instance that owns the call, which
keeps writing it: a standby that fails to restore one record deletes the
owner's copy, and the call cannot be taken over any more. For an own
call, the DEL on the notification connection can hit a peer's DB.
Only delete an unrestorable record when the call is our own, and only on
the write connection, in our own DB. A foreign record that can never be
restored now stays until its owner rewrites or deletes it, or until it
expires (--redis-expires). Seen in a lab with Redis Sentinel, where a
subscriber's DEL was refused only because its node had just become a
replica.
Tests, in auto-daemon-tests-redis-subscribe.pl: a keyspace notification
for a foreign call whose record cannot be parsed is followed by no
command on either connection, and an own record that cannot be parsed at
startup is deleted once, on the write connection only. Without this
change both get a DEL on both connections.
Change-Id: I5d8a53f3515596cf3fc2c58ea97847476385b876
Ref #2211
redis_set_timeout() receives the timeout in microseconds (the
--redis-cmd-timeout value times 1000) and logged it unconverted, so
--redis-cmd-timeout=1000 was reported as:
Setting timeout for Redis commands to 1000000 milliseconds
Log the value in milliseconds, as the message says. The value passed to
hiredis is unchanged.
No functional change. Seen in the log of every connection made with a
command timeout set; there is no test, as no test reads this line.
Change-Id: Ie4d81e4f8baab1969210a589c98245407b088561
Ref #2211
redis_connect() finds "role:" in the INFO reply and compares what
follows with memcmp(): 11 bytes for "role:master", 19 for
"role:active-replica" and 10 for "role:slave". memcmp() may read all n
bytes, so a reply that ends less than 19 bytes after "role:" (for
example a replica's "role:slave\r\n" with nothing after it) is read
past its end. Real Redis and KeyDB servers send many more lines after
the role, so this does not show in practice, but AddressSanitizer
aborts on it (heap-buffer-overflow in the memcmp interceptor) with a
minimal reply, such as one from a test server or a proxy.
Use strncmp(): hiredis NUL-terminates the reply string, so the compare
stops at its end.
Tested by auto-daemon-tests-redis-errors.pl, added later in this
series, which answers a reconnect's INFO with just "role:slave": an
ASan build aborts there without this change. The change is early in
the series so that it can be taken on its own.
Change-Id: Icdf3d3a23464d861f57406436328d0b166794362
Ref #2211
"set rediscmdtimeout 0" reconnects the Redis connections, but each of
the three calls is guarded by rtpe_redis instead of its own pointer:
- with --redis and no notification connection (no --redis-subscribe and
no subscribed keyspace), redis_reconnect(rtpe_redis_notify) locks a
NULL mutex and rtpengine crashes;
- with --redis-write and no --redis, the write connection is never
reconnected, so its command timeout is never cleared.
Guard each call with its own pointer, as the non-zero branch already
does.
Tested by auto-daemon-tests-redis-errors.pl, added later in this
series ("set rediscmdtimeout 0" over the ng "cli" command with --redis
only): without this change rtpengine crashes and never replies. The
change is first in the series so that it can be taken on its own.
Change-Id: Ia458a6eb0b70a6906ab410fd91087ad5af581f3b
Clear the matching per-destination suppression state when a completed telephone-event is deduplicated at media scope. Otherwise later listeners suppress speech until the timestamp-discrepancy timeout expires.
Add repeated-burst coverage for AMR-WB and Opus listeners in both destination orders, with blocked DTMF, prompt playback, and an event sink.
Closes#2193
Change-Id: Ia73e9324ecc6122b090be36c384b21000ef94609
monologue_publish() never copied the offered label into the media,
so sdp_create() omitted a=label from the publish answer and from
subscribe requests for that publisher. Add the missing
media_update_label() call.
Closes#2200
Change-Id: Id9aaf4ab09c21f98d3dcaf78632413b99f4a97ff
The kernel tracks lost packets per SSRC through the `lost_bits` window
in the shared ssrc_stats, which was left zeroed, while userspace starts
its own window as all-received. As userspace already advances `ext_seq`
before a stream is kernelised, the kernel then counted up to 32 packets
it did receive as lost, both per SSRC and in the interface and global
packets_lost counters. Start the shared window as all-received too.
Change-Id: I3d6cd85a1b764a6bb3296daf7489091767449554
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ssrc_collect_metrics() converted the running RFC 3550 jitter estimate
to milliseconds in place on every call timer run with --measure-rtp,
dividing it by clock_rate/1000 each time. The estimator is kept in
RTP timestamp units and also feeds generated RTCP RRs, which therefore
reported a jitter decaying towards zero. Convert a copy instead.
The RTCP test expected the resulting zero jitter; it now gets the
value actually measured from its back-to-back packets.
Change-Id: Ice29029322764fd76374eb3f7dda517570f7f75b
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Just squash passthrough with the !ice flag check,
because essentially doing exactly the same logic.
Change-Id: Ie34b41d52e7128b0fc361a6ad6483c07a0970700
Both mean that the ice towards the offerer gets rejected.
When a certain peer who previously offered ice, sends SDP without ice
(e.g. T.38 re-invite), or the ice was requested to be removed
(when processing answer) with `ICE=remove` then the ice agent is left alive
with its learned ufrag, candidates and succeeded pairs.
The stream endpoint is then taken from that SDP (non-ice path).
If this peer later offers ice again with the same credentials (e.g.
falling back to audio after an error response like 488 for that T.38),
then this stale agent is reused:
`__fill_stream()` ignores the new endpoint because the ice data
considered as unchanged, and then the `ice_update()` drops these candidates as duplicates.
Therefore no connectivity checks run and then the endpoint stays at the old non-iced address.
Media gets bound to the wrong port.
Shutdown the ice agent whenever the peer's own sdp no longer carries ice,
or we get the `ICE=remove`.
Then the offered ice again later starts from scratch.
Change-Id: I5ff3b6454262099022bf4f61c84b978835130dcc
Copy create-answer and subscribe-answer SDP into call-owned storage before parsing, matching offer/answer and publish. Retained stream parameters otherwise reference released NG request buffers and later subscriptions can lose their media type and codecs.
Initialize created-answer monologue session state so generated subscriptions retain a valid SDP origin and session name. Add request-buffer-churn and decoded-audio regressions for created and subscribed endpoints.
Validation on standalone Sipwise master: daemon build, all 89 retained-SDP assertions, all 652 create assertions, and all 932 publish/subscribe assertions pass. The cumulative 26.2.1 release also has isolated browser Opus decoding, privacy, recording, and mixer regression coverage.
Closes#2194
Change-Id: Id8ca0e0ff691356d905ee94c0f41090f973fc70f
Put everything into a dedicated struct
Split out code into separate file
Introduce files for code shared between daemon and kernel module
Update return type to bool
Change-Id: I2cd78dcdef3490bf00eb7e27474303f14cfb1900
Update media player, send timer, jitter buffer objects.
Set final memory arena pointer as the releasing the last reference might
reset it to null.
Exclude media_player stuff from no-transcoding builds and adjust ifdef
guards accordingly.
Closes#2186
Change-Id: I42dc5a449816d765bc4d1da2b35315e0b6f7c29b
Share SSRC dump and restore helpers between ingress and egress, keeping
ssrc_out_table as the outbound list name. Scope each hash lock to its
dump rather than holding both locks throughout media serialization.
Restore records without an outbound list through the original combined
in/out path, including records written before the SSRC hash split.
Adapt the takeover regression test from kbinetruy's PR #2188 to exercise
native and JSON records, sequence rollover, encryption after restart,
and both legacy record layouts. Follow the existing Redis helper,
initializer, iteration, and daemon-test patterns.
Ref #2185
Change-Id: I0ac7e6b351fd3e2bcab38d0318d1787e5332c442
Co-authored-by: Kevin Binetruy <kevin.binetruy@ovhcloud.com>
Since the SSRC contexts were split into separate ingress and egress
entries (cf22d829), only the ingress entries are written to the SSRC
table, and the lines for the outbound index are commented out. The
restore still looks for the out_* keys in the ingress entries, where
they no longer are.
A node that takes over a call therefore starts encrypting from index 0
again. A receiver estimates the ROC from its own state, so if the
sequence number of the call has already wrapped, it rejects every
packet after the takeover.
Write the egress entries to a list of their own, ssrc_out_table, and
restore them from there. Records without the list restore as before.
Fixes#2183Closes#2185
Change-Id: Id28dc86c658f7e94dd87443fc0f09225beaf484e
After a takeover, FOREIGN_MEDIA holds off the media timeout until the
call receives media. The flag is cleared by the first packet that
arrives on any of the call's sockets, before the packet is checked, so
a packet that is then dropped, for example because of strict-source,
ends the grace period without refreshing the stream's last packet time.
A call that was restored longer than the timeout ago is then closed on
the next timer run, before its real media has had a chance to arrive.
With the peer address confirmation now restored from Redis, this is
what happens to an adopted strict-source call when a stray packet
arrives ahead of the peer's first one.
Clear the flag only once a packet has passed the address check. The
timer still clears it for kernel-forwarded media, and the grace period
stays bounded by the timeout after the takeover.
Ref #2184
Change-Id: Ib06a170d271e8aea709abc57cbfd3bcc59196ab5
A call restored from Redis starts with every sfd unconfirmed and every
stream's el_flags at zero, which is EL_DELAYED, regardless of what the
offer asked for or what --endpoint-learning is set to. After a
keyspace-notification failover, all adopted calls therefore re-learn
their peer at the same time, and strict-source has nothing to compare
against until they have.
Write the sfd's confirmed flag and the stream's el_flags and
learned_endpoint to the record, and read them back on restore. The two
stream fields were so far only part of the rollback snapshot. Move them
into the shared stream encoder and decoder, so that both use the same
definition. Records without the new keys restore as before.
Fixes#2182Closes#2184
Change-Id: I6228e55796e7a680b05797df985573e8ab54c653