Ref #2211
When json_restore_call() fails, it sends "DEL <call-id>" twice: on the
write connection, in its own DB, and on the notification connection, in
whatever DB the last keyspace notification selected. For a foreign call
(a keyspace notification, or the restore of a subscribed keyspace at
startup) that key belongs to the instance that owns the call, which
keeps writing it: a standby that fails to restore one record deletes the
owner's copy, and the call cannot be taken over any more. For an own
call, the DEL on the notification connection can hit a peer's DB.
Only delete an unrestorable record when the call is our own, and only on
the write connection, in our own DB. A foreign record that can never be
restored now stays until its owner rewrites or deletes it, or until it
expires (--redis-expires). Seen in a lab with Redis Sentinel, where a
subscriber's DEL was refused only because its node had just become a
replica.
Tests, in auto-daemon-tests-redis-subscribe.pl: a keyspace notification
for a foreign call whose record cannot be parsed is followed by no
command on either connection, and an own record that cannot be parsed at
startup is deleted once, on the write connection only. Without this
change both get a DEL on both connections.
Change-Id: I5d8a53f3515596cf3fc2c58ea97847476385b876
(cherry picked from commit 6b3bf8299b)
Ref #2211
redis_set_timeout() receives the timeout in microseconds (the
--redis-cmd-timeout value times 1000) and logged it unconverted, so
--redis-cmd-timeout=1000 was reported as:
Setting timeout for Redis commands to 1000000 milliseconds
Log the value in milliseconds, as the message says. The value passed to
hiredis is unchanged.
No functional change. Seen in the log of every connection made with a
command timeout set; there is no test, as no test reads this line.
Change-Id: Ie4d81e4f8baab1969210a589c98245407b088561
(cherry picked from commit 1c447b509b)
Ref #2211
redis_connect() finds "role:" in the INFO reply and compares what
follows with memcmp(): 11 bytes for "role:master", 19 for
"role:active-replica" and 10 for "role:slave". memcmp() may read all n
bytes, so a reply that ends less than 19 bytes after "role:" (for
example a replica's "role:slave\r\n" with nothing after it) is read
past its end. Real Redis and KeyDB servers send many more lines after
the role, so this does not show in practice, but AddressSanitizer
aborts on it (heap-buffer-overflow in the memcmp interceptor) with a
minimal reply, such as one from a test server or a proxy.
Use strncmp(): hiredis NUL-terminates the reply string, so the compare
stops at its end.
Tested by auto-daemon-tests-redis-errors.pl, added later in this
series, which answers a reconnect's INFO with just "role:slave": an
ASan build aborts there without this change. The change is early in
the series so that it can be taken on its own.
Change-Id: Icdf3d3a23464d861f57406436328d0b166794362
(cherry picked from commit e5ff3ea8af)
Ref #2211
"set rediscmdtimeout 0" reconnects the Redis connections, but each of
the three calls is guarded by rtpe_redis instead of its own pointer:
- with --redis and no notification connection (no --redis-subscribe and
no subscribed keyspace), redis_reconnect(rtpe_redis_notify) locks a
NULL mutex and rtpengine crashes;
- with --redis-write and no --redis, the write connection is never
reconnected, so its command timeout is never cleared.
Guard each call with its own pointer, as the non-zero branch already
does.
Tested by auto-daemon-tests-redis-errors.pl, added later in this
series ("set rediscmdtimeout 0" over the ng "cli" command with --redis
only): without this change rtpengine crashes and never replies. The
change is first in the series so that it can be taken on its own.
Change-Id: Ia458a6eb0b70a6906ab410fd91087ad5af581f3b
(cherry picked from commit 20bcdce73c)
monologue_publish() never copied the offered label into the media,
so sdp_create() omitted a=label from the publish answer and from
subscribe requests for that publisher. Add the missing
media_update_label() call.
Closes#2200
Change-Id: Id9aaf4ab09c21f98d3dcaf78632413b99f4a97ff
(cherry picked from commit b66c59bed6)
The kernel tracks lost packets per SSRC through the `lost_bits` window
in the shared ssrc_stats, which was left zeroed, while userspace starts
its own window as all-received. As userspace already advances `ext_seq`
before a stream is kernelised, the kernel then counted up to 32 packets
it did receive as lost, both per SSRC and in the interface and global
packets_lost counters. Start the shared window as all-received too.
Change-Id: I3d6cd85a1b764a6bb3296daf7489091767449554
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 3b73432826)
ssrc_collect_metrics() converted the running RFC 3550 jitter estimate
to milliseconds in place on every call timer run with --measure-rtp,
dividing it by clock_rate/1000 each time. The estimator is kept in
RTP timestamp units and also feeds generated RTCP RRs, which therefore
reported a jitter decaying towards zero. Convert a copy instead.
The RTCP test expected the resulting zero jitter; it now gets the
value actually measured from its back-to-back packets.
Change-Id: Ice29029322764fd76374eb3f7dda517570f7f75b
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 4dd1826ffa)
Copy create-answer and subscribe-answer SDP into call-owned storage before parsing, matching offer/answer and publish. Retained stream parameters otherwise reference released NG request buffers and later subscriptions can lose their media type and codecs.
Initialize created-answer monologue session state so generated subscriptions retain a valid SDP origin and session name. Add request-buffer-churn and decoded-audio regressions for created and subscribed endpoints.
Validation on standalone Sipwise master: daemon build, all 89 retained-SDP assertions, all 652 create assertions, and all 932 publish/subscribe assertions pass. The cumulative 26.2.1 release also has isolated browser Opus decoding, privacy, recording, and mixer regression coverage.
Closes#2194
Change-Id: Id8ca0e0ff691356d905ee94c0f41090f973fc70f
(cherry picked from commit 12ecdbc2cf)
Update media player, send timer, jitter buffer objects.
Set final memory arena pointer as the releasing the last reference might
reset it to null.
Exclude media_player stuff from no-transcoding builds and adjust ifdef
guards accordingly.
Closes#2186
Change-Id: I42dc5a449816d765bc4d1da2b35315e0b6f7c29b
(cherry picked from commit 2366392202)
Share SSRC dump and restore helpers between ingress and egress, keeping
ssrc_out_table as the outbound list name. Scope each hash lock to its
dump rather than holding both locks throughout media serialization.
Restore records without an outbound list through the original combined
in/out path, including records written before the SSRC hash split.
Adapt the takeover regression test from kbinetruy's PR #2188 to exercise
native and JSON records, sequence rollover, encryption after restart,
and both legacy record layouts. Follow the existing Redis helper,
initializer, iteration, and daemon-test patterns.
Ref #2185
Change-Id: I0ac7e6b351fd3e2bcab38d0318d1787e5332c442
Co-authored-by: Kevin Binetruy <kevin.binetruy@ovhcloud.com>
(cherry picked from commit afb98f00d3)
Since the SSRC contexts were split into separate ingress and egress
entries (cf22d829), only the ingress entries are written to the SSRC
table, and the lines for the outbound index are commented out. The
restore still looks for the out_* keys in the ingress entries, where
they no longer are.
A node that takes over a call therefore starts encrypting from index 0
again. A receiver estimates the ROC from its own state, so if the
sequence number of the call has already wrapped, it rejects every
packet after the takeover.
Write the egress entries to a list of their own, ssrc_out_table, and
restore them from there. Records without the list restore as before.
Fixes#2183Closes#2185
Change-Id: Id28dc86c658f7e94dd87443fc0f09225beaf484e
(cherry picked from commit ef16e3f1e7)
After a takeover, FOREIGN_MEDIA holds off the media timeout until the
call receives media. The flag is cleared by the first packet that
arrives on any of the call's sockets, before the packet is checked, so
a packet that is then dropped, for example because of strict-source,
ends the grace period without refreshing the stream's last packet time.
A call that was restored longer than the timeout ago is then closed on
the next timer run, before its real media has had a chance to arrive.
With the peer address confirmation now restored from Redis, this is
what happens to an adopted strict-source call when a stray packet
arrives ahead of the peer's first one.
Clear the flag only once a packet has passed the address check. The
timer still clears it for kernel-forwarded media, and the grace period
stays bounded by the timeout after the takeover.
Ref #2184
Change-Id: Ib06a170d271e8aea709abc57cbfd3bcc59196ab5
(cherry picked from commit 1d783ba1b9)
Both mean that the ice towards the offerer gets rejected.
When a certain peer who previously offered ice, sends SDP without ice
(e.g. T.38 re-invite), or the ice was requested to be removed
(when processing answer) with `ICE=remove` then the ice agent is left alive
with its learned ufrag, candidates and succeeded pairs.
The stream endpoint is then taken from that SDP (non-ice path).
If this peer later offers ice again with the same credentials (e.g.
falling back to audio after an error response like 488 for that T.38),
then this stale agent is reused:
`__fill_stream()` ignores the new endpoint because the ice data
considered as unchanged, and then the `ice_update()` drops these candidates as duplicates.
Therefore no connectivity checks run and then the endpoint stays at the old non-iced address.
Media gets bound to the wrong port.
Shutdown the ice agent whenever the peer's own sdp no longer carries ice,
or we get the `ICE=remove`.
Then the offered ice again later starts from scratch.
Change-Id: I5ff3b6454262099022bf4f61c84b978835130dcc
(cherry picked from commit 39fd49b6c4)
Forcing the DTLS role to passive only makes sense for the backwards
(torwards the offerer) direction anyway, as towards the answerer (in an
offer) it's not allowed by the RFC, and in an answer it's not possible
as the active DTLS connection would already be in progress.
Convert DTLS= processing into a list as multiple flags might be
required.
Change-Id: I54239280460e1bbce1966bc17a384489eb978618
Amends: 78ee0451 / Ib1d07831
Related: #2156
Related: #2145
(cherry picked from commit 032bd398b9)
Using the FILLED flag to decide how to resolve actpass is not reliable,
as during a reinvite FILLED would be set for both stream directions.
Instead, immediately clear the ACTIVE flag during the offer phase
(dtls_logic) and assume a strict passive role. Then if the answer
indicates that the connection needs to be active instead, a reset occurs
via dtls_connection_init.
Closes#2156
Change-Id: I0ded94620bcaa481eea920ae0c412c7bb86bff1f
(cherry picked from commit 91d0669499)
DTLS connections created during an offer are passive by default. If the
answer requires a role change to active, don't reset the TLS ID, as we
have no way to communicate this change until a re-invite, which would
then lead to an unexpected reset of the connection.
Related: #2156
Change-Id: I49dcd93d63840a223c41a59c74e2f270d0a08f62
(cherry picked from commit 001d8fe4c6)
The primary pair selection already checks this, but a retransmit can
still trigger a check on a socket that has since been closed.
Closes#2154
Change-Id: I8308693e7269bfa9dcfcb24daf6863669c065139
(cherry picked from commit c22d724aa9)
If a SSRC object is still in use (refcount is >1), keep it in the list,
even if the list is too large.
Otherwise we may end up recreating a new SSRC object for the same SSRC
that still has an object in use elsewhere, making locks unreliable.
Ref #2148
Change-Id: I2afd277696e05c356fc9cf5aea59f51e1fba0784
To not come across the sink's mutation (e.g. `endpoint.address.family`)
just lock way earlier, before to start processing and
translating endpoint to the kernel related structs.
Otherwise the other signaling path may mutate something
which will make the sink unactual and provide
to the `__re_address_translate_ep()` NULLed or garbage structs.
Change-Id: I88d87ae88abe4e37aaa5d3cdda76ed0ca82e4842
16de9688d6 resets the `socket.family` to NULL,
that makes the direct appeal to the corresponding
object unsafe.
E.g. `ps->selected_sfd->socket.family->name` when
filling the streams.
Add a simply guard and log `none` in case
the ip family is NULL.
Change-Id: I7ad7399e32e5d0ffb2fcc919822d05760fdc7421
If the dumper was not opened, then also take care
to clear the recording path, because later
the `response_pcap()` only checks the recording path
itself and not wehther the dumper was actually opened.
Hence NG would report a recording file, which indeed
was never even created.
For consistency reasons add the dumper's guard
in the `response_pcap()`.
Change-Id: Ifc1d5b73785633ffcc6293d64dbfbdea51b54582
The `finish_proc()` misses part of the clean-up
work if the kernel table isn't open.
Only `kernel_del_call()` really depends
on the `kernel.is_open`, local cleanup should still happen.
Change-Id: I8f999dbc48f7dd04b4ca1ba5edbc9218a3c49d29