fix: a claimed gap must be corroborated before it is believed, and its hole must fit

The user's real host reported "connected and live" with one missing-data band over the whole window
and no trace in it. The counter is the only thing that claims a packet existed, and a reservation
was booked on its word alone: a source whose DATA counter counts samples, or frames of an inner
layer, advances by more than 1 per datagram, the client cannot check that claim (it has no clock of
its own to compare against), so it holds each packet, saturates its reorder ring, delivers it in
order and reports the skipped counters as lost. Every frame therefore reported a phantom gap, and
`reserve_hole()` wrote `lost * batch_` NaN samples with no bound but the 256-PACKET cap -- the
sample form of that rule ("the number to weigh against the ring is 256 * batch_, not the 256") was
never implemented. With relative time on -- which that session carries -- each phantom gap is
materialised as NaN slots by construction: the window filled with reservations, there was never
anything missing to repair them, and the gaps also stretched the relative clock, which is why the
status bar read live at ten times the true rate.

`Acquisition::gap_plausible()` now gates every reservation on two independent tests: the arrival
clock must corroborate the claim (that many missing packets would take that many frame intervals;
deliberately permissive, because a false refusal costs a shading and never data), and the ring must
be able to hold the hole. The verdict is taken before the stamps, and relative mode's slot advance
is skipped with it, since the advance and the reservation are the same claim. Refusals are counted
in `StreamStats::gaps_refused` and said once per connection with the numbers, naming the wire shape
they describe.

tools/mock_udpstreamer.py gains `--counter-step N`: a counter that is not a packet counter on an
intact link, which is the one shape the mock could not produce and the shape the report came from.

Verified in both directions. A counter stepping 1000 per datagram reads `holes=0 hole_slots=0
backfilled=0` with the traces present (3 traces, 1.4 M samples) against 7.9 M claimed gaps, and the
warning above names the cause. Every other packet withheld -- a real loss -- still reads
`holes=3579`, so genuine gaps are still reserved: a fix that stopped believing real losses would
have traded one wrong picture for another. The user's control read as "the math channel is the
problem" and was confounded -- disabling it is a structural change, which rebuilds the buffer and
clears the accumulated NaN.
This commit is contained in:
Martino Ferrari committed 2026-10-15 08:13:28 +02:00
1 parent 22e7a8efcc
commit 27c9b34277
9 files changed
+249 -15

No files matched your search

+30 -1
View File
@@ -169,7 +169,36 @@ Part of [CLAUDE.md](../../CLAUDE.md). These are rules that cost a defect to lear
(`kGlitchPackets`, 256) is counted in **packets, not samples**, so one hole's sample cost
is `256 × batch_` and scales with the stream's batch — 25 600 samples at a batch of 100,
2.56 M at a batch of 10 000 — which is the number to weigh against the ring, not the 256.
- **A re-plan drops every hole and restarts the producer's tick.** `rebuild_locked()` replaces
- **A reservation is weighed against the wire, and never booked on the counter's word alone.**
The counter is a *claim* about the stream; the arrival clock is a *measurement* of it, and only
the two together justify writing NaN into the ring (`Acquisition::gap_plausible()`, called
before anything is booked *and* before relative mode's `slot_base_` advance, because the advance
and the reservation are the same claim). Two tests, either one refusing is enough:
- **The arrival clock.** `missing` packets would have taken `missing` frame intervals to arrive,
so a gap whose elapsed arrival time is under half of that is refused. Half and not one because
the socket clock jitters and a drain can compress two frames' arrival times -- and a refused
gap costs a *shading*, never data, so the test is deliberately permissive.
- **The ring.** `missing × batch_` samples become `missing × batch_ / time_stride` slots, and the
hole may not lap `time_capacity`. This is the *sample* form of the rule the 256-packet cap
states in packets -- the cap's own comment says 256 packet spans in one go is an outage, and
the invariant above says the number to weigh against the ring is `256 × batch_` and not the
256 -- and it has to exist in samples because a hole that laps the ring erases the history it
is meant to annotate, can never be backfilled (its slots are already overwritten) and is never
forgotten (`forget_holes_before()` waits for slots the ring has passed), so it leaves a
*permanent* band over a buffer that is all NaN.
What its absence looks like, which is why it is written down: a device whose counter counts
*samples*, or frames of an inner layer, reports a forward jump on every datagram while the wire
delivers on time. Every frame then reserves, and with `relative_time` every frame books NaN
slots by construction -- one band over the whole window with the samples washed out,
`backfilled=0` (no packet was ever missing, so nothing can repair it), and the status bar reading
`live` at the right rate the whole time. The refusal is counted in `StreamStats::gaps_refused`
and said once per connection with the numbers, because the status bar's `gaps` climbs either way
and the state is otherwise invisible; `tools/mock_udpstreamer.py --counter-step N` is the mock
shape (its counter is otherwise always +1, so this is the one wire shape it could not produce),
and `gaps_refused` rising while `nan_slots` stays 0 is the reading. A refused gap is not a silent
one: the timeline is not stretched over loss that did not happen, so it shows as the x
discontinuity it is.
- **A re-plan drops every hole and restarts the producer's tick.** `rebuild_locked()` replaces
the store, so `produced()` restarts at 0 and a hole's `slot_begin`/`tick_begin`/`x_begin`
describe a buffer that is gone; it therefore clears `holes_` and bumps
`ScopeModel::plan_generation()`, which the producer compares against the value it last saw