333 lines
25 KiB
Markdown
333 lines
25 KiB
Markdown
# Verification
|
|
|
|
How to run the headless self-check, the two baselines every run is read against, and the environment variables that schedule each probe.
|
|
|
|
Part of [CLAUDE.md](../../CLAUDE.md).
|
|
|
|
There is no unit test suite. The only automated check is the app's own headless
|
|
self-check, driven by env vars and run under a display (`xvfb-run`, llvmpipe) against
|
|
a live UDPStreamer source:
|
|
|
|
```sh
|
|
xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=8000 UDPSCOPE_ENABLE_ALL_MS=1000 \
|
|
./build/udpscope --server 127.0.0.1 --port 8000
|
|
```
|
|
|
|
Two baselines every run is read against. A clean build emits **four** warnings and no more: two
|
|
`-Wmissing-field-initializers` on `MathNode::source` and `-Woverlength-strings` on **both**
|
|
generated GResource `.c` files — the help pages' and the icons', whose blob crossed C99's
|
|
4095-byte string-literal limit when the two app icons joined the four action ones (a note about a
|
|
generated file, which GCC itself supports). The fifth warning that used to appear here was an unused
|
|
`output_built` in `math_probe()`; it was **deleted** as dead code — redundant with the
|
|
`output_after_add` already printed on the same line — and that deletion is what dropped the baseline
|
|
from five to four. Nothing started reading it. And an exit-dump run emits **no** `Gtk-CRITICAL` at
|
|
all. A fifth warning, or any critical, is a finding.
|
|
|
|
An *incremental* build reads **two** of those four: the generated GResource `.c` files are not
|
|
recompiled when only `src/` changed, so their `-Woverlength-strings` do not reappear.
|
|
|
|
That second baseline used to be "exactly two, `gtk_widget_get_height` and `gtk_widget_get_width`,
|
|
from the dump asking a destroyed widget for its size", and tolerating them was a mistake: the dump
|
|
runs from the destroy handler, so the widget pointer is **freed memory**, and a critical is only
|
|
the lucky reading of it. A run that had built and destroyed other windows first —
|
|
`UDPSCOPE_PROBE_HELP_MS`, which opens three — has had that memory reused, and the same two calls
|
|
then **SIGSEGV in the middle of the dump**, which reads as a truncated dump rather than as a
|
|
crash in a debug line. Every size the dump prints is therefore captured on the *tick*, while the
|
|
window is alive (`App::dbg_*`), and **no `gtk_widget_*` call may appear in the dump** — the
|
|
`panel controls:` line was already written that way for the same reason, and that comment is the
|
|
precedent, not an exception.
|
|
|
|
- `UDPSCOPE_EXIT_AFTER_MS` — run the trigger-on → go-live probe, close the window,
|
|
then dump the whole pipeline state to stdout (frame/sample counts, per-plot painted
|
|
frames, trace bindings, panel sizes, status line in both states).
|
|
- `UDPSCOPE_ENABLE_ALL_MS`, `UDPSCOPE_ARM_TRIGGER_MS`, `UDPSCOPE_PROBE_DISARM_MS`,
|
|
`UDPSCOPE_TRIG_TRACE_MS` — schedule individual probes. `UDPSCOPE_PROBE_DISARM_MS` only
|
|
*starts* its sequence: it switches the trigger on and then polls for a capture to open,
|
|
so the Disarm lands ~300 ms into one rather than wherever a fixed delay happened to fall.
|
|
- `UDPSCOPE_PROBE_TRIGRENUM_MS` — the trigger's capture must survive a **renumbering of its own
|
|
channel**. It sets the reported geometry through the real widgets (1 s window, 10 % pre share,
|
|
hold-off from `--holdoff`) on the mock's *square* alone, polls for a capture to open, and then
|
|
enables the sine — which sorts before the square and so moves the trigger channel's index under
|
|
the engine. Its three lines are in [probe-readings.md](probe-readings.md); **`t0_held=1` is the
|
|
assertion**. Run it with `--holdoff 10`, which makes the stolen-t0 reading unmistakable: a second
|
|
latch barely a second after the first cannot be a hold-off that applied.
|
|
- `UDPSCOPE_PROBE_FIELDS_MS` — change the trigger panel's edge and Normal/Single
|
|
dropdowns. As much a crash test as a state check (see the GTK signal-signature invariant
|
|
below), so time it inside a capture: `[fields-probe] ... capturing=1` followed by no
|
|
SIGSEGV *is* the assertion.
|
|
- `UDPSCOPE_PROBE_WINDOW_MS` — drive the window control every way a user can: type a
|
|
number, change the unit under it, step a decade either way.
|
|
- `UDPSCOPE_PROBE_LAYOUT_MS` — drive the layout editor: switch it on, insert above the
|
|
first row, remove a row, append one at the bottom, switch it off. Its steps are 200 ms
|
|
apart on purpose — the buttons are placed from the rects the *last frame* was laid out
|
|
with, so a synchronous sequence would read every position one action stale. It clicks by
|
|
emitting `::clicked`, never `gtk_widget_activate()`, which holds a `GtkButton` for its
|
|
250 ms `ACTIVATE_TIMEOUT` and then emits into a button the rebuild has already destroyed.
|
|
- `[measure-probe] N measurements on 'X'` (stderr, `UDPSCOPE_PROBE_MEASURE_MS`) — adds one
|
|
measurement of each statistical type (average, top, peak to peak, amplitude) on the first
|
|
*channel* and forces a recompute. It must name a **channel**, not `signals().front()`: the
|
|
first signal of a stream is its timestamp, which is not acquired, and naming it adds four
|
|
measurements that all read `--` *and* makes a performance run look free (each bails before
|
|
collecting anything -- that is how the first baseline of the stall below read 0.4 ms).
|
|
The values are in the exit dump as `measurements: <type>=<value>`; on the fast mock
|
|
(`tools/mock_udpstreamer.py --rate 1300000 --batch 2000`) a unit sine must read
|
|
`peak to peak=2`, `amplitude=1`, `average=0` and **`top=0.636620`** -- 2/pi, the half-wave
|
|
mean of a sine, which is the check that the threshold types are not merely plausible.
|
|
`[measure-trace] view=[..] <type>=<value>... marks=N level:V/time:V...` is a repeating line
|
|
(250 ms) from the probe: the pane's numbers beside the *current* window and the marks actually
|
|
placed, which is the only way a stale result or a mark that should not be drawn shows up as a
|
|
number. It must never print `marks=0` while measurements with `show_on_plot` are on and the
|
|
data is there -- that was the rulers blinking off at a mode change.
|
|
`measure cost: N recomputes, X ms total, Y ms each, data D ms (P%), covered=C buckets=B
|
|
raw=R (F%), N ns/covered sample` follows it: **read it against a run with
|
|
`UDPSCOPE_PROBE_MEASURE_MS` unset**, because a frame mean cannot attribute a cost. The last
|
|
three fields are the assertion, and they are a *ratio*, not a time: `covered` is the samples
|
|
the window spans, `buckets` the LOD buckets the two walks answered from, and `raw` the samples
|
|
they still had to read. **`raw` must stay at a fraction of a percent of `covered`** on a clean
|
|
waveform over a window of millions -- 0.1% is what the pyramid-backed walks give (see the
|
|
measurement-cost invariant in
|
|
[invariants-measurements.md](invariants-measurements.md)). A `raw` near `covered` means the walks went back to reading
|
|
the window, which at 1 MS/s is 19.7 ms per recompute and a third of the run's frames. It is
|
|
*legitimately* higher for a window shorter than a bucket (64 samples) or a signal noisy
|
|
enough to cross a threshold everywhere, so read it on the fast mock
|
|
(`tools/mock_udpstreamer.py --rate 1300000 --batch 2000`), where a window is ~2 M samples.
|
|
`--noise 0.1` there reads **2.0%** at 1.96 ms a recompute against the clean 0.1% at 1.41 ms, and
|
|
that is the interferer's own cost rather than a regression -- the figure to compare a change
|
|
against is the **clean** one.
|
|
Note the **final** `measurements:` line and `[measure-trace]`'s last lines may read `--`:
|
|
`UDPSCOPE_EXIT_AFTER_MS` arms the trigger before it dumps, so the dump can land in `Waiting`
|
|
with nothing bound and the axis at its default. The race-free reading is an earlier
|
|
`[measure-trace]` line, which carries its own `view=`; those lines are also where the
|
|
level-mark fallback is visible (an invalid pass must still print its `level:` marks).
|
|
|
|
- `[pan-probe] asked=[..] was=[..]`, `[pan-probe] after=[..] held=D asked=[..]`, then
|
|
`[pan-probe] zoom asked=[..] after=[..] span=S window=W limited=L`
|
|
(stderr, `UDPSCOPE_PROBE_PAN_MS`) — **`held` must be 1**: a gesture must not move a window
|
|
that is pinned (trigger mode, or a paused view). The probe writes the panned range the way a
|
|
gesture does — on the *member* axis, which `Figure::layout()` adopts into the group — and
|
|
reports the axis the frame after it produced. Time it *inside* one pinned state: a pan during
|
|
a capture whose check lands after the capture has ended is re-framed by the mode change, not
|
|
by the pin, and reads `held=1` with the hold removed (that is how the first version of this
|
|
probe fooled me). The pause probe makes a stable pin, so run them together:
|
|
`UDPSCOPE_PROBE_PAUSE_MS=1500 UDPSCOPE_PROBE_PAN_MS=2200` puts the pan and its check inside
|
|
the pause. With `hold_pinned_window()` disabled it reads `held=0` and the axis sits at the
|
|
asked range. It then asks for a **zoom-in** (a window a quarter of what is on screen) and
|
|
reports `group=`/`member=` beside it: **`allowed` must be 1, with both equal to the ask** --
|
|
that is the single-trigger-mode case, where the wheel has to keep working, and where reading
|
|
the wrong one of the two made a working clamp look broken. All three verdicts are read from the
|
|
*tick*, one frame after the ask: a `g_timeout` was the first version and it drifted past the
|
|
paused/resumed boundary under load. The probe then asks for a **zoom-out** ten times the current span and reports
|
|
the span the next frame produced: **`limited` must be 1**, i.e. `span <= window`. Run it with
|
|
a ring that holds *more* than the window (`--buffer-mb 64` past the window's own seconds) —
|
|
that is the state the old `max(retained, window_s_)` bound leaked in, and it showed up here as
|
|
`span=2.502550 window=2.000000 limited=0` against `span=2.000000` after the fix.
|
|
|
|
- `[zoom-probe] <sweep>/x<step> axis=[..] span=.. n=N idx=[a..b) spp=.. level=L pts=P kept=K
|
|
back=.. ink=[..] x_at_first=.. gap_px=.. end_px=.. rect=[..]` (stderr,
|
|
`UDPSCOPE_PROBE_EVENTZOOM_MS`) — arms the trigger in **single** mode, polls for a **held
|
|
event** (a latch happens when the stream says so; a sweep on a timer has nothing to zoom
|
|
into), then zooms in eight doublings
|
|
twice: `ges/` writes the row's own axis the way a gesture does, `win/` drives the window
|
|
control (`apply_time_window`, which unpins and re-arms the live follow before the pin is
|
|
restated). Each step's reading is taken in the *tick*, after that frame's pin and clamp — a
|
|
`g_timeout` would report the ask. It recomputes what `Renderer::draw_traces` is handed, through
|
|
the same calls: `neighbours()` for `idx`, the 64-spp gate for `level`, then `gather()`.
|
|
**`gap_px` and `end_px` must stay within a pixel or two at every step** — they are how much of
|
|
the left and right margin no geometry reaches, and a `gap_px` that *doubles* per step is the
|
|
"zooming into a held event clips it more and more from the left" defect (374 px of a 716 px
|
|
axis by `ges/x128`), whose cause is a constant offset in data seconds (see the `neighbours()`
|
|
invariant in [invariants-rendering.md](invariants-rendering.md)). `kept`/`back` are the renderer's own increasing-suffix filter counted rather
|
|
than applied: `kept` far below `pts` is a staircase, and a non-zero `back` is a non-monotonic
|
|
gather. **Run it with `--clock-jitter`** (`tools/mock_udpstreamer.py --rate 1300000 --batch
|
|
2000 --clock-jitter 0.4`): the ideal mock's stamps sit on a perfect grid, which is exactly
|
|
what hides this class of bug, and three runs against it reported `gap_px=0.0` throughout while
|
|
the defect was live. The jitter varies each *packet's* spacing rather than offsetting its
|
|
start, because the stream's clock has to stay strictly increasing. The `win/` sweep bottoms
|
|
out at `span=0.05` (`RowSet::set_time_window()`'s floor) and walks to the event's post side,
|
|
since `set_time_window()` frames the newest span before the pin clamps it — both are expected.
|
|
- `UDPSCOPE_PROBE_PAUSE_MS` — pause acquisition mid-run so layout numbers are
|
|
measured on a still picture. It now also asserts what a pause *is*
|
|
(`[pause-probe] <action> paused=D ring_produced=N event_size=M frozen_x=[..] axis=[..]
|
|
glance=[..]` on stderr):
|
|
it presses the **button**, not `Acquisition::set_paused()`, because pausing is a display
|
|
state as well as an acquisition one and the view snapshot and axis pin live in the
|
|
handler. Across the three lines **`ring_produced` must grow** (the producer never
|
|
stops — it fills the live ring exactly as it does with the trigger off), **`event_size`
|
|
must be non-zero exactly while paused** (the plot is bound to the frozen snapshot, not
|
|
the ring) and **`frozen_x` must not move between the pause and the resume**; after the
|
|
resume `event_size` is 0 and the window is back on the newest data. Pausing used to stop
|
|
the producer, which is why the counts grew by nothing and the resume left a hole.
|
|
`frozen_x` is the *rebased* snapshot — the pause view's own stamps are relative to the freeze
|
|
instant now, so the range reads `[-window..0]`-ish rather than stream seconds, and a
|
|
`frozen_x` in the thousands is the rebase gone.
|
|
**`glance=` is the assertion for the paused status field**, and this is the only line that can
|
|
make it: the exit dump's `status bar:` is read live and its two `mode …` lines are the live and
|
|
triggered states. While paused it must read `paused · <rate> · <window> · for <elapsed>`, with
|
|
the elapsed figure **growing between the two paused lines** and **absent on the resume line**.
|
|
It is **wall** time (`g_get_monotonic_time()`), not stream time, because the producer never
|
|
stops while paused and "how long have I been looking at this" is a question about the user, not
|
|
about the stream; it is written through `format_elapsed()` so the field steps to a coarser unit
|
|
instead of growing digits (see the layout rule).
|
|
- `UDPSCOPE_PROBE_COLOR_MS` — recolour one plotted signal and read the colour back out of the
|
|
trace and the chip, round-trip it through the session writer, then clear it. Schedule
|
|
`UDPSCOPE_ENABLE_ALL_MS` before it: with nothing bound it has no trace to recolour and asserts
|
|
nothing. Its four lines are in
|
|
[probe-readings.md](probe-readings.md).
|
|
- `UDPSCOPE_EXPORT_MS` (+ optional `UDPSCOPE_EXPORT_PATH`, default
|
|
`/tmp/udpscope-export`) — collect the export table and write it both ways, skipping
|
|
only the file chooser.
|
|
|
|
- `UDPSCOPE_PROBE_CONDITION_MS` — arms a qualified trigger against the mock and reports the latched
|
|
`t0`, the measured pulse or band width, and sample values for the window condition. Schedules its
|
|
own arm and report stages. `UDPSCOPE_PROBE_CONDITION_SHAPE` picks the leg: `0` pulse width `<`
|
|
(the glitch, the default, needs `--glitch-at`/`--glitch-width`), `1` pulse width `>` (uses a
|
|
1.0 s window and 50 % pre so the opening edge of a 0.2 s pulse is captured), `2` window
|
|
(uses `sine` with a 0.2 s window and 10 % pre), `3` dropout (needs `--quiet-at`; uses `sine`
|
|
with a 1.0 s window and 50 % pre so the last crossing before the outage is captured).
|
|
- `UDPSCOPE_PROBE_STATS_MS` — reports every measurement's accumulator (`n`, mean, sd, lo, hi),
|
|
pauses, reports again with `dn=` per measurement, then resets and reports a third time. Three
|
|
phases: `phase=0`, `phase=1` (after pause) and `phase=2` (after Reset). Needs
|
|
`UDPSCOPE_ENABLE_ALL_MS` ahead of it and `UDPSCOPE_PROBE_MEASURE_MS` to add measurements first.
|
|
- `UDPSCOPE_PROBE_SEARCH_MS` — runs a buffer search for the armed condition, cross-checks the live
|
|
trigger's own `t0` against the nearest hit (`agree=1` is the load-bearing reading), and jumps to
|
|
hit 0. Reports the resulting `mode=`, `axis=` and `marks placed=`.
|
|
|
|
The following mock waveform features are what the qualified-condition probes are checked against,
|
|
as distinct from the fault injection below, which exercises ingest:
|
|
|
|
- `--glitch-at S` / `--glitch-width W` invert the `square` signal for `W` seconds at the start of
|
|
every square period from stream second `S` onwards — a runt pulse of a known width, which is
|
|
what a `<` pulse-width trigger and a glitch search are checked against.
|
|
- `--quiet-at S` freezes `sine` at its held value from stream second `S` onwards, so it stops
|
|
crossing zero: the dropout trigger's input.
|
|
|
|
- `UDPSCOPE_PROBE_RESEQ_MS` — one mid-run reading of the resequencing path. Four cases,
|
|
**run each as its own process** — four back to back starve the mock past its 2 s silence
|
|
timeout and every case then reads a reconnect instead of what it was testing. Common
|
|
conditions for all four: the mock at `--rate 10000 --batch 100 --freq 5`, udpscope with
|
|
no extra options, under `xvfb-run` with
|
|
`UDPSCOPE_EXIT_AFTER_MS=8000 UDPSCOPE_ENABLE_ALL_MS=600 UDPSCOPE_EXIT_DUMP=1
|
|
UDPSCOPE_PROBE_RESEQ_MS=5000`.
|
|
|
|
| case | mock extra flags | key reading |
|
|
| --- | --- | --- |
|
|
| **base** | *(none)* | every counter 0; proves the machinery costs nothing |
|
|
| **reorder** | `--reorder-every 10` | `reseq=51` (mid-run) / `102` (dump); `holes=0`; resequencing inside the window leaves no reservation |
|
|
| **late** | `--reorder-every 10 --reorder-delay-ms 50` | `holes=42 bands=42 bands_repaired=42 nan_slots=0`; every band is the faint, repaired tier |
|
|
| **mixed** | `--drop-every 20 --reorder-every 10 --reorder-delay-ms 50` | `bands=42 bands_repaired=20 nan_slots=2300`; all three band states at once |
|
|
|
|
Full measured readings (probe line first, then exit-dump; `bands_repaired` was added after the
|
|
original probe and any earlier reading without it is obsolete):
|
|
|
|
```
|
|
=== base mock: none
|
|
[reseq-probe] reseq=0 late=0 stale=0 holes=0 hole_slots=0 backfilled=0 open_holes=0 bands=0
|
|
bands_repaired=0 nan_slots=0 scanned=44000 trigger_in_hole=0
|
|
[udpscope] frames=1024 packets=1025 gaps=0 channels=4 span=838.861s stride=1 reseq=0 late=0
|
|
stale=0 holes=0 hole_slots=0 backfilled=0
|
|
[udpscope] channel 0 'sine' stored=95200 published=95000
|
|
|
|
=== reorder mock: --reorder-every 10
|
|
[reseq-probe] reseq=51 late=0 stale=0 holes=0 hole_slots=0 backfilled=0 open_holes=0 bands=0
|
|
bands_repaired=0 nan_slots=0 scanned=44000 trigger_in_hole=0
|
|
[udpscope] frames=1028 packets=1029 gaps=0 channels=4 span=838.861s stride=1 reseq=102 late=0
|
|
stale=0 holes=0 hole_slots=0 backfilled=0
|
|
[udpscope] channel 0 'sine' stored=95400 published=95100
|
|
|
|
=== late mock: --reorder-every 10 --reorder-delay-ms 50
|
|
[reseq-probe] reseq=142 late=48 stale=0 holes=42 hole_slots=4200 backfilled=16800 open_holes=42
|
|
bands=42 bands_repaired=42 nan_slots=0 scanned=44000 trigger_in_hole=0
|
|
[udpscope] frames=1026 packets=1027 gaps=100 channels=4 span=838.861s stride=1 reseq=288
|
|
late=100 stale=0 holes=94 hole_slots=9400 backfilled=37600
|
|
[udpscope] channel 0 'sine' stored=95700 published=95500
|
|
|
|
=== mixed mock: --drop-every 20 --reorder-every 10 --reorder-delay-ms 50
|
|
[reseq-probe] reseq=163 late=24 stale=0 holes=43 hole_slots=4300 backfilled=8000 open_holes=43
|
|
bands=42 bands_repaired=20 nan_slots=2300 scanned=44100 trigger_in_hole=0
|
|
[udpscope] frames=974 packets=975 gaps=101 channels=4 span=838.861s stride=1 reseq=314 late=50
|
|
stale=0 holes=94 hole_slots=9400 backfilled=18400
|
|
[udpscope] channel 0 'sine' stored=95100 published=94900
|
|
```
|
|
|
|
The `mixed` case's `open_holes=43` against `bands=42` is correct: the 43rd hole has scrolled
|
|
out of the visible window, and `build_invalid_spans()` drops a span outside the window rather
|
|
than pinning it to the edge. The per-field reference is in
|
|
[probe-readings.md](probe-readings.md).
|
|
|
|
The mock also carries three **fault injections** that no probe schedules, because what they exercise
|
|
is the ingest rather than the UI. The first: `--time-glitch-at <stream seconds> --time-glitch-by <seconds>` makes
|
|
**one** packet's time signal jump ahead and then carries on from where the true timeline was, which
|
|
is what a device whose clock glitches puts on the wire. Run it with the trigger armed, e.g.
|
|
|
|
```sh
|
|
xvfb-run -a env UDPSCOPE_ENABLE_ALL_MS=400 UDPSCOPE_ARM_TRIGGER_MS=700 UDPSCOPE_EXIT_AFTER_MS=22000 \
|
|
./build/udpscope --server 127.0.0.1 --port 8000 --holdoff 10
|
|
# mock: --rate 10000 --batch 100 --freq 5 --time-glitch-at 6.0 --time-glitch-by 10.0
|
|
```
|
|
|
|
Four readings, all off the once-a-second `[udpscope] trig engine:` line:
|
|
|
|
- exactly one `warning: ... jumped implausibly far ahead of the previous packet (16.01..16.0199
|
|
against a previous 6.0099); bridging that packet ...` — the glitch was **caught**;
|
|
- **no** `timing samples by arrival instead` — one bad packet must not change the time source;
|
|
- `now_t` crosses the glitch continuously (`5.040 → 6.050 → 7.050`), and `history` keeps **growing**
|
|
(`1.01 … 23.15`) instead of collapsing — the ring was not wiped;
|
|
- the latches stay a hold-off apart: `edge at 1.060 / latched at 2.060 / holdoff_until=12.060`, then
|
|
`edge at 12.260 / latched at 13.260 / holdoff_until=23.260` — **11.2 s** for a 10 s hold-off plus
|
|
a 0.9 s post window.
|
|
|
|
Before the fix the same run read a fallback warning, `history` collapsing 4.03 → 0.95, `now_t`
|
|
jumping 5.040 → 16.040, and two latches **1.2 s** apart. Note the window here comes from the saved
|
|
session for that port; the figures above are the default 2 s window (50 % pre), so move
|
|
`~/.config/udpscope/<host>_<port>.toml` aside to reproduce them.
|
|
|
|
The other two are **packet loss** (`--drop-every N`, `--drop-burst K`) and **wire inversion**
|
|
(`--reorder-every N`). Loss withholds a datagram the counter and the stream clock have already
|
|
advanced over; the client counts the forward jump as `counter_gaps`. An inversion withholds a
|
|
packet until the one behind it is sent, so the receiver sees a later counter first; a packet that
|
|
arrives within the 20 ms reorder window is resequenced with no hole, and one that arrives past it
|
|
leaves a committed hole that it then backfills (`UDPSCOPE_PROBE_RESEQ_MS` exercises both paths).
|
|
There is **no interpolation** anywhere below the ring.
|
|
|
|
All of the readings below are off `UDPSCOPE_EXIT_DUMP`'s `[udpscope] frames= packets= gaps=` line,
|
|
over an 8 s run against `--rate 10000 --batch 100 --freq 5` (100 packets a second) except where the
|
|
burst needs the faster stream. **`gaps` is the exact reading** and repeats to the packet; `frames`
|
|
and `packets` move by one or two with the run's own timing. Run each of these on its own — four of
|
|
them back to back in one script starved the mock past the 2 s silence timeout and every case,
|
|
including the clean one, then read a reconnect instead of what it was testing:
|
|
|
|
- **clean**, as the reference: `frames=1029 packets=1030 gaps=0`, no warning.
|
|
- `--drop-every 10`: `frames=925 packets=926 gaps=102`. Every tenth packet never reaches the wire,
|
|
so `packets` falls with `frames` and `gaps` is the count of them. No warning: a one-packet hole is
|
|
a one-packet jump in the stamps, far inside the glitch guard.
|
|
- `--reorder-every 10`: `frames=1028 packets=1029 gaps=0`. The inverted packet lands inside the
|
|
20 ms reorder window and is resequenced: every datagram arrives and `gaps` is 0. The
|
|
pre-resequencing signature was `frames=925 packets=1028 gaps=102` — an inversion cost a counter
|
|
gap and a discarded packet, which is exactly what resequencing removes. Adding
|
|
`--reorder-delay-ms 50` pushes the arrival *past* the window and is the case the
|
|
`UDPSCOPE_PROBE_RESEQ_MS` readings above cover; it is not this row.
|
|
- `--drop-every 1000 --drop-burst 200` at `--rate 100000 --batch 100` (1000 packets a second):
|
|
`frames=7000 packets=7001 gaps=1600`, eight bursts of 200, and still no warning.
|
|
- `--drop-every 1000 --drop-burst 300`, same stream: `frames=6203 packets=6204 gaps=2400` **and the
|
|
guard fires** — `warning: ... jumped implausibly far ahead of the previous packet (1.299..1.29999
|
|
against a previous 0.99899)`, then `timing samples by arrival instead`. 300 packet spans is past
|
|
`kGlitchPackets` (256), so real loss is read as a clock glitch; and since each bridge advances the
|
|
anchor by one packet span while the true timeline is 300 ahead, the next four packets fail too and
|
|
`kFailureLimit` throws the stream onto the arrival clock. A burst between 256 packets and the 2 s
|
|
`silence_timeout_s` is the only window in which this is reachable, and it is a finding, not a
|
|
baseline.
|
|
|
|
Keep a burst under 2 s of wire time whatever the rate: past that the client reconnects, the mock
|
|
resets its counter and clock on the new CONNECT, and the run reads `gaps=0` with a *backwards* stamp
|
|
warning (`does not advance across packets ... 0..0.0099 against a previous 0.3899`) — the reconnect
|
|
path, not the loss path.
|
|
|
|
**Run one self-check at a time.** Two concurrent `xvfb-run -a` instances end up on the
|
|
same display and the second gets no GL context — it dumps `layout over 0 frames` and
|
|
`valid=-1`, which looks like a pipeline failure and is not one. `tools/mock_udpstreamer.py`
|
|
also outlives its `--duration`, so kill it after each run or the leftovers pile up and
|
|
hold ports.
|
|
|
|
The probes are the regression net — when changing UI or plot-host behaviour, extend
|
|
the probe/dump in `src/ui.cpp` rather than adding a separate harness.
|