udpscope/docs/claude/verification.md

# Verification

How to run the headless self-check, the two baselines every run is read against, and the environment variables that schedule each probe.

Part of [CLAUDE.md](../../CLAUDE.md).

There is no unit test suite. The only automated check is the app's own headless
self-check, driven by env vars and run under a display (`xvfb-run`, llvmpipe) against
a live UDPStreamer source:

```sh
xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=8000 UDPSCOPE_ENABLE_ALL_MS=1000 \
  ./build/udpscope --server 127.0.0.1 --port 8000
```

Two baselines every run is read against. A clean build emits **four** warnings and no more: two
`-Wmissing-field-initializers` on `MathNode::source` and `-Woverlength-strings` on **both**
generated GResource `.c` files — the help pages' and the icons', whose blob crossed C99's
4095-byte string-literal limit when the two app icons joined the four action ones (a note about a
generated file, which GCC itself supports). The fifth warning that used to appear here was an unused
`output_built` in `math_probe()`; it was **deleted** as dead code — redundant with the
`output_after_add` already printed on the same line — and that deletion is what dropped the baseline
from five to four. Nothing started reading it. And an exit-dump run emits **no** `Gtk-CRITICAL` at
all. A fifth warning, or any critical, is a finding.

An *incremental* build reads **two** of those four: the generated GResource `.c` files are not
recompiled when only `src/` changed, so their `-Woverlength-strings` do not reappear.

That second baseline used to be "exactly two, `gtk_widget_get_height` and `gtk_widget_get_width`,
from the dump asking a destroyed widget for its size", and tolerating them was a mistake: the dump
runs from the destroy handler, so the widget pointer is **freed memory**, and a critical is only
the lucky reading of it. A run that had built and destroyed other windows first —
`UDPSCOPE_PROBE_HELP_MS`, which opens three — has had that memory reused, and the same two calls
then **SIGSEGV in the middle of the dump**, which reads as a truncated dump rather than as a
crash in a debug line. Every size the dump prints is therefore captured on the *tick*, while the
window is alive (`App::dbg_*`), and **no `gtk_widget_*` call may appear in the dump** — the
`panel controls:` line was already written that way for the same reason, and that comment is the
precedent, not an exception.

- `UDPSCOPE_EXIT_AFTER_MS` — run the trigger-on → go-live probe, close the window,
  then dump the whole pipeline state to stdout (frame/sample counts, per-plot painted
  frames, trace bindings, panel sizes, status line in both states).
- `UDPSCOPE_ENABLE_ALL_MS`, `UDPSCOPE_ARM_TRIGGER_MS`, `UDPSCOPE_PROBE_DISARM_MS`,
  `UDPSCOPE_TRIG_TRACE_MS` — schedule individual probes. `UDPSCOPE_PROBE_DISARM_MS` only
  *starts* its sequence: it switches the trigger on and then polls for a capture to open,
  so the Disarm lands ~300 ms into one rather than wherever a fixed delay happened to fall.
- `UDPSCOPE_PROBE_TRIGRENUM_MS` — the trigger's capture must survive a **renumbering of its own
  channel**. It sets the reported geometry through the real widgets (1 s window, 10 % pre share,
  hold-off from `--holdoff`) on the mock's *square* alone, polls for a capture to open, and then
  enables the sine — which sorts before the square and so moves the trigger channel's index under
  the engine. Its three lines are in [probe-readings.md](probe-readings.md); **`t0_held=1` is the
  assertion**. Run it with `--holdoff 10`, which makes the stolen-t0 reading unmistakable: a second
  latch barely a second after the first cannot be a hold-off that applied.
- `UDPSCOPE_PROBE_FIELDS_MS` — change the trigger panel's edge and Normal/Single
  dropdowns. As much a crash test as a state check (see the GTK signal-signature invariant
  below), so time it inside a capture: `[fields-probe] ... capturing=1` followed by no
  SIGSEGV *is* the assertion.
- `UDPSCOPE_PROBE_WINDOW_MS` — drive the window control every way a user can: type a
  number, change the unit under it, step a decade either way.
- `UDPSCOPE_PROBE_LAYOUT_MS` — drive the layout editor: switch it on, insert above the
  first row, remove a row, append one at the bottom, switch it off. Its steps are 200 ms
  apart on purpose — the buttons are placed from the rects the *last frame* was laid out
  with, so a synchronous sequence would read every position one action stale. It clicks by
  emitting `::clicked`, never `gtk_widget_activate()`, which holds a `GtkButton` for its
  250 ms `ACTIVATE_TIMEOUT` and then emits into a button the rebuild has already destroyed.
- `[measure-probe] N measurements on 'X'` (stderr, `UDPSCOPE_PROBE_MEASURE_MS`) — adds one
  measurement of each statistical type (average, top, peak to peak, amplitude) on the first
  *channel* and forces a recompute. It must name a **channel**, not `signals().front()`: the
  first signal of a stream is its timestamp, which is not acquired, and naming it adds four
  measurements that all read `--` *and* makes a performance run look free (each bails before
  collecting anything -- that is how the first baseline of the stall below read 0.4 ms).
  The values are in the exit dump as `measurements: <type>=<value>`; on the fast mock
  (`tools/mock_udpstreamer.py --rate 1300000 --batch 2000`) a unit sine must read
  `peak to peak=2`, `amplitude=1`, `average=0` and **`top=0.636620`** -- 2/pi, the half-wave
  mean of a sine, which is the check that the threshold types are not merely plausible.
  `[measure-trace] view=[..] <type>=<value>... marks=N level:V/time:V...` is a repeating line
  (250 ms) from the probe: the pane's numbers beside the *current* window and the marks actually
  placed, which is the only way a stale result or a mark that should not be drawn shows up as a
  number. It must never print `marks=0` while measurements with `show_on_plot` are on and the
  data is there -- that was the rulers blinking off at a mode change.
  `measure cost: N recomputes, X ms total, Y ms each, data D ms (P%), covered=C buckets=B
  raw=R (F%), N ns/covered sample` follows it: **read it against a run with
  `UDPSCOPE_PROBE_MEASURE_MS` unset**, because a frame mean cannot attribute a cost. The last
  three fields are the assertion, and they are a *ratio*, not a time: `covered` is the samples
  the window spans, `buckets` the LOD buckets the two walks answered from, and `raw` the samples
  they still had to read. **`raw` must stay at a fraction of a percent of `covered`** on a clean
  waveform over a window of millions -- 0.1% is what the pyramid-backed walks give (see the
  measurement-cost invariant in
  [invariants-measurements.md](invariants-measurements.md)). A `raw` near `covered` means the walks went back to reading
  the window, which at 1 MS/s is 19.7 ms per recompute and a third of the run's frames. It is
  *legitimately* higher for a window shorter than a bucket (64 samples) or a signal noisy
  enough to cross a threshold everywhere, so read it on the fast mock
  (`tools/mock_udpstreamer.py --rate 1300000 --batch 2000`), where a window is ~2 M samples.
  `--noise 0.1` there reads **2.0%** at 1.96 ms a recompute against the clean 0.1% at 1.41 ms, and
  that is the interferer's own cost rather than a regression -- the figure to compare a change
  against is the **clean** one.
  Note the **final** `measurements:` line and `[measure-trace]`'s last lines may read `--`:
  `UDPSCOPE_EXIT_AFTER_MS` arms the trigger before it dumps, so the dump can land in `Waiting`
  with nothing bound and the axis at its default. The race-free reading is an earlier
  `[measure-trace]` line, which carries its own `view=`; those lines are also where the
  level-mark fallback is visible (an invalid pass must still print its `level:` marks).

- `[pan-probe] asked=[..] was=[..]`, `[pan-probe] after=[..] held=D asked=[..]`, then
  `[pan-probe] zoom asked=[..] after=[..] span=S window=W limited=L`
  (stderr, `UDPSCOPE_PROBE_PAN_MS`) — **`held` must be 1**: a gesture must not move a window
  that is pinned (trigger mode, or a paused view). The probe writes the panned range the way a
  gesture does — on the *member* axis, which `Figure::layout()` adopts into the group — and
  reports the axis the frame after it produced. Time it *inside* one pinned state: a pan during
  a capture whose check lands after the capture has ended is re-framed by the mode change, not
  by the pin, and reads `held=1` with the hold removed (that is how the first version of this
  probe fooled me). The pause probe makes a stable pin, so run them together:
  `UDPSCOPE_PROBE_PAUSE_MS=1500 UDPSCOPE_PROBE_PAN_MS=2200` puts the pan and its check inside
  the pause. With `hold_pinned_window()` disabled it reads `held=0` and the axis sits at the
  asked range. It then asks for a **zoom-in** (a window a quarter of what is on screen) and
  reports `group=`/`member=` beside it: **`allowed` must be 1, with both equal to the ask** --
  that is the single-trigger-mode case, where the wheel has to keep working, and where reading
  the wrong one of the two made a working clamp look broken. All three verdicts are read from the
  *tick*, one frame after the ask: a `g_timeout` was the first version and it drifted past the
  paused/resumed boundary under load. The probe then asks for a **zoom-out** ten times the current span and reports
  the span the next frame produced: **`limited` must be 1**, i.e. `span <= window`. Run it with
  a ring that holds *more* than the window (`--buffer-mb 64` past the window's own seconds) —
  that is the state the old `max(retained, window_s_)` bound leaked in, and it showed up here as
  `span=2.502550 window=2.000000 limited=0` against `span=2.000000` after the fix.

- `[zoom-probe] <sweep>/x<step> axis=[..] span=.. n=N idx=[a..b) spp=.. level=L pts=P kept=K
  back=.. ink=[..] x_at_first=.. gap_px=.. end_px=.. rect=[..]` (stderr,
  `UDPSCOPE_PROBE_EVENTZOOM_MS`) — arms the trigger in **single** mode, polls for a **held
  event** (a latch happens when the stream says so; a sweep on a timer has nothing to zoom
  into), then zooms in eight doublings
  twice: `ges/` writes the row's own axis the way a gesture does, `win/` drives the window
  control (`apply_time_window`, which unpins and re-arms the live follow before the pin is
  restated). Each step's reading is taken in the *tick*, after that frame's pin and clamp — a
  `g_timeout` would report the ask. It recomputes what `Renderer::draw_traces` is handed, through
  the same calls: `neighbours()` for `idx`, the 64-spp gate for `level`, then `gather()`.
  **`gap_px` and `end_px` must stay within a pixel or two at every step** — they are how much of
  the left and right margin no geometry reaches, and a `gap_px` that *doubles* per step is the
  "zooming into a held event clips it more and more from the left" defect (374 px of a 716 px
  axis by `ges/x128`), whose cause is a constant offset in data seconds (see the `neighbours()`
  invariant in [invariants-rendering.md](invariants-rendering.md)). `kept`/`back` are the renderer's own increasing-suffix filter counted rather
  than applied: `kept` far below `pts` is a staircase, and a non-zero `back` is a non-monotonic
  gather. **Run it with `--clock-jitter`** (`tools/mock_udpstreamer.py --rate 1300000 --batch
  2000 --clock-jitter 0.4`): the ideal mock's stamps sit on a perfect grid, which is exactly
  what hides this class of bug, and three runs against it reported `gap_px=0.0` throughout while
  the defect was live. The jitter varies each *packet's* spacing rather than offsetting its
  start, because the stream's clock has to stay strictly increasing. The `win/` sweep bottoms
  out at `span=0.05` (`RowSet::set_time_window()`'s floor) and walks to the event's post side,
  since `set_time_window()` frames the newest span before the pin clamps it — both are expected.
- `UDPSCOPE_PROBE_PAUSE_MS` — pause acquisition mid-run so layout numbers are
  measured on a still picture. It now also asserts what a pause *is*
  (`[pause-probe] <action> paused=D ring_produced=N event_size=M frozen_x=[..] axis=[..]
  skips=N glance=[..]` on stderr):
  it presses the **button**, not `Acquisition::set_paused()`, because pausing is a display
  state as well as an acquisition one and the view snapshot and axis pin live in the
  handler. Across the three lines **`ring_produced` must grow** (the producer never
  stops — it fills the live ring exactly as it does with the trigger off), **`event_size`
  must be non-zero exactly while paused** (the plot is bound to the frozen snapshot, not
  the ring) and **`frozen_x` must not move between the pause and the resume**; after the
  resume `event_size` is 0 and the window is back on the newest data. Pausing used to stop
  the producer, which is why the counts grew by nothing and the resume left a hole.
  `frozen_x` is the *rebased* snapshot — the pause view's own stamps are relative to the freeze
  instant now, so the range reads `[-window..0]`-ish rather than stream seconds, and a
  `frozen_x` in the thousands is the rebase gone.
  **`glance=` is the assertion for the paused status field**, and this is the only line that can
  make it: the exit dump's `status bar:` is read live and its two `mode …` lines are the live and
  triggered states. While paused it must read `paused · <rate> · <window> · for <elapsed>`, with
  the elapsed figure **growing between the two paused lines** and **absent on the resume line**.
  It is **wall** time (`g_get_monotonic_time()`), not stream time, because the producer never
  stops while paused and "how long have I been looking at this" is a question about the user, not
  about the stream; it is written through `format_elapsed()` so the field steps to a coarser unit
  instead of growing digits (see the layout rule).
  **`skips=` is the assertion for the render gate** (the end of `tick()`), cumulative over the
  session: the `live` line must read **0** and the `paused` line must read **more than 0**. A
  frozen view is bound to an immutable snapshot on a pinned axis, so a frame whose picture cannot
  have changed is not drawn again — and this is the only line that can say whether that happened,
  since a skipped frame is invisible in every other reading by construction. Non-zero on the
  `live` line is the gate's condition inverted, which would cost dropped samples' worth of
  picture; 0 on the `paused` line is the gate not engaging and the saving not being made.
  The gate cannot stall the picture (`kStaticRenderEvery` renders every few ticks regardless),
  so the renderer's own baselines below are unchanged by it: a measured run read `skips=0`,
  then 33, then 45, with `columns == graded`, `render blank frames: 0`, and the layout floor
  still converged (`y-gutter 104..104`).
- `UDPSCOPE_PROBE_COLOR_MS` — recolour one plotted signal and read the colour back out of the
  trace and the chip, round-trip it through the session writer, then clear it. Schedule
  `UDPSCOPE_ENABLE_ALL_MS` before it: with nothing bound it has no trace to recolour and asserts
  nothing. Its four lines are in
  [probe-readings.md](probe-readings.md).
- `UDPSCOPE_EXPORT_MS` (+ optional `UDPSCOPE_EXPORT_PATH`, default
  `/tmp/udpscope-export`) — collect the export table and write it both ways, skipping
  only the file chooser.

- `UDPSCOPE_PROBE_CONDITION_MS` — arms a qualified trigger against the mock and reports the latched
  `t0`, the measured pulse or band width, and sample values for the window condition. Schedules its
  own arm and report stages. `UDPSCOPE_PROBE_CONDITION_SHAPE` picks the leg: `0` pulse width `<`
  (the glitch, the default, needs `--glitch-at`/`--glitch-width`), `1` pulse width `>` (uses a
  1.0 s window and 50 % pre so the opening edge of a 0.2 s pulse is captured), `2` window
  (uses `sine` with a 0.2 s window and 10 % pre), `3` dropout (needs `--quiet-at`; uses `sine`
  with a 1.0 s window and 50 % pre so the last crossing before the outage is captured).
- `UDPSCOPE_PROBE_STATS_MS` — reports every measurement's accumulator (`n`, mean, sd, lo, hi),
  pauses, reports again with `dn=` per measurement, then resets and reports a third time. Three
  phases: `phase=0`, `phase=1` (after pause) and `phase=2` (after Reset). Needs
  `UDPSCOPE_ENABLE_ALL_MS` ahead of it and `UDPSCOPE_PROBE_MEASURE_MS` to add measurements first.
- `UDPSCOPE_PROBE_SEARCH_MS` — runs a buffer search for the armed condition, cross-checks the live
  trigger's own `t0` against the nearest hit (`agree=1` is the load-bearing reading), and jumps to
  hit 0. Reports the resulting `mode=`, `axis=` and `marks placed=`.

The following mock waveform features are what the qualified-condition probes are checked against,
as distinct from the fault injection below, which exercises ingest:

- `--glitch-at S` / `--glitch-width W` invert the `square` signal for `W` seconds at the start of
  every square period from stream second `S` onwards — a runt pulse of a known width, which is
  what a `<` pulse-width trigger and a glitch search are checked against.
- `--quiet-at S` freezes `sine` at its held value from stream second `S` onwards, so it stops
  crossing zero: the dropout trigger's input.

- `UDPSCOPE_PROBE_RESEQ_MS` — one mid-run reading of the resequencing path. Five cases,
  **run each as its own process** — five back to back starve the mock past its 2 s silence
  timeout and every case then reads a reconnect instead of what it was testing. Common
  conditions for the first four: the mock at `--rate 10000 --batch 100 --freq 5`, udpscope with
  no extra options, under `xvfb-run` with
  `UDPSCOPE_EXIT_AFTER_MS=8000 UDPSCOPE_ENABLE_ALL_MS=600 UDPSCOPE_EXIT_DUMP=1
  UDPSCOPE_PROBE_RESEQ_MS=5000`. Delete `~/.config/udpscope/127.0.0.1_8000.toml` first: a
  session left by an earlier run restores its math channel, and five channels instead of four
  change `channels=`, `span=` and every count derived from them.

  | case | mock extra flags | key reading |
  | --- | --- | --- |
  | **base** | *(none)* | every counter 0; proves the machinery costs nothing |
  | **reorder** | `--reorder-every 10` | `reseq=51` (mid-run) / `102` (dump); `holes=0`; resequencing inside the window leaves no reservation |
  | **late** | `--reorder-every 10 --reorder-delay-ms 50` | `bands_repaired` one short of `bands` with `nan_slots=100`; the band still solid is the one hole whose late packet is in flight, and it is the *only* unrepaired one |
  | **mixed** | `--drop-every 20 --reorder-every 10 --reorder-delay-ms 50` | `bands=43 bands_repaired=20 nan_slots=2200`; all three band states at once |
  | **re-plan** | `--rate 100000` with the `mixed` flags, and `UDPSCOPE_PROBE_WINDOW_MS=4000 UDPSCOPE_PROBE_RESEQ_MS=6000 UDPSCOPE_EXIT_AFTER_MS=9000` | four `buffer rebuilt` lines before the reading, then `bands=64 bands_repaired=16 nan_slots=7800`: **a band is still only marked repaired when its own slots hold values**. A `bands_repaired == bands` beside a non-zero `nan_slots` here is the hole table and the producer's `tick_` surviving a re-plan that reset `produced()` — a band marked settled over samples that are all still NaN. The non-zero `overflows` is this case too: 100 kS/s with a 50 ms reorder delay keeps about 50 packets in flight against 64 slots |

  The re-plan case needs `--rate 100000`, not the other four's 10 000: at 10 kS/s the ring

- `UDPSCOPE_PROBE_CAPTURE_MS` — toggles the packet capture **through the same handler the
  status-bar button uses**, waits, then reads the file back off disk and prints one line:
  `enabled= path= slots= stored= resident= cursor= torn= truncated= header_ok= first_seq=
  last_seq= payload= file= open_ms= t0=`. `stored` is the datagrams-since-open counter and must
  equal the dump's datagram count from the toggle's instant; `resident` is what the file holds,
  which is the slot count once the ring has wrapped; `file` is the reserved size
  `64 + slot_count * slot_bytes` and is a constant for a given N, so a mismatch against the file
  on disk is a reading. Two cases, **run each as its own process**:

  1. **enable at runtime** — the mock at `--rate 20000 --batch 200`, udpscope with
     `UDPSCOPE_ENABLE_ALL_MS=700 UDPSCOPE_PROBE_CAPTURE_MS=1500 UDPSCOPE_EXIT_AFTER_MS=9000`
     and no capture flag: `enabled=1 header_ok=1 torn=0 truncated=0 first_seq=1`, `stored` equal
     to the datagrams since the toggle, `open_ms` a few ms.
  2. **wrap** — the same with `--capture-packets=16` and
     `UDPSCOPE_PROBE_CAPTURE_MS=7000 UDPSCOPE_EXIT_AFTER_MS=10000`: `stored` (870 in the
     reference run) exceeds `slot_count`, `resident=16`, `last_seq - first_seq == 15`,
     `torn=0`, and `cursor` agrees with `seq` (counter mod slot count). A wrap that loses
     ordering, or a `cursor` that disagrees, shows here and nowhere else.

  Then the converter over the wrap case's file: `tools/capture_to_pcap.py <capture> out.pcap`
  must exit 0, print a record count equal to `slot_count` with `torn` 0, and the pcap's first
  and last record timestamps must fall inside the run's wall-clock interval — both checked by a
  `struct.unpack` of the pcap headers, nothing outside the standard library.

- **A gap must not kill a filtered trace.** `--relative-time` with the mock dropping
  (`--drop-every 20`) and `UDPSCOPE_PROBE_MATH_MS=2500 UDPSCOPE_ENABLE_ALL_MS=600
  UDPSCOPE_EXIT_AFTER_MS=9000`: every derived channel's dump line
  `math '<name>': finite=N/4000` must read about 3800 — the tail minus the reserved holes —
  and **`finite=0` beside a non-zero `hole_slots` is the defect** (a biquad that integrated the
  hole's NaN, permanently). The filters' `sd` must stay near the no-drop value; run the same
  command without `--drop-every` as the control, where every `finite` reads 4000.
- `UDPSCOPE_PROBE_CONFIGS_MS` — saves the running session as a configuration named `probe` with a
  note, through `save_configuration_as()` (the same entry point the dialog's button uses), then
  reads it back off disk. Reading: `save-as=1`; `names=` gains `probe`; `active=probe`; and the
  round trip must carry the note **and** the startup settings —
  `comment='written by the configs probe' buffer_mb=8 relative=0 capture=0`. A `buffer_mb=0` there
  is the per-source buffer size not being written, which is the one setting a user notices only
  after a capture has been sized wrong. Run it with a throwaway `XDG_CONFIG_HOME`; the probe
  deletes its own configuration at the end (`after forget names=…`).
- `UDPSCOPE_PROBE_ROWMENU_MS` — gives row 0 a trace, opens the plot's context menu with
  `open_row_menu()` (the one entry point `on_right_click()` wraps) and walks its two views one step
  at a time (`main` → click **Remove signal** → `remove` → click **Back** → `after-back` → click
  **Remove signal** again → `second-open` → click a trace), measuring each view's rows:
  `[rowmenu] <view> N '<label>' h=18 label_x=26 accel_right=R/W ok=1` per row and one
  `rows=/bad=/labels=` line per view, then `[rowmenu] summary main_rows=5 main_bad=0 remove_rows=6
  remove_bad=0 after_back_rows=5 after_back_bad=0 second_rows=6 second_bad=0 remove_row_in_main=1
  remove_row_gone=1 can_open_again=1` and `[rowmenu] remove-clicked 'sine' trace_freed=1`.
  **Every `bad` must be 0** (the same compact row style as the app's own menu: one label column, a
  shortcut flush right, 18 px rows — the **Back** row's icon indents its label, which every menu
  does, so its `label_x` is 52 against the others' 26) and the three booleans must all read 1: the
  remove view *replaces* the main one (`remove_row_gone`, `remove_row_in_main`), and it can be
  entered again after leaving it (`can_open_again`) — that last one is the case a
  `GtkMenuButton`-owned submenu failed, its row going dead on the second click. Each view is
  measured a step after the click that swapped it in, because a popover allocates its new child on
  the next layout pass.
- `UDPSCOPE_PROBE_DEFAULTS_MS` — opens **Calibrate** and reads every spin button in it and in the
  trigger panel: `[defaults] calibrate: 1 0 model=1 0 | trigger: 0.001 0.002 0 0.001 0 50 0.01`. A
  `GtkSpinButton` starts at its range's *minimum*, so a `-1e12` here is an unseeded button the
  dialog's own reader would then apply — which is exactly what the calibration offset did. It then
  calibrates signal 0 through `set_calibration()` and reopens the dialog:
  `[defaults] calibrated: 2.5 -1.25 model=2.5 -1.25` — **the dialog must show the signal's stored
  calibration**, and the two halves must match.
- The **chooser**, display-free: `UDPSCOPE_PROBE_SOURCES=1` prints `[sources] N known:` with one
  entry per configuration (`host:port/name … buffer= relative= note='…'`), and
  `UDPSCOPE_PROBE_PICK_MS=800` opens the chooser, picks the first row and prints
  `[sources] picked host:port … buffer=64 relative=0 capture=0 config=… note='…'` — the settings the
  configuration carried, into the `Config` the model is built from. A legacy file
  (`<host>_<port>.toml`) produces `layout loaded from …/<host>_<port>/default.toml`, an `active`
  holding `default`, and the old file renamed `.imported`.
- **The event cache round-trips, and only where it is available.** `UDPSCOPE_PROBE_EVENTS_MS` reads
  the pane's availability first, in the state the app starts in -- `[events] live: button_visible=0
  pane=0 recording=1`, **no pane and no button without trigger mode** -- then turns trigger mode on
  through the real toggle (`[events] after trigger on: button_visible=1 pane=0`: the button appears,
  the pane stays collapsed) and clicks the button, waits for a latch, then reads the cache and replays
  the newest event through the pane's own row:
  `[events] dir=… n=1 newest: dur=2.000000 pre=1.000000 post=1.000000 edge=… signals=4 samples=20001
  rate=10000 seq=1 thumb_bytes=230454 recorded=1 pane=1`, then
  `[events] activated row0: replaying=1 thumbnail=1 thumb_size=108x36 thumb_visible=1` (the size is
  what says the picture is *on screen*: a paint-able alone is not), `[events] replayed: channels=4
  samples=20001 replaying=1 waveform=1` and `[events] live again: replaying=0 waveform=0`. It ends on
  the pane's availability, through the trigger panel's switch: `[events] recording off, trigger off` and
  `[events] recording off, trigger on` -- once the cache holds events (the probe's own run has written
  some by this point) both read `button_visible=1 pane=1 recording=0`, because a button that shows the
  cache must not go away with the switch that wrote it; against an *empty* cache the same two lines read
  `button_visible=0 pane=0`, which is the empty column the item exists to prevent. **`samples` must equal the
  export's row count for the same window** (`20001` for a 2 s event at 10 kS/s), `thumb_bytes` must be
  a real BMP (`54 + 480*160*3`-ish, i.e. hundreds of kB, not 0), `thumbnail` must be 1 (the row loaded
  the picture) and the last line must read `replaying=0` -- a replay with no way back is a one-way
  door. It then **waits for a second latch** (bounded: twelve retries) and drives the two group layouts
  through the buttons: `[events] compare buttons: cached=2 selected=2 compare=1 accumulate=1` (the
  buttons read the selection), `[events] halve`… `[events] halves a_first=20001 a_second=0 b_first=0
  b_second=20001` -- **the two halves of the ring, and the reading that says a comparison's gap is a
  gap: the first event's channel is finite only in the first half and the second's only in the second**
  -- `[events] compare: kind=1 rows=4 channels=8 samples0=40002 traces0=2 traces1=2 window=4.000
  recording=1 pane=1` (`samples0` must be exactly the stamps pushed, **not more**: a live producer left
  running pushes its own four signals into the group's eight channels), `[events] compare plots:
  area_b=1 rows_a=1 rows_b=1 x_a=[-2.000,0.000] x_b=[-2.000,0.000] same_x=1` (the second plot exists, is
  on screen, mirrors the row count, and both are on one x window), then the sync -- the probe sets a y
  range nobody would arrive at by accident on the *first* plot and reads the second a few frames later:
  `[events] compare sync: y_a=-3.500 y_b=-3.500 mirrored=1` -- and `[events] accumulate: kind=2 rows=1
  channels=8 samples0=33045 selected=2 traces0=8 window=2.000` (`traces0=8`: the four signals the layout
  holds plus one layer each for the second event, **in the same row**, which is what an accumulation
  means). Run it with a throwaway
  `XDG_CACHE_HOME` and `UDPSCOPE_EXIT_AFTER_MS` long enough for the second latch (20 s).
- **A waveform round-trips.** Export from a live run (`UDPSCOPE_EXPORT_MS` with
  `UDPSCOPE_EXPORT_PATH`, against the mock at 10 kS/s), then open that file with `--open` and an
  exit dump. The load must print
  `[udpscope] waveform loaded: … signals=4 samples=20001 span=2.000000s rate=10000 time=[-2.000000..0.000000]`
  — **the same row count, column count, span and time range the export probe reported**
  (`rows=20001 cols=4 time=[-2.000000..0.000000]`) — and the dump must show `channels=4`,
  `frames=0 packets=0` (no acquisition) and, per channel,
  `20001 samples vs 20001 time slots … y=[-1.0000, 1.0000]`. A file that loads with fewer samples
  than it was written with, or with `NaN` slots, is the reader mis-slicing its records.
- **A loaded file can be plotted.** `UDPSCOPE_PROBE_DROP_MS` drives the drop handler the way a drag
  from the signal panel does, on a file opened with `--open`: it must read
  `[drop-probe] signals=4 channels=4 rows=4 slots0=[sine|||]` — **`rows` equal to the column count**,
  one trace per row — and each drop must return `1` and add its trace (`dropped signal 1 -> 1
  slots=[sine|sine|square|]`). `rows=1 slots0=[sine|square|ramp|noise]` with every drop returning `0`
  is the defect this exists for: rows sized four signals to one row fill the only row, so the app
  reads as one that cannot plot anything. A file-mode run must also write **no session** for the
  default `127.0.0.1:8000` (`App::source_port` is 0), or it overwrites that source's saved layout.
- **The chooser's Waveforms page**: `UDPSCOPE_PROBE_WAVEFORMS=1` prints
  `[sources] waveforms: N recent <path>(ok|missing) … page=waveforms`, and `UDPSCOPE_PROBE_WAVE=<path>` sets
  the field the file dialog's callback sets and prints `[sources] picked waveform <path>` — then the
  load line above, with no socket involved.
- **A `--server` launch reads the stored configuration.** With a throwaway `XDG_CONFIG_HOME`
  holding `127.0.0.1_8101/bench.toml` (`buffer_mb = 64`, `relative_time = true`,
  `capture_packets = 500`, `active` = `bench`), `./build/udpscope --server 127.0.0.1 --port 8101`
  must print `[sources] settings from configuration 'bench': buffer=64 relative=1 capture=500`, the
  `relative time:` line, and a `span=` of about 209 s (64 MB at 10 kS/s) — **not** the 200 MB
  default's ~655 s. The same command with `--buffer-mb 8` must read `buffer=8` and `span=26.214s`:
  **an explicit flag wins over the file**. Without this, the settings a host was edited to have were
  ignored whenever the address came from the command line.
- The chooser's **form**, with `UDPSCOPE_PROBE_PICK_MS` scheduling it:
  `UDPSCOPE_PROBE_NEWHOST=host:port` fills it and prints
  `[sources] form: name='probe-host' note='…' buffer=32 relative=1 connect_sensitive=1`, then
  `[sources] configuration created: host:port/probe-host … buffer=32 relative=1` and a
  `picked … config=probe-host`; a following `UDPSCOPE_PROBE_SOURCES=1` must list it
  (`buffer=32 relative=1 note='…'`) — a configuration the form wrote that the chooser does not
  list cannot be edited next time. `UDPSCOPE_PROBE_EDITHOST=host:port` opens the same form on that
  configuration and prints `[sources] edit form: name='…' note='…' buffer=… relative=… button='Save'`
  — **the pre-fill is the assertion**: an edit form that opens empty cannot edit anything — then
  saves and reports the new values. `UDPSCOPE_PROBE_EDITNAME=new-name` renames it as well: one
  `known:` row afterwards, with the old file gone.
- `UDPSCOPE_PROBE_RESEQ_MS`'s neighbour for **reconnect**: the mock gains an EMI case,
  `--blackout-at S --blackout D` — silent from second S for D seconds (D must exceed the
  client's 2 s silence timeout), resuming from *now* so the link's backlog is what the link
  ate. One run (`--blackout-at 4 --blackout 3.5`, `UDPSCOPE_EXIT_AFTER_MS=15000`) has to read:
  every reconnect's `CONFIG:` line followed by `CONFIG unchanged: keeping the buffer and its
  history` (a reconnect that re-plans wipes history, which is the defect this case exists for),
  no `buffer rebuilt` line after the first CONFIG, `frames` climbing again after the blackout
  with `gaps=0`, and a trigger armed *after* recovery (`UDPSCOPE_PROBE_KEYS_MS` past the
  blackout) latching — the dump's `CAPTURING`/`EVENT HELD` and `publish cost: N events` with
  N ≥ 1 are that proof.
  already covers 838 s, so no window the control can reach moves `time_stride`,
  `set_target_span()` returns without rebuilding, and the window probe reports `decim=1`
  throughout — it re-plans nothing. At 100 kS/s the ring covers 83.9 s and the probe's
  500 ms → 500 s → 120 s → 600 s → 60 s walk crosses the boundary four times.

  Full measured readings (probe line first, then exit-dump):

  ```
  === base   mock: none
  [reseq-probe] reseq=0 late=0 stale=0 overflows=0 holes=0 hole_slots=0 backfilled=0 open_holes=0
                bands=0 bands_repaired=0 nan_slots=0 scanned=44200 trigger_in_hole=0
  [udpscope] frames=1032 packets=1033 gaps=0 channels=4 span=838.861s stride=1 reseq=0 late=0
             stale=0 overflows=0 holes=0 hole_slots=0 backfilled=0
  [udpscope]   channel 0 'sine' stored=95700 published=95400

  === reorder   mock: --reorder-every 10
  [reseq-probe] reseq=51 late=0 stale=0 overflows=0 holes=0 hole_slots=0 backfilled=0
                open_holes=0 bands=0 bands_repaired=0 nan_slots=0 scanned=44200
                trigger_in_hole=0
  [udpscope] frames=1027 packets=1028 gaps=0 channels=4 span=838.861s stride=1 reseq=102 late=0
             stale=0 overflows=0 holes=0 hole_slots=0 backfilled=0
  [udpscope]   channel 0 'sine' stored=95300 published=95100

  === late   mock: --reorder-every 10 --reorder-delay-ms 50
  [reseq-probe] reseq=157 late=46 stale=0 overflows=0 holes=40 hole_slots=4000 backfilled=15600
                open_holes=40 bands=40 bands_repaired=39 nan_slots=100 scanned=44000
                trigger_in_hole=0
  [udpscope] frames=1027 packets=1028 gaps=98 channels=4 span=838.861s stride=1 reseq=296
             late=98 stale=0 overflows=0 holes=91 hole_slots=9100 backfilled=36400
  [udpscope]   channel 0 'sine' stored=95300 published=95000

  === mixed   mock: --drop-every 20 --reorder-every 10 --reorder-delay-ms 50
  [reseq-probe] reseq=159 late=25 stale=0 overflows=0 holes=43 hole_slots=4300 backfilled=8400
                open_holes=43 bands=43 bands_repaired=20 nan_slots=2200 scanned=44100
                trigger_in_hole=0
  [udpscope] frames=978 packets=981 gaps=101 channels=4 span=838.861s stride=1 reseq=309 late=50
             stale=0 overflows=0 holes=94 hole_slots=9400 backfilled=18400
  [udpscope]   channel 0 'sine' stored=95400 published=95400

  === re-plan   mock: --rate 100000 --drop-every 20 --reorder-every 10 --reorder-delay-ms 50
  [reseq-probe] reseq=4374 late=72 stale=0 overflows=36 holes=315 hole_slots=31136
                backfilled=26800 open_holes=64 bands=64 bands_repaired=16 nan_slots=7800
                scanned=156600 trigger_in_hole=0
  [udpscope] frames=8949 packets=8953 gaps=614 channels=4 span=83.886s stride=1 reseq=8028
             late=144 stale=0 overflows=36 holes=603 hole_slots=59936 backfilled=55600
  [udpscope]   channel 0 'sine' stored=588600 published=584600
  ```

  The `late` case's `bands_repaired` is one short of `bands` with `nan_slots=100` — exactly one
  packet's worth of slots at stride 1 — because the probe fires 50 ms-delayed releases into a
  window that always has one in flight. An earlier recording of this case read
  `bands_repaired=42 nan_slots=0`; both are healthy, and which one a run gets is where the
  probe's instant falls relative to the newest release. A `bands_repaired` *above* the count of
  holes whose slots hold values is the defect, in either direction.

  `holes` far above `open_holes` in the re-plan case is the two bounds working: a re-plan clears
  the table (the old entries address a buffer that is gone) and `kMaxHoles` caps what survives
  at 64. The `mixed` case's `open_holes=43` against `bands=43` has no such spread because
  nothing re-planned. The per-field reference is in [probe-readings.md](probe-readings.md).

The mock also carries three **fault injections** that no probe schedules, because what they exercise
is the ingest rather than the UI. The first: `--time-glitch-at <stream seconds> --time-glitch-by <seconds>` makes
**one** packet's time signal jump ahead and then carries on from where the true timeline was, which
is what a device whose clock glitches puts on the wire. Run it with the trigger armed, e.g.

```sh
xvfb-run -a env UDPSCOPE_ENABLE_ALL_MS=400 UDPSCOPE_ARM_TRIGGER_MS=700 UDPSCOPE_EXIT_AFTER_MS=22000 \
  ./build/udpscope --server 127.0.0.1 --port 8000 --holdoff 10
# mock: --rate 10000 --batch 100 --freq 5 --time-glitch-at 6.0 --time-glitch-by 10.0
```

Four readings, all off the once-a-second `[udpscope] trig engine:` line:

- exactly one `warning: ... jumped implausibly far ahead of the previous packet (16.01..16.0199
  against a previous 6.0099); bridging that packet ...` — the glitch was **caught**;
- **no** `timing samples by arrival instead` — one bad packet must not change the time source;
- `now_t` crosses the glitch continuously (`5.040 → 6.050 → 7.050`), and `history` keeps **growing**
  (`1.01 … 23.15`) instead of collapsing — the ring was not wiped;
- the latches stay a hold-off apart: `edge at 1.060 / latched at 2.060 / holdoff_until=12.060`, then
  `edge at 12.260 / latched at 13.260 / holdoff_until=23.260` — **11.2 s** for a 10 s hold-off plus
  a 0.9 s post window.

Before the fix the same run read a fallback warning, `history` collapsing 4.03 → 0.95, `now_t`
jumping 5.040 → 16.040, and two latches **1.2 s** apart. Note the window here comes from the saved
session for that port; the figures above are the default 2 s window (50 % pre), so move
`~/.config/udpscope/<host>_<port>.toml` aside to reproduce them.

The other two are **packet loss** (`--drop-every N`, `--drop-burst K`) and **wire inversion**
(`--reorder-every N`). Loss withholds a datagram the counter and the stream clock have already
advanced over; the client counts the forward jump as `counter_gaps`. An inversion withholds a
packet until the one behind it is sent, so the receiver sees a later counter first; a packet that
arrives within the 20 ms reorder window is resequenced with no hole, and one that arrives past it
leaves a committed hole that it then backfills (`UDPSCOPE_PROBE_RESEQ_MS` exercises both paths).
There is **no interpolation** anywhere below the ring.

All of the readings below are off `UDPSCOPE_EXIT_DUMP`'s `[udpscope] frames= packets= gaps=` line,
over an 8 s run against `--rate 10000 --batch 100 --freq 5` (100 packets a second) except where the
burst needs the faster stream. **`gaps` is the exact reading** and repeats to the packet; `frames`
and `packets` move by one or two with the run's own timing. Run each of these on its own — four of
them back to back in one script starved the mock past the 2 s silence timeout and every case,
including the clean one, then read a reconnect instead of what it was testing:

- **clean**, as the reference: `frames=1029 packets=1030 gaps=0`, no warning.
- `--drop-every 10`: `frames=925 packets=926 gaps=102`. Every tenth packet never reaches the wire,
  so `packets` falls with `frames` and `gaps` is the count of them. No warning: a one-packet hole is
  a one-packet jump in the stamps, far inside the glitch guard.
- `--reorder-every 10`: `frames=1028 packets=1029 gaps=0`. The inverted packet lands inside the
  20 ms reorder window and is resequenced: every datagram arrives and `gaps` is 0. The
  pre-resequencing signature was `frames=925 packets=1028 gaps=102` — an inversion cost a counter
  gap and a discarded packet, which is exactly what resequencing removes. Adding
  `--reorder-delay-ms 50` pushes the arrival *past* the window and is the case the
  `UDPSCOPE_PROBE_RESEQ_MS` readings above cover; it is not this row.
- `--drop-every 1000 --drop-burst 200` at `--rate 100000 --batch 100` (1000 packets a second):
  `frames=7000 packets=7001 gaps=1600`, eight bursts of 200, and still no warning.
- `--drop-every 1000 --drop-burst 300`, same stream: `frames=6203 packets=6204 gaps=2400` **and the
  guard fires** — `warning: ... jumped implausibly far ahead of the previous packet (1.299..1.29999
  against a previous 0.99899)`, then `timing samples by arrival instead`. 300 packet spans is past
  `kGlitchPackets` (256), so real loss is read as a clock glitch; and since each bridge advances the
  anchor by one packet span while the true timeline is 300 ahead, the next four packets fail too and
  `kFailureLimit` throws the stream onto the arrival clock. A burst between 256 packets and the 2 s
  `silence_timeout_s` is the only window in which this is reachable, and it is a finding, not a
  baseline.

Keep a burst under 2 s of wire time whatever the rate: past that the client reconnects, the mock
resets its counter and clock on the new CONNECT, and the run reads `gaps=0` with a *backwards* stamp
warning (`does not advance across packets ... 0..0.0099 against a previous 0.3899`) — the reconnect
path, not the loss path.

**Run one self-check at a time.** Two concurrent `xvfb-run -a` instances end up on the
same display and the second gets no GL context — it dumps `layout over 0 frames` and
`valid=-1`, which looks like a pipeline failure and is not one. `tools/mock_udpstreamer.py`
also outlives its `--duration`, so kill it after each run or the leftovers pile up and
hold ports.

The probes are the regression net — when changing UI or plot-host behaviour, extend
the probe/dump in `src/ui.cpp` rather than adding a separate harness.

## The noise probe needs a spiked mock

Every other probe runs against the plain mock. The noise probe reads what a *band* and an *HF reject* do to
impulsive interference, so its mock must have some: an alternating-sign impulse train of 2 samples at
amplitude 0.5 every 5 ms, added to `sine` alone (so a spiked channel can be read against a clean one in the
same run).

```bash
cd /home/martino/Projects/tools/udpscope
python3 tools/mock_udpstreamer.py --port 8108 --rate 1300000 --batch 1000 --duration 16 \
  --spike-every 0.005 --spike-amp 0.5 --spike-width 2 > /tmp/noise-mock.log 2>&1 &
sleep 1.5
xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=14000 UDPSCOPE_ENABLE_ALL_MS=1000 \
  UDPSCOPE_PROBE_NOISE_MS=6000 \
  ./build/udpscope --server 127.0.0.1 --port 8108 2>&1 | grep -E 'noise-probe|CRITICAL'
```

The probe arms at 6 s rather than the 8 s a single-phase probe would use: its first phase enables the signals
it measures and waits 2.5 s for the ring to refill after that structural change. `--spike-every`/`--spike-amp`/
`--spike-width` are the mock's own; `--batch` must stay at or below 1000 (2000 samples x 5 signals x f64 is
80 kB, past UDP's ~65507-byte payload, and the mock dies on its first `sendto` while the run still exits 0).

The three lines and what each must read are in `probe-readings.md`.

`UDPSCOPE_PROBE_NOISE_MS` and `UDPSCOPE_PROBE_MEASURE_MS` must **not** be scheduled in the same run: both push
onto `app.measures`, and the later one would read the earlier one's rows as its own. The noise probe carries
its own measurement legs (see `probe-readings.md`), so nothing is lost by keeping them apart.