udpscope/docs/claude/verification.md

# Verification

How to run the headless self-check, the two baselines every run is read against, and the environment variables that schedule each probe.

Part of [CLAUDE.md](../../CLAUDE.md).

There is no unit test suite. The only automated check is the app's own headless
self-check, driven by env vars and run under a display (`xvfb-run`, llvmpipe) against
a live UDPStreamer source:

```sh
xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=8000 UDPSCOPE_ENABLE_ALL_MS=1000 \
  ./build/udpscope --server 127.0.0.1 --port 8000
```

Two baselines every run is read against. A clean build emits **four** warnings and no more: two
`-Wmissing-field-initializers` on `MathNode::source` and `-Woverlength-strings` on **both**
generated GResource `.c` files — the help pages' and the icons', whose blob crossed C99's
4095-byte string-literal limit when the two app icons joined the four action ones (a note about a
generated file, which GCC itself supports). The fifth warning that used to appear here was an unused
`output_built` in `math_probe()`; it was **deleted** as dead code — redundant with the
`output_after_add` already printed on the same line — and that deletion is what dropped the baseline
from five to four. Nothing started reading it. And an exit-dump run emits **no** `Gtk-CRITICAL` at
all. A fifth warning, or any critical, is a finding.

An *incremental* build reads **two** of those four: the generated GResource `.c` files are not
recompiled when only `src/` changed, so their `-Woverlength-strings` do not reappear.

That second baseline used to be "exactly two, `gtk_widget_get_height` and `gtk_widget_get_width`,
from the dump asking a destroyed widget for its size", and tolerating them was a mistake: the dump
runs from the destroy handler, so the widget pointer is **freed memory**, and a critical is only
the lucky reading of it. A run that had built and destroyed other windows first —
`UDPSCOPE_PROBE_HELP_MS`, which opens three — has had that memory reused, and the same two calls
then **SIGSEGV in the middle of the dump**, which reads as a truncated dump rather than as a
crash in a debug line. Every size the dump prints is therefore captured on the *tick*, while the
window is alive (`App::dbg_*`), and **no `gtk_widget_*` call may appear in the dump** — the
`panel controls:` line was already written that way for the same reason, and that comment is the
precedent, not an exception.

- `UDPSCOPE_EXIT_AFTER_MS` — run the trigger-on → go-live probe, close the window,
  then dump the whole pipeline state to stdout (frame/sample counts, per-plot painted
  frames, trace bindings, panel sizes, status line in both states).
- `UDPSCOPE_ENABLE_ALL_MS`, `UDPSCOPE_ARM_TRIGGER_MS`, `UDPSCOPE_PROBE_DISARM_MS`,
  `UDPSCOPE_TRIG_TRACE_MS` — schedule individual probes. `UDPSCOPE_PROBE_DISARM_MS` only
  *starts* its sequence: it switches the trigger on and then polls for a capture to open,
  so the Disarm lands ~300 ms into one rather than wherever a fixed delay happened to fall.
- `UDPSCOPE_PROBE_TRIGRENUM_MS` — the trigger's capture must survive a **renumbering of its own
  channel**. It sets the reported geometry through the real widgets (1 s window, 10 % pre share,
  hold-off from `--holdoff`) on the mock's *square* alone, polls for a capture to open, and then
  enables the sine — which sorts before the square and so moves the trigger channel's index under
  the engine. Its three lines are in [probe-readings.md](probe-readings.md); **`t0_held=1` is the
  assertion**. Run it with `--holdoff 10`, which makes the stolen-t0 reading unmistakable: a second
  latch barely a second after the first cannot be a hold-off that applied.
- `UDPSCOPE_PROBE_FIELDS_MS` — change the trigger panel's edge and Normal/Single
  dropdowns. As much a crash test as a state check (see the GTK signal-signature invariant
  below), so time it inside a capture: `[fields-probe] ... capturing=1` followed by no
  SIGSEGV *is* the assertion.
- `UDPSCOPE_PROBE_WINDOW_MS` — drive the window control every way a user can: type a
  number, change the unit under it, step a decade either way.
- `UDPSCOPE_PROBE_LAYOUT_MS` — drive the layout editor: switch it on, insert above the
  first row, remove a row, append one at the bottom, switch it off. Its steps are 200 ms
  apart on purpose — the buttons are placed from the rects the *last frame* was laid out
  with, so a synchronous sequence would read every position one action stale. It clicks by
  emitting `::clicked`, never `gtk_widget_activate()`, which holds a `GtkButton` for its
  250 ms `ACTIVATE_TIMEOUT` and then emits into a button the rebuild has already destroyed.
- `[measure-probe] N measurements on 'X'` (stderr, `UDPSCOPE_PROBE_MEASURE_MS`) — adds one
  measurement of each statistical type (average, top, peak to peak, amplitude) on the first
  *channel* and forces a recompute. It must name a **channel**, not `signals().front()`: the
  first signal of a stream is its timestamp, which is not acquired, and naming it adds four
  measurements that all read `--` *and* makes a performance run look free (each bails before
  collecting anything -- that is how the first baseline of the stall below read 0.4 ms).
  The values are in the exit dump as `measurements: <type>=<value>`; on the fast mock
  (`tools/mock_udpstreamer.py --rate 1300000 --batch 2000`) a unit sine must read
  `peak to peak=2`, `amplitude=1`, `average=0` and **`top=0.636620`** -- 2/pi, the half-wave
  mean of a sine, which is the check that the threshold types are not merely plausible.
  `[measure-trace] view=[..] <type>=<value>... marks=N level:V/time:V...` is a repeating line
  (250 ms) from the probe: the pane's numbers beside the *current* window and the marks actually
  placed, which is the only way a stale result or a mark that should not be drawn shows up as a
  number. It must never print `marks=0` while measurements with `show_on_plot` are on and the
  data is there -- that was the rulers blinking off at a mode change.
  `measure cost: N recomputes, X ms total, Y ms each, data D ms (P%), covered=C buckets=B
  raw=R (F%), N ns/covered sample` follows it: **read it against a run with
  `UDPSCOPE_PROBE_MEASURE_MS` unset**, because a frame mean cannot attribute a cost. The last
  three fields are the assertion, and they are a *ratio*, not a time: `covered` is the samples
  the window spans, `buckets` the LOD buckets the two walks answered from, and `raw` the samples
  they still had to read. **`raw` must stay at a fraction of a percent of `covered`** on a clean
  waveform over a window of millions -- 0.1% is what the pyramid-backed walks give (see the
  measurement-cost invariant in
  [invariants-measurements.md](invariants-measurements.md)). A `raw` near `covered` means the walks went back to reading
  the window, which at 1 MS/s is 19.7 ms per recompute and a third of the run's frames. It is
  *legitimately* higher for a window shorter than a bucket (64 samples) or a signal noisy
  enough to cross a threshold everywhere, so read it on the fast mock
  (`tools/mock_udpstreamer.py --rate 1300000 --batch 2000`), where a window is ~2 M samples.
  `--noise 0.1` there reads **2.0%** at 1.96 ms a recompute against the clean 0.1% at 1.41 ms, and
  that is the interferer's own cost rather than a regression -- the figure to compare a change
  against is the **clean** one.
  Note the **final** `measurements:` line and `[measure-trace]`'s last lines may read `--`:
  `UDPSCOPE_EXIT_AFTER_MS` arms the trigger before it dumps, so the dump can land in `Waiting`
  with nothing bound and the axis at its default. The race-free reading is an earlier
  `[measure-trace]` line, which carries its own `view=`; those lines are also where the
  level-mark fallback is visible (an invalid pass must still print its `level:` marks).

- `[pan-probe] asked=[..] was=[..]`, `[pan-probe] after=[..] held=D asked=[..]`, then
  `[pan-probe] zoom asked=[..] after=[..] span=S window=W limited=L`
  (stderr, `UDPSCOPE_PROBE_PAN_MS`) — **`held` must be 1**: a gesture must not move a window
  that is pinned (trigger mode, or a paused view). The probe writes the panned range the way a
  gesture does — on the *member* axis, which `Figure::layout()` adopts into the group — and
  reports the axis the frame after it produced. Time it *inside* one pinned state: a pan during
  a capture whose check lands after the capture has ended is re-framed by the mode change, not
  by the pin, and reads `held=1` with the hold removed (that is how the first version of this
  probe fooled me). The pause probe makes a stable pin, so run them together:
  `UDPSCOPE_PROBE_PAUSE_MS=1500 UDPSCOPE_PROBE_PAN_MS=2200` puts the pan and its check inside
  the pause. With `hold_pinned_window()` disabled it reads `held=0` and the axis sits at the
  asked range. It then asks for a **zoom-in** (a window a quarter of what is on screen) and
  reports `group=`/`member=` beside it: **`allowed` must be 1, with both equal to the ask** --
  that is the single-trigger-mode case, where the wheel has to keep working, and where reading
  the wrong one of the two made a working clamp look broken. All three verdicts are read from the
  *tick*, one frame after the ask: a `g_timeout` was the first version and it drifted past the
  paused/resumed boundary under load. The probe then asks for a **zoom-out** ten times the current span and reports
  the span the next frame produced: **`limited` must be 1**, i.e. `span <= window`. Run it with
  a ring that holds *more* than the window (`--buffer-mb 64` past the window's own seconds) —
  that is the state the old `max(retained, window_s_)` bound leaked in, and it showed up here as
  `span=2.502550 window=2.000000 limited=0` against `span=2.000000` after the fix.

- `[zoom-probe] <sweep>/x<step> axis=[..] span=.. n=N idx=[a..b) spp=.. level=L pts=P kept=K
  back=.. ink=[..] x_at_first=.. gap_px=.. end_px=.. rect=[..]` (stderr,
  `UDPSCOPE_PROBE_EVENTZOOM_MS`) — arms the trigger in **single** mode, polls for a **held
  event** (a latch happens when the stream says so; a sweep on a timer has nothing to zoom
  into), then zooms in eight doublings
  twice: `ges/` writes the row's own axis the way a gesture does, `win/` drives the window
  control (`apply_time_window`, which unpins and re-arms the live follow before the pin is
  restated). Each step's reading is taken in the *tick*, after that frame's pin and clamp — a
  `g_timeout` would report the ask. It recomputes what `Renderer::draw_traces` is handed, through
  the same calls: `neighbours()` for `idx`, the 64-spp gate for `level`, then `gather()`.
  **`gap_px` and `end_px` must stay within a pixel or two at every step** — they are how much of
  the left and right margin no geometry reaches, and a `gap_px` that *doubles* per step is the
  "zooming into a held event clips it more and more from the left" defect (374 px of a 716 px
  axis by `ges/x128`), whose cause is a constant offset in data seconds (see the `neighbours()`
  invariant in [invariants-rendering.md](invariants-rendering.md)). `kept`/`back` are the renderer's own increasing-suffix filter counted rather
  than applied: `kept` far below `pts` is a staircase, and a non-zero `back` is a non-monotonic
  gather. **Run it with `--clock-jitter`** (`tools/mock_udpstreamer.py --rate 1300000 --batch
  2000 --clock-jitter 0.4`): the ideal mock's stamps sit on a perfect grid, which is exactly
  what hides this class of bug, and three runs against it reported `gap_px=0.0` throughout while
  the defect was live. The jitter varies each *packet's* spacing rather than offsetting its
  start, because the stream's clock has to stay strictly increasing. The `win/` sweep bottoms
  out at `span=0.05` (`RowSet::set_time_window()`'s floor) and walks to the event's post side,
  since `set_time_window()` frames the newest span before the pin clamps it — both are expected.
- `UDPSCOPE_PROBE_PAUSE_MS` — pause acquisition mid-run so layout numbers are
  measured on a still picture. It now also asserts what a pause *is*
  (`[pause-probe] <action> paused=D ring_produced=N event_size=M frozen_x=[..] axis=[..]
  glance=[..]` on stderr):
  it presses the **button**, not `Acquisition::set_paused()`, because pausing is a display
  state as well as an acquisition one and the view snapshot and axis pin live in the
  handler. Across the three lines **`ring_produced` must grow** (the producer never
  stops — it fills the live ring exactly as it does with the trigger off), **`event_size`
  must be non-zero exactly while paused** (the plot is bound to the frozen snapshot, not
  the ring) and **`frozen_x` must not move between the pause and the resume**; after the
  resume `event_size` is 0 and the window is back on the newest data. Pausing used to stop
  the producer, which is why the counts grew by nothing and the resume left a hole.
  `frozen_x` is the *rebased* snapshot — the pause view's own stamps are relative to the freeze
  instant now, so the range reads `[-window..0]`-ish rather than stream seconds, and a
  `frozen_x` in the thousands is the rebase gone.
  **`glance=` is the assertion for the paused status field**, and this is the only line that can
  make it: the exit dump's `status bar:` is read live and its two `mode …` lines are the live and
  triggered states. While paused it must read `paused · <rate> · <window> · for <elapsed>`, with
  the elapsed figure **growing between the two paused lines** and **absent on the resume line**.
  It is **wall** time (`g_get_monotonic_time()`), not stream time, because the producer never
  stops while paused and "how long have I been looking at this" is a question about the user, not
  about the stream; it is written through `format_elapsed()` so the field steps to a coarser unit
  instead of growing digits (see the layout rule).
- `UDPSCOPE_PROBE_COLOR_MS` — recolour one plotted signal and read the colour back out of the
  trace and the chip, round-trip it through the session writer, then clear it. Schedule
  `UDPSCOPE_ENABLE_ALL_MS` before it: with nothing bound it has no trace to recolour and asserts
  nothing. Its four lines are in
  [probe-readings.md](probe-readings.md).
- `UDPSCOPE_EXPORT_MS` (+ optional `UDPSCOPE_EXPORT_PATH`, default
  `/tmp/udpscope-export`) — collect the export table and write it both ways, skipping
  only the file chooser.

- `UDPSCOPE_PROBE_CONDITION_MS` — arms a qualified trigger against the mock and reports the latched
  `t0`, the measured pulse or band width, and sample values for the window condition. Schedules its
  own arm and report stages. `UDPSCOPE_PROBE_CONDITION_SHAPE` picks the leg: `0` pulse width `<`
  (the glitch, the default, needs `--glitch-at`/`--glitch-width`), `1` pulse width `>`, `2` window
  (uses `sine` with a 1.0 s window and 50 % pre), `3` dropout (needs `--quiet-at`; also uses
  `sine` with a 1.0 s window and 50 % pre).
- `UDPSCOPE_PROBE_STATS_MS` — reports every measurement's accumulator (`n`, mean, sd, lo, hi),
  pauses, reports again with `dn=` per measurement, then resets and reports a third time. Three
  phases: `phase=0`, `phase=1` (after pause) and `phase=2` (after Reset). Needs
  `UDPSCOPE_ENABLE_ALL_MS` ahead of it and `UDPSCOPE_PROBE_MEASURE_MS` to add measurements first.
- `UDPSCOPE_PROBE_SEARCH_MS` — runs a buffer search for the armed condition, cross-checks the live
  trigger's own `t0` against the nearest hit (`agree=1` is the load-bearing reading), and jumps to
  hit 0. Reports the resulting `mode=`, `axis=` and `marks placed=`.

The following mock waveform features are what the qualified-condition probes are checked against,
as distinct from the fault injection below, which exercises ingest:

- `--glitch-at S` / `--glitch-width W` invert the `square` signal for `W` seconds at the start of
  every square period from stream second `S` onwards — a runt pulse of a known width, which is
  what a `<` pulse-width trigger and a glitch search are checked against.
- `--quiet-at S` freezes `sine` at its held value from stream second `S` onwards, so it stops
  crossing zero: the dropout trigger's input.

The mock also carries one **fault injection** that no probe schedules, because what it exercises is
the ingest rather than the UI: `--time-glitch-at <stream seconds> --time-glitch-by <seconds>` makes
**one** packet's time signal jump ahead and then carries on from where the true timeline was, which
is what a device whose clock glitches puts on the wire. Run it with the trigger armed, e.g.

```sh
xvfb-run -a env UDPSCOPE_ENABLE_ALL_MS=400 UDPSCOPE_ARM_TRIGGER_MS=700 UDPSCOPE_EXIT_AFTER_MS=22000 \
  ./build/udpscope --server 127.0.0.1 --port 8000 --holdoff 10
# mock: --rate 10000 --batch 100 --freq 5 --time-glitch-at 6.0 --time-glitch-by 10.0
```

Four readings, all off the once-a-second `[udpscope] trig engine:` line:

- exactly one `warning: ... jumped implausibly far ahead of the previous packet (16.01..16.0199
  against a previous 6.0099); bridging that packet ...` — the glitch was **caught**;
- **no** `timing samples by arrival instead` — one bad packet must not change the time source;
- `now_t` crosses the glitch continuously (`5.040 → 6.050 → 7.050`), and `history` keeps **growing**
  (`1.01 … 23.15`) instead of collapsing — the ring was not wiped;
- the latches stay a hold-off apart: `edge at 1.060 / latched at 2.060 / holdoff_until=12.060`, then
  `edge at 12.260 / latched at 13.260 / holdoff_until=23.260` — **11.2 s** for a 10 s hold-off plus
  a 0.9 s post window.

Before the fix the same run read a fallback warning, `history` collapsing 4.03 → 0.95, `now_t`
jumping 5.040 → 16.040, and two latches **1.2 s** apart. Note the window here comes from the saved
session for that port; the figures above are the default 2 s window (50 % pre), so move
`~/.config/udpscope/<host>_<port>.toml` aside to reproduce them.

**Run one self-check at a time.** Two concurrent `xvfb-run -a` instances end up on the
same display and the second gets no GL context — it dumps `layout over 0 frames` and
`valid=-1`, which looks like a pipeline failure and is not one. `tools/mock_udpstreamer.py`
also outlives its `--duration`, so kill it after each run or the leftovers pile up and
hold ports.

The probes are the regression net — when changing UI or plot-host behaviour, extend
the probe/dump in `src/ui.cpp` rather than adding a separate harness.