# Verification How to run the headless self-check, the two baselines every run is read against, and the environment variables that schedule each probe. Part of [CLAUDE.md](../../CLAUDE.md). There is no unit test suite. The only automated check is the app's own headless self-check, driven by env vars and run under a display (`xvfb-run`, llvmpipe) against a live UDPStreamer source: ```sh xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=8000 UDPSCOPE_ENABLE_ALL_MS=1000 \ ./build/udpscope --server 127.0.0.1 --port 8000 ``` Two baselines every run is read against. A clean build emits **four** warnings and no more: two `-Wmissing-field-initializers` on `MathNode::source` and `-Woverlength-strings` on **both** generated GResource `.c` files — the help pages' (**514532** bytes) and the icons', whose blob is **8613** against C99's 4095-byte string-literal limit with the five action icons and the two app icons that remain (a note about a generated file, which GCC itself supports) -- re-measured when `udpscope-math-symbolic` was deleted, which did not move the count; re-measure by touching the two `*.gresource.xml` files and rebuilding. The fifth warning that used to appear here was an unused `output_built` in `math_probe()`; it was **deleted** as dead code — redundant with the `output_after_add` already printed on the same line — and that deletion is what dropped the baseline from five to four. Nothing started reading it. And an exit-dump run emits **no** `Gtk-CRITICAL` at all. A fifth warning, or any critical, is a finding. An *incremental* build reads **two** of those four: the generated GResource `.c` files are not recompiled when only `src/` changed, so their `-Woverlength-strings` do not reappear. A third reading every run offers for free: **how many `[udpscope] buffer allocated:` lines it printed.** The line is not behind a probe — it is in every build — and one line is one *real* change of buffer plan, each of which blocks the main thread for the time it takes to touch the whole `--buffer-mb`. A normal run prints two or three. Fifteen at startup is the defect it was added for; see [probe-readings.md](probe-readings.md). That second baseline used to be "exactly two, `gtk_widget_get_height` and `gtk_widget_get_width`, from the dump asking a destroyed widget for its size", and tolerating them was a mistake: the dump runs from the destroy handler, so the widget pointer is **freed memory**, and a critical is only the lucky reading of it. A run that had built and destroyed other windows first — `UDPSCOPE_PROBE_HELP_MS`, which opens three — has had that memory reused, and the same two calls then **SIGSEGV in the middle of the dump**, which reads as a truncated dump rather than as a crash in a debug line. Every size the dump prints is therefore captured on the *tick*, while the window is alive (`App::dbg_*`), and **no `gtk_widget_*` call may appear in the dump** — the `panel controls:` line was already written that way for the same reason, and that comment is the precedent, not an exception. - `UDPSCOPE_EXIT_AFTER_MS` — run the trigger-on → go-live probe, close the window, then dump the whole pipeline state to stdout (frame/sample counts, per-plot painted frames, trace bindings, panel sizes, status line in both states). - `UDPSCOPE_ENABLE_ALL_MS`, `UDPSCOPE_ARM_TRIGGER_MS`, `UDPSCOPE_PROBE_DISARM_MS`, `UDPSCOPE_TRIG_TRACE_MS` — schedule individual probes. `UDPSCOPE_PROBE_DISARM_MS` only *starts* its sequence: it switches the trigger on and then polls for a capture to open, so the Disarm lands ~300 ms into one rather than wherever a fixed delay happened to fall. - `UDPSCOPE_PROBE_TRIGRENUM_MS` — the trigger's capture must survive a **renumbering of its own channel**. It sets the reported geometry through the real widgets (1 s window, 10 % pre share, hold-off from `--holdoff`) on the mock's *square* alone, polls for a capture to open, and then enables the sine — which sorts before the square and so moves the trigger channel's index under the engine. Its three lines are in [probe-readings.md](probe-readings.md); **`t0_held=1` is the assertion**. Run it with `--holdoff 10`, which makes the stolen-t0 reading unmistakable: a second latch barely a second after the first cannot be a hold-off that applied. - `UDPSCOPE_PROBE_FIELDS_MS` — change the trigger panel's edge and Normal/Single dropdowns. As much a crash test as a state check (see the GTK signal-signature invariant below), so time it inside a capture: `[fields-probe] ... capturing=1` followed by no SIGSEGV *is* the assertion. - `UDPSCOPE_PROBE_WINDOW_MS` — drive the window control every way a user can: type a number, change the unit under it, step a decade either way. - `UDPSCOPE_PROBE_LAYOUT_MS` — drive the layout editor: switch it on, insert above the first row, remove a row, append one at the bottom, switch it off. Its steps are 200 ms apart on purpose — the buttons are placed from the rects the *last frame* was laid out with, so a synchronous sequence would read every position one action stale. It clicks by emitting `::clicked`, never `gtk_widget_activate()`, which holds a `GtkButton` for its 250 ms `ACTIVATE_TIMEOUT` and then emits into a button the rebuild has already destroyed. - `[measure-probe] N measurements on 'X'` (stderr, `UDPSCOPE_PROBE_MEASURE_MS`) — adds one measurement of each statistical type (average, top, peak to peak, amplitude) on the first *channel* and forces a recompute. It must name a **channel**, not `signals().front()`: the first signal of a stream is its timestamp, which is not acquired, and naming it adds four measurements that all read `--` *and* makes a performance run look free (each bails before collecting anything -- that is how the first baseline of the stall below read 0.4 ms). The values are in the exit dump as `measurements: =`; on the fast mock (`tools/mock_udpstreamer.py --rate 1300000 --batch 2000`) a unit sine must read `peak to peak=2`, `amplitude=1`, `average=0` and **`top=0.636620`** -- 2/pi, the half-wave mean of a sine, which is the check that the threshold types are not merely plausible. `[measure-trace] view=[..] =... marks=N level:V/time:V...` is a repeating line (250 ms) from the probe: the pane's numbers beside the *current* window and the marks actually placed, which is the only way a stale result or a mark that should not be drawn shows up as a number. It must never print `marks=0` while measurements with `show_on_plot` are on and the data is there -- that was the rulers blinking off at a mode change. `measure cost: N recomputes, X ms total, Y ms each, data D ms (P%), covered=C buckets=B raw=R (F%), N ns/covered sample` follows it: **read it against a run with `UDPSCOPE_PROBE_MEASURE_MS` unset**, because a frame mean cannot attribute a cost. The last three fields are the assertion, and they are a *ratio*, not a time: `covered` is the samples the window spans, `buckets` the LOD buckets the two walks answered from, and `raw` the samples they still had to read. **`raw` must stay at a fraction of a percent of `covered`** on a clean waveform over a window of millions -- 0.1% is what the pyramid-backed walks give (see the measurement-cost invariant in [invariants-measurements.md](invariants-measurements.md)). A `raw` near `covered` means the walks went back to reading the window, which at 1 MS/s is 19.7 ms per recompute and a third of the run's frames. It is *legitimately* higher for a window shorter than a bucket (64 samples) or a signal noisy enough to cross a threshold everywhere, so read it on the fast mock (`tools/mock_udpstreamer.py --rate 1300000 --batch 2000`), where a window is ~2 M samples. `--noise 0.1` there reads **2.0%** at 1.96 ms a recompute against the clean 0.1% at 1.41 ms, and that is the interferer's own cost rather than a regression -- the figure to compare a change against is the **clean** one. Note the **final** `measurements:` line and `[measure-trace]`'s last lines may read `--`: `UDPSCOPE_EXIT_AFTER_MS` arms the trigger before it dumps, so the dump can land in `Waiting` with nothing bound and the axis at its default. The race-free reading is an earlier `[measure-trace]` line, which carries its own `view=`; those lines are also where the level-mark fallback is visible (an invalid pass must still print its `level:` marks). - `[pan-probe] asked=[..] was=[..]`, `[pan-probe] after=[..] held=D asked=[..]`, then `[pan-probe] zoom asked=[..] after=[..] span=S window=W limited=L` (stderr, `UDPSCOPE_PROBE_PAN_MS`) — **`held` must be 1**: a gesture must not move a window that is pinned (trigger mode, or a paused view). The probe writes the panned range the way a gesture does — on the *member* axis, which `Figure::layout()` adopts into the group — and reports the axis the frame after it produced. Time it *inside* one pinned state: a pan during a capture whose check lands after the capture has ended is re-framed by the mode change, not by the pin, and reads `held=1` with the hold removed (that is how the first version of this probe fooled me). The pause probe makes a stable pin, so run them together: `UDPSCOPE_PROBE_PAUSE_MS=1500 UDPSCOPE_PROBE_PAN_MS=2200` puts the pan and its check inside the pause. With `hold_pinned_window()` disabled it reads `held=0` and the axis sits at the asked range. It then asks for a **zoom-in** (a window a quarter of what is on screen) and reports `group=`/`member=` beside it: **`allowed` must be 1, with both equal to the ask** -- that is the single-trigger-mode case, where the wheel has to keep working, and where reading the wrong one of the two made a working clamp look broken. All three verdicts are read from the *tick*, one frame after the ask: a `g_timeout` was the first version and it drifted past the paused/resumed boundary under load. The probe then asks for a **zoom-out** ten times the current span and reports the span the next frame produced: **`limited` must be 1**, i.e. `span <= window`. Run it with a ring that holds *more* than the window (`--buffer-mb 64` past the window's own seconds) — that is the state the old `max(retained, window_s_)` bound leaked in, and it showed up here as `span=2.502550 window=2.000000 limited=0` against `span=2.000000` after the fix. - `[zoom-probe] /x axis=[..] span=.. n=N idx=[a..b) spp=.. level=L pts=P kept=K back=.. ink=[..] x_at_first=.. gap_px=.. end_px=.. rect=[..]` (stderr, `UDPSCOPE_PROBE_EVENTZOOM_MS`) — arms the trigger in **single** mode, polls for a **held event** (a latch happens when the stream says so; a sweep on a timer has nothing to zoom into), then zooms in eight doublings twice: `ges/` writes the row's own axis the way a gesture does, `win/` drives the window control (`apply_time_window`, which unpins and re-arms the live follow before the pin is restated). Each step's reading is taken in the *tick*, after that frame's pin and clamp — a `g_timeout` would report the ask. It recomputes what `Renderer::draw_traces` is handed, through the same calls: `neighbours()` for `idx`, the 64-spp gate for `level`, then `gather()`. **`gap_px` and `end_px` must stay within a pixel or two at every step** — they are how much of the left and right margin no geometry reaches, and a `gap_px` that *doubles* per step is the "zooming into a held event clips it more and more from the left" defect (374 px of a 716 px axis by `ges/x128`), whose cause is a constant offset in data seconds (see the `neighbours()` invariant in [invariants-rendering.md](invariants-rendering.md)). `kept`/`back` are the renderer's own increasing-suffix filter counted rather than applied: `kept` far below `pts` is a staircase, and a non-zero `back` is a non-monotonic gather. **Run it with `--clock-jitter`** (`tools/mock_udpstreamer.py --rate 1300000 --batch 2000 --clock-jitter 0.4`): the ideal mock's stamps sit on a perfect grid, which is exactly what hides this class of bug, and three runs against it reported `gap_px=0.0` throughout while the defect was live. The jitter varies each *packet's* spacing rather than offsetting its start, because the stream's clock has to stay strictly increasing. The `win/` sweep bottoms out at `span=0.05` (`RowSet::set_time_window()`'s floor) and walks to the event's post side, since `set_time_window()` frames the newest span before the pin clamps it — both are expected. - `UDPSCOPE_PROBE_PAUSE_MS` — pause acquisition mid-run so layout numbers are measured on a still picture. It now also asserts what a pause *is* (`[pause-probe] paused=D ring_produced=N event_size=M frozen_x=[..] axis=[..] skips=N glance=[..]` on stderr): it presses the **button**, not `Acquisition::set_paused()`, because pausing is a display state as well as an acquisition one and the view snapshot and axis pin live in the handler. Across the three lines **`ring_produced` must grow** (the producer never stops — it fills the live ring exactly as it does with the trigger off), **`event_size` must be non-zero exactly while paused** (the plot is bound to the frozen snapshot, not the ring) and **`frozen_x` must not move between the pause and the resume**; after the resume `event_size` is 0 and the window is back on the newest data. Pausing used to stop the producer, which is why the counts grew by nothing and the resume left a hole. `frozen_x` is the *rebased* snapshot — the pause view's own stamps are relative to the freeze instant now, so the range reads `[-window..0]`-ish rather than stream seconds, and a `frozen_x` in the thousands is the rebase gone. **`glance=` is the assertion for the paused status field**, and this is the only line that can make it: the exit dump's `status bar:` is read live and its two `mode …` lines are the live and triggered states. While paused it must read `paused · · · for `, with the elapsed figure **growing between the two paused lines** and **absent on the resume line**. It is **wall** time (`g_get_monotonic_time()`), not stream time, because the producer never stops while paused and "how long have I been looking at this" is a question about the user, not about the stream; it is written through `format_elapsed()` so the field steps to a coarser unit instead of growing digits (see the layout rule). **`skips=` is the assertion for the render gate** (the end of `tick()`), cumulative over the session: the `live` line must read **0** and the `paused` line must read **more than 0**. A frozen view is bound to an immutable snapshot on a pinned axis, so a frame whose picture cannot have changed is not drawn again — and this is the only line that can say whether that happened, since a skipped frame is invisible in every other reading by construction. Non-zero on the `live` line is the gate's condition inverted, which would cost dropped samples' worth of picture; 0 on the `paused` line is the gate not engaging and the saving not being made. The gate cannot stall the picture (`kStaticRenderEvery` renders every few ticks regardless), so the renderer's own baselines below are unchanged by it: a measured run read `skips=0`, then 33, then 45, with `columns == graded`, `render blank frames: 0`, and the layout floor still converged (`y-gutter 104..104`). - `UDPSCOPE_PROBE_COLOR_MS` — recolour one plotted signal and read the colour back out of the trace and the chip, round-trip it through the session writer, then clear it. Schedule `UDPSCOPE_ENABLE_ALL_MS` before it: with nothing bound it has no trace to recolour and asserts nothing. Its four lines are in [probe-readings.md](probe-readings.md). - `UDPSCOPE_EXPORT_MS` (+ optional `UDPSCOPE_EXPORT_PATH`, default `/tmp/udpscope-export`) — collect the export table and write it both ways, skipping only the file chooser. - `UDPSCOPE_PROBE_CONDITION_MS` — arms a qualified trigger against the mock and reports the latched `t0`, the measured pulse or band width, and sample values for the window condition. Schedules its own arm and report stages. `UDPSCOPE_PROBE_CONDITION_SHAPE` picks the leg: `0` pulse width `<` (the glitch, the default, needs `--glitch-at`/`--glitch-width`), `1` pulse width `>` (uses a 1.0 s window and 50 % pre so the opening edge of a 0.2 s pulse is captured), `2` window (uses `sine` with a 0.2 s window and 10 % pre), `3` dropout (needs `--quiet-at`; uses `sine` with a 1.0 s window and 50 % pre so the last crossing before the outage is captured). - `UDPSCOPE_PROBE_STATS_MS` — reports every measurement's accumulator (`n`, mean, sd, lo, hi), pauses, reports again with `dn=` per measurement, then resets and reports a third time. Three phases: `phase=0`, `phase=1` (after pause) and `phase=2` (after Reset). Needs `UDPSCOPE_ENABLE_ALL_MS` ahead of it and `UDPSCOPE_PROBE_MEASURE_MS` to add measurements first. - `UDPSCOPE_PROBE_SIDE_TAB_MS=3000` with `UDPSCOPE_EXIT_AFTER_MS=11000` — the measurements on the Measure tab, a forced pass, then the tab switched away and read again two seconds later. **`advanced=4` is the assertion** (every accumulator gained passes while its page was off screen); `advanced=0` is the defect where the accumulation is gated on the visible tab, which would leave a gap in min/max/avg for exactly as long as the trigger was on screen. It enables *and plots* its own signal, so it needs no other probe ahead of it. The reading's own entry is in `probe-readings.md`. - `UDPSCOPE_PROBE_SEARCH_MS` — runs a buffer search for the armed condition, cross-checks the live trigger's own `t0` against the nearest hit (`agree=1` is the load-bearing reading), and jumps to hit 0. Reports the resulting `mode=`, `axis=` and `marks placed=`. The following mock waveform features are what the qualified-condition probes are checked against, as distinct from the fault injection below, which exercises ingest: - `--glitch-at S` / `--glitch-width W` invert the `square` signal for `W` seconds at the start of every square period from stream second `S` onwards — a runt pulse of a known width, which is what a `<` pulse-width trigger and a glitch search are checked against. - `--quiet-at S` freezes `sine` at its held value from stream second `S` onwards, so it stops crossing zero: the dropout trigger's input. - `UDPSCOPE_PROBE_RESEQ_MS` — one mid-run reading of the resequencing path. Five cases, **run each as its own process** — five back to back starve the mock past its 2 s silence timeout and every case then reads a reconnect instead of what it was testing. Common conditions for the first four: the mock at `--rate 10000 --batch 100 --freq 5`, udpscope with no extra options, under `xvfb-run` with `UDPSCOPE_EXIT_AFTER_MS=8000 UDPSCOPE_ENABLE_ALL_MS=600 UDPSCOPE_EXIT_DUMP=1 UDPSCOPE_PROBE_RESEQ_MS=5000`. Delete `~/.config/udpscope/127.0.0.1_8000.toml` first: a session left by an earlier run restores its math channel, and five channels instead of four change `channels=`, `span=` and every count derived from them. | case | mock extra flags | key reading | | --- | --- | --- | | **base** | *(none)* | every counter 0; proves the machinery costs nothing | | **reorder** | `--reorder-every 10` | `reseq=51` (mid-run) / `102` (dump); `holes=0`; resequencing inside the window leaves no reservation | | **late** | `--reorder-every 10 --reorder-delay-ms 50` | `bands_repaired` one short of `bands` with `nan_slots=100`; the band still solid is the one hole whose late packet is in flight, and it is the *only* unrepaired one | | **mixed** | `--drop-every 20 --reorder-every 10 --reorder-delay-ms 50` | `bands=43 bands_repaired=20 nan_slots=2200`; all three band states at once | | **re-plan** | `--rate 100000` with the `mixed` flags, and `UDPSCOPE_PROBE_WINDOW_MS=4000 UDPSCOPE_PROBE_RESEQ_MS=6000 UDPSCOPE_EXIT_AFTER_MS=9000` | four `buffer rebuilt` lines before the reading, then `bands=64 bands_repaired=16 nan_slots=7800`: **a band is still only marked repaired when its own slots hold values**. A `bands_repaired == bands` beside a non-zero `nan_slots` here is the hole table and the producer's `tick_` surviving a re-plan that reset `produced()` — a band marked settled over samples that are all still NaN. The non-zero `overflows` is this case too: 100 kS/s with a 50 ms reorder delay keeps about 50 packets in flight against 64 slots | The re-plan case needs `--rate 100000`, not the other four's 10 000: at 10 kS/s the ring - `UDPSCOPE_PROBE_CAPTURE_MS` — toggles the packet capture **through the same handler the status-bar button uses**, waits, then reads the file back off disk and prints one line: `enabled= path= slots= stored= resident= cursor= torn= truncated= header_ok= first_seq= last_seq= payload= file= open_ms= t0=`. `stored` is the datagrams-since-open counter and must equal the dump's datagram count from the toggle's instant; `resident` is what the file holds, which is the slot count once the ring has wrapped; `file` is the reserved size `64 + slot_count * slot_bytes` and is a constant for a given N, so a mismatch against the file on disk is a reading. Two cases, **run each as its own process**: 1. **enable at runtime** — the mock at `--rate 20000 --batch 200`, udpscope with `UDPSCOPE_ENABLE_ALL_MS=700 UDPSCOPE_PROBE_CAPTURE_MS=1500 UDPSCOPE_EXIT_AFTER_MS=9000` and no capture flag: `enabled=1 header_ok=1 torn=0 truncated=0 first_seq=1`, `stored` equal to the datagrams since the toggle, `open_ms` a few ms. 2. **wrap** — the same with `--capture-packets=16` and `UDPSCOPE_PROBE_CAPTURE_MS=7000 UDPSCOPE_EXIT_AFTER_MS=10000`: `stored` (870 in the reference run) exceeds `slot_count`, `resident=16`, `last_seq - first_seq == 15`, `torn=0`, and `cursor` agrees with `seq` (counter mod slot count). A wrap that loses ordering, or a `cursor` that disagrees, shows here and nowhere else. Then the converter over the wrap case's file: `tools/capture_to_pcap.py out.pcap` must exit 0, print a record count equal to `slot_count` with `torn` 0, and the pcap's first and last record timestamps must fall inside the run's wall-clock interval — both checked by a `struct.unpack` of the pcap headers, nothing outside the standard library. - **A gap must not kill a filtered trace.** `--relative-time` with the mock dropping (`--drop-every 20`) and `UDPSCOPE_PROBE_MATH_MS=2500 UDPSCOPE_ENABLE_ALL_MS=600 UDPSCOPE_EXIT_AFTER_MS=9000`: every derived channel's dump line `math '': finite=N/4000` must read about 3800 — the tail minus the reserved holes — and **`finite=0` beside a non-zero `hole_slots` is the defect** (a biquad that integrated the hole's NaN, permanently). The filters' `sd` must stay near the no-drop value; run the same command without `--drop-every` as the control, where every `finite` reads 4000. - `UDPSCOPE_PROBE_CONFIGS_MS` — saves the running session as a configuration named `probe` with a note, through `save_configuration_as()` (the same entry point the dialog's button uses), then reads it back off disk. Reading: `save-as=1`; `names=` gains `probe`; `active=probe`; and the round trip must carry the note **and** the startup settings — `comment='written by the configs probe' buffer_mb=8 relative=0 capture=0`. A `buffer_mb=0` there is the per-source buffer size not being written, which is the one setting a user notices only after a capture has been sized wrong. Run it with a throwaway `XDG_CONFIG_HOME`; the probe deletes its own configuration at the end (`after forget names=…`). - `UDPSCOPE_PROBE_ROWMENU_MS` — gives row 0 a trace, opens the plot's context menu with `open_row_menu()` (the one entry point `on_right_click()` wraps) and walks its two views one step at a time (`main` → click **Remove signal** → `remove` → click **Back** → `after-back` → click **Remove signal** again → `second-open` → click a trace), measuring each view's rows: `[rowmenu] N '