0 Fork 0

Files

T

Martino FerrariandClaude Opus 4.6 4d859d6a37 docs: the source-plugin framework, the --source options and the source-kinds probe

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-10-31 20:22:22 +01:00

74 KiB

Raw Blame History

Verification

How to run the headless self-check, the two baselines every run is read against, and the environment variables that schedule each probe.

Part of CLAUDE.md.

There is no unit test suite. The only automated check is the app's own headless self-check, driven by env vars and run under a display (xvfb-run, llvmpipe) against a live UDPStreamer source:

xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=8000 UDPSCOPE_ENABLE_ALL_MS=1000 \
  ./build/udpscope --server 127.0.0.1 --port 8000

Two baselines every run is read against. The UDPS runs must be unchanged by the source-plugin work: a UDPS self-check, number-normalised (sed 's/[0-9.]\+/N/g'), diffs empty against one taken before it. A clean build emits three warnings and no more: two -Wmissing-field-initializers on MathNode::source and one -Woverlength-strings on the icons' generated GResource .c, whose blob is 12352 against C99's 4095-byte string-literal limit with the seven action icons and the two app icons that remain (a note about a generated file, which GCC itself supports). The fourth was the help pages' generated .c (516044 bytes); the manual is now a Mallard set installed for Yelp rather than a GResource compiled into the binary, so that bundle — and its warning — is gone, and the baseline moved from four to three. Re-measure by touching each *.gresource.xml under src/ and rebuilding; a third -Wmissing-field-initializers or any other warning is a finding. (The fifth warning that used to appear here was an unused output_built in math_probe(); it was deleted as dead code — redundant with the output_after_add already printed on the same line — which dropped the baseline from five to four.) And an exit-dump run emits no Gtk-CRITICAL at all, nor any Gtk-WARNING: the dump's gtk warnings: line counts GTK's own log (through a writer function — GTK is built with structured logging, so the handler API never sees its messages) and must read 0; see probe-readings.md.

An incremental build reads one of those three: the generated GResource .c is not recompiled when only src/ changed, so its -Woverlength-strings does not reappear.

A third reading every run offers for free: how many [udpscope] buffer allocated: lines it printed. The line is not behind a probe — it is in every build — and one line is one real change of buffer plan, each of which blocks the main thread for the time it takes to touch the whole --buffer-mb. A normal run prints two or three. Fifteen at startup is the defect it was added for; see probe-readings.md.

That second baseline used to be "exactly two, gtk_widget_get_height and gtk_widget_get_width, from the dump asking a destroyed widget for its size", and tolerating them was a mistake: the dump runs from the destroy handler, so the widget pointer is freed memory, and a critical is only the lucky reading of it. A run that had built and destroyed other windows first — UDPSCOPE_PROBE_HELP_MS, which opens the About and shortcut dialogs and then closes them — has had that memory reused, and the same two calls then SIGSEGV in the middle of the dump, which reads as a truncated dump rather than as a crash in a debug line. Every size the dump prints is therefore captured on the tick, while the window is alive (App::dbg_*), and no gtk_widget_* call may appear in the dump — the panel controls: line was already written that way for the same reason, and that comment is the precedent, not an exception.

UDPSCOPE_EXIT_AFTER_MS — run the trigger-on → go-live probe, close the window, then dump the whole pipeline state to stdout (frame/sample counts, per-plot painted frames, trace bindings, panel sizes, status line in both states).
UDPSCOPE_ENABLE_ALL_MS, UDPSCOPE_ARM_TRIGGER_MS, UDPSCOPE_PROBE_DISARM_MS, UDPSCOPE_TRIG_TRACE_MS — schedule individual probes. UDPSCOPE_PROBE_DISARM_MS only starts its sequence: it switches the trigger on and then polls for a capture to open, so the Disarm lands ~300 ms into one rather than wherever a fixed delay happened to fall.
UDPSCOPE_PROBE_FORCE_MS — the trigger panel's Force button: one acquisition at the present moment, with the pre/post window the panel is set to, without waiting for the condition. It is a chain, not an instant — MS is when it starts: it forces trigger mode off through the toggle (a session file remembers the trigger, so the off state is stated rather than assumed) and reads the button's stated state 300 ms later; it then sets a level no mock signal reaches (9 V), a 2 s window at 50 % pre, acquires the mock's sine and turns the trigger on; it presses Force 2.5 s after that (the window write re-plans the buffer and clears the history, so the ring needs the time to fill past the 1 s leg A asks for); it presses a second time on a 100 s window at 99 % pre — a request no run this short can satisfy — 400 ms after the first latch. Run it alone, against the plain mock, with time for both legs:
```
python3 tools/mock_udpstreamer.py --port 8114 --rate 10000 --batch 100 --duration 30 &
sleep 1.5
xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=14000 UDPSCOPE_PROBE_FORCE_MS=3000 \
  ./build/udpscope --server 127.0.0.1 --port 8114 2>&1 | \
  grep -E 'force-probe|trigger:'
```
Being alone is load-bearing twice: the level is chosen so the engine cannot latch on its own, so a latch count that moves is the force's and nothing else's — any other probe that arms a reachable condition adds latches of its own — and UDPSCOPE_ENABLE_ALL_MS before it would acquire four signals, which changes the level's default range and the plan. Two lines per pass are in probe-readings.md; latches=P->Q with Q == P + 1, display=event(frozen), matches_request=1 in leg A, pin_is_snapshot=1 in both are the assertions, and leg B's matches_request=0 is expected — a ring that cannot reach back 99 s is the only state where "pinned to the snapshot, not to the request" is a reading rather than a coincidence.
UDPSCOPE_PROBE_PERSIST_MS (+ UDPSCOPE_PROBE_PERSIST_DEPTH, default 4, 1..8) — the persistence display: trigger mode keeps the last N acquisitions on screen, each older one drawn at half the previous one's brightness. It enables the signals itself (one call, as a restored session does) and arranges one signal per row — the arrangement the app gives an opened file, and the one with room for every age — sets the depth through the panel's own spin, arms the trigger on the mock's square and polls until the retention is full: an event on screen and as many older ones kept as the depth asks for. The depth is left where it put it, so the run's own tail is the display at that depth:
```
python3 tools/mock_udpstreamer.py --port 9541 --rate 500000 --batch 1000 &
PY=$!; sleep 0.5
xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=16000 \
  UDPSCOPE_PROBE_PERSIST_MS=2500 UDPSCOPE_PROBE_PERSIST_DEPTH=4 \
  ./build/udpscope --server 127.0.0.1 --port 9541 2>&1 | grep -E 'persist-probe|persistence:|layout:'
kill $PY
```
Not with UDPSCOPE_ENABLE_ALL_MS: that probe ends by switching a signal off two seconds later, which takes the row that plotted it — and its layers with it — out of the middle of the measurement.

Read it as a pair of runs: the same command at ..._PERSIST_DEPTH=1 is the display without the feature, so its kept=0, layers=0, bytes=0 and its layout: line holding only the signals is the "N = 1 is exactly the old behaviour" reading, and it is also the run the exit dump's frame cost: line is compared against — the tick/draw means are means over the whole run, and the depth is raised at MS, so the difference is a lower bound on what the layers cost. ev_is_snap and pin_is_snap must be 1 in both runs: the older acquisitions are drawn and never pinned, so the axis, the framing and the measurements still describe the current event. alphas= is the decay schedule in use, oldest first — halves, and 1/2^age at that. Both lines and the two exit-dump lines are in probe-readings.md.
UDPSCOPE_PROBE_DENSITY_MS (+ UDPSCOPE_PROBE_DENSITY_TAU, default 1.0 s) — the phosphor density: an intensity image per trace, in that trace's own colour, the axes rect in device pixels up to the plotter's kDensityCellCap (past which it scales the rect down uniformly and magnifies the image back over it with a linear filter), every triggered acquisition accumulated into it (the newest included), drawn in place of the traces, fading by a per-acquisition factor or by a time constant. It states the display off through the panel's own switch first, so the off leg is the run without it, then arms the trigger on the mock's square with one signal per row — the arrangement the persistence probe uses — and only then switches it on, so the acquisition already on screen is the first image. Four legs, each a separate g_timeout, so whole frames happen between them:
```
python3 tools/mock_udpstreamer.py --port 9551 --rate 500000 --batch 1000 &
PY=$!; sleep 0.5
xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=14000 \
  UDPSCOPE_PROBE_DENSITY_MS=2500 ./build/udpscope --server 127.0.0.1 --port 9551 2>&1 | \
  grep -E 'density-probe|density:'
kill $PY
```
The assertions, in order: off reads rows=0 traces=0 cells=0 ink=0.0 peak=0.0000 (there is no grid at all); on reads rows=4 — every row has its own image, one per trace in its own colour, so traces=4 with one signal a row — and peak=1.0000, one sweep's ink being one per cell it covers; step-acquisitions reads peak=1.5000, which is the panel's 0.5 applied to the old image plus the new sweep's own pass; and step-time reads ratio=0.367879, one step of dt = tau through the plot's own decay_density(). A peak in the tens for the on leg is the per-hit deposit -- one sweep counted once per sample that crossed a cell -- which saturates a single acquisition and hides the sweeps behind it. The line also carries the app's own A/B of the frame cost (tick/draw before and after the switch, in one run), and the exit dump's density: line says what the last frame it was on held. Both lines and the dump line are in probe-readings.md.
UDPSCOPE_PROBE_NOTRACE (no value; not a scheduled probe, and off unless it is set) — the per-paint detail behind the exit dump's traceless paints: counter: one [notrace] line for each of the first 40 paints after the first bind that staged no trace, naming the mode, the axis the frame was asked for and every slot's bounds. It is behind a variable because a state that draws nothing lasts as long as it lasts, so an ungated line is one per frame for the whole time -- and it is not printed for a frame a phosphor density is drawing, which stages no trace by construction (the frame's ink is the image) and would otherwise be counted as well. The counter itself is in every build; see probe-readings.md.
UDPSCOPE_PROBE_HEADER (no value; not a scheduled probe) — the once-a-second [udpscope] header: window_button_w=… bar_w=… line. The widths are sampled either way (the exit dump reads them); the line is behind this variable because it is detail behind that reading, not a line every run should print. An unallocated window reads 0 for both.
UDPSCOPE_PROBE_RENDERER (no value; not a scheduled probe) — the renderer pair --renderer should have exported, printed from main() before GTK initialises ([renderer] option=… GSK_RENDERER=… LIBGL_ALWAYS_SOFTWARE=…), then the renderer GTK built ([renderer] realized=…) once the window has one. Run it with --renderer cairo to check the option reaches the environment before gtk_init()/adw_init(); run it without the option to check the default is untouched.
UDPSCOPE_PROBE_TRIGRENUM_MS — the trigger's capture must survive a renumbering of its own channel. It sets the reported geometry through the real widgets (1 s window, 10 % pre share, hold-off from --holdoff) on the mock's square alone, polls for a capture to open, and then enables the sine — which sorts before the square and so moves the trigger channel's index under the engine. Its three lines are in probe-readings.md; t0_held=1 is the assertion. Run it with --holdoff 10, which makes the stolen-t0 reading unmistakable: a second latch barely a second after the first cannot be a hold-off that applied.
UDPSCOPE_PROBE_FIELDS_MS — change the trigger panel's edge and Normal/Single dropdowns. As much a crash test as a state check (see the GTK signal-signature invariant below), so time it inside a capture: [fields-probe] ... capturing=1 followed by no SIGSEGV is the assertion.
UDPSCOPE_PROBE_WINDOW_MS — drive the window control every way a user can: type a number, change the unit under it, step a decade either way.
UDPSCOPE_PROBE_LAYOUT_MS — drive the layout editor: switch it on, insert above the first row, remove a row, append one at the bottom, switch it off. Its steps are 200 ms apart on purpose — the buttons are placed from the rects the last frame was laid out with, so a synchronous sequence would read every position one action stale. It clicks by emitting ::clicked, never gtk_widget_activate(), which holds a GtkButton for its 250 ms ACTIVATE_TIMEOUT and then emits into a button the rebuild has already destroyed.
[levels-probe] N measurement(s) over K channel(s), shares='...' -> W window(s), L level walk(s) (stderr, UDPSCOPE_PROBE_LEVELS_MS) — prices a measurement pane, which is what a report of "it is slow with many measurements" has to become before it is a number. It adds one above per share in UDPSCOPE_PROBE_LEVELS (comma separated, default 50), spread in turn over the first UDPSCOPE_PROBE_LEVELS_CHANNELS channels (default 1), and reports the two things a count does not name: how many windows (one MeasureWindowReader, one fetch, one scan per measured channel) and how many level walks (one accumulator per distinct share) the pass then had to take. 50,50,50,50 on one channel is four pane rows and one walk; 5,10,15,20 is four rows and four; the same four shares over four channels is four windows and four walks. Read the measure cost: line it leaves in the exit dump and the sweep in invariants-measurements.md — and read it against a run with UDPSCOPE_PROBE_LEVELS_MS unset, because the pane's cost is a difference between two tick means. Run it at the scale being asked about: the cost is the window's, so a 2 s window prices a 15 s one at nothing (four measurements over four channels read 35.69 ms a recompute and a 4.58 ms tick mean at a 15 s window against 6.58 ms/1.34 ms for one window at the same window).
[measure-probe] N measurements on 'X' (stderr, UDPSCOPE_PROBE_MEASURE_MS) — adds one measurement of each statistical type (average, top, peak to peak, amplitude) on the first channel and forces a recompute. It must name a channel, not signals().front(): the first signal of a stream is its timestamp, which is not acquired, and naming it adds four measurements that all read -- and makes a performance run look free (each bails before collecting anything -- that is how the first baseline of the stall below read 0.4 ms). The values are in the exit dump as measurements: <type>=<value>; on the fast mock (tools/mock_udpstreamer.py --rate 1300000 --batch 2000) a unit sine must read peak to peak=2 exactly, average=0, and the two flat-level types as their definition says: top=0.957988 and amplitude=1.915983. top is no longer 2/pi -- it is the flat level of the upper side, read from the density of the window's buckets, and a sine has no plateau so the densest band is at its peak; amplitude is top - bottom (a scope's Vamplitude), not the half-swing it once was. The check that the flat levels are levels and not extremes is the same probe against the mock's square (--square-harmonics 30, the control being 0): top/bottom 0.999983/-0.999983 rippled against 1.000000/-1.000000 ideal, amplitude 1.999966 against 2.000000, while peak to peak reads 2.358701 rippled against 2.000000 ideal -- the ±1.179 edges' overshoot, kept by the extremes and excluded by the levels. The per-type values, the reason for each, and the cost of the histogram are in invariants-measurements.md. [measure-trace] mode=M view=[..] s0=S swin=[..] pin=[..|off] for=[..] passes=N <type>=<value>... marks=N level:V/time:V... is a repeating line (250 ms) from the probe: the pane's numbers beside the current window and the marks actually placed, which is the only way a stale result or a mark that should not be drawn shows up as a number. swin is view moved onto the stream's clock by s0 (stream_at_x0()), pin is the pin's own extent in that clock (off when nothing is pinned) and for is the window the printed numbers were computed over -- so a reading that moved while view stood still is a change of s0 (a new snapshot under the same mode) and not a stale pass, and only these fields tell those apart. passes beside them says whether a recompute answered the move. While the view is pinned, for must equal pin -- and swin is free to sit anywhere inside it -- through any number of zooms and pans: a latched event's numbers are the frame's, so a gesture inside the pin changes neither them nor passes. Read the other way round it is the defect: for following swin into the zoom while the values move is the reported "on a still frame the measurements change when I zoom". Neither reading is the old "measurements stand still while the data moves", which was for=[0.47..2.47] against a swin=[1.417..1.467] with passes frozen -- the pane describing a window the plot had left. The line must also never print marks=0 while measurements with show_on_plot are on and the data is there -- that was the rulers blinking off at a mode change. pin=off while the mode is held is not a stale reading but the window control: apply_time_window() unpins, and for those frames the view is live-framed and measuring the viewport is right. measure cost: N recomputes, X ms total, Y ms each, data D ms (P%), covered=C buckets=B raw=R (F%), N ns/covered sample follows it: read it against a run with UDPSCOPE_PROBE_MEASURE_MS unset, because a frame mean cannot attribute a cost. The last three fields are the assertion, and they are a ratio, not a time: covered is the samples the window spans, buckets the LOD buckets the two walks answered from, and raw the samples they still had to read. raw must stay at a fraction of a percent of covered on a clean waveform over a window of millions -- 0.1% is what the pyramid-backed walks give (see the measurement-cost invariant in invariants-measurements.md). A raw near covered means the walks went back to reading the window, which at 1 MS/s is 19.7 ms per recompute and a third of the run's frames. It is legitimately higher for a window shorter than a bucket (64 samples) or a signal noisy enough to cross a threshold everywhere, so read it on the fast mock (tools/mock_udpstreamer.py --rate 1300000 --batch 2000), where a window is ~2 M samples. --noise 0.1 there reads 2.0% at 1.96 ms a recompute against the clean 0.1% at 1.41 ms, and that is the interferer's own cost rather than a regression -- the figure to compare a change against is the clean one. raw is also the guard on the flat-level types, which are the one thing that reads a second time: top/bottom/amplitude fold a histogram out of the same fetched bucket list, so they add buckets and not samples (measured: raw=0.1% and 2.1% with the interferer, against 0.1%/2.1% before those types had a flat level, for 0.44→0.50 ms and 0.52→0.65 ms a pass -- the buckets are the budget). It is the guard on the band averages for the same reason: between is a second level in the same walk, not a second pass, and measured on the fast mock at a 2 s window with the measurements probe adding one of every type, raw stayed at a fraction of a percent of covered — covered=240719031 buckets=3760828 raw=113966 (0.0%) for the ten types before, against covered=255372033 buckets=3989978 raw=134978 (0.1%) for the thirteen after, at 0.8 → 1.1 ns/covered sample (data 156.85 → 178.38 ms: the pane holds three more measurements and one more level, and both walk the same fetched bucket list). Note the final measurements: line and [measure-trace]'s last lines may read --: UDPSCOPE_EXIT_AFTER_MS arms the trigger before it dumps, so the dump can land in Waiting with nothing bound and the axis at its default. The race-free reading is an earlier [measure-trace] line, which carries its own view=; those lines are also where the level-mark fallback is visible (an invalid pass must still print its level: marks).
[bandavg-probe] (stderr, UDPSCOPE_PROBE_BANDAVG_MS) — the band averages: above/below/between, the plain means over one side of a share and over a band, which are different numbers from top/bottom's flat levels (see the band-average invariant in invariants-measurements.md). It adds seven measurements, on the three shapes whose band means can be computed by hand, and the four figures to read are:
- the 1 Hz unit sine at a 2 s window: above 50 = 0.636620 (2/pi) and below 50 = -0.636620 — the exact figures the old top/bottom documented, and the reason this type exists at all; between 50 75 = 0.255873 (3(2-sqrt 3)/pi), an interior mean no side and no whole-window statistic reports;
- the mock's band-limited square (30 harmonics, its default): above 50 = 0.986496, the reading the commit that made top a flat level recorded as the old top;
- the mock's ramp, whose density is uniform: between 50 75 = 0.250000 and above 50 = 0.500000 — a band's mean is that band's centre, so the shape predicts the number;
- and the two nothing-to-measure readings: above 100 on the ramp and between 50 75 on --square-harmonics 0 (whose samples are all exactly +/-1) must be invalid/--, never 0.
Run it at a 2 s window (the probe waits for a full one before it reads anything — the mock delivers about 0.5 MS/s of the 1.3 MS/s it declares, so a freshly enabled ring needs seconds of wall clock), and not with UDPSCOPE_PROBE_MEASURE_MS or UDPSCOPE_PROBE_MEASURES_MS: all three add measurements to one pane. UDPSCOPE_EXIT_AFTER_MS=17000 covers its sequence. The command, both mock flavours and the lines are in probe-readings.md: --square-harmonics 30 for the 0.986496 and the 0.302744, --square-harmonics 0 for the 1.000000 and the empty band.
[measurepin-probe] (stderr, UDPSCOPE_PROBE_MEASUREPIN_MS) — what window a measurement describes: on a still frame the pane is about the frame, so a pan or a zoom changes no number, and on a live view the numbers still follow the window. Three legs — a latched event, a pause taken from it, and a live view after the trigger is switched off — each read the pane before and after a gesture written on the row's own axis (never the window control, which calls off the pin). The assertions, in order: the pinned legs read same=4/4 for_same=1 pass_delta=0 with for= still at the pin (the values bit-identical and no recompute spent on the gesture), and the live leg reads follows=1 with for_same=0, pass_delta>0 and the results window's span narrowed to the axis'. Its sequence is about 12 s, so UDPSCOPE_EXIT_AFTER_MS=20000; run it on the fast mock with a batch the datagram limit allows (--rate 1300000 --batch 1500 --freq 1, not --batch 2000, which is 80 kB of payload and dies with Message too long). The lines, the fields and the settle wait are in probe-readings.md; the window rule itself is in invariants-measurements.md.
[pan-probe] asked=[..] was=[..], [pan-probe] after=[..] held=D asked=[..], then [pan-probe] zoom asked=[..] after=[..] span=S window=W limited=L (stderr, UDPSCOPE_PROBE_PAN_MS) — held must be 1: a gesture must not move a window that is pinned (trigger mode, or a paused view). The probe writes the panned range the way a gesture does — on the member axis, which Figure::layout() adopts into the group — and reports the axis the frame after it produced. Time it inside one pinned state: a pan during a capture whose check lands after the capture has ended is re-framed by the mode change, not by the pin, and reads held=1 with the hold removed (that is how the first version of this probe fooled me). The pause probe makes a stable pin, so run them together: UDPSCOPE_PROBE_PAUSE_MS=1500 UDPSCOPE_PROBE_PAN_MS=2200 puts the pan and its check inside the pause. With hold_pinned_window() disabled it reads held=0 and the axis sits at the asked range. It then asks for a zoom-in (a window a quarter of what is on screen) and reports group=/member= beside it: allowed must be 1, with both equal to the ask -- that is the single-trigger-mode case, where the wheel has to keep working, and where reading the wrong one of the two made a working clamp look broken. All three verdicts are read from the tick, one frame after the ask: a g_timeout was the first version and it drifted past the paused/resumed boundary under load. The probe then asks for a zoom-out ten times the current span and reports the span the next frame produced: limited must be 1, i.e. span <= window. Run it with a ring that holds more than the window (--buffer-mb 64 past the window's own seconds) — that is the state the old max(retained, window_s_) bound leaked in, and it showed up here as span=2.502550 window=2.000000 limited=0 against span=2.000000 after the fix.
[zoom-probe] <sweep>/x<step> axis=[..] span=.. n=N idx=[a..b) spp=.. level=L pts=P kept=K back=.. ink=[..] x_at_first=.. gap_px=.. end_px=.. rect=[..] (stderr, UDPSCOPE_PROBE_EVENTZOOM_MS) — arms the trigger in single mode, polls for a held event (a latch happens when the stream says so; a sweep on a timer has nothing to zoom into), then zooms in eight doublings twice: ges/ writes the row's own axis the way a gesture does, win/ drives the window control (apply_time_window, which unpins and re-arms the live follow before the pin is restated). Each step's reading is taken in the tick, after that frame's pin and clamp — a g_timeout would report the ask. It recomputes what Renderer::draw_traces is handed, through the same calls: neighbours() for idx, the 64-spp gate for level, then gather(). gap_px and end_px must stay within a pixel or two at every step — they are how much of the left and right margin no geometry reaches, and a gap_px that doubles per step is the "zooming into a held event clips it more and more from the left" defect (374 px of a 716 px axis by ges/x128), whose cause is a constant offset in data seconds (see the neighbours() invariant in invariants-rendering.md). kept/back are the renderer's own increasing-suffix filter counted rather than applied: kept far below pts is a staircase, and a non-zero back is a non-monotonic gather. Run it with --clock-jitter (tools/mock_udpstreamer.py --rate 1300000 --batch 2000 --clock-jitter 0.4): the ideal mock's stamps sit on a perfect grid, which is exactly what hides this class of bug, and three runs against it reported gap_px=0.0 throughout while the defect was live. The jitter varies each packet's spacing rather than offsetting its start, because the stream's clock has to stay strictly increasing. The win/ sweep bottoms out at span=0.05 (RowSet::set_time_window()'s floor) and walks to the event's post side, since set_time_window() frames the newest span before the pin clamps it — both are expected.
UDPSCOPE_PROBE_PAUSE_MS — pause acquisition mid-run so layout numbers are measured on a still picture. It now also asserts what a pause is ([pause-probe] <action> paused=D ring_produced=N event_size=M frozen_x=[..] axis=[..] skips=N glance=[..] on stderr): it presses the button, not Acquisition::set_paused(), because pausing is a display state as well as an acquisition one and the view snapshot and axis pin live in the handler. Across the three lines ring_produced must grow (the producer never stops — it fills the live ring exactly as it does with the trigger off), event_size must be non-zero exactly while paused (the plot is bound to the frozen snapshot, not the ring) and frozen_x must not move between the pause and the resume; after the resume event_size is 0 and the window is back on the newest data. Pausing used to stop the producer, which is why the counts grew by nothing and the resume left a hole. frozen_x is the rebased snapshot — the pause view's own stamps are relative to the freeze instant now, so the range reads [-window..0]-ish rather than stream seconds, and a frozen_x in the thousands is the rebase gone. glance= is the assertion for the paused status field, and this is the only line that can make it: the exit dump's status bar: is read live and its two mode … lines are the live and triggered states. While paused it must read paused · <rate> · <window> · for <elapsed>, with the elapsed figure growing between the two paused lines and absent on the resume line. It is wall time (g_get_monotonic_time()), not stream time, because the producer never stops while paused and "how long have I been looking at this" is a question about the user, not about the stream; it is written through format_elapsed() so the field steps to a coarser unit instead of growing digits (see the layout rule). skips= is the assertion for the render gate (the end of tick()), cumulative over the session: the live line must read 0 and the paused line must read more than 0. A frozen view is bound to an immutable snapshot on a pinned axis, so a frame whose picture cannot have changed is not drawn again — and this is the only line that can say whether that happened, since a skipped frame is invisible in every other reading by construction. Non-zero on the live line is the gate's condition inverted, which would cost dropped samples' worth of picture; 0 on the paused line is the gate not engaging and the saving not being made. The gate cannot stall the picture (kStaticRenderEvery renders every few ticks regardless), so the renderer's own baselines below are unchanged by it: a measured run read skips=0, then 33, then 45, with columns == graded, render blank frames: 0, and the layout floor still converged (y-gutter 104..104).
UDPSCOPE_PROBE_COLOR_MS — recolour one plotted signal and read the colour back out of the trace and the chip, round-trip it through the session writer, then clear it. Schedule UDPSCOPE_ENABLE_ALL_MS before it: with nothing bound it has no trace to recolour and asserts nothing. Its four lines are in probe-readings.md.
UDPSCOPE_EXPORT_MS (+ optional UDPSCOPE_EXPORT_PATH, default /tmp/udpscope-export) — collect the export table and write it all three ways (.npy, .csv, .mf3), skipping only the file chooser.
UDPSCOPE_PROBE_CONDITION_MS — arms a qualified trigger against the mock and reports the latched t0, the measured pulse or band width, and sample values for the window condition. Schedules its own arm and report stages. UDPSCOPE_PROBE_CONDITION_SHAPE picks the leg: 0 pulse width < (the glitch, the default, needs --glitch-at/--glitch-width), 1 pulse width > (uses a 1.0 s window and 50 % pre so the opening edge of a 0.2 s pulse is captured), 2 window (uses sine with a 0.2 s window and 10 % pre), 3 dropout (needs --quiet-at; uses sine with a 1.0 s window and 50 % pre so the last crossing before the outage is captured).
UDPSCOPE_PROBE_STATS_MS — reports every measurement's accumulator (n, mean, sd, lo, hi), pauses, reports again with dn= per measurement, then resets and reports a third time. Three phases: phase=0, phase=1 (after pause) and phase=2 (after Reset). Needs UDPSCOPE_ENABLE_ALL_MS ahead of it and UDPSCOPE_PROBE_MEASURE_MS to add measurements first.
UDPSCOPE_PROBE_SIDE_TAB_MS=3000 with UDPSCOPE_EXIT_AFTER_MS=11000 — the measurements on the Measure tab, a forced pass, then the tab switched away and read again two seconds later. advanced=4 is the assertion (every accumulator gained passes while its page was off screen); advanced=0 is the defect where the accumulation is gated on the visible tab, which would leave a gap in min/max/avg for exactly as long as the trigger was on screen. It enables and plots its own signal, so it needs no other probe ahead of it. The reading's own entry is in probe-readings.md.
UDPSCOPE_PROBE_SOURCE_KINDS=2000 with UDPSCOPE_EXIT_AFTER_MS=5000, run as --source probe --source-opt rate=500 -- the source-plugin path end to end. The variable's presence also registers the test-only probe kind (a synthetic sine with one panel), so it is the only way a run sees that kind; its value is the delay in ms. Seven [source-kinds] lines; the reading's own entry is in probe-readings.md.
UDPSCOPE_PROBE_SEARCH_MS — runs a buffer search for the armed condition, cross-checks the live trigger's own t0 against the nearest hit (agree=1 is the load-bearing reading), and jumps to hit 0. Reports the resulting mode=, axis= and marks placed=.

The following mock waveform features are what the qualified-condition probes are checked against, as distinct from the fault injection below, which exercises ingest:

--glitch-at S / --glitch-width W invert the square signal for W seconds at the start of every square period from stream second S onwards — a runt pulse of a known width, which is what a < pulse-width trigger and a glitch search are checked against.
--quiet-at S freezes sine at its held value from stream second S onwards, so it stops crossing zero: the dropout trigger's input.

UDPSCOPE_PROBE_RESEQ_MS — one mid-run reading of the resequencing path. Five cases, run each as its own process — five back to back starve the mock past its 2 s silence timeout and every case then reads a reconnect instead of what it was testing. Common conditions for the first four: the mock at --rate 10000 --batch 100 --freq 5, udpscope with no extra options, under xvfb-run with UDPSCOPE_EXIT_AFTER_MS=8000 UDPSCOPE_ENABLE_ALL_MS=600 UDPSCOPE_EXIT_DUMP=1 UDPSCOPE_PROBE_RESEQ_MS=5000. Move ~/.config/udpscope/127.0.0.1_8000/ aside first (the per-endpoint session is a directory now -- active plus one .toml per named configuration; a bare 127.0.0.1_8000.toml is a pre-migration leftover): a session left by an earlier run restores its math channel, and five channels instead of four change channels=, span= and every count derived from them. It restores the window and the measurements too, which is the subtler trap: a run after a zoom-sweep probe starts at window_s = 0.05 with the previous runs' measurements still in the pane, so measures window: then reports a short-window sawtooth (144 %, marks off screen) on a run that asked for none of it.

case	mock extra flags	key reading
base	(none)	every counter 0; proves the machinery costs nothing
reorder	`--reorder-every 10`	`reseq=51` (mid-run) / `102` (dump); `holes=0`; resequencing inside the window leaves no reservation
late	`--reorder-every 10 --reorder-delay-ms 50`	`bands_repaired` one short of `bands` with `nan_slots=100`; the band still solid is the one hole whose late packet is in flight, and it is the only unrepaired one
mixed	`--drop-every 20 --reorder-every 10 --reorder-delay-ms 50`	`bands=43 bands_repaired=20 nan_slots=2200`; all three band states at once
re-plan	`--rate 100000` with the `mixed` flags, and `UDPSCOPE_PROBE_WINDOW_MS=4000 UDPSCOPE_PROBE_RESEQ_MS=6000 UDPSCOPE_EXIT_AFTER_MS=9000`	four `buffer rebuilt` lines before the reading, then `bands=64 bands_repaired=16 nan_slots=7800`: a band is still only marked repaired when its own slots hold values. A `bands_repaired == bands` beside a non-zero `nan_slots` here is the hole table and the producer's `tick_` surviving a re-plan that reset `produced()` — a band marked settled over samples that are all still NaN. The non-zero `overflows` is this case too: 100 kS/s with a 50 ms reorder delay keeps about 50 packets in flight against 64 slots

The re-plan case needs --rate 100000, not the other four's 10 000: at 10 kS/s the ring

UDPSCOPE_PROBE_CAPTURE_MS — toggles the packet capture through the same handler the status-bar button uses, waits, then reads the file back off disk and prints one line: enabled= path= slots= stored= resident= cursor= torn= truncated= header_ok= first_seq= last_seq= payload= file= open_ms= t0=. stored is the datagrams-since-open counter and must equal the dump's datagram count from the toggle's instant; resident is what the file holds, which is the slot count once the ring has wrapped; file is the reserved size 64 + slot_count * slot_bytes and is a constant for a given N, so a mismatch against the file on disk is a reading. Two cases, run each as its own process:
1. enable at runtime — the mock at --rate 20000 --batch 200, udpscope with UDPSCOPE_ENABLE_ALL_MS=700 UDPSCOPE_PROBE_CAPTURE_MS=1500 UDPSCOPE_EXIT_AFTER_MS=9000 and no capture flag: enabled=1 header_ok=1 torn=0 truncated=0 first_seq=1, stored equal to the datagrams since the toggle, open_ms a few ms.
2. wrap — the same with --capture-packets=16 and UDPSCOPE_PROBE_CAPTURE_MS=7000 UDPSCOPE_EXIT_AFTER_MS=10000: stored (870 in the reference run) exceeds slot_count, resident=16, last_seq - first_seq == 15, torn=0, and cursor agrees with seq (counter mod slot count). A wrap that loses ordering, or a cursor that disagrees, shows here and nowhere else.
Then the converter over the wrap case's file: tools/capture_to_pcap.py <capture> out.pcap must exit 0, print a record count equal to slot_count with torn 0, and the pcap's first and last record timestamps must fall inside the run's wall-clock interval — both checked by a struct.unpack of the pcap headers, nothing outside the standard library.
A gap must not kill a filtered trace. --relative-time with the mock dropping (--drop-every 20) and UDPSCOPE_PROBE_MATH_MS=2500 UDPSCOPE_ENABLE_ALL_MS=600 UDPSCOPE_EXIT_AFTER_MS=9000: every derived channel's dump line math '<name>': finite=N/4000 must read about 3800 — the tail minus the reserved holes — and finite=0 beside a non-zero hole_slots is the defect (a biquad that integrated the hole's NaN, permanently). The filters' sd must stay near the no-drop value; run the same command without --drop-every as the control, where every finite reads 4000.
A math edit on a ring that has already wrapped. UDPSCOPE_PROBE_MATHWRAP_MS — installs one Gain graph over the mock's sine, waits for the ring to saturate, and only then edits the gain, which is the one state UDPSCOPE_PROBE_MATH_MS cannot reach (each of its edits follows a structural change, and a replan restarts every ring at absolute 0). It therefore needs a small buffer and a long run: --buffer-mb 1 against --rate 20000 --batch 200, with UDPSCOPE_PROBE_MATHWRAP_MS=1500 UDPSCOPE_EXIT_AFTER_MS=25000. base > 0 on its second line is the precondition — base=0 means the run tested nothing — and the third line's produced == slots and y=±0.5 are the two assertions; both are in probe-readings.md.
UDPSCOPE_PROBE_CONFIGS_MS — saves the running session as a configuration named probe with a note, through save_configuration_as() (the same entry point the dialog's button uses), then reads it back off disk. Reading: save-as=1; names= gains probe; active=probe; and the round trip must carry the note and the startup settings — comment='written by the configs probe' buffer_mb=8 relative=0 capture=0. A buffer_mb=0 there is the per-source buffer size not being written, which is the one setting a user notices only after a capture has been sized wrong. Run it with a throwaway XDG_CONFIG_HOME; the probe deletes its own configuration at the end (after forget names=…).
UDPSCOPE_PROBE_ROWMENU_MS — gives row 0 a trace, opens the plot's context menu with open_row_menu() (the one entry point on_right_click() wraps) and walks its two views one step at a time (main → click Remove signal → remove → click Back → after-back → click Remove signal again → second-open → click a trace), measuring each view's rows: [rowmenu] <view> N '<label>' h=18 label_x=26 accel_right=R/W ok=1 per row and one rows=/bad=/labels= line per view, then [rowmenu] summary main_rows=5 main_bad=0 remove_rows=6 remove_bad=0 after_back_rows=5 after_back_bad=0 second_rows=6 second_bad=0 remove_row_in_main=1 remove_row_gone=1 can_open_again=1 and [rowmenu] remove-clicked 'sine' trace_freed=1. Every bad must be 0 (the same compact row style as the app's own menu: one label column, a shortcut flush right, 18 px rows — the Back row's icon indents its label, which every menu does, so its label_x is 52 against the others' 26) and the three booleans must all read 1: the remove view replaces the main one (remove_row_gone, remove_row_in_main), and it can be entered again after leaving it (can_open_again) — that last one is the case a GtkMenuButton-owned submenu failed, its row going dead on the second click. Each view is measured a step after the click that swapped it in, because a popover allocates its new child on the next layout pass.
UDPSCOPE_PROBE_HELPUI_MS — the help affordance layer, end to end, in ten steps 300–400 ms apart: the first-run auto-start (run it once with a throwaway XDG_CONFIG_HOME to see auto=1, and again against the same one to see it not repeat), the menu's seven rows and the four entries of the Help submenu, the contextual-help toggle with the count of "(i)" affordances on either side of it, the tour started from the probe and stepped forwards, back and dismissed, and help figures: read in both of the trigger pane's modes. Its lines and what each asserts are in probe-readings.md; the two things to read first are hints=0 while the toggle is off (off is gone, not greyed) and controllers=12 (was 12) after the dismissal (no lingering input grab — a tour of ordinary overlay widgets takes none), and hint=12x12 is the size the shrunk "(i)" must keep (the default menu-button mark is 34x34). The collapsed search: line is the header hint read on the search block's expander while it is collapsed. The chooser's own tour is the other half and has its own entry points — see the chooser bullet below. Give it UDPSCOPE_EXIT_AFTER_MS=9500 UDPSCOPE_EXIT_DUMP=1 to get the help layer:/help figures: dump lines as well, and note the tour auto-starts in any run whose settings file does not carry tour_seen — in the source chooser when there is one, in the window otherwise (see the chooser bullet) — so a probe run against a fresh config directory will have it on screen for the first second and a half, and that start writes tour_seen to whichever config directory the run is using, which is why the second run against the same one reads auto=0.
UDPSCOPE_PROBE_TOUR_SURFACE_MS — the tour's surface switch: it puts the sidebar on Measure, starts the window's tour, and walks it a stop per tick to the signals stop, reading the sidebar's page and the spotlight/card where the walk arrives. It then puts the sidebar away and starts the tour again, which is the same filter's other case (a collapsed pane drops the stops a page switch could have revealed). [toursurface] signals stop=4/5 … side_tab=signals … contains=1 and [toursurface] collapsed: side_stack_mapped=0 steps=2 signals_stop=0 are the two assertions; its lines are in probe-readings.md. Run it alone with UDPSCOPE_EXIT_AFTER_MS=9000 UDPSCOPE_EXIT_DUMP=1 against a live source.
UDPSCOPE_PROBE_DEFAULTS_MS — opens Calibrate and reads every spin button in it and in the trigger panel: [defaults] calibrate: 1 0 model=1 0 | trigger: 0.001 0.002 0 0.001 0 50 0.01. A GtkSpinButton starts at its range's minimum, so a -1e12 here is an unseeded button the dialog's own reader would then apply — which is exactly what the calibration offset did. It then calibrates signal 0 through set_calibration() and reopens the dialog: [defaults] calibrated: 2.5 -1.25 model=2.5 -1.25 — the dialog must show the signal's stored calibration, and the two halves must match.
The chooser, display-free: UDPSCOPE_PROBE_SOURCES=1 prints [sources] N known: with one entry per configuration (host:port/name … buffer= relative= note='…'), and UDPSCOPE_PROBE_PICK_MS=800 opens the chooser, picks the first row and prints [sources] picked host:port … buffer=64 relative=0 capture=0 config=… note='…' — the settings the configuration carried, into the Config the model is built from. With no --server and no probe the chooser blocks in its own nested loop (no frame to end and no dump), so a chooser run is always this probe. It also prints the dialog's frame and its own help layer: [sources] chooser ui: … hints=[2 {chooser_sources chooser_waveform}] hint=12x12 with the setting on, hints=[0 {}] hint=0x0 with it off. UDPSCOPE_PROBE_CHOOSER_HELP=1 drives the Enable contextual help row and reads the file back; UDPSCOPE_PROBE_CHOOSER_TOUR=1 drives the Guided tour row and walks the chooser's own tour to its last stop, printing [sources] chooser tour: … and the handoff (tour_pending=1); UDPSCOPE_PROBE_CHOOSER_TOUR_MS=<ms> walks it a stop per tick instead, printing the page the stack shows at every stop and stepping Back off the stop that stands on the waveforms page — the reading for the surface a stop changes. It starts the walk itself when one is not running, so it does not need a first-run config; pair it with UDPSCOPE_PROBE_PICK_MS after the walk so the application's half follows. A working run: seed a source once (./build/udpscope --server 127.0.0.1 --port N against a mock, under the throwaway XDG_CONFIG_HOME), then run bare with UDPSCOPE_PROBE_CHOOSER_TOUR_MS=2000 UDPSCOPE_PROBE_PICK_MS=4500 UDPSCOPE_EXIT_AFTER_MS=12000 UDPSCOPE_EXIT_DUMP=1. Its lines and what each asserts are in probe-readings.md. A legacy file (<host>_<port>.toml) produces layout loaded from …/<host>_<port>/default.toml, an active holding default, and the old file renamed .imported.
The loading window covers the gap, and only on the path that asked. The same UDPSCOPE_PROBE_PICK_MS run must print [sources] loading: title='Connecting to 127.0.0.1:<port>' detail='planning the N MB sample buffer' between the picked line and the stream's CONFIG:, and [sources] loading: closed after it — the window is raised before the buffer is planned and taken down only once the real window has been presented, which is the whole of what it exists for. UDPSCOPE_PROBE_WAVE=<path> is the file leg of it (title='Opening <basename>' detail='reading the recording', before waveform loaded:). A run that names its source with --server prints no loading: line at all: every other probe here is on that path and must stay on it.
The event cache round-trips, and only where it is available. UDPSCOPE_PROBE_EVENTS_MS reads the pane's availability first, in the state the app starts in -- [events] live: button_visible=0 pane=0 recording=1, no pane and no button without trigger mode -- then turns trigger mode on through the real toggle ([events] after trigger on: button_visible=1 pane=0: the button appears, the pane stays collapsed) and clicks the button, waits for a latch, then reads the cache and replays the newest event through the pane's own row: [events] dir=… n=1 newest: dur=2.000000 pre=1.000000 post=1.000000 edge=… signals=4 samples=20001 rate=10000 seq=1 thumb_bytes=230454 recorded=1 pane=1, then [events] activated row0: replaying=1 thumbnail=1 thumb_size=108x36 thumb_visible=1 (the size is what says the picture is on screen: a paint-able alone is not), [events] replayed: channels=4 samples=20001 replaying=1 waveform=1 and [events] live again: replaying=0 waveform=0. It ends on the pane's availability, through the trigger panel's switch: [events] recording off, trigger off and [events] recording off, trigger on -- once the cache holds events (the probe's own run has written some by this point) both read button_visible=1 pane=1 recording=0, because a button that shows the cache must not go away with the switch that wrote it; against an empty cache the same two lines read button_visible=0 pane=0, which is the empty column the item exists to prevent. samples must equal the export's row count for the same window (20001 for a 2 s event at 10 kS/s), thumb_bytes must be a real BMP (54 + 480*160*3-ish, i.e. hundreds of kB, not 0), thumbnail must be 1 (the row loaded the picture) and the last line must read replaying=0 -- a replay with no way back is a one-way door. It then waits for a second latch (bounded: twelve retries) and drives the two group layouts through the buttons: [events] compare buttons: cached=2 selected=2 compare=1 accumulate=1 (the buttons read the selection), [events] halve… [events] halves a_first=20001 a_second=0 b_first=0 b_second=20001 -- the two halves of the ring, and the reading that says a comparison's gap is a gap: the first event's channel is finite only in the first half and the second's only in the second -- [events] compare: kind=1 rows=4 channels=8 samples0=40002 traces0=2 traces1=2 window=4.000 recording=1 pane=1 (samples0 must be exactly the stamps pushed, not more: a live producer left running pushes its own four signals into the group's eight channels), [events] compare plots: area_b=1 rows_a=1 rows_b=1 x_a=[-2.000,0.000] x_b=[-2.000,0.000] same_x=1 (the second plot exists, is on screen, mirrors the row count, and both are on one x window), then the sync -- the probe sets a y range nobody would arrive at by accident on the first plot and reads the second a few frames later: [events] compare sync: y_a=-3.500 y_b=-3.500 mirrored=1 -- and [events] accumulate: kind=2 rows=1 channels=8 samples0=33045 selected=2 traces0=8 window=2.000 (traces0=8: the four signals the layout holds plus one layer each for the second event, in the same row, which is what an accumulation means). Run it with a throwaway XDG_CACHE_HOME and UDPSCOPE_EXIT_AFTER_MS long enough for the second latch (20 s).
A waveform round-trips. Export from a live run (UDPSCOPE_EXPORT_MS with UDPSCOPE_EXPORT_PATH, against the mock at 10 kS/s), then open that file with --open and an exit dump. The load must print [udpscope] waveform loaded: … signals=4 samples=20001 rows=20001 decim=1 span=2.000000s rate=10000 time=[-2.000000..0.000000] — the same row count, column count, span and time range the export probe reported (rows=20001 cols=4 time=[-2.000000..0.000000]) — and the dump must show channels=4, frames=0 packets=0 (no acquisition) and, per channel, 20001 samples vs 20001 time slots … y=[-1.0000, 1.0000]. A file that loads with fewer samples than it was written with, or with NaN slots, is the reader mis-slicing its records.
A recording that does not fit its buffer is filtered into it, not truncated. The case needs a file bigger than the budget, which the mock cannot export in one window, so write one with NumPy — the dtype is the exporter's own, [('time','<f8'), ('abs_seconds','<f8'), (<signal>,'<f4'), …] — and open it against a small buffer:
```
python3 -c "import numpy as np; n=1000000; t=-4.0+np.arange(n)/50000.0; \
a=np.zeros(n, dtype=[('time','<f8'),('abs_seconds','<f8'),('a','<f4'),('b','<f4'),('c','<f4'),('d','<f4')]); \
a['time']=t; a['abs_seconds']=1000.0+t; a['d']=np.linspace(-1,1,n); np.save('/tmp/long.npy', a)"
xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=4000 ./build/udpscope --open /tmp/long.npy --buffer-mb 1
```
must read samples=31250 rows=1000000 decim=32 span=19.999980s time=[-4.000000..15.999980] and stored=31250 … published=31250 per channel — the whole span, at a thirty-second of the rate. The ramp column is the assertion that the head is there: y=[-0.999969, 0.999969] is the file end to end, where the defect this exists for reads a fraction of it. With every row pushed regardless of the plan, this file arrived as its last 0.9 s and nothing in the log said so.
A loaded file can be plotted. UDPSCOPE_PROBE_DROP_MS drives the drop handler the way a drag from the signal panel does, on a file opened with --open: it must read [drop-probe] signals=4 channels=4 rows=4 slots0=[sine|||] view=[-3.999690..15.999670] repins=2 — rows equal to the column count, one trace per row — and each drop must return 1 and add its trace (dropped signal 1 -> 1 slots=[sine|sine|square|]). rows=1 slots0=[sine|square|ramp|noise] with every drop returning 0 is the defect this exists for: rows sized four signals to one row fill the only row, so the app reads as one that cannot plot anything. A file-mode run must also write no session for the default 127.0.0.1:8000 (App::source_port is 0), or it overwrites that source's saved layout.
A file view states its pin once. repins on the line above must read 2 whenever the probe is scheduled — at one second or at seven — and view must be the file's whole span. A file's extent does not change while it is open, so the tick refreshes the pin's numbers and writes the group's range only when the view first becomes an event. Reading it against the decimated file above (/tmp/long.npy, stamps [-4..16]) is the case that matters: with the pin written as a pre/post split where a range belonged it read view=[3.999690..15.999670] repins=116 at three seconds — the axis alternating between the file's span and a corrupted pin every frame, which on screen is a flicker in which no pan, zoom or click survives to the next frame. A file whose stamps are all positive, or which ends at 0 (an export's own window), hides it: the corrupted pin is then wider than the data, or empty, and the clamp never fires. UDPSCOPE_PROBE_PAN_MS on the same run is the other half — allowed=1 and a view that is still the zoomed range when the drop probe reads it later is interaction actually taking.
The chooser's Waveforms page: UDPSCOPE_PROBE_WAVEFORMS=1 prints [sources] waveforms: N recent <path>(ok|missing) … page=waveforms, and UDPSCOPE_PROBE_WAVE=<path> sets the field the file dialog's callback sets and prints [sources] picked waveform <path> — then the load line above, with no socket involved.
A --server launch reads the stored configuration. With a throwaway XDG_CONFIG_HOME holding 127.0.0.1_8101/bench.toml (buffer_mb = 64, relative_time = true, capture_packets = 500, active = bench), ./build/udpscope --server 127.0.0.1 --port 8101 must print [sources] settings from configuration 'bench': buffer=64 relative=1 capture=500, the relative time: line, and a span= of about 209 s (64 MB at 10 kS/s) — not the 200 MB default's ~655 s. The same command with --buffer-mb 8 must read buffer=8 and span=26.214s: an explicit flag wins over the file. Without this, the settings a host was edited to have were ignored whenever the address came from the command line.
The chooser's form, with UDPSCOPE_PROBE_PICK_MS scheduling it: UDPSCOPE_PROBE_NEWHOST=host:port fills it and prints [sources] form: name='probe-host' note='…' buffer=32 relative=1 connect_sensitive=1, then [sources] configuration created: host:port/probe-host … buffer=32 relative=1 and a picked … config=probe-host; a following UDPSCOPE_PROBE_SOURCES=1 must list it (buffer=32 relative=1 note='…') — a configuration the form wrote that the chooser does not list cannot be edited next time. UDPSCOPE_PROBE_EDITHOST=host:port opens the same form on that configuration and prints [sources] edit form: name='…' note='…' buffer=… relative=… button='Save' — the pre-fill is the assertion: an edit form that opens empty cannot edit anything — then saves and reports the new values. UDPSCOPE_PROBE_EDITNAME=new-name renames it as well: one known: row afterwards, with the old file gone.

UDPSCOPE_PROBE_RESEQ_MS's neighbour for reconnect: the mock gains an EMI case, --blackout-at S --blackout D — silent from second S for D seconds (D must exceed the client's 2 s silence timeout), resuming from now so the link's backlog is what the link ate. One run (--blackout-at 4 --blackout 3.5, UDPSCOPE_EXIT_AFTER_MS=15000) has to read: every reconnect's CONFIG: line followed by CONFIG unchanged: keeping the buffer and its history (a reconnect that re-plans wipes history, which is the defect this case exists for), no buffer rebuilt line after the first CONFIG, frames climbing again after the blackout with gaps=0, and a trigger armed after recovery (UDPSCOPE_PROBE_KEYS_MS past the blackout) latching — the dump's CAPTURING/EVENT HELD and publish cost: N events with N ≥ 1 are that proof. already covers 838 s, so no window the control can reach moves time_stride, set_target_span() returns without rebuilding, and the window probe reports decim=1 throughout — it re-plans nothing. At 100 kS/s the ring covers 83.9 s and the probe's 500 ms → 500 s → 120 s → 600 s → 60 s walk crosses the boundary four times.

Full measured readings (probe line first, then exit-dump):

=== base   mock: none
[reseq-probe] reseq=0 late=0 stale=0 overflows=0 holes=0 hole_slots=0 backfilled=0 open_holes=0
              bands=0 bands_repaired=0 nan_slots=0 scanned=44200 trigger_in_hole=0
[udpscope] frames=1032 packets=1033 gaps=0 channels=4 span=838.861s stride=1 reseq=0 late=0
           stale=0 overflows=0 holes=0 hole_slots=0 backfilled=0
[udpscope]   channel 0 'sine' stored=95700 published=95400

=== reorder   mock: --reorder-every 10
[reseq-probe] reseq=51 late=0 stale=0 overflows=0 holes=0 hole_slots=0 backfilled=0
              open_holes=0 bands=0 bands_repaired=0 nan_slots=0 scanned=44200
              trigger_in_hole=0
[udpscope] frames=1027 packets=1028 gaps=0 channels=4 span=838.861s stride=1 reseq=102 late=0
           stale=0 overflows=0 holes=0 hole_slots=0 backfilled=0
[udpscope]   channel 0 'sine' stored=95300 published=95100

=== late   mock: --reorder-every 10 --reorder-delay-ms 50
[reseq-probe] reseq=157 late=46 stale=0 overflows=0 holes=40 hole_slots=4000 backfilled=15600
              open_holes=40 bands=40 bands_repaired=39 nan_slots=100 scanned=44000
              trigger_in_hole=0
[udpscope] frames=1027 packets=1028 gaps=98 channels=4 span=838.861s stride=1 reseq=296
           late=98 stale=0 overflows=0 holes=91 hole_slots=9100 backfilled=36400
[udpscope]   channel 0 'sine' stored=95300 published=95000

=== mixed   mock: --drop-every 20 --reorder-every 10 --reorder-delay-ms 50
[reseq-probe] reseq=159 late=25 stale=0 overflows=0 holes=43 hole_slots=4300 backfilled=8400
              open_holes=43 bands=43 bands_repaired=20 nan_slots=2200 scanned=44100
              trigger_in_hole=0
[udpscope] frames=978 packets=981 gaps=101 channels=4 span=838.861s stride=1 reseq=309 late=50
           stale=0 overflows=0 holes=94 hole_slots=9400 backfilled=18400
[udpscope]   channel 0 'sine' stored=95400 published=95400

=== re-plan   mock: --rate 100000 --drop-every 20 --reorder-every 10 --reorder-delay-ms 50
[reseq-probe] reseq=4374 late=72 stale=0 overflows=36 holes=315 hole_slots=31136
              backfilled=26800 open_holes=64 bands=64 bands_repaired=16 nan_slots=7800
              scanned=156600 trigger_in_hole=0
[udpscope] frames=8949 packets=8953 gaps=614 channels=4 span=83.886s stride=1 reseq=8028
           late=144 stale=0 overflows=36 holes=603 hole_slots=59936 backfilled=55600
[udpscope]   channel 0 'sine' stored=588600 published=584600

The late case's bands_repaired is one short of bands with nan_slots=100 — exactly one packet's worth of slots at stride 1 — because the probe fires 50 ms-delayed releases into a window that always has one in flight. An earlier recording of this case read bands_repaired=42 nan_slots=0; both are healthy, and which one a run gets is where the probe's instant falls relative to the newest release. A bands_repaired above the count of holes whose slots hold values is the defect, in either direction.

holes far above open_holes in the re-plan case is the two bounds working: a re-plan clears the table (the old entries address a buffer that is gone) and kMaxHoles caps what survives at 64. The mixed case's open_holes=43 against bands=43 has no such spread because nothing re-planned. The per-field reference is in probe-readings.md.

The mock also carries four fault injections that no probe schedules, because what they exercise is the ingest rather than the UI. The first: --time-glitch-at <stream seconds> --time-glitch-by <seconds> makes one packet's time signal jump ahead and then carries on from where the true timeline was, which is what a device whose clock glitches puts on the wire. Run it with the trigger armed, e.g.

xvfb-run -a env UDPSCOPE_ENABLE_ALL_MS=400 UDPSCOPE_ARM_TRIGGER_MS=700 UDPSCOPE_EXIT_AFTER_MS=22000 \
  ./build/udpscope --server 127.0.0.1 --port 8000 --holdoff 10
# mock: --rate 10000 --batch 100 --freq 5 --time-glitch-at 6.0 --time-glitch-by 10.0

Four readings, all off the once-a-second [udpscope] trig engine: line:

exactly one warning: ... jumped implausibly far ahead of the previous packet (16.01..16.0199 against a previous 6.0099); bridging that packet ... — the glitch was caught;
no timing samples by arrival instead — one bad packet must not change the time source;
now_t crosses the glitch continuously (5.040 → 6.050 → 7.050), and history keeps growing (1.01 … 23.15) instead of collapsing — the ring was not wiped;
the latches stay a hold-off apart: edge at 1.060 / latched at 2.060 / holdoff_until=12.060, then edge at 12.260 / latched at 13.260 / holdoff_until=23.260 — 11.2 s for a 10 s hold-off plus a 0.9 s post window.

Before the fix the same run read a fallback warning, history collapsing 4.03 → 0.95, now_t jumping 5.040 → 16.040, and two latches 1.2 s apart. Note the window here comes from the saved session for that port; the figures above are the default 2 s window (50 % pre), so move ~/.config/udpscope/<host>_<port>/ aside to reproduce them.

The other two are packet loss (--drop-every N, --drop-burst K) and wire inversion (--reorder-every N). Loss withholds a datagram the counter and the stream clock have already advanced over; the client counts the forward jump as counter_gaps. An inversion withholds a packet until the one behind it is sent, so the receiver sees a later counter first; a packet that arrives within the 20 ms reorder window is resequenced with no hole, and one that arrives past it leaves a committed hole that it then backfills (UDPSCOPE_PROBE_RESEQ_MS exercises both paths). There is no interpolation anywhere below the ring.

The fourth is the counter itself: --counter-step N advances the header counter by N per datagram instead of one -- a device whose counter counts samples, or frames of an inner layer, rather than packets. Nothing is withheld and every datagram goes out on schedule, so the link is intact and the status bar is right to say live; what is wrong is the counter's claim, which the client cannot check (it has no clock) and only the app can (it has the arrival times). It is the one wire shape the mock could not otherwise produce, and the run that reads it is the reseq probe with relative time on (the mode that frames a counter gap as NaN by construction):

# mock: --counter-step 10
xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=8000 UDPSCOPE_ENABLE_ALL_MS=600 \
  UDPSCOPE_PROBE_RESEQ_MS=5000 UDPSCOPE_EXIT_DUMP=1 \
  ./build/udpscope --server 127.0.0.1 --port 8000 --relative-time

# fixed:  [reseq-probe] … holes=1 hole_slots=900 backfilled=0 gaps_refused=419 open_holes=0
#         bands=0 bands_repaired=0 nan_slots=0 scanned=21700 trigger_in_hole=0
#         status bar: glance=[● live · 9.97 kS/s · 2 s window]
# broken: [reseq-probe] … holes=420 hole_slots=378000 backfilled=0 open_holes=64 bands=64
#         bands_repaired=0 nan_slots=198000 scanned=220000 trigger_in_hole=0
#         status bar: glance=[● live · 101 kS/s · 2 s window]

gaps_refused above 0 with nan_slots at 0 is the assertion — the gap was refused and the samples were kept. The broken reading is nan_slots at 90% of scanned with backfilled=0: a reserved hole per datagram, relative_time framing each one as NaN, and nothing ever missing to backfill it, i.e. one missing-data band over the whole window with no trace in it, under a status bar reading live at ten times the stream's real rate. The same field must be 0 on every honest shape above (--drop-every, --drop-burst 200, --reorder-delay-ms 50, at 10 kS/s: holes=210 hole_slots=21000 nan_slots=11000 scanned=22000 gaps_refused=0 for --drop-every 2, holes=3 hole_slots=60000 nan_slots=20000 for the 200-packet burst at 100 kS/s) — a non-zero reading there means the arrival test is refusing real loss, whose only cost is a gap with no shading.

All of the readings below are off UDPSCOPE_EXIT_DUMP's [udpscope] frames= packets= gaps= line, over an 8 s run against --rate 10000 --batch 100 --freq 5 (100 packets a second) except where the burst needs the faster stream. gaps is the exact reading and repeats to the packet; frames and packets move by one or two with the run's own timing. Run each of these on its own — four of them back to back in one script starved the mock past the 2 s silence timeout and every case, including the clean one, then read a reconnect instead of what it was testing:

clean, as the reference: frames=1029 packets=1030 gaps=0, no warning.
--drop-every 10: frames=925 packets=926 gaps=102. Every tenth packet never reaches the wire, so packets falls with frames and gaps is the count of them. No warning: a one-packet hole is a one-packet jump in the stamps, far inside the glitch guard.
--reorder-every 10: frames=1028 packets=1029 gaps=0. The inverted packet lands inside the 20 ms reorder window and is resequenced: every datagram arrives and gaps is 0. The pre-resequencing signature was frames=925 packets=1028 gaps=102 — an inversion cost a counter gap and a discarded packet, which is exactly what resequencing removes. Adding --reorder-delay-ms 50 pushes the arrival past the window and is the case the UDPSCOPE_PROBE_RESEQ_MS readings above cover; it is not this row.
--drop-every 1000 --drop-burst 200 at --rate 100000 --batch 100 (1000 packets a second): frames=7000 packets=7001 gaps=1600, eight bursts of 200, and still no warning.
--drop-every 1000 --drop-burst 300, same stream: frames=6203 packets=6204 gaps=2400 and the guard fires — warning: ... jumped implausibly far ahead of the previous packet (1.299..1.29999 against a previous 0.99899), then timing samples by arrival instead. 300 packet spans is past kGlitchPackets (256), so real loss is read as a clock glitch; and since each bridge advances the anchor by one packet span while the true timeline is 300 ahead, the next four packets fail too and kFailureLimit throws the stream onto the arrival clock. A burst between 256 packets and the 2 s silence_timeout_s is the only window in which this is reachable, and it is a finding, not a baseline.

Keep a burst under 2 s of wire time whatever the rate: past that the client reconnects, the mock resets its counter and clock on the new CONNECT, and the run reads gaps=0 with a backwards stamp warning (does not advance across packets ... 0..0.0099 against a previous 0.3899) — the reconnect path, not the loss path.

Run one self-check at a time. Two concurrent xvfb-run -a instances end up on the same display and the second gets no GL context — it dumps layout over 0 frames and valid=-1, which looks like a pipeline failure and is not one. tools/mock_udpstreamer.py also outlives its --duration, so kill it after each run or the leftovers pile up and hold ports.

The probes are the regression net — when changing UI or plot-host behaviour, extend the probe/dump in src/ui.cpp rather than adding a separate harness.

The noise probe needs a spiked mock

Every other probe runs against the plain mock. The noise probe reads what a band and an HF reject do to impulsive interference, so its mock must have some: an alternating-sign impulse train of 2 samples at amplitude 0.5 every 5 ms, added to sine alone (so a spiked channel can be read against a clean one in the same run).

cd /home/martino/Projects/tools/udpscope
python3 tools/mock_udpstreamer.py --port 8108 --rate 1300000 --batch 1000 --duration 20 \
  --spike-every 0.005 --spike-amp 0.5 --spike-width 2 --square-harmonics 0 \
  > /tmp/noise-mock.log 2>&1 &
sleep 1.5
xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=18000 UDPSCOPE_ENABLE_ALL_MS=1000 \
  UDPSCOPE_PROBE_NOISE_MS=4000 \
  ./build/udpscope --server 127.0.0.1 --port 8108 2>&1 | grep -E 'noise-probe|CRITICAL'

--square-harmonics 0 is the one flag in that command that is not about the sine, and it is what keeps the probe's last reading exact: the group-delay leg measures a one-pole's step response on square, and the mock's square rings by default (see build-and-run.md). A band-limited edge moves that crossing off tau*ln2 — measured delay=0.002770 ratio=1.255 against the ideal step's 0.002206 / 1.000, and the same figure on every run, so it is a property of the shape and not of the run. The leg's assertion is the 0.7..1.4 band, which the ripple keeps, but the ideal step is what keeps the number the line prints beside want as well. The other legs measure event counts on the sine, which the ripple does not touch; the counts do read a few percent lower under the ripple at this rate, for the producer-cost reason build-and-run.md gives.

UDPSCOPE_PROBE_NOISE_MS is not the time its readings are taken. It times the probe's first stage, which installs a math graph through apply_math_channels() -- and that is a structural rebuild that replaces the buffer and drops its history. Every reading the probe takes is over retained history, so the rebuild has to happen seconds before them, and the chain is what the number is really choosing: 4000 puts the graph in at 4 s, noise_probe at 9 s (it enables the signals it measures itself and waits 2.5 s for the ring to refill after that change), the legs and the five measurements at about 11.5 s, the de-noise channel enabled at 14 s, and the de-noise reading at 16.5 s -- hence the 18 s exit and the 20 s mock. Collapsing the two stages into one is the failure mode: a probe that rebuilt the buffer at 11.5 s and then scanned it immediately would read bare=0 on every search leg, which looks exactly like a broken comparator.

Why 4 s and not 3. enable_all_probe's last act is a timeout of its own, ENABLE_ALL_MS + 2000 -- 3 s under this recipe -- which walks signal_list for the front channel's acquire tick and clicks it. A probe that installs a math graph at that same instant rebuilds the signal panel under it, and the off-step then emits active on a freed GtkCheckButton: GLib-GObject-CRITICAL: instance with invalid (NULL) class pointer, with the backtrace in enable_all_probe()::{lambda}. It is a real fragility of that probe's lookup and not of the feature, and it is why every documented probe time is offset from ENABLE_ALL_MS + 2000, not merely from each other.

The de-noise channel is installed disabled and enabled only for its own reading (stage 3). An acquired channel is a share of the ring, so a fifth one shortens every channel's retained span -- and the six leg counts and pkpk bare all scale with that span. The readings above the de-noise line belong to the older feature and are the proof it still works, so the new channel stays out of the acquisition plan until they have run. --spike-every/--spike-amp/ --spike-width are the mock's own; --batch must stay at or below 1000 (2000 samples x 5 signals x f64 is 80 kB, past UDP's ~65507-byte payload, and the mock dies on its first sendto while the run still exits 0).

The three lines and what each must read are in probe-readings.md.

UDPSCOPE_PROBE_NOISE_MS, UDPSCOPE_PROBE_MEASURE_MS, UDPSCOPE_PROBE_MEASURES_MS, UDPSCOPE_PROBE_BANDAVG_MS, UDPSCOPE_PROBE_MATH_MS and UDPSCOPE_PROBE_MATHWRAP_MS must not be scheduled in the same run. The noise, measurement, measurements and band-average probes all push onto app.measures, and the later one would read the earlier one's rows as its own; the noise, math and math-wrap probes all write app.math, so the later one's graph lands on top of the earlier one's and both sets of readings describe a state neither probe set up. The noise probe carries its own measurement legs (see probe-readings.md), so nothing is lost by keeping them apart.

74 KiB Raw Blame History Unescape Escape

Verification

The noise probe needs a spiked mock

74 KiB

Raw Blame History