diff --git a/docs/superpowers/specs/2026-10-07-scope-features-design.md b/docs/superpowers/specs/2026-10-07-scope-features-design.md new file mode 100644 index 0000000..d80f083 --- /dev/null +++ b/docs/superpowers/specs/2026-10-07-scope-features-design.md @@ -0,0 +1,426 @@ + + + +# Three things a bench scope has and udpscope does not: qualified triggers, measurement statistics, and a search over the buffer + +udpscope triggers on one thing — a level crossing — reports each measurement as a single +number with no sense of how it has varied, and offers no way to ask where in the retained +buffer something happened. A survey of what bench scopes and the open-source viewers provide +turned up a longer list than this; it was narrowed deliberately, and what was cut is named in +**Scope** below. + +The three features chosen are not independent. A qualified trigger condition and a search over +the buffer are the *same question* asked of live data and of recorded data, so they are built +on one evaluator rather than two, and the probe that proves they agree is part of the work. +Measurement statistics are separate but cheap: the pane's recompute pass already produces the +number a statistic accumulates. + +## Scope + +**In:** + +- A shared, allocation-free `EventDetector` extracted out of `TriggerEngine`, carrying four + condition types: edge (as now), pulse width (with glitch as a comparison, not a fifth type), + window, and dropout. +- The trigger panel's new type row and the parameter rows each type needs. +- Per-measurement statistics — count, mean, standard deviation, min, max — accumulated one + observation per recompute, with a pane-wide toggle and a Reset. +- A `Search` section in the trigger panel that scans the retained ring for the trigger's own + condition, marks the hits, and navigates them by reusing the existing pause. +- Four new probes, and the mock waveforms they need. +- `SPECS.md` is not changed; `README.md` and `docs/claude/` are. + +**Out, and deliberately:** + +- **Reference waveforms** — declined explicitly. +- **PNG export** — the `.npy`/`.csv` export covers the need that matters; a screenshot is the + window manager's job. +- **FFT / spectrum** — needs a second `plot::Figure` with its own x units, which is a project + of its own and not a feature of this one. +- **Byte-bounded event history** — a second retention policy beside the ring's, with its own + eviction. Out for the same reason. +- No new measurement types. No change to `measure_scan_buckets()` / + `measure_accumulate_bucket()` or to the hysteresis band. No change to the decimation plan, the + partition, or the two-thread split. + +## Global constraints + +These bind every task below and are not restated in each one. + +- **Two threads only.** `EventDetector::feed()` runs on the producer inside + `Acquisition::on_data()`, under `ScopeModel::mutex()`. It allocates nothing, throws nothing, + does no I/O, and is `noexcept`. The statistics accumulator and the search driver run on the + GTK main thread. +- **Every new source file carries the SPDX MIT header**, matching the existing files. +- **Stream seconds, end to end.** Nothing on the trigger or search path converts a span to a + sample count, and nothing introduces a samples-per-second estimate. That conversion is the + documented cause of the "the post window is empty" defect + (`docs/claude/invariants-trigger-and-pinning.md`). +- **The detector is fed the stored (decimated, averaged) value**, never the wire value — an + event found on a sample the buffer does not hold cannot be cut from it. +- **A pinned axis is pinned from the snapshot's own stamps, never from the request.** Applies to + every pause the search publishes. +- **There is no unit test suite.** Every assertion is a probe line on stderr, driven through the + real widgets, read against the documented baselines: the documented build-warning count and + zero `Gtk-CRITICAL`. +- **No figure is written into a document unless it was measured on this machine.** Costs are + printed by probes and quoted from those runs. +- **Run one probe at a time.** The only exception is the search probe's internal + scan-versus-armed pairing, which is one probe. + +## File structure + +| File | Responsibility | +| --- | --- | +| `src/condition.hpp` / `.cpp` | **New.** `EventCondition`, `TriggerType`, `WidthCmp`, and `EventDetector` — the whole per-sample condition state machine and nothing else. No model, no GTK, no allocation. | +| `src/trigger.hpp` / `.cpp` | `TriggerConfig` grows an `EventCondition`. `TriggerEngine` keeps arming, the hold-off, the pre-window gate, the post window and the mode, and delegates detection to `EventDetector`. | +| `src/search.hpp` / `.cpp` | **New.** `BufferSearch` — the chunked scan driver over the retained ring, the hit list, the cursor into it, and the staleness state. Owns no widgets. | +| `src/measure.hpp` / `.cpp` | `MeasureStatsAccum` (Welford) beside the existing result types. | +| `src/plot_host.hpp` / `.cpp` | `kSearchMarks` cursors per row, created in `install()`, placed by a new `set_search_marks()`. | +| `src/ui.cpp` | The trigger panel's type and parameter rows, the `Search` expander, the statistics toggle and second line, the chunk pump in the frame tick, and the four probes. | +| `src/session.cpp` | The new `TriggerConfig` fields and the statistics toggle. | +| `tools/mock_udpstreamer.py` | `--glitch-at` / `--glitch-width` / `--quiet-at`. | +| `README.md`, `docs/claude/*` | What was built, and the invariants it adds. | + +`src/condition.*` and `src/search.*` are new files rather than additions to `src/trigger.cpp` +and `src/ui.cpp` for two reasons: the detector must be includable by the search without +dragging in the engine's arming state, and `src/ui.cpp` is already the largest file in the +project. + +## 1. The shared event condition + +### What is extracted + +`TriggerEngine::process()` (`src/trigger.cpp:38-106`) currently does three jobs. Lines 78-84 +detect a crossing; line 92 gates it on the pre-trigger window; lines 42-75 run the +pending/post-window/hold-off/mode machine. Only the first is what a search means by "a +condition", so only the first moves. + +```cpp +enum class TriggerType { Edge, PulseWidth, Window, Dropout }; +enum class WidthCmp { Less, Greater, Within, Outside }; + +struct EventCondition { + TriggerType type = TriggerType::Edge; + double level = 0.0; + double level2 = 0.0; ///< Window: the band's other edge. Ignored otherwise. + TriggerEdge edge = TriggerEdge::Rising; + WidthCmp cmp = WidthCmp::Greater; ///< PulseWidth only. + double width_s = 1.0e-3; ///< PulseWidth only. + double width2_s = 2.0e-3; ///< PulseWidth, Within/Outside only. + double timeout_s = 1.0e-3; ///< Dropout only. + bool operator==(const EventCondition&) const noexcept; +}; + +class EventDetector { + public: + /// Adopts the condition. Resets the state iff the condition differs. + void configure(const EventCondition& c) noexcept; + void reset() noexcept; + /// One sample, in stream order. Returns the event's own stream time, or NaN for no event. + double feed(double value, double t) noexcept; +}; +``` + +`feed()` returns a **time** rather than a bool because for two of the four types the event's +instant is not the sample being looked at: a pulse width is only known at the pulse's closing +edge, and a dropout fires on an ordinary sample nowhere near an edge. + +`TriggerEngine::process()` then reads: + +```cpp +const double event_t = detector_.feed(value, now_s); +if (!std::isfinite(event_t)) return false; +if (event_t - oldest_s_ < pre_seconds_) return false; // the gate, now on the event's own t +last_trigger_s_ = event_t; +pending_ = true; +pending_start_s_ = event_t; +``` + +Note the gate and `pending_start_s_` both move from `now_s` to `event_t`. For `Edge` they are +the same value and the behaviour is bit-identical to today; for a long qualifying pulse they +differ, and using `now_s` would measure the post window from the wrong instant. + +`TriggerEngine::configure()` keeps its name-keyed reset exactly as it is +(`docs/claude/invariants-trigger-and-pinning.md`) and additionally calls +`detector_.configure(cfg.condition)`, which resets the detector only when the condition +actually changed — a condition re-applied every packet must not restart the state machine, or a +pulse-width condition could never see a pulse end. + +### The four types + +| Type | Fires when | The instant it calls t0 | +| --- | --- | --- | +| `Edge` | `level` is crossed in `edge`'s direction | the crossing sample | +| `PulseWidth` | a pulse — a crossing in `edge`'s direction, then the crossing back — whose duration satisfies `cmp` against `width_s` (and `width2_s` for `Within`/`Outside`) | **the closing edge**: the width is not known before it | +| `Window` | the signal enters or leaves `[min(level,level2), max(level,level2)]`; `Rising` = enters, `Falling` = leaves, `Both` = either | the crossing sample | +| `Dropout` | `level` has not been crossed for longer than `timeout_s` | the first sample at which the timeout is observed | + +**Glitch is `PulseWidth` with `cmp = Less`** and is not a separate type. That is how bench +scopes spell it, and one state machine cannot disagree with itself about where a pulse ended. +The panel may label the preset "glitch". + +`Dropout`'s t0 is the sample at which the expiry is *observed*, not the exact expiry instant, +for the same reason every readout in this app quotes a stored sample: t0 has to be a sample the +buffer holds, because the cut is a binary search over the time ring and the axis is pinned to +what the cut got. + +A `Window` condition with `level == level2` degrades to an edge on that level, and must — a +band of zero width is not a special mode. + +### Panel and session + +The trigger panel gains a **type** combo above the existing rows, and the rows below it change +with the type: `Edge` shows the level, edge and pre-% rows it shows now; `PulseWidth` adds a +comparison combo and one or two width spins; `Window` turns the single level spin into two; +`Dropout` replaces the width with a timeout. Hold-off, pre-% and Normal/Single are common to +all four and do not move. + +No plot instrument is added. The trigger level has never been drawn on the plot — it is a spin +button (`src/ui.cpp:135`, `:1604`) — so a band needs no second ruler. + +`TriggerConfig` gains one `EventCondition` member; its `level` and `edge` replace the +`TriggerConfig` fields of those names, and every reader moves with them. The session file gains +the new keys **appended**, since the order is load-bearing +(`docs/claude/invariants-session-and-export.md`); a file without them loads as +`TriggerType::Edge` with the level, edge, hold-off, pre-% and mode it already carries, which is +exactly today's behaviour. No version bump, no migration. + +The level's existing default — the middle of the signal's declared range until the user edits it +(`src/ui.cpp:2016-2054`) — applies to `Window` as a band about that middle, and `level2` +defaults with it. + +## 2. Measurement statistics + +### What is accumulated + +Not the samples: `Average` is already the mean of every visible sample. A statistic is of the +**measurement's own reported value over time** — count, mean, standard deviation, min and max of +every value that measurement has produced since the last reset. + +```cpp +/// The variation of one measurement's own value over time. Fixed size, no allocation: it is +/// folded in the pane's existing recompute pass, which must not grow a second pass. +struct MeasureStatsAccum { + std::uint64_t n = 0; + double mean = 0.0; ///< Welford's running mean + double m2 = 0.0; ///< Welford's sum of squared deviations + double lo = 0.0; + double hi = 0.0; + void add(double v) noexcept; + void reset() noexcept; + double sd() const noexcept; ///< sqrt(m2 / (n - 1)), 0 for n < 2 +}; +``` + +Welford rather than `sum` and `sum_of_squares`: on a value with a large offset — a 3.3 V logic +level whose σ is a few millivolts — the naive form loses the variance entirely to cancellation +in `Σv²`. + +### One observation per recompute + +`update_measures()` is throttled deliberately (`src/ui.cpp:3141-3162`): at most one pass per +`kMeasureRefreshUs` while the window scrolls, four times that when a pass is heavy, and exactly +**once when the window settles** — the `!changed` early return at `:3153` is what makes a pinned +window recompute once rather than every frame. The accumulator takes one observation per pass, +so: + +- A scrolling live view contributes on the order of ten observations a second. +- A paused view, a held event or a pinned window contributes **one**. +- `n` is therefore a count of *recomputes* — not of samples, frames or seconds. The pane labels + it `n` and its tooltip says so. Presenting it as a sample count would be a figure the user + acts on and that is false. + +Two exclusions, each of which would corrupt the statistic silently: + +- An **invalid** result (`--`) contributes nothing, to neither `n` nor the extremes. +- The **`measure_last_valid` fallback** (`src/ui.cpp:3363`) is **not** accumulated. It exists so + a level mark does not blink off and is by construction a repeat of an observation already + counted; folding it in pulls the mean toward whatever preceded a dropout and collapses σ. + +### Reset + +An explicit **Reset** button in the pane clears every accumulator. Three changes clear them +implicitly, because each one makes the values before and after a different population: + +- the measurement is edited (signal, type or options); +- a CONFIG arrives — the signal list is rebuilt and a name may now resolve elsewhere; +- the display window changes — a mean over 2 s and a mean over 600 s are different statistics. + +Deliberately **not** cleared by a pause, a resume, an event latch, or a pan/zoom within the +window. Each of those contributes its one settled observation. + +Accumulators are not persisted. A statistic over a session that has ended is not a statistic; +the **toggle's state is** persisted. + +### Display + +One `Statistics` check button at the top of the measurements pane. Off — the default — the pane +is exactly as it is now. On, every row grows a second line beneath it, dimmed and smaller: + +``` +● ● frequency 5.0003 Hz edit + x̄ 5.0001 σ 0.0004 ▾ 4.9989 ▴ 5.0012 n 143 +``` + +Pane-wide rather than per-measurement, so that the second lines align and are comparable, and +because the per-measurement dialog is already the busiest surface in the app. + +The second line is a `GtkLabel` in a vertical box wrapping the existing horizontal row, built in +`measures_row()` (`src/ui.cpp:2495`) and shown or hidden by the toggle — the toggle must **not** +call `rebuild_measures_pane()`, which drops and recreates every label including the ones the +frame tick holds pointers to. All five numbers go through the same `format_measure_value()` the +value above them uses, so units and significant figures agree by construction. + +## 3. Search and navigate + +### What it searches, and with what + +A `Search` expander at the bottom of the trigger panel, searching **the trigger's own +`EventCondition`** read straight off `TriggerConfig`. There is no second condition UI: a +condition is set up once and can either be armed to capture live or scanned for over the buffer +already held. That is what the shared evaluator buys, and it gives the search all four types for +free. + +It runs on the **trigger's channel** over the **retained ring** — not the visible window, which +the user can already read by looking. + +### The scan is exact, chunked, and progressive + +It feeds the same `EventDetector`, sample by sample, in stream order, raw. No bucket-skipping: a +level-0 bucket whose `[lo,hi]` does not straddle the level cannot hold a crossing, so skipping +it is tempting, but the detector is a sequential state machine and synthesising its previous +value across a skipped span is precisely how the search and the live trigger would come to +disagree about where an edge was. The shared evaluator exists to make that impossible. + +A raw pass over the whole ring cannot be one pass under one lock — that holds +`ScopeModel::mutex()` long enough to starve the producer. `BufferSearch` therefore advances in +**bounded chunks pumped from the frame tick**, taking and releasing the lock per chunk, the same +shape `update_measures()` uses to materialise its window in chunks. The detector's state carries +across chunks because it is sequential. The panel shows progress and a Cancel. + +Two limits, both reported in the panel rather than hidden: + +- **The ring keeps filling during the scan** — a pause freezes the view, not the acquisition. + Each chunk re-clamps to the ring's current base; if the base has passed the chunk's start the + scan stops and the panel says so (`searched 412 s of 600 s — the oldest data was overwritten + while scanning`). Reachable only at high rates on a small buffer. +- **The hit list is capped** at `kSearchHitsMax = 10000`, and the panel says + `stopped at 10000 matches`. A condition matching every sample is a mis-set condition; the + answer is a message, not millions of doubles. + +### The hits, and navigating them + +Hits are a `std::vector` of stream times, each the t0 its condition defines. The panel +shows `147 matches` with `◀ prev`, `next ▶` and `3 / 147`, with key bindings while the panel has +focus. + +Navigation reuses the existing pause. Jumping to hit `t` pauses if not already paused and calls +the primitive that is already there (`src/ui.cpp:2188-2199`): + +```cpp +model->publish_pause_view(t - w / 2.0, t + w / 2.0, /*x_offset=*/t); +if (auto snap = model->pause_view(); snap && !snap->stamps.empty()) + rows->pin_pause_window(snap->stamps.front(), snap->stamps.back()); +``` + +`w` is the current display window. `x_offset = t` puts the hit at **x = 0**, exactly where a +held event's edge sits, so the two ways of looking at an event read alike and the x-label logic +needs no new case. The pin comes off the snapshot's own stamps and never off the request: a hit +near the oldest retained sample yields a shorter cut, and pinning the request leaves an empty +left margin that a zoom-in walks into. + +Resuming or clearing the search resumes and unpins exactly as the pause button does. The hit +list survives a resume — it describes data the ring may still hold — but goes stale, so the +panel marks its age, and any structural rebuild (a CONFIG, an enable/disable, a calibration +change, or a window change that moves `time_stride`) clears it outright, because those drop the +history it refers to. + +### Marks + +Cursors are not dynamic: `RowSet` creates them up front in fixed arrays +(`kMeasureMarks = 6`, `src/plot_host.hpp:55`) because a `Plot` cannot give one up. So this adds +`kSearchMarks = 16` cursors per row, created in the same `install()` loop +(`src/plot_host.cpp:185-188`) and taking the same read-only dashed 0.75 px stroke the measure +marks take, parked when unused. + +Every frame, the hits inside the visible window are placed on them, nearest to the view's centre +first, with the current hit in a distinct colour. When more hits are in view than there are +marks the panel reads `16 of 38 marks shown — zoom in to see the rest`, which is a true +statement about the picture rather than a silent truncation. Like the measure marks they are +re-placed every frame, which is what makes them read-only: a gesture moves one and this puts it +back. + +## 4. Verification + +Four probes, each scheduled by an env var, each driving the real widgets, each printing tagged +lines to stderr. Added to `docs/claude/verification.md` and `docs/claude/probe-readings.md` with +the readings a correct run gives — measured, never assumed. + +**The numbers in the sample lines below, and in the pane mock-up above, show the line's *shape* +only.** None of them has been measured; nothing may copy them into a document as a reading. The +figures that go into `verification.md` come from a run. + +**`UDPSCOPE_PROBE_CONDITION_MS`** — for each of `Edge`, `PulseWidth` with `Less`, `PulseWidth` +with `Greater`, `Window` and `Dropout`: set the type and its parameters through the panel, arm, +wait for the latch, print + +``` +[condition-probe] type=pulsewidth cmp=less width=0.0005 t0=4.3021 want=4.3020 ok=1 +``` + +`want` comes from the mock's injection schedule. `ok=1` is the reading. + +**`UDPSCOPE_PROBE_STATS_MS`** — add a `frequency` measurement on the mock's sine, turn the +statistics toggle on, let several recomputes land, print + +``` +[stats-probe] n=14 mean=5.0001 sd=0.0004 lo=4.9989 hi=5.0012 toggle=1 line2=1 +``` + +then press Reset and print it again: `n=0` is the assertion. It must also hold a pause for a +stretch and show `n` advancing by **exactly 1** across it — that is what distinguishes +accumulating per recompute from accumulating per frame. With the `measure_last_valid` fallback +wrongly accumulated, `n` climbs through an injected dropout while the value does not change and +`sd` collapses; that is the second defect this probe exists for. + +**`UDPSCOPE_PROBE_SEARCH_MS`** — configure a glitch condition, scan to completion against the +injected pulse train, print + +``` +[search-probe] matches=7 first=1.3020 last=6.8020 scanned=8.21/8.21 s truncated=0 capped=0 +[search-probe] jump 1/7 t=1.3020 axis=[-1.0000..1.0000] snap=[-1.0000..1.0000] inside=1 x0_hit=1 +``` + +for three successive `next` presses. `matches` is asserted against the injection count, +`inside=1` is the pause-pin invariant, `x0_hit=1` is the hit sitting at x = 0. + +**The cross-check that makes the shared evaluator a claim and not a hope.** The same probe run, +same condition, same mock, **armed** instead of searched: the first latch's t0 must equal the +scan's `first` exactly. + +``` +[search-probe] armed t0=1.3020 vs scanned first=1.3020 agree=1 +``` + +A divergence between the live and the offline path appears here and nowhere else. + +**Costs are printed, not asserted**, because an unmeasured figure drifts: `scan cost: N chunks, +X ms total, worst chunk Y ms` — the **worst chunk** is the invariant worth watching, since it is +what the producer waited on; the total is allowed to be long. Statistics accumulation is folded +into the existing `measure cost:` line. + +**Mock additions:** `--glitch-at ` / `--glitch-width ` inject a narrow pulse into a pulse +train at a known instant, and `--quiet-at ` stops a signal crossing its level from a known +instant. Both are permanent regression cases, like `--time-glitch-at`. + +## Risks + +- **`TriggerConfig`'s `level`/`edge` move into `EventCondition`**, so every reader changes — + the panel, the session, the probes and the status line. Mechanical, but wide, and the + mitigation is that `Edge` behaviour must be bit-identical afterwards: the existing trigger + probes are the regression test and are run before anything new is added. +- **The scan's worst chunk is a latency the producer pays.** The chunk size is the one knob, it + is picked from the measured figure, and the probe prints the worst chunk every run. +- **`kSearchMarks = 16` is a fixed cost in cursors per row** on every plot whether or not a + search is running, as `kMeasureMarks = 6` already is.