udpscope/docs/superpowers/specs/2026-10-07-scope-features-design.md

<!-- SPDX-License-Identifier: MIT -->
<!-- SPDX-FileCopyrightText: 2026 Martino Ferrari -->

# Three things a bench scope has and udpscope does not: qualified triggers, measurement statistics, and a search over the buffer

udpscope triggers on one thing — a level crossing — reports each measurement as a single
number with no sense of how it has varied, and offers no way to ask where in the retained
buffer something happened. A survey of what bench scopes and the open-source viewers provide
turned up a longer list than this; it was narrowed deliberately, and what was cut is named in
**Scope** below.

The three features chosen are not independent. A qualified trigger condition and a search over
the buffer are the *same question* asked of live data and of recorded data, so they are built
on one evaluator rather than two, and the probe that proves they agree is part of the work.
Measurement statistics are separate but cheap: the pane's recompute pass already produces the
number a statistic accumulates.

## Scope

**In:**

- A shared, allocation-free `EventDetector` extracted out of `TriggerEngine`, carrying four
  condition types: edge (as now), pulse width (with glitch as a comparison, not a fifth type),
  window, and dropout.
- The trigger panel's new type row and the parameter rows each type needs.
- Per-measurement statistics — count, mean, standard deviation, min, max — accumulated one
  observation per recompute, with a pane-wide toggle and a Reset.
- A `Search` section in the trigger panel that scans the retained ring for the trigger's own
  condition, marks the hits, and navigates them by reusing the existing pause.
- Four new probes, and the mock waveforms they need.
- `SPECS.md` is not changed; `README.md` and `docs/claude/` are.

**Out, and deliberately:**

- **Reference waveforms** — declined explicitly.
- **PNG export** — the `.npy`/`.csv` export covers the need that matters; a screenshot is the
  window manager's job.
- **FFT / spectrum** — needs a second `plot::Figure` with its own x units, which is a project
  of its own and not a feature of this one.
- **Byte-bounded event history** — a second retention policy beside the ring's, with its own
  eviction. Out for the same reason.
- No new measurement types. No change to `measure_scan_buckets()` /
  `measure_accumulate_bucket()` or to the hysteresis band. No change to the decimation plan, the
  partition, or the two-thread split.

## Global constraints

These bind every task below and are not restated in each one.

- **Two threads only.** `EventDetector::feed()` runs on the producer inside
  `Acquisition::on_data()`, under `ScopeModel::mutex()`. It allocates nothing, throws nothing,
  does no I/O, and is `noexcept`. The statistics accumulator and the search driver run on the
  GTK main thread.
- **Every new source file carries the SPDX MIT header**, matching the existing files.
- **Stream seconds, end to end.** Nothing on the trigger or search path converts a span to a
  sample count, and nothing introduces a samples-per-second estimate. That conversion is the
  documented cause of the "the post window is empty" defect
  (`docs/claude/invariants-trigger-and-pinning.md`).
- **The detector is fed the stored (decimated, averaged) value**, never the wire value — an
  event found on a sample the buffer does not hold cannot be cut from it.
- **A pinned axis is pinned from the snapshot's own stamps, never from the request.** Applies to
  every pause the search publishes.
- **There is no unit test suite.** Every assertion is a probe line on stderr, driven through the
  real widgets, read against the documented baselines: the documented build-warning count and
  zero `Gtk-CRITICAL`.
- **No figure is written into a document unless it was measured on this machine.** Costs are
  printed by probes and quoted from those runs.
- **Run one probe at a time.** The only exception is the search probe's internal
  scan-versus-armed pairing, which is one probe.

## File structure

| File | Responsibility |
| --- | --- |
| `src/condition.hpp` / `.cpp` | **New.** `EventCondition`, `TriggerType`, `WidthCmp`, and `EventDetector` — the whole per-sample condition state machine and nothing else. No model, no GTK, no allocation. |
| `src/trigger.hpp` / `.cpp` | `TriggerConfig` grows an `EventCondition`. `TriggerEngine` keeps arming, the hold-off, the pre-window gate, the post window and the mode, and delegates detection to `EventDetector`. |
| `src/search.hpp` / `.cpp` | **New.** `BufferSearch` — the chunked scan driver over the retained ring, the hit list, the cursor into it, and the staleness state. Owns no widgets. |
| `src/measure.hpp` / `.cpp` | `MeasureStatsAccum` (Welford) beside the existing result types. |
| `src/plot_host.hpp` / `.cpp` | `kSearchMarks` cursors per row, created in `install()`, placed by a new `set_search_marks()`. |
| `src/ui.cpp` | The trigger panel's type and parameter rows, the `Search` expander, the statistics toggle and second line, the chunk pump in the frame tick, and the four probes. |
| `src/session.cpp` | The new `TriggerConfig` fields and the statistics toggle. |
| `tools/mock_udpstreamer.py` | `--glitch-at` / `--glitch-width` / `--quiet-at`. |
| `README.md`, `docs/claude/*` | What was built, and the invariants it adds. |

`src/condition.*` and `src/search.*` are new files rather than additions to `src/trigger.cpp`
and `src/ui.cpp` for two reasons: the detector must be includable by the search without
dragging in the engine's arming state, and `src/ui.cpp` is already the largest file in the
project.

## 1. The shared event condition

### What is extracted

`TriggerEngine::process()` (`src/trigger.cpp:38-106`) currently does three jobs. Lines 78-84
detect a crossing; line 92 gates it on the pre-trigger window; lines 42-75 run the
pending/post-window/hold-off/mode machine. Only the first is what a search means by "a
condition", so only the first moves.

```cpp
enum class TriggerType { Edge, PulseWidth, Window, Dropout };
enum class WidthCmp { Less, Greater, Within, Outside };

struct EventCondition {
  TriggerType type = TriggerType::Edge;
  double level = 0.0;
  double level2 = 0.0;          ///< Window: the band's other edge. Ignored otherwise.
  TriggerEdge edge = TriggerEdge::Rising;
  WidthCmp cmp = WidthCmp::Greater;   ///< PulseWidth only.
  double width_s = 1.0e-3;            ///< PulseWidth only.
  double width2_s = 2.0e-3;           ///< PulseWidth, Within/Outside only.
  double timeout_s = 1.0e-3;          ///< Dropout only.
  bool operator==(const EventCondition&) const noexcept;
};

class EventDetector {
 public:
  /// Adopts the condition. Resets the state iff the condition differs.
  void configure(const EventCondition& c) noexcept;
  void reset() noexcept;
  /// One sample, in stream order. Returns the event's own stream time, or NaN for no event.
  double feed(double value, double t) noexcept;
};
```

`feed()` returns a **time** rather than a bool because for two of the four types the event's
instant is not the sample being looked at: a pulse width is only known at the pulse's closing
edge, and a dropout fires on an ordinary sample nowhere near an edge.

`TriggerEngine::process()` then reads:

```cpp
const double event_t = detector_.feed(value, now_s);
if (!std::isfinite(event_t)) return false;
if (event_t - oldest_s_ < pre_seconds_) return false;   // the gate, now on the event's own t
last_trigger_s_ = event_t;
pending_ = true;
pending_start_s_ = event_t;
```

Note the gate and `pending_start_s_` both move from `now_s` to `event_t`. For `Edge` they are
the same value and the behaviour is bit-identical to today; for a long qualifying pulse they
differ, and using `now_s` would measure the post window from the wrong instant.

`TriggerEngine::configure()` keeps its name-keyed reset exactly as it is
(`docs/claude/invariants-trigger-and-pinning.md`) and additionally calls
`detector_.configure(cfg.condition)`, which resets the detector only when the condition
actually changed — a condition re-applied every packet must not restart the state machine, or a
pulse-width condition could never see a pulse end.

### The four types

| Type | Fires when | The instant it calls t0 |
| --- | --- | --- |
| `Edge` | `level` is crossed in `edge`'s direction | the crossing sample |
| `PulseWidth` | a pulse — a crossing in `edge`'s direction, then the crossing back — whose duration satisfies `cmp` against `width_s` (and `width2_s` for `Within`/`Outside`) | **the closing edge**: the width is not known before it |
| `Window` | the signal enters or leaves `[min(level,level2), max(level,level2)]`; `Rising` = enters, `Falling` = leaves, `Both` = either | the crossing sample |
| `Dropout` | `level` has not been crossed for longer than `timeout_s` | the first sample at which the timeout is observed |

**Glitch is `PulseWidth` with `cmp = Less`** and is not a separate type. That is how bench
scopes spell it, and one state machine cannot disagree with itself about where a pulse ended.
The panel may label the preset "glitch".

`Dropout`'s t0 is the sample at which the expiry is *observed*, not the exact expiry instant,
for the same reason every readout in this app quotes a stored sample: t0 has to be a sample the
buffer holds, because the cut is a binary search over the time ring and the axis is pinned to
what the cut got.

A `Window` condition with `level == level2` degrades to an edge on that level, and must — a
band of zero width is not a special mode.

### Panel and session

The trigger panel gains a **type** combo above the existing rows, and the rows below it change
with the type: `Edge` shows the level, edge and pre-% rows it shows now; `PulseWidth` adds a
comparison combo and one or two width spins; `Window` turns the single level spin into two;
`Dropout` replaces the width with a timeout. Hold-off, pre-% and Normal/Single are common to
all four and do not move.

No plot instrument is added. The trigger level has never been drawn on the plot — it is a spin
button (`src/ui.cpp:135`, `:1604`) — so a band needs no second ruler.

`TriggerConfig` gains one `EventCondition` member; its `level` and `edge` replace the
`TriggerConfig` fields of those names, and every reader moves with them. The session file gains
the new keys **appended**, since the order is load-bearing
(`docs/claude/invariants-session-and-export.md`); a file without them loads as
`TriggerType::Edge` with the level, edge, hold-off, pre-% and mode it already carries, which is
exactly today's behaviour. No version bump, no migration.

The level's existing default — the middle of the signal's declared range until the user edits it
(`src/ui.cpp:2016-2054`) — applies to `Window` as a band about that middle, and `level2`
defaults with it.

## 2. Measurement statistics

### What is accumulated

Not the samples: `Average` is already the mean of every visible sample. A statistic is of the
**measurement's own reported value over time** — count, mean, standard deviation, min and max of
every value that measurement has produced since the last reset.

```cpp
/// The variation of one measurement's own value over time. Fixed size, no allocation: it is
/// folded in the pane's existing recompute pass, which must not grow a second pass.
struct MeasureStatsAccum {
  std::uint64_t n = 0;
  double mean = 0.0;   ///< Welford's running mean
  double m2 = 0.0;     ///< Welford's sum of squared deviations
  double lo = 0.0;
  double hi = 0.0;
  void add(double v) noexcept;
  void reset() noexcept;
  double sd() const noexcept;   ///< sqrt(m2 / (n - 1)), 0 for n < 2
};
```

Welford rather than `sum` and `sum_of_squares`: on a value with a large offset — a 3.3 V logic
level whose σ is a few millivolts — the naive form loses the variance entirely to cancellation
in `Σv²`.

### One observation per recompute

`update_measures()` is throttled deliberately (`src/ui.cpp:3141-3162`): at most one pass per
`kMeasureRefreshUs` while the window scrolls, four times that when a pass is heavy, and exactly
**once when the window settles** — the `!changed` early return at `:3153` is what makes a pinned
window recompute once rather than every frame. The accumulator takes one observation per pass,
so:

- A scrolling live view contributes on the order of ten observations a second.
- A paused view, a held event or a pinned window contributes **one**.
- `n` is therefore a count of *recomputes* — not of samples, frames or seconds. The pane labels
  it `n` and its tooltip says so. Presenting it as a sample count would be a figure the user
  acts on and that is false.

Two exclusions, each of which would corrupt the statistic silently:

- An **invalid** result (`--`) contributes nothing, to neither `n` nor the extremes.
- The **`measure_last_valid` fallback** (`src/ui.cpp:3363`) is **not** accumulated. It exists so
  a level mark does not blink off and is by construction a repeat of an observation already
  counted; folding it in pulls the mean toward whatever preceded a dropout and collapses σ.

### Reset

An explicit **Reset** button in the pane clears every accumulator. Three changes clear them
implicitly, because each one makes the values before and after a different population:

- the measurement is edited (signal, type or options);
- a CONFIG arrives — the signal list is rebuilt and a name may now resolve elsewhere;
- the display window changes — a mean over 2 s and a mean over 600 s are different statistics.

Deliberately **not** cleared by a pause, a resume, an event latch, or a pan/zoom within the
window. Each of those contributes its one settled observation.

Accumulators are not persisted. A statistic over a session that has ended is not a statistic;
the **toggle's state is** persisted.

### Display

One `Statistics` check button at the top of the measurements pane. Off — the default — the pane
is exactly as it is now. On, every row grows a second line beneath it, dimmed and smaller:

```
● ● frequency                    5.0003 Hz   edit
    x̄ 5.0001  σ 0.0004  ▾ 4.9989  ▴ 5.0012  n 143
```

Pane-wide rather than per-measurement, so that the second lines align and are comparable, and
because the per-measurement dialog is already the busiest surface in the app.

The second line is a `GtkLabel` in a vertical box wrapping the existing horizontal row, built in
`measures_row()` (`src/ui.cpp:2495`) and shown or hidden by the toggle — the toggle must **not**
call `rebuild_measures_pane()`, which drops and recreates every label including the ones the
frame tick holds pointers to. All five numbers go through the same `format_measure_value()` the
value above them uses, so units and significant figures agree by construction.

## 3. Search and navigate

### What it searches, and with what

A `Search` expander at the bottom of the trigger panel, searching **the trigger's own
`EventCondition`** read straight off `TriggerConfig`. There is no second condition UI: a
condition is set up once and can either be armed to capture live or scanned for over the buffer
already held. That is what the shared evaluator buys, and it gives the search all four types for
free.

It runs on the **trigger's channel** over the **retained ring** — not the visible window, which
the user can already read by looking.

### The scan is exact, chunked, and progressive

It feeds the same `EventDetector`, sample by sample, in stream order, raw. No bucket-skipping: a
level-0 bucket whose `[lo,hi]` does not straddle the level cannot hold a crossing, so skipping
it is tempting, but the detector is a sequential state machine and synthesising its previous
value across a skipped span is precisely how the search and the live trigger would come to
disagree about where an edge was. The shared evaluator exists to make that impossible.

A raw pass over the whole ring cannot be one pass under one lock — that holds
`ScopeModel::mutex()` long enough to starve the producer. `BufferSearch` therefore advances in
**bounded chunks pumped from the frame tick**, taking and releasing the lock per chunk, the same
shape `update_measures()` uses to materialise its window in chunks. The detector's state carries
across chunks because it is sequential. The panel shows progress and a Cancel.

Two limits, both reported in the panel rather than hidden:

- **The ring keeps filling during the scan** — a pause freezes the view, not the acquisition.
  Each chunk re-clamps to the ring's current base; if the base has passed the chunk's start the
  scan stops and the panel says so (`searched 412 s of 600 s — the oldest data was overwritten
  while scanning`). Reachable only at high rates on a small buffer.
- **The hit list is capped** at `kSearchHitsMax = 10000`, and the panel says
  `stopped at 10000 matches`. A condition matching every sample is a mis-set condition; the
  answer is a message, not millions of doubles.

### The hits, and navigating them

Hits are a `std::vector<double>` of stream times, each the t0 its condition defines. The panel
shows `147 matches` with `◀ prev`, `next ▶` and `3 / 147`, with key bindings while the panel has
focus.

Navigation reuses the existing pause. Jumping to hit `t` pauses if not already paused and calls
the primitive that is already there (`src/ui.cpp:2188-2199`):

```cpp
model->publish_pause_view(t - w / 2.0, t + w / 2.0, /*x_offset=*/t);
if (auto snap = model->pause_view(); snap && !snap->stamps.empty())
  rows->pin_pause_window(snap->stamps.front(), snap->stamps.back());
```

`w` is the current display window. `x_offset = t` puts the hit at **x = 0**, exactly where a
held event's edge sits, so the two ways of looking at an event read alike and the x-label logic
needs no new case. The pin comes off the snapshot's own stamps and never off the request: a hit
near the oldest retained sample yields a shorter cut, and pinning the request leaves an empty
left margin that a zoom-in walks into.

Resuming or clearing the search resumes and unpins exactly as the pause button does. The hit
list survives a resume — it describes data the ring may still hold — but goes stale, so the
panel marks its age, and any structural rebuild (a CONFIG, an enable/disable, a calibration
change, or a window change that moves `time_stride`) clears it outright, because those drop the
history it refers to.

### Marks

Cursors are not dynamic: `RowSet` creates them up front in fixed arrays
(`kMeasureMarks = 6`, `src/plot_host.hpp:55`) because a `Plot` cannot give one up. So this adds
`kSearchMarks = 16` cursors per row, created in the same `install()` loop
(`src/plot_host.cpp:185-188`) and taking the same read-only dashed 0.75 px stroke the measure
marks take, parked when unused.

Every frame, the hits inside the visible window are placed on them, nearest to the view's centre
first, with the current hit in a distinct colour. When more hits are in view than there are
marks the panel reads `16 of 38 marks shown — zoom in to see the rest`, which is a true
statement about the picture rather than a silent truncation. Like the measure marks they are
re-placed every frame, which is what makes them read-only: a gesture moves one and this puts it
back.

## 4. Verification

Four probes, each scheduled by an env var, each driving the real widgets, each printing tagged
lines to stderr. Added to `docs/claude/verification.md` and `docs/claude/probe-readings.md` with
the readings a correct run gives — measured, never assumed.

**The numbers in the sample lines below, and in the pane mock-up above, show the line's *shape*
only.** None of them has been measured; nothing may copy them into a document as a reading. The
figures that go into `verification.md` come from a run.

**`UDPSCOPE_PROBE_CONDITION_MS`** — for each of `Edge`, `PulseWidth` with `Less`, `PulseWidth`
with `Greater`, `Window` and `Dropout`: set the type and its parameters through the panel, arm,
wait for the latch, print

```
[condition-probe] type=pulsewidth cmp=less width=0.0005 t0=4.3021 want=4.3020 ok=1
```

`want` comes from the mock's injection schedule. `ok=1` is the reading.

**`UDPSCOPE_PROBE_STATS_MS`** — add a `frequency` measurement on the mock's sine, turn the
statistics toggle on, let several recomputes land, print

```
[stats-probe] n=14 mean=5.0001 sd=0.0004 lo=4.9989 hi=5.0012 toggle=1 line2=1
```

then press Reset and print it again: `n=0` is the assertion. It must also hold a pause for a
stretch and show `n` advancing by **exactly 1** across it — that is what distinguishes
accumulating per recompute from accumulating per frame. With the `measure_last_valid` fallback
wrongly accumulated, `n` climbs through an injected dropout while the value does not change and
`sd` collapses; that is the second defect this probe exists for.

**`UDPSCOPE_PROBE_SEARCH_MS`** — configure a glitch condition, scan to completion against the
injected pulse train, print

```
[search-probe] matches=7 first=1.3020 last=6.8020 scanned=8.21/8.21 s truncated=0 capped=0
[search-probe] jump 1/7 t=1.3020 axis=[-1.0000..1.0000] snap=[-1.0000..1.0000] inside=1 x0_hit=1
```

for three successive `next` presses. `matches` is asserted against the injection count,
`inside=1` is the pause-pin invariant, `x0_hit=1` is the hit sitting at x = 0.

**The cross-check that makes the shared evaluator a claim and not a hope.** The same probe run,
same condition, same mock, **armed** instead of searched: the first latch's t0 must equal the
scan's `first` exactly.

```
[search-probe] armed t0=1.3020 vs scanned first=1.3020 agree=1
```

A divergence between the live and the offline path appears here and nowhere else.

**Costs are printed, not asserted**, because an unmeasured figure drifts: `scan cost: N chunks,
X ms total, worst chunk Y ms` — the **worst chunk** is the invariant worth watching, since it is
what the producer waited on; the total is allowed to be long. Statistics accumulation is folded
into the existing `measure cost:` line.

**Mock additions:** `--glitch-at <s>` / `--glitch-width <s>` inject a narrow pulse into a pulse
train at a known instant, and `--quiet-at <s>` stops a signal crossing its level from a known
instant. Both are permanent regression cases, like `--time-glitch-at`.

## Risks

- **`TriggerConfig`'s `level`/`edge` move into `EventCondition`**, so every reader changes —
  the panel, the session, the probes and the status line. Mechanical, but wide, and the
  mitigation is that `Edge` behaviour must be bit-identical afterwards: the existing trigger
  probes are the regression test and are run before anything new is added.
- **The scan's worst chunk is a latency the producer pays.** The chunk size is the one knob, it
  is picked from the measured figure, and the probe prints the worst chunk every run.
- **`kSearchMarks = 16` is a fixed cost in cursors per row** on every plot whether or not a
  search is running, as `kMeasureMarks = 6` already is.