udpscope/docs/claude/invariants-measurements.md

# Invariants: the measurements pane

Why the pane is refreshed on a cadence, why neither pass may read its window raw, and how a crossing is counted.

Part of [CLAUDE.md](../../CLAUDE.md). These are rules that cost a defect to learn; read the ones touching the area you are editing before you edit it.

- **The measurements pane is refreshed on a cadence, and its cost is the *window's*, not the
  frame's.** A measurement is a statistic of the visible window and a live view moves that window
  every frame, so "recompute when the window moves" is "recompute every frame" at O(visible
  samples) *per measurement*. With the pane holding four measurements over a 4.7M-sample window
  that measured **13.66 ms of tick mean** (baseline 0.41 ms) and cost the run a third of its
  frames. Three things fixed it, and all three matter: `update_measures()` recomputes at most
  every `kMeasureRefreshUs` (100 ms; 400 ms when a pass costs more than `kMeasureHeavyUs`), once
  when the window settles and once per mode change -- and reading *that* backwards is what made
  it worse first, 457 recomputes in one run, because a pinned window then recomputed every
  frame; the window is materialized in bounded chunks (never two whole-window f32 temporaries)
  and the loop holds it **by reference** (copying both vectors per measurement was 300 MB of
  memcpy a frame); and `measure_stats()` is one shared pass carrying both extremes with their
  first positions, the sum and the count, so average/min/max/peak-to-peak need no pass of their
  own -- `top`/`bottom`/`amplitude` take a second read of that same **bucket** list and never of
  the window (the flat-level invariant below). **Adding a type must not add a pass** -- that is
  what `compute_measure()`'s `stats` argument is for.
  The marks' *ink* is set where they are created (`install()` in `plot_host.cpp`): 0.75 px and
  dashed 4/4 through `Plot::set_cursor_stroke()`, which is per cursor because the trigger marker
  (also a cursor) keeps the solid `line_width_px` stroke. A new measure mark must take that
  stroke -- it is created in the same loop, so it does; a new *instrument* should not.
  The **marks** the pane draws (`set_measure_marks()` -- full-width `h_ruler`s and full-height
  cursors, i.e. the "lines across the plot") are re-placed every frame from the result, so a
  throttled pass is a *stepped* ruler: acceptable, because the ruler and the number beside it
  come from the same pass and agree. What is *not* acceptable is a ruler that appears and
  disappears, so an invalid pass falls back to the last valid result for its **level** marks and
  only drops its **time** marks: a level is a property of the signal (a mean, a top, an
  amplitude) and outlives the window it was measured in, while a time mark is a position *inside*
  that window and may not. Both are emitted in **one pass after the numbers, never inside the
  loop that computes them**: that loop has three `continue`s (an unresolvable signal, a channel
  with no window) and each one was a way for the rulers to blink off and back. Fixing one and not
  the others is how that bug survived a round of "fixes". `clamp fires: N` in the dump is the other half of this: `clamp_to_data()`
  compares with a relative tolerance, because clamping to exactly `window_s_` can leave the
  group's span an ULP above it and a clamp that fires every frame is a re-frame loop, not a
  clamp.
- **A measurement must not read its window raw.** Both passes go through
  `plot::data::MinMaxPyramid`, which the producer already maintains at amortised O(1) -- so the
  pane inherits the summary of the new samples rather than computing one, which is what the
  "evaluate only on new samples" ask comes to here. Throttling the cadence (above) bounded *how
  often*; this bounds *how much*, and it is the larger factor: 19.7 ms per recompute to 1.2--1.6
  ms, reading **0.1 %** of a 2 M-sample window. Pass one (`measure_scan_buckets()`) adds whole
  level-0 buckets -- 64 samples each, carrying min, max an **exact** sum and a finite count -- and
  locates each extreme to a *bucket*; one raw `measure_scan()` of that bucket alone then pins it
  to a sample, two buckets at most whatever the window's size. Pass two
  (`measure_accumulate_bucket()`) folds a bucket's **sums** only when it lies **strictly** on one
  side of its level (`k.lo > level` or `k.hi < level`): then every sample falls in the same bin and
  none is *at* the level, so the fold is exact and not an approximation. It asks **separately**
  whether the bucket enters a side of the hysteresis band (see the invariant below), because a
  single test against `level ± h` refuses exactly the buckets a crossing passes through. A bucket
  wholly past a threshold holds that crossing at its first sample, exactly -- which is where the raw
  walk would have found it. It refuses a bucket that straddles the level, one that enters a side
  somewhere inside itself, and one holding a non-finite sample (the raw walk takes the first
  *finite* sample of the new side and an aggregate cannot say which that is); the caller then opens
  that bucket. **Nothing may be folded before a refusal** -- a refused bucket goes to
  `measure_accumulate()`, which adds the same samples again, and sums added above the `return false`
  double-counted every opened bucket. That read as `top=0.614304` on a unit sine against the 2/pi a
  mean over a half then gave; a *bucketed* window no longer reports `top` from those sums at all
  (see the flat-level invariant below), which makes the rule easier to break silently rather than
  less binding -- the sums are the fallback a window with no histogram reads, so a wrong one is a
  wrong answer waiting for that window. Four things this rests on, each of which breaks it silently:
  * **Exactness is the point, so nothing may stride or decimate.** `peak to peak` must still read
    exactly `2` on a unit sine, and it and `average` still come straight out of the one shared
    `measure_stats()` pass. A bucket's sum is a sum. (`top` has no *exact* reading to protect any
    more: it is a density estimate and not a mean, and the two places that used to assert
    `top == 0.636620` on a sine now assert what the flat level actually reads -- see the flat-level
    invariant below. Everything above is about the *bucket* path being a fast path and not an
    approximation, and that is unchanged.)
  * **The bucket index space must be the window's.** `ChannelBuffer::acquire()` calls
    `pyramid_.drop_before(values_->base())`, so a channel's pyramid is in the value ring's
    absolute index space -- the same space `live_window_locked()` answers in, since a channel's
    stride is always 1. The snapshot's pyramid is built from sample 0 and never dropped from, so
    its space is the snapshot's own. `MeasureWindowReader::origin_` is what turns one into a
    window offset, and a mismatch there measures the wrong samples rather than failing.
  * **The raw path stays, and stays exact**, for the window's partial edge buckets, for the
    samples produced since this frame's `acquire_all()` (one frame of production, so bounded), and
    for the straddling buckets. A window shorter than a bucket degrades to O(window) and must
    still be right.
  * **Both passes walk the same fetched bucket list** (`App::measure_bucket_scratch`, reused
    across recomputes -- a 2 M-sample window is 32 k buckets). Fetching twice is a second walk of
    the pyramid for nothing.
  `measure cost:`'s `covered`/`buckets`/`raw` is the guard, and it needs the fast mock to say
  anything.
- **`Top`/`Bottom`/`Amplitude` read a *flat* level, and the density that finds it is the buckets',
  not the samples'.** A band-limited edge overshoots and rings -- the mock's `square` at 30
  harmonics reaches ±1.179 where its plateau is ±1 -- so a measurement that reads the extremes, or
  a mean over a side of a level, reports the ringing rather than the level the signal dwells at:
  that is what made this change (`amplitude` read `1.179350` and `top` `0.986496` on the rippled
  square, against `1.179350` of overshoot where the plateau is `1`). A flat level is a *density*
  question: the plateau holds most of the samples and the ringing a few percent. So
  `measure_scan_buckets()` folds a 64-bin histogram of the whole buckets' *midpoints* while it has
  the list in hand -- one observation per bucket, each bin carrying its buckets' **exact** sum and
  finite count -- and `measure_flat_levels()` takes, either side of the split, the densest bin that
  holds at least half the densest one's buckets, **the outermost of those**, and the exact mean of
  the samples of the buckets within **5 % of the window's peak-to-peak** of it. 5 % is *under* the
  ~9 %-of-swing Gibbs overshoot, so the ringing's excursion falls outside the band while a
  plateau's own ripple stays inside it; the outermost-of-the-equally-dense rule is what makes a
  waveform with *no* plateau -- a sine, a ramp -- read its peak instead of an arbitrary mid-range
  bin that a shifted window would move. `Amplitude` is then `top - bottom`, a bench scope's
  `Vamplitude`, and **not** the half-swing it used to be, nor the peak-to-peak swing.
  The readings, fast mock, 2 s window, `[measure-trace]`, harmonics `0` → `30`: `top` `1.000000` →
  `0.999983`, `bottom` `-1.000000` → `-0.999983`, `amplitude` `2.000000` → `1.999966`, against
  `peak to peak` `2.000000` → `2.358701` -- the extremes keep the overshoot, and that separation
  *is* the measurement. On the probe's 1 Hz ±1 sine: `top` `0.957988` (it was `0.636620`, the
  half-wave mean), `amplitude` `1.915983`, and `peak to peak` still **exactly `2`**; `[stats-probe]`
  reads `top` `mean=0.957905 sd=0.0014 lo=0.951967 hi=0.962348` over a scrolling window, which is
  the stability the value band buys -- the answer does not follow the window's alignment.
  Five things this rests on, each of which breaks something silently:
  * **Nothing may be read raw for it.** The histogram is the *same* fetched bucket list, walked a
    second time inside `measure_scan_buckets()`; a histogram of the window's *samples* would be a
    second O(window) walk, which is exactly what the pyramid path exists to avoid. `measure cost:`
    is the guard and its `raw` does not move: the probe's pane reads
    `covered=59684972 buckets=932446 raw=47640 (0.1%)` against the baseline's
    `covered=61183221 buckets=955856 raw=47402 (0.1%)` -- the same `raw`, 0.44 → 0.50 ms a pass
    (and `--noise 0.1`, `2.1%` both sides, 0.52 → 0.65 ms).
  * **It is gated, and the gate is what keeps an unrelated pane free.** `measure_scan_buckets()`'s
    `flat_hist` comes from `App::MeasureWindow::flat_wanted`, set by the first loop over the pane
    and only for one of the three types. A pane of means and extremes pays nothing.
  * **The bin range is the *buckets'*, not the window's.** The window's partial edge buckets and the
    samples produced since `acquire_all()` are read raw and are not in the list, so
    `measure_scan_buckets()` tracks its own `blo`/`bhi` over the **non-empty** buckets only -- an
    all-non-finite bucket is an empty aggregate whose `lo`/`hi` say nothing, and taking them would
    put a bucket at some arbitrary value. A degenerate range (`bhi == blo`) leaves `folded` at 0,
    which is the caller's "no histogram at all".
  * **The *window's* peak-to-peak is what the band is a share of**, and the *split* is sized exactly
    as `measure_set_level()` sizes a level, so "the top" and "the level a period is counted about"
    are placed by one rule. The band is a *value* band, which is why the reading does not move when
    the window does.
  * **The fallback is the split, and it is what a filtered or sub-bucket window reads.** `hf_fc > 0`
    disables the pyramid (above) and a window shorter than a bucket has no bucket to fold, so
    `top`/`bottom` fall back to the mean of each side of the split -- and `amplitude` to the
    window's own peak to peak, which is the same number on a waveform with nothing to exclude.
    There is no flat level in that mode to subtract, so `amplitude == top - bottom` is *not* an
    invariant to lean on there. Measured on the rippled `square` at `hf_fc=100`:
    `top=0.973824 bottom=-0.991731 amplitude=2.354173 peak to peak=2.354173`, `buckets=0
    raw=200.0%` -- two raw walks and no density estimate, which is the honest reading of a window
    the pyramid cannot summarize.
- **`MeasureOptions::noise_pct` is a percentage, clamped in `measure_set_level()` to half the room to the
  nearer extreme**, so a band can never invert its own thresholds. It moves the **edge decision only**: the
  sums still split at the bare level, which is the *fallback* `Top`/`Bottom` read on a window with no
  bucket histogram -- a band that moved that split would change what a filtered or sub-bucket window
  reports, for no gain to a crossing count.
- **`measure_accumulate` loops samples outer, levels inner, and that is not backwards.** The filter holds one
  state for the whole walk, so levels-outer would step it once per level *per sample* and hand every level but
  the last a different filter state.
- **A non-zero `hf_fc` disables *both* bucket fast paths**, with the single
  `if (!filtered && reader.bucket_range(...))`. An empty `buckets` makes `bulk_from == bulk_to == n` *and*
  leaves `lo_search` / `hi_search` at 0, so `pin()` becomes a no-op -- and that is the right answer, because a
  bucket is an aggregate of unfiltered samples and cannot be mixed into a filtered walk in either direction.
- **A crossing is counted with hysteresis, and the band moves the *edge decision* only.**
  `MeasureLevel::hysteresis` is a Schmitt trigger about `level`: an edge is counted once the signal
  has passed `level + h` going up or `level - h` going down, and a sample inside the band decides
  nothing. Without it `period`, `frequency` and `pulse duration` are unusable on anything real --
  the mid level is exactly where a waveform moves *fastest*, so an interferer faster than the
  fundamental recrosses a single level several times per true crossing. Measured on a 5 Hz sine with
  a 500 Hz tone at a tenth of the amplitude (`tools/mock_udpstreamer.py --noise 0.1`): 9 true
  crossings counted as **79**, `period=0.025622 freq=39.029` for a true 0.2 s / 5 Hz, against
  `period=0.200188` with the band. Four rules, each of which was wrong first:
  * **`measure_set_level()` is the only place the band is sized**, and `src/ui.cpp` calls it rather
    than assigning `.level` itself: two expressions for the width is how the raw walk and the bucket
    path end up disagreeing about where a crossing was. `h = min(0.10 * (hi - lo), 0.5 * min(hi -
    level, level - lo))`. The 0.10 is measured -- 5 % reads a 0.1 interferer and fails at 0.2
    (`period=0.075`), 10 % holds to 0.3 and to white noise at sigma=0.05.
  * **The clamp is load-bearing.** `PulseDuration` lets the user put the split near an extreme, and
    a fixed tenth of the span there puts the upper threshold *above every sample*: the walk never
    decides that side and the measurement reads nothing at all, which is worse than reading
    something noisy. On a ringing 3.3 V logic signal at a 90 % split: `enters=0` with a fixed band,
    `enters=4 period=0.200000` with the clamp.
  * **The `sum`/`count` split stays at the bare `level`.** Two things read it and neither may be
    moved by a band: a crossing's *side* is decided by the sample's own value against the level
    (`v == level` is the below side, as it always was), and `Top`/`Bottom` fall back to the mean of
    each side on a window with no bucket histogram. The band is about *deciding a crossing*, not
    about where a sample lives, so folding it in would change a filtered window's `top` for no gain
    to any crossing count.
  * **`h = 0` must reduce exactly to the single-level walk**, including the `v == level` skip, so the
    band is a parameter of the engine rather than a mode of it.
  The cost is the noise's, not the band's: the clean fast mock reads `raw=119412 (0.1%)` of 215 M at
  1.41 ms a recompute and with the 0.1 interferer `raw=2.0%` at 1.96 ms. A
  `/tmp/check_measure.cpp`-style check is how the walk itself was verified (`g++ -std=c++20 -O2 -I
  src ... src/measure.cpp`): over a clean sine, sines with 0.1 and 0.3 interferers, 10 %/50 % duty
  squares and a 90 % threshold, the **bucket walk must equal the raw walk exactly** -- same
  `enters`, same `count[]`, identical `first_enter`/`last_enter` to 0.0 -- which is the assertion
  that the fast path is a fast path and not an approximation. The app's reading is
  `[measures-probe]` against `--noise 0.1`.
- **A statistic's observation is one *recompute*, not one sample.** The pane already refreshes on
  a cadence (at most one pass per 100 ms while the window scrolls, exactly once when it
  settles), and the accumulator is fed from that pass's result — so `n` counts windows measured.
  Feeding it per sample would make `n` a sample count and the mean a mean of *samples*, which is
  not what any of these measurements measure: a frequency over a window is not a property of a
  sample. The reading that names this defect is `[stats-probe]`'s `n` in the thousands rather
  than the tens.
- **The accumulator is Welford's, and the reason is the same one the time ring is `f64` for.** A
  measurement's values are routinely large with a small spread — an absolute stream time is the
  obvious case — and `Σx²  - (Σx)²/n` cancels to noise there. The standalone check is that the
  spread of `{2,4,4,4,5,5,7,9}` survives a `+1e9` offset unchanged.
- **What resets the statistics is "what is being measured changed", not "what is in view
  changed".** An explicit Reset, an add/edit/delete (all three funnel through
  `rebuild_measures_pane()`), a CONFIG, and a change of display window each change the
  *question*, so the history is cleared. A pause, a resume, a trigger latch, a pan and a zoom do
  not: they change which samples answer it, which is precisely what averaging over many windows
  is for. The pane-wide toggle does not reset either — it only shows and hides labels that are
  always created and always fed, so reading the plot for a moment does not cost the history.