udpscope/docs/superpowers/specs/2026-10-13-noise-rejection-design.md

Noise rejection: design

Noise rejection in three places — the measurements pane, the trigger, and a new math node — aimed at one problem: electromagnetic peaks. EM interference is impulsive. It arrives as a few samples of large excursion, not as a raised noise floor, and that distinction decides every choice below. An impulse recrosses a threshold several times per true crossing, so it inflates an edge count; it moves a min, a max and a peak-to-peak outright; and it is the one kind of noise a low-pass filter handles badly, smearing the spike across its settling time instead of removing it while rounding the real edge beside it.

Each of the three places therefore gets two independent, optional controls:

	amplitude qualifier	frequency qualifier
Trigger	hysteresis band about the level, in signal units	one-pole HF reject on the comparator's input, fc in Hz
Measurements	hysteresis band, % of the window's peak-to-peak	one-pole HF reject over the measured window, fc in Hz
Math	—	the de-noise node: median over N samples, then a one-pole at fc

Nothing here changes a default. Every control is off, or set to the value the code already uses, so the whole documented reading set must come out of this work unchanged. That is the first verification assertion, not an afterthought.

The decisions taken, and what each one rules out

Decision	Alternative rejected
The trigger band is absolute, in the channel's own units	A percentage of a running peak-to-peak: the producer has no measured span, so it would need per-channel sliding-window state in the hot loop and a warm-up during which the band is wrong — which shows up as a missed or early latch, the one failure a trigger must not have
band == 0 takes today's crossing test verbatim, with the latched-side comparator used only when the band is non-zero	One unified latched-side comparator: at v == level exactly the two are not equivalent (today it fires, a zero-width band decides nothing), and feed_window's own comment records a ±1 square against a [1, 1] band firing 19 times on exact float equality. probe-readings.md's [condition-probe] table of four exact latch times is a regression suite in disguise; unifying would require re-measuring all of it to prove a change nobody asked for
The band and the filter live inside EventDetector	Filtering in Acquisition before engine.process(): the buffer search deliberately runs the same evaluator (condition.hpp:46-49), so anything upstream of the detector has to be replicated there, and the live trigger and the search would drift apart
No shared hysteresis primitive extracted	A common side_of() header used by measure.cpp and condition.hpp: the genuinely shared part is about five lines, and the two call sites want different things from it — measure.cpp continues on an in-band sample while counting enters, widths and folded buckets, the detector needs a side to feed four state machines. Three similar lines beat a premature abstraction; the invariant is stated once and pointed at
The measurement filter is a real one-pole at a user fc, and it disables the pyramid fast path	Measuring the pyramid's per-bucket means, which would be free and keep the fast path — but its corner is wherever the bucket size puts it (fs/64), so it is HF reject at one fixed, rate-dependent frequency rather than a cutoff the user can aim at the interference they actually have
The filter runs inside each of the two raw walks	Materializing a filtered copy of the window: MeasureChunk's comment (measure.hpp:108-115) records that widening the window cost 60 % of the pane's time before a single statistic had been computed, and 17 MB at a 2.2 M-sample window
One combined de-noise node (median then pole)	A plain median node chained to the existing LowPass: two boxes for one idea, and it lets the user wire them in the wrong order — a pole before a median lets one spike contaminate the filter state for its whole settling time, which defeats the purpose
The median window is N samples, fixed	A duration converted to N from dt: N would then grow with decimation, so a modest duration at a long window becomes a large sort in the producer's hot loop and needs a hidden cap the user did not set
MathOp::DeNoise is appended, at index 12	Inserting it beside the filters in math_op_at(), where it belongs in the dropdown: a node persists as its index through that accessor (layout_store.cpp:358-365), so inserting would silently reinterpret every node in every existing session file
The median's ring is pooled in MathRuntime	A double[9] member of MathNodeRt: 72 bytes added to every node, doubling a struct whose shape README's math-cost section records as a deliberate choice, to serve the one op whose state size is a parameter
A new impulsive mock source	Verifying against --noise (Gaussian): a median is specifically the wrong tool for a raised noise floor and the right one for impulses, so a Gaussian test would prove nothing about the feature

1. The trigger

Config

EventCondition (condition.hpp:27) gains two fields:

double noise_band = 0.0;  ///< hysteresis half-width about `level`, signal units. 0 = off.
double hf_fc = 0.0;       ///< HF reject corner on the comparator's input, Hz. 0 = off.

Both must join operator== (condition.hpp:37-41). This is not bookkeeping: EventDetector::configure() returns early when the condition compares equal (condition.cpp:9) and the producer calls it on every packet, so a field omitted from the comparison is a control that silently never takes effect.

The filter needs the sample interval to turn fc into a coefficient, and the detector is fed (value, t) pairs rather than a rate. configure() therefore takes dt as a second argument and keeps it as a dt_ member, and the early return compares the new dt against dt_ as well as the condition against the stored one — a re-plan changes dt, and a coefficient left over from the previous partition is a filter at the wrong corner.

Comparator

EventDetector gains int side_ = 0 (0 undecided, ±1 above/below) and one pole state word. crossed_up/crossed_down (condition.hpp:77-82) stop being pure functions of prev_ and become flags computed once per sample in feed():

• noise_band == 0 → prev_ < level && v >= level and its mirror, bit for bit as today.
• noise_band > 0 → v >= level + band latches +1, v <= level - band latches -1, anything between decides nothing; a crossing is a change of latch.

Computing the pair once also removes a duplicated computation: feed_edge, feed_pulse and feed_dropout each call both predicates today.

feed_window (condition.cpp:115) does not use those predicates, so it takes the band explicitly on each edge of its own band: becoming inside requires [lo+band, hi-band], becoming outside requires leaving [lo-band, hi+band]. A band wide enough to close that inner region is clamped to just under half the window's width — the same shape of clamp measure_set_level() already applies for the same reason, and it keeps the panel's existing refusal of a zero-width window meaningful.

reset() (condition.cpp:14) clears side_ and the pole state.

The filter

One pole on the comparator's input, y += a·(x − y), with a = 1 − exp(−2π·fc·dt) built in configure() and clamped below Nyquist as biquad() does (math_channel.cpp:28). fc = 0 gives a = 1, i.e. y = x, so "off" needs no branch in the inner loop.

Two consequences, both stated rather than discovered:

• The latch moves later by roughly 1/(2π·fc) — the pole's group delay. This is inherent, it is what a bench scope's HF reject also does, and it is why the control is off by default.
• A NaN stays benign. feed()'s comment (condition.hpp:68-70) records that a NaN sample cannot fire because every comparison against it is false, and that rejecting one would leave prev_ stale and let the next sample fire a crossing that today cannot happen. The filter preserves this: a non-finite input returns NaN (which flows into the existing comparisons, all false) and leaves the pole state alone.

last_value() (condition.hpp:73) comes to report the comparator's input rather than the raw sample. Its only consumer is the v=%.4g field of the trig-engine diagnostic (acquisition.cpp:918), whose purpose is to say what the comparator saw.

Panel

A trigger_band_spin and a trigger_hf_spin in build_trigger_panel() (ui_trigger.cpp:219), read in publish_trigger_fields() (:56), shown for all four condition types in sync_trigger_rows() (:94). The band takes the level spin's unit, digits and range, floored at 0.

The search gets both for free

EventDetector holds no arming state precisely so the buffer search can run the same evaluator over retained samples (condition.hpp:46-49). A band or a filter set in the panel therefore changes what a search finds, consistently, with no second code path and no second thing to keep in step.

Not persisted

The trigger has never been persisted, so neither control is either; a restart comes up with both off. That is also what makes every current [condition-probe] latch time hold unchanged.

2. The measurements

Config

MeasureOptions (measure.hpp:37) gains two fields:

double noise_pct = 10.0;  ///< hysteresis half-width, % of the window's peak-to-peak. 0 = off.
double hf_fc = 0.0;       ///< HF reject corner, Hz. 0 = off.

noise_pct = 10.0 reproduces today's kHysteresisFrac = 0.10 (measure.cpp:36) exactly, so every current reading is identical. measure_set_level() (measure.hpp:195, measure.cpp:40) already takes a pct — the threshold percent — so it gains a second, separately named noise_frac argument for the band, replacing its read of the file-scope kHysteresisFrac; the constant stays as MeasureOptions::noise_pct's default so its measured-value rationale — 5 % reads a 0.1 interferer correctly and fails at 0.2, 10 % holds to 0.3 and to white noise at σ=0.05 — does not become an orphan comment. The clamp to half the room between the level and the nearer extreme survives untouched: it is what keeps a threshold at 90 % measurable instead of reading enters=0.

The band reaches the edge types only

MeasureLevel::hysteresis's comment (measure.hpp:165-169) is explicit that the band moves the edge decision only, and that the sum/count split stays at the bare level — because that split is Top/Bottom, the mean of the samples above or below the threshold, and banding it would change the documented exact top = 2/π on a unit sine.

So noise_pct is meaningful for PulseDuration, Period and Frequency, and does nothing for Top/Bottom. The file's own rule (measure.hpp:34-36) is that an option which does nothing for a type is left out of the dialog rather than stored and ignored, so this adds a measure_uses_noise() beside measure_uses_threshold() and measure_uses_side() (measure.hpp:242-245), and measure_dialog_sync() (ui_measure.cpp:90) shows the field for those three types only.

A raised band also costs reads: measure_accumulate_bucket() already refuses a bucket where the signal passes level ± hysteresis (measure.hpp:211-216), so a wider band opens more buckets. The existing measure cost: line is where that shows up.

HF reject reaches every type

The filter changes the samples rather than a decision, so min, max, average, peak to peak, amplitude, top, bottom and every crossing type all read the filtered signal. The control is therefore shown for all types.

It is seeded with the window's first sample, so the step transient at the left edge does not read as a measurement. (Priming from samples before the window is possible — the ring holds the history — and is deliberately not done: it would make a measurement depend on data outside the window it reports on.)

What it costs

hf_fc > 0 disables both bucket fast paths, measure_scan_buckets() (measure.hpp:147) and measure_accumulate_bucket() (:217): a bucket's lo/hi/sum describe raw samples and cannot answer for a filtered signal. Both passes walk raw, which is the 1.2–1.6 ms → ~20 ms step at a 2.2 M-sample window that README's measurement-cost section was written about. Against the pane's 250 ms cadence that is a few percent of a core and a visible frame hitch.

This is accepted deliberately: the control is per-measurement and off by default, so nothing gets slower unasked, and measure cost:'s covered/buckets/raw makes the cost a reading rather than a mystery. raw ≈ covered is the signature of a filtered measurement.

The filter runs inside each of the two raw walks — three flops on a sample the walk is already reading — rather than materializing a filtered window once. Filtering twice is deterministic and free compared with the alternative measured in MeasureChunk's comment.

NaN keeps the existing rule: the walks already skip a non-finite sample, and the filter state is left alone, so a hole stays a hole and the filter resumes on real samples.

Two documented deviations

1. docs/claude/invariants-readouts-and-cursors.md holds that a readout quotes a stored sample and never a value between two. A filtered max is a value that was never stored, and with show_on_plot its ruler sits at a level the drawn — raw — trace never reaches. This is inherent to filtering, so it becomes a stated exception in that invariant rather than a silent violation.
2. The documented exact baselines (peak to peak = 2, top = 0.636620) hold with the filter off. With it on they move, correctly; the probe reads both.

Persistence

Free. A measurement is written as keyed TOML (layout_store.cpp:659-672) and read key by key (:544-556), so two new keys load in an old session at their defaults.

3. The de-noise math node

The op, appended

MathOp::DeNoise goes last in the enum (math_channel.hpp:15-28) and at index 12 in math_op_at() (math_channel.cpp:99-114), with Div given an explicit case 11 so the current default no longer swallows it; math_op_count() becomes 13 and math_op_name() gains "de-noise".

Indices 0–11 keeping their meaning is exactly what leaves every existing session file reading correctly, because a node persists as its index through that accessor. The price is that de-noise sits last in the dialog's dropdown rather than beside the filters, which is the right side of that trade.

Parameters

MathNode (math_channel.hpp:48-72) gains int med_n = 5; — odd, 3 to 9, rounded to odd and clamped. fc is reused for the HF reject; q is unused. fc = 0 means median only, and it expresses "off" without a branch: the pole's coefficient a = 1 − exp(−2π·fc·dt) is exactly 1 at fc = 0.

The coefficient is built in math_prepare() (math_channel.cpp:140-165) where dt is known and rebuilt on a dt change, exactly as the biquads are, with the same sub-Nyquist clamp (:28).

Predicates

math_op_uses_filter() (math_channel.cpp:135) keeps meaning biquad — it gates coefficient construction at :162, and a de-noise node must not be given biquad coefficients. The dialog gets a new math_op_uses_fc() (filter ‖ de-noise) and a math_op_uses_median() for the new spin row. math_dialog_sync_form() (ui_math.cpp:889-981) already shows and hides one widget set per op, and q's row being visible for BandPass alone is the precedent for a per-op subset of the filter row.

Node text

math_node_text() (layout_store.cpp:358) emits a 13th pipe field. math_node_from_text() accepts 12 or 13 fields, defaulting med_n when it reads 12, so existing sessions load unchanged. A file written by this build and read by an older one is rejected by that build's f.size() != 12u check (:385); there is no forward-compatibility requirement here, but it is recorded rather than left as a surprise.

Runtime state, pooled

MathRuntime (math_channel.hpp:115-120) gains one std::vector<double> holding the median rings of whichever nodes need them; MathNodeRt gains an offset into it, a ring position and the effective N. Not a fixed array member, for the reason in the decisions table. One indirection on a path already doing N compares is free, and the choice is reversible if measurement disagrees.

Evaluation

In math_step() (math_channel.cpp:167-230), the new case is: push the input into the ring, take the median, then one pole step. Median first — de-spiking before smoothing is the whole point, and the reverse lets one spike contaminate the filter state for its entire settling time.

The median is an insertion sort of at most nine doubles in a local array: ~20 compares worst case, ~4 at N=3, against a documented 80–100 ns per slot for a whole three-graph runtime. That ratio is measured and reported through the existing math-cost line, not assumed.

Warm-up: before the ring holds med_n samples the node outputs the median of what it holds, which is a correct median of a shorter window. Deliberately not NaN — in this application a NaN output means a hole in the data (math_channel.cpp:64-70), and spending that meaning on a start-up transient would make a gap ambiguous.

NaN matches biquad_step exactly: a non-finite input returns NaN and leaves both the median ring and the pole state alone, so the hole stays visible and the node resumes on real samples.

4. Verification

There is no unit test suite; the check is the application's own headless self-check under xvfb-run against tools/mock_udpstreamer.py, read against two baselines (an exact warning count, and no Gtk-CRITICAL).

The regression assertion, first

Every default is today's behaviour, so the documented reading set must be unchanged:

• [condition-probe]'s four exact latch times (t0=1.710000, 2.160100, 1.706700, 4.120200);
• peak to peak=2, amplitude=1, average=0, top=0.636620;
• measure cost: with raw ≪ covered;
• the build's warning count — adding a field to MathNode leaves ui_probe.cpp:304 and :313 at one -Wmissing-field-initializers each, so four on a clean build and two on an incremental one still holds.

The mock needs an impulsive source

tools/mock_udpstreamer.py offers --noise (Gaussian), --glitch-at/--glitch-width (a single glitch at one time) and --clock-jitter. None of them is a stream of EM-like peaks, and a median is the wrong tool for Gaussian noise, so none of the three pieces could be shown to work against the mock as it stands.

Add --spike-every S --spike-amp A --spike-width N: a recurring impulse of N samples at amplitude A, every S seconds. One flag set, and it is what every reading below is measured on.

The four readings

1. Trigger band — a new [condition-probe] leg on a slow sine plus spikes. With noise_band = 0 the latch lands on a spike; with the band above the spike amplitude it lands on the sine's own crossing time. The latched t0 is the assertion, and the four existing shapes are re-read unchanged in the same run.
2. Trigger HF reject — the same leg with the band off and hf_fc set. The latch moves later by roughly 1/(2π·fc), reported beside the prediction. A latch that does not move is the filter not being in the comparator's path.
3. Measurements — Frequency on the spiky mock read three ways: noise_pct = 0 (counts spikes, frequency far too high), = 10 (the true frequency), and hf_fc set (the true frequency and raw ≈ covered, which is the fast-path loss as a reading). Plus peak to peak with the filter off and on, since that is the number the filter is supposed to move.
4. The de-noise node — math_probe (ui_probe.cpp:304) gains a de-noise graph on the spiky channel. Its output's peak-to-peak must fall to the clean waveform's while a LowPass at the same fc does not: that contrast is the entire justification for the node existing. Plus the per-slot cost through the existing math-cost line.

5. Documentation

File	What it gains
README.md	A noise-rejection section covering the three places, and the filtered-readout deviation stated rather than quietly taken
docs/claude/invariants-readouts-and-cursors.md	The stated exception: a filtered readout is not a stored sample, and its on-plot ruler can sit where the raw trace never goes
docs/claude/invariants-measurements.md	A filter on a measurement disables both bucket fast paths, and raw ≈ covered is its signature
docs/claude/invariants-trigger-and-pinning.md	The band, the band == 0 verbatim path and why, and the latch delay a filter adds
docs/claude/invariants-math-channels.md	Op indices are append-only because the session format stores them; the pooled median state
docs/claude/verification.md, docs/claude/probe-readings.md	The four readings above and what a wrong value names
docs/claude/build-and-run.md	The new mock flags

6. Out of scope

• Persisting the trigger. The trigger has never been persisted; the band and the filter follow that, rather than this change being the one that introduces trigger persistence.
• Any averaging acquisition mode. Noise rejection here is per-measurement, per-trigger and per-node; nothing changes what the ingest stores, so a raw trace stays raw and every reading remains traceable to stored samples.
• Noise rejection on the cursors or the search's own UI. The search inherits the trigger's condition and so inherits both controls; nothing further is added.