fix: a loaded recording is filtered into its buffer, not truncated

The loader pushed one slot per file row while the plan decimated by
eight, so a 10 M-row file overran the ring and only its last 4.19 s
survived -- a 20 s recording that read as a 4 s one. Each group of
`time_stride` rows is now averaged into one slot at the group's mean
stamp, as the packet path already does, and the load line reports
rows and decimation so a filtered file cannot be mistaken for a
truncated one.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Martino FerrariandClaude Opus 4.6 committed 2026-10-14 00:54:31 +02:00
1 parent 50a1ddca62
commit 345145c52b
7 files changed
+104 -11

No files matched your search

+11
View File
@@ -63,6 +63,17 @@ Part of [CLAUDE.md](../../CLAUDE.md). These are rules that cost a defect to lear
length. The LOD min/max pyramid still preserves the extremes *of what is stored*; a spike
narrower than a group is now filtered away rather than aliased, which is the correct
trade and is why the status line's `rate` field is worth reading.
- **A file is ingested too, so the loader decimates by the same factor.** `load_waveform_into_model()`
sets the target span to the recording's own length and enables every column, and the plan that
comes back may well decimate: 20 s of 10 channels at 500 kS/s is 480 MB of rings, and the budget is
`--buffer-mb`. The loader therefore reads `partition().time_stride` **after** the enables and pushes
one slot per that many rows, carrying the group's mean at the group's mean stamp — the packet path's
filter, written a second time because a file does not arrive in packets. Pushing every row instead
overruns the ring and leaves the file as **its own tail**, with nothing to say so: a 20 s event
opened as its last 4 s, `stored=10000000` against `published=2097152` in the channel dump, and a
trace that looks like a short recording rather than a truncated one. The `waveform loaded:` line
carries `rows=` and `decim=` beside `samples=` for exactly this reason — `samples * decim == rows`
is the arithmetic that says the whole file is in the buffer.
- **One bad packet is bridged; only a run of them changes the time source.** The stream-time
checks in `build_stamps()` reject three things: stamps that are not finite, a packet with no
span at all, and a packet whose stamps do not advance past the previous packet's last. A
+5 -2
View File
@@ -979,7 +979,8 @@ configuration that carries settings, is the defect this exists for — a saved h
[sources] chooser ui: page='sources' title='Sources' icon=yes page='waveforms' title='Waveforms' icon=yes visible='sources' switcher_wired=1 buttons=[prefs=1 new_host=1 open=1 cancel=1]
[sources] waveforms: 1 recent /tmp/rt.npy(ok) page=waveforms
[sources] picked waveform /tmp/rt.npy
[udpscope] waveform loaded: /tmp/rt.npy signals=4 samples=20001 span=2.000000s rate=10000 time=[-2.000000..0.000000]
[udpscope] waveform loaded: /tmp/rt.npy signals=4 samples=20001 rows=20001 decim=1 span=2.000000s rate=10000 time=[-2.000000..0.000000]
[udpscope] waveform loaded: /tmp/long.npy signals=4 samples=31250 rows=1000000 decim=32 span=19.999980s rate=50000 time=[-4.000000..15.999980]
```
| field | value it must read | the defect a wrong reading names |
@@ -987,7 +988,9 @@ configuration that carries settings, is the defect this exists for — a saved h
| `chooser ui:` | both pages with a `title` and an `icon=yes`, `switcher_wired=1`, and all four buttons `1` | the chooser's `.ui` losing an id: the code names the missing one and refuses the window, and this line is the frame that *did* build -- a page with `icon=NO` is one the switcher cannot draw |
| `waveforms:` | one entry per remembered export, with `(ok)` or `(missing)`, and `page=waveforms` once the page is shown | the Waveforms page not built, or a stale path offered as if it were openable |
| `picked waveform` | the selected path | a pick that carries no file: `--open` is what the mode reads |
| `signals=`, `samples=` | the export's `cols=` and `rows=` | a reader mis-slicing its records, or the columns bound to the wrong channels |
| `signals=`, `rows=` | the export's `cols=` and `rows=` | a reader mis-slicing its records, or the columns bound to the wrong channels |
| `samples=` × `decim=` | `rows`, to within the last partial group | the loader not filtering the file into the plan it built: every row pushed into a ring that cannot hold them leaves the file as its own tail (`stored=` far above `published=` in the channel dump, and a 20 s recording on screen as its last 4 s) |
| `decim=` | 1 whenever the recording fits the buffer at its own rate; a power of two above it | a decimation that moved without the span moving — the second line above is the reading to compare against: `samples` fell 32-fold and `span=` did **not** |
| `span=`, `rate=`, `time=[…]` | the exported window and rate, `[-window, 0]` | the plan not covering the whole file (the tail arrives as the whole picture) or the display epoch not pinned to the file's last stamp |
| `frames=0 packets=0` beside `channels=N` | 0 — no acquisition | a socket opened for a file, which would also make the starvation notice live |
+16 -1
View File
@@ -354,12 +354,27 @@ as distinct from the fault injection below, which exercises ingest:
- **A waveform round-trips.** Export from a live run (`UDPSCOPE_EXPORT_MS` with
`UDPSCOPE_EXPORT_PATH`, against the mock at 10 kS/s), then open that file with `--open` and an
exit dump. The load must print
`[udpscope] waveform loaded: … signals=4 samples=20001 span=2.000000s rate=10000 time=[-2.000000..0.000000]`
`[udpscope] waveform loaded: … signals=4 samples=20001 rows=20001 decim=1 span=2.000000s rate=10000 time=[-2.000000..0.000000]`
— **the same row count, column count, span and time range the export probe reported**
(`rows=20001 cols=4 time=[-2.000000..0.000000]`) — and the dump must show `channels=4`,
`frames=0 packets=0` (no acquisition) and, per channel,
`20001 samples vs 20001 time slots … y=[-1.0000, 1.0000]`. A file that loads with fewer samples
than it was written with, or with `NaN` slots, is the reader mis-slicing its records.
- **A recording that does not fit its buffer is *filtered* into it, not truncated.** The case needs a
file bigger than the budget, which the mock cannot export in one window, so write one with NumPy —
the dtype is the exporter's own, `[('time','<f8'), ('abs_seconds','<f8'), (<signal>,'<f4'), …]` — and
open it against a small buffer:
```bash
python3 -c "import numpy as np; n=1000000; t=-4.0+np.arange(n)/50000.0; \
a=np.zeros(n, dtype=[('time','<f8'),('abs_seconds','<f8'),('a','<f4'),('b','<f4'),('c','<f4'),('d','<f4')]); \
a['time']=t; a['abs_seconds']=1000.0+t; a['d']=np.linspace(-1,1,n); np.save('/tmp/long.npy', a)"
xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=4000 ./build/udpscope --open /tmp/long.npy --buffer-mb 1
```
must read `samples=31250 rows=1000000 decim=32 span=19.999980s time=[-4.000000..15.999980]` and
`stored=31250 … published=31250` per channel — **the whole span, at a thirty-second of the rate**.
The ramp column is the assertion that the *head* is there: `y=[-0.999969, 0.999969]` is the file end
to end, where the defect this exists for reads a fraction of it. With every row pushed regardless of
the plan, this file arrived as its **last 0.9 s** and nothing in the log said so.
- **A loaded file can be plotted.** `UDPSCOPE_PROBE_DROP_MS` drives the drop handler the way a drag
from the signal panel does, on a file opened with `--open`: it must read
`[drop-probe] signals=4 channels=4 rows=4 slots0=[sine|||]` — **`rows` equal to the column count**,