feat: the export writes an MDF 3.30 file, so a window loads in ScryLab

A third format beside `.npy` and `.csv`, written by hand in `src/export.cpp` --
MF4 is HDF5 and would mean a library, MF3 is a handful of fixed blocks. The
layout is the asammdf 8.8.27 reader's own field list, byte-diffed against a
known-good 3.30 file it wrote; the shape is the one ScryLab's data-source
contract asks for: one channel group, record-interleaved, the master as the x
axis, and the unit in the channel's conversion block.

So in MDF the master is `abs_seconds`, named `time` -- the clock that is the
same in every display mode -- and the plotted axis goes out as an ordinary
channel, `time_relative`. `ExportTable::Column` gained the CONFIG's unit, which
only this writer spells out; the `.npy`/`.csv` bytes are unchanged, and
`write_export()` now picks the writer for both the dialog and the probe.
This commit is contained in:
Martino Ferrari committed 2026-10-14 15:56:27 +02:00
1 parent 761615c2fe
commit 908286ee89
10 files changed
+505 -28

No files matched your search

@@ -52,6 +52,40 @@ Part of [CLAUDE.md](../../CLAUDE.md). These are rules that cost a defect to lear
- **The export is collected at click time, not in the file-dialog callback.** The live
view keeps scrolling while the chooser is up, so a late collection would write a
window the user never saw. The writer is picked by the chosen path's extension.
- **The MDF export is MDF 3.30 written by hand, and its layout is the reader's, not the
standard's prose.** No dependency is added: MF4 is HDF5 and would need a library, while MF3 is a
handful of fixed-size blocks, so `write_mdf()` spells them out. The field list comes from
`asammdf` 8.8.27 (`blocks/v2_v3_blocks.py`, `blocks/mdf_v3.py`, which parses MF3 by explicit
offsets) and was confirmed byte for byte against a known-good 3.30 file asammdf wrote (one
group, a `time` master and two `V` channels). The shape, in file order: the **ID block** is 64
bytes with *no id/length header* (`"MDF 3.30"`, byte order, version `330`) — a writer that
emits `ID` + size + links at offset 0 produces something no reader accepts, since the first real
block starts at 64; the **HD block** is 208 (164 common + 44 of the 3.2-era time fields) and
points at the data group; a **TX** comment follows it; then the **samples**; the **DG** is 28; a
**CC** (conversion, 46 — a `NONE` conversion is the common fields alone) then a **CN** (channel,
228) precede each channel; and the **CG** (30) is last. Address fields are absolute file offsets
written *forward*, so the whole address map is computed before a byte is written, and every
block length is checked after it is built: a field at the wrong width does not fail the write, it
leaves the reader in the middle of the next block.
- **The data group's data address holds the raw records, with no `DT` header** — which is what
asammdf's *writer* does and what its *reader* assumes (`DataBlock` is documented as "a pseudo
block not defined by the MDF 3 standard"). This is the one place the file follows the reader
rather than a narrow reading of the standard, and it is the assumption ScryLab compatibility
rests on: a file with a `DT` block there would have its header read as sample bytes.
- **In MDF the master is `abs_seconds`, named `time`, and the relative axis is a channel
`time_relative`** — the same two numbers the `.npy`/`.csv` table holds, in the other roles. A
master off the relative column is 0 at a different sample in each display mode, so it would put a
live export and a held event's export on two timelines, which is the one thing the second column
exists to avoid; both carry the unit `s`, and the signal columns follow in the `.npy`/`.csv`
order. **The unit** is the client's CONFIG unit (a math channel's own), and it is written into
the CC block because that is the only place MDF 3 keeps one — and the place ScryLab reads a
signal's unit from. `ExportTable::Column` gained the field for it; the two older writers ignore
it, and their bytes are unchanged. **The names** are sanitised and de-duplicated against each
other *and* against the two reserved names before being cut to 31 bytes (the CN name field is 32
with its NUL): ScryLab imports only the **first** occurrence of a channel name, so a cut made
before de-duplication could silently drop a signal. `write_export()` is the one place the
extension picks the writer (`.csv`, `.mf3`/`.mdf`, anything else `.npy`), and both the dialog and
the self-check's export probe come through it so the two cannot offer different formats.
- **The chooser runs before `ScopeModel` exists, and `--server` is what decides.** `Config` carries
`server_specified`, set only by the `--server` branch, and `main()` asks only when it is false —
so declining costs no buffer and no socket, and `ScopeModel`, `Acquisition`, `RowSet` and
+18 -1
View File
@@ -126,10 +126,27 @@ Two lines of the self-check output are assertions in disguise:
An export taken **mid-capture** reports `mode=live` with `time=[-pre..e]` where `e` is how far
the post window has filled — the export is what is on screen, and during a capture that is a
partly filled window. Pin the probe outside a capture when checking the `rows=` count.
The two `bytes=` lines are the writers: the `.npy` is
The three `bytes=` lines are the writers: the `.npy` is
`64·⌈(10+len(header)+1)/64⌉ + rows·(16 + 4·cols)`, so a size that is not that is a
mispacked record. Read it back with `pd.DataFrame(np.load(path))` after changing
either writer.
The `.mf3` line's size is the MDF 3.30 layout the writer spells out, and every term of it is
observable: `272 + comment + rows·record + 28 + channels·274 + 30`, where the ID block is 64,
the HD block 208, the comment is a `TX` block of its text plus 5 bytes, `record` is
`16 + 4·cols` (two `f64` time columns then one `f32` per signal), the data group is 28, each of
`channels = cols + 2` gets a 46-byte conversion block and a 228-byte channel block, and the
channel group is 30. Read the file back with asammdf: `MDF(path).info()` must show one group
whose `channel 0` is `name="time" type=master`, and `get(name, group=0).unit` must be the unit
the next line reports.
`[export-probe] mdf master=time master_unit=s channels=N record=B units=['s' 's' 'V' …]`
is the other half, and it is the assertion on the units: `channels` and `record` are the two
numbers the `.mf3` size above is built from (so `record` must be `16 + 4·cols` and `channels`
`cols + 2`), and `units` must be **the CONFIG's**, in the table's column order, behind the
master's and the relative axis's two `s` — one `'s' 's'` pair plus one entry per exported
column, each exactly the unit the stream sent (an empty one reads `''`). A `V` where the mock
sent `s`, or a missing entry, is the unit not reaching the conversion block, which is the only
thing ScryLab reads a signal's unit from; `write_npy`/`write_csv` must be unchanged either way
(their bytes are pinned by the two lines above).
- `[window-probe] <action> label=.. entry=.. unit=U window=W event_span=E cap=C decim=M
stored=R aa=F buffered=B` (stderr, `UDPSCOPE_PROBE_WINDOW_MS`) — **`window` must equal
`event_span`** on every line (the trigger's event span is the display window; a capture
+3 -3
View File
@@ -15,7 +15,7 @@ xvfb-run -a env UDPSCOPE_EXIT_AFTER_MS=8000 UDPSCOPE_ENABLE_ALL_MS=1000 \
Two baselines every run is read against. A clean build emits **four** warnings and no more: two
`-Wmissing-field-initializers` on `MathNode::source` and `-Woverlength-strings` on **both**
generated GResource `.c` files — the help pages' (**514660** bytes) and the icons', whose blob is
generated GResource `.c` files — the help pages' (**515700** bytes) and the icons', whose blob is
**12352** against C99's 4095-byte string-literal limit with the seven action icons and the two app icons
that remain (a note about a generated file, which GCC itself supports) -- re-measured when
the tab work changed the set (that icon left, the two tab glyphs arrived, and this file's
@@ -194,8 +194,8 @@ precedent, not an exception.
nothing. Its four lines are in
[probe-readings.md](probe-readings.md).
- `UDPSCOPE_EXPORT_MS` (+ optional `UDPSCOPE_EXPORT_PATH`, default
`/tmp/udpscope-export`) — collect the export table and write it both ways, skipping
only the file chooser.
`/tmp/udpscope-export`) — collect the export table and write it all three ways (`.npy`,
`.csv`, `.mf3`), skipping only the file chooser.
- `UDPSCOPE_PROBE_CONDITION_MS` — arms a qualified trigger against the mock and reports the latched
`t0`, the measured pulse or band width, and sample values for the window condition. Schedules its