Martino Ferrari
22e7a8efcc
`top`, `bottom` and `amplitude` became the *flat levels* in e83ad86 -- the level the
signal dwells at, read from the density of the window's buckets -- and that change kept
the split's own two means as its internal fallback for a window the pyramid cannot
summarize. Those means are now three types of their own, so the two different questions
about one split can be asked side by side: `above %` and `below %` are the plain mean of
the samples on one side of a share -- the reading the old `top`/`bottom` gave -- and
`between % %` is the mean of the band between two shares, which no single split can
answer, because a band's two edges are two thresholds.
Nothing about the arithmetic is new, and nothing is a new pass. `Above`/`Below` are the
`sum[]`/`count[]` split `measure_accumulate()` already folds -- the very figures
`Top`/`Bottom` fall back to -- and `Between` is the **difference of two levels' upper
sides**, both of which come out of one pass two over one window. It rides the bucket
fast path unchanged: a bucket wholly inside the band is strictly above the lower level
and strictly below the upper one, so both accumulators fold it whole, and only the
buckets that straddle either edge are opened. `measure_level_pcts()` is the one place the
shares are read, and it clamps both and puts `between`'s pair **in order**, so "between
80 % and 20 %" is the band between 20 % and 80 % -- normalized rather than refused,
because the shares name two edges rather than a direction. Nothing to measure reads `--`,
as for every other type: a share at 100 % has no sample above it, and a band can hold
none at all.
Measured, `[bandavg-probe]`, fast mock at a 2 s window (`--rate 1300000 --batch 1000
--freq 1`):
sine `above 50` 0.636620 (2/pi), `below 50` -0.636620, `between 50 75` 0.255873
square `above 50` 0.986496 (30 harmonics) / 1.000000 (--square-harmonics 0),
`between 50 75` 0.302744 / --
ramp `between 50 75` 0.250000 and `above 50` 0.500000, `above 100` --
The first three are the figures the old `top`/`bottom` documented; the ramp's are what a
uniform density predicts (a band's mean is its own centre); the ideal square's empty band
is the nothing-to-measure rule. The pane offers the types and edits their percentages
through the dialog's second share -- which only `between` shows -- and a measurement's
pair survives add/edit/delete and the session file (`threshold2`), where a file written
before it existed loads at the default.
No raw pass: with the measurements probe adding one of every type at a 2 s window, `raw`
went from 113966 of 240719031 covered (0.0%) to 134978 of 255372033 (0.1%), 0.8 -> 1.1
ns a covered sample.
Docs: the band-average invariant (and the two assertions the flat-level change moved),
the probe readings for the new probe, the verification recipe, the manual's measurements
page and the README.
165 lines
14 KiB
Markdown
165 lines
14 KiB
Markdown
# Invariants: the session file, the export and the source chooser
|
|
|
|
What a session persists and in what order, when a save is refused, and when an export is collected.
|
|
|
|
Part of [CLAUDE.md](../../CLAUDE.md). These are rules that cost a defect to learn; read the ones touching the area you are editing before you edit it.
|
|
|
|
- **The trigger is persisted, settings and state both.** Its condition, its channel (by *name*), its
|
|
pre-trigger share, hold-off and mode, and whether trigger mode was on — written even while it is off,
|
|
because a trigger configured once and switched off is still the trigger you come back to. The two
|
|
enumerations are stored as the *words the panel's dropdowns show*, not as their indexes: an index ties
|
|
the file to a declaration order, which is the lesson `math_op_at()` carries, and the words are shared
|
|
with the panel so the two cannot drift. The channel resolves when the acquired channels exist
|
|
(`rebuild_trigger_signals()` picks up `App::pending_trigger_channel`, or `apply_trigger_layout()`
|
|
resolves it itself when they are already there) — a name that resolved to nothing would silently move
|
|
the trigger to channel 0.
|
|
- **A session that loaded is what locks the two panes.** Locked is the returning user's mode: the
|
|
trigger pane shows one compact line, the signal list only the signals that are on, and neither offers
|
|
a control that changes the configuration. A *first run* — no session file for that source — never
|
|
reaches `apply_layout()` and stays in configuration mode, which is what a first run is for. The mode
|
|
is a mode and not a setting: it is deliberately not persisted, so a scope reopened tomorrow starts
|
|
where a returning user expects to start. **Dragging a row onto a plot is not configuration and survives
|
|
the lock**: filling plots is what the list is *for*, so a locked row keeps its `GtkDragSource` — from
|
|
anywhere on it, the colour chip included, which in configuration mode has a click gesture that claims
|
|
the sequence precisely so a drag cannot start there. `drag_sources` on the `signals panel:` line is the
|
|
reading, and it must be `rows - 1`.
|
|
- **The session is the layout.** `LayoutState` persists what makes a reopened scope look like the
|
|
one that was closed: the window, the rows and their slot signals, the calibration, **the enabled
|
|
signals**, **the chosen trace colours** (`[[color]]`, `name` + `rgb = "#RRGGBB"` — only the
|
|
*overrides*, so a stream's new signals still take the positional palette and a saved session does
|
|
not freeze a colour that is meant to follow the signal list), **the math channels with their
|
|
graphs**, **the measurements pane**, and **`measure_stats`** — the measurements statistics
|
|
toggle; written as a bare top-level boolean (`measure_stats = true/false`), defaults to `false`
|
|
when absent. Signal identity
|
|
is the *name* throughout (a CONFIG renumbers channels), 'enabled' is read off the channels the
|
|
model built -- an enabled signal is exactly one that produced a channel -- and `apply_layout()`
|
|
installs the graphs **before** assigning the rows, since a row may hold a math channel's name and
|
|
that name is only a signal once its graph is in. Two format rules, both learned the hard way:
|
|
write an op or a measure *type* as its index through `math_op_at()`/`measure_type_at()`, never as
|
|
the raw enum value -- those accessors' orders are the dialog's, not the declaration order, so
|
|
writing one and reading the other silently renames operations. And a top-level key written after
|
|
a `[[table]]` belongs to *that table* in TOML: `enabled = [...]` emitted after `[[calibration]]`
|
|
was parsed into the last calibration record and the enable state was lost without a word.
|
|
`mkdirs()` used to create only components with a trailing slash, so `~/.config/udpscope` was never
|
|
made and the *first* save failed silently -- worth knowing when a layout change "does not stick".
|
|
- **Measurement options persist under *named* keys -- `threshold`, `threshold2`, `noise` and `hf` -- so a
|
|
file written before them loads at the defaults**, which are exactly the behaviour it was written against:
|
|
`noise = 10` is the
|
|
hysteresis band the app has always counted with, `hf = 0` is no filter, and a file with no
|
|
`threshold2` -- every file written before `between` existed -- leaves that measurement at
|
|
`threshold2 = 75`, which is the value the dialog offers a new one. The *pair* is stored as written
|
|
rather than as ordered: `measure_level_pcts()` is what puts a reversed pair in order on read, so a
|
|
file records what the user typed and a measurement file round-trips unchanged.
|
|
- **The math node string is thirteen pipe fields, and the reader accepts twelve *or* thirteen.** `med_n` is
|
|
the thirteenth, after `y`; a twelve-field node is a file written before the de-noise node existed, and its
|
|
`med_n` stays at the struct's default -- which is what such a file meant. Field 0 is the op as a
|
|
`math_op_at()` index, which is why `MathOp` may only be *appended* to.
|
|
- **The export is collected at click time, not in the file-dialog callback.** The live
|
|
view keeps scrolling while the chooser is up, so a late collection would write a
|
|
window the user never saw. The writer is picked by the chosen path's extension.
|
|
- **The MDF export is MDF 3.30 written by hand, and its layout is the reader's, not the
|
|
standard's prose.** No dependency is added: MF4 is HDF5 and would need a library, while MF3 is a
|
|
handful of fixed-size blocks, so `write_mdf()` spells them out. The field list comes from
|
|
`asammdf` 8.8.27 (`blocks/v2_v3_blocks.py`, `blocks/mdf_v3.py`, which parses MF3 by explicit
|
|
offsets) and was confirmed byte for byte against a known-good 3.30 file asammdf wrote (one
|
|
group, a `time` master and two `V` channels). The shape, in file order: the **ID block** is 64
|
|
bytes with *no id/length header* (`"MDF 3.30"`, byte order, version `330`) — a writer that
|
|
emits `ID` + size + links at offset 0 produces something no reader accepts, since the first real
|
|
block starts at 64; the **HD block** is 208 (164 common + 44 of the 3.2-era time fields) and
|
|
points at the data group; a **TX** comment follows it; then the **samples**; the **DG** is 28; a
|
|
**CC** (conversion, 46 — a `NONE` conversion is the common fields alone) then a **CN** (channel,
|
|
228) precede each channel; and the **CG** (30) is last. Address fields are absolute file offsets
|
|
written *forward*, so the whole address map is computed before a byte is written, and every
|
|
block length is checked after it is built: a field at the wrong width does not fail the write, it
|
|
leaves the reader in the middle of the next block.
|
|
- **The data group's data address holds the raw records, with no `DT` header** — which is what
|
|
asammdf's *writer* does and what its *reader* assumes (`DataBlock` is documented as "a pseudo
|
|
block not defined by the MDF 3 standard"). This is the one place the file follows the reader
|
|
rather than a narrow reading of the standard, and it is the assumption ScryLab compatibility
|
|
rests on: a file with a `DT` block there would have its header read as sample bytes.
|
|
- **In MDF the master is `abs_seconds`, named `time`, and the relative axis is a channel
|
|
`time_relative`** — the same two numbers the `.npy`/`.csv` table holds, in the other roles. A
|
|
master off the relative column is 0 at a different sample in each display mode, so it would put a
|
|
live export and a held event's export on two timelines, which is the one thing the second column
|
|
exists to avoid; both carry the unit `s`, and the signal columns follow in the `.npy`/`.csv`
|
|
order. **The unit** is the client's CONFIG unit (a math channel's own), and it is written into
|
|
the CC block because that is the only place MDF 3 keeps one — and the place ScryLab reads a
|
|
signal's unit from. `ExportTable::Column` gained the field for it; the two older writers ignore
|
|
it, and their bytes are unchanged. **The names** are sanitised and de-duplicated against each
|
|
other *and* against the two reserved names before being cut to 31 bytes (the CN name field is 32
|
|
with its NUL): ScryLab imports only the **first** occurrence of a channel name, so a cut made
|
|
before de-duplication could silently drop a signal. `write_export()` is the one place the
|
|
extension picks the writer (`.csv`, `.mf3`/`.mdf`, anything else `.npy`), and both the dialog and
|
|
the self-check's export probe come through it so the two cannot offer different formats.
|
|
- **The chooser runs before `ScopeModel` exists, and `--server` is what decides.** `Config` carries
|
|
`server_specified`, set only by the `--server` branch, and `main()` asks only when it is false —
|
|
so declining costs no buffer and no socket, and `ScopeModel`, `Acquisition`, `RowSet` and
|
|
`run_ui()` need no notion of "no source yet". It is `--server` alone and not "no arguments":
|
|
`--multicast` still needs a server to fetch CONFIG from. **Every documented probe passes
|
|
`--server`**, which is what makes the feature invisible to the regression net — a probe added
|
|
without it would reach the chooser and hang. `gtk_init()` plus a nested `GMainLoop`, and then
|
|
`run_ui()`'s `GtkApplication` afterwards in the same process, is a verified combination
|
|
(`gtk_init()` is idempotent; running your own loop is the documented GTK4 replacement for the
|
|
removed `gtk_dialog_run()`).
|
|
- **A session file's name cannot say how its source was reached, so the file says it.**
|
|
`<host>_<port>.toml` carries neither the multicast group nor the interface, so a chooser row
|
|
could not reconnect to a multicast source at all. `LayoutState::multicast`/`interface_addr` are
|
|
written in the **top block, before the first `[[table]]`** — a bare key after a table header is
|
|
that table's key in TOML, which is exactly how `enabled` was parsed into the last
|
|
`[[calibration]]` record and lost without a word. A file from an older build has neither key and
|
|
loads as unicast on the default route. The chooser's list filters to sessions that **name at
|
|
least one signal**, because a probe run leaves a file per throwaway port and all of them hold
|
|
`signals = ["", "", "", ""]`; `known_sources()` splits the filename at the **last** `_`, since
|
|
`sanitize()` turns anything unsafe in a host into one.
|
|
- **`save_layout_now()` refuses to write while `signals()` is empty.** A run against a source
|
|
whose streamer was down receives no CONFIG, so every slot name stays empty and the save would
|
|
overwrite a configured session with `signals = ["", ...]`. The chooser then filters that file
|
|
out — a source whose streamer was merely down during one run would silently vanish from the
|
|
only list it can be picked from, for ever. Keeping the old file loses nothing: a run that
|
|
configured nothing had nothing worth persisting.
|
|
- **It refuses again while `replaying` is set: a file view is not the source's session.** Same
|
|
argument, different cause — this view configured nothing about the stream. A plain replay leaves
|
|
the rows where the user put them, so its save would merely be pointless; a **group** rebinds every
|
|
slot to its own renamed channels and the save wrote `signals = ["square (A)", …]` into the file.
|
|
Nothing noticed until the way back: *Go live* re-applies the session, those names match no live
|
|
signal, so nothing was enabled and **the live view came back with no channels at all** — on the
|
|
window the poisoned file carried, which is also how it swallowed the window restore below it. The
|
|
reading is `[events] live restored (after a group)`: the symptom was a stuck `pending=1` beside
|
|
`channels=0`, because the restore waits for a channel list that was never going to arrive. The
|
|
explicit *Save as* is deliberately **not** guarded: that one is a user's own answer to "save this
|
|
view", and its dialog reports the outcome.
|
|
|
|
## A configuration is a named session, and a source may hold several
|
|
|
|
One source, several arrangements: the same streamer serves different experiments, and retyping a
|
|
layout to look at the second one is how a session gets lost. Storage is therefore a **directory
|
|
per source** — `~/.config/udpscope/<host>_<port>/` — holding one `<name>.toml` per configuration
|
|
plus an `active` file naming the one the source is on. `configuration_path()` is the only
|
|
expression of the scheme, and the name is `sanitize()`d exactly as the host is, so a name typed
|
|
with a slash cannot escape the directory.
|
|
|
|
A configuration carries the whole session *and* the settings that can only be decided before the
|
|
first socket: `buffer_mb`, `relative_time`, `capture_packets`. The startup chooser applies them
|
|
(`apply_source()`), which works because `choose_source()` runs before `ScopeModel` is constructed
|
|
— the one moment a buffer size can be applied at all. Switching configurations inside a running
|
|
app applies what can change live and says in the status line that the rest takes effect on the
|
|
next start; pretending otherwise would be a configuration that half-applies.
|
|
|
|
A configuration's *startup* settings reach the model from two directions, and both are needed:
|
|
`apply_source()` for a source picked in the chooser, and `apply_stored_settings()` for a source
|
|
named with `--server`, which never reaches the chooser at all. Without the second, a saved host's
|
|
buffer size, clock and capture ring were silently ignored — the layout still applied, because the
|
|
app loads that separately once a CONFIG names the signals — so an edit to any of the three read as
|
|
one that never took effect. **A flag the user typed wins over the file** (`--buffer-mb`,
|
|
`--relative-time`, `--capture-packets`): the file fills in what the command line did not say, and
|
|
`Config::buffer_mb_specified` is what tells "not asked" from "asked for the default".
|
|
|
|
The `active` file is written on every save and by the chooser's pick, so "which configuration was
|
|
I on" survives a restart. A session written before configurations existed is **imported once** as
|
|
the configuration named `default`, and the old file is renamed to `<host>_<port>.toml.imported`
|
|
rather than deleted: an import that cannot be undone by hand is a migration nobody can back out
|
|
of.
|
|
|
|
Nothing is lost by switching: the dialog saves the current session *before* loading the picked
|
|
one, which is the only ordering in which a switch cannot lose what it switches from.
|