udpscope/docs/claude/invariants-session-and-export.md

# Invariants: the session file, the export and the source chooser

What a session persists and in what order, when a save is refused, and when an export is collected.

Part of [CLAUDE.md](../../CLAUDE.md). These are rules that cost a defect to learn; read the ones touching the area you are editing before you edit it.

- **The trigger is persisted, settings and state both.** Its condition, its channel (by *name*), its
  pre-trigger share, hold-off and mode, and whether trigger mode was on — written even while it is off,
  because a trigger configured once and switched off is still the trigger you come back to. The two
  enumerations are stored as the *words the panel's dropdowns show*, not as their indexes: an index ties
  the file to a declaration order, which is the lesson `math_op_at()` carries, and the words are shared
  with the panel so the two cannot drift. The channel resolves when the acquired channels exist
  (`rebuild_trigger_signals()` picks up `App::pending_trigger_channel`, or `apply_trigger_layout()`
  resolves it itself when they are already there) — a name that resolved to nothing would silently move
  the trigger to channel 0.
- **A session that loaded is what locks the trigger pane.** Locked is the returning user's mode: the
  trigger pane shows one compact line and offers no control that changes the configuration (the signals
  have no lock). A *first run* — no session file for that source — never
  reaches `apply_layout()` and stays in configuration mode, which is what a first run is for. The mode
  is a mode and not a setting: it is deliberately not persisted, so a scope reopened tomorrow starts
  where a returning user expects to start. **Dragging a signal onto a plot is not configuration and survives
  the lock**: it is the signal *badge's* drag (the list's rows are no longer drag sources, and the badges do
  not follow the lock). `[sigbadges] drag_sources` is the reading, and `drag_sources` on the `signals panel:`
  line must be 0.
- **The session is the layout.** `LayoutState` persists what makes a reopened scope look like the
  one that was closed: the window, the rows and their slot signals, the calibration, **the unit renames**
  (`[[unit]]`, `name` + `unit`, only the overrides; `ScopeModel::set_signal_unit` re-applies them to every CONFIG,
  and `set_signals()` takes them off the incoming units before the "same stream" comparison so a reconnect still
  keeps its history), **the enabled
  signals**, **the chosen trace colours** (`[[color]]`, `name` + `rgb = "#RRGGBB"` — only the
  *overrides*, so a stream's new signals still take the positional palette and a saved session does
  not freeze a colour that is meant to follow the signal list), **the math channels with their
  graphs**, **the measurements pane**, and **`measure_stats`** — the measurements statistics
  toggle; written as a bare top-level boolean (`measure_stats = true/false`), defaults to `false`
  when absent, and **`dock`** -- `LayoutState::dock_shown`, the Dock button's state (the user's last word on it,
  `App::sidebar_wanted`): a bare top-level boolean written where `side_tab` used to be, default `true`.
  It is the **only** thing the dock persists -- its 280 px width is fixed, so there is nothing to store --
  and **the open slide-over page is deliberately not persisted** (a panel that opens by itself at start-up
  is a surprise, and a page id from another source kind may not exist). The parser ignores unknown keys
  (its key chain has no erroring final `else`), so a file written before the tabs became a dock
  (`side_tab = "measure"`, or `side_tab = "source-ao"`) still loads and opens with the dock shown;
  nothing reads the old key, and `[dock-session]` asserts it. Signal identity
  is the *name* throughout (a CONFIG renumbers channels), 'enabled' is read off the channels the
  model built -- an enabled signal is exactly one that produced a channel -- and `apply_layout()`
  installs the graphs **before** assigning the rows, since a row may hold a math channel's name and
  that name is only a signal once its graph is in. Two format rules, both learned the hard way:
  write an op or a measure *type* as its index through `math_op_at()`/`measure_type_at()`, never as
  the raw enum value -- those accessors' orders are the dialog's, not the declaration order, so
  writing one and reading the other silently renames operations. And a top-level key written after
  a `[[table]]` belongs to *that table* in TOML: `enabled = [...]` emitted after `[[calibration]]`
  was parsed into the last calibration record and the enable state was lost without a word.
  `mkdirs()` used to create only components with a trailing slash, so `~/.config/udpscope` was never
  made and the *first* save failed silently -- worth knowing when a layout change "does not stick".
- **A `[[plot]]` table is one *plot* (one cell), and carries `row` and `width` for the horizontal split.**
  `plots = N` counts plots, as it always did. `row` is the 0-based **figure row** the plot sits in and
  `width` the cell's share of that row's width (`LayoutState::Row::row`, `::width`); the tables run in
  the flattened order, so a row's cells are consecutive and left to right. The row's height share
  `weight` stays per *figure row* but is **written on every cell of it** (so a table read alone is
  complete) and **read from the row's first cell**; `collect_layout()` takes it from
  `row_weights()[figure_row_of(plot)]`, not from `row_weights()[plot]` -- the latter is a plot index into
  a figure-row vector, right until the first split. The file is written `row`, `width`, `weight`,
  `signals`; a state built with `row < 0` (nothing in the app does) omits `row` and `width`.
  Three loading rules, all in `apply_layout()` and none of them a refusal, because a session is the
  user's arrangement and not worth losing to a malformed key:
  (1) **a file with no `row` loads as one cell per row** -- every session written before the split --
  with the per-plot `weight`s as the row heights, exactly as it opened before;
  (2) **a `row` sequence that cannot be a layout loads the same way**: some plot without a `row`, a row
  number that goes backwards or skips one, a first row that is not 0, more than 4 cells in a row, or
  more than 8 plots; the signals stay on the plots they were on (`invalid-rows` is the reading);
  (3) **cell widths are applied after the arrangement**, because `set_cells_per_row()` resets them, and
  only for rows of two or more cells -- a one-cell row has no widths and its `width` is ignored.
  Both new keys are parsed with the file's other forgiving rules: `row` must be an integer >= 0 and
  `width` a number > 0, anything else leaves the default. A save after loading a pre-split file writes
  the `row`/`width` keys, which is how an old session is upgraded without anyone asking.
- **Measurement options persist under *named* keys -- `threshold`, `threshold2`, `noise` and `hf` -- so a
  file written before them loads at the defaults**, which are exactly the behaviour it was written against:
  `noise = 10` is the
  hysteresis band the app has always counted with, `hf = 0` is no filter, and a file with no
  `threshold2` -- every file written before `between` existed -- leaves that measurement at
  `threshold2 = 75`, which is the value the dialog offers a new one. The *pair* is stored as written
  rather than as ordered: `measure_level_pcts()` is what puts a reversed pair in order on read, so a
  file records what the user typed and a measurement file round-trips unchanged.
- **The math node string is thirteen pipe fields, and the reader accepts twelve *or* thirteen.** `med_n` is
  the thirteenth, after `y`; a twelve-field node is a file written before the de-noise node existed, and its
  `med_n` stays at the struct's default -- which is what such a file meant. Field 0 is the op as a
  `math_op_at()` index, which is why `MathOp` may only be *appended* to.
- **The export is collected at click time, not in the file-dialog callback.** The live
  view keeps scrolling while the chooser is up, so a late collection would write a
  window the user never saw. The writer is picked by the chosen path's extension.
- **The MDF export is MDF 3.30 written by hand, and its layout is the reader's, not the
  standard's prose.** No dependency is added: MF4 is HDF5 and would need a library, while MF3 is a
  handful of fixed-size blocks, so `write_mdf()` spells them out. The field list comes from
  `asammdf` 8.8.27 (`blocks/v2_v3_blocks.py`, `blocks/mdf_v3.py`, which parses MF3 by explicit
  offsets) and was confirmed byte for byte against a known-good 3.30 file asammdf wrote (one
  group, a `time` master and two `V` channels). The shape, in file order: the **ID block** is 64
  bytes with *no id/length header* (`"MDF     3.30"`, byte order, version `330`) — a writer that
  emits `ID` + size + links at offset 0 produces something no reader accepts, since the first real
  block starts at 64; the **HD block** is 208 (164 common + 44 of the 3.2-era time fields) and
  points at the data group; a **TX** comment follows it; then the **samples**; the **DG** is 28; a
  **CC** (conversion, 46 — a `NONE` conversion is the common fields alone) then a **CN** (channel,
  228) precede each channel; and the **CG** (30) is last. Address fields are absolute file offsets
  written *forward*, so the whole address map is computed before a byte is written, and every
  block length is checked after it is built: a field at the wrong width does not fail the write, it
  leaves the reader in the middle of the next block.
- **The data group's data address holds the raw records, with no `DT` header** — which is what
  asammdf's *writer* does and what its *reader* assumes (`DataBlock` is documented as "a pseudo
  block not defined by the MDF 3 standard"). This is the one place the file follows the reader
  rather than a narrow reading of the standard, and it is the assumption ScryLab compatibility
  rests on: a file with a `DT` block there would have its header read as sample bytes.
- **In MDF the master is `abs_seconds`, named `time`, and the relative axis is a channel
  `time_relative`** — the same two numbers the `.npy`/`.csv` table holds, in the other roles. A
  master off the relative column is 0 at a different sample in each display mode, so it would put a
  live export and a held event's export on two timelines, which is the one thing the second column
  exists to avoid; both carry the unit `s`, and the signal columns follow in the `.npy`/`.csv`
  order. **The unit** is the client's CONFIG unit (a math channel's own), and it is written into
  the CC block because that is the only place MDF 3 keeps one — and the place ScryLab reads a
  signal's unit from. `ExportTable::Column` gained the field for it; the two older writers ignore
  it, and their bytes are unchanged. **The names** are sanitised and de-duplicated against each
  other *and* against the two reserved names before being cut to 31 bytes (the CN name field is 32
  with its NUL): ScryLab imports only the **first** occurrence of a channel name, so a cut made
  before de-duplication could silently drop a signal. `write_export()` is the one place the
  extension picks the writer (`.csv`, `.mf3`/`.mdf`, anything else `.npy`), and both the dialog and
  the self-check's export probe come through it so the two cannot offer different formats.
- **The chooser runs before `ScopeModel` exists, and `--server` is what decides.** `Config` carries
  `server_specified`, set only by the `--server` branch, and `main()` asks only when it is false —
  so declining costs no buffer and no socket, and `ScopeModel`, `Acquisition`, `RowSet` and
  `run_ui()` need no notion of "no source yet". It is `--server` alone and not "no arguments":
  `--multicast` still needs a server to fetch CONFIG from. **Every documented probe passes
  `--server`**, which is what makes the feature invisible to the regression net — a probe added
  without it would reach the chooser and hang. `gtk_init()` plus a nested `GMainLoop`, and then
  `run_ui()`'s `GtkApplication` afterwards in the same process, is a verified combination
  (`gtk_init()` is idempotent; running your own loop is the documented GTK4 replacement for the
  removed `gtk_dialog_run()`).
- **A session file's name cannot say how its source was reached, so the file says it.**
  `<host>_<port>.toml` carries neither the multicast group nor the interface, so a chooser row
  could not reconnect to a multicast source at all. `LayoutState::multicast`/`interface_addr` are
  written in the **top block, before the first `[[table]]`** — a bare key after a table header is
  that table's key in TOML, which is exactly how `enabled` was parsed into the last
  `[[calibration]]` record and lost without a word. A file from an older build has neither key and
  loads as unicast on the default route. The chooser's list filters to sessions that **name at
  least one signal**, because a probe run leaves a file per throwaway port and all of them hold
  `signals = ["", "", "", ""]`; `known_sources()` splits the filename at the **last** `_`, since
  `sanitize()` turns anything unsafe in a host into one.
- **`save_layout_now()` refuses to write while `signals()` is empty.** A run against a source
  whose streamer was down receives no CONFIG, so every slot name stays empty and the save would
  overwrite a configured session with `signals = ["", ...]`. The chooser then filters that file
  out — a source whose streamer was merely down during one run would silently vanish from the
  only list it can be picked from, for ever. Keeping the old file loses nothing: a run that
  configured nothing had nothing worth persisting.
- **It refuses again while `replaying` is set: a file view is not the source's session.** Same
  argument, different cause — this view configured nothing about the stream. A plain replay leaves
  the rows where the user put them, so its save would merely be pointless; a **group** rebinds every
  slot to its own renamed channels and the save wrote `signals = ["square (A)", …]` into the file.
  Nothing noticed until the way back: *Go live* re-applies the session, those names match no live
  signal, so nothing was enabled and **the live view came back with no channels at all** — on the
  window the poisoned file carried, which is also how it swallowed the window restore below it. The
  reading is `[events] live restored (after a group)`: the symptom was a stuck `pending=1` beside
  `channels=0`, because the restore waits for a channel list that was never going to arrive. The
  explicit *Save as* is deliberately **not** guarded: that one is a user's own answer to "save this
  view", and its dialog reports the outcome.

## A configuration is a named session, and a source may hold several

One source, several arrangements: the same streamer serves different experiments, and retyping a
layout to look at the second one is how a session gets lost. Storage is therefore a **directory
per source** — `~/.config/udpscope/<host>_<port>/` — holding one `<name>.toml` per configuration
plus an `active` file naming the one the source is on. `configuration_path()` is the only
expression of the scheme, and the name is `sanitize()`d exactly as the host is, so a name typed
with a slash cannot escape the directory.

A configuration carries the whole session *and* the settings that can only be decided before the
first socket: `buffer_samples`, `relative_time`, `capture_packets`. The startup chooser applies them
(`apply_source()`), which works because `choose_source()` runs before `ScopeModel` is constructed
— the one moment a buffer size can be applied at all. Switching configurations inside a running
app applies what can change live and says in the status line that the rest takes effect on the
next start; pretending otherwise would be a configuration that half-applies.

A configuration's *startup* settings reach the model from two directions, and both are needed:
`apply_source()` for a source picked in the chooser, and `apply_stored_settings()` for a source
named with `--server`, which never reaches the chooser at all. Without the second, a saved host's
buffer size, clock and capture ring were silently ignored — the layout still applied, because the
app loads that separately once a CONFIG names the signals — so an edit to any of the three read as
one that never took effect. **A flag the user typed wins over the file** (`--buffer-samples`,
`--relative-time`, `--capture-packets`): the file fills in what the command line did not say, and
`Config::buffer_samples_specified` is what tells "not asked" from "asked for the default".

The `active` file is written on every save and by the chooser's pick, so "which configuration was
I on" survives a restart. A session written before configurations existed is **imported once** as
the configuration named `default`, and the old file is renamed to `<host>_<port>.toml.imported`
rather than deleted: an import that cannot be undone by hand is a migration nobody can back out
of.

Nothing is lost by switching: the dialog saves the current session *before* loading the picked
one, which is the only ordering in which a switch cannot lose what it switches from.

- **The event cache is keyed on the source *and its active configuration*, and the key moves
  with the switch.** An event's trigger, its channel set and its units are the configuration's,
  so the directory is `<base>/<host>-<port>/<configuration>/` (`EventStore`'s third constructor
  argument) and `retarget_event_store()` -- called next to every `App::config_name` write and
  resolving the name from `active_configuration()` when it is empty -- re-derives it, refreshing
  both views of the list only when the directory changed. A store built without a configuration
  keeps the legacy `<base>/<source>/` path, and the **first construction under a configuration
  adopts the loose event files it finds there** (`*.npy`/`*.txt`/`*.bmp`, moved into the
  configuration's folder): an upgrade keeps the events it had. The key is a *sanitized* path
  component, the same rule the configuration files use, so a name typed with a slash cannot
  leave the directory.

- **The session remembers extra windows; a launch never opens them.** Each scope or detached
  window is a `[[window]]` block (`name`, `detached`, and its own `[[window.plot]]` rows -- the
  same keys the main `[[plot]]` rows carry), written from the open windows' present arrangement
  plus the ones already closed (`collect_layout()`). Loading puts **all** of them in
  `App::closed_windows`, never into `app.scopes`: the overflow menu's *Reopen closed windows (N)*
  is the only path that opens them (`open_scope_window_from_state()`, the one constructor every
  window uses, then its rows). Closing a window records it (by name, replaced, so close/reopen
  cycles do not multiply entries); **attaching** records nothing -- its figure is on the main
  window again -- and a detached figure survives a quit through its window's entry instead of
  being lost from the main layout.