udpscope/docs/claude/invariants-session-and-export.md

Invariants: the session file, the export and the source chooser

What a session persists and in what order, when a save is refused, and when an export is collected.

Part of CLAUDE.md. These are rules that cost a defect to learn; read the ones touching the area you are editing before you edit it.

• The trigger is persisted, settings and state both. Its condition, its channel (by name), its pre-trigger share, hold-off and mode, and whether trigger mode was on — written even while it is off, because a trigger configured once and switched off is still the trigger you come back to. The two enumerations are stored as the words the panel's dropdowns show, not as their indexes: an index ties the file to a declaration order, which is the lesson math_op_at() carries, and the words are shared with the panel so the two cannot drift. The channel resolves when the acquired channels exist (rebuild_trigger_signals() picks up App::pending_trigger_channel, or apply_trigger_layout() resolves it itself when they are already there) — a name that resolved to nothing would silently move the trigger to channel 0.
• A session that loaded is what locks the trigger pane. Locked is the returning user's mode: the trigger pane shows one compact line and offers no control that changes the configuration (the signals have no lock). A first run — no session file for that source — never reaches apply_layout() and stays in configuration mode, which is what a first run is for. The mode is a mode and not a setting: it is deliberately not persisted, so a scope reopened tomorrow starts where a returning user expects to start. Dragging a signal onto a plot is not configuration and survives the lock: it is the signal badge's drag (the list's rows are no longer drag sources, and the badges do not follow the lock). [sigbadges] drag_sources is the reading, and drag_sources on the signals panel: line must be 0.
• The session is the layout. LayoutState persists what makes a reopened scope look like the one that was closed: the window, the rows and their slot signals, the calibration, the unit renames ([[unit]], name + unit, only the overrides; ScopeModel::set_signal_unit re-applies them to every CONFIG, and set_signals() takes them off the incoming units before the "same stream" comparison so a reconnect still keeps its history), the enabled signals, the chosen trace colours ([[color]], name + rgb = "#RRGGBB" — only the overrides, so a stream's new signals still take the positional palette and a saved session does not freeze a colour that is meant to follow the signal list), the math channels with their graphs, the measurements pane, and measure_stats — the measurements statistics toggle; written as a bare top-level boolean (measure_stats = true/false), defaults to false when absent, and dock -- LayoutState::dock_shown, the Dock button's state (the user's last word on it, App::sidebar_wanted): a bare top-level boolean written where side_tab used to be, default true. It is the only thing the dock persists -- its 280 px width is fixed, so there is nothing to store -- and the open slide-over page is deliberately not persisted (a panel that opens by itself at start-up is a surprise, and a page id from another source kind may not exist). The parser ignores unknown keys (its key chain has no erroring final else), so a file written before the tabs became a dock (side_tab = "measure", or side_tab = "source-ao") still loads and opens with the dock shown; nothing reads the old key, and [dock-session] asserts it. Signal identity is the name throughout (a CONFIG renumbers channels), 'enabled' is read off the channels the model built -- an enabled signal is exactly one that produced a channel -- and apply_layout() installs the graphs before assigning the rows, since a row may hold a math channel's name and that name is only a signal once its graph is in. Two format rules, both learned the hard way: write an op or a measure type as its index through math_op_at()/measure_type_at(), never as the raw enum value -- those accessors' orders are the dialog's, not the declaration order, so writing one and reading the other silently renames operations. And a top-level key written after a [[table]] belongs to that table in TOML: enabled = [...] emitted after [[calibration]] was parsed into the last calibration record and the enable state was lost without a word. mkdirs() used to create only components with a trailing slash, so ~/.config/udpscope was never made and the first save failed silently -- worth knowing when a layout change "does not stick".
• Measurement options persist under named keys -- threshold, threshold2, noise and hf -- so a file written before them loads at the defaults, which are exactly the behaviour it was written against: noise = 10 is the hysteresis band the app has always counted with, hf = 0 is no filter, and a file with no threshold2 -- every file written before between existed -- leaves that measurement at threshold2 = 75, which is the value the dialog offers a new one. The pair is stored as written rather than as ordered: measure_level_pcts() is what puts a reversed pair in order on read, so a file records what the user typed and a measurement file round-trips unchanged.
• The math node string is thirteen pipe fields, and the reader accepts twelve or thirteen. med_n is the thirteenth, after y; a twelve-field node is a file written before the de-noise node existed, and its med_n stays at the struct's default -- which is what such a file meant. Field 0 is the op as a math_op_at() index, which is why MathOp may only be appended to.
• The export is collected at click time, not in the file-dialog callback. The live view keeps scrolling while the chooser is up, so a late collection would write a window the user never saw. The writer is picked by the chosen path's extension.
• The MDF export is MDF 3.30 written by hand, and its layout is the reader's, not the standard's prose. No dependency is added: MF4 is HDF5 and would need a library, while MF3 is a handful of fixed-size blocks, so write_mdf() spells them out. The field list comes from asammdf 8.8.27 (blocks/v2_v3_blocks.py, blocks/mdf_v3.py, which parses MF3 by explicit offsets) and was confirmed byte for byte against a known-good 3.30 file asammdf wrote (one group, a time master and two V channels). The shape, in file order: the ID block is 64 bytes with no id/length header ("MDF 3.30", byte order, version 330) — a writer that emits ID + size + links at offset 0 produces something no reader accepts, since the first real block starts at 64; the HD block is 208 (164 common + 44 of the 3.2-era time fields) and points at the data group; a TX comment follows it; then the samples; the DG is 28; a CC (conversion, 46 — a NONE conversion is the common fields alone) then a CN (channel, 228) precede each channel; and the CG (30) is last. Address fields are absolute file offsets written forward, so the whole address map is computed before a byte is written, and every block length is checked after it is built: a field at the wrong width does not fail the write, it leaves the reader in the middle of the next block.
• The data group's data address holds the raw records, with no DT header — which is what asammdf's writer does and what its reader assumes (DataBlock is documented as "a pseudo block not defined by the MDF 3 standard"). This is the one place the file follows the reader rather than a narrow reading of the standard, and it is the assumption ScryLab compatibility rests on: a file with a DT block there would have its header read as sample bytes.
• In MDF the master is abs_seconds, named time, and the relative axis is a channel time_relative — the same two numbers the .npy/.csv table holds, in the other roles. A master off the relative column is 0 at a different sample in each display mode, so it would put a live export and a held event's export on two timelines, which is the one thing the second column exists to avoid; both carry the unit s, and the signal columns follow in the .npy/.csv order. The unit is the client's CONFIG unit (a math channel's own), and it is written into the CC block because that is the only place MDF 3 keeps one — and the place ScryLab reads a signal's unit from. ExportTable::Column gained the field for it; the two older writers ignore it, and their bytes are unchanged. The names are sanitised and de-duplicated against each other and against the two reserved names before being cut to 31 bytes (the CN name field is 32 with its NUL): ScryLab imports only the first occurrence of a channel name, so a cut made before de-duplication could silently drop a signal. write_export() is the one place the extension picks the writer (.csv, .mf3/.mdf, anything else .npy), and both the dialog and the self-check's export probe come through it so the two cannot offer different formats.
• The chooser runs before ScopeModel exists, and --server is what decides. Config carries server_specified, set only by the --server branch, and main() asks only when it is false — so declining costs no buffer and no socket, and ScopeModel, Acquisition, RowSet and run_ui() need no notion of "no source yet". It is --server alone and not "no arguments": --multicast still needs a server to fetch CONFIG from. Every documented probe passes --server, which is what makes the feature invisible to the regression net — a probe added without it would reach the chooser and hang. gtk_init() plus a nested GMainLoop, and then run_ui()'s GtkApplication afterwards in the same process, is a verified combination (gtk_init() is idempotent; running your own loop is the documented GTK4 replacement for the removed gtk_dialog_run()).
• A session file's name cannot say how its source was reached, so the file says it. <host>_<port>.toml carries neither the multicast group nor the interface, so a chooser row could not reconnect to a multicast source at all. LayoutState::multicast/interface_addr are written in the top block, before the first [[table]] — a bare key after a table header is that table's key in TOML, which is exactly how enabled was parsed into the last [[calibration]] record and lost without a word. A file from an older build has neither key and loads as unicast on the default route. The chooser's list filters to sessions that name at least one signal, because a probe run leaves a file per throwaway port and all of them hold signals = ["", "", "", ""]; known_sources() splits the filename at the last _, since sanitize() turns anything unsafe in a host into one.
• save_layout_now() refuses to write while signals() is empty. A run against a source whose streamer was down receives no CONFIG, so every slot name stays empty and the save would overwrite a configured session with signals = ["", ...]. The chooser then filters that file out — a source whose streamer was merely down during one run would silently vanish from the only list it can be picked from, for ever. Keeping the old file loses nothing: a run that configured nothing had nothing worth persisting.
• It refuses again while replaying is set: a file view is not the source's session. Same argument, different cause — this view configured nothing about the stream. A plain replay leaves the rows where the user put them, so its save would merely be pointless; a group rebinds every slot to its own renamed channels and the save wrote signals = ["square (A)", …] into the file. Nothing noticed until the way back: Go live re-applies the session, those names match no live signal, so nothing was enabled and the live view came back with no channels at all — on the window the poisoned file carried, which is also how it swallowed the window restore below it. The reading is [events] live restored (after a group): the symptom was a stuck pending=1 beside channels=0, because the restore waits for a channel list that was never going to arrive. The explicit Save as is deliberately not guarded: that one is a user's own answer to "save this view", and its dialog reports the outcome.

A configuration is a named session, and a source may hold several

One source, several arrangements: the same streamer serves different experiments, and retyping a layout to look at the second one is how a session gets lost. Storage is therefore a directory per source — ~/.config/udpscope/<host>_<port>/ — holding one <name>.toml per configuration plus an active file naming the one the source is on. configuration_path() is the only expression of the scheme, and the name is sanitize()d exactly as the host is, so a name typed with a slash cannot escape the directory.

A configuration carries the whole session and the settings that can only be decided before the first socket: buffer_samples, relative_time, capture_packets. The startup chooser applies them (apply_source()), which works because choose_source() runs before ScopeModel is constructed — the one moment a buffer size can be applied at all. Switching configurations inside a running app applies what can change live and says in the status line that the rest takes effect on the next start; pretending otherwise would be a configuration that half-applies.

A configuration's startup settings reach the model from two directions, and both are needed: apply_source() for a source picked in the chooser, and apply_stored_settings() for a source named with --server, which never reaches the chooser at all. Without the second, a saved host's buffer size, clock and capture ring were silently ignored — the layout still applied, because the app loads that separately once a CONFIG names the signals — so an edit to any of the three read as one that never took effect. A flag the user typed wins over the file (--buffer-samples, --relative-time, --capture-packets): the file fills in what the command line did not say, and Config::buffer_samples_specified is what tells "not asked" from "asked for the default".

The active file is written on every save and by the chooser's pick, so "which configuration was I on" survives a restart. A session written before configurations existed is imported once as the configuration named default, and the old file is renamed to <host>_<port>.toml.imported rather than deleted: an import that cannot be undone by hand is a migration nobody can back out of.

Nothing is lost by switching: the dialog saves the current session before loading the picked one, which is the only ordering in which a switch cannot lose what it switches from.