udpscope/docs/claude/invariants-session-and-export.md

Invariants: the session file, the export and the source chooser

What a session persists and in what order, when a save is refused, and when an export is collected.

Part of CLAUDE.md. These are rules that cost a defect to learn; read the ones touching the area you are editing before you edit it.

• The trigger is persisted, settings and state both. Its condition, its channel (by name), its pre-trigger share, hold-off and mode, and whether trigger mode was on — written even while it is off, because a trigger configured once and switched off is still the trigger you come back to. The two enumerations are stored as the words the panel's dropdowns show, not as their indexes: an index ties the file to a declaration order, which is the lesson math_op_at() carries, and the words are shared with the panel so the two cannot drift. The channel resolves when the acquired channels exist (rebuild_trigger_signals() picks up App::pending_trigger_channel, or apply_trigger_layout() resolves it itself when they are already there) — a name that resolved to nothing would silently move the trigger to channel 0.
• A session that loaded is what locks the trigger pane. Locked is the returning user's mode: the trigger pane shows one compact line and offers no control that changes the configuration (the signals have no lock). A first run — no session file for that source — never reaches apply_layout() and stays in configuration mode, which is what a first run is for. The mode is a mode and not a setting: it is deliberately not persisted, so a scope reopened tomorrow starts where a returning user expects to start. Dragging a signal onto a plot is not configuration and survives the lock: it is the signal badge's drag (the list's rows are no longer drag sources, and the badges do not follow the lock). [sigbadges] drag_sources is the reading, and drag_sources on the signals panel: line must be 0.
• The session is the layout. LayoutState persists what makes a reopened scope look like the one that was closed: the window, the rows and their slot signals, the calibration, the unit renames ([[unit]], name + unit, only the overrides; ScopeModel::set_signal_unit re-applies them to every CONFIG, and set_signals() takes them off the incoming units before the "same stream" comparison so a reconnect still keeps its history), the enabled signals, the chosen trace colours ([[color]], name + rgb = "#RRGGBB" — only the overrides, so a stream's new signals still take the positional palette and a saved session does not freeze a colour that is meant to follow the signal list), the math channels with their graphs, the measurements pane, and measure_stats — the measurements statistics toggle; written as a bare top-level boolean (measure_stats = true/false), defaults to false when absent, and dock -- LayoutState::dock_shown, the Dock button's state (the user's last word on it, App::sidebar_wanted): a bare top-level boolean written where side_tab used to be, default true. It is the only thing the dock persists -- its 280 px width is fixed, so there is nothing to store -- and the open slide-over page is deliberately not persisted (a panel that opens by itself at start-up is a surprise, and a page id from another source kind may not exist). The parser ignores unknown keys (its key chain has no erroring final else), so a file written before the tabs became a dock (side_tab = "measure", or side_tab = "source-ao") still loads and opens with the dock shown; nothing reads the old key, and [dock-session] asserts it. Signal identity is the name throughout (a CONFIG renumbers channels), 'enabled' is read off the channels the model built -- an enabled signal is exactly one that produced a channel -- and apply_layout() installs the graphs before assigning the rows, since a row may hold a math channel's name and that name is only a signal once its graph is in. Two format rules, both learned the hard way: write an op or a measure type as its index through math_op_at()/measure_type_at(), never as the raw enum value -- those accessors' orders are the dialog's, not the declaration order, so writing one and reading the other silently renames operations. And a top-level key written after a [[table]] belongs to that table in TOML: enabled = [...] emitted after [[calibration]] was parsed into the last calibration record and the enable state was lost without a word. mkdirs() used to create only components with a trailing slash, so ~/.config/udpscope was never made and the first save failed silently -- worth knowing when a layout change "does not stick".
• A [[plot]] table is one plot (one cell), and carries row and width for the horizontal split. plots = N counts plots, as it always did. row is the 0-based figure row the plot sits in and width the cell's share of that row's width (LayoutState::Row::row, ::width); the tables run in the flattened order, so a row's cells are consecutive and left to right. The row's height share weight stays per figure row but is written on every cell of it (so a table read alone is complete) and read from the row's first cell; collect_layout() takes it from row_weights()[figure_row_of(plot)], not from row_weights()[plot] -- the latter is a plot index into a figure-row vector, right until the first split. The file is written row, width, weight, signals; a state built with row < 0 (nothing in the app does) omits row and width. Three loading rules, all in apply_layout() and none of them a refusal, because a session is the user's arrangement and not worth losing to a malformed key: (1) a file with no row loads as one cell per row -- every session written before the split -- with the per-plot weights as the row heights, exactly as it opened before; (2) a row sequence that cannot be a layout loads the same way: some plot without a row, a row number that goes backwards or skips one, a first row that is not 0, more than 4 cells in a row, or more than 8 plots; the signals stay on the plots they were on (invalid-rows is the reading); (3) cell widths are applied after the arrangement, because set_cells_per_row() resets them, and only for rows of two or more cells -- a one-cell row has no widths and its width is ignored. Both new keys are parsed with the file's other forgiving rules: row must be an integer >= 0 and width a number > 0, anything else leaves the default. A save after loading a pre-split file writes the row/width keys, which is how an old session is upgraded without anyone asking.
• Measurement options persist under named keys -- threshold, threshold2, noise and hf -- so a file written before them loads at the defaults, which are exactly the behaviour it was written against: noise = 10 is the hysteresis band the app has always counted with, hf = 0 is no filter, and a file with no threshold2 -- every file written before between existed -- leaves that measurement at threshold2 = 75, which is the value the dialog offers a new one. The pair is stored as written rather than as ordered: measure_level_pcts() is what puts a reversed pair in order on read, so a file records what the user typed and a measurement file round-trips unchanged.
• The math node string is thirteen pipe fields, and the reader accepts twelve or thirteen. med_n is the thirteenth, after y; a twelve-field node is a file written before the de-noise node existed, and its med_n stays at the struct's default -- which is what such a file meant. Field 0 is the op as a math_op_at() index, which is why MathOp may only be appended to.
• The export is collected at click time, not in the file-dialog callback. The live view keeps scrolling while the chooser is up, so a late collection would write a window the user never saw. The writer is picked by the chosen path's extension.
• The MDF export is MDF 3.30 written by hand, and its layout is the reader's, not the standard's prose. No dependency is added: MF4 is HDF5 and would need a library, while MF3 is a handful of fixed-size blocks, so write_mdf() spells them out. The field list comes from asammdf 8.8.27 (blocks/v2_v3_blocks.py, blocks/mdf_v3.py, which parses MF3 by explicit offsets) and was confirmed byte for byte against a known-good 3.30 file asammdf wrote (one group, a time master and two V channels). The shape, in file order: the ID block is 64 bytes with no id/length header ("MDF 3.30", byte order, version 330) — a writer that emits ID + size + links at offset 0 produces something no reader accepts, since the first real block starts at 64; the HD block is 208 (164 common + 44 of the 3.2-era time fields) and points at the data group; a TX comment follows it; then the samples; the DG is 28; a CC (conversion, 46 — a NONE conversion is the common fields alone) then a CN (channel, 228) precede each channel; and the CG (30) is last. Address fields are absolute file offsets written forward, so the whole address map is computed before a byte is written, and every block length is checked after it is built: a field at the wrong width does not fail the write, it leaves the reader in the middle of the next block.
• The data group's data address holds the raw records, with no DT header — which is what asammdf's writer does and what its reader assumes (DataBlock is documented as "a pseudo block not defined by the MDF 3 standard"). This is the one place the file follows the reader rather than a narrow reading of the standard, and it is the assumption ScryLab compatibility rests on: a file with a DT block there would have its header read as sample bytes.
• In MDF the master is abs_seconds, named time, and the relative axis is a channel time_relative — the same two numbers the .npy/.csv table holds, in the other roles. A master off the relative column is 0 at a different sample in each display mode, so it would put a live export and a held event's export on two timelines, which is the one thing the second column exists to avoid; both carry the unit s, and the signal columns follow in the .npy/.csv order. The unit is the client's CONFIG unit (a math channel's own), and it is written into the CC block because that is the only place MDF 3 keeps one — and the place ScryLab reads a signal's unit from. ExportTable::Column gained the field for it; the two older writers ignore it, and their bytes are unchanged. The names are sanitised and de-duplicated against each other and against the two reserved names before being cut to 31 bytes (the CN name field is 32 with its NUL): ScryLab imports only the first occurrence of a channel name, so a cut made before de-duplication could silently drop a signal. write_export() is the one place the extension picks the writer (.csv, .mf3/.mdf, anything else .npy), and both the dialog and the self-check's export probe come through it so the two cannot offer different formats.
• The chooser runs before ScopeModel exists, and --server is what decides. Config carries server_specified, set only by the --server branch, and main() asks only when it is false — so declining costs no buffer and no socket, and ScopeModel, Acquisition, RowSet and run_ui() need no notion of "no source yet". It is --server alone and not "no arguments": --multicast still needs a server to fetch CONFIG from. Every documented probe passes --server, which is what makes the feature invisible to the regression net — a probe added without it would reach the chooser and hang. gtk_init() plus a nested GMainLoop, and then run_ui()'s GtkApplication afterwards in the same process, is a verified combination (gtk_init() is idempotent; running your own loop is the documented GTK4 replacement for the removed gtk_dialog_run()).
• A session file's name cannot say how its source was reached, so the file says it. <host>_<port>.toml carries neither the multicast group nor the interface, so a chooser row could not reconnect to a multicast source at all. LayoutState::multicast/interface_addr are written in the top block, before the first [[table]] — a bare key after a table header is that table's key in TOML, which is exactly how enabled was parsed into the last [[calibration]] record and lost without a word. A file from an older build has neither key and loads as unicast on the default route. The chooser's list filters to sessions that name at least one signal, because a probe run leaves a file per throwaway port and all of them hold signals = ["", "", "", ""]; known_sources() splits the filename at the last _, since sanitize() turns anything unsafe in a host into one.
• save_layout_now() refuses to write while signals() is empty. A run against a source whose streamer was down receives no CONFIG, so every slot name stays empty and the save would overwrite a configured session with signals = ["", ...]. The chooser then filters that file out — a source whose streamer was merely down during one run would silently vanish from the only list it can be picked from, for ever. Keeping the old file loses nothing: a run that configured nothing had nothing worth persisting.
• It refuses again while replaying is set: a file view is not the source's session. Same argument, different cause — this view configured nothing about the stream. A plain replay leaves the rows where the user put them, so its save would merely be pointless; a group rebinds every slot to its own renamed channels and the save wrote signals = ["square (A)", …] into the file. Nothing noticed until the way back: Go live re-applies the session, those names match no live signal, so nothing was enabled and the live view came back with no channels at all — on the window the poisoned file carried, which is also how it swallowed the window restore below it. The reading is [events] live restored (after a group): the symptom was a stuck pending=1 beside channels=0, because the restore waits for a channel list that was never going to arrive. The explicit Save as is deliberately not guarded: that one is a user's own answer to "save this view", and its dialog reports the outcome.

A configuration is a named session, and a source may hold several

One source, several arrangements: the same streamer serves different experiments, and retyping a layout to look at the second one is how a session gets lost. Storage is therefore a directory per source — ~/.config/udpscope/<host>_<port>/ — holding one <name>.toml per configuration plus an active file naming the one the source is on. configuration_path() is the only expression of the scheme, and the name is sanitize()d exactly as the host is, so a name typed with a slash cannot escape the directory.

A configuration carries the whole session and the settings that can only be decided before the first socket: buffer_samples, relative_time, capture_packets. The startup chooser applies them (apply_source()), which works because choose_source() runs before ScopeModel is constructed — the one moment a buffer size can be applied at all. Switching configurations inside a running app applies what can change live and says in the status line that the rest takes effect on the next start; pretending otherwise would be a configuration that half-applies.

A configuration's startup settings reach the model from two directions, and both are needed: apply_source() for a source picked in the chooser, and apply_stored_settings() for a source named with --server, which never reaches the chooser at all. Without the second, a saved host's buffer size, clock and capture ring were silently ignored — the layout still applied, because the app loads that separately once a CONFIG names the signals — so an edit to any of the three read as one that never took effect. A flag the user typed wins over the file (--buffer-samples, --relative-time, --capture-packets): the file fills in what the command line did not say, and Config::buffer_samples_specified is what tells "not asked" from "asked for the default".

The active file is written on every save and by the chooser's pick, so "which configuration was I on" survives a restart. A session written before configurations existed is imported once as the configuration named default, and the old file is renamed to <host>_<port>.toml.imported rather than deleted: an import that cannot be undone by hand is a migration nobody can back out of.

Nothing is lost by switching: the dialog saves the current session before loading the picked one, which is the only ordering in which a switch cannot lose what it switches from.

• The event cache is keyed on the source and its active configuration, and the key moves with the switch. An event's trigger, its channel set and its units are the configuration's, so the directory is <base>/<host>-<port>/<configuration>/ (EventStore's third constructor argument) and retarget_event_store() -- called next to every App::config_name write and resolving the name from active_configuration() when it is empty -- re-derives it, refreshing both views of the list only when the directory changed. A store built without a configuration keeps the legacy <base>/<source>/ path, and the first construction under a configuration adopts the loose event files it finds there (*.npy/*.txt/*.bmp, moved into the configuration's folder): an upgrade keeps the events it had. The key is a sanitized path component, the same rule the configuration files use, so a name typed with a slash cannot leave the directory.

• The session remembers extra windows; a launch never opens them. Each scope or detached window is a [[window]] block (name, detached, and its own [[window.plot]] rows -- the same keys the main [[plot]] rows carry), written from the open windows' present arrangement plus the ones already closed (collect_layout()). Loading puts all of them in App::closed_windows, never into app.scopes: the overflow menu's Reopen closed windows (N) is the only path that opens them (open_scope_window_from_state(), the one constructor every window uses, then its rows). Closing a window records it (by name, replaced, so close/reopen cycles do not multiply entries); attaching records nothing -- its figure is on the main window again -- and a detached figure survives a quit through its window's entry instead of being lost from the main layout.