docs: the sample budget in samples per signal, the memory limit and the planner probe lines
Converts every --buffer-mb / buffer_mb mention to --buffer-samples / buffer_samples, records the unit change as a deviation (README, SPECS, CLAUDE.md), documents the buffer: and planner: dump lines and the new Preferences row, and updates the help pages and the help-shot script. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
1 parent
034c82f7d7
commit
1627abcd4a
19 files changed
+166
-93
No files matched your search
@@ -22,8 +22,9 @@ It is used for exactly one widget — the shortcut overview is an `AdwShortcutsD
|
||||
GTK4's own `GtkShortcutsWindow` is deprecated as of 4.18 and `AdwShortcutsDialog` is its
|
||||
replacement.
|
||||
|
||||
Options: `--server`, `--port`, `--multicast`, `--interface`, `--buffer-mb`
|
||||
(size of the sample buffer, default 200), `--max-window` (largest time
|
||||
Options: `--server`, `--port`, `--multicast`, `--interface`, `--buffer-samples`
|
||||
(samples kept per acquired signal, default 10M; see "The sample buffer is budgeted in samples" below),
|
||||
`--memory-limit-mb` (hard ceiling on the ring storage, default 8192), `--max-window` (largest time
|
||||
window the toolbar offers, default 600), `--holdoff` (default trigger hold-off),
|
||||
`--recv-buffer`, `--plots` (the number of stacked plots to start with, 1..8; rows are
|
||||
added and removed from the **Layout editor** afterwards), `--relative-time` (see below),
|
||||
@@ -33,16 +34,36 @@ added and removed from the **Layout editor** afterwards), `--relative-time` (see
|
||||
`GSK_RENDERER` and `LIBGL_ALWAYS_SOFTWARE` before GTK initialises — see the SSH
|
||||
limitation below).
|
||||
|
||||
**What a large `--buffer-mb` costs at startup, and what the window says while it happens.** The
|
||||
buffer's rings are value-initialised, so allocating one touches every byte of `--buffer-mb` --- about
|
||||
**The sample buffer is budgeted in samples per signal, under a memory limit.** `--buffer-samples N`
|
||||
(`250000`, `64k`, `100M`, `1.5G`; k, M and G are decimal, a *sample* count and not a byte size) is how
|
||||
many samples to keep for **every acquired signal**, and for the shared time ring. A ring is a power of
|
||||
two, so the request is rounded **down** to one --- "up to" --- and the UI shows both figures: the
|
||||
default 10 M asks for, and gets, 8 388 608 per signal, and `--buffer-samples 100M` gets 67 108 864. What that
|
||||
costs in bytes follows from the channel count (`ticks x 8` for the clock, doubled in relative mode, plus
|
||||
`ticks x 4` per channel), and the status popover and the badge text quote the *planned* bytes rather than a
|
||||
figure the user typed. The ceiling on those bytes is a separate application setting,
|
||||
**`--memory-limit-mb`** (default 8192 MiB; also a *Memory limit* row in **Preferences**, which re-plans the
|
||||
live buffer when changed). `plan_partition()` starts at the power-of-two floor of the request and
|
||||
**halves** it until the rings fit under the limit, and throws only when 128 samples do not fit. A clipped
|
||||
plan is never silent: stderr gets one line (`[udpscope] buffer limited to 8.39 M samples per signal (100 M
|
||||
requested) by the 256 MiB memory limit; the rings take 192 MiB`), the status popover's buffer row names the
|
||||
limit, and the source form's note says so, assuming one signal there because the channel count is not
|
||||
known before a stream is. `--buffer-mb` and the stored `buffer_mb` key are gone, with no compatibility
|
||||
shim: the flag is an unknown argument, and an old `buffer_mb` in a saved configuration is ignored (the
|
||||
source falls back to the default until it is saved again as `buffer_samples`). **This is a deliberate
|
||||
change of the budget's unit** (it was megabytes of ring, which made the retained history depend on the
|
||||
channel count and on relative time); it is recorded under "Deviations from the spec" in `SPECS.md`.
|
||||
|
||||
**What a large buffer costs at startup, and what the window says while it happens.** The
|
||||
buffer's rings are value-initialised, so allocating one touches every byte of the planned rings --- about
|
||||
a second per gigabyte, mostly first-touch page faults --- and it happens on the producer thread
|
||||
holding the lock the frame tick also takes. It is therefore a real freeze and not a slow paint. Two
|
||||
things follow. First, a rebuild whose plan describes the storage already allocated **clears** the
|
||||
store instead of allocating a new one, which is what most structural changes are: startup used to ask
|
||||
for fifteen allocations and now asks for two or three (3.36 s → 1.18 s at `--buffer-mb 1000`), and
|
||||
for fifteen allocations and now asks for two or three (3.36 s → 1.18 s at a gigabyte of rings), and
|
||||
every enable of a signal is applied in one batch rather than re-planning per signal. Second, the
|
||||
window is **presented before** the buffer is planned and carries a notice naming what it is waiting
|
||||
for --- `connecting to 127.0.0.1:8000 — planning the 4000 MB buffer` --- because the remaining second
|
||||
for --- `connecting to 127.0.0.1:8000 — planning the buffer: 134 M samples per signal` --- because the remaining second
|
||||
or so at a multi-gigabyte buffer is unavoidable, and a blank window that says nothing is the part of
|
||||
it that reads as a hang. Each allocation also names itself on stderr
|
||||
(`[udpscope] buffer allocated: 4 channels, 839 s span, 74 ms`), so a run that prints a column of them
|
||||
@@ -740,8 +761,12 @@ The history cost of relative mode is measured rather than computed:
|
||||
| stream time (default) | **83.886 s** |
|
||||
| `--relative-time` | **41.943 s** |
|
||||
|
||||
Conditions: the mock at 100 kS/s, 4 channels at stride 1, the default `--buffer-mb 200`.
|
||||
Relative mode retains **exactly half** the history — not the ~22% the original spec claimed, and
|
||||
Conditions: the mock at 100 kS/s, 4 channels at stride 1, the old byte budget (`--buffer-mb 200`, since removed). **That
|
||||
no longer happens**: the buffer is budgeted in samples per signal, so both modes retain the same span
|
||||
(8 388 608 ticks at the default) and relative mode instead costs a third more *memory* (32 against 24
|
||||
bytes a tick with four channels; the `planner:` dump line in `docs/claude/probe-readings.md` reads it); only a memory limit that
|
||||
bites clips it first. The paragraph below is the measurement that motivated it.
|
||||
Under the old byte budget relative mode retained **exactly half** the history — not the ~22% the original spec claimed, and
|
||||
not the 75% a per-byte calculation suggests. The reason is the floor: relative mode keeps a second
|
||||
f64 ring for absolute time beside the stream clock's, so a tick's cost goes from
|
||||
`8 + 4×4 = 24` bytes to `16 + 4×4 = 32` bytes — a third more. But `Partition::time_capacity` is
|
||||
@@ -889,7 +914,7 @@ Three things worth knowing:
|
||||
replayed samples age out of the ring as the stream refills it.
|
||||
- **Opening an event, a comparison or going back to *Live* is one re-plan of the buffer**, not one
|
||||
per channel. Each of those switches rebuilds the signal set, and a rebuild that moves the plan
|
||||
allocates --- and so touches --- the whole of `--buffer-mb`; enabling a comparison's eight channels
|
||||
allocates --- and so touches --- the whole of the planned rings; enabling a comparison's eight channels
|
||||
one at a time was eight of those, which is most of what made the switch feel slow. They are now
|
||||
enabled in a single call, and a rebuild whose plan is unchanged clears the buffer instead of
|
||||
allocating a new one.
|
||||
@@ -923,15 +948,15 @@ Only `.npy` is read: it is the format the item asked for and the one that carrie
|
||||
export is for other tools.
|
||||
|
||||
**A recording longer than the buffer can hold at its own rate is resampled, never cut short.** The
|
||||
file is pushed into the same ring a stream fills, and that ring is sized by `--buffer-mb`: 20 s of ten
|
||||
channels at 500 kS/s is 480 MB of rings, which a 200 MB buffer cannot hold. The loader therefore reads
|
||||
file is pushed into the same ring a stream fills, and that ring is sized by `--buffer-samples`: 20 s of ten
|
||||
channels at 500 kS/s is 10 M samples a channel, which the default 8.39 M cannot hold. The loader therefore reads
|
||||
the decimation the plan chose and pushes one stored sample per group of file rows, carrying the
|
||||
group's **mean** — the same anti-aliased filter the live ingest applies, so a long event opens as the
|
||||
whole recording at a lower sample rate instead of as a fraction of itself. It used to push every row
|
||||
regardless, which overran the ring and left the file as **its own tail**: a 20 s event opened showing
|
||||
only its last 4 s, with a correct-looking 20 s header line above it. The load line now names both
|
||||
numbers, `samples=31250 rows=1000000 decim=32`, so what is on screen is never in doubt; give the run a
|
||||
larger `--buffer-mb` (the source's configuration carries one) to see such a file sample for sample.
|
||||
larger `--buffer-samples` (the source's configuration carries one) to see such a file sample for sample.
|
||||
|
||||
## Named configurations
|
||||
|
||||
@@ -953,18 +978,18 @@ measurements — **and the settings that can only be decided at startup**, which
|
||||
here rather than only on the command line:
|
||||
|
||||
```toml
|
||||
comment = "5 kHz sine, trigger on Ch1, 8 MB is plenty"
|
||||
buffer_mb = 8
|
||||
comment = "5 kHz sine, trigger on Ch1, 256 k samples is plenty"
|
||||
buffer_samples = 262144
|
||||
relative_time = false
|
||||
capture_packets = 0
|
||||
```
|
||||
|
||||
The **buffer size** is the one that matters most: two sources on one machine rarely want the same
|
||||
The **buffer size** (samples per signal) is the one that matters most: two sources on one machine rarely want the same
|
||||
ring, and the value is applied before the model exists, which is the only moment it can be — by
|
||||
whichever of the chooser's rows opened the session, **and by `apply_stored_settings()` when the
|
||||
source was named on the command line**. A saved host's settings are read either way: opening
|
||||
`udpscope --server 10.0.0.5 --port 8000` gets that source's stored buffer size, clock and capture
|
||||
ring, and a flag the user actually typed wins over the file (`--buffer-mb`, `--relative-time`,
|
||||
ring, and a flag the user actually typed wins over the file (`--buffer-samples`, `--relative-time`,
|
||||
`--capture-packets`) because it is the more recent statement of intent. Switching configurations *inside* a running
|
||||
app applies everything that can change live (layout, plots, math, measurements, enables) and says
|
||||
so in the status line about the rest (buffer, clock, packet capture): they take effect on the next
|
||||
@@ -1004,7 +1029,11 @@ applies as it is made — there is no Apply button, and a **Reset to defaults**
|
||||
dialog puts everything back — and the rows are the ones every GNOME application shows: a title, a
|
||||
subtitle under it where the setting needs explaining, and the control on the right. A key that is
|
||||
absent means its default, so the file only mentions what changed, and a missing file is a first run
|
||||
rather than an error. The directory field shows the directory in use: the default
|
||||
rather than an error. The **Sample buffer** group holds the *Memory limit (MB)* row (`memory_limit_mb`,
|
||||
default 8192): the hard ceiling on the ring storage for the whole application, which the planner halves the
|
||||
sample budget to fit. It is the one planner input that changes while the app runs, so changing it
|
||||
re-plans the live buffer --- dropping history only if the storage actually changes --- except while a
|
||||
recording or a replay is on screen, where it is stored for the next plan. The directory field shows the directory in use: the default
|
||||
is filled in rather than described, and typing it back leaves the setting unset, so it goes on
|
||||
following the cache directory instead of pinning one path for good. It can be typed, or picked with the folder
|
||||
button beside it, which writes what it found into the same field -- the field stays the one value, and
|
||||
@@ -1247,11 +1276,11 @@ allocates one `BufferStore` per CONFIG: one shared time-base ring plus one value
|
||||
ring per channel (N enabled signals -- see the limitation on multi-element signals
|
||||
below). The spec's second buffer is the latched `EventSnapshot`, which is already
|
||||
an immutable copy of the captured window (see "Trigger" below); a second live ring
|
||||
bought nothing and cost half the memory budget, so `--buffer-mb 200` really is
|
||||
200 MB. What it *would* buy is priced in the dump's `publish cost:` line — see
|
||||
bought nothing and cost half the memory, so the planned bytes the status line quotes are
|
||||
the whole of it. What it *would* buy is priced in the dump's `publish cost:` line — see
|
||||
"There is one live buffer, not two" under the review findings.
|
||||
`plan_partition()` sizes the rings to cover `--max-window` seconds and then grows
|
||||
them with the rest of the budget. When a channel's declared sample rate is lower
|
||||
`plan_partition()` sizes the rings to the sample budget (rounded down to a power of two, halved
|
||||
until it fits the memory limit) and then decimates the shared clock until `--max-window` seconds fit. When a channel's declared sample rate is lower
|
||||
than the fastest channel's, it is decimated by the integer ratio
|
||||
(`Partition::stride`); when the requested window does not fit the budget, the shared
|
||||
clock is resampled too (`Partition::time_stride`, an averaging decimation — see
|
||||
@@ -1437,8 +1466,8 @@ tick and 12.8 ms of draw a frame and 4.9 MB uploaded** on llvmpipe, and reads **
|
||||
5.6 ms and 1.0 MB** at the cap.
|
||||
|
||||
**And it says what it costs.** A retained acquisition is a whole window copied out of the ring, so
|
||||
the retention is bounded by two things: the depth, and **`--buffer-mb`** — persistence may hold at
|
||||
most as much again as the buffer does, which is the bound a long window on a fast stream reaches
|
||||
the retention is bounded by two things: the depth, and **the planned buffer** — persistence may hold at
|
||||
most as many bytes again as the rings take (`persist_budget_bytes()`), which is the bound a long window on a fast stream reaches
|
||||
long before the eighth layer. The other bound is the plot's own: a layer takes a *free trace slot
|
||||
in the row that plots its signal*, layer by layer, so a row of four signals — the layout the app
|
||||
starts with — has room for one older acquisition of each, while a row per signal has room for all
|
||||
@@ -1654,7 +1683,7 @@ enabled only to feed a math channel.
|
||||
several, and there was no way to see which sources the app already knew about. The chooser lists
|
||||
one row per stored session: the server and port it was named after, how many plots and signals it
|
||||
held, and its multicast group when it had one, newest first. `New host…` is a four-field form —
|
||||
server, port, multicast group, interface — and nothing more: `--buffer-mb` and the rest are tuning
|
||||
server, port, multicast group, interface — and nothing more: `--buffer-samples` and the rest are tuning
|
||||
rather than "which source", and a form that asks for them is a form you have to read before using
|
||||
the app. Cancel exits before anything is allocated, because declining to choose a source is not a
|
||||
failure.
|
||||
@@ -1858,7 +1887,7 @@ fixes above bound *how often* and *how much per sample*; what is left is the ari
|
||||
scale: a pass is priced per measured **channel** (one fetch of the window's level-0 buckets, one
|
||||
statistics scan) and per **distinct share** in the pane (one level accumulator, walked over those
|
||||
buckets), and the window's own sample count multiplies both. Measured with the new
|
||||
`[levels-probe]` sweep on the mock at a **15 s** window and `--buffer-mb 4000` (so `time_stride`
|
||||
`[levels-probe]` sweep on the mock at a **15 s** window and `--buffer-samples 100M` (so `time_stride`
|
||||
stays 1 and the window holds 11.8 M samples, 184 k buckets), four signals, one row, on llvmpipe
|
||||
— `measure cost:`'s `each` per recompute, and the tick's mean:
|
||||
|
||||
@@ -2001,9 +2030,9 @@ a sample and runs out on a clock. Widening only the clock keeps the cost where i
|
||||
the rings are the memory budget, and the values are the overwhelming majority of them. The
|
||||
one consequence worth knowing is that `plan_partition()`'s growth loop can no longer count
|
||||
slots, because a time slot and a value sample are now different sizes — it weighs the
|
||||
budget in **bytes** (`sizeof(f64)` for the shared clock plus `sizeof(f32) / stride[i]` per
|
||||
channel) and the power-of-two rounding absorbs the rest: a three-channel stream at
|
||||
`--buffer-mb 200` retains the same 8.39 s span it did before. `EventSnapshot::stamps` is
|
||||
budget in **bytes** against the memory limit (`sizeof(f64)` for the shared clock plus `sizeof(f32)` per
|
||||
channel); the sample budget itself is a count of ticks, so a three-channel stream at
|
||||
`--buffer-samples 10M` retains the same 8.39 M ticks it did before. `EventSnapshot::stamps` is
|
||||
`f64` for the same reason rather than for symmetry — an *event's* stamps are rebased onto
|
||||
its edge and so are small, but a *pause* view is deliberately not rebased and carries full
|
||||
stream time, which quantised at exactly the rate the live ring did.
|
||||
@@ -2646,7 +2675,7 @@ This reverses the earlier "the app does no decimation at all" decision, and keep
|
||||
reason: a naive stride is what made a 2 kHz sine on a 1 MHz clock plot as a
|
||||
full-amplitude 125 Hz sine — a plausible-looking waveform that was never in the data.
|
||||
The headless check is the same aliasing case with the filter in place: at
|
||||
`--buffer-mb 1 --freq 2000` with a 60 s window the clock decimates by 16 to a 625 S/s
|
||||
`--buffer-samples 65536 --freq 2000` with a 60 s window the clock decimates by 16 to a 625 S/s
|
||||
stored rate (312.5 Hz corner) and the channel dump reads `y=[-0.059, 0.079]`, roughly
|
||||
the boxcar's predicted 6%. A `y=[-1, 1]` there would mean the filter had been lost.
|
||||
|
||||
@@ -2912,7 +2941,7 @@ now — they are listed so they are not mistaken for anything else:
|
||||
smaller than the stall: an A/B with and without a pause read `gaps=0` both ways, so
|
||||
what a second ring would buy is a smoother frame at a latch, not recovered packets.
|
||||
- **The buffer is allocated at the first CONFIG, not at initialisation.**
|
||||
The spec's "N+1 split" needs N, which only a CONFIG gives; `--buffer-mb` is the
|
||||
The spec's "N+1 split" needs N, which only a CONFIG gives; `--buffer-samples` is the
|
||||
*budget*, and the power-of-two ring rounding means the final
|
||||
allocation lands in `(budget/2, budget]` rather than exactly on it. Every CONFIG,
|
||||
enable/disable and calibration re-partitions (and drops the history) because the
|
||||
|
||||
Reference in new issue
Block a user