docs: the reseq probe's readings, and the invariants resequencing changes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
1 parent
4549c622fc
commit
eacb3f5efc
11 files changed
+258
-23
No files matched your search
@@ -26,7 +26,7 @@ Options: `--server`, `--port`, `--multicast`, `--interface`, `--buffer-mb`
|
|||||||
(size of the sample buffer, default 200), `--max-window` (largest time
|
(size of the sample buffer, default 200), `--max-window` (largest time
|
||||||
window the toolbar offers, default 600), `--holdoff` (default trigger hold-off),
|
window the toolbar offers, default 600), `--holdoff` (default trigger hold-off),
|
||||||
`--recv-buffer`, `--plots` (the number of stacked plots to start with, 1..8; rows are
|
`--recv-buffer`, `--plots` (the number of stacked plots to start with, 1..8; rows are
|
||||||
added and removed from the **Layout editor** afterwards).
|
added and removed from the **Layout editor** afterwards), `--relative-time` (see below).
|
||||||
|
|
||||||
Every CONFIG is dumped to stderr once -- publish mode and, per signal, its name,
|
Every CONFIG is dumped to stderr once -- publish mode and, per signal, its name,
|
||||||
element count, rate, time mode and time-signal index -- because a signal that
|
element count, rate, time mode and time-signal index -- because a signal that
|
||||||
@@ -363,6 +363,50 @@ latch, a pan and a zoom do **not** reset: they change which samples are in view,
|
|||||||
exactly what a statistic over many windows is meant to average. The toggle itself does not
|
exactly what a statistic over many windows is meant to average. The toggle itself does not
|
||||||
reset either, so turning the line off to read the plot and back on again keeps the history.
|
reset either, so turning the line off to read the plot and back on again keeps the history.
|
||||||
|
|
||||||
|
## Resequencing and relative timing
|
||||||
|
|
||||||
|
**A shaded band marks samples the stream says exist and have not arrived.** When a packet is
|
||||||
|
received that reveals a gap in the counter, a run of NaN samples is reserved for the missing
|
||||||
|
packets; the span is shaded in the plot and the counters in the status bar show it. The band
|
||||||
|
carries three states:
|
||||||
|
|
||||||
|
- **Solid** while the hole still lacks packets.
|
||||||
|
- **Faint** once the late packets arrive and repair the values — the data is now correct, but
|
||||||
|
the record remains that this region arrived out of order.
|
||||||
|
- **Absent** once that part of the stream scrolls out of the ring's buffer.
|
||||||
|
|
||||||
|
Two things are **not** repaired when a late packet arrives: **math channels** and **the trigger**.
|
||||||
|
A backfilled slot's value is corrected in the ring and in the pyramid, so the raw trace and the
|
||||||
|
readouts are right. The math engine and the trigger already ran over the placeholder, however, and
|
||||||
|
are not replayed: a math trace keeps its gap across a repaired hole, and if the trigger latched
|
||||||
|
adjacent to one, that latch was on the data the engine actually saw.
|
||||||
|
|
||||||
|
**Known limitations.** The fade is all-or-nothing at the hole level — a hole with one packet still
|
||||||
|
missing shades solidly, even if all but one slot are already repaired. And the two tints are not
|
||||||
|
yet configurable; the preferences dialog in `todo.md` is where that would land. Do not use a
|
||||||
|
command-line flag for either; none exists.
|
||||||
|
|
||||||
|
**`--relative-time` makes the x axis follow the sample count and the declared rate**, rather than
|
||||||
|
the stream's own clock. The packet counter says exactly how many samples each gap spans, so a
|
||||||
|
reserved hole is a precise arithmetic range. In stream mode the same hole is estimated from the
|
||||||
|
declared cadence, which is the existing glitch-bridge mechanism reused.
|
||||||
|
|
||||||
|
The history cost of relative mode is measured rather than computed:
|
||||||
|
|
||||||
|
| mode | retained span |
|
||||||
|
| --- | --- |
|
||||||
|
| stream time (default) | **83.886 s** |
|
||||||
|
| `--relative-time` | **41.943 s** |
|
||||||
|
|
||||||
|
Conditions: the mock at 100 kS/s, 4 channels at stride 1, the default `--buffer-mb 200`.
|
||||||
|
Relative mode retains **exactly half** the history — not the ~22% the original spec claimed, and
|
||||||
|
not the 75% a per-byte calculation suggests. The reason is the floor: relative mode keeps a second
|
||||||
|
f64 ring for absolute time beside the stream clock's, so a tick's cost goes from
|
||||||
|
`8 + 4×4 = 24` bytes to `16 + 4×4 = 32` bytes — a third more. But `Partition::time_capacity` is
|
||||||
|
floored to a **power of two** (`src/model.cpp:83`, `:110`), so a third more weight does not cost a
|
||||||
|
third of the span; it costs a whole halving. The measured figures above are the correct ones; a
|
||||||
|
per-byte computation gives the wrong answer for this reason.
|
||||||
|
|
||||||
## The session layout file
|
## The session layout file
|
||||||
|
|
||||||
What you arranged is remembered per data source, in
|
What you arranged is remembered per data source, in
|
||||||
|
|||||||
@@ -36,3 +36,16 @@
|
|||||||
- the time should be computed from the time associated with the signal (not from the client but from the streamed data)
|
- the time should be computed from the time associated with the signal (not from the client but from the streamed data)
|
||||||
- in trigger mode data visualisation should not be refreshed till next event, that's why 2 identical buffer, one is the currently
|
- in trigger mode data visualisation should not be refreshed till next event, that's why 2 identical buffer, one is the currently
|
||||||
viewed event, the other used to acquire live data for next event
|
viewed event, the other used to acquire live data for next event
|
||||||
|
|
||||||
|
## Deviations from the spec
|
||||||
|
|
||||||
|
The following were built differently from what the spec states, for documented reasons in `README.md`:
|
||||||
|
|
||||||
|
- **`todo.md` item 2 — relative timing.** Implemented as `--relative-time`. The retained history
|
||||||
|
in this mode is **41.943 s** against the default **83.886 s** (measured at 100 kS/s, 4 channels,
|
||||||
|
`--buffer-mb 200`): exactly half, because a second f64 ring for absolute time raises a tick's
|
||||||
|
cost from 24 to 32 bytes — a third more — and `Partition::time_capacity` is floored to a power
|
||||||
|
of two, so a third more weight costs a full halving. The spec and the original plan both quoted
|
||||||
|
~22%, which is wrong; the per-byte figure of 75% is also wrong for the same reason.
|
||||||
|
Absolute stream time is preserved in a separate ring and used for the export's `abs_seconds`
|
||||||
|
column, exactly as `todo.md` requires.
|
||||||
@@ -62,6 +62,28 @@ handler keyed by a pointer-to-pointer (`gpointer slot` cast to `AdwDialog**`), s
|
|||||||
handler clears either pointer without knowing which dialog it is. Presenting either on this
|
handler clears either pointer without knowing which dialog it is. Presenting either on this
|
||||||
app's plain `GtkApplicationWindow` is supported and warning-free; it was checked.
|
app's plain `GtkApplicationWindow` is supported and warning-free; it was checked.
|
||||||
|
|
||||||
|
`deps/udpsclient` is a vendored C99 source pair and its diff from the upstream source
|
||||||
|
is intentional. Four things were changed:
|
||||||
|
|
||||||
|
1. **A reorder ring of reassembled payloads.** When a packet arrives with a higher counter
|
||||||
|
than expected, the raw datagram is held in a fixed-size ring rather than decoded immediately.
|
||||||
|
A deadline (`now + reorder_window_s`, default 0.020 s) is set for the gap; the deadline is
|
||||||
|
served by a third `clamp_deadline()` call on the `select()` budget
|
||||||
|
(`udps_client.c:1145-1152`) so a quiet stream does not wait for the next arrival.
|
||||||
|
2. **Late delivery replacing the outright discard.** A packet that arrives past the window — or
|
||||||
|
when the ring is drained by the deadline — is delivered to the callback flagged `late` rather
|
||||||
|
than discarded with `stats.stale_packets++; return 0`. That flag is what makes backfill
|
||||||
|
possible at all; without it the samples are gone before udpscope can act.
|
||||||
|
3. **The third `clamp_deadline()`.** The existing pair handles the keepalive and the silence
|
||||||
|
timeout; the reorder deadline is a third call to the same helper, which already understood
|
||||||
|
the pattern.
|
||||||
|
4. **`reorder_window_s = 0` as the switch.** `udps_client_config_t::reorder_window_s`,
|
||||||
|
defaulted to `0.020` in `udps_client_default_config()` (`deps/udpsclient/udps_client.c:1190`),
|
||||||
|
is the only knob. Setting it to zero disables resequencing and restores the previous
|
||||||
|
discard-on-late behaviour exactly — so a reported regression is bisectable without reverting
|
||||||
|
the source. **There is no command-line flag for the reorder window.** It is not surfaced on
|
||||||
|
udpscope's command line; do not document a `--reorder-window-ms`.
|
||||||
|
|
||||||
`deps/plotter` is upstream's own repository and this app's figure support was added
|
`deps/plotter` is upstream's own repository and this app's figure support was added
|
||||||
there rather than worked around here: `Figure::set_min_y_gutter()` (a floor under the
|
there rather than worked around here: `Figure::set_min_y_gutter()` (a floor under the
|
||||||
common left gutter), `Figure::set_right_gutter()` / `Plot::set_right_gutter()` (an empty
|
common left gutter), `Figure::set_right_gutter()` / `Plot::set_right_gutter()` (an empty
|
||||||
|
|||||||
@@ -96,6 +96,26 @@ Part of [CLAUDE.md](../../CLAUDE.md). These are rules that cost a defect to lear
|
|||||||
the export guard in [probe-readings.md](probe-readings.md) for why it is no longer exactly one
|
the export guard in [probe-readings.md](probe-readings.md) for why it is no longer exactly one
|
||||||
value). Three steps at 2, 2.5 and
|
value). Three steps at 2, 2.5 and
|
||||||
3 ULP of *the stream's uptime* — microseconds, not femtoseconds — is this defect.
|
3 ULP of *the stream's uptime* — microseconds, not femtoseconds — is this defect.
|
||||||
|
- **Append is not the only write.** A backfill amends slots that a reservation wrote as NaN, and
|
||||||
|
the amend is split across threads for a reason:
|
||||||
|
- **Ring amend — producer thread.** `RingBuffer<T>::amend(absolute, v)` is a single relaxed
|
||||||
|
store, bounds-checked against `[produced_ - capacity, produced_)`. The same race class as a
|
||||||
|
push, and an `f32` cannot tear. Measurements and export read the ring directly, so they see
|
||||||
|
a repair immediately and no second notification is needed.
|
||||||
|
- **Pyramid amend — consumer (GTK main thread), inside `ChannelBuffer::acquire()`.** The
|
||||||
|
producer collects amended values in a plain `std::vector<Amend>` member and the consumer
|
||||||
|
drains it after its own append pass. `acquire()` is called under `model.mutex()`, and the
|
||||||
|
producer holds that same mutex for all of `on_data`, so the vector is correctly serialised
|
||||||
|
with no lock-free queue. The reason the pyramid amend cannot also be on the producer: a
|
||||||
|
bucket is a 32-byte non-atomic aggregate that the renderer reads **unlocked at paint time**,
|
||||||
|
so a producer writing it could be caught mid-bucket — a `count` that does not match its
|
||||||
|
`sum`, producing a wrong mean from a structure whose readers trust `count == 0` to mean
|
||||||
|
*absent*. On the consumer that cannot happen.
|
||||||
|
- **A hole is stamped at reservation; backfill changes values only.** The f64 time ring is
|
||||||
|
never amended. `neighbours()' binary search requires a monotonic array, and `window_x0_`
|
||||||
|
must remain valid; pre-stamping the hole at the declared cadence (exact in relative mode,
|
||||||
|
the bridge mechanism in stream mode) is what keeps both invariants, and it is why a late
|
||||||
|
packet's counter, not its arrival stamp, is what addresses the hole.
|
||||||
- **A dropped time slot is clamped, never answered with a sentinel.** `acquire()` pins
|
- **A dropped time slot is clamped, never answered with a sentinel.** `acquire()` pins
|
||||||
the published window to the value ring's base and both rings keep sliding afterwards —
|
the published window to the value ring's base and both rings keep sliding afterwards —
|
||||||
their capacities are equal, so there is no slack — which means the oldest few samples
|
their capacities are equal, so there is no slack — which means the oldest few samples
|
||||||
|
|||||||
@@ -104,6 +104,29 @@ Part of [CLAUDE.md](../../CLAUDE.md). These are rules that cost a defect to lear
|
|||||||
figure rebuild — which `assign()` already forces. A signal on no plot gets
|
figure rebuild — which `assign()` already forces. A signal on no plot gets
|
||||||
`RowSet::unplotted_color()` (grey) rather than no chip, so the panel's rows stay the
|
`RowSet::unplotted_color()` (grey) rather than no chip, so the panel's rows stay the
|
||||||
same height.
|
same height.
|
||||||
|
- **The invalid-span band is data-anchored and reserves no layout space.** `Plot::set_invalid_spans()`
|
||||||
|
draws a tinted quad over the span it names; unlike `Plot::set_x_note()` (the `t0` caption), which
|
||||||
|
is chrome drawn at a fixed corner of the axes regardless of the data, a band pans and zooms with
|
||||||
|
the samples it accuses. The two are the complementary cases: the caption is chrome that data does
|
||||||
|
not move, the band is annotation that data does. Both are alike in the second count: neither
|
||||||
|
reserves layout space and neither can be picked, so a hole cannot move the layout the labels were
|
||||||
|
measured into.
|
||||||
|
|
||||||
|
The band has a **one-device-pixel floor on its width**. A hole of one sample in a window of hundreds
|
||||||
|
of thousands is a hundredth of a column, and a fill rounded to zero would drop exactly the case
|
||||||
|
the band is most needed for — a sub-pixel hole is invisible as a gap in the polyline and would be
|
||||||
|
invisible as a band too without the floor.
|
||||||
|
|
||||||
|
The band carries **three states** and they are a property of the hole, not of the plot: solid
|
||||||
|
(`Style::invalid_fill`) while the hole still lacks packets, faint (`Style::invalid_repaired_fill`)
|
||||||
|
once `Hole::repaired()` is true, absent once the hole's slots age out of the ring and
|
||||||
|
`forget_holes_before()` drops it from the table. A repaired hole is **not** retired early: the band
|
||||||
|
is a record of what the window was missing, and retiring it on repair would erase the report the
|
||||||
|
moment it became explicable — the faint tier is how the display says "this was missing but arrived
|
||||||
|
late". `RowSet` sends one `plot::InvalidSpan` per hole to every row — a gap is in the stream, not
|
||||||
|
in one signal — and only in `Live` and `Capturing`, because a `Held` or `Paused` snapshot carries
|
||||||
|
its own stamps that the live hole table does not address.
|
||||||
|
|
||||||
- **`RowSet::signal_color()` is the only route to a trace's colour.** The user can override the
|
- **`RowSet::signal_color()` is the only route to a trace's colour.** The user can override the
|
||||||
palette entry per signal (clicking its chip; `set_signal_color()` / `clear_signal_color()`,
|
palette entry per signal (clicking its chip; `set_signal_color()` / `clear_signal_color()`,
|
||||||
keyed by **name**, because a palette index is positional and a CONFIG renumbers), and that
|
keyed by **name**, because a palette index is positional and a CONFIG renumbers), and that
|
||||||
|
|||||||
@@ -122,6 +122,10 @@ Part of [CLAUDE.md](../../CLAUDE.md). These are rules that cost a defect to lear
|
|||||||
export.** Both need the same absolute index range over the time ring and every value
|
export.** Both need the same absolute index range over the time ring and every value
|
||||||
ring, or an export and a capture of the same window would hold different samples. Call
|
ring, or an export and a capture of the same window would hold different samples. Call
|
||||||
it under `mutex()`.
|
it under `mutex()`.
|
||||||
|
- **The trigger engine is not fed a reserved hole's samples.** A NaN comparison is false anyway,
|
||||||
|
so the engine would not latch; but the real reason the samples are withheld is the other one:
|
||||||
|
feeding them would advance the hold-off clock across time nothing was sampled in. The probe
|
||||||
|
asserts `trigger_in_hole=0` rather than relying on NaN's arithmetic properties.
|
||||||
- **`EventDetector` owns every condition shape and holds no arming state; `TriggerEngine` owns
|
- **`EventDetector` owns every condition shape and holds no arming state; `TriggerEngine` owns
|
||||||
the arming, the hold-off and the post window and holds no condition.** The split is what lets
|
the arming, the hold-off and the post window and holds no condition.** The split is what lets
|
||||||
the buffer search ask the trigger's exact question without owning a trigger, and it is only
|
the buffer search ask the trigger's exact question without owning a trigger, and it is only
|
||||||
|
|||||||
@@ -586,6 +586,53 @@ Two lines of the self-check output are assertions in disguise:
|
|||||||
it afterwards.
|
it afterwards.
|
||||||
- `[keys] a: armed 0->1 | space: paused 0->1 | t: trigger panel: 0->1 | … | editable focused: 'm' refused=1` (stderr, `UDPSCOPE_PROBE_KEYS_MS`) — **every `X->Y` pair must differ** and **`refused` must be 1**. The refusal is the whole reason the controller is in the bubble phase: every bare-letter accel is also a character, so an accel that fires while a `GtkEditable` has focus makes the window entry unusable. A pair reading `0->0` is an action whose dispatch does not reach the handler a click reaches. Note the format is `<key>: <widget-description>: <before>-><after>` — the name passed to `before_after()` includes the widget's description, so `t: trigger panel: 0->1` is the correct form (not `t: trigger panel 0->1` without the colon before the values).
|
- `[keys] a: armed 0->1 | space: paused 0->1 | t: trigger panel: 0->1 | … | editable focused: 'm' refused=1` (stderr, `UDPSCOPE_PROBE_KEYS_MS`) — **every `X->Y` pair must differ** and **`refused` must be 1**. The refusal is the whole reason the controller is in the bubble phase: every bare-letter accel is also a character, so an accel that fires while a `GtkEditable` has focus makes the window entry unusable. A pair reading `0->0` is an action whose dispatch does not reach the handler a click reaches. Note the format is `<key>: <widget-description>: <before>-><after>` — the name passed to `before_after()` includes the widget's description, so `t: trigger panel: 0->1` is the correct form (not `t: trigger panel 0->1` without the colon before the values).
|
||||||
- `[color] '<name>' at row R slot S: trace=#RRGGBB chip_agrees=1 (want 1)`, then `[color] override: trace=#123456 chip_agrees=1`, `[color] session: entries=1 rgb=#123456` and `[color] default: trace=#RRGGBB chip_agrees=1` (stderr, `UDPSCOPE_PROBE_COLOR_MS`) — **all three `chip_agrees` must be 1**, the override line's `trace` must be `#123456`, the session line must read `entries=1 rgb=#123456`, and the default line's `trace` must be back to the first line's. Run it with `UDPSCOPE_ENABLE_ALL_MS` ahead of it, or there is no bound trace and the probe prints `no bound trace to recolour` instead of asserting anything; on the standard mock the first line reads `'sine' at row 0 slot 0: trace=#4FA3FF`. Two things make this an assertion rather than a tautology. The colours are read out of **`plot::Plot::trace().color`** — what the renderer will stroke, reached through `Row::plot` *after* `relayout_areas()`, since a new colour is a new figure — and out of the chip's own `<span foreground="…">` markup, not by asking `RowSet::signal_color()` twice, which would pass against a chip that never reached the trace. And the override is `#123456`, deliberately in no palette slot: an override equal to the entry it replaces passes against an implementation that ignores overrides entirely. The session leg round-trips `collect_layout()` → `save_layout()` → `load_layout()` on a temp path and deletes it, because a probe must not leave a colour in the real session file for a later run to inherit.
|
- `[color] '<name>' at row R slot S: trace=#RRGGBB chip_agrees=1 (want 1)`, then `[color] override: trace=#123456 chip_agrees=1`, `[color] session: entries=1 rgb=#123456` and `[color] default: trace=#RRGGBB chip_agrees=1` (stderr, `UDPSCOPE_PROBE_COLOR_MS`) — **all three `chip_agrees` must be 1**, the override line's `trace` must be `#123456`, the session line must read `entries=1 rgb=#123456`, and the default line's `trace` must be back to the first line's. Run it with `UDPSCOPE_ENABLE_ALL_MS` ahead of it, or there is no bound trace and the probe prints `no bound trace to recolour` instead of asserting anything; on the standard mock the first line reads `'sine' at row 0 slot 0: trace=#4FA3FF`. Two things make this an assertion rather than a tautology. The colours are read out of **`plot::Plot::trace().color`** — what the renderer will stroke, reached through `Row::plot` *after* `relayout_areas()`, since a new colour is a new figure — and out of the chip's own `<span foreground="…">` markup, not by asking `RowSet::signal_color()` twice, which would pass against a chip that never reached the trace. And the override is `#123456`, deliberately in no palette slot: an override equal to the entry it replaces passes against an implementation that ignores overrides entirely. The session leg round-trips `collect_layout()` → `save_layout()` → `load_layout()` on a temp path and deletes it, because a probe must not leave a colour in the real session file for a later run to inherit.
|
||||||
|
- `[reseq-probe] reseq=N late=N stale=N holes=N hole_slots=N backfilled=N open_holes=N bands=N
|
||||||
|
bands_repaired=N nan_slots=N scanned=N trigger_in_hole=D` (stderr, `UDPSCOPE_PROBE_RESEQ_MS`) —
|
||||||
|
the resequencing path's mid-run state; the measured readings and the per-case command are in
|
||||||
|
[verification.md](verification.md). Six things to read carefully:
|
||||||
|
|
||||||
|
1. **`backfilled` and `hole_slots` are not comparable directly.** `hole_slots` counts
|
||||||
|
*time-ring* slots once per hole; `backfilled` counts *value-ring* slots repaired and
|
||||||
|
therefore sums over every non-math channel. On a stream with 4 non-math channels they
|
||||||
|
stand a factor of 4 apart. `hole_slots=4200 backfilled=16800` in the `late` case is
|
||||||
|
healthy, not an error; `backfilled` below `4 × hole_slots` would mean some channels
|
||||||
|
went unrepaired.
|
||||||
|
|
||||||
|
2. **`nan_slots=0` beside a large `holes` is healthy, not a contradiction.** A hole is
|
||||||
|
retired from the table only when its slots age out of the ring
|
||||||
|
(`ScopeModel::forget_holes_before()`), *never* on repair. So the `late` case reads
|
||||||
|
`holes=42 nan_slots=0`: every hole was fully repaired, and all 42 entries stay in the
|
||||||
|
table with no placeholder left in the data. What *would* be a defect is `nan_slots > 0`
|
||||||
|
with `bands_repaired == bands`, which would mean a band claimed data was recovered while
|
||||||
|
placeholders remained. The discriminator is `bands_repaired`, not `holes`.
|
||||||
|
|
||||||
|
3. **`bands` is the visible spans, `open_holes` the whole table.** `bands` counts what the
|
||||||
|
plot is actually shading in the visible window; `open_holes` counts every live hole entry
|
||||||
|
whether or not it is in view. `bands` below `open_holes` is normal — in the `mixed` case
|
||||||
|
`open_holes=43` against `bands=42` because the 43rd hole has scrolled out of the visible
|
||||||
|
window and `build_invalid_spans()` drops it. `bands` *above* `open_holes` would name a
|
||||||
|
stale span left on a plot after its hole was retired.
|
||||||
|
|
||||||
|
4. **`bands_repaired` is the faint tier.** A hole whose late packets have all arrived is
|
||||||
|
still drawn, in `Style::invalid_repaired_fill` rather than `Style::invalid_fill`, so the
|
||||||
|
display distinguishes "there is no data here" from "there was none when this window was
|
||||||
|
first drawn". The three states are: solid while the hole still lacks packets, faint once
|
||||||
|
`Hole::repaired()` is true, absent once the slots age out and `forget_holes_before()`
|
||||||
|
drops it. `bands_repaired == bands` with `nan_slots=0` is the fully recovered reading;
|
||||||
|
`bands_repaired=0` with a non-zero `backfilled` would mean the repair never reached the
|
||||||
|
display.
|
||||||
|
|
||||||
|
5. **`trigger_in_hole` must be 0, always.** A `1` means the trigger engine was fed a
|
||||||
|
reserved span's samples. The engine is specifically not fed them; a NaN comparison is
|
||||||
|
false anyway, but the real reason is that feeding them would advance the hold-off clock
|
||||||
|
across time nothing was sampled in.
|
||||||
|
|
||||||
|
6. **The discriminator for whether a reservation sized the hole right is the exit dump's
|
||||||
|
per-channel `stored=` line, not `frames=`.** `frames` counts delivered packets and is
|
||||||
|
blind to the slots a reservation pushed on their behalf. `stored` reflects what the ring
|
||||||
|
actually holds, so a reservation that mismeasured the hole shows up there and nowhere
|
||||||
|
else.
|
||||||
|
|
||||||
- `[zoomkeys] ctrl+plus x=[..]->[..] y_held=1 | alt+plus y=[..]->[..] x_held=1` (stderr, `UDPSCOPE_PROBE_ZOOMKEYS_MS`) — **both `_held` must be 1 and both zooms must still happen.** Run it inside a pinned state, where a frame cannot re-frame the other axis for its own reasons: `UDPSCOPE_PROBE_PAUSE_MS=1500 UDPSCOPE_PROBE_ZOOMKEYS_MS=2200`. A `_held=0` is the modifier not reaching the plotter's controller; an unchanged range on the *zoomed* axis is the opposite failure, the key doing nothing. Ctrl restricts to x and Alt to y; Ctrl+Alt zooms neither (no special case needed — neither flag alone fires).
|
- `[zoomkeys] ctrl+plus x=[..]->[..] y_held=1 | alt+plus y=[..]->[..] x_held=1` (stderr, `UDPSCOPE_PROBE_ZOOMKEYS_MS`) — **both `_held` must be 1 and both zooms must still happen.** Run it inside a pinned state, where a frame cannot re-frame the other axis for its own reasons: `UDPSCOPE_PROBE_PAUSE_MS=1500 UDPSCOPE_PROBE_ZOOMKEYS_MS=2200`. A `_held=0` is the modifier not reaching the plotter's controller; an unchanged range on the *zoomed* axis is the opposite failure, the key doing nothing. Ctrl restricts to x and Alt to y; Ctrl+Alt zooms neither (no special case needed — neither flag alone fires).
|
||||||
- `status bar: glance=[● live · 1.09 MS/s · 2 s window] max_glance_chars=53 detail_fields=11 popover_rebuilds=0` (exit dump) — **`max_glance_chars` must stay small** (under about 60) and **`popover_rebuilds` must be 0**. The status row is directly above the plot, so this is the "layout must not depend on content" rule read from the bar's own side: the state field is one word from a fixed set (`connecting`, `live`, `paused`, `armed`, `capturing`, `event held`, `disarmed` — a word added to it must be **no wider than the widest already there**, which is what let `connecting` join at ten characters beside `event held`; the dead-port glance reads `max_glance_chars=51` against a live run's 54), the rate field is `%.3g` (so `10 kS/s`, never `10.0 kS/s`), and the two fields a *state* adds are both elapsed times — trigger mode's `last event … ago` and a pause's `for <elapsed>` — which step units rather than growing digits (`format_elapsed()`). A **paused** glance is not in the dump at all: it is read by `[pause-probe]`'s `glance=` field, since the dump's own line is taken live. `detail_fields` is the popover's label count, a literal, so adding a label without thinking about layout shows up here. The glance rate format matters: `%.3g` drops the trailing zero that a `%.4g` or a fixed-precision format would keep — a rate that reads differently in the dump and in the help page is a format disagreement, and `14-status.txt`'s example must match it.
|
- `status bar: glance=[● live · 1.09 MS/s · 2 s window] max_glance_chars=53 detail_fields=11 popover_rebuilds=0` (exit dump) — **`max_glance_chars` must stay small** (under about 60) and **`popover_rebuilds` must be 0**. The status row is directly above the plot, so this is the "layout must not depend on content" rule read from the bar's own side: the state field is one word from a fixed set (`connecting`, `live`, `paused`, `armed`, `capturing`, `event held`, `disarmed` — a word added to it must be **no wider than the widest already there**, which is what let `connecting` join at ten characters beside `event held`; the dead-port glance reads `max_glance_chars=51` against a live run's 54), the rate field is `%.3g` (so `10 kS/s`, never `10.0 kS/s`), and the two fields a *state* adds are both elapsed times — trigger mode's `last event … ago` and a pause's `for <elapsed>` — which step units rather than growing digits (`format_elapsed()`). A **paused** glance is not in the dump at all: it is read by `[pause-probe]`'s `glance=` field, since the dump's own line is taken live. `detail_fields` is the popover's label count, a literal, so adding a label without thinking about layout shows up here. The glance rate format matters: `%.3g` drops the trailing zero that a `%.4g` or a fixed-precision format would keep — a rate that reads differently in the dump and in the help page is a format disagreement, and `14-status.txt`'s example must match it.
|
||||||
|
|
||||||
|
|||||||
+63
-11
@@ -201,6 +201,59 @@ as distinct from the fault injection below, which exercises ingest:
|
|||||||
- `--quiet-at S` freezes `sine` at its held value from stream second `S` onwards, so it stops
|
- `--quiet-at S` freezes `sine` at its held value from stream second `S` onwards, so it stops
|
||||||
crossing zero: the dropout trigger's input.
|
crossing zero: the dropout trigger's input.
|
||||||
|
|
||||||
|
- `UDPSCOPE_PROBE_RESEQ_MS` — one mid-run reading of the resequencing path. Four cases,
|
||||||
|
**run each as its own process** — four back to back starve the mock past its 2 s silence
|
||||||
|
timeout and every case then reads a reconnect instead of what it was testing. Common
|
||||||
|
conditions for all four: the mock at `--rate 10000 --batch 100 --freq 5`, udpscope with
|
||||||
|
no extra options, under `xvfb-run` with
|
||||||
|
`UDPSCOPE_EXIT_AFTER_MS=8000 UDPSCOPE_ENABLE_ALL_MS=600 UDPSCOPE_EXIT_DUMP=1
|
||||||
|
UDPSCOPE_PROBE_RESEQ_MS=5000`.
|
||||||
|
|
||||||
|
| case | mock extra flags | key reading |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| **base** | *(none)* | every counter 0; proves the machinery costs nothing |
|
||||||
|
| **reorder** | `--reorder-every 10` | `reseq=51` (mid-run) / `102` (dump); `holes=0`; resequencing inside the window leaves no reservation |
|
||||||
|
| **late** | `--reorder-every 10 --reorder-delay-ms 50` | `holes=42 bands=42 bands_repaired=42 nan_slots=0`; every band is the faint, repaired tier |
|
||||||
|
| **mixed** | `--drop-every 20 --reorder-every 10 --reorder-delay-ms 50` | `bands=42 bands_repaired=20 nan_slots=2300`; all three band states at once |
|
||||||
|
|
||||||
|
Full measured readings (probe line first, then exit-dump; `bands_repaired` was added after the
|
||||||
|
original probe and any earlier reading without it is obsolete):
|
||||||
|
|
||||||
|
```
|
||||||
|
=== base mock: none
|
||||||
|
[reseq-probe] reseq=0 late=0 stale=0 holes=0 hole_slots=0 backfilled=0 open_holes=0 bands=0
|
||||||
|
bands_repaired=0 nan_slots=0 scanned=44000 trigger_in_hole=0
|
||||||
|
[udpscope] frames=1024 packets=1025 gaps=0 channels=4 span=838.861s stride=1 reseq=0 late=0
|
||||||
|
stale=0 holes=0 hole_slots=0 backfilled=0
|
||||||
|
[udpscope] channel 0 'sine' stored=95200 published=95000
|
||||||
|
|
||||||
|
=== reorder mock: --reorder-every 10
|
||||||
|
[reseq-probe] reseq=51 late=0 stale=0 holes=0 hole_slots=0 backfilled=0 open_holes=0 bands=0
|
||||||
|
bands_repaired=0 nan_slots=0 scanned=44000 trigger_in_hole=0
|
||||||
|
[udpscope] frames=1028 packets=1029 gaps=0 channels=4 span=838.861s stride=1 reseq=102 late=0
|
||||||
|
stale=0 holes=0 hole_slots=0 backfilled=0
|
||||||
|
[udpscope] channel 0 'sine' stored=95400 published=95100
|
||||||
|
|
||||||
|
=== late mock: --reorder-every 10 --reorder-delay-ms 50
|
||||||
|
[reseq-probe] reseq=142 late=48 stale=0 holes=42 hole_slots=4200 backfilled=16800 open_holes=42
|
||||||
|
bands=42 bands_repaired=42 nan_slots=0 scanned=44000 trigger_in_hole=0
|
||||||
|
[udpscope] frames=1026 packets=1027 gaps=100 channels=4 span=838.861s stride=1 reseq=288
|
||||||
|
late=100 stale=0 holes=94 hole_slots=9400 backfilled=37600
|
||||||
|
[udpscope] channel 0 'sine' stored=95700 published=95500
|
||||||
|
|
||||||
|
=== mixed mock: --drop-every 20 --reorder-every 10 --reorder-delay-ms 50
|
||||||
|
[reseq-probe] reseq=163 late=24 stale=0 holes=43 hole_slots=4300 backfilled=8000 open_holes=43
|
||||||
|
bands=42 bands_repaired=20 nan_slots=2300 scanned=44100 trigger_in_hole=0
|
||||||
|
[udpscope] frames=974 packets=975 gaps=101 channels=4 span=838.861s stride=1 reseq=314 late=50
|
||||||
|
stale=0 holes=94 hole_slots=9400 backfilled=18400
|
||||||
|
[udpscope] channel 0 'sine' stored=95100 published=94900
|
||||||
|
```
|
||||||
|
|
||||||
|
The `mixed` case's `open_holes=43` against `bands=42` is correct: the 43rd hole has scrolled
|
||||||
|
out of the visible window, and `build_invalid_spans()` drops a span outside the window rather
|
||||||
|
than pinning it to the edge. The per-field reference is in
|
||||||
|
[probe-readings.md](probe-readings.md).
|
||||||
|
|
||||||
The mock also carries three **fault injections** that no probe schedules, because what they exercise
|
The mock also carries three **fault injections** that no probe schedules, because what they exercise
|
||||||
is the ingest rather than the UI. The first: `--time-glitch-at <stream seconds> --time-glitch-by <seconds>` makes
|
is the ingest rather than the UI. The first: `--time-glitch-at <stream seconds> --time-glitch-by <seconds>` makes
|
||||||
**one** packet's time signal jump ahead and then carries on from where the true timeline was, which
|
**one** packet's time signal jump ahead and then carries on from where the true timeline was, which
|
||||||
@@ -229,12 +282,12 @@ session for that port; the figures above are the default 2 s window (50 % pre),
|
|||||||
`~/.config/udpscope/<host>_<port>.toml` aside to reproduce them.
|
`~/.config/udpscope/<host>_<port>.toml` aside to reproduce them.
|
||||||
|
|
||||||
The other two are **packet loss** (`--drop-every N`, `--drop-burst K`) and **wire inversion**
|
The other two are **packet loss** (`--drop-every N`, `--drop-burst K`) and **wire inversion**
|
||||||
(`--reorder-every N`). Both withhold a datagram the counter and the stream clock have already
|
(`--reorder-every N`). Loss withholds a datagram the counter and the stream clock have already
|
||||||
advanced over, which is what makes them loss and reordering rather than a slower producer: the
|
advanced over; the client counts the forward jump as `counter_gaps`. An inversion withholds a
|
||||||
client orders the stream by the header's counter alone (`udps_client.c:504`), counting a forward
|
packet until the one behind it is sent, so the receiver sees a later counter first; a packet that
|
||||||
jump as `counter_gaps` and rejecting anything that does not advance it as `stale_packets`. There is
|
arrives within the 20 ms reorder window is resequenced with no hole, and one that arrives past it
|
||||||
**no resequencing and no interpolation** anywhere below the ring — an inverted packet's samples are
|
leaves a committed hole that it then backfills (`UDPSCOPE_PROBE_RESEQ_MS` exercises both paths).
|
||||||
discarded and the hole they leave is a jump in x, which is the behaviour these options are for.
|
There is **no interpolation** anywhere below the ring.
|
||||||
|
|
||||||
All of the readings below are off `UDPSCOPE_EXIT_DUMP`'s `[udpscope] frames= packets= gaps=` line,
|
All of the readings below are off `UDPSCOPE_EXIT_DUMP`'s `[udpscope] frames= packets= gaps=` line,
|
||||||
over an 8 s run against `--rate 10000 --batch 100 --freq 5` (100 packets a second) except where the
|
over an 8 s run against `--rate 10000 --batch 100 --freq 5` (100 packets a second) except where the
|
||||||
@@ -247,11 +300,10 @@ including the clean one, then read a reconnect instead of what it was testing:
|
|||||||
- `--drop-every 10`: `frames=925 packets=926 gaps=102`. Every tenth packet never reaches the wire,
|
- `--drop-every 10`: `frames=925 packets=926 gaps=102`. Every tenth packet never reaches the wire,
|
||||||
so `packets` falls with `frames` and `gaps` is the count of them. No warning: a one-packet hole is
|
so `packets` falls with `frames` and `gaps` is the count of them. No warning: a one-packet hole is
|
||||||
a one-packet jump in the stamps, far inside the glitch guard.
|
a one-packet jump in the stamps, far inside the glitch guard.
|
||||||
- `--reorder-every 10`: `frames=925 packets=1028 gaps=102`. This is the signature to look for —
|
- `--reorder-every 10`: `frames=1028 packets=1029 gaps=0`. The inverted packet lands inside the
|
||||||
`packets` is the clean figure (every datagram arrived) while `frames` is the dropped one, because
|
20 ms reorder window and is resequenced: every datagram arrives and `gaps` is 0. The pre-resequencing
|
||||||
the overtaken packet is decoded far enough to be read as stale and then thrown away. One inversion
|
signature was `frames=925 packets=1028 gaps=102`; that is now the reading for
|
||||||
costs one `gaps` (for the packet that overtook it) **and** one `stale_packets`, which udpscope does
|
`--reorder-every 10 --reorder-delay-ms 50`, which forces a late arrival past the window.
|
||||||
not currently display; the ~104 missing frames are the only place it shows.
|
|
||||||
- `--drop-every 1000 --drop-burst 200` at `--rate 100000 --batch 100` (1000 packets a second):
|
- `--drop-every 1000 --drop-burst 200` at `--rate 100000 --batch 100` (1000 packets a second):
|
||||||
`frames=7000 packets=7001 gaps=1600`, eight bursts of 200, and still no warning.
|
`frames=7000 packets=7001 gaps=1600`, eight bursts of 200, and still no warning.
|
||||||
- `--drop-every 1000 --drop-burst 300`, same stream: `frames=6203 packets=6204 gaps=2400` **and the
|
- `--drop-every 1000 --drop-burst 300`, same stream: `frames=6203 packets=6204 gaps=2400` **and the
|
||||||
|
|||||||
@@ -140,12 +140,18 @@ cannot be misdiagnosed.
|
|||||||
`todo.md` requires absolute stream time to remain available for export. There is exactly one f64
|
`todo.md` requires absolute stream time to remain available for export. There is exactly one f64
|
||||||
time ring per `BufferStore`, shared by every channel of that buffer, and relative mode takes it.
|
time ring per `BufferStore`, shared by every channel of that buffer, and relative mode takes it.
|
||||||
Export therefore needs a **second** f64 ring carrying the absolute stamp under the same group-mean
|
Export therefore needs a **second** f64 ring carrying the absolute stamp under the same group-mean
|
||||||
rule, written in relative mode only and read by export and by nothing else — eight bytes per
|
rule, written in **relative mode only** and read by export and by nothing else — eight bytes per
|
||||||
stored slot.
|
stored slot. The absolute ring is not kept unconditionally; it is budgeted along with the main
|
||||||
|
ring and the value rings, which is why it halves the retained span rather than costing a
|
||||||
|
proportional fraction (see SPECS.md for the measured figures and the power-of-two explanation).
|
||||||
|
|
||||||
This cost is paid rather than avoided. Without it, selecting relative mode silently degrades the
|
This cost is paid rather than avoided. Without it, selecting relative mode silently degrades the
|
||||||
one output `todo.md` says must keep absolute time.
|
one output `todo.md` says must keep absolute time.
|
||||||
|
|
||||||
|
The measured history cost: **83.886 s** in stream mode and **41.943 s** in relative mode, at
|
||||||
|
100 kS/s, 4 channels, `--buffer-mb 200`. The plan's ~22% figure and the per-byte 75% figure are
|
||||||
|
both wrong because `Partition::time_capacity` is floored to a power of two.
|
||||||
|
|
||||||
## The receive path
|
## The receive path
|
||||||
|
|
||||||
### In the client: a reorder ring of reassembled payloads
|
### In the client: a reorder ring of reassembled payloads
|
||||||
@@ -226,9 +232,11 @@ thread folds what was pushed into the pyramid. That split carries straight over
|
|||||||
`[produced_ - capacity, produced_)`, a single relaxed store. The same race class as a push, and
|
`[produced_ - capacity, produced_)`, a single relaxed store. The same race class as a push, and
|
||||||
an `f32` cannot tear. Measurements and export read the ring directly, so they see a repair
|
an `f32` cannot tear. Measurements and export read the ring directly, so they see a repair
|
||||||
immediately.
|
immediately.
|
||||||
- **Pyramid amend — main thread, inside `acquire()`.** The producer enqueues `(slot, value)` on an
|
- **Pyramid amend — main thread, inside `acquire()`.** The producer appends amended `(slot, value)`
|
||||||
SPSC queue; `ChannelBuffer::acquire()` drains it **after** its append pass, so every amended
|
pairs to a plain `std::vector<Amend>` member; `ChannelBuffer::acquire()` drains it **after** its
|
||||||
index is already published.
|
append pass, so every amended index is already published. No SPSC queue is needed: `acquire()` is
|
||||||
|
called under `model.mutex()` and the producer holds that same mutex for all of `on_data`, so the
|
||||||
|
vector is correctly serialised without lock-free machinery.
|
||||||
|
|
||||||
The thread split is not a stylistic choice. A bucket is 32 bytes and is not atomic, so amending it
|
The thread split is not a stylistic choice. A bucket is 32 bytes and is not atomic, so amending it
|
||||||
from the producer would let a reader see a half-updated bucket whose `sum` does not match its
|
from the producer would let a reader see a half-updated bucket whose `sum` does not match its
|
||||||
|
|||||||
@@ -2,13 +2,14 @@
|
|||||||
1. the antialisaing strategy with phosphore effect should be applied with more stregth
|
1. the antialisaing strategy with phosphore effect should be applied with more stregth
|
||||||
2. the lines should be more tick
|
2. the lines should be more tick
|
||||||
3. the sample points should be bigger
|
3. the sample points should be bigger
|
||||||
2. relative timing:
|
~~2. relative timing:
|
||||||
- in order to avoid problem with time jitter user can decide to go in relative time mode
|
- in order to avoid problem with time jitter user can decide to go in relative time mode
|
||||||
- in this mode absolute signal time is only use for export
|
- in this mode absolute signal time is only use for export
|
||||||
- the plot time depends just on signal sample rate and frame number, e.g.:
|
- the plot time depends just on signal sample rate and frame number, e.g.:
|
||||||
- user is acquiring a signal with 1MSps and 100_000 samples per udp frame
|
- user is acquiring a signal with 1MSps and 100_000 samples per udp frame
|
||||||
- last frame as number <N> and previous <N-2> in this case a 100_000 indexes must be skipped
|
- last frame as number <N> and previous <N-2> in this case a 100_000 indexes must be skipped
|
||||||
- time is simply: `((<N_frame> - <N_last>) * <n_samples_per_frame> - <sample_index>) / <sample rate>` (or something like that)
|
- time is simply: `((<N_frame> - <N_last>) * <n_samples_per_frame> - <sample_index>) / <sample rate>` (or something like that)~~
|
||||||
|
done: `--relative-time`; see README and SPECS.md for the measured history cost.
|
||||||
3. named configurations: in order to have multiples configurations for the same sources configuration can also have comments
|
3. named configurations: in order to have multiples configurations for the same sources configuration can also have comments
|
||||||
4. optional udp frame storage for debugging of strange behaviours (store last <N>, default = 1000, udp packets binary content on disk for forensic analyss)
|
4. optional udp frame storage for debugging of strange behaviours (store last <N>, default = 1000, udp packets binary content on disk for forensic analyss)
|
||||||
5. open exported binary waveforms to view/analyse, in the source selection dialog there should be the option to open exported waveform (maybe a tab with latest opened file and option to open new ones?)
|
5. open exported binary waveforms to view/analyse, in the source selection dialog there should be the option to open exported waveform (maybe a tab with latest opened file and option to open new ones?)
|
||||||
|
|||||||
@@ -271,10 +271,11 @@ def main():
|
|||||||
type=int,
|
type=int,
|
||||||
default=0,
|
default=0,
|
||||||
help="hold every Nth packet back and send it after the next one that goes out -- "
|
help="hold every Nth packet back and send it after the next one that goes out -- "
|
||||||
"an inversion on the wire. The receiver takes the later counter first, so the held "
|
"an inversion on the wire. What the receiver does depends on how late the held "
|
||||||
"packet no longer advances the counter and is stale by definition: one inversion "
|
"packet arrives: inside the 20 ms reorder window it is resequenced with no hole; "
|
||||||
"costs one counter gap, one stale packet, and that packet's samples for good. A "
|
"past it a hole is committed first and the packet backfills it. "
|
||||||
"--drop-every that swallows the packet after a held one deepens the inversion "
|
"--reorder-delay-ms controls which path is taken (see its help). "
|
||||||
|
"A --drop-every that swallows the packet after a held one deepens the inversion "
|
||||||
"rather than cancelling it, since the held packet waits for the next packet "
|
"rather than cancelling it, since the held packet waits for the next packet "
|
||||||
"actually sent. 0 = never",
|
"actually sent. 0 = never",
|
||||||
)
|
)
|
||||||
|
|||||||
Reference in new issue
Block a user