Findings from the review of the sliding-window change:
- A level >= 1 gather emitted only complete, wholly-retained buckets, so the
geometry stopped at the last bucket boundary inside the range: the newest
samples of a live window -- up to `span - 1` of them, about a pixel at the
level level_for() picks -- were not drawn, on the right edge, which is the
part of a scrolling trace that gets watched, and the oldest retained samples
fell outside the geometry too (the level-1 gather of a sliding window started
51 samples in). The two partial edge buckets are now gathered as raw samples
through copy_raw() and bracketed around the envelope -- the same exactness
split reduce/sum/sumsq already use -- so the emitted geometry spans the
requested range exactly. The pyramid still stores no partial bucket, so append
stays amortised O(1); at 1 MS/s the demo stages 511 extra sub-pixel segments
and still draws at 1.05 ms/frame, and the screenshot's trace now reaches one
column further at each end.
- `drop_before`'s doc claimed a no-op "for an empty pyramid": false, and
backwards -- the restart branch is exactly what an empty pyramid hits, which is
what StreamSeries::acquire() depends on when the ring saturated before the
first acquire. The precondition is `absolute_sample <= window_base()`, and the
restart is documented as what it is.
- The pyramid's memory figure was 3x optimistic: an entry is 24 bytes (MinMax
plus the exact sum/sumsq doubles), so level 0 costs 0.375n bytes and the whole
pyramid ~0.75n -- about 19% of 4-byte raw samples, not the 6% a min/max-only
index would cost. Corrected in the header, spec 4.2 and 4.5, with the trade
stated (the doubles buy O(log n) sum/mean/RMS/integral from whole buckets),
and the plan's Task 3 sketch no longer describes per-level prefix sums, a
level() span or an unaligned-end widening that does not exist.
Tests: `gather_at_a_lod_level_reaches_both_ends_of_the_window` pins that the
first and last emitted points are the oldest and newest retained samples -- with
spike values no bucket envelope can produce -- at level 1, at the level a dense
view picks, and for a static series whose end is unaligned. It fails against the
previous commit (first point at the first whole bucket), and so does the sliding
gather mapping check, which now accounts for the raw edge pairs.
Frame cost was O(capacity x levels) once the ring saturated: acquire() saw the
window base move (it always moves once the ring wraps), cleared the pyramid and
re-appended all 1,048,576 retained samples, and append() walked every live level
with no early stop -- 13,647,936 bucket merges and 73.8 ms per frame at a
1M-sample window, scaling with the window, which is what capped the 1 MS/s demo
at 32-58 ms/frame while it was still filling.
Index the pyramid in absolute bucket space instead. Level l bucket k covers the
samples [k * (kBucket << l), (k + 1) * (kBucket << l)), so a bucket's span never
depends on where the window starts and the window can slide over the aggregate
instead of forcing it to be rebuilt. Each level keeps its complete buckets in a
deque plus the absolute base of its first entry; append() folds a sample into the
single in-progress level-0 bucket and, only when that bucket completes, cascades
upward, stopping at the first level that gains no new entry; a new drop_before()
slides the window by discarding leading buckets and remembers where the window
begins. acquire() is now: drop to the new base, then append exactly what was
produced -- 256 merges and 0.034 ms per frame for 8192 new samples, independent
of the window, and memory bounded by the window (32,767 entries after 50M samples
through a 1M ring, the same as after 10M). A producer that laps the consumer by
more than capacity() between two acquires restarts the window rather than
splicing a gap.
Interface: `level(l) -> std::span<const MinMax>` cannot survive a ring and is
replaced by level_base/level_size/bucket in absolute bucket coordinates, plus
window_base/window_end/drop_before. The *_buckets queries stay in window
coordinates (index 0 = oldest retained sample), so Series keeps its window
semantics for callers; its exactness split now aligns in absolute space
(window_base() + the ceil/floor) because the bucket grid is absolute. The sums
move from per-level prefix sums to per-bucket ones -- the only representation a
sliding front can keep exact -- and StreamSeries::y_bounds() becomes
reduce(0, size()), exact in O(log n + kBucket) per frame instead of a scan of the
window. The pyramid owns the published window now, so there is no second copy of
base/end to keep in step, and gather() emits only buckets that are both complete
and wholly retained.
Tests: test-only merge counters (mirroring PLOT_PYRAMID_COUNT_COVERS, absent from
every non-test build) pin an acquire proportional to production and independent
of capacity, an append cost amortised over 16 live levels, and bounded memory;
brute-force checks pin absolute bucket aggregates across slides, sliding-window
reduce/sum/sumsq over unaligned ranges, and the level-1 gather mapping. The demo
now draws 946 samples/px at 1.03 ms/frame (967 fps) where the saturated window
cost 54.96 ms/frame (18 fps).
The completion check was scheduling-dependent and the test could lose its TSan
overlap window. The producer now signals done with a release store, the consumer
reads in a do/while keyed on that flag so at least one read is guaranteed by
construction, and a started/reading handshake makes the producer's remaining
pushes provably overlap the reader's loop. Bounded to 4096 pushes on the 256-slot
ring, and the assertions moved after join() so a failure reports instead of
terminating. With the pre-fix ring restored, four TSan runs still flag push vs
copy_from, so the case keeps its teeth.
The pyramid's public query surface is the bucket-grained trio
(reduce_buckets/sum_buckets/sumsq_buckets); reduce() was the one
non-*_buckets query and silently over-covered non-aligned ranges, which
is exactly the misuse the exactness split exists to prevent. Exact range
queries stay on Series.
- delete the declaration, its doc paragraph and the definition; no
helper became dead (fold_range/cover_buckets/ceil_to_bucket are all
still used by the *_buckets queries)
- pyramid_clear_and_rebuild: complete bucket 0 and assert its extrema via
reduce_buckets(0, kBucket) -- a pyramid has no exact sub-bucket query
- stream_series_acquire_publishes_snapshot: s.pyramid().reduce(0, 100)
-> s.reduce(0, 100), keeping the == 99.0 assertion
cover_buckets chose one level per pass, so an odd first whole bucket (half of all
measurement ranges) never reached a level above 0 and folded every remaining
bucket at level 0 -- 15624 entries for a 1e6-sample pyramid. Pick the largest
aligned, in-range, materialised level at each cursor instead; the new cost test
counts folded entries through a PLOT_PYRAMID_COUNT_COVERS-only hook (absent from
normal builds) and fails on the old body.
Ring slots are now relaxed std::atomic: overwrite-on-full means the producer
reuses the slot of a snapshot's oldest sample while the consumer reads it, which
was a data race, not just a stale value. copy_from loads explicitly instead of
std::copy; push stays lock-free. A new producer/consumer case is the one TSan
flags without this.
Also extends the wrapped-window test with x0/bounds/pyramid/sum/sumsq/gather
checks, and corrects the StreamSeries threading note to the sample-count bound.
Martino Ferrari