Frame cost was O(capacity x levels) once the ring saturated: acquire() saw the window base move (it always moves once the ring wraps), cleared the pyramid and re-appended all 1,048,576 retained samples, and append() walked every live level with no early stop -- 13,647,936 bucket merges and 73.8 ms per frame at a 1M-sample window, scaling with the window, which is what capped the 1 MS/s demo at 32-58 ms/frame while it was still filling. Index the pyramid in absolute bucket space instead. Level l bucket k covers the samples [k * (kBucket << l), (k + 1) * (kBucket << l)), so a bucket's span never depends on where the window starts and the window can slide over the aggregate instead of forcing it to be rebuilt. Each level keeps its complete buckets in a deque plus the absolute base of its first entry; append() folds a sample into the single in-progress level-0 bucket and, only when that bucket completes, cascades upward, stopping at the first level that gains no new entry; a new drop_before() slides the window by discarding leading buckets and remembers where the window begins. acquire() is now: drop to the new base, then append exactly what was produced -- 256 merges and 0.034 ms per frame for 8192 new samples, independent of the window, and memory bounded by the window (32,767 entries after 50M samples through a 1M ring, the same as after 10M). A producer that laps the consumer by more than capacity() between two acquires restarts the window rather than splicing a gap. Interface: `level(l) -> std::span<const MinMax>` cannot survive a ring and is replaced by level_base/level_size/bucket in absolute bucket coordinates, plus window_base/window_end/drop_before. The *_buckets queries stay in window coordinates (index 0 = oldest retained sample), so Series keeps its window semantics for callers; its exactness split now aligns in absolute space (window_base() + the ceil/floor) because the bucket grid is absolute. The sums move from per-level prefix sums to per-bucket ones -- the only representation a sliding front can keep exact -- and StreamSeries::y_bounds() becomes reduce(0, size()), exact in O(log n + kBucket) per frame instead of a scan of the window. The pyramid owns the published window now, so there is no second copy of base/end to keep in step, and gather() emits only buckets that are both complete and wholly retained. Tests: test-only merge counters (mirroring PLOT_PYRAMID_COUNT_COVERS, absent from every non-test build) pin an acquire proportional to production and independent of capacity, an append cost amortised over 16 live levels, and bounded memory; brute-force checks pin absolute bucket aggregates across slides, sliding-window reduce/sum/sumsq over unaligned ranges, and the level-1 gather mapping. The demo now draws 946 samples/px at 1.03 ms/frame (967 fps) where the saturated window cost 54.96 ms/frame (18 fps).
140 lines
6.4 KiB
C++
140 lines
6.4 KiB
C++
#pragma once
|
|
#include <deque>
|
|
#include <vector>
|
|
|
|
#include <plot/core/types.hpp>
|
|
|
|
namespace plot::data {
|
|
|
|
/// Aggregate of one bucket: the extremes of the samples it covers.
|
|
struct MinMax {
|
|
f32 min = 0.0f;
|
|
f32 max = 0.0f;
|
|
};
|
|
|
|
/// Samples per level-0 bucket. Fixed so the LOD geometry is identical for every
|
|
/// series and the level maths is a shift.
|
|
inline constexpr u32 kBucket = 64;
|
|
|
|
/// Incremental min/max LOD pyramid, indexed in absolute bucket space.
|
|
///
|
|
/// Level `l` bucket `k` covers the absolute samples
|
|
/// `[k * (kBucket << l), (k + 1) * (kBucket << l))`. Bucket boundaries are
|
|
/// therefore fixed for the pyramid's whole life: appending never re-buckets an
|
|
/// existing aggregate, and the window a caller reads can slide over the same
|
|
/// buckets instead of being rebuilt from the raw samples.
|
|
///
|
|
/// Each level holds its complete buckets in a deque (O(1) push_back *and*
|
|
/// pop_front) plus a `base`, the absolute level-`l` bucket index of its first
|
|
/// entry. `append()` folds a sample into the in-progress level-0 bucket and, when
|
|
/// that bucket completes, pushes it and walks up -- stopping at the first level
|
|
/// that gains no new entry. Level `l` gains one entry per `kBucket << l` samples,
|
|
/// so an append costs amortised O(1) and never walks every live level.
|
|
///
|
|
/// `drop_before(u)` slides the window's leading edge to absolute sample `u`: each
|
|
/// level discards the buckets whose span starts before `u`, so the memory is
|
|
/// bounded by the retained window (about n/4 bytes, ~6% of 4-byte raw samples)
|
|
/// and not by the number of samples ever appended. Every stored entry is the
|
|
/// complete aggregate of its own span and every stored span lies inside the
|
|
/// retained window, so no aggregate is ever recomputed or invalidated.
|
|
///
|
|
/// Queries are in *window* coordinates: index 0 is the oldest retained sample,
|
|
/// `size()` is the retained sample count, and `window_base()` is the absolute
|
|
/// index window index 0 refers to. That is the same window `Series` publishes,
|
|
/// which is why a `StreamSeries` can hand the pyramid a moving base and keep
|
|
/// answering in its own coordinates.
|
|
///
|
|
/// A pyramid stores *aggregates*, so it can only answer exactly for whole
|
|
/// buckets; a range with partial edge buckets would silently pull in samples
|
|
/// outside the range. Every query here is therefore bucket-grained and named
|
|
/// *_buckets, and exact range queries live on Series, which owns the raw samples.
|
|
class MinMaxPyramid {
|
|
public:
|
|
MinMaxPyramid() = default;
|
|
|
|
/// Sizes the level table for a window of `n` samples so a following run of
|
|
/// appends never grows the level vector. The bucket deques themselves grow in
|
|
/// chunks: a sliding window cannot know their final length up front.
|
|
void reserve(u64 n);
|
|
void append(f32 v);
|
|
/// Slides the retained window's leading edge to absolute sample index
|
|
/// `absolute_sample`, dropping every aggregate that covered a sample before it.
|
|
/// Monotonic for arguments ahead of the base; a no-op for arguments at or
|
|
/// behind it, and for an empty pyramid.
|
|
///
|
|
/// An argument past `window_end()` (a producer that outran the consumer by more
|
|
/// than the ring held) drops everything and restarts the window there: the
|
|
/// samples in between were never appended, so they are not representable.
|
|
void drop_before(u64 absolute_sample);
|
|
void clear();
|
|
|
|
/// Samples in the retained window.
|
|
u64 size() const noexcept { return count_ - base_; }
|
|
/// Absolute index of the sample window index 0 refers to.
|
|
u64 window_base() const noexcept { return base_; }
|
|
/// Absolute index one past the last appended sample: the first sample index a
|
|
/// following append() will occupy.
|
|
u64 window_end() const noexcept { return count_; }
|
|
/// At least 1, even when empty, so level_count() - 1 is always valid.
|
|
u32 level_count() const noexcept { return live_levels_; }
|
|
|
|
/// Absolute level-`l` bucket index of level `l`'s first entry, and how many
|
|
/// entries it holds. They cover `[level_base(l), level_base(l) + level_size(l))`:
|
|
/// entry `i` is the aggregate of the absolute samples
|
|
/// `[i * (kBucket << l), (i + 1) * (kBucket << l))`. A level that exists but
|
|
/// holds nothing, or a level past level_count(), reports size 0 (its base is
|
|
/// then meaningless).
|
|
u64 level_base(u32 l) const noexcept;
|
|
u64 level_size(u32 l) const noexcept;
|
|
/// Aggregate of the absolute level-`l` bucket `i`. Precondition:
|
|
/// level_base(l) <= i < level_base(l) + level_size(l).
|
|
MinMax bucket(u32 l, u64 i) const;
|
|
|
|
/// Min/max over the whole level-0 buckets fully contained in the window range
|
|
/// [begin, end). Partial edge buckets are excluded, never approximated; an
|
|
/// empty result is a degenerate Range with span() == 0.
|
|
Range reduce_buckets(u64 begin, u64 end) const;
|
|
/// Sum of the samples in those same whole buckets, accumulated in double.
|
|
f64 sum_buckets(u64 begin, u64 end) const;
|
|
/// Sum of squares of the samples in those same whole buckets.
|
|
f64 sumsq_buckets(u64 begin, u64 end) const;
|
|
/// Pyramid level whose buckets hold about `samples_per_pixel` samples, for a
|
|
/// window of `n` samples: 0 when a pixel covers at most one bucket. Clamped to
|
|
/// the levels that exist, to levels with more than one bucket, and to levels
|
|
/// that give the window at least two buckets.
|
|
u32 level_for(u64 n, f64 samples_per_pixel) const;
|
|
/// Index of the sample nearest x on the uniform grid (x0 + i*dx), clamped to
|
|
/// [0, size() - 1]. Returns 0 for an empty pyramid or a non-positive dx.
|
|
u64 snap(f64 x, f64 x0, f64 dx) const;
|
|
|
|
private:
|
|
/// One bucket: its extremes plus the raw-sum aggregates Series folds over.
|
|
struct Entry {
|
|
MinMax mm;
|
|
f64 sum = 0.0;
|
|
f64 sumsq = 0.0;
|
|
};
|
|
/// One level's complete buckets, in absolute bucket-index order.
|
|
struct Level {
|
|
std::deque<Entry> entries;
|
|
u64 base = 0; ///< absolute level-l bucket index of entries.front()
|
|
};
|
|
|
|
void add_level();
|
|
/// Appends `e` at absolute level-`l` bucket `k`, which must be one past the
|
|
/// level's last entry.
|
|
void push(u32 l, u64 k, const Entry& e);
|
|
/// Entry for absolute level-`l` bucket `i`; the caller has checked the range.
|
|
const Entry& at(u32 l, u64 i) const {
|
|
return levels_[l].entries[static_cast<usize>(i - levels_[l].base)];
|
|
}
|
|
|
|
u64 count_ = 0; ///< absolute index one past the last appended sample
|
|
u64 base_ = 0; ///< absolute index of the window's oldest sample
|
|
u32 live_levels_ = 1;
|
|
std::vector<Level> levels_{1};
|
|
Entry pending_; ///< the in-progress level-0 bucket, not yet an entry
|
|
};
|
|
|
|
} // namespace plot::data
|