perf(data): absolute-space pyramid so acquire() is O(produced), not O(window)

Frame cost was O(capacity x levels) once the ring saturated: acquire() saw the
window base move (it always moves once the ring wraps), cleared the pyramid and
re-appended all 1,048,576 retained samples, and append() walked every live level
with no early stop -- 13,647,936 bucket merges and 73.8 ms per frame at a
1M-sample window, scaling with the window, which is what capped the 1 MS/s demo
at 32-58 ms/frame while it was still filling.

Index the pyramid in absolute bucket space instead. Level l bucket k covers the
samples [k * (kBucket << l), (k + 1) * (kBucket << l)), so a bucket's span never
depends on where the window starts and the window can slide over the aggregate
instead of forcing it to be rebuilt. Each level keeps its complete buckets in a
deque plus the absolute base of its first entry; append() folds a sample into the
single in-progress level-0 bucket and, only when that bucket completes, cascades
upward, stopping at the first level that gains no new entry; a new drop_before()
slides the window by discarding leading buckets and remembers where the window
begins. acquire() is now: drop to the new base, then append exactly what was
produced -- 256 merges and 0.034 ms per frame for 8192 new samples, independent
of the window, and memory bounded by the window (32,767 entries after 50M samples
through a 1M ring, the same as after 10M). A producer that laps the consumer by
more than capacity() between two acquires restarts the window rather than
splicing a gap.

Interface: `level(l) -> std::span<const MinMax>` cannot survive a ring and is
replaced by level_base/level_size/bucket in absolute bucket coordinates, plus
window_base/window_end/drop_before. The *_buckets queries stay in window
coordinates (index 0 = oldest retained sample), so Series keeps its window
semantics for callers; its exactness split now aligns in absolute space
(window_base() + the ceil/floor) because the bucket grid is absolute. The sums
move from per-level prefix sums to per-bucket ones -- the only representation a
sliding front can keep exact -- and StreamSeries::y_bounds() becomes
reduce(0, size()), exact in O(log n + kBucket) per frame instead of a scan of the
window. The pyramid owns the published window now, so there is no second copy of
base/end to keep in step, and gather() emits only buckets that are both complete
and wholly retained.

Tests: test-only merge counters (mirroring PLOT_PYRAMID_COUNT_COVERS, absent from
every non-test build) pin an acquire proportional to production and independent
of capacity, an append cost amortised over 16 live levels, and bounded memory;
brute-force checks pin absolute bucket aggregates across slides, sliding-window
reduce/sum/sumsq over unaligned ranges, and the level-1 gather mapping. The demo
now draws 946 samples/px at 1.03 ms/frame (967 fps) where the saturated window
cost 54.96 ms/frame (18 fps).
This commit is contained in:
Martino Ferrari committed 2026-09-28 00:05:15 +02:00
1 parent 4b649a2783
commit 0ac5d47fa6
13 files changed
+769 -244

No files matched your search

+149 -114
View File
@@ -1,8 +1,10 @@
#include <plot/data/detail/cover_count.hpp>
#include <plot/data/detail/merge_count.hpp>
#include <plot/data/minmax_pyramid.hpp>
#include <algorithm>
#include <cmath>
#include <cstddef>
namespace plot::data {
namespace {
@@ -16,18 +18,19 @@ constexpr u64 buckets_for(u64 n, u32 l) {
return n / span + (n % span == 0 ? u64{0} : u64{1});
}
/// Ceiling division: first level-l bucket whose span starts at or after v.
constexpr u64 ceil_div(u64 v, u64 d) { return v / d + (v % d == 0 ? u64{0} : u64{1}); }
/// First level-0 bucket boundary at or after v.
constexpr u64 ceil_to_bucket(u64 v) {
return (v / kBucket + (v % kBucket == 0 ? u64{0} : u64{1})) * kBucket;
}
constexpr u64 ceil_to_bucket(u64 v) { return ceil_div(v, kBucket) * kBucket; }
/// Canonical, coarsest-first decomposition of the level-0 bucket span [b0, b1)
/// into pyramid entries, calling fn(level, index) for each.
/// into pyramid entries, calling fn(level, absolute_bucket_index) for each.
///
/// A level-l entry is emitted only when the whole level-l bucket exists (all
/// kBucket << l samples are present, so its aggregate is final). Callers
/// guarantee b1 <= ceil(size() / kBucket), so the only possible remainder is the
/// single trailing partial level-0 bucket, which is emitted at level 0.
/// guarantee b0 >= the first whole bucket of the window and b1 <= the number of
/// complete buckets, so every emission both exists and lies inside the window.
///
/// The level is chosen per *emission*, not once per pass: at each cursor the
/// largest level is taken whose bucket is aligned with the cursor, lies wholly
@@ -40,16 +43,18 @@ constexpr u64 ceil_to_bucket(u64 v) {
/// no coarser level is ever reachable from an odd cursor.
template <typename Fn>
void cover_buckets(const MinMaxPyramid& p, u64 b0, u64 b1, Fn&& fn) {
const u64 complete = p.size() / kBucket;
const u64 limit = std::min(b1, complete);
// One past the last complete level-0 bucket: levels sit on top of those, so
// this also bounds every coarser bucket the cursor may take.
const u64 limit = std::min(b1, p.level_base(0) + p.level_size(0));
const u32 top = p.level_count() - 1; // level_count() is at least 1
u64 b = b0;
while (b < b1) {
u32 li = top;
for (;;) {
const u64 lb = u64{1} << li;
if ((b % lb) == 0 && b + lb <= limit &&
(b / lb) < static_cast<u64>(p.level(li).size())) {
const u64 i = b >> li;
if ((b % lb) == 0 && b + lb <= limit && i >= p.level_base(li) &&
i - p.level_base(li) < p.level_size(li)) {
break;
}
if (li == 0) break;
@@ -69,7 +74,7 @@ Range fold_range(const MinMaxPyramid& p, u64 b0, u64 b1) {
Range r{};
bool first = true;
cover_buckets(p, b0, b1, [&](u32 l, u64 i) {
const MinMax m = p.level(l)[static_cast<usize>(i)];
const MinMax m = p.bucket(l, i);
if (first) {
r.min = f64(m.min);
r.max = f64(m.max);
@@ -84,140 +89,170 @@ Range fold_range(const MinMaxPyramid& p, u64 b0, u64 b1) {
} // namespace
void MinMaxPyramid::add_level() {
levels_.emplace_back();
sums_.emplace_back();
sumsqs_.emplace_back();
}
void MinMaxPyramid::add_level() { levels_.emplace_back(); }
void MinMaxPyramid::reserve(u64 n) {
if (n > reserved_n_) reserved_n_ = n;
const u64 want = std::max(std::max(n, size_), u64{2});
// Materialise every level n samples can need -- level 0 plus one per doubling
// of the bucket -- so a run of appends never grows the level vector. See the
// header for why the bucket deques themselves cannot be pre-sized.
u32 want = 1;
for (u64 span = level_span(0); span <= n && want < 63; span <<= 1) ++want;
if (levels_.size() < want) levels_.resize(want);
}
// Materialise every level n samples can need -- buckets *and* prefix sums --
// and size each one up front, so a run of appends allocates nothing at all.
u32 max_levels = 1;
for (u64 span = level_span(0); span <= want && max_levels < 63; span <<= 1) ++max_levels;
while (levels_.size() < max_levels) add_level();
for (u32 l = 0; l < levels_.size(); ++l) {
const usize want_l = static_cast<usize>(buckets_for(want, l));
levels_[l].reserve(want_l);
sums_[l].reserve(want_l);
sumsqs_[l].reserve(want_l);
}
void MinMaxPyramid::push(u32 l, u64 k, const Entry& e) {
Level& lv = levels_[l];
// The first entry of a level fixes its base; every later entry is the next
// absolute bucket, which is what keeps a level one contiguous run.
if (lv.entries.empty()) lv.base = k;
lv.entries.push_back(e);
#ifdef PLOT_PYRAMID_COUNT_MERGES
detail::merge_count().fetch_add(1, std::memory_order_relaxed);
#endif
}
void MinMaxPyramid::append(f32 v) {
const u64 prev = size_;
const f64 vd = f64(v);
// sum_[l].back() is the sum over every sample appended so far, and the newest
// sample always lands in the last bucket of every level, so the last prefix
// entry of every level is exactly the running total.
const f64 total_before = sums_[0].empty() ? 0.0 : sums_[0].back();
const f64 total = total_before + vd;
const f64 sq_before = sumsqs_[0].empty() ? 0.0 : sumsqs_[0].back();
const f64 total_sq = sq_before + vd * vd;
const u64 idx = prev / kBucket;
if (idx == levels_[0].size()) {
levels_[0].push_back(MinMax{v, v});
sums_[0].push_back(total);
sumsqs_[0].push_back(total_sq);
// The new sample always lands in the in-progress level-0 bucket, which is the
// only bucket any append ever touches: a bucket becomes an entry exactly once,
// complete, and is never rewritten.
if (count_ % kBucket == 0) {
pending_.mm.min = v;
pending_.mm.max = v;
pending_.sum = vd;
pending_.sumsq = vd * vd;
} else {
MinMax& cur = levels_[0][static_cast<usize>(idx)];
cur.min = std::min(cur.min, v);
cur.max = std::max(cur.max, v);
sums_[0][static_cast<usize>(idx)] = total;
sumsqs_[0][static_cast<usize>(idx)] = total_sq;
pending_.mm.min = std::min(pending_.mm.min, v);
pending_.mm.max = std::max(pending_.mm.max, v);
pending_.sum += vd;
pending_.sumsq += vd * vd;
}
++count_;
if (count_ % kBucket != 0) return; // the bucket is still partial: nothing to push
const u64 k0 = count_ / kBucket - 1; // the level-0 bucket that just completed
if (k0 < level_base(0)) return; // its span starts before the window: never read
const Entry done = pending_;
push(0, k0, done);
// Cascade: level l gains an entry exactly when kBucket << l divides the sample
// count, so the walk stops at the first level that gained nothing -- amortised
// O(1) per sample (1/kBucket + 1/(2*kBucket) + ... = 1/(kBucket/2) merges).
for (u32 l = 1; l < 63; ++l) {
const u64 span = level_span(l);
if (count_ % span != 0) break;
const u64 k = count_ / span - 1; // the level-l bucket that just completed
const u64 child = 2 * k; // its left child, at level l-1
if (child < level_base(l - 1)) break; // a child is gone: this parent is not retained
if (l == static_cast<u32>(levels_.size())) add_level();
if (l >= live_levels_) live_levels_ = l + 1;
const Entry& a = at(l - 1, child);
const Entry& b = at(l - 1, child + 1);
Entry m;
m.mm.min = std::min(a.mm.min, b.mm.min);
m.mm.max = std::max(a.mm.max, b.mm.max);
m.sum = a.sum + b.sum;
m.sumsq = a.sumsq + b.sumsq;
push(l, k, m);
}
}
void MinMaxPyramid::drop_before(u64 absolute_sample) {
if (absolute_sample <= base_) return;
if (absolute_sample > count_) {
// The producer produced more than the ring held between two acquires: every
// appended sample is gone and the ones in between were never appended, so the
// pyramid restarts empty at the new base rather than splicing a gap.
base_ = absolute_sample;
count_ = absolute_sample;
pending_ = Entry{};
live_levels_ = 1;
for (u32 l = 0; l < levels_.size(); ++l) {
levels_[l].entries.clear();
levels_[l].base = ceil_div(absolute_sample, level_span(l));
}
return;
}
for (u32 l = 1; l < 63; ++l) {
// A level only carries data once the level below has more than one bucket: a
// one-child parent would just duplicate it.
if (levels_[l - 1].size() < 2) break;
if (l == live_levels_) {
++live_levels_;
if (l == levels_.size()) { // no reserve() sized this level up front
add_level();
if (reserved_n_ != 0) {
const usize want = static_cast<usize>(buckets_for(reserved_n_, l));
levels_[l].reserve(want);
sums_[l].reserve(want);
sumsqs_[l].reserve(want);
}
}
}
const u64 ci = prev / level_span(l - 1); // current bucket at level l-1
const u64 parent = ci / 2; // its parent at level l
MinMax m;
if ((ci & 1u) != 0) {
const MinMax& a = levels_[l - 1][static_cast<usize>(ci - 1)];
const MinMax& b = levels_[l - 1][static_cast<usize>(ci)];
m.min = std::min(a.min, b.min);
m.max = std::max(a.max, b.max);
base_ = absolute_sample;
for (u32 l = 0; l < levels_.size(); ++l) {
Level& lv = levels_[l];
// A bucket is dropped once its span starts before the window: the first
// retained bucket of every level is the first one wholly inside the window,
// which is exactly what the queries assume.
const u64 keep = ceil_div(base_, level_span(l));
if (keep <= lv.base) continue;
const u64 have = static_cast<u64>(lv.entries.size());
const u64 drop = std::min(keep - lv.base, have);
lv.entries.erase(lv.entries.begin(),
lv.entries.begin() + static_cast<std::ptrdiff_t>(drop));
// The base is where the next entry goes: past the last one if the level was
// emptied, which is how the cursor catches up with the sliding window.
if (drop == have) {
lv.base = keep;
} else {
m = levels_[l - 1][static_cast<usize>(ci)];
}
if (parent == levels_[l].size()) {
levels_[l].push_back(m);
sums_[l].push_back(total);
sumsqs_[l].push_back(total_sq);
} else {
levels_[l][static_cast<usize>(parent)] = m;
sums_[l][static_cast<usize>(parent)] = total;
sumsqs_[l][static_cast<usize>(parent)] = total_sq;
lv.base += drop;
}
}
size_ = prev + 1;
// The in-progress level-0 bucket may now start before the window's base: its
// aggregate covers samples that are gone. It belongs to the bucket the window
// starts inside, which is never read (the queries take whole buckets and the
// Series scans the two partial edges raw), so discarding it is lossless.
if (ceil_div(base_, kBucket) > count_ / kBucket) pending_ = Entry{};
while (live_levels_ > 1 && level_size(live_levels_ - 1) == 0) --live_levels_;
}
void MinMaxPyramid::clear() {
size_ = 0;
count_ = 0;
base_ = 0;
live_levels_ = 1;
for (auto& lv : levels_) lv.clear();
for (auto& s : sums_) s.clear();
for (auto& s : sumsqs_) s.clear();
levels_.resize(1);
levels_[0].entries.clear();
levels_[0].base = 0;
pending_ = Entry{};
}
std::span<const MinMax> MinMaxPyramid::level(u32 l) const {
if (l >= level_count()) return {};
return levels_[l];
u64 MinMaxPyramid::level_base(u32 l) const noexcept {
return l < levels_.size() ? levels_[l].base : u64{0};
}
u64 MinMaxPyramid::level_size(u32 l) const noexcept {
return l < levels_.size() ? static_cast<u64>(levels_[l].entries.size()) : u64{0};
}
MinMax MinMaxPyramid::bucket(u32 l, u64 i) const { return at(l, i).mm; }
Range MinMaxPyramid::reduce_buckets(u64 begin, u64 end) const {
const u64 e = std::min(end, size_);
const u64 b = std::min(begin, e);
const u64 b0 = ceil_to_bucket(b) / kBucket; // first whole bucket inside
const u64 b1 = e / kBucket; // one past the last whole bucket
const u64 e = std::min(end, size());
if (begin >= e) return Range{0.0, 0.0};
// The bucket grid is absolute while these are window coordinates: window index
// i is absolute window_base() + i, so the first whole bucket inside the range is
// the first absolute bucket boundary at or after that.
const u64 b0 = ceil_to_bucket(base_ + begin) / kBucket;
const u64 b1 = (base_ + e) / kBucket;
if (b1 <= b0) return Range{0.0, 0.0};
return fold_range(*this, b0, b1);
}
f64 MinMaxPyramid::sum_buckets(u64 begin, u64 end) const {
const u64 e = std::min(end, size_);
const u64 b = std::min(begin, e);
const u64 b0 = ceil_to_bucket(b) / kBucket;
const u64 b1 = e / kBucket;
const u64 e = std::min(end, size());
if (begin >= e) return 0.0;
const u64 b0 = ceil_to_bucket(base_ + begin) / kBucket;
const u64 b1 = (base_ + e) / kBucket;
if (b1 <= b0) return 0.0;
f64 total = 0.0;
cover_buckets(*this, b0, b1, [&](u32 l, u64 i) {
const f64 prev = i == 0 ? 0.0 : sums_[l][static_cast<usize>(i - 1)];
total += sums_[l][static_cast<usize>(i)] - prev;
});
cover_buckets(*this, b0, b1, [&](u32 l, u64 i) { total += at(l, i).sum; });
return total;
}
f64 MinMaxPyramid::sumsq_buckets(u64 begin, u64 end) const {
const u64 e = std::min(end, size_);
const u64 b = std::min(begin, e);
const u64 b0 = ceil_to_bucket(b) / kBucket;
const u64 b1 = e / kBucket;
const u64 e = std::min(end, size());
if (begin >= e) return 0.0;
const u64 b0 = ceil_to_bucket(base_ + begin) / kBucket;
const u64 b1 = (base_ + e) / kBucket;
if (b1 <= b0) return 0.0;
f64 total = 0.0;
cover_buckets(*this, b0, b1, [&](u32 l, u64 i) {
const f64 prev = i == 0 ? 0.0 : sumsqs_[l][static_cast<usize>(i - 1)];
total += sumsqs_[l][static_cast<usize>(i)] - prev;
});
cover_buckets(*this, b0, b1, [&](u32 l, u64 i) { total += at(l, i).sumsq; });
return total;
}
@@ -229,7 +264,7 @@ u32 MinMaxPyramid::level_for(u64 n, f64 samples_per_pixel) const {
const f64 lg = std::floor(std::log2(ratio));
u32 l = lg >= f64(62) ? 62u : static_cast<u32>(lg);
while (l > 0) {
const bool exists = l < level_count() && level(l).size() >= 2;
const bool exists = l < level_count() && level_size(l) >= 2;
if (exists && buckets_for(n, l) >= 2) break;
--l;
}
@@ -237,14 +272,14 @@ u32 MinMaxPyramid::level_for(u64 n, f64 samples_per_pixel) const {
}
u64 MinMaxPyramid::snap(f64 x, f64 x0, f64 dx) const {
if (dx <= 0.0 || size_ == 0) return 0;
if (dx <= 0.0 || size() == 0) return 0;
const f64 i = (x - x0) / dx;
// Bound the round before it: llround() is undefined for values a long long
// cannot hold, and this also maps NaN to index 0.
if (!(i > 0.0)) return 0;
if (i >= f64(size_)) return size_ - 1;
if (i >= f64(size())) return size() - 1;
const u64 idx = static_cast<u64>(std::llround(i));
return idx >= size_ ? size_ - 1 : idx;
return idx >= size() ? size() - 1 : idx;
}
} // namespace plot::data
+34 -19
View File
@@ -12,23 +12,30 @@ constexpr u64 kGatherChunk = 256;
} // namespace
Series::QueryParts Series::split_query(u64 begin, u64 end) {
Series::QueryParts Series::split_query(u64 begin, u64 end) const {
QueryParts p;
// First level-0 bucket boundary at or after begin, and the last at or before
// end: [interior_begin, interior_end) is exactly the part the pyramid can
// answer with whole buckets.
const u64 up = (begin / kBucket + (begin % kBucket == 0 ? u64{0} : u64{1})) * kBucket;
const u64 down = (end / kBucket) * kBucket;
// The pyramid's buckets are absolute while these are window coordinates, so its
// level-0 boundaries sit at window index `window_base() rounded up to the next
// multiple of kBucket`, and every kBucket after that. Align the split in
// absolute coordinates and come back to the window: aligning in window
// coordinates would treat index 0 as a boundary and, for a window that slid
// into the middle of a bucket, route samples through the pyramid that it does
// not cover.
const u64 origin = pyramid().window_base();
const u64 a_begin = origin + begin;
const u64 a_end = origin + end;
const u64 up = (a_begin / kBucket + (a_begin % kBucket == 0 ? u64{0} : u64{1})) * kBucket;
const u64 down = (a_end / kBucket) * kBucket;
if (up < down) {
p.interior_begin = up;
p.interior_end = down;
p.interior_begin = up - origin; // up >= a_begin >= origin
p.interior_end = down - origin;
}
// The two edge spans never overlap and never exceed one bucket each: when the
// interior is empty they tile [begin, end) together, otherwise they are the
// head and tail buckets half inside the range.
p.left_begin = begin;
p.left_end = std::min(end, up);
p.right_begin = std::max(down, p.left_end);
p.left_end = std::min(end, up - origin);
p.right_begin = std::max(down > origin ? down - origin : u64{0}, p.left_end);
p.right_end = end;
return p;
}
@@ -132,17 +139,25 @@ void Series::gather_impl(u32 level, u64 begin, u64 end, std::vector<f32>& out_xy
return;
}
// Level >= 1: one envelope pair per bucket, x at the bucket's first sample
// (min) and at its last sample (max), which keeps x monotonic while covering
// the bucket's span.
// Level >= 1: one envelope pair per *retained* bucket the range overlaps, x at
// the bucket's first sample (min) and at its last sample (max), which keeps x
// monotonic while covering the bucket's span. The pyramid's buckets are
// absolute, so map the window range onto its grid and clamp to the buckets that
// are both complete and wholly inside the window: a bucket that is only partly
// retained aggregates samples the window no longer has, and a partial trailing
// bucket is not stored at all.
const u64 span = u64{kBucket} << level;
const std::span<const MinMax> buckets = pyramid().level(level);
const u64 first = b / span;
const u64 last = std::min<u64>((e - 1) / span + 1, static_cast<u64>(buckets.size()));
const u64 origin = pyramid().window_base();
const u64 lbase = pyramid().level_base(level);
const u64 lend = lbase + pyramid().level_size(level);
if (lbase >= lend) return;
const u64 a_end = origin + e;
const u64 first = std::max((origin + b) / span, lbase);
const u64 last = std::min((a_end - 1) / span + 1, lend);
for (u64 bi = first; bi < last; ++bi) {
const MinMax m = buckets[static_cast<usize>(bi)];
const u64 first_sample = bi * span;
const u64 last_sample = std::min(first_sample + span, n) - 1;
const MinMax m = pyramid().bucket(level, bi);
const u64 first_sample = bi * span - origin; // >= 0: stored buckets start inside the window
const u64 last_sample = std::min((bi + 1) * span, a_end) - 1 - origin;
out_xy.push_back(f32(x_0 + f64(first_sample) * step));
out_xy.push_back(m.min);
out_xy.push_back(f32(x_0 + f64(last_sample) * step));
+35 -57
View File
@@ -6,14 +6,18 @@
namespace plot::data {
namespace {
/// Samples copied per ring read on the ingest path, so rebuilding a saturated
/// window costs no allocation.
/// Samples copied per ring read on the ingest path, so appending the samples
/// produced between two frames costs no allocation.
constexpr u64 kReadChunk = 256;
} // namespace
StreamSeries::StreamSeries(u64 capacity, f64 x0, f64 dx)
: ring_(capacity), x0_(x0), dx_(dx) {}
: ring_(capacity), x0_(x0), dx_(dx) {
// Size the pyramid's level table for a full window up front: the ring never
// holds more than capacity() samples, so appending never grows it.
pyramid_.reserve(capacity);
}
void StreamSeries::push(f32 y) { ring_.push(y); }
@@ -27,62 +31,35 @@ void StreamSeries::acquire() {
const u64 p = ring_.produced();
const u64 cap = ring_.capacity();
const u64 base = p > cap ? p - cap : 0; // oldest index the ring still holds
const u64 n = p - base; // retained, published sample count
if (base != base_) {
// The window slid: samples the pyramid covered are gone, and because bucket
// boundaries are absolute, extending it incrementally would mis-align every
// bucket. Rebuild from the retained samples instead.
pyramid_.clear();
pyramid_.reserve(n);
f32 lo = 0.0f, hi = 0.0f;
bool first = true;
f32 chunk[kReadChunk];
for (u64 i = 0; i < n;) {
const u64 take = std::min<u64>(kReadChunk, n - i);
ring_.copy_from(base + i, take, chunk);
for (u64 k = 0; k < take; ++k) {
const f32 v = chunk[k];
pyramid_.append(v);
if (first) {
lo = v;
hi = v;
first = false;
} else {
lo = std::min(lo, v);
hi = std::max(hi, v);
}
}
i += take;
}
y_bounds_ = first ? Range{0.0, 0.0} : Range{f64(lo), f64(hi)};
base_ = base;
} else {
// Contiguous: only the samples produced since the last acquire() are new.
if (n == 0) y_bounds_ = Range{0.0, 0.0};
f32 chunk[kReadChunk];
for (u64 i = end_ - base_; i < n;) {
const u64 take = std::min<u64>(kReadChunk, n - i);
ring_.copy_from(base + i, take, chunk);
for (u64 k = 0; k < take; ++k) {
const f32 v = chunk[k];
pyramid_.append(v);
if (i + k == 0) {
y_bounds_ = Range{f64(v), f64(v)};
} else {
y_bounds_.min = std::min(y_bounds_.min, f64(v));
y_bounds_.max = std::max(y_bounds_.max, f64(v));
}
}
i += take;
}
// Slide the pyramid's window: the buckets that covered samples the ring has
// overwritten are dropped, and the rest stay exactly as they were built --
// their boundaries are absolute, so nothing has to be recomputed. This is also
// where a producer that lapped the consumer by more than the ring holds is
// handled: the pyramid restarts at the new base.
pyramid_.drop_before(base);
// Append exactly the samples produced since the last acquire(). Normally that
// is one frame's worth; after a lapped producer it is the whole window.
f32 chunk[kReadChunk];
for (u64 a = pyramid_.window_end(); a < p;) {
const u64 take = std::min<u64>(kReadChunk, p - a);
ring_.copy_from(a, take, chunk);
for (u64 k = 0; k < take; ++k) pyramid_.append(chunk[k]);
a += take;
}
end_ = p;
// Exact bounds for the published window, in the same O(log n + kBucket) as any
// other exact query over it: the bucket-aligned interior comes from the
// pyramid and the two partial edge spans are scanned through copy_raw(). An
// incrementally extended bound cannot work here -- it would have to know
// whether the extreme it is holding has just slid out of the window.
y_bounds_ = reduce(0, size());
}
u64 StreamSeries::size() const { return end_ - base_; }
u64 StreamSeries::size() const { return pyramid_.size(); }
f64 StreamSeries::x0() const { return x0_ + f64(base_) * dx_; }
f64 StreamSeries::x0() const { return x0_ + f64(pyramid_.window_base()) * dx_; }
f64 StreamSeries::dx() const { return dx_; }
@@ -105,9 +82,10 @@ void StreamSeries::copy_raw(u64 begin, u64 end, f32* out) const {
const u64 n = size();
const u64 b = std::min(begin, n);
const u64 e = std::min(std::max(end, b), n);
// Retained index b is absolute base_ + b: fixed for the whole copy, so a
// pushing producer cannot shift the window midway.
ring_.copy_from(base_ + b, e - b, out);
// Retained index b is absolute window_base() + b: fixed for the whole copy, so
// a pushing producer cannot shift the window midway (the pyramid is written by
// this thread alone, so its window cannot move under the read either).
ring_.copy_from(pyramid_.window_base() + b, e - b, out);
std::fill(out + static_cast<std::ptrdiff_t>(e - b),
out + static_cast<std::ptrdiff_t>(want), 0.0f);
}