perf(data): absolute-space pyramid so acquire() is O(produced), not O(window)
Frame cost was O(capacity x levels) once the ring saturated: acquire() saw the window base move (it always moves once the ring wraps), cleared the pyramid and re-appended all 1,048,576 retained samples, and append() walked every live level with no early stop -- 13,647,936 bucket merges and 73.8 ms per frame at a 1M-sample window, scaling with the window, which is what capped the 1 MS/s demo at 32-58 ms/frame while it was still filling. Index the pyramid in absolute bucket space instead. Level l bucket k covers the samples [k * (kBucket << l), (k + 1) * (kBucket << l)), so a bucket's span never depends on where the window starts and the window can slide over the aggregate instead of forcing it to be rebuilt. Each level keeps its complete buckets in a deque plus the absolute base of its first entry; append() folds a sample into the single in-progress level-0 bucket and, only when that bucket completes, cascades upward, stopping at the first level that gains no new entry; a new drop_before() slides the window by discarding leading buckets and remembers where the window begins. acquire() is now: drop to the new base, then append exactly what was produced -- 256 merges and 0.034 ms per frame for 8192 new samples, independent of the window, and memory bounded by the window (32,767 entries after 50M samples through a 1M ring, the same as after 10M). A producer that laps the consumer by more than capacity() between two acquires restarts the window rather than splicing a gap. Interface: `level(l) -> std::span<const MinMax>` cannot survive a ring and is replaced by level_base/level_size/bucket in absolute bucket coordinates, plus window_base/window_end/drop_before. The *_buckets queries stay in window coordinates (index 0 = oldest retained sample), so Series keeps its window semantics for callers; its exactness split now aligns in absolute space (window_base() + the ceil/floor) because the bucket grid is absolute. The sums move from per-level prefix sums to per-bucket ones -- the only representation a sliding front can keep exact -- and StreamSeries::y_bounds() becomes reduce(0, size()), exact in O(log n + kBucket) per frame instead of a scan of the window. The pyramid owns the published window now, so there is no second copy of base/end to keep in step, and gather() emits only buckets that are both complete and wholly retained. Tests: test-only merge counters (mirroring PLOT_PYRAMID_COUNT_COVERS, absent from every non-test build) pin an acquire proportional to production and independent of capacity, an append cost amortised over 16 live levels, and bounded memory; brute-force checks pin absolute bucket aggregates across slides, sliding-window reduce/sum/sumsq over unaligned ranges, and the level-1 gather mapping. The demo now draws 946 samples/px at 1.03 ms/frame (967 fps) where the saturated window cost 54.96 ms/frame (18 fps).
This commit is contained in:
1 parent
4b649a2783
commit
0ac5d47fa6
13 files changed
+769
-244
No files matched your search
+149
-114
@@ -1,8 +1,10 @@
|
||||
#include <plot/data/detail/cover_count.hpp>
|
||||
#include <plot/data/detail/merge_count.hpp>
|
||||
#include <plot/data/minmax_pyramid.hpp>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
#include <cstddef>
|
||||
|
||||
namespace plot::data {
|
||||
namespace {
|
||||
@@ -16,18 +18,19 @@ constexpr u64 buckets_for(u64 n, u32 l) {
|
||||
return n / span + (n % span == 0 ? u64{0} : u64{1});
|
||||
}
|
||||
|
||||
/// Ceiling division: first level-l bucket whose span starts at or after v.
|
||||
constexpr u64 ceil_div(u64 v, u64 d) { return v / d + (v % d == 0 ? u64{0} : u64{1}); }
|
||||
|
||||
/// First level-0 bucket boundary at or after v.
|
||||
constexpr u64 ceil_to_bucket(u64 v) {
|
||||
return (v / kBucket + (v % kBucket == 0 ? u64{0} : u64{1})) * kBucket;
|
||||
}
|
||||
constexpr u64 ceil_to_bucket(u64 v) { return ceil_div(v, kBucket) * kBucket; }
|
||||
|
||||
/// Canonical, coarsest-first decomposition of the level-0 bucket span [b0, b1)
|
||||
/// into pyramid entries, calling fn(level, index) for each.
|
||||
/// into pyramid entries, calling fn(level, absolute_bucket_index) for each.
|
||||
///
|
||||
/// A level-l entry is emitted only when the whole level-l bucket exists (all
|
||||
/// kBucket << l samples are present, so its aggregate is final). Callers
|
||||
/// guarantee b1 <= ceil(size() / kBucket), so the only possible remainder is the
|
||||
/// single trailing partial level-0 bucket, which is emitted at level 0.
|
||||
/// guarantee b0 >= the first whole bucket of the window and b1 <= the number of
|
||||
/// complete buckets, so every emission both exists and lies inside the window.
|
||||
///
|
||||
/// The level is chosen per *emission*, not once per pass: at each cursor the
|
||||
/// largest level is taken whose bucket is aligned with the cursor, lies wholly
|
||||
@@ -40,16 +43,18 @@ constexpr u64 ceil_to_bucket(u64 v) {
|
||||
/// no coarser level is ever reachable from an odd cursor.
|
||||
template <typename Fn>
|
||||
void cover_buckets(const MinMaxPyramid& p, u64 b0, u64 b1, Fn&& fn) {
|
||||
const u64 complete = p.size() / kBucket;
|
||||
const u64 limit = std::min(b1, complete);
|
||||
// One past the last complete level-0 bucket: levels sit on top of those, so
|
||||
// this also bounds every coarser bucket the cursor may take.
|
||||
const u64 limit = std::min(b1, p.level_base(0) + p.level_size(0));
|
||||
const u32 top = p.level_count() - 1; // level_count() is at least 1
|
||||
u64 b = b0;
|
||||
while (b < b1) {
|
||||
u32 li = top;
|
||||
for (;;) {
|
||||
const u64 lb = u64{1} << li;
|
||||
if ((b % lb) == 0 && b + lb <= limit &&
|
||||
(b / lb) < static_cast<u64>(p.level(li).size())) {
|
||||
const u64 i = b >> li;
|
||||
if ((b % lb) == 0 && b + lb <= limit && i >= p.level_base(li) &&
|
||||
i - p.level_base(li) < p.level_size(li)) {
|
||||
break;
|
||||
}
|
||||
if (li == 0) break;
|
||||
@@ -69,7 +74,7 @@ Range fold_range(const MinMaxPyramid& p, u64 b0, u64 b1) {
|
||||
Range r{};
|
||||
bool first = true;
|
||||
cover_buckets(p, b0, b1, [&](u32 l, u64 i) {
|
||||
const MinMax m = p.level(l)[static_cast<usize>(i)];
|
||||
const MinMax m = p.bucket(l, i);
|
||||
if (first) {
|
||||
r.min = f64(m.min);
|
||||
r.max = f64(m.max);
|
||||
@@ -84,140 +89,170 @@ Range fold_range(const MinMaxPyramid& p, u64 b0, u64 b1) {
|
||||
|
||||
} // namespace
|
||||
|
||||
void MinMaxPyramid::add_level() {
|
||||
levels_.emplace_back();
|
||||
sums_.emplace_back();
|
||||
sumsqs_.emplace_back();
|
||||
}
|
||||
void MinMaxPyramid::add_level() { levels_.emplace_back(); }
|
||||
|
||||
void MinMaxPyramid::reserve(u64 n) {
|
||||
if (n > reserved_n_) reserved_n_ = n;
|
||||
const u64 want = std::max(std::max(n, size_), u64{2});
|
||||
// Materialise every level n samples can need -- level 0 plus one per doubling
|
||||
// of the bucket -- so a run of appends never grows the level vector. See the
|
||||
// header for why the bucket deques themselves cannot be pre-sized.
|
||||
u32 want = 1;
|
||||
for (u64 span = level_span(0); span <= n && want < 63; span <<= 1) ++want;
|
||||
if (levels_.size() < want) levels_.resize(want);
|
||||
}
|
||||
|
||||
// Materialise every level n samples can need -- buckets *and* prefix sums --
|
||||
// and size each one up front, so a run of appends allocates nothing at all.
|
||||
u32 max_levels = 1;
|
||||
for (u64 span = level_span(0); span <= want && max_levels < 63; span <<= 1) ++max_levels;
|
||||
while (levels_.size() < max_levels) add_level();
|
||||
for (u32 l = 0; l < levels_.size(); ++l) {
|
||||
const usize want_l = static_cast<usize>(buckets_for(want, l));
|
||||
levels_[l].reserve(want_l);
|
||||
sums_[l].reserve(want_l);
|
||||
sumsqs_[l].reserve(want_l);
|
||||
}
|
||||
void MinMaxPyramid::push(u32 l, u64 k, const Entry& e) {
|
||||
Level& lv = levels_[l];
|
||||
// The first entry of a level fixes its base; every later entry is the next
|
||||
// absolute bucket, which is what keeps a level one contiguous run.
|
||||
if (lv.entries.empty()) lv.base = k;
|
||||
lv.entries.push_back(e);
|
||||
#ifdef PLOT_PYRAMID_COUNT_MERGES
|
||||
detail::merge_count().fetch_add(1, std::memory_order_relaxed);
|
||||
#endif
|
||||
}
|
||||
|
||||
void MinMaxPyramid::append(f32 v) {
|
||||
const u64 prev = size_;
|
||||
const f64 vd = f64(v);
|
||||
// sum_[l].back() is the sum over every sample appended so far, and the newest
|
||||
// sample always lands in the last bucket of every level, so the last prefix
|
||||
// entry of every level is exactly the running total.
|
||||
const f64 total_before = sums_[0].empty() ? 0.0 : sums_[0].back();
|
||||
const f64 total = total_before + vd;
|
||||
const f64 sq_before = sumsqs_[0].empty() ? 0.0 : sumsqs_[0].back();
|
||||
const f64 total_sq = sq_before + vd * vd;
|
||||
|
||||
const u64 idx = prev / kBucket;
|
||||
if (idx == levels_[0].size()) {
|
||||
levels_[0].push_back(MinMax{v, v});
|
||||
sums_[0].push_back(total);
|
||||
sumsqs_[0].push_back(total_sq);
|
||||
// The new sample always lands in the in-progress level-0 bucket, which is the
|
||||
// only bucket any append ever touches: a bucket becomes an entry exactly once,
|
||||
// complete, and is never rewritten.
|
||||
if (count_ % kBucket == 0) {
|
||||
pending_.mm.min = v;
|
||||
pending_.mm.max = v;
|
||||
pending_.sum = vd;
|
||||
pending_.sumsq = vd * vd;
|
||||
} else {
|
||||
MinMax& cur = levels_[0][static_cast<usize>(idx)];
|
||||
cur.min = std::min(cur.min, v);
|
||||
cur.max = std::max(cur.max, v);
|
||||
sums_[0][static_cast<usize>(idx)] = total;
|
||||
sumsqs_[0][static_cast<usize>(idx)] = total_sq;
|
||||
pending_.mm.min = std::min(pending_.mm.min, v);
|
||||
pending_.mm.max = std::max(pending_.mm.max, v);
|
||||
pending_.sum += vd;
|
||||
pending_.sumsq += vd * vd;
|
||||
}
|
||||
++count_;
|
||||
if (count_ % kBucket != 0) return; // the bucket is still partial: nothing to push
|
||||
|
||||
const u64 k0 = count_ / kBucket - 1; // the level-0 bucket that just completed
|
||||
if (k0 < level_base(0)) return; // its span starts before the window: never read
|
||||
const Entry done = pending_;
|
||||
push(0, k0, done);
|
||||
|
||||
// Cascade: level l gains an entry exactly when kBucket << l divides the sample
|
||||
// count, so the walk stops at the first level that gained nothing -- amortised
|
||||
// O(1) per sample (1/kBucket + 1/(2*kBucket) + ... = 1/(kBucket/2) merges).
|
||||
for (u32 l = 1; l < 63; ++l) {
|
||||
const u64 span = level_span(l);
|
||||
if (count_ % span != 0) break;
|
||||
const u64 k = count_ / span - 1; // the level-l bucket that just completed
|
||||
const u64 child = 2 * k; // its left child, at level l-1
|
||||
if (child < level_base(l - 1)) break; // a child is gone: this parent is not retained
|
||||
if (l == static_cast<u32>(levels_.size())) add_level();
|
||||
if (l >= live_levels_) live_levels_ = l + 1;
|
||||
const Entry& a = at(l - 1, child);
|
||||
const Entry& b = at(l - 1, child + 1);
|
||||
Entry m;
|
||||
m.mm.min = std::min(a.mm.min, b.mm.min);
|
||||
m.mm.max = std::max(a.mm.max, b.mm.max);
|
||||
m.sum = a.sum + b.sum;
|
||||
m.sumsq = a.sumsq + b.sumsq;
|
||||
push(l, k, m);
|
||||
}
|
||||
}
|
||||
|
||||
void MinMaxPyramid::drop_before(u64 absolute_sample) {
|
||||
if (absolute_sample <= base_) return;
|
||||
if (absolute_sample > count_) {
|
||||
// The producer produced more than the ring held between two acquires: every
|
||||
// appended sample is gone and the ones in between were never appended, so the
|
||||
// pyramid restarts empty at the new base rather than splicing a gap.
|
||||
base_ = absolute_sample;
|
||||
count_ = absolute_sample;
|
||||
pending_ = Entry{};
|
||||
live_levels_ = 1;
|
||||
for (u32 l = 0; l < levels_.size(); ++l) {
|
||||
levels_[l].entries.clear();
|
||||
levels_[l].base = ceil_div(absolute_sample, level_span(l));
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
for (u32 l = 1; l < 63; ++l) {
|
||||
// A level only carries data once the level below has more than one bucket: a
|
||||
// one-child parent would just duplicate it.
|
||||
if (levels_[l - 1].size() < 2) break;
|
||||
if (l == live_levels_) {
|
||||
++live_levels_;
|
||||
if (l == levels_.size()) { // no reserve() sized this level up front
|
||||
add_level();
|
||||
if (reserved_n_ != 0) {
|
||||
const usize want = static_cast<usize>(buckets_for(reserved_n_, l));
|
||||
levels_[l].reserve(want);
|
||||
sums_[l].reserve(want);
|
||||
sumsqs_[l].reserve(want);
|
||||
}
|
||||
}
|
||||
}
|
||||
const u64 ci = prev / level_span(l - 1); // current bucket at level l-1
|
||||
const u64 parent = ci / 2; // its parent at level l
|
||||
MinMax m;
|
||||
if ((ci & 1u) != 0) {
|
||||
const MinMax& a = levels_[l - 1][static_cast<usize>(ci - 1)];
|
||||
const MinMax& b = levels_[l - 1][static_cast<usize>(ci)];
|
||||
m.min = std::min(a.min, b.min);
|
||||
m.max = std::max(a.max, b.max);
|
||||
base_ = absolute_sample;
|
||||
for (u32 l = 0; l < levels_.size(); ++l) {
|
||||
Level& lv = levels_[l];
|
||||
// A bucket is dropped once its span starts before the window: the first
|
||||
// retained bucket of every level is the first one wholly inside the window,
|
||||
// which is exactly what the queries assume.
|
||||
const u64 keep = ceil_div(base_, level_span(l));
|
||||
if (keep <= lv.base) continue;
|
||||
const u64 have = static_cast<u64>(lv.entries.size());
|
||||
const u64 drop = std::min(keep - lv.base, have);
|
||||
lv.entries.erase(lv.entries.begin(),
|
||||
lv.entries.begin() + static_cast<std::ptrdiff_t>(drop));
|
||||
// The base is where the next entry goes: past the last one if the level was
|
||||
// emptied, which is how the cursor catches up with the sliding window.
|
||||
if (drop == have) {
|
||||
lv.base = keep;
|
||||
} else {
|
||||
m = levels_[l - 1][static_cast<usize>(ci)];
|
||||
}
|
||||
if (parent == levels_[l].size()) {
|
||||
levels_[l].push_back(m);
|
||||
sums_[l].push_back(total);
|
||||
sumsqs_[l].push_back(total_sq);
|
||||
} else {
|
||||
levels_[l][static_cast<usize>(parent)] = m;
|
||||
sums_[l][static_cast<usize>(parent)] = total;
|
||||
sumsqs_[l][static_cast<usize>(parent)] = total_sq;
|
||||
lv.base += drop;
|
||||
}
|
||||
}
|
||||
size_ = prev + 1;
|
||||
// The in-progress level-0 bucket may now start before the window's base: its
|
||||
// aggregate covers samples that are gone. It belongs to the bucket the window
|
||||
// starts inside, which is never read (the queries take whole buckets and the
|
||||
// Series scans the two partial edges raw), so discarding it is lossless.
|
||||
if (ceil_div(base_, kBucket) > count_ / kBucket) pending_ = Entry{};
|
||||
while (live_levels_ > 1 && level_size(live_levels_ - 1) == 0) --live_levels_;
|
||||
}
|
||||
|
||||
void MinMaxPyramid::clear() {
|
||||
size_ = 0;
|
||||
count_ = 0;
|
||||
base_ = 0;
|
||||
live_levels_ = 1;
|
||||
for (auto& lv : levels_) lv.clear();
|
||||
for (auto& s : sums_) s.clear();
|
||||
for (auto& s : sumsqs_) s.clear();
|
||||
levels_.resize(1);
|
||||
levels_[0].entries.clear();
|
||||
levels_[0].base = 0;
|
||||
pending_ = Entry{};
|
||||
}
|
||||
|
||||
std::span<const MinMax> MinMaxPyramid::level(u32 l) const {
|
||||
if (l >= level_count()) return {};
|
||||
return levels_[l];
|
||||
u64 MinMaxPyramid::level_base(u32 l) const noexcept {
|
||||
return l < levels_.size() ? levels_[l].base : u64{0};
|
||||
}
|
||||
|
||||
u64 MinMaxPyramid::level_size(u32 l) const noexcept {
|
||||
return l < levels_.size() ? static_cast<u64>(levels_[l].entries.size()) : u64{0};
|
||||
}
|
||||
|
||||
MinMax MinMaxPyramid::bucket(u32 l, u64 i) const { return at(l, i).mm; }
|
||||
|
||||
Range MinMaxPyramid::reduce_buckets(u64 begin, u64 end) const {
|
||||
const u64 e = std::min(end, size_);
|
||||
const u64 b = std::min(begin, e);
|
||||
const u64 b0 = ceil_to_bucket(b) / kBucket; // first whole bucket inside
|
||||
const u64 b1 = e / kBucket; // one past the last whole bucket
|
||||
const u64 e = std::min(end, size());
|
||||
if (begin >= e) return Range{0.0, 0.0};
|
||||
// The bucket grid is absolute while these are window coordinates: window index
|
||||
// i is absolute window_base() + i, so the first whole bucket inside the range is
|
||||
// the first absolute bucket boundary at or after that.
|
||||
const u64 b0 = ceil_to_bucket(base_ + begin) / kBucket;
|
||||
const u64 b1 = (base_ + e) / kBucket;
|
||||
if (b1 <= b0) return Range{0.0, 0.0};
|
||||
return fold_range(*this, b0, b1);
|
||||
}
|
||||
|
||||
f64 MinMaxPyramid::sum_buckets(u64 begin, u64 end) const {
|
||||
const u64 e = std::min(end, size_);
|
||||
const u64 b = std::min(begin, e);
|
||||
const u64 b0 = ceil_to_bucket(b) / kBucket;
|
||||
const u64 b1 = e / kBucket;
|
||||
const u64 e = std::min(end, size());
|
||||
if (begin >= e) return 0.0;
|
||||
const u64 b0 = ceil_to_bucket(base_ + begin) / kBucket;
|
||||
const u64 b1 = (base_ + e) / kBucket;
|
||||
if (b1 <= b0) return 0.0;
|
||||
f64 total = 0.0;
|
||||
cover_buckets(*this, b0, b1, [&](u32 l, u64 i) {
|
||||
const f64 prev = i == 0 ? 0.0 : sums_[l][static_cast<usize>(i - 1)];
|
||||
total += sums_[l][static_cast<usize>(i)] - prev;
|
||||
});
|
||||
cover_buckets(*this, b0, b1, [&](u32 l, u64 i) { total += at(l, i).sum; });
|
||||
return total;
|
||||
}
|
||||
|
||||
f64 MinMaxPyramid::sumsq_buckets(u64 begin, u64 end) const {
|
||||
const u64 e = std::min(end, size_);
|
||||
const u64 b = std::min(begin, e);
|
||||
const u64 b0 = ceil_to_bucket(b) / kBucket;
|
||||
const u64 b1 = e / kBucket;
|
||||
const u64 e = std::min(end, size());
|
||||
if (begin >= e) return 0.0;
|
||||
const u64 b0 = ceil_to_bucket(base_ + begin) / kBucket;
|
||||
const u64 b1 = (base_ + e) / kBucket;
|
||||
if (b1 <= b0) return 0.0;
|
||||
f64 total = 0.0;
|
||||
cover_buckets(*this, b0, b1, [&](u32 l, u64 i) {
|
||||
const f64 prev = i == 0 ? 0.0 : sumsqs_[l][static_cast<usize>(i - 1)];
|
||||
total += sumsqs_[l][static_cast<usize>(i)] - prev;
|
||||
});
|
||||
cover_buckets(*this, b0, b1, [&](u32 l, u64 i) { total += at(l, i).sumsq; });
|
||||
return total;
|
||||
}
|
||||
|
||||
@@ -229,7 +264,7 @@ u32 MinMaxPyramid::level_for(u64 n, f64 samples_per_pixel) const {
|
||||
const f64 lg = std::floor(std::log2(ratio));
|
||||
u32 l = lg >= f64(62) ? 62u : static_cast<u32>(lg);
|
||||
while (l > 0) {
|
||||
const bool exists = l < level_count() && level(l).size() >= 2;
|
||||
const bool exists = l < level_count() && level_size(l) >= 2;
|
||||
if (exists && buckets_for(n, l) >= 2) break;
|
||||
--l;
|
||||
}
|
||||
@@ -237,14 +272,14 @@ u32 MinMaxPyramid::level_for(u64 n, f64 samples_per_pixel) const {
|
||||
}
|
||||
|
||||
u64 MinMaxPyramid::snap(f64 x, f64 x0, f64 dx) const {
|
||||
if (dx <= 0.0 || size_ == 0) return 0;
|
||||
if (dx <= 0.0 || size() == 0) return 0;
|
||||
const f64 i = (x - x0) / dx;
|
||||
// Bound the round before it: llround() is undefined for values a long long
|
||||
// cannot hold, and this also maps NaN to index 0.
|
||||
if (!(i > 0.0)) return 0;
|
||||
if (i >= f64(size_)) return size_ - 1;
|
||||
if (i >= f64(size())) return size() - 1;
|
||||
const u64 idx = static_cast<u64>(std::llround(i));
|
||||
return idx >= size_ ? size_ - 1 : idx;
|
||||
return idx >= size() ? size() - 1 : idx;
|
||||
}
|
||||
|
||||
} // namespace plot::data
|
||||
+34
-19
@@ -12,23 +12,30 @@ constexpr u64 kGatherChunk = 256;
|
||||
|
||||
} // namespace
|
||||
|
||||
Series::QueryParts Series::split_query(u64 begin, u64 end) {
|
||||
Series::QueryParts Series::split_query(u64 begin, u64 end) const {
|
||||
QueryParts p;
|
||||
// First level-0 bucket boundary at or after begin, and the last at or before
|
||||
// end: [interior_begin, interior_end) is exactly the part the pyramid can
|
||||
// answer with whole buckets.
|
||||
const u64 up = (begin / kBucket + (begin % kBucket == 0 ? u64{0} : u64{1})) * kBucket;
|
||||
const u64 down = (end / kBucket) * kBucket;
|
||||
// The pyramid's buckets are absolute while these are window coordinates, so its
|
||||
// level-0 boundaries sit at window index `window_base() rounded up to the next
|
||||
// multiple of kBucket`, and every kBucket after that. Align the split in
|
||||
// absolute coordinates and come back to the window: aligning in window
|
||||
// coordinates would treat index 0 as a boundary and, for a window that slid
|
||||
// into the middle of a bucket, route samples through the pyramid that it does
|
||||
// not cover.
|
||||
const u64 origin = pyramid().window_base();
|
||||
const u64 a_begin = origin + begin;
|
||||
const u64 a_end = origin + end;
|
||||
const u64 up = (a_begin / kBucket + (a_begin % kBucket == 0 ? u64{0} : u64{1})) * kBucket;
|
||||
const u64 down = (a_end / kBucket) * kBucket;
|
||||
if (up < down) {
|
||||
p.interior_begin = up;
|
||||
p.interior_end = down;
|
||||
p.interior_begin = up - origin; // up >= a_begin >= origin
|
||||
p.interior_end = down - origin;
|
||||
}
|
||||
// The two edge spans never overlap and never exceed one bucket each: when the
|
||||
// interior is empty they tile [begin, end) together, otherwise they are the
|
||||
// head and tail buckets half inside the range.
|
||||
p.left_begin = begin;
|
||||
p.left_end = std::min(end, up);
|
||||
p.right_begin = std::max(down, p.left_end);
|
||||
p.left_end = std::min(end, up - origin);
|
||||
p.right_begin = std::max(down > origin ? down - origin : u64{0}, p.left_end);
|
||||
p.right_end = end;
|
||||
return p;
|
||||
}
|
||||
@@ -132,17 +139,25 @@ void Series::gather_impl(u32 level, u64 begin, u64 end, std::vector<f32>& out_xy
|
||||
return;
|
||||
}
|
||||
|
||||
// Level >= 1: one envelope pair per bucket, x at the bucket's first sample
|
||||
// (min) and at its last sample (max), which keeps x monotonic while covering
|
||||
// the bucket's span.
|
||||
// Level >= 1: one envelope pair per *retained* bucket the range overlaps, x at
|
||||
// the bucket's first sample (min) and at its last sample (max), which keeps x
|
||||
// monotonic while covering the bucket's span. The pyramid's buckets are
|
||||
// absolute, so map the window range onto its grid and clamp to the buckets that
|
||||
// are both complete and wholly inside the window: a bucket that is only partly
|
||||
// retained aggregates samples the window no longer has, and a partial trailing
|
||||
// bucket is not stored at all.
|
||||
const u64 span = u64{kBucket} << level;
|
||||
const std::span<const MinMax> buckets = pyramid().level(level);
|
||||
const u64 first = b / span;
|
||||
const u64 last = std::min<u64>((e - 1) / span + 1, static_cast<u64>(buckets.size()));
|
||||
const u64 origin = pyramid().window_base();
|
||||
const u64 lbase = pyramid().level_base(level);
|
||||
const u64 lend = lbase + pyramid().level_size(level);
|
||||
if (lbase >= lend) return;
|
||||
const u64 a_end = origin + e;
|
||||
const u64 first = std::max((origin + b) / span, lbase);
|
||||
const u64 last = std::min((a_end - 1) / span + 1, lend);
|
||||
for (u64 bi = first; bi < last; ++bi) {
|
||||
const MinMax m = buckets[static_cast<usize>(bi)];
|
||||
const u64 first_sample = bi * span;
|
||||
const u64 last_sample = std::min(first_sample + span, n) - 1;
|
||||
const MinMax m = pyramid().bucket(level, bi);
|
||||
const u64 first_sample = bi * span - origin; // >= 0: stored buckets start inside the window
|
||||
const u64 last_sample = std::min((bi + 1) * span, a_end) - 1 - origin;
|
||||
out_xy.push_back(f32(x_0 + f64(first_sample) * step));
|
||||
out_xy.push_back(m.min);
|
||||
out_xy.push_back(f32(x_0 + f64(last_sample) * step));
|
||||
|
||||
+35
-57
@@ -6,14 +6,18 @@
|
||||
namespace plot::data {
|
||||
namespace {
|
||||
|
||||
/// Samples copied per ring read on the ingest path, so rebuilding a saturated
|
||||
/// window costs no allocation.
|
||||
/// Samples copied per ring read on the ingest path, so appending the samples
|
||||
/// produced between two frames costs no allocation.
|
||||
constexpr u64 kReadChunk = 256;
|
||||
|
||||
} // namespace
|
||||
|
||||
StreamSeries::StreamSeries(u64 capacity, f64 x0, f64 dx)
|
||||
: ring_(capacity), x0_(x0), dx_(dx) {}
|
||||
: ring_(capacity), x0_(x0), dx_(dx) {
|
||||
// Size the pyramid's level table for a full window up front: the ring never
|
||||
// holds more than capacity() samples, so appending never grows it.
|
||||
pyramid_.reserve(capacity);
|
||||
}
|
||||
|
||||
void StreamSeries::push(f32 y) { ring_.push(y); }
|
||||
|
||||
@@ -27,62 +31,35 @@ void StreamSeries::acquire() {
|
||||
const u64 p = ring_.produced();
|
||||
const u64 cap = ring_.capacity();
|
||||
const u64 base = p > cap ? p - cap : 0; // oldest index the ring still holds
|
||||
const u64 n = p - base; // retained, published sample count
|
||||
|
||||
if (base != base_) {
|
||||
// The window slid: samples the pyramid covered are gone, and because bucket
|
||||
// boundaries are absolute, extending it incrementally would mis-align every
|
||||
// bucket. Rebuild from the retained samples instead.
|
||||
pyramid_.clear();
|
||||
pyramid_.reserve(n);
|
||||
f32 lo = 0.0f, hi = 0.0f;
|
||||
bool first = true;
|
||||
f32 chunk[kReadChunk];
|
||||
for (u64 i = 0; i < n;) {
|
||||
const u64 take = std::min<u64>(kReadChunk, n - i);
|
||||
ring_.copy_from(base + i, take, chunk);
|
||||
for (u64 k = 0; k < take; ++k) {
|
||||
const f32 v = chunk[k];
|
||||
pyramid_.append(v);
|
||||
if (first) {
|
||||
lo = v;
|
||||
hi = v;
|
||||
first = false;
|
||||
} else {
|
||||
lo = std::min(lo, v);
|
||||
hi = std::max(hi, v);
|
||||
}
|
||||
}
|
||||
i += take;
|
||||
}
|
||||
y_bounds_ = first ? Range{0.0, 0.0} : Range{f64(lo), f64(hi)};
|
||||
base_ = base;
|
||||
} else {
|
||||
// Contiguous: only the samples produced since the last acquire() are new.
|
||||
if (n == 0) y_bounds_ = Range{0.0, 0.0};
|
||||
f32 chunk[kReadChunk];
|
||||
for (u64 i = end_ - base_; i < n;) {
|
||||
const u64 take = std::min<u64>(kReadChunk, n - i);
|
||||
ring_.copy_from(base + i, take, chunk);
|
||||
for (u64 k = 0; k < take; ++k) {
|
||||
const f32 v = chunk[k];
|
||||
pyramid_.append(v);
|
||||
if (i + k == 0) {
|
||||
y_bounds_ = Range{f64(v), f64(v)};
|
||||
} else {
|
||||
y_bounds_.min = std::min(y_bounds_.min, f64(v));
|
||||
y_bounds_.max = std::max(y_bounds_.max, f64(v));
|
||||
}
|
||||
}
|
||||
i += take;
|
||||
}
|
||||
// Slide the pyramid's window: the buckets that covered samples the ring has
|
||||
// overwritten are dropped, and the rest stay exactly as they were built --
|
||||
// their boundaries are absolute, so nothing has to be recomputed. This is also
|
||||
// where a producer that lapped the consumer by more than the ring holds is
|
||||
// handled: the pyramid restarts at the new base.
|
||||
pyramid_.drop_before(base);
|
||||
|
||||
// Append exactly the samples produced since the last acquire(). Normally that
|
||||
// is one frame's worth; after a lapped producer it is the whole window.
|
||||
f32 chunk[kReadChunk];
|
||||
for (u64 a = pyramid_.window_end(); a < p;) {
|
||||
const u64 take = std::min<u64>(kReadChunk, p - a);
|
||||
ring_.copy_from(a, take, chunk);
|
||||
for (u64 k = 0; k < take; ++k) pyramid_.append(chunk[k]);
|
||||
a += take;
|
||||
}
|
||||
end_ = p;
|
||||
|
||||
// Exact bounds for the published window, in the same O(log n + kBucket) as any
|
||||
// other exact query over it: the bucket-aligned interior comes from the
|
||||
// pyramid and the two partial edge spans are scanned through copy_raw(). An
|
||||
// incrementally extended bound cannot work here -- it would have to know
|
||||
// whether the extreme it is holding has just slid out of the window.
|
||||
y_bounds_ = reduce(0, size());
|
||||
}
|
||||
|
||||
u64 StreamSeries::size() const { return end_ - base_; }
|
||||
u64 StreamSeries::size() const { return pyramid_.size(); }
|
||||
|
||||
f64 StreamSeries::x0() const { return x0_ + f64(base_) * dx_; }
|
||||
f64 StreamSeries::x0() const { return x0_ + f64(pyramid_.window_base()) * dx_; }
|
||||
|
||||
f64 StreamSeries::dx() const { return dx_; }
|
||||
|
||||
@@ -105,9 +82,10 @@ void StreamSeries::copy_raw(u64 begin, u64 end, f32* out) const {
|
||||
const u64 n = size();
|
||||
const u64 b = std::min(begin, n);
|
||||
const u64 e = std::min(std::max(end, b), n);
|
||||
// Retained index b is absolute base_ + b: fixed for the whole copy, so a
|
||||
// pushing producer cannot shift the window midway.
|
||||
ring_.copy_from(base_ + b, e - b, out);
|
||||
// Retained index b is absolute window_base() + b: fixed for the whole copy, so
|
||||
// a pushing producer cannot shift the window midway (the pyramid is written by
|
||||
// this thread alone, so its window cannot move under the read either).
|
||||
ring_.copy_from(pyramid_.window_base() + b, e - b, out);
|
||||
std::fill(out + static_cast<std::ptrdiff_t>(e - b),
|
||||
out + static_cast<std::ptrdiff_t>(want), 0.0f);
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user