plotter/docs/superpowers/specs/2026-09-27-plot-library-design.md

# `plot` — embeddable OpenGL time-series plotting library

**Date:** 2026-09-27
**Status:** approved (design reviewed section-by-section with the requester)

## Goal

A C++20 library that renders real-time, high-sample-rate time-series plots with
grid, axes, cursors and measurements, and that can be embedded into an existing
Qt6, GTK4, Dear ImGui, or raw GLFW application without owning the window, the GL
context, or the event loop.

## Non-goals (this cut)

- 3D surfaces, scatter/bar/pie charts, polar plots.
- Data I/O (CSV/HDF5 readers), file formats, network sources.
- Font rendering beyond the built-in bitmap atlas.
- Any host API not listed above.

These are explicitly out of scope so the first cut is a coherent vertical slice.
The layering (§3) leaves room for them without redesign.

## Constraints (fixed by the requester)

| Constraint | Value |
|---|---|
| Core dependencies | **zero.** No GLM, glad, GLEW, or bundled Khronos headers. |
| GL baseline | **3.3 core**, with a runtime-selected 4.3+ fast path. |
| Reference host | GLFW, plus ImGui and Qt6 and GTK4 adapters in the same cut. |
| First deliverable | buildable **end-to-end** skeleton: no stub bodies on the demo path. |
| Language | C++20, GCC/Clang, Linux first (no portability blockers introduced). |

## 1. Environment (verified on the development machine)

GCC 16.2.1, CMake 4.4.3, Ninja 1.13.2. Qt6 6.11.2 with `Qt6OpenGLWidgets`,
GTK4 4.22.5 with `gtkglarea.h`, GLFW 3.5.1, EGL 1.5 + `libEGL_mesa` (surfaceless
headless contexts available), `glm` and `GL/glew.h` installed but **unused**.
Dear ImGui is absent and is provided via `FetchContent` (network verified).

## 2. The embeddability contract

The library never creates a window, never creates or shares a GL context, and
never runs an event loop. Its render entry point is:

```cpp
plot::Renderer r;              // constructed with the caller's context current
r.init();
r.begin_frame(fb_w, fb_h, viewport_rect_in_pixels);   // in the caller's FBO
r.draw(plot);
r.end_frame();
```

Input arrives as plain value types (`PointerEvent`, `WheelEvent`, `KeyEvent`) that
each adapter translates from its host toolkit. Nothing in `plot::core` names a
toolkit, so a consumer links only the adapter they need.

## 3. Layering

```
adapters/{glfw,imgui,qt6,gtk4}   own the surface; translate host events
        ↓
plot/     Plot · Axis · Trace · Grid · Cursor · Measure · Annotation
        ↓
render/   Renderer · Style · TextBackend · primitive batches
        ↓
data/     Series · ArraySeries · StreamSeries · MinMaxPyramid
        ↓
gl/       runtime loader · RAII handles · capability probe
        ↓
core/ + math/   types · Color · Rect · vec2 · mat3 · Transform
```

Each layer depends only downward. `adapters/*` and `examples/*` are optional
CMake targets guarded by `PLOT_BUILD_*` options and by dependency discovery.

## 4. Performance architecture

These four decisions are what make 1M+ samples/frame viable. Each replaces an
approach that fails at high sample rates.

### 4.1 Data-space vertices, uniform-only pan and zoom

Vertex buffers hold raw *data* coordinates. Screen mapping is
`x' = x * sx + tx`, `y' = y * sy + ty` evaluated in the vertex shader from a
`Transform` uniform. The CPU never transforms a vertex, and a pan or zoom changes
one uniform and no vertex *values*.

The staged window is view-dependent: each frame the renderer gathers just the
visible sample range (or the visible buckets of the chosen LOD level) and uploads
it. That per-frame cost is **O(visible pixels)** — about two segments per pixel
column, flat in series length — so the property this section exists for holds.
A staging cache keyed on `(series, level, first, last, viewport)` would let a
static plot skip the redundant gather and upload entirely; it is not implemented,
because it changes no asymptotics and would add an invalidation surface (new
samples, LOD changes, series mutation, streaming window slide) whose bugs are
worse than the redundant work it removes. Recorded here as deliberate future
work, not as a missing feature.

Log axes are handled inside the same transform (`log10` before scaling) so no
separate code path or pre-transformed buffer is needed.

### 4.2 Min/max LOD pyramid

Every series owns a mip chain: level 0 buckets `kBucket = 64` consecutive samples
and stores one entry per bucket; each subsequent level halves the bucket count.
Each entry is 24 bytes — a `MinMax` pair (8 bytes) plus the exact `sum` and `sumsq`
doubles the range queries fold over (16 bytes) — so level 0 costs
`24n/64 = 0.375n` bytes and the geometric sum `1 + ½ + ¼ + …` brings the pyramid to
≈`0.75n` bytes, about 19% of the `4n` bytes of raw samples, not 2×. A min/max-only
index would be `n/4`, ≈6%: the two doubles are the trade that makes an interior
sum/mean/RMS/integral O(log n) whole-bucket additions instead of a scan of raw
samples, at every level, for every series that uses the measurement index.

- Built incrementally, in **absolute bucket space**: level `l` bucket `k` covers
  samples `[k · (kBucket << l), (k + 1) · (kBucket << l))`. A bucket's span
  therefore never depends on where the current window starts, which is what lets a
  window slide over a pyramid instead of forcing it to be rebuilt. `append()`
  folds a sample into the single in-progress level-0 bucket and, exactly when that
  bucket completes, pushes it and merges upward, stopping at the first level that
  gains no new entry: level `l` gains one entry per `kBucket << l` samples, so an
  append costs amortised O(1) (`1/kBucket + 1/(2·kBucket) + … = 2/kBucket` merges)
  and never walks the live levels.
- A sliding window (a `StreamSeries` ring) drops the leading buckets it no longer
  holds and appends only the newly produced samples, so nothing is recomputed and
  memory stays bounded by the retained window rather than the total ever pushed.
- The renderer picks the level where samples-per-pixel ≈ 2, so a 100M-sample
  series rasterises ≈ `2 · viewport_width` vertices instead of 100M.
- Buckets preserve extremes, so spikes are never dropped by decimation.

### 4.3 One rasteriser, two sampling levels

Every trace, every frame, is drawn by one path: instanced anti-aliased quads.
One instance per segment, four corners expanded from `gl_VertexID` in the vertex
shader, a fragment shader that does distance-to-segment SDF for smooth edges and
joins. There is no second primitive anywhere in the renderer — in particular no
`GL_LINE_STRIP` path — so a sparse trace and a dense one differ in the geometry
*staged*, never in the call that draws it.

What the samples-per-pixel threshold selects is therefore the gather level:

| samples/px | geometry staged |
|---|---|
| `< dense_threshold_spp` (1.5) | the visible window's raw samples: a level-0 `Series::gather()`, a polyline through the window, one instance per sample pair |
| `>= dense_threshold_spp` (1.5) | the pyramid level whose buckets are ~1 per pixel (`pyramid().level_for(n, samples_per_pixel)`): a min/max envelope pair per bucket, so O(pixels) instances instead of O(samples) |

`Style::dense_threshold_spp` is that threshold and nothing more: it picks the
gather level, changes no draw call, and the two sides of it are the same shader.
The dense side is where the work is saved (a 100M-sample window stages O(pixels)
vertices); the sparse side stages the window itself, which at ~1 sample per pixel
is ~1 instance per column anyway.

A geometry-shader expansion path was rejected: poor performance and it is the
one stage macOS cannot be relied on for.

### 4.4 Interval reduction reuses the pyramid

Cursor measurements (`Δx`, `Δy`, slope, RMS, mean, min/max, integral over a
dragged region) are answered from the same pyramid instead of by scanning the
series. `Series::reduce`/`sum`/`sumsq` cover the bucket-aligned interior with
pyramid aggregates and scan at most two partial edge buckets (≤ 2 × kBucket = 128
samples), so a query's cost is independent of series length. `Series::reduce` is
the single implementation used by measurement, auto-scale and hit-testing.

The split matters: a min/max pyramid stores *aggregates per bucket*, so it can
only answer exactly for whole buckets — a range with partial edge buckets would
otherwise silently include samples outside the range. Exactness therefore lives
on `Series` (which owns the raw samples), and `MinMaxPyramid` exposes only
bucket-exact queries, named `*_buckets` so they cannot be mistaken for exact ones.

The bucket grid is absolute while a sliding window's base is not, so that split is
computed in absolute coordinates — `Series` asks the pyramid for `window_base()`
and aligns `ceil`/`floor` there before mapping back to window indices. That is
also how a stream's `y_bounds()` stays exact in O(log n + kBucket) per frame: it
is `reduce(0, size())` over the window rather than a scan of it.

### 4.5 Decimation approach — decision

| | Approach | Verdict |
|---|---|---|
| **A** | **CPU min/max pyramid, incremental** | **chosen** — works on GL 3.3, deterministic, unit-testable with no GPU, O(1) amortised per sample, and it doubles as the measurement index. Cost: one CPU pass per sample, plus ≈19% memory (24-byte buckets: extremes plus exact per-bucket sums). |
| B | GPU compute decimation every frame | rejected — re-reads the whole series each frame (real bandwidth at 1M samples/frame), no 3.3 fallback, untestable headlessly. |
| C | no decimation | rejected — dies above ~500k samples. |

The 4.3+ fast path therefore applies to *transfer*, not to decimation: a
persistent-mapped, `glFenceSync`-guarded triple-buffered ring replaces the
staging-upload path. Both paths share identical semantics and are selected by
the capability probe.

## 5. Zero-dependency mechanisms

- **Math** — hand-written `vec2`, `mat3`, `Transform`: ~250 lines, no templates
  in public headers, no consumer-visible dependency.
- **GL loader** — a hand-maintained X-macro list `PLOT_GL_FUNCS(X)` covering only
  the ≈60 entry points actually used, with our own typedefs and enums. No
  `glcorearb.h` bundling and no code-generation step. Resolution order:
  context-provided `getProcAddress` (glX / EGL / wgl) → `dlsym`/`GetProcAddress`
  on the system GL library.
- **Text** — a `TextBackend` interface plus a built-in 8×16 bitmap-glyph atlas.
  Axis labelling needs no third-party code; a FreeType or stb_truetype backend
  can be added behind the same interface without touching callers.

There is no `third_party/` directory. Adapters use `find_package` first and
`FetchContent` only as a fallback (ImGui; GLFW when absent).

## 6. Public API sketch

Signatures here are the contract the implementation plan and adapters rely on.

```cpp
namespace plot {

using f32 = float;  using f64 = double;
using u32 = std::uint32_t;  using u64 = std::uint64_t;
constexpr u64 kInvalid = ~u64{0};

struct Range { f64 min, max; };
struct Rect  { f32 x, y, w, h; };            // pixels, origin top-left
struct Color { f32 r, g, b, a; static Color hex(u32 rgb, f32 a = 1.0f); };

enum class ErrorCode { Ok, InvalidArgument, OutOfMemory, GLFailure, Unsupported, NotImplemented };
struct Status { ErrorCode code = ErrorCode::Ok; const char* message = ""; explicit operator bool() const; };

// math/transform.hpp
struct Transform {
  f64 sx, tx, sy, ty; bool log_x = false, log_y = false;
  f64 to_screen_x(f64) const; f64 to_data_x(f64) const;
  f64 to_screen_y(f64) const; f64 to_data_y(f64) const;
};

namespace gl {
  using ProcLoader = void* (*)(const char*);
  bool load(ProcLoader);                 // nullptr → platform default
  struct Caps { int major, minor; bool compute, persistent_map, buffer_storage; };
  const Caps& caps();
  class Buffer; class VertexArray; class Program; class Texture; class Framebuffer; class Fence;
}

namespace data {
  struct MinMax { f32 min, max; };
  constexpr u32 kBucket = 64;

  class MinMaxPyramid {
   public:
    void reserve(u64 n); void append(f32 v); void clear();
    // Absolute bucket space: level l entry i covers the absolute samples
    // [i * (kBucket << l), (i + 1) * (kBucket << l)). The window slides over the
    // same buckets; queries are in window coordinates (index 0 = oldest retained).
    void drop_before(u64 absolute_sample);
    u64 size() const;                                        // retained window size
    u64 window_base() const;                                 // abs index of index 0
    u64 window_end() const;                                  // abs index past the last
    u32 level_count() const;
    u64 level_base(u32 l) const, level_size(u32 l) const;
    MinMax bucket(u32 l, u64 absolute_bucket) const;
    u32 level_for(u64 n, f64 samples_per_pixel) const;
    u64 snap(f64 x, f64 x0, f64 dx) const;                   // nearest sample index
    // Whole-bucket only: aggregates over the level-0 buckets fully contained in
    // [begin, end). Partial edges are excluded, not approximated. Callers wanting
    // an exact answer over an arbitrary range use Series::reduce/sum/sumsq.
    Range reduce_buckets(u64 begin, u64 end) const;
    f64   sum_buckets(u64 begin, u64 end) const;
    f64   sumsq_buckets(u64 begin, u64 end) const;
  };

  class Series {                                             // abstract
   public:
    virtual ~Series() = default;
    virtual u64 size() const = 0;
    virtual f64 x0() const = 0;                                // x of sample 0
    virtual f64 dx() const = 0;                                // uniform x spacing
    virtual Range x_bounds() const = 0;
    virtual Range y_bounds() const = 0;
    virtual const MinMaxPyramid& pyramid() const = 0;
    virtual void gather(u32 level, u64 begin, u64 end, std::vector<f32>& out_xy) const = 0;

    // Exact, O(kBucket) worst case regardless of series length: the bucket-aligned
    // interior comes from the pyramid, the at-most-two partial edge buckets are
    // read through copy_raw() and scanned. Implemented once, non-virtually, in
    // src/data/series.cpp. Preconditions: begin < end, end <= size().
    Range reduce(u64 begin, u64 end) const;
    f64   sum(u64 begin, u64 end) const;
    f64   sumsq(u64 begin, u64 end) const;

   protected:
    // Copies raw y samples for [begin, end) into `out`, which the caller sized to
    // (end - begin). Handles ring wraparound. Only ever called with small edge
    // spans by the queries above; not on a per-sample path.
    virtual void copy_raw(u64 begin, u64 end, f32* out) const = 0;
  };

  class ArraySeries final : public Series {                  // own storage, uniform dx
   public:
    ArraySeries(std::vector<f32> y, f64 x0, f64 dx);
    void reset(std::vector<f32> y, f64 x0, f64 dx);
    void commit();                                           // rebuild the pyramid
  };

  class StreamSeries final : public Series {                 // SPSC, producer + consumer
   public:
    explicit StreamSeries(u64 capacity, f64 x0, f64 dx);
    void push(f32 y);                                        // producer thread only
    void push(const f32* ys, u64 n);                         // producer thread only
    u64 produced() const;                                    // atomic, monotonic
    void acquire();                                          // consumer: snapshot + extend pyramid
  };
}

enum class AxisPlace { Bottom, Top, Left, Right };

struct AxisDesc { std::string label; Range range; bool auto_scale = true; AxisPlace place; bool log = false; };
struct TraceDesc { const data::Series* series; Color color; f32 width = 1.5f; u32 axis_x = 0, axis_y = 0; };

class Plot {
 public:
  u32 add_x_axis(AxisDesc); u32 add_y_axis(AxisDesc);
  u32 add_trace(TraceDesc);
  u32 add_cursor(f64 x, Color);
  u32 add_measure(u32 cursor_a, u32 cursor_b);
  void set_viewport(Rect px);
  void set_x_range(u32 axis, Range); void set_y_range(u32 axis, Range);
  void auto_fit_x(); void auto_fit_y(u32 axis);
  struct Measurement { f64 dx, dy, slope, rms, mean, ymin, ymax, integral; bool valid; };
  const Measurement& measurement(u32 id) const;              // recomputed on cursor move
};

class Renderer {
 public:
  bool init(Status* = nullptr);                              // context must be current
  void begin_frame(u32 fb_w, u32 fb_h, Rect viewport_px);
  void draw(Plot&);
  void end_frame();
  Style& style();
};

namespace input {
  enum class Button { Left, Middle, Right };
  enum class Mod { None = 0, Shift = 1, Ctrl = 2, Alt = 4, Super = 8 };
  struct PointerEvent { f32 x, y, dx, dy; Button button; u32 mods; bool down, pressed, released; };
  struct WheelEvent   { f32 x, y, dx, dy; u32 mods; };
  struct KeyEvent     { int key; u32 mods; bool down; };

  enum class DragMode { None, Pan, BoxZoom, MoveCursor, AxisZoomX, AxisZoomY };
  class Controller {
   public:
    explicit Controller(Plot&);
    void set_viewport(Rect);
    void on_pointer(const PointerEvent&);
    void on_wheel(const WheelEvent&);
    void on_key(const KeyEvent&);
    DragMode drag_mode() const;
    Rect selection_rect() const;                             // valid while BoxZoom
    u64 hovered_cursor() const;                              // kInvalid when none
  };
}
}  // namespace plot
```

## 7. Threading model

- All GL calls happen on the thread that has the context current; `Renderer` is
  not thread-safe and does not need to be.
- Producers write to `data::StreamSeries` from an acquisition thread through a
  lock-free SPSC ring. No mutex on the sample path.
- A frame begins with `StreamSeries::acquire()`, which publishes a consistent
  `[0, produced)` snapshot: it slides the pyramid's retained window to the new
  base, dropping the leading buckets the ring has overwritten, then extends the
  pyramid by exactly the newly produced samples. Its cost is **O(samples produced
  since the last acquire)** — independent of both the window size and the series
  length — and so is its memory, because the drops bound the pyramid to the
  retained window instead of the total ever pushed. A frame's total cost is
  therefore O(new samples + visible pixels), and no part of the frame path is
  proportional to the window. The one exception is a producer that laps the
  consumer by more than `capacity()` samples between two acquires: the samples in
  between were never appended, so that acquire re-reads the retained window from
  scratch. Readers use the snapshot; the producer never blocks.
- `Renderer`, `Plot`, and `Controller` are confined to the render thread.

## 8. Error handling

- Library-originated failures never throw: every one of them returns `Status`
  (or `bool` plus an out-parameter `Status*`), and no code path reports an error
  by throwing. An allocation failure is the host's process running out of memory,
  not a library error, and the entry points that allocate host memory --
  constructing a series, `Plot::add_*`, a frame that stages geometry,
  `Renderer::dump_frame_ppm()` -- propagate `std::bad_alloc` from the container
  that ran out. They are therefore not annotated `noexcept`; the entries that can
  neither fail nor allocate are, so the surface says per entry point what it does
  rather than claiming a blanket `noexcept`.
- GL failures are surfaced as `ErrorCode::GLFailure` from `Renderer::init` and
  recorded for the frame: `Renderer::end_frame()` drains `glGetError()` in a loop
  and sets the flag `last_frame_had_gl_error()` reports, so the first error cannot
  hide the ones after it and a stale error cannot be attributed to a later frame.
  The instance stores report their own refusals through the same flag: a storage,
  map or fence the driver refuses drops that store to the orphan-and-refill
  baseline (see §4.5) and marks the frame. There is no `PLOT_GL_CHECK()` macro and
  no build-dependent error checking -- the drain runs in every build, and it is
  the flag, not a macro, that a host observes.
- Unsupported-but-optional capabilities (compute, persistent mapping) degrade
  silently by selecting the 3.3 path; unsupported-and-required ones return
  `ErrorCode::Unsupported` from `init`.

## 9. Verification

| Layer | How it is verified |
|---|---|
| core/math, data, plot/ticks, input | CTest unit tests, no GL required: ring-buffer wrap/overwrite/tearing, pyramid build and `reduce()` vs. brute force, level selection, tick generation, transform round-trip, zoom-anchor invariance, pan invariance. |
| render, plot | **headless EGL surfaceless context + FBO**, rasterise a known sine, `glReadPixels` and assert ink present in expected columns and background elsewhere; assert the trace covers ≥95% of viewport width at 1M samples. |
| adapters | Launch each demo (`glfw`, `imgui`, `qt`, `gtk`) on the development display, screenshot, confirm grid + axes + trace + cursor + measurement readout render. |

Headless-EGL risk: if Mesa surfaceless context creation fails on this machine the
fallback is GLX on a hidden window; this is validated in the first GL task, before
three of the four adapters depend on it.

## 10. Milestones

1. Repo skeleton, CMake, core+math with tests.
2. GL loader, handles, capability probe; headless EGL harness proven.
3. Data layer: ring buffer, pyramid, series.
4. Render layer: programs, buffers, batching, bitmap text.
5. Plot layer: axes, ticks, grid, trace, cursor, measurement.
6. Input layer: events, controller.
7. Renderer + headless render smoke test (pixel assertions).
8. GLFW adapter + 1M-sample demo.
9. ImGui adapter + demo.
10. Qt6 adapter + demo.
11. GTK4 adapter + demo.
12. Install/export, full verification pass, screenshots.