plotter/docs/superpowers/specs/2026-09-27-plot-library-design.md

plot — embeddable OpenGL time-series plotting library

Date: 2026-09-27 Status: approved (design reviewed section-by-section with the requester)

Goal

A C++20 library that renders real-time, high-sample-rate time-series plots with grid, axes, cursors and measurements, and that can be embedded into an existing Qt6, GTK4, Dear ImGui, or raw GLFW application without owning the window, the GL context, or the event loop.

Non-goals (this cut)

• 3D surfaces, scatter/bar/pie charts, polar plots.
• Data I/O (CSV/HDF5 readers), file formats, network sources.
• Font rendering beyond the built-in bitmap atlas.
• Any host API not listed above.

These are explicitly out of scope so the first cut is a coherent vertical slice. The layering (§3) leaves room for them without redesign.

Constraints (fixed by the requester)

Constraint	Value
Core dependencies	zero. No GLM, glad, GLEW, or bundled Khronos headers.
GL baseline	3.3 core, with a runtime-selected 4.3+ fast path.
Reference host	GLFW, plus ImGui and Qt6 and GTK4 adapters in the same cut.
First deliverable	buildable end-to-end skeleton: no stub bodies on the demo path.
Language	C++20, GCC/Clang, Linux first (no portability blockers introduced).

1. Environment (verified on the development machine)

GCC 16.2.1, CMake 4.4.3, Ninja 1.13.2. Qt6 6.11.2 with Qt6OpenGLWidgets, GTK4 4.22.5 with gtkglarea.h, GLFW 3.5.1, EGL 1.5 + libEGL_mesa (surfaceless headless contexts available), glm and GL/glew.h installed but unused. Dear ImGui is absent and is provided via FetchContent (network verified).

2. The embeddability contract

The library never creates a window, never creates or shares a GL context, and never runs an event loop. Its render entry point is:

plot::Renderer r;              // constructed with the caller's context current
r.init();
r.begin_frame(fb_w, fb_h, viewport_rect_in_pixels);   // in the caller's FBO
r.draw(plot);
r.end_frame();

Input arrives as plain value types (PointerEvent, WheelEvent, KeyEvent) that each adapter translates from its host toolkit. Nothing in plot::core names a toolkit, so a consumer links only the adapter they need.

3. Layering

adapters/{glfw,imgui,qt6,gtk4}   own the surface; translate host events
        ↓
plot/     Plot · Axis · Trace · Grid · Cursor · Measure · Annotation
        ↓
render/   Renderer · Style · TextBackend · primitive batches
        ↓
data/     Series · ArraySeries · StreamSeries · MinMaxPyramid
        ↓
gl/       runtime loader · RAII handles · capability probe
        ↓
core/ + math/   types · Color · Rect · vec2 · mat3 · Transform

Each layer depends only downward. adapters/* and examples/* are optional CMake targets guarded by PLOT_BUILD_* options and by dependency discovery.

4. Performance architecture

These four decisions are what make 1M+ samples/frame viable. Each replaces an approach that fails at high sample rates.

4.1 Data-space vertices, uniform-only pan and zoom

Vertex buffers hold raw data coordinates. Screen mapping is x' = x * sx + tx, y' = y * sy + ty evaluated in the vertex shader from a Transform uniform. The CPU never transforms a vertex, and a pan or zoom changes one uniform and no vertex values.

The staged window is view-dependent: each frame the renderer gathers just the visible sample range (or the visible buckets of the chosen LOD level) and uploads it. That per-frame cost is O(visible pixels) — about two segments per pixel column, flat in series length — so the property this section exists for holds. A staging cache keyed on (series, level, first, last, viewport) would let a static plot skip the redundant gather and upload entirely; it is not implemented, because it changes no asymptotics and would add an invalidation surface (new samples, LOD changes, series mutation, streaming window slide) whose bugs are worse than the redundant work it removes. Recorded here as deliberate future work, not as a missing feature.

Log axes are handled inside the same transform (log10 before scaling) so no separate code path or pre-transformed buffer is needed.

4.2 Min/max LOD pyramid

Every series owns a mip chain: level 0 buckets kBucket = 64 consecutive samples and stores one entry per bucket; each subsequent level halves the bucket count. Each entry is 24 bytes — a MinMax pair (8 bytes) plus the exact sum and sumsq doubles the range queries fold over (16 bytes) — so level 0 costs 24n/64 = 0.375n bytes and the geometric sum 1 + ½ + ¼ + … brings the pyramid to ≈0.75n bytes, about 19% of the 4n bytes of raw samples, not 2×. A min/max-only index would be n/4, ≈6%: the two doubles are the trade that makes an interior sum/mean/RMS/integral O(log n) whole-bucket additions instead of a scan of raw samples, at every level, for every series that uses the measurement index.

• Built incrementally, in absolute bucket space: level l bucket k covers samples [k · (kBucket << l), (k + 1) · (kBucket << l)). A bucket's span therefore never depends on where the current window starts, which is what lets a window slide over a pyramid instead of forcing it to be rebuilt. append() folds a sample into the single in-progress level-0 bucket and, exactly when that bucket completes, pushes it and merges upward, stopping at the first level that gains no new entry: level l gains one entry per kBucket << l samples, so an append costs amortised O(1) (1/kBucket + 1/(2·kBucket) + … = 2/kBucket merges) and never walks the live levels.
• A sliding window (a StreamSeries ring) drops the leading buckets it no longer holds and appends only the newly produced samples, so nothing is recomputed and memory stays bounded by the retained window rather than the total ever pushed.
• The renderer picks the level where samples-per-pixel ≈ 2, so a 100M-sample series rasterises ≈ 2 · viewport_width vertices instead of 100M.
• Buckets preserve extremes, so spikes are never dropped by decimation.

4.3 One rasteriser, two sampling levels

Every trace, every frame, is drawn by one path: instanced anti-aliased quads. One instance per segment, four corners expanded from gl_VertexID in the vertex shader, a fragment shader that does distance-to-segment SDF for smooth edges and joins. There is no second primitive anywhere in the renderer — in particular no GL_LINE_STRIP path — so a sparse trace and a dense one differ in the geometry staged, never in the call that draws it.

What the samples-per-pixel threshold selects is therefore the gather level:

samples/px	geometry staged
< dense_threshold_spp (1.5)	the visible window's raw samples: a level-0 Series::gather(), a polyline through the window, one instance per sample pair
>= dense_threshold_spp (1.5)	the pyramid level whose buckets are ~1 per pixel (pyramid().level_for(n, samples_per_pixel)): a min/max envelope pair per bucket, so O(pixels) instances instead of O(samples)

Style::dense_threshold_spp is that threshold and nothing more: it picks the gather level, changes no draw call, and the two sides of it are the same shader. The dense side is where the work is saved (a 100M-sample window stages O(pixels) vertices); the sparse side stages the window itself, which at ~1 sample per pixel is ~1 instance per column anyway.

A geometry-shader expansion path was rejected: poor performance and it is the one stage macOS cannot be relied on for.

4.4 Interval reduction reuses the pyramid

Cursor measurements (Δx, Δy, slope, RMS, mean, min/max, integral over a dragged region) are answered from the same pyramid instead of by scanning the series. Series::reduce/sum/sumsq cover the bucket-aligned interior with pyramid aggregates and scan at most two partial edge buckets (≤ 2 × kBucket = 128 samples), so a query's cost is independent of series length. Series::reduce is the single implementation used by measurement, auto-scale and hit-testing.

The split matters: a min/max pyramid stores aggregates per bucket, so it can only answer exactly for whole buckets — a range with partial edge buckets would otherwise silently include samples outside the range. Exactness therefore lives on Series (which owns the raw samples), and MinMaxPyramid exposes only bucket-exact queries, named *_buckets so they cannot be mistaken for exact ones.

The bucket grid is absolute while a sliding window's base is not, so that split is computed in absolute coordinates — Series asks the pyramid for window_base() and aligns ceil/floor there before mapping back to window indices. That is also how a stream's y_bounds() stays exact in O(log n + kBucket) per frame: it is reduce(0, size()) over the window rather than a scan of it.

4.5 Decimation approach — decision

	Approach	Verdict
A	CPU min/max pyramid, incremental	chosen — works on GL 3.3, deterministic, unit-testable with no GPU, O(1) amortised per sample, and it doubles as the measurement index. Cost: one CPU pass per sample, plus ≈19% memory (24-byte buckets: extremes plus exact per-bucket sums).
B	GPU compute decimation every frame	rejected — re-reads the whole series each frame (real bandwidth at 1M samples/frame), no 3.3 fallback, untestable headlessly.
C	no decimation	rejected — dies above ~500k samples.

The 4.3+ fast path therefore applies to transfer, not to decimation: a persistent-mapped, glFenceSync-guarded triple-buffered ring replaces the staging-upload path. Both paths share identical semantics and are selected by the capability probe.

5. Zero-dependency mechanisms

• Math — hand-written vec2, mat3, Transform: ~250 lines, no templates in public headers, no consumer-visible dependency.
• GL loader — a hand-maintained X-macro list PLOT_GL_FUNCS(X) covering only the ≈60 entry points actually used, with our own typedefs and enums. No glcorearb.h bundling and no code-generation step. Resolution order: context-provided getProcAddress (glX / EGL / wgl) → dlsym/GetProcAddress on the system GL library.
• Text — a TextBackend interface plus a built-in 8×16 bitmap-glyph atlas. Axis labelling needs no third-party code; a FreeType or stb_truetype backend can be added behind the same interface without touching callers.

There is no third_party/ directory. Adapters use find_package first and FetchContent only as a fallback (ImGui; GLFW when absent).

6. Public API sketch

Signatures here are the contract the implementation plan and adapters rely on.

namespace plot {

using f32 = float;  using f64 = double;
using u32 = std::uint32_t;  using u64 = std::uint64_t;
constexpr u64 kInvalid = ~u64{0};

struct Range { f64 min, max; };
struct Rect  { f32 x, y, w, h; };            // pixels, origin top-left
struct Color { f32 r, g, b, a; static Color hex(u32 rgb, f32 a = 1.0f); };

enum class ErrorCode { Ok, InvalidArgument, OutOfMemory, GLFailure, Unsupported, NotImplemented };
struct Status { ErrorCode code = ErrorCode::Ok; const char* message = ""; explicit operator bool() const; };

// math/transform.hpp
struct Transform {
  f64 sx, tx, sy, ty; bool log_x = false, log_y = false;
  f64 to_screen_x(f64) const; f64 to_data_x(f64) const;
  f64 to_screen_y(f64) const; f64 to_data_y(f64) const;
};

namespace gl {
  using ProcLoader = void* (*)(const char*);
  bool load(ProcLoader);                 // nullptr → platform default
  struct Caps { int major, minor; bool compute, persistent_map, buffer_storage; };
  const Caps& caps();
  class Buffer; class VertexArray; class Program; class Texture; class Framebuffer; class Fence;
}

namespace data {
  struct MinMax { f32 min, max; };
  constexpr u32 kBucket = 64;

  class MinMaxPyramid {
   public:
    void reserve(u64 n); void append(f32 v); void clear();
    // Absolute bucket space: level l entry i covers the absolute samples
    // [i * (kBucket << l), (i + 1) * (kBucket << l)). The window slides over the
    // same buckets; queries are in window coordinates (index 0 = oldest retained).
    void drop_before(u64 absolute_sample);
    u64 size() const;                                        // retained window size
    u64 window_base() const;                                 // abs index of index 0
    u64 window_end() const;                                  // abs index past the last
    u32 level_count() const;
    u64 level_base(u32 l) const, level_size(u32 l) const;
    MinMax bucket(u32 l, u64 absolute_bucket) const;
    u32 level_for(u64 n, f64 samples_per_pixel) const;
    u64 snap(f64 x, f64 x0, f64 dx) const;                   // nearest sample index
    // Whole-bucket only: aggregates over the level-0 buckets fully contained in
    // [begin, end). Partial edges are excluded, not approximated. Callers wanting
    // an exact answer over an arbitrary range use Series::reduce/sum/sumsq.
    Range reduce_buckets(u64 begin, u64 end) const;
    f64   sum_buckets(u64 begin, u64 end) const;
    f64   sumsq_buckets(u64 begin, u64 end) const;
  };

  class Series {                                             // abstract
   public:
    virtual ~Series() = default;
    virtual u64 size() const = 0;
    virtual f64 x0() const = 0;                                // x of sample 0
    virtual f64 dx() const = 0;                                // uniform x spacing
    virtual Range x_bounds() const = 0;
    virtual Range y_bounds() const = 0;
    virtual const MinMaxPyramid& pyramid() const = 0;
    virtual void gather(u32 level, u64 begin, u64 end, std::vector<f32>& out_xy) const = 0;

    // Exact, O(kBucket) worst case regardless of series length: the bucket-aligned
    // interior comes from the pyramid, the at-most-two partial edge buckets are
    // read through copy_raw() and scanned. Implemented once, non-virtually, in
    // src/data/series.cpp. Preconditions: begin < end, end <= size().
    Range reduce(u64 begin, u64 end) const;
    f64   sum(u64 begin, u64 end) const;
    f64   sumsq(u64 begin, u64 end) const;

   protected:
    // Copies raw y samples for [begin, end) into `out`, which the caller sized to
    // (end - begin). Handles ring wraparound. Only ever called with small edge
    // spans by the queries above; not on a per-sample path.
    virtual void copy_raw(u64 begin, u64 end, f32* out) const = 0;
  };

  class ArraySeries final : public Series {                  // own storage, uniform dx
   public:
    ArraySeries(std::vector<f32> y, f64 x0, f64 dx);
    void reset(std::vector<f32> y, f64 x0, f64 dx);
    void commit();                                           // rebuild the pyramid
  };

  class StreamSeries final : public Series {                 // SPSC, producer + consumer
   public:
    explicit StreamSeries(u64 capacity, f64 x0, f64 dx);
    void push(f32 y);                                        // producer thread only
    void push(const f32* ys, u64 n);                         // producer thread only
    u64 produced() const;                                    // atomic, monotonic
    void acquire();                                          // consumer: snapshot + extend pyramid
  };
}

enum class AxisPlace { Bottom, Top, Left, Right };

struct AxisDesc { std::string label; Range range; bool auto_scale = true; AxisPlace place; bool log = false; };
struct TraceDesc { const data::Series* series; Color color; f32 width = 1.5f; u32 axis_x = 0, axis_y = 0; };

class Plot {
 public:
  u32 add_x_axis(AxisDesc); u32 add_y_axis(AxisDesc);
  u32 add_trace(TraceDesc);
  u32 add_cursor(f64 x, Color);
  u32 add_measure(u32 cursor_a, u32 cursor_b);
  void set_viewport(Rect px);
  void set_x_range(u32 axis, Range); void set_y_range(u32 axis, Range);
  void auto_fit_x(); void auto_fit_y(u32 axis);
  struct Measurement { f64 dx, dy, slope, rms, mean, ymin, ymax, integral; bool valid; };
  const Measurement& measurement(u32 id) const;              // recomputed on cursor move
};

class Renderer {
 public:
  bool init(Status* = nullptr);                              // context must be current
  void begin_frame(u32 fb_w, u32 fb_h, Rect viewport_px);
  void draw(Plot&);
  void end_frame();
  Style& style();
};

namespace input {
  enum class Button { Left, Middle, Right };
  enum class Mod { None = 0, Shift = 1, Ctrl = 2, Alt = 4, Super = 8 };
  struct PointerEvent { f32 x, y, dx, dy; Button button; u32 mods; bool down, pressed, released; };
  struct WheelEvent   { f32 x, y, dx, dy; u32 mods; };
  struct KeyEvent     { int key; u32 mods; bool down; };

  enum class DragMode { None, Pan, BoxZoom, MoveCursor, AxisZoomX, AxisZoomY };
  class Controller {
   public:
    explicit Controller(Plot&);
    void set_viewport(Rect);
    void on_pointer(const PointerEvent&);
    void on_wheel(const WheelEvent&);
    void on_key(const KeyEvent&);
    DragMode drag_mode() const;
    Rect selection_rect() const;                             // valid while BoxZoom
    u64 hovered_cursor() const;                              // kInvalid when none
  };
}
}  // namespace plot

7. Threading model

• All GL calls happen on the thread that has the context current; Renderer is not thread-safe and does not need to be.
• Producers write to data::StreamSeries from an acquisition thread through a lock-free SPSC ring. No mutex on the sample path.
• A frame begins with StreamSeries::acquire(), which publishes a consistent [0, produced) snapshot: it slides the pyramid's retained window to the new base, dropping the leading buckets the ring has overwritten, then extends the pyramid by exactly the newly produced samples. Its cost is O(samples produced since the last acquire) — independent of both the window size and the series length — and so is its memory, because the drops bound the pyramid to the retained window instead of the total ever pushed. A frame's total cost is therefore O(new samples + visible pixels), and no part of the frame path is proportional to the window. The one exception is a producer that laps the consumer by more than capacity() samples between two acquires: the samples in between were never appended, so that acquire re-reads the retained window from scratch. Readers use the snapshot; the producer never blocks.
• Renderer, Plot, and Controller are confined to the render thread.

8. Error handling

• Library-originated failures never throw: every one of them returns Status (or bool plus an out-parameter Status*), and no code path reports an error by throwing. An allocation failure is the host's process running out of memory, not a library error, and the entry points that allocate host memory -- constructing a series, Plot::add_*, a frame that stages geometry, Renderer::dump_frame_ppm() -- propagate std::bad_alloc from the container that ran out. They are therefore not annotated noexcept; the entries that can neither fail nor allocate are, so the surface says per entry point what it does rather than claiming a blanket noexcept.
• GL failures are surfaced as ErrorCode::GLFailure from Renderer::init and recorded for the frame: Renderer::end_frame() drains glGetError() in a loop and sets the flag last_frame_had_gl_error() reports, so the first error cannot hide the ones after it and a stale error cannot be attributed to a later frame. The instance stores report their own refusals through the same flag: a storage, map or fence the driver refuses drops that store to the orphan-and-refill baseline (see §4.5) and marks the frame. There is no PLOT_GL_CHECK() macro and no build-dependent error checking -- the drain runs in every build, and it is the flag, not a macro, that a host observes.
• Unsupported-but-optional capabilities (compute, persistent mapping) degrade silently by selecting the 3.3 path; unsupported-and-required ones return ErrorCode::Unsupported from init.

9. Verification

Layer	How it is verified
core/math, data, plot/ticks, input	CTest unit tests, no GL required: ring-buffer wrap/overwrite/tearing, pyramid build and reduce() vs. brute force, level selection, tick generation, transform round-trip, zoom-anchor invariance, pan invariance.
render, plot	headless EGL surfaceless context + FBO, rasterise a known sine, glReadPixels and assert ink present in expected columns and background elsewhere; assert the trace covers ≥95% of viewport width at 1M samples.
adapters	Launch each demo (glfw, imgui, qt, gtk) on the development display, screenshot, confirm grid + axes + trace + cursor + measurement readout render.

Headless-EGL risk: if Mesa surfaceless context creation fails on this machine the fallback is GLX on a hidden window; this is validated in the first GL task, before three of the four adapters depend on it.

10. Milestones

1. Repo skeleton, CMake, core+math with tests.
2. GL loader, handles, capability probe; headless EGL harness proven.
3. Data layer: ring buffer, pyramid, series.
4. Render layer: programs, buffers, batching, bitmap text.
5. Plot layer: axes, ticks, grid, trace, cursor, measurement.
6. Input layer: events, controller.
7. Renderer + headless render smoke test (pixel assertions).
8. GLFW adapter + 1M-sample demo.
9. ImGui adapter + demo.
10. Qt6 adapter + demo.
11. GTK4 adapter + demo.
12. Install/export, full verification pass, screenshots.