MARTe-Integrated-Components/docs/superpowers/specs/2026-06-26-stress-suite-report-integration-design.md

Stress Suite Report Integration — Design

Date: 2026-06-26

Context

A capacity/stress suite already exists in Test/E2E/chain/ but is untracked in git and not wired into the report:

• stress.py — declarative matrix, 27 cases across 7 load axes, all valid.
• stress_run.py — orchestrator: runs each case, captures cpu/peakRSS via proc_perf.snapshot(), zoom p50/p95 latency, gates survival+liveness (hard) and RSS/zoom-p95 (soft), writes stress_results.json.
• run_stress.sh — standalone wrapper (build + env + invoke stress_run.py).
• client/main.go — stress mode (liveness + zoom-latency recorder).

The suite is functional (smoke-tested ds_size_1000 → PASS end-to-end). The report pipeline (run_chain_e2e.sh, collect.py, report_build.py, E2E_Report.typ) has zero stress references.

This work covers the user's three requested stress axes — signal size, client count, source count — which already map onto existing matrix axes ({ds,hub}_signal_elements, hub_ws_clients / ds_subscriber_hubs, hub_source_count), plus the report integration.

Separately, the UDPSClient reassembly cap was just raised from 64 KB to 1 MiB (UDPS_CLIENT_MAX_PACKET_BYTES = 1048576), with exact assembledBytes delivery. The stress matrix still enforces a now-outdated sub-64 KB single-datagram ceiling.

Goals

1. Extend the signal-size axis into the multi-fragment regime (exercise the fixed reassembly path) and lift the outdated 64 KB validation cap.
2. Run the stress suite as an opt-in phase of run_chain_e2e.sh (--stress).
3. Surface stress results in the PDF: a table, per-axis scaling curves, and progression/regression vs the previous run (tracked in history.jsonl).

Non-goals: waveform fidelity under stress (UDP loss is expected; the hub decimates to PushRate), and changing the stress harness's gating semantics.

Design

1. Matrix extension (stress.py)

• Extend _DS_SIZE and _HUB_SIZE with multi-fragment cases: add 50000, 100000, 250000 elements (≈200 KB, 400 KB, ~1 MB float32 packets).
• Large packets get a slower producer to keep bandwidth realistic (mirroring s51's 100 Hz / 10 ms cycle): set producer_hz/row_dt per case so a ~1 MB packet is not emitted at 1 kHz (= 1 GB/s). sampling_rate stays elements / row_dt so the FirstSample per-cycle window equals one producer cycle (keeps the hub ring's time axis monotonic — same constraint scenarios.py enforces).
• Lift the single-datagram validation in validate_case() from 64 KB to the 1 MiB deliverable cap (UDPS_CLIENT_MAX_PACKET_BYTES = 1048576). Update the module docstring (lines 44–47) to reflect multi-fragment support.

2. Report data wiring (run_chain_e2e.sh, report_build.py)

• run_chain_e2e.sh gains --stress (default off). When set, after the correctness phase and before collect.py/report_build.py, it runs the stress phase, reusing run_stress.sh's env/LD_LIBRARY_PATH setup and the stress_run.py invocation, producing stress_results.json in Build/x86-linux/E2E/chain/stress/. Honors --skip-build.
• report_build.py reads stress_results.json if present and folds a stress block into report_data.json:
  • stress.overall, stress.cases[] (id, axis, level, status, survival, marte_cpu_s/rss, hub_cpu_s/rss, zoom p50/p95, fails).
  • stress.by_axis — cases grouped by axis, sorted by level (for the curves).
• When stress was not run, report_data.json has no stress block and the PDF omits the section (no stale data).

3. PDF section + scaling plots (report_build.py, E2E_Report.typ)

• New = Stress Tests section in E2E_Report.typ, after the Performance section.
• Table — one row per case: axis, level, status, marteCPU, marteRSS, hubRSS, zoom p50/p95 (matching the existing Performance table style).
• Scaling curves — report_build.py generates one PNG per axis (reusing its matplotlib trend-plot helper): load level (x) vs the relevant metric(s) — RSS & CPU vs level for size/count/sources, zoom-p95 vs reqrate, frames/RSS vs ws_clients. PNGs land alongside the existing trend plots and are embedded in the Typst section.

4. Regression + history (report_build.py)

• Append a stress summary to each history.jsonl run record: per-case key metrics (marte/hub RSS & CPU, zoom-p95, min_frames).
• Extend the _DIRECTION map: RSS/CPU/zoom-p95 → lower-is-better; min_frames → higher-is-better. Surface stress regressions in the existing headline/delta mechanism (▲/▼ vs last run), consistent with E2E metrics.
• First-run guard: when no prior stress history exists, show baseline values with no delta (existing is_first_run handling).

5. --stress flag plumbing (run_chain_e2e.sh)

• Add --stress to the arg parser; default off (correctness-only stays the fast default).
• Update the run_chain_e2e.sh header comment and the CLAUDE.md E2E paragraph to document --stress.

Full flow

run_chain_e2e.sh [--stress]
  ├─ correctness phase → results.json + perf JSON   (unchanged)
  ├─ [if --stress] stress phase → stress_results.json
  ├─ collect.py        → unit tests + coverage        (unchanged)
  └─ report_build.py   → report_data.json (+stress block, +history/regression)
        └─ E2E_Report.typ → PDF (Stress Tests: table + per-axis scaling curves)

Verification

• python3 stress.py validates the extended matrix (multi-fragment cases pass the lifted cap).
• A multi-fragment size case (e.g. ds_size_100000) runs PASS via run_stress.sh --only ds_size_100000 (exercises reassembly).
• run_chain_e2e.sh --stress --skip-build produces a PDF with the Stress Tests section (table + curves); a run without --stress omits it.
• Second --stress run shows progression/regression deltas vs the first.

Risks

• Bandwidth at large element counts — mitigated by the slower producer; verify the hub keeps up (survival gate) and doesn't OOM (RSS gate; ~1 MB/slot ×4 slots ×N sources).
• Runtime — 27+ cases at ~6–8 s each adds minutes; mitigated by opt-in --stress.
• history.jsonl already polluted by prior single-scenario runs — stress history starts fresh (new key); no rewrite of existing E2E history.