MARTe-Integrated-Components/docs/superpowers/specs/2026-06-26-stress-suite-report-integration-design.md

# Stress Suite Report Integration — Design

Date: 2026-06-26

## Context

A capacity/stress suite already exists in `Test/E2E/chain/` but is **untracked in
git** and **not wired into the report**:

- `stress.py` — declarative matrix, 27 cases across 7 load axes, all valid.
- `stress_run.py` — orchestrator: runs each case, captures cpu/peakRSS via
  `proc_perf.snapshot()`, zoom p50/p95 latency, gates survival+liveness (hard) and
  RSS/zoom-p95 (soft), writes `stress_results.json`.
- `run_stress.sh` — standalone wrapper (build + env + invoke `stress_run.py`).
- `client/main.go` — `stress` mode (liveness + zoom-latency recorder).

The suite is functional (smoke-tested `ds_size_1000` → PASS end-to-end). The report
pipeline (`run_chain_e2e.sh`, `collect.py`, `report_build.py`, `E2E_Report.typ`) has
**zero** stress references.

This work covers the user's three requested stress axes — **signal size**, **client
count**, **source count** — which already map onto existing matrix axes
(`{ds,hub}_signal_elements`, `hub_ws_clients` / `ds_subscriber_hubs`,
`hub_source_count`), plus the report integration.

Separately, the UDPSClient reassembly cap was just raised from 64 KB to 1 MiB
(`UDPS_CLIENT_MAX_PACKET_BYTES = 1048576`), with exact `assembledBytes` delivery.
The stress matrix still enforces a now-outdated sub-64 KB single-datagram ceiling.

## Goals

1. Extend the signal-size axis into the multi-fragment regime (exercise the fixed
   reassembly path) and lift the outdated 64 KB validation cap.
2. Run the stress suite as an opt-in phase of `run_chain_e2e.sh` (`--stress`).
3. Surface stress results in the PDF: a table, per-axis scaling curves, and
   progression/regression vs the previous run (tracked in `history.jsonl`).

Non-goals: waveform fidelity under stress (UDP loss is expected; the hub decimates
to PushRate), and changing the stress harness's gating semantics.

## Design

### 1. Matrix extension (`stress.py`)

- Extend `_DS_SIZE` and `_HUB_SIZE` with multi-fragment cases: add `50000`,
  `100000`, `250000` elements (≈200 KB, 400 KB, ~1 MB float32 packets).
- Large packets get a **slower producer** to keep bandwidth realistic (mirroring
  s51's 100 Hz / 10 ms cycle): set `producer_hz`/`row_dt` per case so a ~1 MB packet
  is not emitted at 1 kHz (= 1 GB/s). `sampling_rate` stays `elements / row_dt` so
  the FirstSample per-cycle window equals one producer cycle (keeps the hub ring's
  time axis monotonic — same constraint `scenarios.py` enforces).
- Lift the single-datagram validation in `validate_case()` from 64 KB to the 1 MiB
  deliverable cap (`UDPS_CLIENT_MAX_PACKET_BYTES = 1048576`). Update the module
  docstring (lines 44–47) to reflect multi-fragment support.

### 2. Report data wiring (`run_chain_e2e.sh`, `report_build.py`)

- `run_chain_e2e.sh` gains `--stress` (default off). When set, after the correctness
  phase and before `collect.py`/`report_build.py`, it runs the stress phase, reusing
  `run_stress.sh`'s env/LD_LIBRARY_PATH setup and the `stress_run.py` invocation,
  producing `stress_results.json` in `Build/x86-linux/E2E/chain/stress/`. Honors
  `--skip-build`.
- `report_build.py` reads `stress_results.json` *if present* and folds a `stress`
  block into `report_data.json`:
  - `stress.overall`, `stress.cases[]` (id, axis, level, status, survival,
    marte_cpu_s/rss, hub_cpu_s/rss, zoom p50/p95, fails).
  - `stress.by_axis` — cases grouped by axis, sorted by level (for the curves).
- When stress was not run, `report_data.json` has no `stress` block and the PDF omits
  the section (no stale data).

### 3. PDF section + scaling plots (`report_build.py`, `E2E_Report.typ`)

- New `= Stress Tests` section in `E2E_Report.typ`, after the Performance section.
- **Table** — one row per case: axis, level, status, marteCPU, marteRSS, hubRSS,
  zoom p50/p95 (matching the existing Performance table style).
- **Scaling curves** — `report_build.py` generates one PNG per axis (reusing its
  matplotlib trend-plot helper): load level (x) vs the relevant metric(s) — RSS & CPU
  vs level for size/count/sources, zoom-p95 vs reqrate, frames/RSS vs ws_clients.
  PNGs land alongside the existing trend plots and are embedded in the Typst section.

### 4. Regression + history (`report_build.py`)

- Append a `stress` summary to each `history.jsonl` run record: per-case key metrics
  (marte/hub RSS & CPU, zoom-p95, min_frames).
- Extend the `_DIRECTION` map: RSS/CPU/zoom-p95 → lower-is-better; min_frames →
  higher-is-better. Surface stress regressions in the existing headline/delta
  mechanism (▲/▼ vs last run), consistent with E2E metrics.
- First-run guard: when no prior stress history exists, show baseline values with no
  delta (existing `is_first_run` handling).

### 5. `--stress` flag plumbing (`run_chain_e2e.sh`)

- Add `--stress` to the arg parser; default off (correctness-only stays the fast
  default).
- Update the `run_chain_e2e.sh` header comment and the `CLAUDE.md` E2E paragraph to
  document `--stress`.

## Full flow

```
run_chain_e2e.sh [--stress]
  ├─ correctness phase → results.json + perf JSON   (unchanged)
  ├─ [if --stress] stress phase → stress_results.json
  ├─ collect.py        → unit tests + coverage        (unchanged)
  └─ report_build.py   → report_data.json (+stress block, +history/regression)
        └─ E2E_Report.typ → PDF (Stress Tests: table + per-axis scaling curves)
```

## Verification

- `python3 stress.py` validates the extended matrix (multi-fragment cases pass the
  lifted cap).
- A multi-fragment size case (e.g. `ds_size_100000`) runs PASS via `run_stress.sh
  --only ds_size_100000` (exercises reassembly).
- `run_chain_e2e.sh --stress --skip-build` produces a PDF with the Stress Tests
  section (table + curves); a run without `--stress` omits it.
- Second `--stress` run shows progression/regression deltas vs the first.

## Risks

- **Bandwidth at large element counts** — mitigated by the slower producer; verify the
  hub keeps up (survival gate) and doesn't OOM (RSS gate; ~1 MB/slot ×4 slots ×N
  sources).
- **Runtime** — 27+ cases at ~6–8 s each adds minutes; mitigated by opt-in `--stress`.
- **history.jsonl already polluted** by prior single-scenario runs — stress history
  starts fresh (new key); no rewrite of existing E2E history.