diff --git a/AGENTS.md b/AGENTS.md index 6f4d1ca..1a74526 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -89,7 +89,7 @@ clangd. The CMake-based clients (ImGui, Qt) also export it into their `build/`. cd Common/Client/go && go build ./... cd Client/debugger && go build ./... cd Client/udpstreamer && go build ./... # or: cd Client/webui && go build -cd Test/E2E/chain/client && go build ./... # chain-client (E2E driver) + its tests +cd Test/E2E/suite/client && go build ./... # chain-client (E2E driver) + its tests # ImGui desktop client (needs SDL2; fetches Dear ImGui + ImPlot via FetchContent) cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build @@ -113,13 +113,13 @@ cd Client/streamhub-qt && cmake -B build && cmake --build build ### Go tests ```bash -cd Test/E2E/chain/client && go test ./... +cd Test/E2E/suite/client && go test ./... ``` ### Python framework tests (E2E framework logic, no live stack needed) ```bash -cd Test/E2E/chain && python3 -m unittest tests_py +cd Test/E2E/suite && python3 -m unittest tests_py ``` ### E2E / demo scripts (build + launch the full stack) @@ -129,8 +129,8 @@ cd Test/E2E/chain && python3 -m unittest tests_py | `./run_streamhub.sh` | Launch MARTe2 app (UDPStreamer) + StreamHub, optionally web UI (`-w`) and ImGui client (`-g`). Ports documented in its header. | | `./run_combined_test.sh` | Combined streaming + debug integration test. `-d` auto-starts the debugger web UI. | | `./run_e2e_test.sh` | Older standalone StreamHub E2E (`-s` skips build). | -| `./Test/E2E/chain/run_chain_e2e.sh` | **Streaming-chain E2E suite**: per-scenario generates data + cfgs, runs MARTe2+StreamHub, drives the Go `chain-client` (live/zoom/window/trigger), validates the recorded waveform against an analytic/fed oracle, runs unit suites + coverage, builds a Typst PDF report. Flags: `--skip-build`, `--only `, `--cpp-coverage`, `--pdf-only`, `--stress`. | -| `./Test/E2E/chain/run_stress.sh` | **Capacity/stress harness**: sweeps one load axis at a time (signal size/count, subscriber fan-out, source count, WS-client count, zoom rate), gates on survival+liveness (hard) and RSS+zoom-p95 latency (soft). Flags: `--skip-build`, `--only `, `--axis `. | +| `./Test/E2E/suite/run_e2e.sh` | **Streaming-chain E2E suite**: per-scenario generates data + cfgs, runs MARTe2+StreamHub, drives the Go `chain-client` (live/zoom/window/trigger), validates the recorded waveform against an analytic/fed oracle, runs unit suites + coverage, builds a Typst PDF report. Flags: `--skip-build`, `--only `, `--cpp-coverage`, `--pdf-only`, `--stress`. | +| `./Test/E2E/suite/run_stress.sh` | **Capacity/stress harness**: sweeps one load axis at a time (signal size/count, subscriber fan-out, source count, WS-client count, zoom rate), gates on survival+liveness (hard) and RSS+zoom-p95 latency (soft). Flags: `--skip-build`, `--only `, `--axis `. | The E2E suite produces `Build/x86-linux/E2E/chain/` artifacts: `report_data.json`, `history.jsonl` (per-run headline metrics for trend/regression tracking), @@ -246,7 +246,7 @@ default 1, `Decimation` default 1, `FlushIntervalSec` default 5, ### E2E scenario / stress matrix -`Test/E2E/chain/scenarios.py` is a *curated covering set*: every configurable +`Test/E2E/suite/scenarios.py` is a *curated covering set*: every configurable UDPStreamer option value appears in at least one scenario, plus high-risk interactions. `stress.py` is the capacity sibling — it sweeps one load axis at a time and records survival/liveness (hard gates) and RSS/zoom-p95 latency diff --git a/CLAUDE.md b/CLAUDE.md index 7bf7c26..72213b6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -39,7 +39,7 @@ cd Client/streamhub-qt && cmake -B build && cmake --build build End-to-end demo scripts (build + launch full stack, see headers for ports/options): `./run_combined_test.sh`, `./run_streamhub.sh`. -**Streaming-chain E2E suite** (`Test/E2E/chain/`): `./run_chain_e2e.sh [--skip-build] [--only ] [--cpp-coverage]` drives the full chain per scenario (`scenarios.py`) — generates typed/shaped input + both cfgs, runs MARTe2+StreamHub, records via the Go `chain-client` (live/zoom/window/trigger), and validates the recorded waveform against an analytic/fed oracle (`validate_waveform.py`: fidelity gates correctness, sine shape-fit is a gross-sanity gate + tracked metric pending Phase-A timestamp calibration). It then runs the unit suites + coverage (`collect.py`: C++ GTest, Go, Python; `--cpp-coverage` does an instrumented `--coverage` rebuild, captures with lcov restricted to `Source/*`+`Test/*`, then restores the clean build), consolidates everything into `report_data.json` with per-field progression/regression vs the previous run and trend plots (`report_build.py`, history in `Build/x86-linux/E2E/chain/history.jsonl`), and compiles a Typst PDF (`E2E_Report.typ`). Python framework unit tests: `python3 -m unittest tests_py` (in `Test/E2E/chain/`). +**Streaming-chain E2E suite** (`Test/E2E/suite/`): `./run_e2e.sh [--skip-build] [--only ] [--cpp-coverage]` drives the full chain per scenario (`scenarios.py`) — generates typed/shaped input + both cfgs, runs MARTe2+StreamHub, records via the Go `chain-client` (live/zoom/window/trigger), and validates the recorded waveform against an analytic/fed oracle (`validate_waveform.py`: fidelity gates correctness, sine shape-fit is a gross-sanity gate + tracked metric pending Phase-A timestamp calibration). It then runs the unit suites + coverage (`collect.py`: C++ GTest, Go, Python; `--cpp-coverage` does an instrumented `--coverage` rebuild, captures with lcov restricted to `Source/*`+`Test/*`, then restores the clean build), consolidates everything into `report_data.json` with per-field progression/regression vs the previous run and trend plots (`report_build.py`, history in `Build/x86-linux/E2E/chain/history.jsonl`), and compiles a Typst PDF (`E2E_Report.typ`). Python framework unit tests: `python3 -m unittest tests_py` (in `Test/E2E/suite/`). Build output goes to `Build/x86-linux/` (shared libs per component, `.ex` executables). diff --git a/Test/E2E/chain/E2E_Report.typ b/Test/E2E/suite/E2E_Report.typ similarity index 100% rename from Test/E2E/chain/E2E_Report.typ rename to Test/E2E/suite/E2E_Report.typ diff --git a/Test/E2E/chain/__pycache__/collect.cpython-314.pyc b/Test/E2E/suite/__pycache__/collect.cpython-314.pyc similarity index 100% rename from Test/E2E/chain/__pycache__/collect.cpython-314.pyc rename to Test/E2E/suite/__pycache__/collect.cpython-314.pyc diff --git a/Test/E2E/chain/__pycache__/gen_cfg.cpython-314.pyc b/Test/E2E/suite/__pycache__/gen_cfg.cpython-314.pyc similarity index 100% rename from Test/E2E/chain/__pycache__/gen_cfg.cpython-314.pyc rename to Test/E2E/suite/__pycache__/gen_cfg.cpython-314.pyc diff --git a/Test/E2E/chain/__pycache__/gen_data.cpython-314.pyc b/Test/E2E/suite/__pycache__/gen_data.cpython-314.pyc similarity index 100% rename from Test/E2E/chain/__pycache__/gen_data.cpython-314.pyc rename to Test/E2E/suite/__pycache__/gen_data.cpython-314.pyc diff --git a/Test/E2E/chain/__pycache__/proc_perf.cpython-314.pyc b/Test/E2E/suite/__pycache__/proc_perf.cpython-314.pyc similarity index 100% rename from Test/E2E/chain/__pycache__/proc_perf.cpython-314.pyc rename to Test/E2E/suite/__pycache__/proc_perf.cpython-314.pyc diff --git a/Test/E2E/chain/__pycache__/scenarios.cpython-314.pyc b/Test/E2E/suite/__pycache__/scenarios.cpython-314.pyc similarity index 100% rename from Test/E2E/chain/__pycache__/scenarios.cpython-314.pyc rename to Test/E2E/suite/__pycache__/scenarios.cpython-314.pyc diff --git a/Test/E2E/chain/__pycache__/stress.cpython-314.pyc b/Test/E2E/suite/__pycache__/stress.cpython-314.pyc similarity index 99% rename from Test/E2E/chain/__pycache__/stress.cpython-314.pyc rename to Test/E2E/suite/__pycache__/stress.cpython-314.pyc index 89d39dc..b26b95b 100644 Binary files a/Test/E2E/chain/__pycache__/stress.cpython-314.pyc and b/Test/E2E/suite/__pycache__/stress.cpython-314.pyc differ diff --git a/Test/E2E/chain/__pycache__/stress_run.cpython-314.pyc b/Test/E2E/suite/__pycache__/stress_run.cpython-314.pyc similarity index 100% rename from Test/E2E/chain/__pycache__/stress_run.cpython-314.pyc rename to Test/E2E/suite/__pycache__/stress_run.cpython-314.pyc diff --git a/Test/E2E/chain/__pycache__/tests_py.cpython-314.pyc b/Test/E2E/suite/__pycache__/tests_py.cpython-314.pyc similarity index 100% rename from Test/E2E/chain/__pycache__/tests_py.cpython-314.pyc rename to Test/E2E/suite/__pycache__/tests_py.cpython-314.pyc diff --git a/Test/E2E/chain/__pycache__/validate_waveform.cpython-314.pyc b/Test/E2E/suite/__pycache__/validate_waveform.cpython-314.pyc similarity index 100% rename from Test/E2E/chain/__pycache__/validate_waveform.cpython-314.pyc rename to Test/E2E/suite/__pycache__/validate_waveform.cpython-314.pyc diff --git a/Test/E2E/chain/client/chain-client b/Test/E2E/suite/client/chain-client similarity index 95% rename from Test/E2E/chain/client/chain-client rename to Test/E2E/suite/client/chain-client index 12c6287..1b7c984 100755 Binary files a/Test/E2E/chain/client/chain-client and b/Test/E2E/suite/client/chain-client differ diff --git a/Test/E2E/chain/client/go.mod b/Test/E2E/suite/client/go.mod similarity index 100% rename from Test/E2E/chain/client/go.mod rename to Test/E2E/suite/client/go.mod diff --git a/Test/E2E/chain/client/go.sum b/Test/E2E/suite/client/go.sum similarity index 100% rename from Test/E2E/chain/client/go.sum rename to Test/E2E/suite/client/go.sum diff --git a/Test/E2E/chain/client/main.go b/Test/E2E/suite/client/main.go similarity index 100% rename from Test/E2E/chain/client/main.go rename to Test/E2E/suite/client/main.go diff --git a/Test/E2E/chain/client/main_test.go b/Test/E2E/suite/client/main_test.go similarity index 100% rename from Test/E2E/chain/client/main_test.go rename to Test/E2E/suite/client/main_test.go diff --git a/Test/E2E/chain/collect.py b/Test/E2E/suite/collect.py similarity index 99% rename from Test/E2E/chain/collect.py rename to Test/E2E/suite/collect.py index cd354c9..d42535b 100644 --- a/Test/E2E/chain/collect.py +++ b/Test/E2E/suite/collect.py @@ -109,7 +109,7 @@ def integration_suite(int_bin, work, timeout=220): def go_all_suites(repo, work): """Run Go test suites across all project modules and aggregate results.""" modules = [ - (os.path.join(repo, "Test/E2E/chain/client"), + (os.path.join(repo, "Test/E2E/suite/client"), "Go (chain-client)"), (os.path.join(repo, "Common/Client/go"), "Go (common udpsprotocol + wshub)"), diff --git a/Test/E2E/chain/gen_cfg.py b/Test/E2E/suite/gen_cfg.py similarity index 100% rename from Test/E2E/chain/gen_cfg.py rename to Test/E2E/suite/gen_cfg.py diff --git a/Test/E2E/chain/gen_data.py b/Test/E2E/suite/gen_data.py similarity index 100% rename from Test/E2E/chain/gen_data.py rename to Test/E2E/suite/gen_data.py diff --git a/Test/E2E/chain/plots.py b/Test/E2E/suite/plots.py similarity index 100% rename from Test/E2E/chain/plots.py rename to Test/E2E/suite/plots.py diff --git a/Test/E2E/chain/proc_perf.py b/Test/E2E/suite/proc_perf.py similarity index 100% rename from Test/E2E/chain/proc_perf.py rename to Test/E2E/suite/proc_perf.py diff --git a/Test/E2E/chain/report_build.py b/Test/E2E/suite/report_build.py similarity index 100% rename from Test/E2E/chain/report_build.py rename to Test/E2E/suite/report_build.py diff --git a/Test/E2E/chain/run_chain_e2e.sh b/Test/E2E/suite/run_e2e.sh similarity index 97% rename from Test/E2E/chain/run_chain_e2e.sh rename to Test/E2E/suite/run_e2e.sh index 7b222a6..9add131 100755 --- a/Test/E2E/chain/run_chain_e2e.sh +++ b/Test/E2E/suite/run_e2e.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# run_chain_e2e.sh — Full-chain E2E orchestrator for the streaming chain +# run_e2e.sh — Full-chain E2E orchestrator for the streaming chain # # MARTe2 app (FileReader -> IOGAM -> UDPStreamer) # -> UDPS -> StreamHub -> chain-client (record + zoom/window/trigger) @@ -10,7 +10,7 @@ # against the analytic/fed oracle, renders plots, and aggregates results.json. # Artifacts: Build/x86-linux/E2E/chain/ (report) and /tmp/chain_e2e/ (scratch). # -# Usage: ./run_chain_e2e.sh [--skip-build] [--only ] [--pdf-only] +# Usage: ./run_e2e.sh [--skip-build] [--only ] [--pdf-only] set -u SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -88,8 +88,8 @@ fi # ── Scenario list (id|ws_port|udp_port0|network|oracle|trig|checks) ────────── LIST="$(${PY} - "${ONLY}" <<'PY' import sys, os -sys.path.insert(0, os.path.dirname(os.path.abspath("Test/E2E/chain/scenarios.py"))) -sys.path.insert(0, os.path.join(os.getcwd(), "Test/E2E/chain")) +sys.path.insert(0, os.path.dirname(os.path.abspath("Test/E2E/suite/scenarios.py"))) +sys.path.insert(0, os.path.join(os.getcwd(), "Test/E2E/suite")) import scenarios as S only = sys.argv[1] if len(sys.argv) > 1 else "" for s in S.SCENARIOS: diff --git a/Test/E2E/chain/run_stress.sh b/Test/E2E/suite/run_stress.sh similarity index 97% rename from Test/E2E/chain/run_stress.sh rename to Test/E2E/suite/run_stress.sh index 499cb14..fd78068 100755 --- a/Test/E2E/chain/run_stress.sh +++ b/Test/E2E/suite/run_stress.sh @@ -8,7 +8,7 @@ # It sweeps one load axis at a time (signal size/count, subscriber fan-out, source # count, WS-client count, zoom request rate — see stress.py) and gates each case on # survival + liveness (hard) and RSS + zoom-p95 latency (soft). This is the -# capacity sibling of run_chain_e2e.sh (which gates waveform correctness). +# capacity sibling of run_e2e.sh (which gates waveform correctness). # # Usage: ./run_stress.sh [--skip-build] [--only ] [--axis ] set -u diff --git a/Test/E2E/chain/scenarios.py b/Test/E2E/suite/scenarios.py similarity index 100% rename from Test/E2E/chain/scenarios.py rename to Test/E2E/suite/scenarios.py diff --git a/Test/E2E/chain/stress.py b/Test/E2E/suite/stress.py similarity index 100% rename from Test/E2E/chain/stress.py rename to Test/E2E/suite/stress.py diff --git a/Test/E2E/chain/stress_run.py b/Test/E2E/suite/stress_run.py similarity index 99% rename from Test/E2E/chain/stress_run.py rename to Test/E2E/suite/stress_run.py index 95761b1..338db7e 100755 --- a/Test/E2E/chain/stress_run.py +++ b/Test/E2E/suite/stress_run.py @@ -2,7 +2,7 @@ """ stress_run.py — Orchestrator for the streaming-chain stress matrix (stress.py). -Where run_chain_e2e.sh drives scenarios.py for *correctness*, this drives +Where run_e2e.sh drives scenarios.py for *correctness*, this drives STRESS_CASES for *capacity*. Per case it: 1. generates the FileReader input + MARTe app cfg + 1..M StreamHub cfgs diff --git a/Test/E2E/chain/tests_py.py b/Test/E2E/suite/tests_py.py similarity index 100% rename from Test/E2E/chain/tests_py.py rename to Test/E2E/suite/tests_py.py diff --git a/Test/E2E/chain/validate_waveform.py b/Test/E2E/suite/validate_waveform.py similarity index 100% rename from Test/E2E/chain/validate_waveform.py rename to Test/E2E/suite/validate_waveform.py