refactor(e2e): rename Test/E2E/chain -> Test/E2E/suite, run_chain_e2e.sh -> run_e2e.sh
This commit is contained in:
Martino Ferrari
@@ -89,7 +89,7 @@ clangd. The CMake-based clients (ImGui, Qt) also export it into their `build/`.
|
||||
cd Common/Client/go && go build ./...
|
||||
cd Client/debugger && go build ./...
|
||||
cd Client/udpstreamer && go build ./... # or: cd Client/webui && go build
|
||||
cd Test/E2E/chain/client && go build ./... # chain-client (E2E driver) + its tests
|
||||
cd Test/E2E/suite/client && go build ./... # chain-client (E2E driver) + its tests
|
||||
|
||||
# ImGui desktop client (needs SDL2; fetches Dear ImGui + ImPlot via FetchContent)
|
||||
cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build
|
||||
@@ -113,13 +113,13 @@ cd Client/streamhub-qt && cmake -B build && cmake --build build
|
||||
### Go tests
|
||||
|
||||
```bash
|
||||
cd Test/E2E/chain/client && go test ./...
|
||||
cd Test/E2E/suite/client && go test ./...
|
||||
```
|
||||
|
||||
### Python framework tests (E2E framework logic, no live stack needed)
|
||||
|
||||
```bash
|
||||
cd Test/E2E/chain && python3 -m unittest tests_py
|
||||
cd Test/E2E/suite && python3 -m unittest tests_py
|
||||
```
|
||||
|
||||
### E2E / demo scripts (build + launch the full stack)
|
||||
@@ -129,8 +129,8 @@ cd Test/E2E/chain && python3 -m unittest tests_py
|
||||
| `./run_streamhub.sh` | Launch MARTe2 app (UDPStreamer) + StreamHub, optionally web UI (`-w`) and ImGui client (`-g`). Ports documented in its header. |
|
||||
| `./run_combined_test.sh` | Combined streaming + debug integration test. `-d` auto-starts the debugger web UI. |
|
||||
| `./run_e2e_test.sh` | Older standalone StreamHub E2E (`-s` skips build). |
|
||||
| `./Test/E2E/chain/run_chain_e2e.sh` | **Streaming-chain E2E suite**: per-scenario generates data + cfgs, runs MARTe2+StreamHub, drives the Go `chain-client` (live/zoom/window/trigger), validates the recorded waveform against an analytic/fed oracle, runs unit suites + coverage, builds a Typst PDF report. Flags: `--skip-build`, `--only <id>`, `--cpp-coverage`, `--pdf-only`, `--stress`. |
|
||||
| `./Test/E2E/chain/run_stress.sh` | **Capacity/stress harness**: sweeps one load axis at a time (signal size/count, subscriber fan-out, source count, WS-client count, zoom rate), gates on survival+liveness (hard) and RSS+zoom-p95 latency (soft). Flags: `--skip-build`, `--only <id>`, `--axis <axis>`. |
|
||||
| `./Test/E2E/suite/run_e2e.sh` | **Streaming-chain E2E suite**: per-scenario generates data + cfgs, runs MARTe2+StreamHub, drives the Go `chain-client` (live/zoom/window/trigger), validates the recorded waveform against an analytic/fed oracle, runs unit suites + coverage, builds a Typst PDF report. Flags: `--skip-build`, `--only <id>`, `--cpp-coverage`, `--pdf-only`, `--stress`. |
|
||||
| `./Test/E2E/suite/run_stress.sh` | **Capacity/stress harness**: sweeps one load axis at a time (signal size/count, subscriber fan-out, source count, WS-client count, zoom rate), gates on survival+liveness (hard) and RSS+zoom-p95 latency (soft). Flags: `--skip-build`, `--only <id>`, `--axis <axis>`. |
|
||||
|
||||
The E2E suite produces `Build/x86-linux/E2E/chain/` artifacts: `report_data.json`,
|
||||
`history.jsonl` (per-run headline metrics for trend/regression tracking),
|
||||
@@ -246,7 +246,7 @@ default 1, `Decimation` default 1, `FlushIntervalSec` default 5,
|
||||
|
||||
### E2E scenario / stress matrix
|
||||
|
||||
`Test/E2E/chain/scenarios.py` is a *curated covering set*: every configurable
|
||||
`Test/E2E/suite/scenarios.py` is a *curated covering set*: every configurable
|
||||
UDPStreamer option value appears in at least one scenario, plus high-risk
|
||||
interactions. `stress.py` is the capacity sibling — it sweeps one load axis at
|
||||
a time and records survival/liveness (hard gates) and RSS/zoom-p95 latency
|
||||
|
||||
@@ -39,7 +39,7 @@ cd Client/streamhub-qt && cmake -B build && cmake --build build
|
||||
|
||||
End-to-end demo scripts (build + launch full stack, see headers for ports/options): `./run_combined_test.sh`, `./run_streamhub.sh`.
|
||||
|
||||
**Streaming-chain E2E suite** (`Test/E2E/chain/`): `./run_chain_e2e.sh [--skip-build] [--only <id>] [--cpp-coverage]` drives the full chain per scenario (`scenarios.py`) — generates typed/shaped input + both cfgs, runs MARTe2+StreamHub, records via the Go `chain-client` (live/zoom/window/trigger), and validates the recorded waveform against an analytic/fed oracle (`validate_waveform.py`: fidelity gates correctness, sine shape-fit is a gross-sanity gate + tracked metric pending Phase-A timestamp calibration). It then runs the unit suites + coverage (`collect.py`: C++ GTest, Go, Python; `--cpp-coverage` does an instrumented `--coverage` rebuild, captures with lcov restricted to `Source/*`+`Test/*`, then restores the clean build), consolidates everything into `report_data.json` with per-field progression/regression vs the previous run and trend plots (`report_build.py`, history in `Build/x86-linux/E2E/chain/history.jsonl`), and compiles a Typst PDF (`E2E_Report.typ`). Python framework unit tests: `python3 -m unittest tests_py` (in `Test/E2E/chain/`).
|
||||
**Streaming-chain E2E suite** (`Test/E2E/suite/`): `./run_e2e.sh [--skip-build] [--only <id>] [--cpp-coverage]` drives the full chain per scenario (`scenarios.py`) — generates typed/shaped input + both cfgs, runs MARTe2+StreamHub, records via the Go `chain-client` (live/zoom/window/trigger), and validates the recorded waveform against an analytic/fed oracle (`validate_waveform.py`: fidelity gates correctness, sine shape-fit is a gross-sanity gate + tracked metric pending Phase-A timestamp calibration). It then runs the unit suites + coverage (`collect.py`: C++ GTest, Go, Python; `--cpp-coverage` does an instrumented `--coverage` rebuild, captures with lcov restricted to `Source/*`+`Test/*`, then restores the clean build), consolidates everything into `report_data.json` with per-field progression/regression vs the previous run and trend plots (`report_build.py`, history in `Build/x86-linux/E2E/chain/history.jsonl`), and compiles a Typst PDF (`E2E_Report.typ`). Python framework unit tests: `python3 -m unittest tests_py` (in `Test/E2E/suite/`).
|
||||
|
||||
Build output goes to `Build/x86-linux/` (shared libs per component, `.ex` executables).
|
||||
|
||||
|
||||
BIN
Binary file not shown.
Binary file not shown.
@@ -109,7 +109,7 @@ def integration_suite(int_bin, work, timeout=220):
|
||||
def go_all_suites(repo, work):
|
||||
"""Run Go test suites across all project modules and aggregate results."""
|
||||
modules = [
|
||||
(os.path.join(repo, "Test/E2E/chain/client"),
|
||||
(os.path.join(repo, "Test/E2E/suite/client"),
|
||||
"Go (chain-client)"),
|
||||
(os.path.join(repo, "Common/Client/go"),
|
||||
"Go (common udpsprotocol + wshub)"),
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# run_chain_e2e.sh — Full-chain E2E orchestrator for the streaming chain
|
||||
# run_e2e.sh — Full-chain E2E orchestrator for the streaming chain
|
||||
#
|
||||
# MARTe2 app (FileReader -> IOGAM -> UDPStreamer)
|
||||
# -> UDPS -> StreamHub -> chain-client (record + zoom/window/trigger)
|
||||
@@ -10,7 +10,7 @@
|
||||
# against the analytic/fed oracle, renders plots, and aggregates results.json.
|
||||
# Artifacts: Build/x86-linux/E2E/chain/ (report) and /tmp/chain_e2e/ (scratch).
|
||||
#
|
||||
# Usage: ./run_chain_e2e.sh [--skip-build] [--only <id>] [--pdf-only]
|
||||
# Usage: ./run_e2e.sh [--skip-build] [--only <id>] [--pdf-only]
|
||||
set -u
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
@@ -88,8 +88,8 @@ fi
|
||||
# ── Scenario list (id|ws_port|udp_port0|network|oracle|trig|checks) ──────────
|
||||
LIST="$(${PY} - "${ONLY}" <<'PY'
|
||||
import sys, os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath("Test/E2E/chain/scenarios.py")))
|
||||
sys.path.insert(0, os.path.join(os.getcwd(), "Test/E2E/chain"))
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath("Test/E2E/suite/scenarios.py")))
|
||||
sys.path.insert(0, os.path.join(os.getcwd(), "Test/E2E/suite"))
|
||||
import scenarios as S
|
||||
only = sys.argv[1] if len(sys.argv) > 1 else ""
|
||||
for s in S.SCENARIOS:
|
||||
@@ -8,7 +8,7 @@
|
||||
# It sweeps one load axis at a time (signal size/count, subscriber fan-out, source
|
||||
# count, WS-client count, zoom request rate — see stress.py) and gates each case on
|
||||
# survival + liveness (hard) and RSS + zoom-p95 latency (soft). This is the
|
||||
# capacity sibling of run_chain_e2e.sh (which gates waveform correctness).
|
||||
# capacity sibling of run_e2e.sh (which gates waveform correctness).
|
||||
#
|
||||
# Usage: ./run_stress.sh [--skip-build] [--only <id>] [--axis <axis>]
|
||||
set -u
|
||||
@@ -2,7 +2,7 @@
|
||||
"""
|
||||
stress_run.py — Orchestrator for the streaming-chain stress matrix (stress.py).
|
||||
|
||||
Where run_chain_e2e.sh drives scenarios.py for *correctness*, this drives
|
||||
Where run_e2e.sh drives scenarios.py for *correctness*, this drives
|
||||
STRESS_CASES for *capacity*. Per case it:
|
||||
|
||||
1. generates the FileReader input + MARTe app cfg + 1..M StreamHub cfgs
|
||||
Reference in New Issue
Block a user