refactor(e2e): rename Test/E2E/chain -> Test/E2E/suite, run_chain_e2e.sh -> run_e2e.sh
This commit is contained in:
1 parent
1fcc4e4e6d
commit
69e52af20b
30 files changed
+14
-14
No files matched your search
@@ -89,7 +89,7 @@ clangd. The CMake-based clients (ImGui, Qt) also export it into their `build/`.
|
|||||||
cd Common/Client/go && go build ./...
|
cd Common/Client/go && go build ./...
|
||||||
cd Client/debugger && go build ./...
|
cd Client/debugger && go build ./...
|
||||||
cd Client/udpstreamer && go build ./... # or: cd Client/webui && go build
|
cd Client/udpstreamer && go build ./... # or: cd Client/webui && go build
|
||||||
cd Test/E2E/chain/client && go build ./... # chain-client (E2E driver) + its tests
|
cd Test/E2E/suite/client && go build ./... # chain-client (E2E driver) + its tests
|
||||||
|
|
||||||
# ImGui desktop client (needs SDL2; fetches Dear ImGui + ImPlot via FetchContent)
|
# ImGui desktop client (needs SDL2; fetches Dear ImGui + ImPlot via FetchContent)
|
||||||
cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build
|
cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build
|
||||||
@@ -113,13 +113,13 @@ cd Client/streamhub-qt && cmake -B build && cmake --build build
|
|||||||
### Go tests
|
### Go tests
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd Test/E2E/chain/client && go test ./...
|
cd Test/E2E/suite/client && go test ./...
|
||||||
```
|
```
|
||||||
|
|
||||||
### Python framework tests (E2E framework logic, no live stack needed)
|
### Python framework tests (E2E framework logic, no live stack needed)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd Test/E2E/chain && python3 -m unittest tests_py
|
cd Test/E2E/suite && python3 -m unittest tests_py
|
||||||
```
|
```
|
||||||
|
|
||||||
### E2E / demo scripts (build + launch the full stack)
|
### E2E / demo scripts (build + launch the full stack)
|
||||||
@@ -129,8 +129,8 @@ cd Test/E2E/chain && python3 -m unittest tests_py
|
|||||||
| `./run_streamhub.sh` | Launch MARTe2 app (UDPStreamer) + StreamHub, optionally web UI (`-w`) and ImGui client (`-g`). Ports documented in its header. |
|
| `./run_streamhub.sh` | Launch MARTe2 app (UDPStreamer) + StreamHub, optionally web UI (`-w`) and ImGui client (`-g`). Ports documented in its header. |
|
||||||
| `./run_combined_test.sh` | Combined streaming + debug integration test. `-d` auto-starts the debugger web UI. |
|
| `./run_combined_test.sh` | Combined streaming + debug integration test. `-d` auto-starts the debugger web UI. |
|
||||||
| `./run_e2e_test.sh` | Older standalone StreamHub E2E (`-s` skips build). |
|
| `./run_e2e_test.sh` | Older standalone StreamHub E2E (`-s` skips build). |
|
||||||
| `./Test/E2E/chain/run_chain_e2e.sh` | **Streaming-chain E2E suite**: per-scenario generates data + cfgs, runs MARTe2+StreamHub, drives the Go `chain-client` (live/zoom/window/trigger), validates the recorded waveform against an analytic/fed oracle, runs unit suites + coverage, builds a Typst PDF report. Flags: `--skip-build`, `--only <id>`, `--cpp-coverage`, `--pdf-only`, `--stress`. |
|
| `./Test/E2E/suite/run_e2e.sh` | **Streaming-chain E2E suite**: per-scenario generates data + cfgs, runs MARTe2+StreamHub, drives the Go `chain-client` (live/zoom/window/trigger), validates the recorded waveform against an analytic/fed oracle, runs unit suites + coverage, builds a Typst PDF report. Flags: `--skip-build`, `--only <id>`, `--cpp-coverage`, `--pdf-only`, `--stress`. |
|
||||||
| `./Test/E2E/chain/run_stress.sh` | **Capacity/stress harness**: sweeps one load axis at a time (signal size/count, subscriber fan-out, source count, WS-client count, zoom rate), gates on survival+liveness (hard) and RSS+zoom-p95 latency (soft). Flags: `--skip-build`, `--only <id>`, `--axis <axis>`. |
|
| `./Test/E2E/suite/run_stress.sh` | **Capacity/stress harness**: sweeps one load axis at a time (signal size/count, subscriber fan-out, source count, WS-client count, zoom rate), gates on survival+liveness (hard) and RSS+zoom-p95 latency (soft). Flags: `--skip-build`, `--only <id>`, `--axis <axis>`. |
|
||||||
|
|
||||||
The E2E suite produces `Build/x86-linux/E2E/chain/` artifacts: `report_data.json`,
|
The E2E suite produces `Build/x86-linux/E2E/chain/` artifacts: `report_data.json`,
|
||||||
`history.jsonl` (per-run headline metrics for trend/regression tracking),
|
`history.jsonl` (per-run headline metrics for trend/regression tracking),
|
||||||
@@ -246,7 +246,7 @@ default 1, `Decimation` default 1, `FlushIntervalSec` default 5,
|
|||||||
|
|
||||||
### E2E scenario / stress matrix
|
### E2E scenario / stress matrix
|
||||||
|
|
||||||
`Test/E2E/chain/scenarios.py` is a *curated covering set*: every configurable
|
`Test/E2E/suite/scenarios.py` is a *curated covering set*: every configurable
|
||||||
UDPStreamer option value appears in at least one scenario, plus high-risk
|
UDPStreamer option value appears in at least one scenario, plus high-risk
|
||||||
interactions. `stress.py` is the capacity sibling — it sweeps one load axis at
|
interactions. `stress.py` is the capacity sibling — it sweeps one load axis at
|
||||||
a time and records survival/liveness (hard gates) and RSS/zoom-p95 latency
|
a time and records survival/liveness (hard gates) and RSS/zoom-p95 latency
|
||||||
|
|||||||
@@ -39,7 +39,7 @@ cd Client/streamhub-qt && cmake -B build && cmake --build build
|
|||||||
|
|
||||||
End-to-end demo scripts (build + launch full stack, see headers for ports/options): `./run_combined_test.sh`, `./run_streamhub.sh`.
|
End-to-end demo scripts (build + launch full stack, see headers for ports/options): `./run_combined_test.sh`, `./run_streamhub.sh`.
|
||||||
|
|
||||||
**Streaming-chain E2E suite** (`Test/E2E/chain/`): `./run_chain_e2e.sh [--skip-build] [--only <id>] [--cpp-coverage]` drives the full chain per scenario (`scenarios.py`) — generates typed/shaped input + both cfgs, runs MARTe2+StreamHub, records via the Go `chain-client` (live/zoom/window/trigger), and validates the recorded waveform against an analytic/fed oracle (`validate_waveform.py`: fidelity gates correctness, sine shape-fit is a gross-sanity gate + tracked metric pending Phase-A timestamp calibration). It then runs the unit suites + coverage (`collect.py`: C++ GTest, Go, Python; `--cpp-coverage` does an instrumented `--coverage` rebuild, captures with lcov restricted to `Source/*`+`Test/*`, then restores the clean build), consolidates everything into `report_data.json` with per-field progression/regression vs the previous run and trend plots (`report_build.py`, history in `Build/x86-linux/E2E/chain/history.jsonl`), and compiles a Typst PDF (`E2E_Report.typ`). Python framework unit tests: `python3 -m unittest tests_py` (in `Test/E2E/chain/`).
|
**Streaming-chain E2E suite** (`Test/E2E/suite/`): `./run_e2e.sh [--skip-build] [--only <id>] [--cpp-coverage]` drives the full chain per scenario (`scenarios.py`) — generates typed/shaped input + both cfgs, runs MARTe2+StreamHub, records via the Go `chain-client` (live/zoom/window/trigger), and validates the recorded waveform against an analytic/fed oracle (`validate_waveform.py`: fidelity gates correctness, sine shape-fit is a gross-sanity gate + tracked metric pending Phase-A timestamp calibration). It then runs the unit suites + coverage (`collect.py`: C++ GTest, Go, Python; `--cpp-coverage` does an instrumented `--coverage` rebuild, captures with lcov restricted to `Source/*`+`Test/*`, then restores the clean build), consolidates everything into `report_data.json` with per-field progression/regression vs the previous run and trend plots (`report_build.py`, history in `Build/x86-linux/E2E/chain/history.jsonl`), and compiles a Typst PDF (`E2E_Report.typ`). Python framework unit tests: `python3 -m unittest tests_py` (in `Test/E2E/suite/`).
|
||||||
|
|
||||||
Build output goes to `Build/x86-linux/` (shared libs per component, `.ex` executables).
|
Build output goes to `Build/x86-linux/` (shared libs per component, `.ex` executables).
|
||||||
|
|
||||||
|
|||||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
BIN
Binary file not shown.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Binary file not shown.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -109,7 +109,7 @@ def integration_suite(int_bin, work, timeout=220):
|
|||||||
def go_all_suites(repo, work):
|
def go_all_suites(repo, work):
|
||||||
"""Run Go test suites across all project modules and aggregate results."""
|
"""Run Go test suites across all project modules and aggregate results."""
|
||||||
modules = [
|
modules = [
|
||||||
(os.path.join(repo, "Test/E2E/chain/client"),
|
(os.path.join(repo, "Test/E2E/suite/client"),
|
||||||
"Go (chain-client)"),
|
"Go (chain-client)"),
|
||||||
(os.path.join(repo, "Common/Client/go"),
|
(os.path.join(repo, "Common/Client/go"),
|
||||||
"Go (common udpsprotocol + wshub)"),
|
"Go (common udpsprotocol + wshub)"),
|
||||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# run_chain_e2e.sh — Full-chain E2E orchestrator for the streaming chain
|
# run_e2e.sh — Full-chain E2E orchestrator for the streaming chain
|
||||||
#
|
#
|
||||||
# MARTe2 app (FileReader -> IOGAM -> UDPStreamer)
|
# MARTe2 app (FileReader -> IOGAM -> UDPStreamer)
|
||||||
# -> UDPS -> StreamHub -> chain-client (record + zoom/window/trigger)
|
# -> UDPS -> StreamHub -> chain-client (record + zoom/window/trigger)
|
||||||
@@ -10,7 +10,7 @@
|
|||||||
# against the analytic/fed oracle, renders plots, and aggregates results.json.
|
# against the analytic/fed oracle, renders plots, and aggregates results.json.
|
||||||
# Artifacts: Build/x86-linux/E2E/chain/ (report) and /tmp/chain_e2e/ (scratch).
|
# Artifacts: Build/x86-linux/E2E/chain/ (report) and /tmp/chain_e2e/ (scratch).
|
||||||
#
|
#
|
||||||
# Usage: ./run_chain_e2e.sh [--skip-build] [--only <id>] [--pdf-only]
|
# Usage: ./run_e2e.sh [--skip-build] [--only <id>] [--pdf-only]
|
||||||
set -u
|
set -u
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
@@ -88,8 +88,8 @@ fi
|
|||||||
# ── Scenario list (id|ws_port|udp_port0|network|oracle|trig|checks) ──────────
|
# ── Scenario list (id|ws_port|udp_port0|network|oracle|trig|checks) ──────────
|
||||||
LIST="$(${PY} - "${ONLY}" <<'PY'
|
LIST="$(${PY} - "${ONLY}" <<'PY'
|
||||||
import sys, os
|
import sys, os
|
||||||
sys.path.insert(0, os.path.dirname(os.path.abspath("Test/E2E/chain/scenarios.py")))
|
sys.path.insert(0, os.path.dirname(os.path.abspath("Test/E2E/suite/scenarios.py")))
|
||||||
sys.path.insert(0, os.path.join(os.getcwd(), "Test/E2E/chain"))
|
sys.path.insert(0, os.path.join(os.getcwd(), "Test/E2E/suite"))
|
||||||
import scenarios as S
|
import scenarios as S
|
||||||
only = sys.argv[1] if len(sys.argv) > 1 else ""
|
only = sys.argv[1] if len(sys.argv) > 1 else ""
|
||||||
for s in S.SCENARIOS:
|
for s in S.SCENARIOS:
|
||||||
@@ -8,7 +8,7 @@
|
|||||||
# It sweeps one load axis at a time (signal size/count, subscriber fan-out, source
|
# It sweeps one load axis at a time (signal size/count, subscriber fan-out, source
|
||||||
# count, WS-client count, zoom request rate — see stress.py) and gates each case on
|
# count, WS-client count, zoom request rate — see stress.py) and gates each case on
|
||||||
# survival + liveness (hard) and RSS + zoom-p95 latency (soft). This is the
|
# survival + liveness (hard) and RSS + zoom-p95 latency (soft). This is the
|
||||||
# capacity sibling of run_chain_e2e.sh (which gates waveform correctness).
|
# capacity sibling of run_e2e.sh (which gates waveform correctness).
|
||||||
#
|
#
|
||||||
# Usage: ./run_stress.sh [--skip-build] [--only <id>] [--axis <axis>]
|
# Usage: ./run_stress.sh [--skip-build] [--only <id>] [--axis <axis>]
|
||||||
set -u
|
set -u
|
||||||
File renamed without changes.
File renamed without changes.
@@ -2,7 +2,7 @@
|
|||||||
"""
|
"""
|
||||||
stress_run.py — Orchestrator for the streaming-chain stress matrix (stress.py).
|
stress_run.py — Orchestrator for the streaming-chain stress matrix (stress.py).
|
||||||
|
|
||||||
Where run_chain_e2e.sh drives scenarios.py for *correctness*, this drives
|
Where run_e2e.sh drives scenarios.py for *correctness*, this drives
|
||||||
STRESS_CASES for *capacity*. Per case it:
|
STRESS_CASES for *capacity*. Per case it:
|
||||||
|
|
||||||
1. generates the FileReader input + MARTe app cfg + 1..M StreamHub cfgs
|
1. generates the FileReader input + MARTe app cfg + 1..M StreamHub cfgs
|
||||||
File renamed without changes.
File renamed without changes.
Reference in new issue
Block a user