fix(e2e): isolate coverage-pass WORK dir; filter chain-only e2e report section

The final whole-branch review (post Task 10) found two real cross-task
integration bugs:

- run_e2e.sh's coverage-instrumented scenario re-run only rebound OUT_DIR
  in its subshell, not WORK. proc_perf.py/plots.py write perf_*.json and
  wave_*.png into WORK, so every --cpp-coverage run (the default) silently
  clobbered the primary pass's perf/waveform data with the instrumented
  re-run's numbers before report_build.py read them -- defeating the
  "uncontaminated performance metrics" goal of the coverage-double-run
  design. Fixed by rebinding WORK the same way OUT_DIR already was.

- report_build.py's build_e2e() iterated all results["scenarios"] with no
  kind filter, so the direct/recorder/debug/tcplogger scenarios (already
  covered by their own dedicated report sections since Task 8) also leaked
  into the chain-only Scenarios/Performance sections and headline e2e
  pass/fail count as degenerate rows. Fixed by filtering to kind=="chain".

Verified end-to-end with a full ./run_e2e.sh run: coverage pass now writes
to /tmp/chain_e2e/coverage_pass/ (confirmed via log), and report_data.json's
e2e section now reports 51 (chain-only) scenarios instead of all 56.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Martino Ferrari
2026-07-01 23:59:54 +02:00
parent 07b6b4898a
commit 9a39cf923a
2 changed files with 13 additions and 5 deletions
+2
View File
@@ -74,6 +74,8 @@ def build_e2e(results, work):
scen = [] scen = []
corrs, rss_vals, cpu_vals, tput_vals = [], [], [], [] corrs, rss_vals, cpu_vals, tput_vals = [], [], [], []
for r in results.get("scenarios", []): for r in results.get("scenarios", []):
if r.get("kind", "chain") != "chain":
continue
sid = r["id"] sid = r["id"]
metrics = r.get("metrics", {}) metrics = r.get("metrics", {})
sigs = [] sigs = []
+11 -5
View File
@@ -452,14 +452,20 @@ if [ "${CPP_COV}" -eq 1 ]; then
# only hit via the direct/recorder/debug E2E flows). Re-running the full # only hit via the direct/recorder/debug E2E flows). Re-running the full
# scenario matrix here — against the SAME instrumented binaries, before # scenario matrix here — against the SAME instrumented binaries, before
# collect.py's lcov capture — lets those .gcda files accumulate E2E-path # collect.py's lcov capture — lets those .gcda files accumulate E2E-path
# coverage too. Run in a subshell with OUT_DIR overridden so the coverage # coverage too. Run in a subshell with OUT_DIR *and* WORK overridden so the
# pass's logs/results land in a discardable subdirectory and none of its # coverage pass's logs/results/perf-JSON/waveform-PNG scratch files land in
# SCEN_IDS/OUT_DIR/trap state leaks back into this shell (the authoritative # discardable subdirectories and none of its SCEN_IDS/OUT_DIR/WORK/trap
# results.json was already written above, from the primary pass). # state leaks back into this shell (the authoritative results.json was
# already written above, from the primary pass). WORK must be isolated too,
# not just OUT_DIR: run_scenario_matrix writes perf_*/wave_*.png/metrics_*
# under WORK, and report_build.py reads those files straight from WORK
# after this pass — without isolation the coverage-instrumented re-run
# would silently clobber the primary pass's perf/waveform data.
echo " re-running scenario matrix under instrumented binaries (coverage pass) ..." echo " re-running scenario matrix under instrumented binaries (coverage pass) ..."
( (
OUT_DIR="${OUT_DIR}/coverage_pass" OUT_DIR="${OUT_DIR}/coverage_pass"
mkdir -p "${OUT_DIR}" WORK="${WORK}/coverage_pass"
mkdir -p "${OUT_DIR}" "${WORK}"
run_scenario_matrix "coverage" run_scenario_matrix "coverage"
) || true ) || true