fix(e2e): isolate coverage-pass WORK dir; filter chain-only e2e report section
The final whole-branch review (post Task 10) found two real cross-task integration bugs: - run_e2e.sh's coverage-instrumented scenario re-run only rebound OUT_DIR in its subshell, not WORK. proc_perf.py/plots.py write perf_*.json and wave_*.png into WORK, so every --cpp-coverage run (the default) silently clobbered the primary pass's perf/waveform data with the instrumented re-run's numbers before report_build.py read them -- defeating the "uncontaminated performance metrics" goal of the coverage-double-run design. Fixed by rebinding WORK the same way OUT_DIR already was. - report_build.py's build_e2e() iterated all results["scenarios"] with no kind filter, so the direct/recorder/debug/tcplogger scenarios (already covered by their own dedicated report sections since Task 8) also leaked into the chain-only Scenarios/Performance sections and headline e2e pass/fail count as degenerate rows. Fixed by filtering to kind=="chain". Verified end-to-end with a full ./run_e2e.sh run: coverage pass now writes to /tmp/chain_e2e/coverage_pass/ (confirmed via log), and report_data.json's e2e section now reports 51 (chain-only) scenarios instead of all 56. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -74,6 +74,8 @@ def build_e2e(results, work):
|
||||
scen = []
|
||||
corrs, rss_vals, cpu_vals, tput_vals = [], [], [], []
|
||||
for r in results.get("scenarios", []):
|
||||
if r.get("kind", "chain") != "chain":
|
||||
continue
|
||||
sid = r["id"]
|
||||
metrics = r.get("metrics", {})
|
||||
sigs = []
|
||||
|
||||
@@ -452,14 +452,20 @@ if [ "${CPP_COV}" -eq 1 ]; then
|
||||
# only hit via the direct/recorder/debug E2E flows). Re-running the full
|
||||
# scenario matrix here — against the SAME instrumented binaries, before
|
||||
# collect.py's lcov capture — lets those .gcda files accumulate E2E-path
|
||||
# coverage too. Run in a subshell with OUT_DIR overridden so the coverage
|
||||
# pass's logs/results land in a discardable subdirectory and none of its
|
||||
# SCEN_IDS/OUT_DIR/trap state leaks back into this shell (the authoritative
|
||||
# results.json was already written above, from the primary pass).
|
||||
# coverage too. Run in a subshell with OUT_DIR *and* WORK overridden so the
|
||||
# coverage pass's logs/results/perf-JSON/waveform-PNG scratch files land in
|
||||
# discardable subdirectories and none of its SCEN_IDS/OUT_DIR/WORK/trap
|
||||
# state leaks back into this shell (the authoritative results.json was
|
||||
# already written above, from the primary pass). WORK must be isolated too,
|
||||
# not just OUT_DIR: run_scenario_matrix writes perf_*/wave_*.png/metrics_*
|
||||
# under WORK, and report_build.py reads those files straight from WORK
|
||||
# after this pass — without isolation the coverage-instrumented re-run
|
||||
# would silently clobber the primary pass's perf/waveform data.
|
||||
echo " re-running scenario matrix under instrumented binaries (coverage pass) ..."
|
||||
(
|
||||
OUT_DIR="${OUT_DIR}/coverage_pass"
|
||||
mkdir -p "${OUT_DIR}"
|
||||
WORK="${WORK}/coverage_pass"
|
||||
mkdir -p "${OUT_DIR}" "${WORK}"
|
||||
run_scenario_matrix "coverage"
|
||||
) || true
|
||||
|
||||
|
||||
Reference in New Issue
Block a user