Martino Ferrari
f132a5ec92
`remove_region()` now clears or decrements the published `hot_.region` with the cache it re-indexes, so a frame after a removal highlights the region the handle actually names. The new case covers both branches (a handle above the hole follows it down, the removed region's handle is cleared with its zone), and the coverage document is regenerated by the command that measured it: 95.2% of src/, every file at or above 80%.
162 lines
9.0 KiB
Markdown
162 lines
9.0 KiB
Markdown
# Coverage
|
|
|
|
`plot` measures line coverage of the compiled library bodies under `src/` and enforces a floor on
|
|
it, so a regression fails a command instead of quietly lowering a number in a document.
|
|
|
|
**The table and the floor verdict below are generated.** `tools/coverage.sh` rewrites the region
|
|
between `<!-- BEGIN GENERATED: tools/coverage.sh -->` and
|
|
`<!-- END GENERATED: tools/coverage.sh -->` from the report that run just captured, by calling
|
|
`tools/coverage_floor.py --write-doc`. Do not edit inside those markers: the numbers there come
|
|
from the command, not from a hand-typed copy, and a run on an unchanged tree rewrites the region
|
|
with identical content.
|
|
|
|
## The one command
|
|
|
|
```sh
|
|
tools/coverage.sh # from the repository root
|
|
cmake --build build/cov --target coverage # the same thing, through CMake
|
|
```
|
|
|
|
The script configures `build/cov` from the `linux-coverage` preset, builds it, runs the whole
|
|
`ctest` suite, captures `src/**` with `lcov` into `build/cov/coverage.info`, renders the HTML report
|
|
to `build/cov/html/index.html`, prints the per-file table, rewrites the generated region of this
|
|
file, and **exits non-zero when a floor is violated**. It needs `cmake`, `ninja`, `ctest`, `lcov`,
|
|
`genhtml` and `python3` (Arch: `pacman -S cmake ninja lcov python`); it exits `2` with a message
|
|
when one is missing.
|
|
|
|
## The floors
|
|
|
|
| Scope | Floor | Enforced by |
|
|
| --- | --- | --- |
|
|
| Every file under `src/` | **80%** of its instrumented lines | `tools/coverage_floor.py` (`--per-file`, default 80) |
|
|
| `src/` as a whole | **90%** of instrumented lines | `tools/coverage_floor.py` (`--aggregate`, default 90) |
|
|
|
|
`tools/coverage_floor.py build/cov/coverage.info` reads the lcov tracefile on its own and exits:
|
|
|
|
* `0` — every floor holds;
|
|
* `1` — a floor is violated; every offending file is printed with its rate and counts;
|
|
* `2` — the report cannot be read at all (missing file, or no `src/` records in it), because a
|
|
report this tool cannot find must never read as a pass.
|
|
|
|
## The measured table
|
|
|
|
<!-- BEGIN GENERATED: tools/coverage.sh -->
|
|
|
|
```
|
|
coverage_floor: ok -- 23 files under src/, aggregate 95.2% lines (3302/3469), every file at or above 80%, aggregate target 90%
|
|
```
|
|
|
|
| File | Lines | Line rate | Functions |
|
|
| --- | ---: | ---: | ---: |
|
|
| `src/data/array_series.cpp` | 43/43 | 100.0% | 100.0% |
|
|
| `src/data/minmax_pyramid.cpp` | 163/163 | 100.0% | 100.0% |
|
|
| `src/data/series.cpp` | 105/107 | 98.1% | 100.0% |
|
|
| `src/data/stream_series.cpp` | 39/39 | 100.0% | 100.0% |
|
|
| `src/gl/handle.cpp` | 256/256 | 100.0% | 100.0% |
|
|
| `src/gl/loader.cpp` | 81/81 | 100.0% | 100.0% |
|
|
| `src/input/controller.cpp` | 329/343 | 95.9% | 100.0% |
|
|
| `src/math/transform.cpp` | 28/28 | 100.0% | 100.0% |
|
|
| `src/plot/axis.cpp` | 12/12 | 100.0% | 100.0% |
|
|
| `src/plot/figure.cpp` | 218/223 | 97.8% | 95.5% |
|
|
| `src/plot/grid.cpp` | 15/16 | 93.8% | 100.0% |
|
|
| `src/plot/measure.cpp` | 21/21 | 100.0% | 100.0% |
|
|
| `src/plot/plot.cpp` | 743/763 | 97.4% | 98.9% |
|
|
| `src/plot/ticks.cpp` | 93/99 | 93.9% | 100.0% |
|
|
| `src/render/buffer.cpp` | 88/98 | 89.8% | 100.0% |
|
|
| `src/render/embedded_mono_font.cpp` | 45/45 | 100.0% | 100.0% |
|
|
| `src/render/primitives.cpp` | 71/75 | 94.7% | 100.0% |
|
|
| `src/render/program.cpp` | 26/27 | 96.3% | 100.0% |
|
|
| `src/render/renderer.cpp` | 530/609 | 87.0% | 93.7% |
|
|
| `src/render/text.cpp` | 27/30 | 90.0% | 100.0% |
|
|
| `src/render/text_atlas.cpp` | 40/41 | 97.6% | 100.0% |
|
|
| `src/render/vertex_array.cpp` | 59/59 | 100.0% | 100.0% |
|
|
| `src/text/true_type_font.cpp` | 270/291 | 92.8% | 96.8% |
|
|
| **src/ aggregate** | **3302/3469** | **95.2%** | **98.3%** |
|
|
|
|
<!-- END GENERATED: tools/coverage.sh -->
|
|
|
|
## What the report measures
|
|
|
|
* `src/**` — the compiled library bodies. These are what a caller of `plot::core` executes.
|
|
* Lines and functions. **Branch coverage is off** (`--rc branch_coverage=0`): the data-structure and
|
|
GL code is dense with short-circuit conditions whose branch counts mostly measure how the tests
|
|
were written, so a branch floor would be noise rather than a contract.
|
|
* Three of the 26 translation units produce no record at all, because gcov only writes counters for
|
|
code that runs: `src/core/status.cpp` (`status.o` is never pulled out of `libplot_core.a` — no
|
|
test calls `plot::to_string`), `src/render/shaders.cpp` (string literals and inline definitions),
|
|
and `src/text/stb_truetype_impl.cpp` (its lines belong to the vendored
|
|
`third_party/stb_truetype.h`, which is filtered out). Should a record ever arrive with `LF:0`, the
|
|
checker exempts that file from the per-file floor rather than dividing by zero.
|
|
* Not measured: the public headers under `include/plot/` (inline and template code, exercised
|
|
through their callers but not held to a per-file rate); `tests/` and the generated header-check
|
|
translation units; `adapters/` — only the two header-only seams the unit suite drives appear in a
|
|
raw capture, and the adapter bodies need a window, which the demos cover and this report does not;
|
|
and the C++ standard library headers gcov also records.
|
|
|
|
## Reproducibility
|
|
|
|
gcov's counters accumulate in `.gcda` files, so a run that inherits the previous run's counters
|
|
reports a superset of what it actually exercised — the classic reason a coverage number drifts on a
|
|
second run with no source change, and here it is measurable: `build/cov` before this script existed
|
|
had been reused across revisions, and its accumulated counters reported **93.0% (3206/3447)** for
|
|
the same tree that a clean single run reports as **92.9% (3203/3447)**. Three lines of hit in
|
|
`plot.cpp`, `renderer.cpp` and `buffer.cpp` were left over from an earlier build.
|
|
|
|
`tools/coverage.sh` therefore deletes every `.gcda` under the build tree
|
|
(`find "$BUILD" -name '*.gcda' -delete`) **before** it runs the suite, so the captured counters are
|
|
exactly the ones that run produced. `lcov --zerocounters --directory "$BUILD"` is the equivalent
|
|
call — on lcov 2.5 it unlinks the `.gcda` files and reports success even on a tree that has none
|
|
(checked) — and the script keeps the `find` because it is one explicit step whose meaning does not
|
|
depend on the lcov version, and because clearing counters must never be the thing that breaks a
|
|
fresh tree. The capture also runs with `--rc geninfo_unexecuted_blocks=1` (the "unexecuted
|
|
block" notes come from libstdc++ headers on this toolchain; this records those blocks as
|
|
unexecuted, which is what they are) and `--ignore-errors inconsistent,inconsistent` for the same
|
|
source. `lcov --extract` then narrows the tracefile to `src/**`, and the HTML directory is removed
|
|
before `genhtml` re-creates it.
|
|
|
|
**The numbers are stable; the raw counts are not.** Two consecutive runs on the same tree produce
|
|
per-file `LF`/`LH`/`FNF`/`FNH` tallies that are identical for all 23 files — verified by diffing the
|
|
two tracefiles — but the per-line hit *counts* differ by a few parts per million, because a handful
|
|
of tests drive a producer thread whose iteration count follows wall-clock timing. Everything the
|
|
floor reads (the `LF`/`LH` tallies, and therefore every rate in the generated table above) is
|
|
unaffected.
|
|
|
|
## Adjusting the floors, and seeing the gate fail
|
|
|
|
Both floors are flags, which is how the failure path is demonstrated — and how a later task tightens
|
|
the contract. Against the report in the generated table above, a floor raised above every file's
|
|
rate fails the check and names each offender:
|
|
|
|
```sh
|
|
# Any file not at 100% fails: a floor that is guaranteed to bite, whatever the
|
|
# current rates are. The list of offenders is the files in the table above whose
|
|
# rate is below 100%.
|
|
python3 tools/coverage_floor.py --per-file 100 build/cov/coverage.info
|
|
# coverage_floor: FAILED -- <n> floor(s) violated in build/cov/coverage.info:
|
|
# <file>: <rate>% lines (<hit>/<found>) is below the 100% per-file floor
|
|
# ...
|
|
# $? == 1
|
|
|
|
# The same, for the aggregate: a target above the measured aggregate fails with one line.
|
|
python3 tools/coverage_floor.py --aggregate 99 build/cov/coverage.info
|
|
# coverage_floor: FAILED -- 1 floor(s) violated in build/cov/coverage.info:
|
|
# src/ aggregate: <rate>% lines (<hit>/<found>) is below the 99% aggregate floor
|
|
# $? == 1
|
|
|
|
# A floor at or below the measured rates passes.
|
|
python3 tools/coverage_floor.py build/cov/coverage.info
|
|
# coverage_floor: ok -- ...
|
|
# $? == 0
|
|
```
|
|
|
|
## Why the check is a CMake target and not a ctest case
|
|
|
|
A ctest entry registered only in the coverage preset would either fail in a freshly configured tree
|
|
— no report exists until `tools/coverage.sh` has run once, so the preset's own clean-tree command
|
|
(`cmake --preset linux-coverage && cmake --build build/cov && ctest --test-dir build/cov`) would
|
|
then fail for a reason that has nothing to do with the code — or it would have to build the library
|
|
and re-run the whole suite inside the test, duplicating the script. A custom target instead: it is
|
|
not part of `all`, so it cannot slow or destabilise the `linux-debug`, `linux-asan` or
|
|
`linux-release` builds, and it is not even created in those presets, because they leave
|
|
`PLOT_COVERAGE` off and the option's guard rejects a tree that carries no `--coverage` flags.
|