Skip to content

Instrument first-party libraries in coverage reports - #8322

Open
Amaury Chamayou (achamayou) wants to merge 6 commits into
mainfrom
achamayou-coverage-instrumentation
Open

Amaury Chamayou (achamayou) wants to merge 6 commits into
mainfrom
achamayou-coverage-instrumentation

Conversation

@achamayou

@achamayou Amaury Chamayou (achamayou) commented Sep 9, 2026

Copy link
Copy Markdown
Member

Instrument first-party C++ libraries under COVERAGE=ON, propagating coverage-runtime linkage to their consumers while reporting only linked binaries.

Add per-file JSON export, framework/sample totals, and a CI guard for missing or unhit implementation code. Preserve existing exclusions and support source paths in renamed checkouts.

The expanded coverage denominator makes older percentages incomparable. A passing full Coverage run is still outstanding; earlier concurrent e2e runs did not pass.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@achamayou
Amaury Chamayou (achamayou) marked this pull request as ready for review September 17, 2026 08:51
Copilot AI lite review requested due to automatic review settings September 17, 2026 08:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Resolve the coverage-history baseline and renamed-checkout path handling issues.

Pull request overview

Expands C++ coverage to first-party libraries with JSON reporting, instrumentation checks, and CI updates.

Changes:

  • Instruments libraries and propagates coverage runtime linkage.
  • Adds scoped totals, path handling, JSON export, and sentinel validation.
  • Updates coverage CI and testing documentation.
File summaries
File Description
scripts/tests/coverage_report_test.py Tests coverage parsing and validation.
scripts/coverage.sh Adds JSON export and path handling.
scripts/coverage_report.py Generates scoped totals and instrumentation checks.
cmake/tools.cmake Instruments libraries and propagates runtime linkage.
cmake/crypto.cmake Enables coverage for the crypto library.
cmake/ccf_app.cmake Enables coverage for CCF targets.
.github/workflows/coverage.yml Runs coverage reporting and CI validation.
.github/skills/testing/SKILL.md Documents the updated coverage workflow.
Review details

Suppressed comments (2)

.github/workflows/coverage.yml:91

  • This run now emits a different TOTAL denominator because first-party library objects are included, but the existing history step still feeds pre-instrumentation logs into coverage_summary.py alongside this value. The resulting trend chart presents incomparable percentages as a continuous history; please reset/mark the baseline at this change or suppress the old points until a new baseline exists.
          python3 ../scripts/coverage_report.py coverage_html/coverage.json \
            --check-instrumentation | tee coverage_html/summary.md

scripts/coverage_report.py:46

  • When the export retains the relative path produced by -ffile-prefix-map (for example CCF/src/crypto/base64.cpp), this prepends the renamed checkout and normalises to CCF/src/... instead of src/.... The framework files are then classified as Other, and every instrumentation sentinel is reported missing in the renamed-checkout case. Treat the mapped CCF/ prefix as rooted at source_dir.parent before applying the root stripping, and add that relative-path form to the regression test.
            path = (source_dir / entry["filename"]).resolve()
            filename = path.as_posix()
            for root in source_roots:
                if path.is_relative_to(root):
                    filename = path.relative_to(root).as_posix()
  • Files reviewed: 8/8 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants