Skip to content

ci: speed up test-coverage workflow with 4 CPUs, sharding, and path filtering - #769

Merged
randy3k merged 3 commits into
masterfrom
perf/test-coverage
Sep 16, 2026
Merged

randy3k merged 3 commits into
masterfrom
perf/test-coverage

Conversation

@randy3k

@randy3k randy3k commented Sep 15, 2026

Copy link
Copy Markdown
Member

Summary

This PR speeds up the test-coverage workflow through three complementary optimizations:

  1. Parallel test execution with 4 vCPUs: Sets TESTTHAT_CPUS: 4 in the test-coverage workflow to take full advantage of standard GitHub Actions runners (4 vCPUs on Linux).
  2. Path filtering: Adds paths-ignore for markdown documentation (**.md) and .gitignore to prevent unnecessary CI runs on doc-only changes.
  3. 4-way balanced test sharding: Partitions all 61 test files across 4 balanced matrix shards running in parallel, uploading separate flags (shard-1, shard-2, shard-3, shard-4) to Codecov for complete aggregation:
    • Shard 1 (^[a-c]): 2,920 lines, 82 tests (contains completion)
    • Shard 2 (^[d-i]): 3,928 lines, 116 tests (definition, formatting, hover, index, etc.)
    • Shard 3 (^[j-r]): 3,144 lines, 100 tests (lintr, refactor, rename, references, etc.)
    • Shard 4 (^[s-z]): 3,494 lines, 101 tests (semantic-tokens, symbol, task, type-hierarchy, etc.)

Runtime Comparison

  • Baseline (single thread, all tests): ~44 minutes
  • 4 vCPUs (all tests): ~32 minutes
  • 2-way naive sharding (unbalanced, shard 1 took 77% of assertions): ~30 minutes
  • 4-way balanced sharding: expected ~8–10 minutes per shard

@randy3k randy3k changed the title ci: speed up test coverage with 4 CPUs and path filtering ci: speed up test-coverage workflow with 4 CPUs, sharding, and path filtering Sep 15, 2026

@renkun-ken renkun-ken left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Thanks!

@randy3k
randy3k merged commit 7d0d1d5 into master Sep 16, 2026
8 checks passed
@randy3k
randy3k deleted the perf/test-coverage branch September 16, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants