Skip to content

perf(coverage): reduce RPC data and unnecessary serializations - #10781

Merged
sheremet-va merged 1 commit into
vitest-dev:mainfrom
AriPerkkio:perf/coverage-serialization
Jul 23, 2026
Merged

perf(coverage): reduce RPC data and unnecessary serializations#10781
sheremet-va merged 1 commit into
vitest-dev:mainfrom
AriPerkkio:perf/coverage-serialization

Conversation

@AriPerkkio

@AriPerkkio AriPerkkio commented Jul 15, 2026

Copy link
Copy Markdown
Member

Description

Adds two performance improvements I've had in mind long time:

  1. Currently we are sending coverage JSON data from Node side workers to main thread, and then writing that to file system. Now we'll write the data directly in the worker and just pass the path to main thread. Initially I thought writing in workers would be worse, but it should actually be faster. RPC always requires serialization that's costly.

  2. When reviewing fix(browser): disable client cdp API when allowWrite/allowExec: false #10444 I realized browser mode v8 coverage is doing unnecessary RPC hops:

  • Browser command to call CDP on server side to get V8 script coverage, pass that to browser via WS RPC
  • In browser we just filter out stuff from the JSON
  • Send the filtered JSON again to main thread to the coverage provider.

Now we'll instead just write the results to file system in the first call, and never pass the V8 script coverage to browser.

LLM Generated analysis of perf changes

Prompted Claude to run vitest-coverage-large locally with main and this PR. There's noticeable perf change in browser + v8, as now we don't bounce the results unnecessary over the WS RPC. 🔥

Performance

Details Measured with [`coverage-large`](https://github.com/AriPerkkio/coverage-large) using `pkg.pr.new` builds of this PR and its parent commit. The fixture was generated in a payload-heavy shape where each test file imports a 300-module graph, so that per-test coverage payloads are large (~6 MB for `v8`, ~1.9 GB total across the run):
COVERED_FILE_COUNT=300 UNCOVERED_FILE_COUNT=0 FUNCTION_COUNT=100 IMPORTS_PER_TEST=300 node ./generate-files.mjs

Durations are medians of 3 runs (2 runs for istanbul). Memory is the main process' peak RSS.

Duration

Case Before (0f8e63f) After (b7e2ad6) Speed-up
node + v8 39.0 s 33.9 s ~1.15×
node + istanbul 527 s 365 s ~1.4×
browser + v8 103 s 38.2 s ~2.7×
node + v8, default fixture ¹ 28.3 s 26.8 s ~1.06×

Main process peak memory

Case Before (0f8e63f) After (b7e2ad6) Change
node + v8 1 322 MB 1 245 MB −6 %
node + istanbul 4 111 MB 3 121 MB −24 %
browser + v8 1 370 MB 1 297 MB −5 %
Details

¹ No-regression control: the stock coverage-large shape (600 + 600 files, one covered file per test) has tiny per-test payloads, so little change is expected — or observed.

browser + istanbul is excluded from the payload-heavy comparison: at this payload size it does not complete reliably on either side of this PR (before: 190–201 of 300 test files in 26–28 min; after: 206 of 300). The istanbul coverage map only exists inside the browser page, so it must cross the WebSocket in both versions — this PR removes the return trip and main-process re-serialization, but not the dominant page-side transfer. On the stock fixture it completes fine on both sides (139–181 s before, ~136 s after).

LLM Generated sequence diagrams before-after

Output of following prompt below.

Now, draw a sequence diagram that shows how data is moved across main thread, workers and browser. Show how it was done previously and how it's now.

Details

How coverage results move between processes

Node pools (threads, forks, vmThreads, vmForks)

Before — full results cross the worker↔main boundary and are serialized twice more on the main process:

sequenceDiagram
    participant Worker as Test worker
    participant Main as Main process
    participant FS as File system

    Worker->>Worker: takeCoverage() - collect and filter results
    Worker->>Main: onAfterSuiteRun(results)<br/>structured clone (threads) / v8.serialize (forks)<br/>full results, MBs on large projects
    Main->>Main: JSON.stringify(results)
    Main->>FS: write .tmp/coverage-N.json
    Note over Main,FS: at report time
    FS->>Main: read + JSON.parse each file
    Main->>Main: merge, remap, report
Loading

After — workers write their own results in parallel; only a filename crosses the boundary:

sequenceDiagram
    participant Worker as Test worker
    participant Main as Main process
    participant FS as File system

    Worker->>Worker: takeCoverage() - collect and filter results
    Worker->>FS: JSON.stringify(results)<br/>write .tmp/coverage-uuid.json
    Worker->>Main: onAfterSuiteRun(filename)<br/>plain string, few bytes
    Note over Main,FS: at report time
    FS->>Main: read + JSON.parse each file
    Main->>Main: merge, remap, report
Loading

Browser mode

Before — the v8 CDP payload made a round trip to the page and back over WebSocket (flatted), unfiltered on the way out:

sequenceDiagram
    participant Page as Tester page
    participant Main as Main process
    participant FS as File system

    rect rgba(128, 128, 128, 0.1)
    Note right of Page: v8
    Page->>Main: __vitest_takeV8Coverage command
    Main->>Main: Profiler.takePreciseCoverage via CDP
    Main->>Page: unfiltered results over WebSocket<br/>MBs, includes node_modules and vite client
    Page->>Page: filter + rewrite URLs
    Page->>Main: onAfterSuiteRun(results) over WebSocket - MBs
    end

    rect rgba(128, 128, 128, 0.1)
    Note right of Page: istanbul
    Page->>Main: onAfterSuiteRun(coverage map) over WebSocket - full map
    end

    Main->>Main: JSON.stringify(results)
    Main->>FS: write .tmp/coverage-N.json
Loading

After — v8 results never leave the main process; istanbul's map crosses the WebSocket once (it only exists in the page):

sequenceDiagram
    participant Page as Tester page
    participant Main as Main process
    participant FS as File system

    rect rgba(128, 128, 128, 0.1)
    Note right of Page: v8
    Page->>Main: __vitest_takeV8Coverage(location.href) - few bytes
    Main->>Main: Profiler.takePreciseCoverage via CDP<br/>filter + rewrite URLs
    Main->>FS: write .tmp/coverage-uuid.json
    Main->>Page: filename
    end

    rect rgba(128, 128, 128, 0.1)
    Note right of Page: istanbul
    Page->>Main: __vitest_writeCoverageFile(coverage map)<br/>one WebSocket transfer
    Main->>FS: write .tmp/coverage-uuid.json
    Main->>Page: filename
    end

    Page->>Main: onAfterSuiteRun(filename) - few bytes
Loading

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
  • Ideally, include a test that fails without this PR but passes with it.
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.
  • Please check Allow edits by maintainers to make review process faster. Note that this option is not available for repositories that are owned by Github organizations.

Tests

  • Run the tests with pnpm test:ci.

Documentation

  • If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

  • Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

@AriPerkkio AriPerkkio left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment this PR is 100% Fable-slop - I took quick look at the changes and they'll need some adjusting. And testing of course. Opening draft PR for visibility at this point.

@netlify

netlify Bot commented Jul 15, 2026

Copy link
Copy Markdown

Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit 06660ba
🔍 Latest deploy log https://app.netlify.com/projects/vitest-dev/deploys/6a61bc3c949c1f0008316705
😎 Deploy Preview https://deploy-preview-10781--vitest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@pkg-pr-new

pkg-pr-new Bot commented Jul 16, 2026

Copy link
Copy Markdown
@vitest/browser

npm i https://pkg.pr.new/@vitest/browser@3684f15

@vitest/browser-playwright

npm i https://pkg.pr.new/@vitest/browser-playwright@3684f15

@vitest/browser-preview

npm i https://pkg.pr.new/@vitest/browser-preview@3684f15

@vitest/coverage-istanbul

npm i https://pkg.pr.new/@vitest/coverage-istanbul@3684f15

@vitest/coverage-v8

npm i https://pkg.pr.new/@vitest/coverage-v8@3684f15

@vitest/expect

npm i https://pkg.pr.new/@vitest/expect@3684f15

@vitest/mocker

npm i https://pkg.pr.new/@vitest/mocker@3684f15

@vitest/pretty-format

npm i https://pkg.pr.new/@vitest/pretty-format@3684f15

@vitest/snapshot

npm i https://pkg.pr.new/@vitest/snapshot@3684f15

@vitest/spy

npm i https://pkg.pr.new/@vitest/spy@3684f15

@vitest/ui

npm i https://pkg.pr.new/@vitest/ui@3684f15

@vitest/utils

npm i https://pkg.pr.new/@vitest/utils@3684f15

vitest

npm i https://pkg.pr.new/vitest@3684f15

@vitest/web-worker

npm i https://pkg.pr.new/@vitest/web-worker@3684f15

commit: 3684f15

@vitest-ecosystem-ci

vitest-ecosystem-ci Bot commented Jul 21, 2026

Copy link
Copy Markdown

📝 Ran ecosystem CI: Open

suite result
vitest-coverage-large-browser ✅ success

@vitest-ecosystem-ci

vitest-ecosystem-ci Bot commented Jul 21, 2026

Copy link
Copy Markdown

📝 Ran ecosystem CI: Open

suite result
vitest-coverage-large-node ✅ success

@AriPerkkio
AriPerkkio force-pushed the perf/coverage-serialization branch 2 times, most recently from b7e2ad6 to 06660ba Compare July 23, 2026 07:01
@AriPerkkio
AriPerkkio force-pushed the perf/coverage-serialization branch from 06660ba to 3684f15 Compare July 23, 2026 07:10
@AriPerkkio
AriPerkkio marked this pull request as ready for review July 23, 2026 07:21
@AriPerkkio
AriPerkkio requested review from hi-ogawa and sheremet-va and removed request for hi-ogawa July 23, 2026 07:21
@sheremet-va
sheremet-va merged commit 7e7e35c into vitest-dev:main Jul 23, 2026
27 of 28 checks passed
@AriPerkkio
AriPerkkio deleted the perf/coverage-serialization branch July 23, 2026 09:21
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 7, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Sponsor
SponsoredKunjungi sekarang
Promo