Repository navigation
Support auto-refreshing Test Coverage from an external report #26266
Description
Problem
The Python extension renders coverage only for test runs started by the Test Explorer.
Coverage reports generated by external commands cannot refresh the Test Coverage view or editor decorations, even when a valid local coverage.xml already exists.
Why this matters
AI coding agents are increasingly integrated with local test execution.
They can run tests and generate coverage reports through existing project commands,
but their textual output alone is not an ideal verification surface.
Automatically rendering a locally generated coverage report in VS Code provides a visible,
independently inspectable signal:
developers can immediately review which files and lines were covered without rerunning the tests through the UI.
This makes the Test Coverage view a practical verification layer for automated test runs,
including those initiated by AI agents.
Requested feature
Add an opt-in setting that watches a local coverage report and automatically refreshes coverage when the report changes.
{
"python.testing.coverageReport": {
"path": "${workspaceFolder}/coverage.xml",
"format": "cobertura",
"autoRefresh": true
}
}
Expected behavior
- Watch the configured local report when autoRefresh is enabled.
- Parse it after a successful file update.
- Update the existing Test Coverage view and inline coverage decorations.
- Map report paths to workspace files.
- Do not rerun pytest.
- Show clear errors for missing or invalid reports.
Activity
darshpatel1052 commentedon Nov 3, 2026
Hi, I’d like to take this on. I’d first check how externally generated Cobertura data could feed the existing coverage view, then scope an opt-in report watcher with path mapping and invalid-file handling. Would you prefer a manual import command first, or is watching the configured report the right starting point?
xiaoyao9184 commentedon Nov 3, 2026
Thank you. I would prefer a manual import/refresh command as the first step.
Once such a command exists, an agent can run tests through the CLI and then use
VS Code/MCP automation to refresh the coverage display. That would already be a
useful incremental result, without requiring a file watcher in the initial
implementation.
My longer-term goal is: I am exploring a workflow where VS Code becomes
more of a read-and-validate environment. Agents edit and run verification steps;
humans read and validate the resulting artifacts. I am also looking at related
extensions, including Coverage Gutters, but not perfect.
I do not know whether another extension already provides it.
In that model, a Cobertura report should be treated like a source artifact: it
can be versioned, read from a file, and rendered solely to help humans
understand the result. The main design question is how a coverage renderer,
once separated from a live test run, should relate to the Test Results view.
My current view is that test results should eventually also have a standard,
versionable file representation. Then an agent only needs to run CLI commands
and produce result artifacts, while VS Code focuses on rendering those artifacts
for human review and understanding.
darshpatel1052 commentedon Nov 3, 2026
Thanks, a manual import/refresh command sounds like a useful first step. I’d keep the file watcher out of that change. Could a maintainer confirm that scope and assign this to me if it’s available?