Repository navigation
test: fix flaky specs that relied on the individual-test rerun - #54901
Conversation
… job script/spec-runner.js had an --enableRerun mode that, after a failing run, parsed the JUnit report and re-ran each failed test on its own in a fresh Electron. CI passed --enableRerun=3 on the macOS, Windows and Linux test legs, so a test that only passes when it has the machine to itself turned the job green while still being recorded as a failure. In practice the mode only ever made one solo attempt per test (a failed rerun exited the runner immediately), and later failures were never rerun once an earlier one failed again. Remove the mode: the JUnit parsing, the solo rerun loop, the per-launch --log-file renaming and the --files filtering that only reruns used. A failing run now exits 1 straight away. Drop --enableRerun=3 from the test workflow and the @xmldom/xmldom dev dependency that only the JUnit parser used. vitest's in-suite retry (retry: 3 on CI) is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
The autoUpdater MSIX suite installs its signing certificate and registers Electron as a sparse package in a before hook, each through PowerShell with a 30s timeout. While other spec files run alongside it, these two commands regularly take longer than that on windows-arm64 (the hook failed after roughly 60-90s), but they complete when the suite runs on its own, which is why the job only went green through the end-of-job solo rerun. vitest does not retry a failed hook, so with the rerun gone that would fail the job. This is a timeout increase on purpose: the operations themselves are just slow on a loaded host. Give those two setup commands 120s each and the hook 300s; every other PowerShell call keeps the 30s limit. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
"File System API, denies permission when trying to create a writable file handle" failed every in-suite attempt on Linux and macOS with "SecurityError: User activation is required to request permissions", and the permission request handler was never called, so the browser-side FileSystemAccessPermissionContext saw no transient activation. It passed when re-run on its own. The test grants activation and calls createWritable() in the same executeJavaScript(code, true). The renderer reports the activation on the frame's channel-associated LocalFrameHost interface, but createWritable() reaches the browser on the file handle's own Mojo pipe, so the permission check can run before the activation update has been processed. Activate the page in a separate executeJavaScript() call first: its reply comes back on the channel-associated ElectronFrame interface, after the activation update. Do the same in the matching "allows permission" test, which makes the same request. This is the most likely explanation from the code and the in-suite traces; the failure doesn't reproduce on an idle Linux machine. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
The Network Service crash recovery specs crash the service and then sleep for a fixed 500ms before asserting that the next net.fetch() succeeds. On busy Windows runners the NetworkContext (or, for the utility process, the URLLoaderFactory update) isn't back yet after 500ms, and the fetch fails with net::ERR_FAILED. The test then only passed when re-run on its own. Poll a probe server with net.fetch() until a request succeeds, then make the asserted request. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
The api-net-spec.js fixture runs itUtility test bodies in a utility process and does postMessage(result) immediately followed by process.exit(0). The result travels over the parent port's message pipe and the exit is reported by the process host, so the exit can reach the parent first (or the message can be lost entirely when the process goes away before the port is flushed). On Windows this failed a scattering of net specs with "utility process exited (0) before posting a result", which then passed when re-run on their own. Make the fixture wait for an acknowledgement from the parent before it exits, on the failure paths too, and have waitForUtilityResult() send that acknowledgement once it has the result. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
"exposes "exit" event to preload script" registers a process 'exit' handler that sends an IPC, then navigates the hidden window to http://www.google.com. That is a cross-site navigation, so the 'exit' handler runs while the old document is being unloaded in a renderer that is about to go away, and on busy macOS runners the IPC sometimes never arrived (the test timed out after 30s and passed when re-run on its own). It also made the test depend on reaching the internet. Navigate to spec/fixtures/pages/blank.html instead. The unload then stays in the same renderer and the IPC is sent before the new page commits. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
"supports generating snapshots via v8.setHeapSnapshotNearHeapLimit" forks oom-grow.js with a 50MB old space limit and waits for it to run out of memory. The fixture grew the heap by one 1000-character string per setImmediate, so it took tens of thousands of event loop turns (about 8s on an idle Linux machine) to get near the limit, and on busy macos-x64 runners that went past the 60s timeout. It passed when re-run on its own. Allocate an array of 128K numbers per turn instead. The test now takes about 1s locally (1.2-1.7s with every core busy) and still produces the snapshot it checks for. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
The navigator.usb and navigator.hid "does not crash when the requesting webContents is destroyed from the select-*-device handler" specs call guest.destroy() in the handler and then check isDestroyed() after a single setTimeout() tick. destroy() only posts the teardown to the UI thread, so on a busy runner the check can run first and fail with "expected false to be true" (seen on macos-x64). Poll isDestroyed() until it is true instead. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
"visibilityState changes when window is hidden" failed all four in-suite attempts on a macos-x64 main push with "expected 'hidden' to equal 'visible'" on the page's first report, before the test hides the window, and passed when re-run on its own. These specs run in the parallel phase, where other spec files open windows at the same time. On macOS a window that another one covers is occluded, and Chromium then reports its page as hidden. Tag the describe serial so its windows are the only ones on screen. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
"protocol module handle is fast" compares the time to fetch a 128MB body directly and through a protocol.handle() interceptor and requires the median ratio to stay under 3. It ran in the parallel phase, where other spec files' Electron processes compete for the CPU and can slow one side of a pair more than the other; on the Chromium roll PR and on release branches it failed with "median ratio >= 3" and passed when re-run on its own. Tag it serial so the measurement has the machine to itself. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
The "preload code cache" suite's before hook loads a probe preload and waits up to 5s for its cache file to appear, to learn the process lock half of the file name. The browser writes that file from a BEST_EFFORT thread pool task, which a busy machine can hold back for longer than that; on macos-arm64 the hook failed with "timed out waiting for probe cache file to be written" and passed when re-run on its own. vitest does not retry hooks, so give this wait 30s (it returns as soon as the file is there) and the hook 60s. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
The "import / require parity" suite ran its fixture app, which starts a
main, a utility and a renderer process, in a before hook with the 20s
fixture timeout. On busy windows-x64 runners it sometimes did not finish
in time ("Timed out after 20000ms waiting for ... import-require-parity to
exit") and the suite only passed when re-run on its own. vitest does not
retry hooks, so with that rerun gone a single slow launch would fail the
job.
Run the fixture from the first test instead and share the result with the
other two. A failed or timed-out run is forgotten, so vitest's retry of
that test starts a fresh one, like every other fixture launch in this file.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
cleanupWebContents() takes a snapshot of webContents.getAllWebContents() and destroys each entry in turn. Destroying one can take others in the snapshot down with it (its guests or devtools, for example), and calling destroy() on one of those throws "Object has been destroyed". That failed the "chrome extensions" suite's after hook on linux-x64, and hooks are not retried, so the suite only passed when re-run on its own. Skip entries that are already destroyed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
Since Chromium 157 refuses requestFullscreen() from a view that doesn't have focus (SuppressFullscreenFromUnfocusedView, the same change #54883 handled for the simpleFullScreen spec), "pressing ESC should emit the leave-html-full-screen event" has needed in-suite retries on macOS. The test used a hidden window, whose page can never be focused. Use a shown window, take app activation on macOS as #54883 does, focus the window and then the <webview> element, and wait until the guest page reports document.hasFocus() before calling requestFullscreen(). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
|
Generated by Claude Code |
|
I think we should land #54439 first. |
|
The test jobs failing on this head (linux 3 on x64/arm64/ubsan/asan, win 2 on x64/arm64) all fail on one spec: "should recover net.fetch after Network Service crash (main process)". This PR's change to probe Generated by Claude Code |
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
|
Removing draft status now that 54439 has landed (CC @dsanders11) |
ckerr
left a comment
There was a problem hiding this comment.
The first two inline comments (the utility process dropping messages on exit, and webContents.executeJavaScript(code, true) not telling the browser about the activation) are Electron bugs that these test changes work around, and apps can hit them too. The rest are smaller: hooks still exposed now that the rerun is gone, a probe that can hang, coverage, and cleanup.
@claude please address these review comments.
| // Posts |result| and exits with |code| once the parent has acknowledged it | ||
| // (see waitForUtilityResult in spec/lib/utility-process-helpers.ts). Exiting | ||
| // right after posting lets the exit overtake the message, or lose it. | ||
| function finish(result, code) { |
There was a problem hiding this comment.
The ack handshake works around a bug in Electron itself. UtilityProcessWrapper::HandleTermination calls CloseConnectorPort() before emitting 'exit' (shell/browser/api/electron_api_utility_process.cc:337), so any parentPort message that's still unread in the pipe gets dropped. An app doing parentPort.postMessage(result); process.exit(0) hits the same loss. Could we drain the connector before closing it, or before emitting 'exit'? That fixes it for apps too, and this fixture could stay simple.
There was a problem hiding this comment.
Opened #54924 to deliver pending parentPort messages before 'exit'. Once it lands, the ack handshake in this fixture can go.
Generated by Claude Code
There was a problem hiding this comment.
Thanks for opening #54924. Leaving this open until it lands; once it does, the ack handshake here can go.
| // call, the request can be handled before the activation update and is | ||
| // rejected with "User activation is required to request permissions". The | ||
| // reply to this call is ordered after the activation update. | ||
| const activatePage = async (w: BrowserWindow) => { |
There was a problem hiding this comment.
This works around webContents.executeJavaScript(code, true) never telling the browser about the activation. webFrameMain.executeJavaScript goes through RenderFrameHostImpl::ExecuteJavaScriptForTests, which calls UpdateUserActivationState browser-side first. That's why the permission-scope specs at L1951/L2039 (which use mainFrame.executeJavaScript) don't flake. Apps calling webContents.executeJavaScript('handle.createWritable()', true) hit the same race. Calling the existing WebContents::NotifyUserActivation() in ExecuteJavaScriptInRenderer when has_user_gesture is set (electron_api_web_contents.cc:5929) would fix both. At minimum, these two tests could use w.webContents.mainFrame.executeJavaScript(..., true) instead of the extra round trip.
There was a problem hiding this comment.
Opened #54923 to update the browser's user activation when userGesture is set. It calls RenderFrameHostImpl::UpdateUserActivationState() directly, like ExecuteJavaScriptForTests() does. It doesn't use WebContents::NotifyUserActivation(), because RenderFrameHost::NotifyUserActivation() only forwards to the renderer, so the race would remain.
Generated by Claude Code
There was a problem hiding this comment.
Done in 8690035: the two createWritable tests now run through w.webContents.mainFrame.executeJavaScript(..., true) and the extra activation round trip is gone. The webContents.executeJavaScript product fix will be a separate PR.
Generated by Claude Code
There was a problem hiding this comment.
Both crash-recovery specs now wait through waitForNetworkServiceRecovery(), which races each probe against a timeout and, for the utility process variant, the child's exit, so a stalled request or a dead child fails fast with its cause. The network-restart-test.js child is now registered with deferKillUtilityProcess so that a failed wait doesn't leak it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
The per-test waits on the cache write still used waitFor's 5s default, while the beforeAll probe waiting on the same thread pool write had 30s. They now share cacheWriteTimeout, and the suite's test timeout leaves room for it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
Remove-AppxPackage runs in a hook under the same load as setup, and hooks are not retried. Both hook timeouts are now derived from powershellSetupTimeout so that they cover the commands they run. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
The page now navigates to the suite's server on 127.0.0.1, so 'exit' is again covered on a cross-site, process-swapping navigation, as it was with google.com, without depending on the internet. The spec checks that the navigation did swap processes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
test: fix flaky specs that relied on the individual-test rerun (#54901) * build: stop re-running failed specs individually at the end of a test job script/spec-runner.js had an --enableRerun mode that, after a failing run, parsed the JUnit report and re-ran each failed test on its own in a fresh Electron. CI passed --enableRerun=3 on the macOS, Windows and Linux test legs, so a test that only passes when it has the machine to itself turned the job green while still being recorded as a failure. In practice the mode only ever made one solo attempt per test (a failed rerun exited the runner immediately), and later failures were never rerun once an earlier one failed again. Remove the mode: the JUnit parsing, the solo rerun loop, the per-launch --log-file renaming and the --files filtering that only reruns used. A failing run now exits 1 straight away. Drop --enableRerun=3 from the test workflow and the @xmldom/xmldom dev dependency that only the JUnit parser used. vitest's in-suite retry (retry: 3 on CI) is unchanged. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: give the MSIX suite's setup PowerShell commands more time The autoUpdater MSIX suite installs its signing certificate and registers Electron as a sparse package in a before hook, each through PowerShell with a 30s timeout. While other spec files run alongside it, these two commands regularly take longer than that on windows-arm64 (the hook failed after roughly 60-90s), but they complete when the suite runs on its own, which is why the job only went green through the end-of-job solo rerun. vitest does not retry a failed hook, so with the rerun gone that would fail the job. This is a timeout increase on purpose: the operations themselves are just slow on a loaded host. Give those two setup commands 120s each and the hook 300s; every other PowerShell call keeps the 30s limit. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: make sure the browser sees user activation before createWritable() "File System API, denies permission when trying to create a writable file handle" failed every in-suite attempt on Linux and macOS with "SecurityError: User activation is required to request permissions", and the permission request handler was never called, so the browser-side FileSystemAccessPermissionContext saw no transient activation. It passed when re-run on its own. The test grants activation and calls createWritable() in the same executeJavaScript(code, true). The renderer reports the activation on the frame's channel-associated LocalFrameHost interface, but createWritable() reaches the browser on the file handle's own Mojo pipe, so the permission check can run before the activation update has been processed. Activate the page in a separate executeJavaScript() call first: its reply comes back on the channel-associated ElectronFrame interface, after the activation update. Do the same in the matching "allows permission" test, which makes the same request. This is the most likely explanation from the code and the in-suite traces; the failure doesn't reproduce on an idle Linux machine. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: poll for the restarted Network Service instead of sleeping 500ms The Network Service crash recovery specs crash the service and then sleep for a fixed 500ms before asserting that the next net.fetch() succeeds. On busy Windows runners the NetworkContext (or, for the utility process, the URLLoaderFactory update) isn't back yet after 500ms, and the fetch fails with net::ERR_FAILED. The test then only passed when re-run on its own. Poll a probe server with net.fetch() until a request succeeds, then make the asserted request. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: acknowledge utility process results before the child exits The api-net-spec.js fixture runs itUtility test bodies in a utility process and does postMessage(result) immediately followed by process.exit(0). The result travels over the parent port's message pipe and the exit is reported by the process host, so the exit can reach the parent first (or the message can be lost entirely when the process goes away before the port is flushed). On Windows this failed a scattering of net specs with "utility process exited (0) before posting a result", which then passed when re-run on their own. Make the fixture wait for an acknowledgement from the parent before it exits, on the failure paths too, and have waitForUtilityResult() send that acknowledgement once it has the result. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: navigate to a local page in the sandbox "exit" event spec "exposes "exit" event to preload script" registers a process 'exit' handler that sends an IPC, then navigates the hidden window to http://www.google.com. That is a cross-site navigation, so the 'exit' handler runs while the old document is being unloaded in a renderer that is about to go away, and on busy macOS runners the IPC sometimes never arrived (the test timed out after 30s and passed when re-run on its own). It also made the test depend on reaching the internet. Navigate to spec/fixtures/pages/blank.html instead. The unload then stays in the same renderer and the IPC is sent before the new page commits. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: reach the heap limit quickly in the heap snapshot utility spec "supports generating snapshots via v8.setHeapSnapshotNearHeapLimit" forks oom-grow.js with a 50MB old space limit and waits for it to run out of memory. The fixture grew the heap by one 1000-character string per setImmediate, so it took tens of thousands of event loop turns (about 8s on an idle Linux machine) to get near the limit, and on busy macos-x64 runners that went past the 60s timeout. It passed when re-run on its own. Allocate an array of 128K numbers per turn instead. The test now takes about 1s locally (1.2-1.7s with every core busy) and still produces the snapshot it checks for. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: wait for the guest to be destroyed in the USB/HID chooser specs The navigator.usb and navigator.hid "does not crash when the requesting webContents is destroyed from the select-*-device handler" specs call guest.destroy() in the handler and then check isDestroyed() after a single setTimeout() tick. destroy() only posts the teardown to the UI thread, so on a busy runner the check can run first and fail with "expected false to be true" (seen on macos-x64). Poll isDestroyed() until it is true instead. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: run the document.visibilityState/hidden specs serially "visibilityState changes when window is hidden" failed all four in-suite attempts on a macos-x64 main push with "expected 'hidden' to equal 'visible'" on the page's first report, before the test hides the window, and passed when re-run on its own. These specs run in the parallel phase, where other spec files open windows at the same time. On macOS a window that another one covers is occluded, and Chromium then reports its page as hidden. Tag the describe serial so its windows are the only ones on screen. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: run the protocol.handle throughput spec serially "protocol module handle is fast" compares the time to fetch a 128MB body directly and through a protocol.handle() interceptor and requires the median ratio to stay under 3. It ran in the parallel phase, where other spec files' Electron processes compete for the CPU and can slow one side of a pair more than the other; on the Chromium roll PR and on release branches it failed with "median ratio >= 3" and passed when re-run on its own. Tag it serial so the measurement has the machine to itself. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: wait longer for the preload code cache probe in its before hook The "preload code cache" suite's before hook loads a probe preload and waits up to 5s for its cache file to appear, to learn the process lock half of the file name. The browser writes that file from a BEST_EFFORT thread pool task, which a busy machine can hold back for longer than that; on macos-arm64 the hook failed with "timed out waiting for probe cache file to be written" and passed when re-run on its own. vitest does not retry hooks, so give this wait 30s (it returns as soon as the file is there) and the hook 60s. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: run the import/require parity fixture from a test, not a hook The "import / require parity" suite ran its fixture app, which starts a main, a utility and a renderer process, in a before hook with the 20s fixture timeout. On busy windows-x64 runners it sometimes did not finish in time ("Timed out after 20000ms waiting for ... import-require-parity to exit") and the suite only passed when re-run on its own. vitest does not retry hooks, so with that rerun gone a single slow launch would fail the job. Run the fixture from the first test instead and share the result with the other two. A failed or timed-out run is forgotten, so vitest's retry of that test starts a fresh one, like every other fixture launch in this file. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: skip already-destroyed webContents in cleanupWebContents() cleanupWebContents() takes a snapshot of webContents.getAllWebContents() and destroys each entry in turn. Destroying one can take others in the snapshot down with it (its guests or devtools, for example), and calling destroy() on one of those throws "Object has been destroyed". That failed the "chrome extensions" suite's after hook on linux-x64, and hooks are not retried, so the suite only passed when re-run on its own. Skip entries that are already destroyed. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: focus the webview before requesting fullscreen in the ESC spec Since Chromium 157 refuses requestFullscreen() from a view that doesn't have focus (SuppressFullscreenFromUnfocusedView, the same change #54883 handled for the simpleFullScreen spec), "pressing ESC should emit the leave-html-full-screen event" has needed in-suite retries on macOS. The test used a hidden window, whose page can never be focused. Use a shown window, take app activation on macOS as #54883 does, focus the window and then the <webview> element, and wait until the guest page reports document.hasFocus() before calling requestFullscreen(). Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: share the Network Service recovery probe between the crash specs Both crash-recovery specs now wait through waitForNetworkServiceRecovery(), which races each probe against a timeout and, for the utility process variant, the child's exit, so a stalled request or a dead child fails fast with its cause. The network-restart-test.js child is now registered with deferKillUtilityProcess so that a failed wait doesn't leak it. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: refocus the window on each poll in the webview ESC spec Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: give every preload code cache write wait the same longer timeout The per-test waits on the cache write still used waitFor's 5s default, while the beforeAll probe waiting on the same thread pool write had 30s. They now share cacheWriteTimeout, and the suite's test timeout leaves room for it. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: give the MSIX afterAll unregistration the setup timeout too Remove-AppxPackage runs in a hook under the same load as setup, and hooks are not retried. Both hook timeouts are now derived from powershellSetupTimeout so that they cover the commands they run. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: navigate the sandbox exit-event page cross-site to a local server The page now navigates to the suite's server on 127.0.0.1, so 'exit' is again covered on a cross-site, process-swapping navigation, as it was with google.com, without depending on the internet. The spec checks that the navigation did swap processes. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: run createWritable() through webFrameMain.executeJavaScript() webFrameMain.executeJavaScript() updates the browser process's user activation state before it runs the script, so the permission request no longer races the activation update. This replaces the extra activation round trip. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * build: reject a leftover --enableRerun in the spec runner It is no longer a known option, so it would otherwise be forwarded to the Electron workers as a switch that silently does nothing. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: break comments added in this PR at sentence and clause boundaries Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: keep the individual-test rerun for now (moved to a follow-up) Per review, this PR now only fixes the specs that passed thanks to the end-of-job solo rerun, so the fixes can be backported. Restore the spec runner's --enableRerun mode, its @xmldom/xmldom dependency, --enableRerun=3 in the test workflow and the vitest.config.ts comment to main's versions. Removing the rerun moves to a follow-up once main has been clean for a few days. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: go back to a same-site navigation in the sandbox exit-event spec A cross-site navigation unloads the old page through RenderFrameHostImpl::Unload, which gives the renderer only kUnloadTimeout (500ms) before the process is torn down. That is the race this spec originally lost on busy runners. Navigate to the local blank.html again, so the unload stays in the same renderer, and drop the process-swap check and the IPC handshake that only the cross-site variant needed. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: retry a Network Service recovery probe that times out A probe that got no response within 5s threw, and waitUntil fails on a thrown error, so one slow probe right after the crash failed the test although the next one would have succeeded. A timed-out probe now counts as a failed attempt; only the utility process exiting still fails right away. Since a probe can now be abandoned while its fetch is still in flight, the utility process fixture echoes an id sent with each fetch, and the spec waits for the reply with that id, so a late reply isn't taken as the reply to a later probe or to the final fetch. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: handle a rejected 'exit' wait before the first recovery probe once(child, 'exit') rejects if the utility process emits 'error', and nothing chained off it until the first recovery probe. An 'error' before then was an unhandled rejection, which fails the whole run rather than this test. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: log errors in the utility net fixture while it waits for the ack The fixture now stays up for a round trip after posting its result. In that window, leftover async work from the test body (a socket 'error' with no listener, a late unhandled rejection) could kill it with a non-zero code and fail a test that passed. While waiting for the ack, log such errors instead. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
test: fix flaky specs that relied on the individual-test rerun (#54901) * build: stop re-running failed specs individually at the end of a test job script/spec-runner.js had an --enableRerun mode that, after a failing run, parsed the JUnit report and re-ran each failed test on its own in a fresh Electron. CI passed --enableRerun=3 on the macOS, Windows and Linux test legs, so a test that only passes when it has the machine to itself turned the job green while still being recorded as a failure. In practice the mode only ever made one solo attempt per test (a failed rerun exited the runner immediately), and later failures were never rerun once an earlier one failed again. Remove the mode: the JUnit parsing, the solo rerun loop, the per-launch --log-file renaming and the --files filtering that only reruns used. A failing run now exits 1 straight away. Drop --enableRerun=3 from the test workflow and the @xmldom/xmldom dev dependency that only the JUnit parser used. vitest's in-suite retry (retry: 3 on CI) is unchanged. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: give the MSIX suite's setup PowerShell commands more time The autoUpdater MSIX suite installs its signing certificate and registers Electron as a sparse package in a before hook, each through PowerShell with a 30s timeout. While other spec files run alongside it, these two commands regularly take longer than that on windows-arm64 (the hook failed after roughly 60-90s), but they complete when the suite runs on its own, which is why the job only went green through the end-of-job solo rerun. vitest does not retry a failed hook, so with the rerun gone that would fail the job. This is a timeout increase on purpose: the operations themselves are just slow on a loaded host. Give those two setup commands 120s each and the hook 300s; every other PowerShell call keeps the 30s limit. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: make sure the browser sees user activation before createWritable() "File System API, denies permission when trying to create a writable file handle" failed every in-suite attempt on Linux and macOS with "SecurityError: User activation is required to request permissions", and the permission request handler was never called, so the browser-side FileSystemAccessPermissionContext saw no transient activation. It passed when re-run on its own. The test grants activation and calls createWritable() in the same executeJavaScript(code, true). The renderer reports the activation on the frame's channel-associated LocalFrameHost interface, but createWritable() reaches the browser on the file handle's own Mojo pipe, so the permission check can run before the activation update has been processed. Activate the page in a separate executeJavaScript() call first: its reply comes back on the channel-associated ElectronFrame interface, after the activation update. Do the same in the matching "allows permission" test, which makes the same request. This is the most likely explanation from the code and the in-suite traces; the failure doesn't reproduce on an idle Linux machine. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: poll for the restarted Network Service instead of sleeping 500ms The Network Service crash recovery specs crash the service and then sleep for a fixed 500ms before asserting that the next net.fetch() succeeds. On busy Windows runners the NetworkContext (or, for the utility process, the URLLoaderFactory update) isn't back yet after 500ms, and the fetch fails with net::ERR_FAILED. The test then only passed when re-run on its own. Poll a probe server with net.fetch() until a request succeeds, then make the asserted request. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: acknowledge utility process results before the child exits The api-net-spec.js fixture runs itUtility test bodies in a utility process and does postMessage(result) immediately followed by process.exit(0). The result travels over the parent port's message pipe and the exit is reported by the process host, so the exit can reach the parent first (or the message can be lost entirely when the process goes away before the port is flushed). On Windows this failed a scattering of net specs with "utility process exited (0) before posting a result", which then passed when re-run on their own. Make the fixture wait for an acknowledgement from the parent before it exits, on the failure paths too, and have waitForUtilityResult() send that acknowledgement once it has the result. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: navigate to a local page in the sandbox "exit" event spec "exposes "exit" event to preload script" registers a process 'exit' handler that sends an IPC, then navigates the hidden window to http://www.google.com. That is a cross-site navigation, so the 'exit' handler runs while the old document is being unloaded in a renderer that is about to go away, and on busy macOS runners the IPC sometimes never arrived (the test timed out after 30s and passed when re-run on its own). It also made the test depend on reaching the internet. Navigate to spec/fixtures/pages/blank.html instead. The unload then stays in the same renderer and the IPC is sent before the new page commits. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: reach the heap limit quickly in the heap snapshot utility spec "supports generating snapshots via v8.setHeapSnapshotNearHeapLimit" forks oom-grow.js with a 50MB old space limit and waits for it to run out of memory. The fixture grew the heap by one 1000-character string per setImmediate, so it took tens of thousands of event loop turns (about 8s on an idle Linux machine) to get near the limit, and on busy macos-x64 runners that went past the 60s timeout. It passed when re-run on its own. Allocate an array of 128K numbers per turn instead. The test now takes about 1s locally (1.2-1.7s with every core busy) and still produces the snapshot it checks for. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: wait for the guest to be destroyed in the USB/HID chooser specs The navigator.usb and navigator.hid "does not crash when the requesting webContents is destroyed from the select-*-device handler" specs call guest.destroy() in the handler and then check isDestroyed() after a single setTimeout() tick. destroy() only posts the teardown to the UI thread, so on a busy runner the check can run first and fail with "expected false to be true" (seen on macos-x64). Poll isDestroyed() until it is true instead. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: run the document.visibilityState/hidden specs serially "visibilityState changes when window is hidden" failed all four in-suite attempts on a macos-x64 main push with "expected 'hidden' to equal 'visible'" on the page's first report, before the test hides the window, and passed when re-run on its own. These specs run in the parallel phase, where other spec files open windows at the same time. On macOS a window that another one covers is occluded, and Chromium then reports its page as hidden. Tag the describe serial so its windows are the only ones on screen. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: run the protocol.handle throughput spec serially "protocol module handle is fast" compares the time to fetch a 128MB body directly and through a protocol.handle() interceptor and requires the median ratio to stay under 3. It ran in the parallel phase, where other spec files' Electron processes compete for the CPU and can slow one side of a pair more than the other; on the Chromium roll PR and on release branches it failed with "median ratio >= 3" and passed when re-run on its own. Tag it serial so the measurement has the machine to itself. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: wait longer for the preload code cache probe in its before hook The "preload code cache" suite's before hook loads a probe preload and waits up to 5s for its cache file to appear, to learn the process lock half of the file name. The browser writes that file from a BEST_EFFORT thread pool task, which a busy machine can hold back for longer than that; on macos-arm64 the hook failed with "timed out waiting for probe cache file to be written" and passed when re-run on its own. vitest does not retry hooks, so give this wait 30s (it returns as soon as the file is there) and the hook 60s. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: run the import/require parity fixture from a test, not a hook The "import / require parity" suite ran its fixture app, which starts a main, a utility and a renderer process, in a before hook with the 20s fixture timeout. On busy windows-x64 runners it sometimes did not finish in time ("Timed out after 20000ms waiting for ... import-require-parity to exit") and the suite only passed when re-run on its own. vitest does not retry hooks, so with that rerun gone a single slow launch would fail the job. Run the fixture from the first test instead and share the result with the other two. A failed or timed-out run is forgotten, so vitest's retry of that test starts a fresh one, like every other fixture launch in this file. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: skip already-destroyed webContents in cleanupWebContents() cleanupWebContents() takes a snapshot of webContents.getAllWebContents() and destroys each entry in turn. Destroying one can take others in the snapshot down with it (its guests or devtools, for example), and calling destroy() on one of those throws "Object has been destroyed". That failed the "chrome extensions" suite's after hook on linux-x64, and hooks are not retried, so the suite only passed when re-run on its own. Skip entries that are already destroyed. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: focus the webview before requesting fullscreen in the ESC spec Since Chromium 157 refuses requestFullscreen() from a view that doesn't have focus (SuppressFullscreenFromUnfocusedView, the same change #54883 handled for the simpleFullScreen spec), "pressing ESC should emit the leave-html-full-screen event" has needed in-suite retries on macOS. The test used a hidden window, whose page can never be focused. Use a shown window, take app activation on macOS as #54883 does, focus the window and then the <webview> element, and wait until the guest page reports document.hasFocus() before calling requestFullscreen(). Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: share the Network Service recovery probe between the crash specs Both crash-recovery specs now wait through waitForNetworkServiceRecovery(), which races each probe against a timeout and, for the utility process variant, the child's exit, so a stalled request or a dead child fails fast with its cause. The network-restart-test.js child is now registered with deferKillUtilityProcess so that a failed wait doesn't leak it. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: refocus the window on each poll in the webview ESC spec Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: give every preload code cache write wait the same longer timeout The per-test waits on the cache write still used waitFor's 5s default, while the beforeAll probe waiting on the same thread pool write had 30s. They now share cacheWriteTimeout, and the suite's test timeout leaves room for it. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: give the MSIX afterAll unregistration the setup timeout too Remove-AppxPackage runs in a hook under the same load as setup, and hooks are not retried. Both hook timeouts are now derived from powershellSetupTimeout so that they cover the commands they run. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: navigate the sandbox exit-event page cross-site to a local server The page now navigates to the suite's server on 127.0.0.1, so 'exit' is again covered on a cross-site, process-swapping navigation, as it was with google.com, without depending on the internet. The spec checks that the navigation did swap processes. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: run createWritable() through webFrameMain.executeJavaScript() webFrameMain.executeJavaScript() updates the browser process's user activation state before it runs the script, so the permission request no longer races the activation update. This replaces the extra activation round trip. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * build: reject a leftover --enableRerun in the spec runner It is no longer a known option, so it would otherwise be forwarded to the Electron workers as a switch that silently does nothing. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: break comments added in this PR at sentence and clause boundaries Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: keep the individual-test rerun for now (moved to a follow-up) Per review, this PR now only fixes the specs that passed thanks to the end-of-job solo rerun, so the fixes can be backported. Restore the spec runner's --enableRerun mode, its @xmldom/xmldom dependency, --enableRerun=3 in the test workflow and the vitest.config.ts comment to main's versions. Removing the rerun moves to a follow-up once main has been clean for a few days. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: go back to a same-site navigation in the sandbox exit-event spec A cross-site navigation unloads the old page through RenderFrameHostImpl::Unload, which gives the renderer only kUnloadTimeout (500ms) before the process is torn down. That is the race this spec originally lost on busy runners. Navigate to the local blank.html again, so the unload stays in the same renderer, and drop the process-swap check and the IPC handshake that only the cross-site variant needed. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: retry a Network Service recovery probe that times out A probe that got no response within 5s threw, and waitUntil fails on a thrown error, so one slow probe right after the crash failed the test although the next one would have succeeded. A timed-out probe now counts as a failed attempt; only the utility process exiting still fails right away. Since a probe can now be abandoned while its fetch is still in flight, the utility process fixture echoes an id sent with each fetch, and the spec waits for the reply with that id, so a late reply isn't taken as the reply to a later probe or to the final fetch. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: handle a rejected 'exit' wait before the first recovery probe once(child, 'exit') rejects if the utility process emits 'error', and nothing chained off it until the first recovery probe. An 'error' before then was an unhandled rejection, which fails the whole run rather than this test. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: log errors in the utility net fixture while it waits for the ack The fixture now stays up for a round trip after posting its result. In that window, leftover async work from the test body (a socket 'error' with no listener, a late unhandled rejection) could kill it with a non-zero code and fail a test that passed. While waiting for the ack, log such errors instead. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
test: fix flaky specs that relied on the individual-test rerun (#54901) * build: stop re-running failed specs individually at the end of a test job script/spec-runner.js had an --enableRerun mode that, after a failing run, parsed the JUnit report and re-ran each failed test on its own in a fresh Electron. CI passed --enableRerun=3 on the macOS, Windows and Linux test legs, so a test that only passes when it has the machine to itself turned the job green while still being recorded as a failure. In practice the mode only ever made one solo attempt per test (a failed rerun exited the runner immediately), and later failures were never rerun once an earlier one failed again. Remove the mode: the JUnit parsing, the solo rerun loop, the per-launch --log-file renaming and the --files filtering that only reruns used. A failing run now exits 1 straight away. Drop --enableRerun=3 from the test workflow and the @xmldom/xmldom dev dependency that only the JUnit parser used. vitest's in-suite retry (retry: 3 on CI) is unchanged. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: give the MSIX suite's setup PowerShell commands more time The autoUpdater MSIX suite installs its signing certificate and registers Electron as a sparse package in a before hook, each through PowerShell with a 30s timeout. While other spec files run alongside it, these two commands regularly take longer than that on windows-arm64 (the hook failed after roughly 60-90s), but they complete when the suite runs on its own, which is why the job only went green through the end-of-job solo rerun. vitest does not retry a failed hook, so with the rerun gone that would fail the job. This is a timeout increase on purpose: the operations themselves are just slow on a loaded host. Give those two setup commands 120s each and the hook 300s; every other PowerShell call keeps the 30s limit. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: make sure the browser sees user activation before createWritable() "File System API, denies permission when trying to create a writable file handle" failed every in-suite attempt on Linux and macOS with "SecurityError: User activation is required to request permissions", and the permission request handler was never called, so the browser-side FileSystemAccessPermissionContext saw no transient activation. It passed when re-run on its own. The test grants activation and calls createWritable() in the same executeJavaScript(code, true). The renderer reports the activation on the frame's channel-associated LocalFrameHost interface, but createWritable() reaches the browser on the file handle's own Mojo pipe, so the permission check can run before the activation update has been processed. Activate the page in a separate executeJavaScript() call first: its reply comes back on the channel-associated ElectronFrame interface, after the activation update. Do the same in the matching "allows permission" test, which makes the same request. This is the most likely explanation from the code and the in-suite traces; the failure doesn't reproduce on an idle Linux machine. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: poll for the restarted Network Service instead of sleeping 500ms The Network Service crash recovery specs crash the service and then sleep for a fixed 500ms before asserting that the next net.fetch() succeeds. On busy Windows runners the NetworkContext (or, for the utility process, the URLLoaderFactory update) isn't back yet after 500ms, and the fetch fails with net::ERR_FAILED. The test then only passed when re-run on its own. Poll a probe server with net.fetch() until a request succeeds, then make the asserted request. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: acknowledge utility process results before the child exits The api-net-spec.js fixture runs itUtility test bodies in a utility process and does postMessage(result) immediately followed by process.exit(0). The result travels over the parent port's message pipe and the exit is reported by the process host, so the exit can reach the parent first (or the message can be lost entirely when the process goes away before the port is flushed). On Windows this failed a scattering of net specs with "utility process exited (0) before posting a result", which then passed when re-run on their own. Make the fixture wait for an acknowledgement from the parent before it exits, on the failure paths too, and have waitForUtilityResult() send that acknowledgement once it has the result. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: navigate to a local page in the sandbox "exit" event spec "exposes "exit" event to preload script" registers a process 'exit' handler that sends an IPC, then navigates the hidden window to http://www.google.com. That is a cross-site navigation, so the 'exit' handler runs while the old document is being unloaded in a renderer that is about to go away, and on busy macOS runners the IPC sometimes never arrived (the test timed out after 30s and passed when re-run on its own). It also made the test depend on reaching the internet. Navigate to spec/fixtures/pages/blank.html instead. The unload then stays in the same renderer and the IPC is sent before the new page commits. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: reach the heap limit quickly in the heap snapshot utility spec "supports generating snapshots via v8.setHeapSnapshotNearHeapLimit" forks oom-grow.js with a 50MB old space limit and waits for it to run out of memory. The fixture grew the heap by one 1000-character string per setImmediate, so it took tens of thousands of event loop turns (about 8s on an idle Linux machine) to get near the limit, and on busy macos-x64 runners that went past the 60s timeout. It passed when re-run on its own. Allocate an array of 128K numbers per turn instead. The test now takes about 1s locally (1.2-1.7s with every core busy) and still produces the snapshot it checks for. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: wait for the guest to be destroyed in the USB/HID chooser specs The navigator.usb and navigator.hid "does not crash when the requesting webContents is destroyed from the select-*-device handler" specs call guest.destroy() in the handler and then check isDestroyed() after a single setTimeout() tick. destroy() only posts the teardown to the UI thread, so on a busy runner the check can run first and fail with "expected false to be true" (seen on macos-x64). Poll isDestroyed() until it is true instead. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: run the document.visibilityState/hidden specs serially "visibilityState changes when window is hidden" failed all four in-suite attempts on a macos-x64 main push with "expected 'hidden' to equal 'visible'" on the page's first report, before the test hides the window, and passed when re-run on its own. These specs run in the parallel phase, where other spec files open windows at the same time. On macOS a window that another one covers is occluded, and Chromium then reports its page as hidden. Tag the describe serial so its windows are the only ones on screen. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: run the protocol.handle throughput spec serially "protocol module handle is fast" compares the time to fetch a 128MB body directly and through a protocol.handle() interceptor and requires the median ratio to stay under 3. It ran in the parallel phase, where other spec files' Electron processes compete for the CPU and can slow one side of a pair more than the other; on the Chromium roll PR and on release branches it failed with "median ratio >= 3" and passed when re-run on its own. Tag it serial so the measurement has the machine to itself. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: wait longer for the preload code cache probe in its before hook The "preload code cache" suite's before hook loads a probe preload and waits up to 5s for its cache file to appear, to learn the process lock half of the file name. The browser writes that file from a BEST_EFFORT thread pool task, which a busy machine can hold back for longer than that; on macos-arm64 the hook failed with "timed out waiting for probe cache file to be written" and passed when re-run on its own. vitest does not retry hooks, so give this wait 30s (it returns as soon as the file is there) and the hook 60s. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: run the import/require parity fixture from a test, not a hook The "import / require parity" suite ran its fixture app, which starts a main, a utility and a renderer process, in a before hook with the 20s fixture timeout. On busy windows-x64 runners it sometimes did not finish in time ("Timed out after 20000ms waiting for ... import-require-parity to exit") and the suite only passed when re-run on its own. vitest does not retry hooks, so with that rerun gone a single slow launch would fail the job. Run the fixture from the first test instead and share the result with the other two. A failed or timed-out run is forgotten, so vitest's retry of that test starts a fresh one, like every other fixture launch in this file. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: skip already-destroyed webContents in cleanupWebContents() cleanupWebContents() takes a snapshot of webContents.getAllWebContents() and destroys each entry in turn. Destroying one can take others in the snapshot down with it (its guests or devtools, for example), and calling destroy() on one of those throws "Object has been destroyed". That failed the "chrome extensions" suite's after hook on linux-x64, and hooks are not retried, so the suite only passed when re-run on its own. Skip entries that are already destroyed. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: focus the webview before requesting fullscreen in the ESC spec Since Chromium 157 refuses requestFullscreen() from a view that doesn't have focus (SuppressFullscreenFromUnfocusedView, the same change #54883 handled for the simpleFullScreen spec), "pressing ESC should emit the leave-html-full-screen event" has needed in-suite retries on macOS. The test used a hidden window, whose page can never be focused. Use a shown window, take app activation on macOS as #54883 does, focus the window and then the <webview> element, and wait until the guest page reports document.hasFocus() before calling requestFullscreen(). Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: share the Network Service recovery probe between the crash specs Both crash-recovery specs now wait through waitForNetworkServiceRecovery(), which races each probe against a timeout and, for the utility process variant, the child's exit, so a stalled request or a dead child fails fast with its cause. The network-restart-test.js child is now registered with deferKillUtilityProcess so that a failed wait doesn't leak it. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: refocus the window on each poll in the webview ESC spec Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: give every preload code cache write wait the same longer timeout The per-test waits on the cache write still used waitFor's 5s default, while the beforeAll probe waiting on the same thread pool write had 30s. They now share cacheWriteTimeout, and the suite's test timeout leaves room for it. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: give the MSIX afterAll unregistration the setup timeout too Remove-AppxPackage runs in a hook under the same load as setup, and hooks are not retried. Both hook timeouts are now derived from powershellSetupTimeout so that they cover the commands they run. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: navigate the sandbox exit-event page cross-site to a local server The page now navigates to the suite's server on 127.0.0.1, so 'exit' is again covered on a cross-site, process-swapping navigation, as it was with google.com, without depending on the internet. The spec checks that the navigation did swap processes. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: run createWritable() through webFrameMain.executeJavaScript() webFrameMain.executeJavaScript() updates the browser process's user activation state before it runs the script, so the permission request no longer races the activation update. This replaces the extra activation round trip. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * build: reject a leftover --enableRerun in the spec runner It is no longer a known option, so it would otherwise be forwarded to the Electron workers as a switch that silently does nothing. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: break comments added in this PR at sentence and clause boundaries Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: keep the individual-test rerun for now (moved to a follow-up) Per review, this PR now only fixes the specs that passed thanks to the end-of-job solo rerun, so the fixes can be backported. Restore the spec runner's --enableRerun mode, its @xmldom/xmldom dependency, --enableRerun=3 in the test workflow and the vitest.config.ts comment to main's versions. Removing the rerun moves to a follow-up once main has been clean for a few days. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: go back to a same-site navigation in the sandbox exit-event spec A cross-site navigation unloads the old page through RenderFrameHostImpl::Unload, which gives the renderer only kUnloadTimeout (500ms) before the process is torn down. That is the race this spec originally lost on busy runners. Navigate to the local blank.html again, so the unload stays in the same renderer, and drop the process-swap check and the IPC handshake that only the cross-site variant needed. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: retry a Network Service recovery probe that times out A probe that got no response within 5s threw, and waitUntil fails on a thrown error, so one slow probe right after the crash failed the test although the next one would have succeeded. A timed-out probe now counts as a failed attempt; only the utility process exiting still fails right away. Since a probe can now be abandoned while its fetch is still in flight, the utility process fixture echoes an id sent with each fetch, and the spec waits for the reply with that id, so a late reply isn't taken as the reply to a later probe or to the final fetch. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: handle a rejected 'exit' wait before the first recovery probe once(child, 'exit') rejects if the utility process emits 'error', and nothing chained off it until the first recovery probe. An 'error' before then was an unhandled rejection, which fails the whole run rather than this test. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: log errors in the utility net fixture while it waits for the ack The fixture now stays up for a round trip after posting its result. In that window, leftover async work from the test body (a socket 'error' with no listener, a late unhandled rejection) could kill it with a non-zero code and fail a test that passed. While waiting for the ack, log such errors instead. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Move the Atomics.wait() call in the utilityProcess message-before-exit test into a documented sleepSync() spec helper. With pending parentPort messages now dispatched before 'exit', the api-net-spec.js fixture can post its result and exit right away again, so remove the ack handshake that #54901 added as a workaround, along with the error logging that covered the wait for the ack. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
…4924) * fix: deliver pending utilityProcess messages before emitting exit HandleTermination closed the parentPort connector before emitting 'exit', so messages the child had posted but the browser had not dispatched yet were dropped. The connector dispatches one message per task, and the exit notification can already be queued ahead of those tasks, so a child that posts several messages and then calls process.exit() lost all but the first. Dispatch whatever is already queued on the port before closing it. Browser shutdown still closes the port without dispatching, since the child is still running then. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: extract sleepSync helper and drop utility-process ack handshake Move the Atomics.wait() call in the utilityProcess message-before-exit test into a documented sleepSync() spec helper. With pending parentPort messages now dispatched before 'exit', the api-net-spec.js fixture can post its result and exit right away again, so remove the ack handshake that #54901 added as a workaround, along with the error logging that covered the wait for the ack. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS --------- Co-authored-by: Claude <noreply@anthropic.com>
Manual backport of #54901 to 42-x-y. Conflicts: dropped the navigator.usb/navigator.hid chooser hunks in spec/chromium.spec.ts (those tests do not exist on 42-x-y), and resolved spec/api-net.spec.ts against 42-x-y's older setTimeout(500) wait and #54925's net.fetch probe loop (both superseded by waitForNetworkServiceRecovery). (cherry picked from commit fc354a2) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
…4956) fix: deliver pending utilityProcess messages before emitting exit (#54924) * fix: deliver pending utilityProcess messages before emitting exit HandleTermination closed the parentPort connector before emitting 'exit', so messages the child had posted but the browser had not dispatched yet were dropped. The connector dispatches one message per task, and the exit notification can already be queued ahead of those tasks, so a child that posts several messages and then calls process.exit() lost all but the first. Dispatch whatever is already queued on the port before closing it. Browser shutdown still closes the port without dispatching, since the child is still running then. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: extract sleepSync helper and drop utility-process ack handshake Move the Atomics.wait() call in the utilityProcess message-before-exit test into a documented sleepSync() spec helper. With pending parentPort messages now dispatched before 'exit', the api-net-spec.js fixture can post its result and exit right away again, so remove the ack handshake that #54901 added as a workaround, along with the error logging that covered the wait for the ack. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS (cherry picked from commit 4a802b0) Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS Co-authored-by: Claude <noreply@anthropic.com>
…4960) fix: deliver pending utilityProcess messages before emitting exit (#54924) * fix: deliver pending utilityProcess messages before emitting exit HandleTermination closed the parentPort connector before emitting 'exit', so messages the child had posted but the browser had not dispatched yet were dropped. The connector dispatches one message per task, and the exit notification can already be queued ahead of those tasks, so a child that posts several messages and then calls process.exit() lost all but the first. Dispatch whatever is already queued on the port before closing it. Browser shutdown still closes the port without dispatching, since the child is still running then. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS * test: extract sleepSync helper and drop utility-process ack handshake Move the Atomics.wait() call in the utilityProcess message-before-exit test into a documented sleepSync() spec helper. With pending parentPort messages now dispatched before 'exit', the api-net-spec.js fixture can post its result and exit right away again, so remove the ack handshake that #54901 added as a workaround, along with the error logging that covered the wait for the ack. Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS (cherry picked from commit 4a802b0) Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>
… on failure (#54940) The setHeapSnapshotNearHeapLimit spec removed its diagnostic directory only when it passed, and nothing killed its utility process if an attempt failed or timed out. CI retries this test, and on upstream main it has been running close to its 60 s budget: passing macos-x64 runs take up to 56 s, and main run 38585284171 needed a retry on linux-x64-asan (98.6 s) and on macos-x64 (94.3 s). A timed-out attempt's child kept running during the retry, and each failed attempt left its directory behind in the temp dir. Register both cleanups with defer() as soon as they are needed: deferKillUtilityProcess() for the child right after fork(), and a forced fs.rm() for the directory right after mkdtemp(). defer() runs them in reverse order, so the child is killed before its directory is removed. The automatic snapshot path and the exactly-one, non-empty snapshot assertions are unchanged. The fixture that made the test slow is sped up in #54901. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
… on failure (#54980) The setHeapSnapshotNearHeapLimit spec removed its diagnostic directory only when it passed, and nothing killed its utility process if an attempt failed or timed out. CI retries this test, and on upstream main it has been running close to its 60 s budget: passing macos-x64 runs take up to 56 s, and main run 38585284171 needed a retry on linux-x64-asan (98.6 s) and on macos-x64 (94.3 s). A timed-out attempt's child kept running during the retry, and each failed attempt left its directory behind in the temp dir. Register both cleanups with defer() as soon as they are needed: deferKillUtilityProcess() for the child right after fork(), and a forced fs.rm() for the directory right after mkdtemp(). defer() runs them in reverse order, so the child is killed before its directory is removed. The automatic snapshot path and the exactly-one, non-empty snapshot assertions are unchanged. The fixture that made the test slow is sped up in #54901. Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com> Co-authored-by: Charles Kerr <charles@charleskerr.com>
… on failure (#54979) The setHeapSnapshotNearHeapLimit spec removed its diagnostic directory only when it passed, and nothing killed its utility process if an attempt failed or timed out. CI retries this test, and on upstream main it has been running close to its 60 s budget: passing macos-x64 runs take up to 56 s, and main run 38585284171 needed a retry on linux-x64-asan (98.6 s) and on macos-x64 (94.3 s). A timed-out attempt's child kept running during the retry, and each failed attempt left its directory behind in the temp dir. Register both cleanups with defer() as soon as they are needed: deferKillUtilityProcess() for the child right after fork(), and a forced fs.rm() for the directory right after mkdtemp(). defer() runs them in reverse order, so the child is killed before its directory is removed. The automatic snapshot path and the exactly-one, non-empty snapshot assertions are unchanged. The fixture that made the test slow is sped up in #54901. Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com> Co-authored-by: Charles Kerr <charles@charleskerr.com>
… on failure (#54978) * test: clean up the near-heap-limit snapshot test's child and temp dir on failure The setHeapSnapshotNearHeapLimit spec removed its diagnostic directory only when it passed, and nothing killed its utility process if an attempt failed or timed out. CI retries this test, and on upstream main it has been running close to its 60 s budget: passing macos-x64 runs take up to 56 s, and main run 38585284171 needed a retry on linux-x64-asan (98.6 s) and on macos-x64 (94.3 s). A timed-out attempt's child kept running during the retry, and each failed attempt left its directory behind in the temp dir. Register both cleanups with defer() as soon as they are needed: deferKillUtilityProcess() for the child right after fork(), and a forced fs.rm() for the directory right after mkdtemp(). defer() runs them in reverse order, so the child is killed before its directory is removed. The automatic snapshot path and the exactly-one, non-empty snapshot assertions are unchanged. The fixture that made the test slow is sped up in #54901. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Charles Kerr <charles@charleskerr.com> * test: add missing defer import to utility process spec The backport uses defer() from spec-helpers, but this branch never got the import change from #54861. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018UVE7tmkQ9eUoAG1a4aJqQ --------- Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com> Co-authored-by: Charles Kerr <charles@charleskerr.com> Co-authored-by: Claude <noreply@anthropic.com>
… on failure (#54977) * test: clean up the near-heap-limit snapshot test's child and temp dir on failure The setHeapSnapshotNearHeapLimit spec removed its diagnostic directory only when it passed, and nothing killed its utility process if an attempt failed or timed out. CI retries this test, and on upstream main it has been running close to its 60 s budget: passing macos-x64 runs take up to 56 s, and main run 38585284171 needed a retry on linux-x64-asan (98.6 s) and on macos-x64 (94.3 s). A timed-out attempt's child kept running during the retry, and each failed attempt left its directory behind in the temp dir. Register both cleanups with defer() as soon as they are needed: deferKillUtilityProcess() for the child right after fork(), and a forced fs.rm() for the directory right after mkdtemp(). defer() runs them in reverse order, so the child is killed before its directory is removed. The automatic snapshot path and the exactly-one, non-empty snapshot assertions are unchanged. The fixture that made the test slow is sped up in #54901. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Charles Kerr <charles@charleskerr.com> * test: add missing defer import to utility process spec The backport uses defer() from spec-helpers, but this branch never got the import change from #54861. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018UVE7tmkQ9eUoAG1a4aJqQ --------- Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com> Co-authored-by: Charles Kerr <charles@charleskerr.com> Co-authored-by: Claude <noreply@anthropic.com>
test: fix flaky specs that relied on the individual-test rerun (#54901) Manual backport of #54901 to 42-x-y. Conflicts: dropped the navigator.usb/navigator.hid chooser hunks in spec/chromium.spec.ts (those tests do not exist on 42-x-y), and resolved spec/api-net.spec.ts against 42-x-y's older setTimeout(500) wait and #54925's net.fetch probe loop (both superseded by waitForNetworkServiceRecovery). (cherry picked from commit fc354a2) Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS Co-authored-by: Claude <noreply@anthropic.com>
HandleTermination closed the parentPort connector before emitting 'exit', so messages the child had posted but the browser had not dispatched yet were dropped. The connector dispatches one message per task, and the exit notification can already be queued ahead of those tasks, so a child that posts several messages and then calls process.exit() lost all but the first. Dispatch whatever is already queued on the port before closing it. Browser shutdown still closes the port without dispatching, since the child is still running then. Also extract a sleepSync helper for the new spec, and drop the utility-process ack handshake that #54901 added as a workaround, since pending parentPort messages are now dispatched before 'exit'. Manual backport of #54924. Conflicts resolved: the header only adds the DispatchPendingMessages() declaration (43-x-y has no CloseStdioReadFds()), and the spec imports sleepSync into 43-x-y's existing spec-helpers import. (cherry picked from commit 4a802b0) (cherry picked from commit 53d6d27) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
…4962) HandleTermination closed the parentPort connector before emitting 'exit', so messages the child had posted but the browser had not dispatched yet were dropped. The connector dispatches one message per task, and the exit notification can already be queued ahead of those tasks, so a child that posts several messages and then calls process.exit() lost all but the first. Dispatch whatever is already queued on the port before closing it. Browser shutdown still closes the port without dispatching, since the child is still running then. Also extract a sleepSync helper for the new spec, and drop the utility-process ack handshake that #54901 added as a workaround, since pending parentPort messages are now dispatched before 'exit'. Manual backport of #54924. Conflicts resolved: the header only adds the DispatchPendingMessages() declaration (43-x-y has no CloseStdioReadFds()), and the spec imports sleepSync into 43-x-y's existing spec-helpers import. (cherry picked from commit 4a802b0) (cherry picked from commit 53d6d27) Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS Co-authored-by: Claude <noreply@anthropic.com>
…5000) HandleTermination closed the parentPort connector before emitting 'exit', so messages the child had posted but the browser had not dispatched yet were dropped. The connector dispatches one message per task, and the exit notification can already be queued ahead of those tasks, so a child that posts several messages and then calls process.exit() lost all but the first. Dispatch whatever is already queued on the port before closing it. Browser shutdown still closes the port without dispatching, since the child is still running then. Also extract a sleepSync helper for the new spec, and drop the utility-process ack handshake that #54901 added as a workaround, since pending parentPort messages are now dispatched before 'exit'. Manual backport of #54924. Conflicts resolved: the header only adds the DispatchPendingMessages() declaration (42-x-y has no CloseStdioReadFds()). The spec's sleepSync import merged cleanly into 42-x-y's existing spec-helpers import. (cherry picked from commit 4a802b0) (cherry picked from commit 53d6d27) Claude-Session: https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS Co-authored-by: Claude <noreply@anthropic.com>
Requested by John Kleinschmidt · Slack thread
Description of Change
This fixes the specs on main that only passed thanks to the spec runner's
--enableRerunmode, which re-runs every test that failed in a CI test job on its own in a fresh Electron at the end of the job and turns the job green if those solo runs pass. In the last 14 days, 38 of 65 main push builds went green only because of this solo rerun. Removing the rerun itself is a follow-up, to open once main has been clean for a few days (per review); it is parked onclaude/remove-spec-solo-rerun-only. This PR leaves the spec runner, the test workflow and the dependencies unchanged, so the fixes can be backported. Note that vitest's in-suiteretry: 3on CI does not retrybeforeAll/afterAllhooks, so the hook-level fixes below matter once the rerun is gone.The fixes, one commit each:
beforeAllhook installs a certificate and registers a sparse package with a 30s PowerShell timeout, which a busy host regularly overruns; those two setup commands now get 120s and the hook 300s. This one is a deliberate timeout increase, since the commands are just slow under load and finish when run alone.createWritable()with "User activation is required" because the activation and the request travel on different Mojo pipes in the sameexecuteJavaScript()call; the test now runscreateWritable()throughwebFrameMain'smainFrame.executeJavaScript(..., true), which sets the user activation on the browser side before the script runs (likely cause from the code and traces, not reproduced locally).setHeapSnapshotNearHeapLimit: the fixture grew the heap one short string per tick and could exceed 60s on macos-x64; it now allocates in large chunks (about 8s down to about 1s locally).isDestroyed()one timer tick afterdestroy(), which only posts the teardown; they now poll until it is destroyed.hidden; the describe is now serial.beforeAllhook: the probe cache file is written by a best-effort thread pool task that can take longer than 5s on a busy machine; the hook now waits up to 30s for it.beforeAllhook with a 20s timeout and no retry; it now runs from the first test, so a slow launch is retried like the other fixture launches in that file.afterAllhook ("Object has been destroyed"):cleanupWebContents()calleddestroy()on webContents that had already gone away with an earlier one; it now skips destroyed entries.requestFullscreen()from an unfocused view, the test now uses a shown window, takes app activation on macOS as test: focus the page before requesting fullscreen in simpleFullScreen spec #54883 did, and focuses the<webview>until the guest page has focus.Validation (after merging main with #54439's vitest API/expect conversion and #54902): spec and script typecheck,
script/lint.js --javascriptandoxfmt --checkpass. The Linux-runnable fixes were run together 3 times in a row with every core busy, plus once idle, and passed every time (apart from the twonet.fetchDNS-failure tests, which fail in that container regardless). The MSIX and utility-process Network Service crash recovery fixes (Windows, and a DCHECK-only testing binding) and the macOS-only aspects of the sandbox exit, heap snapshot, visibilityState and webview fullscreen fixes are only validated on Linux or not at all, so this PR's CI is their first real run.Not changed: "webFrameMain disposes detached frame after cross-origin navigation" (one main occurrence where all four in-suite attempts timed out after 10s each, so a longer wait isn't clearly a fix), and single occurrences without a clear cause (autoUpdater darwin invalid zip signature, two cpp heap timeouts, ServiceWorkerMain "starts previously stopped worker", requestSingleInstanceLock, crashReporter OOM crash keys). The contentTracing "accepts an empty config" rescue predates #54661.
Checklist
Release Notes
Notes: none
🤖 Generated with Claude Code
https://claude.ai/code/session_01LPMvJBQiyP6b7QjatACmnS
Generated by Claude Code