Skip to content

test(javascript): run what the minifier writes for swc's execution tests - #22355

Closed
aryanraj45 wants to merge 2 commits into
webpack:mainfrom
aryanraj45:test/js-minify-swc-exec
Closed

aryanraj45 wants to merge 2 commits into
webpack:mainfrom
aryanraj45:test/js-minify-swc-exec

Conversation

@aryanraj45

@aryanraj45 aryanraj45 commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Summary

The stdout suite now runs swc's execution tests as well as terser's. There are two sets. The first is the tests in exec.rs: swc's harness runs each input with node and requires the minified output to print the same. The second is the fixtures that carry an expected.stdout, leaving out swc's port of terser's own tests, which the suite already runs. Each test is minified by webpack under its own options and under the options webpack ships with (compress: { passes: 2 }, mangle: true), run in terser's sandbox, and has to print what its input prints.

Of the 502 exec tests, the sandbox runs 497 inputs; the other 5 throw or print nothing there. Of the 150 fixtures, 135 print their expected.stdout in the sandbox. Of the other 15, 13 read what only node gives them, such as process or a timer, and 2 are modules. A floor on each count keeps a reader that stopped matching from passing quietly.

24 outputs print something other than their input. Published terser's output prints the same thing in every one of them, so they are terser's own miscompiles, and webpack adds none. An output therefore fails only where the reference's output does not fail the same way. The suite takes about 30 s, up from 25 s.

The readers for swc's fixtures, exec.rs and mangle.rs, and the probe of which options terser reads, move unchanged from the js-minify suite into test/helpers/swcMinifierTests.js so both suites share them. The js-minify leg still passes all 232 tests. The js-minify-stdout CI leg now also checks out swc.

What kind of change does this PR introduce?

test

Did you add tests for your changes?

Yes, the corpus is the test. I broke the checks and each phase on purpose, then restored them:

  • With the reference check turned off, both swc tests fail on the 24 outputs terser also gets wrong.
  • Skipping every input fails both floors, at 0.
  • In the scope phase, not marking eval's scopes fails 18 runs here, where terser's corpus fails 5 files.
  • Skipping the unused phase's second pass fails 72 runs here, beside 40 files in terser's corpus.
  • Not rebuilding scope before mangling in the minify phase fails 3 runs here, beside 94 files.
  • Descending past a visitor that stopped the walk fails 1 run here, beside 1 file.
  • Never hoisting fails nothing here, since the hoist phase only skips or hands the scope to terser's own. The byte-identical suite catches it in 17 files.
  • Each earlier sabotage (print, parse, output, mangle) still fails the same terser files as before, and now both swc tests too.

Does this PR introduce a breaking change?

No. Only tests and CI change.

If relevant, what needs to be documented once your changes are merged or what have you already documented?

n/a

Use of AI
AI was used. I found the execution tests in swc's tree and reproduced how its harness runs them. It checked each of the 24 differing outputs against published terser, measured how many inputs terser's sandbox can run, and broke the new phases one by one to measure what each corpus catches. I reviewed the result.

Summary by CodeRabbit

  • Tests
    • JavaScript minification tests now cover cases from both Terser and SWC, including checks of generated output and execution results.
    • Existing Terser test coverage remains in place, alongside eligible SWC fixtures and execution cases when the SWC corpus is available.

The readers for swc's fixtures, exec.rs and mangle.rs, and the probe of which
options terser reads, move out of the js-minify suite unchanged, so the stdout
suite can run the same tests. js-minify still passes all 232.
swc's exec.rs tests and its fixtures with an expected.stdout now run in the
stdout suite under their own options and webpack's: 497 and 135 whose input
terser's sandbox runs. 24 outputs print otherwise, each as terser's own does,
so an output fails only where the reference's does not. A broken scope phase
fails 18 runs here, unused 72.
@changeset-bot

changeset-bot Bot commented Sep 29, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 4af7359

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: webpack/webpack/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 158738f7-23bc-45c5-b35c-3354f20c2f49

📥 Commits

Reviewing files that changed from the base of the PR and between d7f7e56 and 4af7359.

📒 Files selected for processing (4)
  • .github/workflows/test.yml
  • test/helpers/swcMinifierTests.js
  • test/specCases/js-minify-stdout.spectest.js
  • test/specCases/js-minify.spectest.js

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

The test workflow now checks out the SWC corpus. Shared helpers read SWC cases and options, and the stdout suite checks eligible SWC cases alongside the existing Terser cases.

Changes

Minifier corpus tests

Layer / File(s) Summary
SWC corpus readers and shared option detection
test/helpers/swcMinifierTests.js, test/specCases/js-minify.spectest.js
The shared helper reads SWC fixture, exec, and mangle tests, detects module inputs, and filters options using Terser’s supported keys. The existing SWC readers and option detection now use these helpers.
Shared output checks and corpus integration
test/specCases/js-minify-stdout.spectest.js, .github/workflows/test.yml
The stdout suite routes Terser cases through shared output checks and adds eligible SWC exec and fixture cases. The workflow checks out both corpora.

Suggested reviewers: alexander-akait

Priority: ⬇️ Low

Change: Other

Merge Risk: ⚪ Minimal · up to 4af73

This change only extends test coverage: webpack's minifier output is now also checked against SWC's execution tests, and CI checks out the SWC corpus for that job. Production behavior is unchanged, and no merge-blocking issue was found.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 4af73

The added execution is confined to the CI test flow and uses its existing sandbox. The external corpus is pinned and the job has read-only repository permissions. The sandbox’s isolation could not be independently verified, so the risk is not assessed as minimal.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The visible expansion is execution of more externally sourced cases in the stdout CI process, not a new production-facing entrypoint. The sandbox’s effective containment is unknown.

Trust Boundaries and Controls

  • observed — SWC corpus contents cross from a pinned submodule into sandbox.runCode. Eligibility checks filter cases, and the job’s declared repository permission is read-only; neither establishes the sandbox’s internal authority restrictions.

Hardening Proposals

  • proposed — Verify the pinned Terser sandbox’s available process, filesystem, network, and child-process authority, and confirm that the configured timeout covers its execution path before relying on it as an isolation control.
🚥 Pre-merge checks | ✅ 3 | ❓ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title uses valid Conventional Commit syntax with the allowed type test, the optional scope javascript, and a subject that describes the SWC execution-test changes. The branch prefix is not pro… Provide the branch name or branch prefix to confirm that test matches it.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title uses valid Conventional Commit syntax with the allowed type test, the optional scope javascript, and a subject that describes the SWC execution-test changes. The branch prefix is not provided, so that requirement cannot be verified.

Warning

Some tools did not complete. Review the errors below.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

test/helpers/swcMinifierTests.js

ESLint failed to execute (timeout).

test/specCases/js-minify-stdout.spectest.js

ESLint skipped: the matched ESLint configuration already failed (timeout).

test/specCases/js-minify.spectest.js

ESLint skipped: the matched ESLint configuration already failed (timeout).


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

Reported by Types Coverage #239 for 4af7359, computed by this pull request's own code.

Types Coverage

Coverage after merging test/js-minify-swc-exec into main will be
97.05%
Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
bin
   webpack.js98.82%100%100%98.82%103
declarations
   WebpackOptions.ts99.94%100%100%99.94%4147
examples
   build-common.js100%100%100%100%
   buildAll.js100%100%100%100%
   examples.js100%100%100%100%
   template-common.js98.21%100%100%98.21%72
examples/custom-javascript-parser
   test.filter.js100%100%100%100%
examples/custom-javascript-parser/internals
   acorn-parse.js100%100%100%100%
   meriyah-parse.js100%100%100%100%
   oxc-parse.js100%100%100%100%
examples/markdown
   webpack.config.mjs100%100%100%100%
examples/module-federation
   test.filter.js100%100%100%100%
examples/reexport-components
   test.filter.js100%100%100%100%
examples/ssr
   test.filter.js100%100%100%100%
examples/typescript
   test.filter.js100%100%100%100%
examples/typescript-non-erasable
   test.filter.js50%100%100%50%5
examples/virtual-modules
   test.filter.js100%100%100%100%
examples/wasm-bindgen-esm
   test.filter.js100%100%100%100%
examples/wasm-complex
   test.filter.js100%100%100%100%
examples/wasm-emscripten
   test.filter.js100%100%100%100%
examples/wasm-simple
   test.filter.js100%100%100%100%
examples/wasm-simple-source-phase
   test.filter.js100%100%100%100%
lib
   Chunk.js100%100%100%100%
   Compilation.js98.59%100%100%98.59%1757, 2078, 2085, 2093, 2115, 2118, 23, 3054, 3533–3534, 3566, 4155, 4159, 4286, 4307, 4432, 4469, 4476, 4529–4530, 4534, 4539, 4555–4556, 4570–4571, 4576–4577, 5100, 5126, 5956, 5973, 5991, 6007, 6022, 6047, 6050, 6418, 6421–6422, 6427, 6442, 6525, 6549, 6663, 837–838
   Compiler.js98.81%100%100%98.81%1116, 1153, 1156, 1336–1337, 1345, 848, 934, 975
   Dependency.js100%100%100%100%
   DependencyTemplate.js100%100%100%100%
   EntryPlugin.js100%100%100%100%
   Entrypoint.js100%100%100%100%
   ExternalsPlugin.js100%100%100%100%
   FileSystemInfo.js100%100%100%100%
   InitFragment.js100%100%100%100%
   LibraryTemplatePlugin.js100%100%100%100%
   ModuleFilenameHelpers.js100%100%100%100%
   ModuleNotFoundError.js100%100%100%100%
   MultiCompiler.js98.88%100%100%98.88%460, 532, 719, 784
   NormalModule.js100%100%100%100%
   NullFactory.js100%100%100%100%
   ProgressPlugin.js100%100%100%100%
   RawModule.js100%100%100%100%
   RequestShortener.js100%100%100%100%
   SingleEntryPlugin.js100%100%100%100%
   WebpackError.js100%100%100%100%
   index.js99.77%100%100%99.77%206
   webpack.js97.84%100%100%97.84%278, 300, 302
lib/asset
   AssetBytesGenerator.js100%100%100%100%
   AssetBytesParser.js100%100%100%100%
   AssetGenerator.js100%100%100%100%
   AssetModule.js100%100%100%100%
   AssetModulesPlugin.js98.15%100%100%98.15%338, 362, 365, 495, 57, 62
   AssetParser.js100%100%100%100%
   AssetSourceGenerator.js100%100%100%100%
   AssetSourceParser.js100%100%100%100%
   RawDataUrlModule.js100%100%100%100%
   WebManifestGenerator.js100%100%100%100%
   WebManifestParser.js100%100%100%100%
lib/async-modules
   AsyncModuleHelpers.js100%100%100%100%
   AwaitDependenciesInitFragment.js100%100%100%100%
   InferAsyncModulesPlugin.js100%100%100%100%
   isGeneratorLowered.js100%100%100%100%
lib/bun
   BunTargetPlugin.js100%100%100%100%
lib/cache
   AddBuildDependenciesPlugin.js100%100%100%100%
   AddManagedPathsPlugin.js100%100%100%100%
   Cache.js96.49%100%100%96.49%10, 104
   CacheFacade.js100%100%100%100%
   IdleFileCachePlugin.js97.92%100%100%97.92%75, 87, 95
   MemoryCachePlugin.js92%100%100%92%33, 42
   MemoryWithGcCachePlugin.js93.42%100%100%93.42%106, 120–121, 130, 88
   PackFileCacheStrategy.js96.54%100%100%96.54%1304, 1404, 1408, 1470, 1716, 1800, 1823, 1853, 682, 701, 711–713, 715, 731–732, 737, 740, 742, 747, 752, 777, 783, 817, 823, 829, 834, 845, 854, 859–860, 862, 879, 885–886, 888
   ResolverCachePlugin.js100%100%100%100%
   getLazyHashedEtag.js100%100%100%100%
   mergeEtags.js100%100%100%100%
lib/config
   OptionsApply.js100%100%100%100%
   PlatformPlugin.js100%100%100%100%
   WebpackOptionsApply.js100%100%100%100%
   WebpackOptionsDefaulter.js100%100%100%100%
   browserslistTargetHandler.js100%100%100%100%
   cli.js98.61%100%100%98.61%116, 546, 578, 628, 9
   defaults.js99.44%100%100%99.44%1623–1625, 1633, 283, 286, 291, 295
   defineConfig.js100%100%100%100%
   getClaimedAssetTypes.js100%100%100%100%
   normalization.js99.12%100%100%99.12%279–280, 346, 361
   target.js100%100%100%100%
   validateSchema.js94.67%100%100%94.67%100, 87, 89, 98
lib/container
   ContainerEntryDependency.js100%100%100%100%
   ContainerEntryModule.js100%100%100%100%
   ContainerEntryModuleFactory.js100%100%100%100%
   ContainerExposedDependency.js100%100%100%100%
   ContainerPlugin.js100%100%100%100%
   ContainerReferencePlugin.js100%100%100%100%
   FallbackDependency.js100%100%100%100%
   FallbackItemDependency.js100%100%100%100%
   FallbackModule.js100%100%100%100%
   FallbackModuleFactory.js100%100%100%100%
   HoistContainerReferencesPlugin.js100%100%100%100%
   ModuleFederationPlugin.js100%100%100%100%
   RemoteModule.js100%100%100%100%
   RemoteRuntimeModule.js100%100%100%100%
   RemoteToExternalDependency.js100%100%100%100%
   declaredRemotes.js100%100%100%100%
   moduleFederationHooks.js100%100%100%100%
   options.js100%100%100%100%
lib/context
   ContextExclusionPlugin.js100%100%100%100%
   ContextModule.js99.88%100%100%99.88%1494
   ContextModuleFactory.js95.66%100%100%95.66%230, 257, 276, 280–281, 289, 465, 486, 491, 532, 541, 543, 547, 550, 556–557
   ContextReplacementPlugin.js100%100%100%100%
lib/css
   CssGenerator.js99.47%100%100%99.47%1085, 136, 842
   CssInjectStyleRuntimeModule.js100%100%100%100%
   CssLoadingRuntimeModule.js100%100%100%100%
   CssModule.js83.16%100%100%83.16%197, 202–210, 212–217
   CssModulesPlugin.js99.86%100%100%99.86%886
   CssParser.js99.44%100%100%99.44%2227, 24, 2861, 2864, 2870, 2911, 2914, 2953, 2956, 3293, 4698, 4701, 843, 959, 990
   CssServerStylesRuntimeModule.js100%100%100%100%
   conventions.js98.65%100%100%98.65%93
   cssMinify.js96.30%100%100%96.30%127, 136
   data.js99.83%100%100%99.83%224, 4043
   syntax-parser.js99.71%100%100%99.71%10136–10137, 10174, 10355, 10401, 12267, 12765, 13118, 14702, 2396, 3095, 3097, 3102, 3121, 3866, 3872, 7082, 7120, 7127, 9601–9602
   syntax-printer.js99.23%100%100%99.23%2282–2283, 2285–2286, 2288–2289, 2294, 2555–2558, 3341, 3474, 3583, 4022, 4271, 5050, 621, 80
   syntax.js100%100%100%100%
   topologicalSort.js100%100%100%100%
lib/debug
   ProfilingPlugin.js96.22%100%100%96.22%169, 425, 554, 561, 570, 574, 583, 588, 594, 599, 92
lib/define
   DefinePlugin.js98.85%100%100%98.85%1495, 268–269, 285, 304, 378, 936, 938, 943, 967
   DotenvPlugin.js98.07%100%100%98.07%391, 439, 452–453
   EnvironmentPlugin.js97.14%100%100%97.14%49
lib/deno
   DenoTargetPlugin.js100%100%100%100%
lib/dependencies
   ConstDependency.js100%100%100%100%
   ContextElementDependency.js100%100%100%100%
   NullDependency.js100%100%100%100%
lib/dependencies/amd
   AMDDefineDependency.js100%100%100%100%
   AMDDefineDependencyParserPlugin.js100%100%100%100%
   AMDPlugin.js99.31%100%100%99.31%54
   AMDRequireArrayDependency.js100%100%100%100%
   AMDRequireContextDependency.js100%100%100%100%
   AMDRequireDependenciesBlock.js100%100%100%100%
   AMDRequireDependenciesBlockParserPlugin.js100%100%100%100%
   AMDRequireDependency.js100%100%100%100%
   AMDRequireItemDependency.js100%100%100%100%
   

@codecov

codecov Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.74%. Comparing base (d7f7e56) to head (4af7359).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #22355      +/-   ##
==========================================
+ Coverage   94.50%   95.74%   +1.23%     
==========================================
  Files         767      770       +3     
  Lines      109227   109487     +260     
  Branches    34138    34199      +61     
==========================================
+ Hits       103230   104825    +1595     
+ Misses       5997     4662    -1335     
Flag Coverage Δ
css-parsing 25.63% <ø> (ø)
html5lib 27.22% <ø> (ø)
integration 84.17% <ø> (+1.83%) ⬆️
syntax-equivalence 79.17% <ø> (ø)
test262 48.50% <ø> (-0.01%) ⬇️
unit 76.24% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

Reported by Code Size #3231 for 4af7359, computed by this pull request's own code.

Generated code size

Comparing base (d7f7e56) to head (4af7359). Merging this pull request will not change the code webpack generates.

What moved Tests Assets Gzip Raw Test edit
Changed, test untouched 0 0 — — —
Changed, test edited 0 0 — — —
New 0 0 — — —
Deleted 0 0 — — —
Unchanged — 8279 — — —

Read Changed, test untouched first: the test is byte-identical on both sides, so webpack generated the difference. Changed, test edited moved partly because the test did — Test edit is how many bytes the test gained, and a bundle that grew by less is not a regression. New and Deleted are whole files, not deltas — a new asset was created, not grown, so it reads gray (⚪ added) — and adding tests cannot bury a real change. Gzip decides, because that is what users download; raw is the tiebreak, and brotli and zstd are per asset below.

No asset changed size.

No runtime gained or lost a runtime module.

Built test/configCases with the defaults a user gets: 2287 test(s), 8279 asset(s), 96 emitted nothing.

@codspeed

codspeed Bot commented Sep 29, 2026

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 1 regressed benchmark
✅ 399 untouched benchmarks
⏩ 6 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Memory benchmark "asset-modules-source", scenario '{"name":"mode-production","mode":"production"}' 5.9 MB 8.4 MB -29.92%
⚡ Memory unit benchmark "js-parser-unit", source 'lodash-es' 305.8 KB 106.5 KB ×2.9
⚡ Memory benchmark "css-modules", scenario '{"name":"mode-development","mode":"development"}' 12.3 MB 8.9 MB +38.23%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing aryanraj45:test/js-minify-swc-exec (4af7359) with main (d7f7e56)

Open in CodSpeed

Footnotes

  1. 6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Copy link
Copy Markdown
Member

Refactored and shipped in #22389 (building on #22352), thanks @aryanraj45!


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants