test_runner: support test order randomization by pmarchini · Pull Request #61747 · nodejs/node

pmarchini · 2026-02-08T22:09:32Z

This PR adds deterministic randomization to the Node.js test runner at both levels:

test file execution order
the order of tests inside each file

--test-randomize enables randomized execution, and --test-random-seed=<seed> replays the same order deterministically (and implicitly enables randomization).
When no seed is provided, one is generated and printed in the summary so runs are easy to reproduce.

nodejs-github-bot · 2026-02-08T22:09:37Z

Review requested:

@nodejs/config
@nodejs/test_runner

codecov · 2026-02-08T23:37:49Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.75%. Comparing base (ae2ffce) to head (84b6260).
⚠️ Report is 53 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff            @@
##             main   #61747    +/-   ##
========================================
  Coverage   89.75%   89.75%            
========================================
  Files         674      675     +1     
  Lines      204416   204828   +412     
  Branches    39285    39352    +67     
========================================
+ Hits       183472   183851   +379     
- Misses      13227    13257    +30     
- Partials     7717     7720     +3

Files with missing lines	Coverage Δ
lib/internal/test_runner/runner.js	`93.63% <100.00%> (+0.49%)`	⬆️
lib/internal/test_runner/test.js	`97.38% <100.00%> (+0.05%)`	⬆️
lib/internal/test_runner/utils.js	`65.61% <100.00%> (+3.58%)`	⬆️
src/node_options.cc	`76.47% <100.00%> (+0.19%)`	⬆️
src/node_options.h	`97.92% <100.00%> (+0.03%)`	⬆️

... and 62 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

doc/api/cli.md

doc/api/test.md

lib/internal/test_runner/runner.js

doc/api/cli.md

pmarchini · 2026-02-10T22:19:13Z

lib/internal/test_runner/test.js

    this.pendingSubtests = [];
    this.readySubtests = new SafeMap();
    this.unfinishedSubtests = new SafeSet();
    this.subtestsPromise = null;
    this.subtests = [];
+    this.nextReportOrder = 1;


I'm still not 100% sure about this approach...I'm still thinking about a less invasive solution

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Feb 8, 2026

pmarchini force-pushed the test_runner/deterministic-test-randomizer branch 2 times, most recently from 5556788 to e6f9a59 Compare February 8, 2026 22:20

pmarchini added test_runner Issues and PRs related to the test runner subsystem. semver-minor PRs that contain new features and should be released in the next minor version. labels Feb 8, 2026

pmarchini force-pushed the test_runner/deterministic-test-randomizer branch from e6f9a59 to b3b2388 Compare February 9, 2026 08:09

pmarchini marked this pull request as draft February 9, 2026 08:42

pmarchini force-pushed the test_runner/deterministic-test-randomizer branch from b3b2388 to 08db017 Compare February 9, 2026 09:19

pmarchini marked this pull request as ready for review February 9, 2026 09:19

pmarchini requested review from JakobJingleheimer, MoLow and atlowChemi February 9, 2026 11:19