Mix test parameters into the per-test :rand seed - #15869
Merged
Merged
Conversation
ExUnit seeds each test process with {phash2(module), phash2(name), seed}.
parameterize re-runs the module per parameter set without changing the
test name, so every parameter set of a test started from the same :rand
state and drew the same random sequence.
Hash the parameters together with the name when they are present. Tests
without parameters keep their previous seed, so existing --seed
reproductions are unaffected.
Assisted-by: Claude Code:claude-fable-5-1
Signed-off-by: Jechol Lee <mr.jechol@gmail.com>
Member
|
💚 💙 💜 💛 ❤️ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ExUnit seeds each test process with
{phash2(module), phash2(name), suite_seed}.parameterizere-runs the module once per parameter set, changing onlytest.parametersand leavingtest.nameas is, so every parameterized run of a test starts from the same:randstate and draws the same random sequence.In practice this bites any test helper that generates random unique values. With Ecto's SQL sandbox, every parameter set inserts the same random email, and the unique index turns that into lock waits between the concurrent transactions, so the parameterized tests end up running one after another.
This PR hashes the parameters together with the name when they are present, so each parameter set gets its own seed. Tests without parameters keep their previous seed, so existing
--seedreproductions are unaffected and the constants in the existing "seed is predictable" test still pass.short_hash/3in the same file already takes this approach fortmp_dirpaths.The added test fails on the current runner and passes with the change.
Reproduction: https://github.com/jechol/repro-exunit-parameterize-seed
The code was written with the help of an AI coding agent and reviewed by me; see the
Assisted-bytag in the commit.