Skip to content

Comments

Attempt to make flaky test case more reliable#1013

Closed
senarclens wants to merge 10 commits intoexercism:mainfrom
senarclens:main
Closed

Attempt to make flaky test case more reliable#1013
senarclens wants to merge 10 commits intoexercism:mainfrom
senarclens:main

Conversation

@senarclens
Copy link
Contributor

Testing for code not to be vulnerable to race conditions is inherently hard. The prior version of the test unfortunately led to a lot of false negatives (ie the test passed about 1/3 of the time even the tested code wasn't thread safe). With these minimal adjustments the test is much more reliable (although not perfect).

senarclens and others added 10 commits November 19, 2024 21:04
thanks to Manuel Grobbauer for pointing this out
started by fixing a typo (exiting -> exciting) but then tried to reduce
the length of some sentences; added hints that references are expected
where appropriate to allow the reader to focus on the problem at hand
test used to pass a lot even if code wasn't thread safe
this is still possible now but less likely
avoid clang-format issue
@github-actions
Copy link
Contributor

Hello. Thanks for opening a PR on Exercism 🙂

We ask that all changes to Exercism are discussed on our Community Forum before being opened on GitHub. To enforce this, we automatically close all PRs that are submitted. That doesn't mean your PR is rejected but that we want the initial discussion about it to happen on our forum where a wide range of key contributors across the Exercism ecosystem can weigh in.

You can use this link to copy this into a new topic on the forum. If we decide the PR is appropriate, we'll reopen it and continue with it, so please don't delete your local branch.

If you're interested in learning more about this auto-responder, please read this blog post.


Note: If this PR has been pre-approved, please link back to this PR on the forum thread and a maintainer or staff member will reopen it.

@github-actions
Copy link
Contributor

This PR touches files which potentially affect the outcome of the tests of an exercise. This will cause all students' solutions to affected exercises to be re-tested.

If this PR does not affect the result of the test (or, for example, adds an edge case that is not worth rerunning all tests for), please add the following to the merge-commit message which will stops student's tests from re-running. Please copy-paste to avoid typos.

[no important files changed]

For more information, refer to the documentation. If you are unsure whether to add the message or not, please ping @exercism/maintainers-admin in a comment. Thank you!

@github-actions github-actions bot closed this Feb 22, 2026
@siebenschlaefer
Copy link
Contributor

siebenschlaefer commented Feb 22, 2026

I don't understand the reason for this PR.
The test is named "Can handle concurrent transactions" and is meant to catch solutions that do not handle concurrent deposits and withdrawals correctly. Reducing the number of threads makes it less likely that the test catches invalid solutions. Or am I misunderstanding something?

BTW: Thanks for catching the typo in the test name.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants