From: "ono-max (Naoto Ono) via ruby-core" Date: 2024-08-02T06:50:15+00:00 Subject: [ruby-core:118777] [Ruby master Misc#20661] Stop retrying tests in `make test-all` command by default Issue #20661 has been reported by ono-max (Naoto Ono). ---------------------------------------- Misc #20661: Stop retrying tests in `make test-all` command by default https://bugs.ruby-lang.org/issues/20661 * Author: ono-max (Naoto Ono) * Status: Open * Assignee: ono-max (Naoto Ono) ---------------------------------------- ## Summary Currently, tests are retried once if they fail when tests are executed as parallel tests in the `make test-all` command by default. This is to prevent test failures caused by parallel testing. That's why failed tests are executed serially, not in parallel. Source code: https://github.com/ruby/ruby/blob/master/tool/lib/test/unit.rb#L728-L751 However, there is a problem, which might hide a real issue like https://bugs.ruby-lang.org/issues/20314. To prevent missing this "real" problem, stopping the retry of tests in the `make test-all` command by default would be better. ## Concerns when stopping the retry of tests in the `make test-all` command Here are some concerns about stopping the retry mechanism in `make test-all` command. #### 1. There are some flaky tests in the test suite. Are we okay with that? Yes, [all flaky tests are tracked by Launchable](https://app.launchableinc.com/organizations/ruby/workspaces/ruby/insights/unhealthy-tests), and they're mainly monitored by @mame. Thus, they'll be fixed in the future. #### 2. When a test that is not related to my change fails, what should we do? Just click the "Re-run jobs" button via the GitHub UI. -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- ruby-core@ml.ruby-lang.org To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/