From: jaruga@... Date: 2020-06-08T15:32:07+00:00 Subject: [ruby-core:98677] [Ruby master Bug#16936] `make test-all TESTS="-n !/Foo#method/"` not skipping the test case Issue #16936 has been updated by jaruga (Jun Aruga). It seems there is an issue `[142/140]` when using `\^` in the pattern. ``` $ make V=1 test-all TESTS="test/-ext-/bug_reporter/test_bug_reporter.rb test/ruby/test_process.rb -v -n \!/\^test_bug_reporter_add\$$/ -n \!/\^test_status_quit\$$/" 2>&1 | tee make.log ... [ 1/140] TestBugReporter#test_bug_reporter_add = 0.44 s ... [128/140] TestProcess#test_status_quit = 0.31 s ... [142/140] TestProcess#test_waitall = 0.18 s Finished tests in 25.888517s, 5.4851 tests/s, 39.3997 assertions/s. 142 tests, 1020 assertions, 0 failures, 0 errors, 0 skips ``` Here is okay case without using `\^`. ``` $ make V=1 test-all TESTS="test/-ext-/bug_reporter/test_bug_reporter.rb test/ruby/test_process.rb -v -n \!/test_bug_reporter_add\$$/ -n \!/test_status_quit\$$/" 2>&1 | tee make.log ... [140/140] TestProcess#test_waitall = 0.18 s Finished tests in 25.745323s, 5.4379 tests/s, 39.0362 assertions/s. 140 tests, 1005 assertions, 0 failures, 0 errors, 0 skips ``` ---------------------------------------- Bug #16936: `make test-all TESTS="-n !/Foo#method/"` not skipping the test case https://bugs.ruby-lang.org/issues/16936#change-86024 * Author: jaruga (Jun Aruga) * Status: Closed * Priority: Normal * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN ---------------------------------------- On the current latest master `cf1adf985ab78507250db0e217a0fdd779e2c6e6`. ``` $ autoconf $ ./configure --prefix=$(pwd)/dest $ make ``` The following `make test-all` works as I mentioned at #16935. ``` $ make test-all TESTS="-v -n /\^TestBugReporter#test_bug_reporter_add\$$/ -n /\^TestProcess#test_status_quit\$$/" ... [1/0] TestBugReporter#test_bug_reporter_add = 0.41 s [2/0] TestProcess#test_status_quit = 0.35 s Finished tests in 9.392046s, 0.2129 tests/s, 1.5971 assertions/s. 2 tests, 15 assertions, 0 failures, 0 errors, 0 skips ... ``` But it seems that the following `TestBugReporter#test_bug_reporter_add` and `TestProcess#test_status_quit` are not actually skipped. Is it a bug? ``` $ make test-all TESTS="test/-ext-/bug_reporter/test_bug_reporter.rb test/ruby/test_process.rb -v -n \!/\^TestBugReporter#test_bug_reporter_add\$$/ -n \!/\^TestProcess#test_status_quit\$$/" 2>&1 | tee make.log Run options:- --seed=21367 "--ruby=./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems" --excludes-dir=./test/excludes --name=!/memory_leak/ -v -n "!/^TestBugReporter#test_bug_reporter_add$/" -n "!/^TestProcess#test_status_quit$/" # Running tests: [ 1/142] TestBugReporter#test_bug_reporter_add = 0.49 s ... [128/142] TestProcess#test_status_quit = 0.30 s ... Finished tests in 25.978847s, 5.4660 tests/s, 39.2627 assertions/s. 142 tests, 1020 assertions, 0 failures, 0 errors, 0 skips ``` I want to skip the specific test cases by `make test-all TESTS="..."` like this. I want to specify both the testing class (ex. `TestBugReporter`) and method (ex. `test_bug_reporter_add`) with the regular expression perfect matching to avoid unintended test cases are skipped. ``` $ make test-all TESTS="-v -n \!/\^TestBugReporter#test_bug_reporter_add\$$/ -n \!/\^TestProcess#test_status_quit\$$/" ``` The following `make test` works skipping `test_bug_reporter_add` and `test_status_quit` methods. ``` $ make test-all TESTS="test/-ext-/bug_reporter/test_bug_reporter.rb test/ruby/test_process.rb -v -n \!/test_bug_reporter_add\$$/ -n \!/test_status_quit\$$/" 2>&1 | tee make2.log ... 140 tests, 1005 assertions, 0 failures, 0 errors, 0 skips ``` -- https://bugs.ruby-lang.org/ Unsubscribe: