From: "jaruga (Jun Aruga)" Date: 2022-11-25T10:41:00+00:00 Subject: [ruby-core:111008] [Ruby master Misc#19149] Minimal covered tests with the --enable-yjit case? Issue #19149 has been updated by jaruga (Jun Aruga). maximecb (Maxime Chevalier-Boisvert) wrote in #note-1: > If you want to thoroughly test everything, you could run make check two times, once with YJIT, and once without. > > If you just run: > ``` > $ make check RUN_OPTS="--yjit" > ``` > > This will run YJIT with its default options, which is what most people would use when running YJIT. If you specify `--yjit-call-threshold=1 --yjit-verify-ctx`, it will make YJIT compile every single method and run extra verifications. This is what we do on the CI to be extra thorough, but it will be slower. > > If you just want to quickly verify that YJIT works on the host platform, you could just run `make btest` with `--yjit` for some quick minimal verification. We do already test YJIT pretty well on the GitHub CI and on `rubyci.org`. Thank you for clarifying it. I understood it. I am still trying to find a good balance of running the YJIT tests between the running time and the coverage. In the case of "MJIT", while the MJIT is supported as a default. And we don't run the tests with RUN_OPTS="--mjit" in the Feodra Ruby's RPM recipe file, while I notice there are tests with `RUN_OPTS="--mjit"` now at . Because we can run the the following MJIT tests within the `make check` without `RUN_OPTS="--mjit"`. Here are the test results on the yesterday's latest master `66e5200ba435361624caa3e23db7962d906b70db`. ``` $ grep MJIT_SUPPORT rbconfig.rb CONFIG["MJIT_SUPPORT"] = "yes" $ make test-all TESTS=test/ruby/test_mjit.rb revision.h updated Run options: --seed=16704 "--ruby=./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems" --excludes-dir=./test/excludes --name=!/memory_leak/ # Running tests: Finished tests in 132.081114s, 0.7950 tests/s, 4.2020 assertions/s. 105 tests, 555 assertions, 0 failures, 0 errors, 4 skips ruby -v: ruby 3.2.0dev (2022-11-24T06:13:00Z master 66e5200ba4) [x86_64-linux] ``` However, we cannot run the following YJIT tests without `RUN_OPTS="--yjit"`. The tests were skipped. ``` $ grep YJIT_SUPPORT rbconfig.rb CONFIG["YJIT_SUPPORT"] = "dev" $ make test-all TESTS=test/ruby/test_yjit.rb revision.h updated Run options: --seed=38677 "--ruby=./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems" --excludes-dir=./test/excludes --name=!/memory_leak/ # Running tests: Finished tests in 0.017483s, 0.0000 tests/s, 0.0000 assertions/s. 0 tests, 0 assertions, 0 failures, 0 errors, 0 skips ruby -v: ruby 3.2.0dev (2022-11-24T06:13:00Z master 66e5200ba4) [x86_64-linux] ``` What does the difference of the ideas come from? ---------------------------------------- Misc #19149: Minimal covered tests with the --enable-yjit case? https://bugs.ruby-lang.org/issues/19149#change-100258 * Author: jaruga (Jun Aruga) * Status: Open * Priority: Normal ---------------------------------------- In the [Fedora Ruby's RPM recipe file](https://src.fedoraproject.org/rpms/ruby/blob/rawhide/f/ruby.spec), we were running the commands below. ``` $ ./autogen.sh $ ./configure ... $ make $ make check ``` ## What is the minimal covered tests? Now the we want to add the `--enable-yjit` option. Could you tell me what is the minimal covered test commands in the `--enable-yjit` case? ``` $ ./autogen.sh $ ./configure --enable-yjit ... $ make $ ?? ``` Then do we need to run the `make check` 2 times with both with yjit and without yjit as follows? The yjit command options `--yjit-call-threshold=1 --yjit-verify-ctx` comes from the . ``` $ make check $ make check RUN_OPTS="--yjit-call-threshold=1 --yjit-verify-ctx" ``` Or is it good enough to run the `make check` and the specific tests with the yjit options as follows? ``` $ make check $ make test-all RUN_OPTS="--yjit-call-threshold=1 --yjit-verify-ctx" TESTS="test/ruby/{test_yjit_exit_locations.rb,test_yjit.rb}" ``` Or is it good enough to run the `make check` with the YJIT options as follows? ``` $ make check RUN_OPTS="--yjit-call-threshold=1 --yjit-verify-ctx" ``` ## YJIT command options Could you explain why the command options `--yjit-call-threshold=1 --yjit-verify-ctx` above is better to test the YJIT cases rather than just `--yjit`? ## Ideal situation I want to see the just running `make check` covers necessary cases in YJIT. Because it is convenience, and I think users tend to be satisfied with only running the `make check`. What do you think? ``` $ ./autogen.sh $ ./configure --enable-yjit ... $ make $ make check ``` I tried to inject the YJIT command options in a test file for that. Perhaps it might be like this. But so far I am not succeeded. ``` diff --git a/test/lib/jit_support.rb b/test/lib/jit_support.rb index 26f8542dc2..3fce402e32 100644 --- a/test/lib/jit_support.rb +++ b/test/lib/jit_support.rb @@ -69,8 +69,10 @@ def supported? end def yjit_supported? + return @yjit_supported if defined?(@yjit_supported) # e.g. x86_64-linux, x64-mswin64_140, x64-mingw32, x64-mingw-ucrt - RUBY_PLATFORM.match?(/^(x86_64|x64|arm64|aarch64)-/) + @yjit_supported = RbConfig::CONFIG["YJIT_SUPPORT"] != 'no' && + RUBY_PLATFORM.match?(/^(x86_64|x64|arm64|aarch64)-/) end def remove_mjit_logs(stderr) diff --git a/test/ruby/test_yjit.rb b/test/ruby/test_yjit.rb index 9ab058d97b..10c8e3b891 100644 --- a/test/ruby/test_yjit.rb +++ b/test/ruby/test_yjit.rb @@ -8,7 +8,7 @@ require 'tmpdir' require_relative '../lib/jit_support' -return unless defined?(RubyVM::YJIT) && RubyVM::YJIT.enabled? +return unless JITSupport.yjit_supported? # Tests for YJIT with assertions on compilation and side exits # insipired by the MJIT tests in test/ruby/test_mjit.rb ``` -- https://bugs.ruby-lang.org/ Unsubscribe: