| Age | Commit message (Collapse) | Author |
|---|
|
This change implements a fallback mode for the `--yjit-dump-disasm`
development command-line option to make it usable in release builds.
Previously, using the option with release builds of YJIT yielded only
a warning asking the user to build with `--enable-yjit=dev`.
While builds that use the `disasm` feature still give the best output,
just having the comments is useful enough for many kinds of debugging.
Having it usable in release builds is nice for new hackers, too, since
this allows for tinkering without having to learn how to build YJIT in
development mode.
Sample output on A64:
```
# regenerate_branch
# Insn: 0001 opt_send_without_block (stack_size: 1)
# guard known object with singleton class
0x11f7e0034: 4b 00 00 58 03 00 00 14 08 ce 9c 04 01 00 00
0x11f7e0043: 00 3f 00 0b eb 81 06 01 54 1f 20 03 d5
# RUBY_VM_CHECK_INTS(ec)
0x11f7e0050: 8b 02 42 b8 cb 07 01 35
# stack overflow check
0x11f7e0058: ab 62 02 91 7f 02 0b eb 69 07 01 54
# save PC to CFP
0x11f7e0064: 0b 3b 9a d2 2b 2f a0 f2 0b 00 cc f2 6b 02 00
0x11f7e0073: f8 ab 82 00 91
```
To ensure this feature doesn't incur too much cost when running without
the `--yjit-dump-disasm` option, I checked that there is no significant
impact to compile time and memory usage with the `compile_time_ns` and
`yjit_alloc_size` entry in `RubyVM::YJIT.runtime_stats`. For each
sample, I ran 3 iterations of the `lobsters` YJIT benchmark. The
statistics summary and done with the `summary` function in R.
Compile time, sample size of 60, lower is better:
```
Before After
Min. :2.054e+09 Min. :2.028e+09
1st Qu.:2.069e+09 1st Qu.:2.044e+09
Median :2.081e+09 Median :2.060e+09
Mean :2.089e+09 Mean :2.066e+09
3rd Qu.:2.109e+09 3rd Qu.:2.085e+09
Max. :2.146e+09 Max. :2.144e+09
```
Allocation size, sample size of 20, lower is better:
```
Before After
Min. :21804742 Min. :21794082
1st Qu.:21826682 1st Qu.:21816282
Median :21844042 Median :21826814
Mean :21960664 Mean :22026291
3rd Qu.:21861228 3rd Qu.:22040439
Max. :22587426 Max. :22930614
```
The `yjit_alloc_size` samples are noisy, but since the average increased
by only 0.3%, and the median is lower, I feel safe saying that there is
no significant change.
|
|
Type check now done in rb_iseqw_to_iseq().
|
|
This mainly aims to make `--yjit-dump-disasm=<relative_path>` more
usable. Previously, it crashed if the program did chdir(2), since it
opened the dump file every time when appending.
Tested with:
./miniruby --yjit-dump-disasm=. --yjit-call-threshold=1 -e 'Dir.chdir("/") {}'
And the `lobsters` benchmark.
|
|
We've long had a size restriction on the code memory region such that a
u32 could refer to everything. This commit capitalizes on this
restriction by shrinking the size of `CodePtr` to be 4 bytes from 8.
To derive a full raw pointer from a `CodePtr`, one needs a base pointer.
Both `CodeBlock` and `VirtualMemory` can be used for this purpose. The
base pointer is readily available everywhere, except for in the case of
the `jit_return` "branch". Generalize lea_label() to lea_jump_target()
in the IR to delay deriving the `jit_return` address until `compile()`,
when the base pointer is available.
On railsbench, this yields roughly a 1% reduction to `yjit_alloc_size`
(58,397,765 to 57,742,248).
|
|
Notes:
Merged-By: maximecb <[email protected]>
|
|
* YJIT: Avoid splitting mov for small values on arm64
* Fix a comment
Co-authored-by: Alan Wu <[email protected]>
* YJIT: Test the 0xffff boundary
---------
Co-authored-by: Alan Wu <[email protected]>
Notes:
Merged-By: k0kubun <[email protected]>
|
|
`Rc` and `RefCell` both incur runtime space costs.
In addition, `RefCell` has given us some headaches with the
non obvious borrow panics it likes to throw out. The latest
one started with 7fd53eeb46db261bbc20025cdab70096245a5cbe
and is yet to be resolved.
Since we already rely on the GC to properly reclaim memory for `Block`
and `Branch`, we might as well stop paying the overhead of `Rc` and
`RefCell`. The `RefCell` panics go away with this change, too.
On 25 iterations of `railsbench` with a stats build I got
`yjit_alloc_size: 8,386,129 => 7,348,637`, with the new memory size 87.6%
of the status quo. This makes the metadata and machine code size roughly
line up one-to-one.
The general idea here is to use `&` shared references with
[interior mutability][1] with `Cell`, which doesn't take any extra
space. The `noalias` requirement that `&mut` imposes is way too hard to
meet and verify. Imagine replacing places where we would've gotten
`BorrowError` from `RefCell` with Rust/LLVM miscompiling us due to aliasing
violations. With shared references, we don't have to think about subtle
cases like the GC _sometimes_ calling the mark callback while codegen
has an aliasing reference in a stack frame below. We mostly only need to
worry about liveness, with which the GC already helps.
There is now a clean split between blocks and branches that are not yet
fully constructed and ones that are "in-service", so to speak. Working
with `PendingBranch` and `JITState` don't really involve `unsafe` stuff.
This change allows `Branch` and `Block` to not have as many optional
fields as many of them are only optional during compilation. Fields that
change post-compilation are wrapped in `Cell` to facilitate mutation
through shared references.
I do some `unsafe` dances here. I've included just a couple tests to run
with Miri (`cargo +nightly miri test miri`). We can add more Miri tests
if desired.
[1]: https://doc.rust-lang.org/std/cell/struct.UnsafeCell.html
Notes:
Merged: https://github.com/ruby/ruby/pull/7443
|
|
Notes:
Merged-By: maximecb <[email protected]>
|
|
* YJIT: Allow testing assembler with disasm
* YJIT: Drop new dependencies
* YJIT: Avoid address manipulation
* YJIT: Introduce assert_disasm! macro
* YJIT: Update the comment about assert_disasm
Notes:
Merged-By: maximecb <[email protected]>
|
|
We set the block address as soon as we make the block, so there is no
point in making it `Option<CodePtr>`. No memory saving, unfortunately,
as `mem::size_of::<Block>() = 176` before and after this change. Still
a simplification for the logic, though.
Notes:
Merged: https://github.com/ruby/ruby/pull/7243
|
|
* YJIT: Colorize outlined code differently
on --yjit-dump-disasm
* YJIT: Reduce the number of escape sequences
Notes:
Merged-By: k0kubun <[email protected]>
|
|
Notes:
Merged-By: k0kubun <[email protected]>
|
|
YJIT: Respect writable_addrs on --yjit-dump-iseq-disasm
as well
Notes:
Merged-By: maximecb <[email protected]>
|
|
YJIT: Skip dumping code for the other cb
on --yjit-dump-disasm
Notes:
Merged-By: maximecb <[email protected]>
|
|
Notes:
Merged-By: maximecb <[email protected]>
|
|
* YJIT: Allow --yjit-dump-disasm to dump into a file
* YJIT: Move IO implementation to disasm.rs
* YJIT: More consistent naming
Notes:
Merged-By: k0kubun <[email protected]>
|
|
* YJIT: Count freed ISEQs
* YJIT: Avoid creating payloads for non-JITed ISEQs
Notes:
Merged-By: k0kubun <[email protected]>
|
|
* fixes more clippy warnings
* Fix x86 c_callable to have doc_strings
Notes:
Merged-By: maximecb <[email protected]>
|
|
Notes:
Merged-By: maximecb <[email protected]>
|
|
(https://github.com/Shopify/ruby/pull/430)
* Add --yjit-dump-disasm to dump every compiled code
* Just use get_option
* Carve out disasm_from_addr
* Avoid push_str with format!
* Share the logic through asm.compile
* This seems to negatively impact the compilation speed
Notes:
Merged: https://github.com/ruby/ruby/pull/6289
|
|
disasm (https://github.com/Shopify/ruby/pull/398)
|
|
|
|
Notes:
Merged: https://github.com/ruby/ruby/pull/6278
|
|
Follow up https://github.com/ruby/ruby/commit/0514d81
Rust YJIT requires Rust 1.60.0 or later. So, `extern crate` looks unnecessary
because it can use the following Rust 2018 edition feature:
https://doc.rust-lang.org/stable/edition-guide/rust-2018/path-changes.html#no-more-extern-crate
It passes the following tests.
```console
% cd yjit
% cargo test --features asm_comments,disasm
(snip)
test result: ok. 56 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
```
Notes:
Merged-By: maximecb <[email protected]>
|
|
`rustc` performs in depth dead code analysis and issues warning
even for things like unused struct fields and unconstructed enum
variants. This was annoying for us during the port but hopefully
they are less of an issue now.
This patch enables all the unused warnings we disabled and address
all the warnings we previously ignored. Generally, the approach I've
taken is to use `cfg!` instead of using the `cfg` attribute and
to delete code where it makes sense. I've put `#[allow(unused)]`
on things we intentionally keep around for printf style debugging
and on items that are too annoying to keep warning-free in all
build configs.
Notes:
Merged-By: maximecb <[email protected]>
|
|
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.
The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.
Because we want to be careful, YJIT is guarded behind a configure
option:
```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```
By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.
The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.
The CI tests have been updated and do not take any more resources than
before.
The development history of the Rust port is available at the following
commit for interested parties:
https://github.com/Shopify/ruby/commit/1fd9573d8b4b65219f1c2407f30a0a60e537f8be
Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.
[issue]: https://bugs.ruby-lang.org/issues/18481
Co-authored-by: Maxime Chevalier-Boisvert <[email protected]>
Co-authored-by: Noah Gibbs <[email protected]>
Co-authored-by: Kevin Newton <[email protected]>
Notes:
Merged: https://github.com/ruby/ruby/pull/5826
|
