| Age | Commit message (Collapse) | Author |
|---|
|
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.
The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.
Because we want to be careful, YJIT is guarded behind a configure
option:
```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```
By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.
The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.
The CI tests have been updated and do not take any more resources than
before.
The development history of the Rust port is available at the following
commit for interested parties:
https://github.com/Shopify/ruby/commit/1fd9573d8b4b65219f1c2407f30a0a60e537f8be
Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.
[issue]: https://bugs.ruby-lang.org/issues/18481
Co-authored-by: Maxime Chevalier-Boisvert <[email protected]>
Co-authored-by: Noah Gibbs <[email protected]>
Co-authored-by: Kevin Newton <[email protected]>
Notes:
Merged: https://github.com/ruby/ruby/pull/5826
|
|
No one reads it at the moment and it's heap allocated.
Notes:
Merged: https://github.com/ruby/ruby/pull/5278
Merged-By: XrXr
|
|
Previously, YJIT assumed that it's always possible to generate a new
basic block when servicing a stub in branch_stub_hit(). When YJIT is out
of executable memory, for example, this assumption doesn't hold up.
Add handling to branch_stub_hit() for servicing stubs without consuming
more executable memory by adding a code path that exits to the
interpreter at the location the branch stub represents. The new code
path reconstructs interpreter state in branch_stub_hit() and then exits
with a new snippet called `code_for_exit_from_stub` that returns
`Qundef` from the YJIT native stack frame.
As this change adds another place where we regenerate code from
`branch_t`, extract the logic for it into a new function and call it
regenerate_branch(). While we are at it, make the branch shrinking code
path in branch_stub_hit() more explicit.
This new functionality is hard to test without full support for out of
memory conditions. To verify this change, I ran
`RUBY_YJIT_ENABLE=1 make check -j12` with the following patch to stress
test the new code path:
```diff
diff --git a/yjit_core.c b/yjit_core.c
index 4ab63d9806..5788b8c5ed 100644
--- a/yjit_core.c
+++ b/yjit_core.c
@@ -878,8 +878,12 @@ branch_stub_hit(branch_t *branch, const uint32_t target_idx, rb_execution_contex
cb_set_write_ptr(cb, branch->end_addr);
}
+if (rand() < RAND_MAX/2) {
// Compile the new block version
p_block = gen_block_version(target, target_ctx, ec);
+}else{
+ p_block = NULL;
+}
if (!p_block && branch_modified) {
// We couldn't generate a new block for the branch, but we modified the branch.
```
We can enable the new test along with other OOM tests once full support
lands.
Other small changes:
* yjit_utils.c (print_str): Update to work with new native frame shape.
Follow up for 8fa0ee4d404.
* yjit_iface.c (rb_yjit_init): Run yjit_init_core() after
yjit_init_codegen() so `cb` and `ocb` are available.
Notes:
Merged: https://github.com/ruby/ruby/pull/5180
Merged-By: XrXr
|
|
This commit adds an entry_exit field to block_t for use in
invalidate_block_version(). By patching the start of the block while
invalidating it, invalidate_block_version() can function correctly
while there is no executable memory left for new branch stubs.
This change additionally fixes correctness for situations where we
cannot patch incoming jumps to the invalidated block. In situations
such as Shopify/yjit#226, the address to the start of the block
is saved and used later, possibly after the block is invalidated.
The assume_* family of function now generate block->entry_exit before
remembering blocks for invalidation.
RubyVM::YJIT.simulate_oom! is introduced for testing out of memory
conditions. The test for it is disabled for now because OOM triggers
other failure conditions not addressed by this commit.
Fixes Shopify/yjit#226
Notes:
Merged: https://github.com/ruby/ruby/pull/5145
|
|
* New code page allocation logic
* Fix leaked globals
* Fix leaked symbols, yjit asm tests
* Make COUNTED_EXIT take a jit argument, so we can eliminate global ocb
* Remove extra whitespace
* Change block start_pos/end_pos to be pointers instead of uint32_t
* Change branch end_pos and start_pos to end_addr, start_addr
Notes:
Merged-By: maximecb <[email protected]>
|
|
|
|
For upstreaming, we want functions we export either prefixed with "rb_"
or made static. Historically we haven't been following this rule, so we
were "leaking" a lot of symbols as `make leak-globals` would tell us.
This change unifies everything YJIT into a single compilation unit,
yjit.o, and makes everything unprefixed static to pass `make leak-globals`.
This manual "unified build" setup is similar to that of vm.o.
Having everything in one compilation unit allows static functions to
be visible across YJIT files and removes the need for declarations in
headers in some cases. Unnecessary declarations were removed.
Other changes of note:
- switched to MJIT_SYMBOL_EXPORT_BEGIN which indicates stuff as being
off limits for native extensions
- the first include of each YJIT file is change to be "internal.h"
- undefined MAP_STACK before explicitly redefining it since it
collide's with a definition in system headers. Consider renaming?
|
|
|
|
|
|
This allows a block version to have dependencies on multiple CMEs.
|
|
|
|
|
|
|
|
|
|
|
|
The JIT entry point passes the CFP as RSI and the EC as RDI. Lets match
that so we don't have to shuffle registers around.
|
|
|
|
|
|
This adds guards
|
|
* Add ETYPE_TRUE and ETYPE_FALSE
* Implement checktype
* Implement concatstrings
* Update deps
|
|
Detect types from putobject and getinlinecache
|
|
* Implement getblockparamproxy
* Parallel runner: wait for timeout thread to terminate after killing
Or else the leak cheaker could sees the thread as running and cause test
failures in test-tool.
* Add a comment, use jne
* Comment about where 0x3 comes from
|
|
* Malloc branch entries
* Add ASM comment for stack overflow check
* WIP
* Fix branch GC code. Add rb_darray_remove_unordered().
* Fix block end_pos after branch rewriting. Remove dst_patched bits.
|
|
|
|
|
|
|
|
Make sure `opt_getinlinecache` is in a block all on its own, and
invalidate it from the interpreter when `opt_setinlinecache`.
It will recompile with a filled cache the second time around.
This lets YJIT runs well when the IC for constant is cold.
|
|
Fixes bug involving ractors and branch stubs.
|
|
|
|
|
|
The old code decides branch->shape based on the write position of the
native code block, which is unsound in case the block already exists
and no new code is written to the write position.
Make this decision with the start address of the target block instead.
Also handle when the branch becomes smaller after patching.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Renaming uJIT to YJIT. AKA s/ujit/yjit/g.
|
