summaryrefslogtreecommitdiff
path: root/gc.c
AgeCommit message (Collapse)Author
2022-03-03Dedup superclass array in leaf sibling classesJohn Hawthorn
Previously, we would build a new `superclasses` array for each class, even though for all immediate subclasses of a class, the array is identical. This avoids duplicating the arrays on leaf classes (those without subclasses) by calculating and storing a "superclasses including self" array on a class when it's first inherited and sharing that among all superclasses. An additional trick used is that the "superclass array including self" is valid as "self"'s superclass array. It just has it's own class at the end. We can use this to avoid an extra pointer of storage and can use one bit of a flag to track that we've "upgraded" the array. Notes: Merged: https://github.com/ruby/ruby/pull/5604
2022-02-23Constant time class to class ancestor lookupJohn Hawthorn
Previously when checking ancestors, we would walk all the way up the ancestry chain checking each parent for a matching class or module. I believe this was especially unfriendly to CPU cache since for each step we need to check two cache lines (the class and class ext). This check is used quite often in: * case statements * rescue statements * Calling protected methods * Class#is_a? * Module#=== * Module#<=> I believe it's most common to check a class against a parent class, to this commit aims to improve that (unfortunately does not help checking for an included Module). This is done by storing on each class the number and an array of all parent classes, in order (BasicObject is at index 0). Using this we can check whether a class is a subclass of another in constant time since we know the location to expect it in the hierarchy. Notes: Merged: https://github.com/ruby/ruby/pull/5568
2022-02-16Change darray size to size_t and add functions that use GC mallocPeter Zhu
Changes size and capacity of darray to size_t to support more elements. Adds functions to darray that use GC allocation functions. Notes: Merged: https://github.com/ruby/ruby/pull/5546
2022-02-16`wmap#each` should check liveness of keysKoichi Sasada
`ObjectSpace::WeakMap#each*` should check key's liveness. fix [Bug #18586] Notes: Merged: https://github.com/ruby/ruby/pull/5556
2022-02-14fix GC event synchronizationKoichi Sasada
(1) gc_verify_internal_consistency() use barrier locking for consistency while `during_gc == true` at the end of the sweep on `RGENGC_CHECK_MODE >= 2`. (2) `rb_objspace_reachable_objects_from()` is called without VM synchronization and it checks `during_gc != true`. So (1) and (2) causes BUG because of `during_gc == true`. To prevent this error, wait for VM barrier on `during_gc == false` and introduce VM locking on `rb_objspace_reachable_objects_from()`. http://ci.rvm.jp/results/trunk-asserts@phosphorus-docker/3830088 Notes: Merged: https://github.com/ruby/ruby/pull/5552
2022-02-10Free cached mark stack chunks when freeing objspacePeter Zhu
Cached mark stack chunks should also be freed when freeing objspace. Notes: Merged: https://github.com/ruby/ruby/pull/5536
2022-02-03Move total_freed_pages to size poolPeter Zhu
Notes: Merged: https://github.com/ruby/ruby/pull/5523
2022-02-03Move total_allocated_pages to size poolPeter Zhu
Notes: Merged: https://github.com/ruby/ruby/pull/5523
2022-02-03Fix case when gc_marks_continue does not yield slotsPeter Zhu
gc_marks_continue will start sweeping when it finishes marking. However, if the heap we are trying to allocate into is full, then the sweeping may not yield any free slots. If we don't call gc_sweep_continue immediate after this, then another GC will be started halfway during lazy sweeping. gc_sweep_continue will either grow the heap or finish sweeping. Notes: Merged: https://github.com/ruby/ruby/pull/5521
2022-02-02Decouple GC slot sizes from RVALUEPeter Zhu
Add a new macro BASE_SLOT_SIZE that determines the slot size. For Variable Width Allocation (compiled with USE_RVARGC=1), all slot sizes are powers-of-2 multiples of BASE_SLOT_SIZE. For USE_RVARGC=0, BASE_SLOT_SIZE is set to sizeof(RVALUE). Notes: Merged: https://github.com/ruby/ruby/pull/5517
2022-01-31Fix heap page iteration in gc_verify_heap_pagePeter Zhu
The for loops are not correctly iterating heap pages in gc_verify_heap_page. Notes: Merged: https://github.com/ruby/ruby/pull/5503
2022-01-29[Bug#18556] Fallback `MAP_ ANONYMOUS`Nobuyoshi Nakada
Define `MAP_ANONYMOUS` to `MAP_ANON` if undefined on old systems. Notes: Merged: https://github.com/ruby/ruby/pull/5506 Merged-By: nobu <[email protected]>
2022-01-26Fix typo in assertion in gc.cPeter Zhu
2022-01-26Unpoison the cached object in the exact sizeNobuyoshi Nakada
2022-01-25Call rb_id_table_foreach_values insteadPeter Zhu
These places never replace the value, so call rb_id_table_foreach_values instead of rb_id_table_foreach_values_with_replace. Notes: Merged: https://github.com/ruby/ruby/pull/5486
2022-01-25Rename rb_id_table_foreach_with_replacePeter Zhu
Renames rb_id_table_foreach_with_replace to rb_id_table_foreach_values_with_replace and passes only the value to the callback. We can use this in GC compaction when we cannot access the global symbol array. Notes: Merged: https://github.com/ruby/ruby/pull/5486
2022-01-25Remove redundant if statement in try_movePeter Zhu
The if statement is redundant since if `index == 0` then `BITS_BITLENGTH * index == 0`. Notes: Merged: https://github.com/ruby/ruby/pull/5479
2022-01-24Keep right operand within width when right shiftingPeter Zhu
NUM_IN_PAGE could return a value much larger than 64. According to the C11 spec 6.5.7 paragraph 3 this is undefined behavior: > If the value of the right operand is negative or is greater than or > equal to the width of the promoted left operand, the behavior is > undefined. On most platforms, this is usually not a problem as the architecture will mask off all out-of-range bits. Notes: Merged: https://github.com/ruby/ruby/pull/5478
2022-01-24[wasm] Disallow compactionPeter Zhu
WebAssembly doesn't support signals so we can't use read barriers so we can't use compaction. Notes: Merged: https://github.com/ruby/ruby/pull/5475
2022-01-19Fix format size qualifier on IL32P64Nobuyoshi Nakada
2022-01-19[wasm] gc.c: scan wasm locals and c stack to mark living objectsYuta Saito
WebAssembly has function local infinite registers and stack values, but there is no way to scan the values in a call stack for now. This implementation uses Asyncify to spilling out wasm locals into linear memory. Notes: Merged: https://github.com/ruby/ruby/pull/5407
2022-01-19[wasm] gc.c: disable read signal barrier for wasiYuta Saito
WASI currently does not yet support signal Notes: Merged: https://github.com/ruby/ruby/pull/5407
2022-01-19[wasm] eval_inter.h gc.c vm_core.h: include wasm/setjmp.h instead of sysroot ↵Yuta Saito
header Notes: Merged: https://github.com/ruby/ruby/pull/5407
2022-01-14Separately allocate class_serial on 32-bit systemsPeter Zhu
On 32-bit systems, VWA causes class_serial to not be aligned (it only guarantees 4 byte alignment but class_serial is 8 bytes and requires 8 byte alignment). This commit uses a hack to allocate class_serial through malloc. Once VWA allocates with 8 byte alignment in the future, we will revert this commit. Notes: Merged: https://github.com/ruby/ruby/pull/5442
2022-01-07Improve string info in rb_raw_obj_infoPeter Zhu
Improve rb_raw_obj_info to output additional into about strings including the length, capacity, and whether or not it is embedded. Notes: Merged: https://github.com/ruby/ruby/pull/5414
2022-01-05Remove assertion causing read barrier to triggerPeter Zhu
GET_HEAP_PAGE reads the page. If during compaction there is a read barrier on the page, it causes the read barrier to trigger. Notes: Merged: https://github.com/ruby/ruby/pull/5394
2022-01-04Switch `is_pointer_to_heap` to use library bsearchMatt Valentine-House
This commit switches from a custom implemented bsearch algorithm to use the one provided by the C standard library. Because `is_pointer_to_heap` will only return true if the pointer being searched for is a valid slot starting address within the heap page body, we've extracted the bsearch call site into a more general function so we can use it elsewhere. The new function `heap_page_for_ptr` returns the heap page for any heap page pointer, regardless of whether that is at the start of a slot or in the middle of one. We then use this function as the basis of `is_pointer_to_heap`. Notes: Merged: https://github.com/ruby/ruby/pull/5187
2022-01-04[Feature #18364] Add GC.stat_heap to get stats for memory heapsPeter Zhu
GC.stat_heap will return stats for memory heaps. This is used for the Variable Width Allocation feature. Notes: Merged: https://github.com/ruby/ruby/pull/5177
2022-01-01Negative RBOOL usageNobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/5385
2021-12-29On 64bit macOS, enlarge heap pages to reduce mmap calls [Bug #18447]Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/5371
2021-12-26Remove deprecate rb_cData [Bug #18433]Nobuyoshi Nakada
Also enable the warning for T_DATA allocator. Notes: Merged: https://github.com/ruby/ruby/pull/5348
2021-12-23`finalize_deferred` doesn't need VM lockKoichi Sasada
`finalize_list()` acquires VM lock to manipulate objspace state. Notes: Merged: https://github.com/ruby/ruby/pull/5326
2021-12-23undef `rb_vm_lookup_overloaded_cme()`Koichi Sasada
Some callable method entries (cme) can be a key of `overloaded_cme_table` and the keys should be pinned because the table is numtable (VALUE is a key). Before the patch GC checks the cme is in `overloaded_cme_table` by looking up the table, but it needs VM locking. It works well in normal GC marking because it is protected by the VM lock, but it doesn't work on `rb_objspace_reachable_objects_from` because it doesn't use VM lock. Now, the number of target cmes are small enough, I decide to pin down all possible cmes instead of using looking up the table. Notes: Merged: https://github.com/ruby/ruby/pull/5327
2021-12-21make `overloaded_cme_table` truly weak key mapKoichi Sasada
`overloaded_cme_table` keeps cme -> monly_cme pairs to manage corresponding `monly_cme` for `cme`. The lifetime of the `monly_cme` should be longer than `monly_cme`, but the previous patch losts the reference to the living `monly_cme`. Now `overloaded_cme_table` values are always root (keys are only weak reference), it means `monly_cme` does not freed until corresponding `cme` is invalidated. To make managing easy, move `overloaded_cme_table` to `rb_vm_t`. Notes: Merged: https://github.com/ruby/ruby/pull/5316
2021-12-21`mandatory_only_cme` should not be in `def`Koichi Sasada
`def` (`rb_method_definition_t`) is shared by multiple callable method entries (cme, `rb_callable_method_entry_t`). There are two issues: * old -> young reference: `cme1->def->mandatory_only_cme = monly_cme` if `cme1` is young and `monly_cme` is young, there is no problem. Howevr, another old `cme2` can refer `def`, in this case, old `cme2` points young `monly_cme` and it violates gengc assumption. * cme can have different `defined_class` but `monly_cme` only has one `defined_class`. It does not make sense and `monly_cme` should be created for a cme (not `def`). To solve these issues, this patch allocates `monly_cme` per `cme`. `cme` does not have another room to store a pointer to the `monly_cme`, so this patch introduces `overloaded_cme_table`, which is weak key map `[cme] -> [monly_cme]`. `def::body::iseqptr::monly_cme` is deleted. The first issue is reported by Alan Wu. Notes: Merged: https://github.com/ruby/ruby/pull/5311
2021-12-20Show whether object is garbage in rb_raw_obj_info()Alan Wu
When using `rp(obj)` for debugging during development, it may be useful to know that an object is soon to be swept. Add a new letter to the object dump for whether the object is garbage. It's easy to forget about lazy sweep. Notes: Merged: https://github.com/ruby/ruby/pull/5310 Merged-By: XrXr
2021-12-14Remove compaction support detection using sysconfPeter Zhu
Except on Windows and MinGW, we can only use compaction on systems that use mmap (only systems that use mmap can use the read barrier that compaction requires). We don't need to separately detect whether we can support compaction or not. Notes: Merged: https://github.com/ruby/ruby/pull/5260
2021-12-07Fixed the check order in wmap_live_p [Bug #18392]Nobuyoshi Nakada
Check if the object is a pointer to heap before check the flag in that object. Notes: Merged: https://github.com/ruby/ruby/pull/5224
2021-12-07ObjectSpace::WeakMap#inspect: check if living object [Bug #18392]Nobuyoshi Nakada
Notes: Merged: https://github.com/ruby/ruby/pull/5224
2021-12-03Refactor GC functions to have consistent namingPeter Zhu
Refactor function names for consistency. Function with name xyz_page should have a corresponding function named xyz_plane. Notes: Merged: https://github.com/ruby/ruby/pull/5204
2021-12-02Lazily create singletons on instance_{exec,eval} (#5146)John Hawthorn
* Lazily create singletons on instance_{exec,eval} Previously when instance_exec or instance_eval was called on an object, that object would be given a singleton class so that method definitions inside the block would be added to the object rather than its class. This commit aims to improve performance by delaying the creation of the singleton class unless/until one is needed for method definition. Most of the time instance_eval is used without any method definition. This was implemented by adding a flag to the cref indicating that it represents a singleton of the object rather than a class itself. In this case CREF_CLASS returns the object's existing class, but in cases that we are defining a method (either via definemethod or VM_SPECIAL_OBJECT_CBASE which is used for undef and alias). This also happens to fix what I believe is a bug. Previously instance_eval behaved differently with regards to constant access for true/false/nil than for all other objects. I don't think this was intentional. String::Foo = "foo" "".instance_eval("Foo") # => "foo" Integer::Foo = "foo" 123.instance_eval("Foo") # => "foo" TrueClass::Foo = "foo" true.instance_eval("Foo") # NameError: uninitialized constant Foo This also slightly changes the error message when trying to define a method through instance_eval on an object which can't have a singleton class. Before: $ ruby -e '123.instance_eval { def foo; end }' -e:1:in `block in <main>': no class/module to add method (TypeError) After: $ ./ruby -e '123.instance_eval { def foo; end }' -e:1:in `block in <main>': can't define singleton (TypeError) IMO this error is a small improvement on the original and better matches the (both old and new) message when definging a method using `def self.` $ ruby -e '123.instance_eval{ def self.foo; end }' -e:1:in `block in <main>': can't define singleton (TypeError) Co-authored-by: Matthew Draper <[email protected]> * Remove "under" argument from yield_under * Move CREF_SINGLETON_SET into vm_cref_new * Simplify vm_get_const_base * Fix leaf VM_SPECIAL_OBJECT_CONST_BASE Co-authored-by: Matthew Draper <[email protected]> Notes: Merged-By: jhawthorn <[email protected]>
2021-12-02Don't clear the constant cache when finishing compactionMatt Valentine-House
References are being updated correctly, so this is no longer necessary Notes: Merged: https://github.com/ruby/ruby/pull/5198
2021-12-02Cast tv_usec to int32_t to fit in tv_nsecYuta Saito
suseconds_t, which is the type of tv_usec, may be defined with a longer size type than tv_nsec's type (long). So usec to nsec conversion needs an explicit casting. Notes: Merged: https://github.com/ruby/ruby/pull/5200
2021-11-26Fix a function name in an error messageKazuhiro NISHIYAMA
Notes: Merged: https://github.com/ruby/ruby/pull/5181
2021-11-26Remove unused function `size_pool_for_size`Kazuhiro NISHIYAMA
``` compiling ../gc.c ../gc.c:2444:1: warning: unused function 'size_pool_for_size' [-Wunused-function] size_pool_for_size(rb_objspace_t *objspace, size_t size) ^ 1 warning generated. ``` Notes: Merged: https://github.com/ruby/ruby/pull/5181
2021-11-26initialize allocated memory by VWA for assertionsKoichi Sasada
When `RGENGC_CHECK_MODE` is enable, `obj_memsize_of` is called in `newobj_init` and it expect the memory is zero-cleared.
2021-11-25Revert "Add GC.stat_size_pool to get stats for a size pool"Peter Zhu
This reverts commit 6157619bb68e4307cdf065cb73d5bfcec30d042d. We'll wait for comments in the open ticket: https://bugs.ruby-lang.org/issues/18364 Notes: Merged: https://github.com/ruby/ruby/pull/5176
2021-11-25Add GC.stat_size_pool to get stats for a size poolPeter Zhu
GC.stat_size_pool will return stats for a particular size pool. This is used for the Variable Width Allocation feature. Notes: Merged: https://github.com/ruby/ruby/pull/5169
2021-11-23Speed up Ractors for Variable Width AllocationPeter Zhu
This commit adds a Ractor cache for every size pool. Previously, all VWA allocated objects used the slowpath and locked the VM. On a micro-benchmark that benchmarks String allocation: VWA turned off: 29.196591 0.889709 30.086300 ( 9.434059) VWA before this commit: 29.279486 41.477869 70.757355 ( 12.527379) VWA after this commit: 16.782903 0.557117 17.340020 ( 4.255603) Notes: Merged: https://github.com/ruby/ruby/pull/5151
2021-11-22Removes unused HEAP_PAGE_BITMAP_PLANES constant from gc.cJemma Issroff
Notes: Merged: https://github.com/ruby/ruby/pull/4154