summaryrefslogtreecommitdiff
path: root/prism_compile.c
AgeCommit message (Collapse)Author
2024-11-13Mark strings returned by Symbol#to_s as chilled (#12065)Jean byroot Boussier
* Use FL_USER0 for ELTS_SHARED This makes space in RString for two bits for chilled strings. * Mark strings returned by `Symbol#to_s` as chilled [Feature #20350] `STR_CHILLED` now spans on two user flags. If one bit is set it marks a chilled string literal, if it's the other it marks a `Symbol#to_s` chilled string. Since it's not possible, and doesn't make much sense to include debug info when `--debug-frozen-string-literal` is set, we can't include allocation source, but we can safely include the symbol name in the warning message, making it much easier to find the source of the issue. Co-Authored-By: Étienne Barrié <[email protected]> --------- Co-authored-by: Étienne Barrié <[email protected]> Co-authored-by: Jean Boussier <[email protected]>
2024-11-08Fix memory leak in prism when syntax error in iseq compilationPeter Zhu
If there's a syntax error during iseq compilation then prism would leak memory because it would not free the pm_parse_result_t. This commit changes pm_iseq_new_with_opt to have a rb_protect to catch when an error is raised, and return NULL and set error_state to a value that can be raised by calling rb_jump_tag after memory has been freed. For example: 10.times do 10_000.times do eval("/[/=~s") rescue SyntaxError end puts `ps -o rss= -p #{$$}` end Before: 39280 68736 99232 128864 158896 188208 217344 246304 275376 304592 After: 12192 13200 14256 14848 16000 16000 16000 16064 17232 17952 Notes: Merged: https://github.com/ruby/ruby/pull/12036
2024-11-04YJIT: Replace Array#each only when YJIT is enabled (#11955)Takashi Kokubun
* YJIT: Replace Array#each only when YJIT is enabled * Add comments about BUILTIN_ATTR_C_TRACE * Make Ruby Array#each available with --yjit as well * Fix all paths that expect a C location * Use method_basic_definition_p to detect patches * Copy a comment about C_TRACE flag to compilers * Rephrase a comment about add_yjit_hook * Give METHOD_ENTRY_BASIC flag to Array#each * Add --yjit-c-builtin option * Allow inconsistent source_location in test-spec * Refactor a check of BUILTIN_ATTR_C_TRACE * Set METHOD_ENTRY_BASIC without touching vm->running Notes: Merged-By: maximecb <[email protected]>
2024-10-21Show where mutated chilled strings were allocatedÉtienne Barrié
[Feature #20205] The warning now suggests running with --debug-frozen-string-literal: ``` test.rb:3: warning: literal string will be frozen in the future (run with --debug-frozen-string-literal for more information) ``` When using --debug-frozen-string-literal, the location where the string was created is shown: ``` test.rb:3: warning: literal string will be frozen in the future test.rb:1: info: the string was created here ``` When resurrecting strings and debug mode is not enabled, the overhead is a simple FL_TEST_RAW. When mutating chilled strings and deprecation warnings are not enabled, the overhead is a simple warning category enabled check. Co-authored-by: Jean Boussier <[email protected]> Co-authored-by: Nobuyoshi Nakada <[email protected]> Co-authored-by: Jean Boussier <[email protected]> Notes: Merged: https://github.com/ruby/ruby/pull/11893
2024-10-18Point keyword->table into iseq local tableKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11912
2024-10-16Fix memory leak in syntax error in prismPeter Zhu
If there is a syntax error, there could be an ast_node in the result. This could get leaked if there is a syntax error so parsing could not complete (parsed is not set to true). For example, the following script leaks memory: 10.times do 10_000.times do eval("def foo(...) super(...) {}; end") rescue SyntaxError end puts `ps -o rss= -p #{$$}` end Before: 31328 42768 53856 65120 76208 86768 97856 109120 120208 131296 After: 20944 20944 20944 20944 20944 20944 20944 20944 20944 20944 Notes: Merged: https://github.com/ruby/ruby/pull/11901
2024-10-08Cast via `uintptr_t` function pointer between object pointerNobuyoshi Nakada
- ISO C forbids conversion of function pointer to object pointer type - ISO C forbids conversion of object pointer to function pointer type
2024-10-04Fix intermediate array off-by-one errorKevin Newton
Co-authored-by: Adam Hess <[email protected]> Notes: Merged: https://github.com/ruby/ruby/pull/11800
2024-10-01Fix compile issue with a short-circuited if/unless condition and `defined?`Luke Gruber
This caused an issue when `defined?` was in the `if` condition. Its instructions weren't appended to the instruction sequence even though it was compiled if a compile-time known logical short-circuit happened before the `defined?`. The catch table entry (`defined?` compilation produces a catch table entry) was still on the iseq even though the instructions weren't there. This caused faulty exception handling in the method. The solution is to no add the catch table entry for `defined?` after a compile-time known logical short circuit. This shouldn't touch much code, it's only for cases like the following, which can occur during debugging: if false && defined?(Some::CONSTANT) "more code..." end Fixes [Bug #20501] Notes: Merged: https://github.com/ruby/ruby/pull/11554
2024-09-30Don't create empty string for interpolationPeter Zhu
We don't need to create an empty string for interpolation unless it is the only element. For example: "#{hello} world" Before: 0000 putobject "" ( 1)[Li] 0002 putself 0003 opt_send_without_block <calldata!mid:hello, argc:0, FCALL|VCALL|ARGS_SIMPLE> 0005 dup 0006 objtostring <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE> 0008 anytostring 0009 putobject " world" 0011 concatstrings 3 0013 leave After: 0000 putself ( 1)[Li] 0001 opt_send_without_block <calldata!mid:hello, argc:0, FCALL|VCALL|ARGS_SIMPLE> 0003 dup 0004 objtostring <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE> 0006 anytostring 0007 putobject " world" 0009 concatstrings 2 0011 leave Notes: Merged: https://github.com/ruby/ruby/pull/11728
2024-09-30s/reproducable/reproducible/ydah
Notes: Merged: https://github.com/ruby/ruby/pull/11718
2024-09-27Fix up compiling popped ranges with non-optimizable boundsKevin Newton
Fixes [Bug #20763] Notes: Merged: https://github.com/ruby/ruby/pull/11714
2024-09-25Potentially fix ASAN checks for GC-ing operandKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11685
2024-09-25Fix up new types for block arguments and splats in prism compilerKevin Newton
2024-09-24Further split up pm_compile_node to work on -O0 buildsKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11679
2024-09-23Revert "GC guard current_string in the putobject instruction"Peter Zhu
This reverts commit 69f28ab715a02692fb2a9128bed46044963cbb50. This commit is being reverted because it does not fix the ASAN issue in the objtostring instruction.
2024-09-20GC guard current_string in the putobject instructionPeter Zhu
This is a band-aid solution for #11655 that only applies the fix for the putobject instruction before the objtostring instruction. This should help fix the use-after-poison in the ASAN CI. http://ci.rvm.jp/logfiles/brlog.trunk_asan.20240920-082802 Notes: Merged: https://github.com/ruby/ruby/pull/11656
2024-09-19Replace RB_OBJ_WRITTEN with RB_OBJ_WRITE in pm_compile_scope_nodePeter Zhu
Notes: Merged: https://github.com/ruby/ruby/pull/11647
2024-09-18Raise a compile error for break/next/redo inside eval in cases where it is ↵Jeremy Evans
optimized away In cases where break/next/redo are not valid syntax, they should raise a SyntaxError even if inside a conditional block that is optimized away. Fixes [Bug #20597] Co-authored-by: Kevin Newton <[email protected]> Notes: Merged: https://github.com/ruby/ruby/pull/11099 Merged-By: jeremyevans <[email protected]>
2024-09-18Fix evaluation order issue in f(**h, &h.delete(key))Jeremy Evans
Previously, this would delete the key in `h` before keyword splatting `h`. This goes against how ruby handles `f(*a, &a.pop)` and similar expressions. Fix this by having the compiler check whether the block pass expression is safe. If it is not safe, then dup the keyword splatted hash before evaluating the block pass expression. For expression: `h=nil; f(**h, &h.delete(:key))` VM instructions before: ``` 0000 putnil ( 1)[Li] 0001 setlocal_WC_0 h@0 0003 putself 0004 getlocal_WC_0 h@0 0006 getlocal_WC_0 h@0 0008 putobject :key 0010 opt_send_without_block <calldata!mid:delete, argc:1, ARGS_SIMPLE> 0012 splatkw 0013 send <calldata!mid:f, argc:1, ARGS_BLOCKARG|FCALL|KW_SPLAT>, nil 0016 leave ``` VM instructions after: ``` 0000 putnil ( 1)[Li] 0001 setlocal_WC_0 h@0 0003 putself 0004 putspecialobject 1 0006 newhash 0 0008 getlocal_WC_0 h@0 0010 opt_send_without_block <calldata!mid:core#hash_merge_kwd, argc:2, ARGS_SIMPLE> 0012 getlocal_WC_0 h@0 0014 putobject :key 0016 opt_send_without_block <calldata!mid:delete, argc:1, ARGS_SIMPLE> 0018 send <calldata!mid:f, argc:1, ARGS_BLOCKARG|FCALL|KW_SPLAT|KW_SPLAT_MUT>, nil 0021 leave ``` This is the same as 07d3bf4832532ae7446c9a6924d79aed60a7a9a5, except that it removes unnecessary hash allocations when using the prism compiler. Fixes [Bug #20640] Notes: Merged: https://github.com/ruby/ruby/pull/11645 Merged-By: jeremyevans <[email protected]>
2024-09-18Revert "Fix evaluation order issue in f(**h, &h.delete(key))"Jeremy Evans
This reverts commit 07d3bf4832532ae7446c9a6924d79aed60a7a9a5. No failures in the pull request CI, but there are now allocation test failures.
2024-09-18Fix evaluation order issue in f(**h, &h.delete(key))Jeremy Evans
Previously, this would delete the key in h before keyword splatting h. This goes against how ruby handles f(*a, &a.pop) and similar expressions. Fix this by having the compiler check whether the block pass expression is safe. If it is not safe, then dup the keyword splatted hash before evaluating the block pass expression. For expression: `h=nil; f(**h, &h.delete(:key))` VM instructions before: ``` 0000 putnil ( 1)[Li] 0001 setlocal_WC_0 h@0 0003 putself 0004 getlocal_WC_0 h@0 0006 getlocal_WC_0 h@0 0008 putobject :key 0010 opt_send_without_block <calldata!mid:delete, argc:1, ARGS_SIMPLE> 0012 splatkw 0013 send <calldata!mid:f, argc:1, ARGS_BLOCKARG|FCALL|KW_SPLAT>, nil 0016 leave ``` VM instructions after: ``` 0000 putnil ( 1)[Li] 0001 setlocal_WC_0 h@0 0003 putself 0004 putspecialobject 1 0006 newhash 0 0008 getlocal_WC_0 h@0 0010 opt_send_without_block <calldata!mid:core#hash_merge_kwd, argc:2, ARGS_SIMPLE> 0012 getlocal_WC_0 h@0 0014 putobject :key 0016 opt_send_without_block <calldata!mid:delete, argc:1, ARGS_SIMPLE> 0018 send <calldata!mid:f, argc:1, ARGS_BLOCKARG|FCALL|KW_SPLAT|KW_SPLAT_MUT>, nil 0021 leave ``` Fixes [Bug #20640] Notes: Merged: https://github.com/ruby/ruby/pull/11206 Merged-By: jeremyevans <[email protected]>
2024-09-18Fix coding issue in prism_compile.cLuke Gruber
Make sure to set back `ISEQ_COMPILE_DATA(iseq)->current_block` for forwarding super nodes with a block. Fixes [Bug #20740] Notes: Merged: https://github.com/ruby/ruby/pull/11621
2024-09-16[PRISM] Assume an eval context for RubyVM::ISEQ compileKevin Newton
Fixes [Bug #20741] Notes: Merged: https://github.com/ruby/ruby/pull/11632
2024-09-16[PRISM] Match defined behavior for explicit blockKevin Newton
Fixes [Bug #20748] Notes: Merged: https://github.com/ruby/ruby/pull/11633
2024-09-12[PRISM] Fix up pm_compile_branch_condition issue with single insn iseqsKevin Newton
2024-09-12[PRISM] Check error type for parsing directoryKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11497
2024-09-12Fix opening multibyte character filepath on WindowsKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11497
2024-09-12Fix FILE_SHARE_* permissions for Windows in read_entire_fileKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11497
2024-09-12[PRISM] Move compile scope node to its own functionKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11497
2024-09-12[PRISM] Move case node compilation into its own functionKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11497
2024-09-12Switch the default parser from parse.y to PrismKevin Newton
This commit switches the default parser to Prism. There are a couple of additional changes related to this that are a part of this as well to make this happen. * Switch the default parser in parse.h * Remove the Prism-specific workflow and add a parse.y-specific workflow to CI so that it continues to be tested * Update a few test exclusions since Prism has the correct behavior but parse.y doesn't per https://bugs.ruby-lang.org/issues/20504. * Skips a couple of tests on RBS which are failing because they are using RubyVM::AbstractSyntaxTree.of. Fixes [Feature #20564] Notes: Merged: https://github.com/ruby/ruby/pull/11497
2024-09-11Fix issue with super and forwarding arguments in prism_compile.cLuke Gruber
Fixes [Bug #20720] Notes: Merged: https://github.com/ruby/ruby/pull/11565
2024-09-03[PRISM] Fix up compile warning for sign comparisonKevin Newton
2024-08-29[PRISM] Handle RubyVM.keep_script_linesKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11501
2024-08-29prism_compile.c: Fix read_entire_file() for platforms without file mmapYuta Saito
Apply similar fix to https://github.com/ruby/prism/pull/2944 Notes: Merged: https://github.com/ruby/ruby/pull/11500
2024-08-29[PRISM] Respect PM_REGULAR_EXPRESSION_FLAGS_FORCED_BINARY_ENCODING flagKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11482
2024-08-29[PRISM] Copy the rest of the setup_args_dup_rest_p functionKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11496
2024-08-28[PRISM] Use node flags for dup_rest detection instead of loopingAlan Wu
Notes: Merged: https://github.com/ruby/ruby/pull/11489
2024-08-28[PRISM] Set use_block parameter flag for forwardable methodsAlan Wu
Match logic in compile.c:2133. Without this, the unused block warning code allocates an array, causing TestAllocation to fail. Notes: Merged: https://github.com/ruby/ruby/pull/11484
2024-08-28[PRISM] Field renamingKevin Newton
Rename some fields that do not quite make sense. * CaseMatchNode#consequent -> CaseMatchNode#else_clause * CaseNode#consequent -> CaseNode#else_clause * IfNode#consequent -> IfNode#subsequent * RescueNode#consequent -> RescueNode#subsequent * UnlessNode#consequent -> UnlessNode#else_clause Notes: Merged: https://github.com/ruby/ruby/pull/11480
2024-08-28[PRISM] Improve `dup_rest` optimization targetingAlan Wu
Part of implementing 3de20efc308cccc38bf9dacfffca6c626d039a06 in prism_compile.c. Down to 2 failures from 30 failures in test/ruby/test_allocation.rb. Notes: Merged: https://github.com/ruby/ruby/pull/11479
2024-08-27[PRISM] Wait for data before reading pipes and chardevsAlan Wu
With the parse.y parser, when a fifo (named pipe) is passed to Kernel#load and friends, we wait for data to be available first before reading. Note that with fifos, opening with `O_RDONLY|O_NONBLOCK` and then reading will look like EOF with read(2) returning 0, but data can become available later. The prism compiler needs to match this behavior to pass `test_loading_fifo_{fd_leak,threading_raise,threading_success}`. I chose to use IO#read to do this. An alternative way to match behavior would be to use open_load_file() from ruby.c like parse.y, but I opted to only allocate an IO to deal with threading when reading from pipes and character devices. The memory mapping code seems to work fine for regular files. Notes: Merged: https://github.com/ruby/ruby/pull/11472
2024-08-27[PRISM] Fix allocations for keyword splat paramseileencodes
Fixes the following allocations tests: * `test_keyword_and_keyword_splat_parameter` * `test_keyword_parameter` * `test_keyword_splat_parameter` * `test_no_array_allocation_with_splat_and_nonstatic_keywords` * `test_no_parameters` * `test_positional_splat_and_keyword_splat_parameter` * `test_ruby2_keywords` * Checks for `first_chunk` and if `stack_length == 0` to match the upstream parser. Otherwise, this optimization is skipped. * Subtracts the index, otherwise it will skip the hash allocation for the following: `keyword(*empty_array, a: 2, **empty_hash)`. * Sets `dup_rest` in order to determine when to set the correct flags * Doesn't set `VM_CALL_KW_SPLAT_MUT` flag unless `dup_rest` doesn't match `initial_dup_rest`. Given the following code: ```ruby keyword(*empty_array, a: 2) ``` Instructions before: ``` == disasm: #<ISeq:[email protected]:4 (4,0)-(8,3)> local table (size: 2, argc: 1 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 2] empty_hash@0<Arg>[ 1] empty_array@1 0000 newarray 0 ( 5)[LiCa] 0002 setlocal_WC_0 empty_array@1 0004 putself ( 7)[Li] 0005 getlocal_WC_0 empty_array@1 0007 splatarray true 0009 putobject :a 0011 putobject 2 0013 newhash 2 0015 opt_send_without_block <calldata!mid:keyword, argc:2, ARGS_SPLAT|ARGS_SPLAT_MUT|FCALL|KW_SPLAT> 0017 leave ( 8)[Re] ``` Instructions after: ``` == disasm: #<ISeq:[email protected]:4 (4,0)-(8,3)> local table (size: 2, argc: 1 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1]) [ 2] empty_hash@0<Arg>[ 1] empty_array@1 0000 newarray 0 ( 5)[LiCa] 0002 setlocal_WC_0 empty_array@1 0004 putself ( 7)[Li] 0005 getlocal_WC_0 empty_array@1 0007 splatarray false 0009 putobject {:a=>2} 0011 opt_send_without_block <calldata!mid:keyword, argc:2, ARGS_SPLAT|FCALL|KW_SPLAT> 0013 leave ( 8)[Re] ``` Differences: * `splatarray` is `false` not `true * `putobject`, `putobject`, `newhash` is simply `putobject` with optimizations on * Remove `ARGS_SPLAT_MUT` flag Related: ruby/prism#2994 Co-authored-by: Kevin Newton <[email protected]> Notes: Merged: https://github.com/ruby/ruby/pull/11438
2024-08-21[PRISM] Reset $. when done reading STDINKevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11425
2024-08-21[PRISM] Fix TestTRICK#test_ksk_1Kevin Newton
If an array element is a static literal that does not result in a intermediate array, it still needs to be compiled normally. Notes: Merged: https://github.com/ruby/ruby/pull/11426
2024-08-21[PRISM] Implement unused block warningeileencodes
Related: ruby/prism#2935 Notes: Merged: https://github.com/ruby/ruby/pull/11415
2024-08-11compile.c: don't allocate empty default values listJean Boussier
It just wastes memory. Notes: Merged: https://github.com/ruby/ruby/pull/11361
2024-07-24[PRISM] Add anon_* flags for iseqs with anonymous * and **Kevin Newton
Notes: Merged: https://github.com/ruby/ruby/pull/11236
2024-07-23[PRISM] Fix block param instructions when it has a receviereileencodes
In the following code: ```ruby def foo(&blk) blk.call end ``` Prism was using the `getblockparam` instruction but it should be `getblockparamproxy`. In this case we have a receiver, if it's a local variable read node, then it needs to go through the path to use `getblockparamproxy`. Before: ``` == disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(3,3)> 0000 definemethod :foo, foo ( 1)[Li] 0003 putobject :foo 0005 leave == disasm: #<ISeq:[email protected]:1 (1,0)-(3,3)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: 0, kw: -1@-1, kwrest: -1]) [ 1] blk@0<Block> 0000 getblockparam blk@0, 0 ( 2)[LiCa] 0003 opt_send_without_block <calldata!mid:call, argc:0, ARGS_SIMPLE> 0005 leave ( 3)[Re] ``` After: ``` == disasm: #<ISeq:<main>@test2.rb:1 (1,0)-(3,3)> 0000 definemethod :foo, foo ( 1)[Li] 0003 putobject :foo 0005 leave == disasm: #<ISeq:[email protected]:1 (1,0)-(3,3)> local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: 0, kw: -1@-1, kwrest: -1]) [ 1] blk@0<Block> 0000 getblockparamproxy blk@0, 0 ( 2)[LiCa] 0003 opt_send_without_block <calldata!mid:call, argc:0, ARGS_SIMPLE> 0005 leave ( 3)[Re] ``` Fixes `test_getblockparamproxy` and `test_ifunc_getblockparamproxy` in test_yjit.rb. Related to ruby/prism#2935. Notes: Merged: https://github.com/ruby/ruby/pull/11231
generated by cgit v1.2.3 (git 2.25.1) at 2025-05-25 06:08:33 +0000