Crashes with complex autoindent #55

aycabta · 2019-12-26T08:01:58Z

Reported by @mame:

(((
)
)

Reported at #16453 by @JunichiIto

[<<FOO]
hello
FOO

ruby#55 We were not taking into account what happens if multiple opening braces on one line. If a user did type in multiple opeing braces the first newline would reset the depth_difference counter and the next type the user typed a closing brace depth_difference would be lower than 0. Any time we tell the terminal to start at a negative position it crashes. It seemed to me that the easiest fix was to not rely on the spaces from the previous line, but just count all of the braces to find our total position. I am a bit consurned about my solution since there didn't seem to be tests for existing behavior, so I'm not sure if I'm removing needed functionality. There is also a different method that calculates the position without doing a new line. This method was also calculating the wrong number and needed to be updated in the same way. It seems to me that check_corresponding_token_depth method and the check_newline_depth_difference method share similar logic. It might be nice to merge those together if possible (and maybe even make a small class for that.) Example of code that would make it crash: ``` zerg:irb Ben$ irb 2.7.0 :001 > [[[ 2.7.0 :002 > ] 2.7.0 :003 > ] 2.7.0 :004 > ] ```

ruby#55 We would crash if we used the on_heredoc_beg tag inside of braces. Example of code that used to cause a crash: ``` zerg:irb Ben$ irb 2.7.0 :001"> [<<TEST] 2.7.0 :002"> foo 2.7.0 :003 > TEST => ["foo\n"] ``` I change the code to assume that on_heredoc_beg should not be indented. I think this is a safe assumption since white space inside of those blocks actually changes the contents of the string.

ruby#55 We were not taking into account what happens if multiple opening braces on one line. If a user did type in multiple opeing braces the first newline would reset the depth_difference counter and the next type the user typed a closing brace depth_difference would be lower than 0. Any time we tell the terminal to start at a negative position it crashes. It seemed to me that the easiest fix was to not rely on the spaces from the previous line, but just count all of the braces to find our total position. I am a bit consurned about my solution since there didn't seem to be tests for existing behavior, so I'm not sure if I'm removing needed functionality. There is also a different method that calculates the position without doing a new line. This method was also calculating the wrong number and needed to be updated in the same way. It seems to me that check_corresponding_token_depth method and the check_newline_depth_difference method share similar logic. It might be nice to merge those together if possible (and maybe even make a small class for that.) Example of code that would make it crash: ``` zerg:irb Ben$ irb 2.7.0 :001 > [[[ 2.7.0 :002 > ] 2.7.0 :003 > ] 2.7.0 :004 > ] ```

ruby#55 We would crash if we used the on_heredoc_beg tag inside of braces. Example of code that used to cause a crash: ``` zerg:irb Ben$ irb 2.7.0 :001"> [<<TEST] 2.7.0 :002"> foo 2.7.0 :003 > TEST => ["foo\n"] ``` I change the code to assume that on_heredoc_beg should not be indented. I think this is a safe assumption since white space inside of those blocks actually changes the contents of the string.

ruby#55 We were not taking into account what happens if multiple opening braces on one line. If a user did type in multiple opening braces the first newline would reset the depth_difference counter and the next type the user typed a closing brace depth_difference would be lower than 0. Any time we tell the terminal to start at a negative position it crashes. It seemed to me that the easiest fix was to not rely on the spaces from the previous line, but just count all of the braces to find our total position. I am a bit concerned about my solution since there didn't seem to be tests for existing behavior, so I'm not sure if I'm removing needed functionality. There is also a different method that calculates the position without doing a new line. This method was also calculating the wrong number and needed to be updated in the same way. It seems to me that check_corresponding_token_depth method and the check_newline_depth_difference method share similar logic. It might be nice to merge those together if possible (and maybe even make a small class for that.) Example of code that would make it crash: ``` zerg:irb Ben$ irb 2.7.0 :001 > [[[ 2.7.0 :002 > ] 2.7.0 :003 > ] 2.7.0 :004 > ] ```

ruby#55 We would crash if we used the on_heredoc_beg tag inside of braces. Example of code that used to cause a crash: ``` zerg:irb Ben$ irb 2.7.0 :001"> [<<TEST] 2.7.0 :002"> foo 2.7.0 :003 > TEST => ["foo\n"] ``` I change the code to assume that on_heredoc_beg should not be indented. I think this is a safe assumption since white space inside of those blocks actually changes the contents of the string.

BenKanouse · 2019-12-27T02:42:30Z

I attempted to fix this issue here: #56

Please let me know if you have any feedback.

Thanks!

ruby#55 There are two issues in this commit. Issue 1 multiple open braces on a line: The first issue is that when we put multiple opening braces on a line the spaces_of_nest array keeps getting '0' added to it. This means that when we pop off of this array we are saying that we should be in position zero for the next line. This is an issue because we don't always want to be in position 0 after a closing brace. Example: ``` [[[ ] ] ] ``` In the above example the 'spaces_of_nest' array looks like this after the first line is entered: [0,0,0]. We really want to be indented 4 spaces for the 1st closing brace 2 for the 2nd and 0 for the 3rd. i.e. we want it to be: [0,2,4]. Issue 2: We were subtracting from the depth in check_newline_depth_difference on every open brace. This is the right thing to do if the opening and closing brace are on the same line. For example in a method definition we have an opening and closing parentheses we want to add 1 to our depth, and then remove it. ``` def foo() end ``` However this isn't the correct behavior when the brace spans multiple lines. If a brace spans multiple lines we don't want to subtract from check_newline_depth_difference and we want to treat the braces the same way as we do `end` and allow check_corresponding_token_depth to pop the correct depth. Example of bad behavior: ``` def foo() [ ] puts 'bar' end ``` Example of desired behavior: ``` def foo() [ ] puts 'bar' end ```

BenKanouse · 2020-01-02T17:24:50Z

I believe that this PR fixes both issues: #58

It also adds test for both issues. I'm looking forward to any feedback. Thanks!

ruby#55 If we had put multiple open braces on a line the with no closing brace spaces_of_nest array keeps getting '0' added to it. This means that when we pop off of this array we are saying that we should be in position zero for the next line. This is an issue because we don't always want to be in position 0 after a closing brace. Example: ``` [[[ ] ] ] ``` In the above example the 'spaces_of_nest' array looks like this after the first line is entered: [0,0,0]. We really want to be indented 4 spaces for the 1st closing brace 2 for the 2nd and 0 for the 3rd. i.e. we want it to be: [0,2,4]. We also saw this issue with a heredoc inside of an array. ``` [<<FOO] hello FOO ```

aycabta · 2020-01-07T07:22:58Z

This is fixed by #58.

Thank you, @BenKanouse!

ruby/irb#55 If we had put multiple open braces on a line the with no closing brace spaces_of_nest array keeps getting '0' added to it. This means that when we pop off of this array we are saying that we should be in position zero for the next line. This is an issue because we don't always want to be in position 0 after a closing brace. Example: ``` [[[ ] ] ] ``` In the above example the 'spaces_of_nest' array looks like this after the first line is entered: [0,0,0]. We really want to be indented 4 spaces for the 1st closing brace 2 for the 2nd and 0 for the 3rd. i.e. we want it to be: [0,2,4]. We also saw this issue with a heredoc inside of an array. ``` [<<FOO] hello FOO ``` ruby/irb@80c69c8272

* [ruby/reline] Sort completion list #### Legacy mode: ```console $ irb --legacy irb(main):001:0> l[TAB][TAB] lambda load local_variables loop ``` #### Before this patch: ```console $ irb irb(main):001:0> l[TAB][TAB] local_variables loop lambda load ``` #### After this patch: ```console $ irb irb(main):001:0> l[TAB][TAB] lambda load local_variables loop ``` ruby/reline@6074069c7d * Drop an invalid char as UTF-8 * Add test_completion_with_indent_and_completer_quote_characters This is for 8a70524. * [ruby/irb] Add tests for RubyLex The set_auto_indent method calculates the correct number of spaces for indenting a line. We think there might be a few bugs in this method so we are testing the current functionality to make sure nothing breaks when we address those bugs. Example test failure: ``` 1) Failure: TestIRB::TestRubyLex#test_auto_indent [/Users/Ben/Projects/irb/test/irb/test_ruby_lex.rb:75]: Calculated the wrong number of spaces for: def each_top_level_statement initialize_input catch(:TERM_INPUT) do loop do begin prompt unless l = lex throw :TERM_INPUT if @line == '' else . <10> expected but was <12>. ``` ruby/irb@752d5597ab * [ruby/reline] Degenerate the terminal size to [$LINES, $COLUMNS] if it is unknown This is a workaround for ruby/irb#50 ruby/reline@5725677d1a * [ruby/irb] Fix newline depth with multiple braces This commit fixes the check_newline_depth_difference method to multiple open braces on one line into account. Before this change we were subtracting from the depth in check_newline_depth_difference on every open brace. This is the right thing to do if the opening and closing brace are on the same line. For example in a method definition we have an opening and closing parentheses we want to add 1 to our depth, and then remove it. ``` def foo() end ``` However this isn't the correct behavior when the brace spans multiple lines. If a brace spans multiple lines we don't want to subtract from check_newline_depth_difference and we want to treat the braces the same way as we do `end` and allow check_corresponding_token_depth to pop the correct depth. Example of bad behavior: ``` def foo() [ ] puts 'bar' end ``` Example of desired behavior: ``` def foo() [ ] puts 'bar' end ``` ruby/irb@7dc8af01e0 * text/readline/test_readline.rb - fix skip on Reline (#2743) TestRelineAsReadline#test_input_metachar passes on MinGW * Add "require 'openstruct'" what is forgotten * [ruby/irb] Fix lib name of OpenStruct ruby/irb@1f3a84ab6b * Add load path and require for ruby/ruby * Rescue EOFError If C-d is pressed before IRB is ready, IRB crashes because EOFError occurs. * Complete indented and quoted string correctly def foo ''.upca[TAB] This will be completed to be: def foo ''.upcase The indent was gone. This commit fixes the bug. * [ruby/irb] Fix crashing when multiple open braces per line ruby/irb#55 If we had put multiple open braces on a line the with no closing brace spaces_of_nest array keeps getting '0' added to it. This means that when we pop off of this array we are saying that we should be in position zero for the next line. This is an issue because we don't always want to be in position 0 after a closing brace. Example: ``` [[[ ] ] ] ``` In the above example the 'spaces_of_nest' array looks like this after the first line is entered: [0,0,0]. We really want to be indented 4 spaces for the 1st closing brace 2 for the 2nd and 0 for the 3rd. i.e. we want it to be: [0,2,4]. We also saw this issue with a heredoc inside of an array. ``` [<<FOO] hello FOO ``` ruby/irb@80c69c8272 * Support history-size in .inputrc correctly * Introduce an abstracted structure about the encoding of Reline The command prompt on Windows always uses Unicode to take input and print output but most Reline implementation depends on Encoding.default_external. This commit introduces an abstracted structure about the encoding of Reline. * Remove an unused setting variable * Use Reline.encoding_system_needs if exists * Add tests for vi_insert and vi_add * Implement vi_insert_at_bol and vi_add_at_eol * [ruby/reline] Implement vi_to_next_char ruby/reline@066ecb0a21 * [ruby/reline] Implement vi_prev_char and vi_to_prev_char ruby/reline@0ad3ee63fa * [ruby/readline-ext] Include ruby/assert.h in ruby/ruby.h so that assertions can be there ruby/readline-ext@4d44c12832 * Stop using minitest dependent methods * Skip a test that uses assert_ruby_status if it doesn't exist * Use omit instead of skip * Check DONT_RUN_RELINE_TEST envvar * [ruby/irb] Add newline_before_multiline_output ruby/irb@9eb1801a66 * [ruby/irb] Fix compatibility with rails before 5.2 Rails before 5.2 added Array#append as an alias to Array#<< , so that it expects only one argument. However ruby-2.5 added Array#append as an alias to Array#push which takes any number of arguments. If irb completion is used in `rails c` (for example "IO.<tab>") it fails with: irb/completion.rb:206:in `<<': wrong number of arguments (given 3, expected 1) (ArgumentError) Using Array#push instead of Array#append fixes compatibility. ruby/irb@5b7bbf9c34 * Reline: Use a more robust detection of MinTTY The previous detection per get_screen_size fails when stdout is passed to a pipe. That is the case when running ruby tests in parallel ("-j" switch). In this case Reline believes that it's running on MinTTY and the tests are running with ANSI IOGate instead of the Windows adapter on MINGW. So parallel test results were different to that of a single process. This commit fixes these differencies. The code is taken from git sources and translated to ruby. NtQueryObject() is replaced by GetFileInformationByHandleEx(), because NtQueryObject() is undocumented and is more difficult to use: https://github.com/git-for-windows/git/blob/c5a03b1e29c69f3f06c8fabd92493edb73469176/compat/winansi.c#L558 * Reline: Fix changed test results due to change to UTF-8 on Windows In commit f8ea286 the Reline encoding for native windows console was changed to hardcoded UTF-8. This caused failures in reline and readline tests, but they were hidden, because parallel ruby tests incorrectly used Reline::ANSI as IOGate. Tests failures were raised in single process mode, but not with -j switch. This patch corrects encodings on native Windows console. * [ruby/irb] [ruby/irb] Rewrite an expression to detect multiline ruby/irb@ed5cf375a6 ruby/irb@5b7bbf9c34 * [ruby/reline] Implement vi_change_meta ruby/reline@8538e0e10f * Always refer to Reline::IOGate.encoding * Always use UTF-8 for Reline::GeneralIO on Windows * Use test_mode on Reline::History::Test for encoding * [ruby/reline] Support GNOME style Home/End key sequences [Bug #16510] ruby/reline@788f0df845 * [ruby/irb] Add a new easter egg: dancing ruby ruby/irb@e37dc7e58e * [ruby/irb] Exclude useless files from RDoc ruby/irb@8f1ab2400c * [ruby/irb] Exclude useless files from RDoc * Fix inaccuracy in encoding tests These tests assume Encoding.find('locale') == Encoding.find('external') and fail if they are distinct. * [ruby/reline] Fix Reline::Windows#scroll_down I mistook Right and Bottom. ruby/reline@8be401c5f5 * [ruby/reline] Bypass cursor down when a char is rendered at eol on Windows A newline is automatically inserted if a character is rendered at eol on Windows command prompt. ruby/reline@4bfea07e4a * [ruby/reline] Organize special keys escape sequences ruby/reline@41deb1a3d9 * [ruby/readline-ext] Remove unnecessary -I$(top_srcdir) when it's an individual gem ruby/readline-ext@efaca4a5f4 * [ruby/readline-ext] Check TestRelineAsReadline existance ruby/readline-ext@c0a6303168 * [ruby/readline-ext] The ruby/assert.h is adopted by Ruby 2.7 or later ruby/readline-ext@106c31fc1b * Revert "[ruby/readline-ext] Include ruby/assert.h in ruby/ruby.h so that assertions can be there" This reverts commit 425b206. This cherry-pick was a mistake. * [ruby/readline-ext] Use require check instead of DONT_RUN_RELINE_TEST env ruby/readline-ext@1df99d1481 * [ruby/readline-ext] Add spec.extensions ruby/readline-ext@8c33abb13c * [ruby/readline-ext] Use rake/extensiokntask to build ruby/readline-ext@b0b5f709bd * Fix readline build dependency * [ruby/irb] Add test_complete_symbol ruby/irb@dbbf086c1f * [ruby/irb] Check doc namespace correctly IRB::InputCompletor::PerfectMatchedProc crashes when doc not found because a variable name was incorrect. ruby/irb@889fd4928f * [ruby/irb] Fix auto indent with closed brace A closed brace in auto-indent shouldn't affect the next brace in the same line, but it behaves like below: p() { } It's a bug. ruby/irb@fbe59e344f * [ruby/irb] Use 0.step instead of (..0).each for Ruby 2.5 ruby/irb@5d628ca40e * Revert "[ruby/irb] Add test_complete_symbol" This reverts commit 3af3431. * [ruby/irb] fix reserved words and completion for them ruby/irb@6184b227ad * Add test_complete_symbol The previous version of the test method used a symbol, ":abcdefg" to complete but longer symbols that can be completed are defined by other test methods of other libs. * test/irb/test_completion.rb: suppress a warning: unused literal ignored * [ruby/reline] Use IO#write instead of IO#print IO#print always adds a string of $\ automatically. ruby/reline@a93119c847 * [ruby/irb] Version 1.2.2 ruby/irb@a71753f15a * [ruby/reline] Version 0.1.3 ruby/reline@ea2b182466 * [ruby/irb] Include easter-egg.rb in gemspec `irb` doesn't run because this file isn't included in the gem. ruby/irb@73cda56d25 * [ruby/irb] Version 1.2.3 ruby/irb@dd56e06df5 * support multi-run test for test_readline.rb * [ruby/irb] `yield` outside method definition is a syntax error ruby/irb@dbc7b059c7 * test/readline - allow ENV control of test class creation In ruby/ruby, the tests run on both readline & reline by creating four test classes: ``` TestReadline TestReadlineHistory TestRelineAsReadline TestRelineAsReadlineHistory ``` Reline inports the test files and uses them in its CI. Adding the ENV control allows it to only run the `TestRelineAsReadline` classes. * Omit test_using_quoting_detection_proc_with_multibyte_input temporarily for random order test * support random order test. test_readline: HISTORY should be empty. test_using_quoting_detection_proc: test_using_quoting_detection_proc_with_multibyte_input: Readline.completer_quote_characters= and Readline.completer_word_break_characters= doesn't accept nil, so skip if previous values are nil. * Set Readline.completion_append_character = nil always GNU Readline add a white space when Readline.completion_append_character is not initialized. * Fix a typo [ci skip] * skip test if Reline.completion_proc is nil. Some other tests can set Reline.completion_proc, so if it is nil, simply skip this test. * Reset Reline.point TestRelineAsReadline#test_insert_text expects Readline.point == 0 at the beginning of the test, but a test violate this assumption. * Convert incompatible encoding symbol names * Ignore incompatible convert of symbols * Add workaround for test-bundler failure https://github.com/ruby/actions/runs/500526558?check_suite_focus=true#step:16:127 ``` Failures: 1) Bundler.setup when Bundler is bundled doesn't blow up Failure/Error: expect(err).to be_empty expected `"fatal: not a git repository (or any of the parent directories): .git\nfatal: not a git repository (o...the parent directories): .git\nfatal: not a git repository (or any of the parent directories): .git".empty?` to return true, got false Commands: $ /home/runner/work/actions/actions/snapshot-master/ruby \ -I/home/runner/work/actions/actions/snapshot-master/lib:/home/runner/work/actions/actions/snapshot-master/spec/bundler \ -rsupport/hax -rsupport/artifice/fail \ /home/runner/work/actions/actions/snapshot-master/libexec/bundle install --retry 0 Resolving dependencies... Using bundler 2.1.4 Bundle complete! 1 Gemfile dependency, 1 gem now installed. Use `bundle info [gemname]` to see where a bundled gem is installed. fatal: not a git repository (or any of the parent directories): .git fatal: not a git repository (or any of the parent directories): .git fatal: not a git repository (or any of the parent directories): .git # $? => 0 $ /home/runner/work/actions/actions/snapshot-master/ruby \ -I/home/runner/work/actions/actions/snapshot-master/lib:/home/runner/work/actions/actions/snapshot-master/spec/bundler \ -rsupport/hax -rsupport/artifice/fail \ /home/runner/work/actions/actions/snapshot-master/libexec/bundle exec ruby -e \ require\ \'bundler\'\;\ Bundler.setup fatal: not a git repository (or any of the parent directories): .git fatal: not a git repository (or any of the parent directories): .git fatal: not a git repository (or any of the parent directories): .git # $? => 0 # ./spec/bundler/runtime/setup_spec.rb:1056:in `block (3 levels) in <top (required)>' # ./spec/bundler/spec_helper.rb:111:in `block (3 levels) in <top (required)>' # ./spec/bundler/spec_helper.rb:111:in `block (2 levels) in <top (required)>' # ./spec/bundler/spec_helper.rb:78:in `block (2 levels) in <top (required)>' make: *** [yes-test-bundler] Error 1 ``` * [ruby/irb] Unnamed groups are not captured when named groups are used ruby/irb@0a641a69b0 * [ruby/reline] Work with wrong $/ value correctly ruby/reline@962ebf5a1b * [ruby/irb] Detect multiple lines output simplify The old implementation performance test code: require 'objspace' puts "%.5g MB" % (ObjectSpace.memsize_of_all * 0.001 * 0.001) /\A.*\Z/ !~ ('abc' * 20_000_000) puts "%.5g MB" % (ObjectSpace.memsize_of_all * 0.001 * 0.001) and run `time test.rb`: 2.5868 MB 62.226 MB real 0m1.307s user 0m0.452s sys 0m0.797s The new implementation performance test code: require 'objspace' puts "%.5g MB" % (ObjectSpace.memsize_of_all * 0.001 * 0.001) ('abc' * 20_000_000).include?("\n") puts "%.5g MB" % (ObjectSpace.memsize_of_all * 0.001 * 0.001) and run `time test.rb`: 2.5861 MB 62.226 MB real 0m0.132s user 0m0.088s sys 0m0.042s ruby/irb@40d6610baf * [ruby/reline] Suppress error in case INPUTRC env is empty ruby/reline@bce7e7562b * [ruby/reline] Add yamatanooroti rendering test ruby/reline@f092519525 * [ruby/reline] Rename test suite name of yamatanooroti test ruby/reline@b0f32f5de4 * [ruby/reline] Add a comment why rescue yamatanooroti loading error on the test ruby/reline@2a8061daec * [ruby/irb] Suppress crashing when EncodingError has occurred without lineno ruby/irb@13572d8cdc * [ruby/reline] Suppress error when check ambiguous char width in LANG=C ruby/reline@623dffdd75 * [ruby/io-console] Enable only interrupt bits on `intr: true` ruby/io-console@baaf929041 * [ruby/io-console] bump up to 0.5.4 * [ruby/io-console] Update the minimum requirement of Ruby version ruby/io-console@73e7b6318a * [ruby/io-console] Filter Ruby engine name rather than just /ruby/ This breaks tests using this path on JRuby because the `jruby` executable turns into `jjruby` after the sub. ruby/io-console@e5951aa34c * [ruby/io-console] bump up to 0.5.5 * [ruby/io-console] Prefer keyword arguments ruby/io-console@5facbfc4c8 * [ruby/io-console] [DOC] Improved about `intr:` ruby/io-console@82b630cd79 * [ruby/io-console] Just ignore the extension on other than CRuby ruby/io-console@41b6f09574 * [ruby/io-console] bump up to 0.5.6 Co-authored-by: KOBAYASHI Shuji <[email protected]> Co-authored-by: Ben <[email protected]> Co-authored-by: Yusuke Endoh <[email protected]> Co-authored-by: MSP-Greg <[email protected]> Co-authored-by: Nobuyoshi Nakada <[email protected]> Co-authored-by: Kenta Murata <[email protected]> Co-authored-by: Lars Kanis <[email protected]> Co-authored-by: Lars Kanis <[email protected]> Co-authored-by: Alan Wu <[email protected]> Co-authored-by: Hiroshi SHIBATA <[email protected]> Co-authored-by: Nobuhiro IMAI <[email protected]> Co-authored-by: Nick Lewis <[email protected]> Co-authored-by: S-H-GAMELINKS <[email protected]> Co-authored-by: Koichi Sasada <[email protected]> Co-authored-by: Kazuhiro NISHIYAMA <[email protected]> Co-authored-by: Charles Oliver Nutter <[email protected]>

ruby#55 If we had put multiple open braces on a line the with no closing brace spaces_of_nest array keeps getting '0' added to it. This means that when we pop off of this array we are saying that we should be in position zero for the next line. This is an issue because we don't always want to be in position 0 after a closing brace. Example: ``` [[[ ] ] ] ``` In the above example the 'spaces_of_nest' array looks like this after the first line is entered: [0,0,0]. We really want to be indented 4 spaces for the 1st closing brace 2 for the 2nd and 0 for the 3rd. i.e. we want it to be: [0,2,4]. We also saw this issue with a heredoc inside of an array. ``` [<<FOO] hello FOO ```

* [ruby/reline] Sort completion list #### Legacy mode: ```console $ irb --legacy irb(main):001:0> l[TAB][TAB] lambda load local_variables loop ``` #### Before this patch: ```console $ irb irb(main):001:0> l[TAB][TAB] local_variables loop lambda load ``` #### After this patch: ```console $ irb irb(main):001:0> l[TAB][TAB] lambda load local_variables loop ``` ruby/reline@6074069c7d * Drop an invalid char as UTF-8 * Add test_completion_with_indent_and_completer_quote_characters This is for 8a705245e55575d4d310a2e956b89a36a5931971. * [ruby/irb] Add tests for RubyLex The set_auto_indent method calculates the correct number of spaces for indenting a line. We think there might be a few bugs in this method so we are testing the current functionality to make sure nothing breaks when we address those bugs. Example test failure: ``` 1) Failure: TestIRB::TestRubyLex#test_auto_indent [/Users/Ben/Projects/irb/test/irb/test_ruby_lex.rb:75]: Calculated the wrong number of spaces for: def each_top_level_statement initialize_input catch(:TERM_INPUT) do loop do begin prompt unless l = lex throw :TERM_INPUT if @line == '' else . <10> expected but was <12>. ``` ruby/irb@752d5597ab * [ruby/reline] Degenerate the terminal size to [$LINES, $COLUMNS] if it is unknown This is a workaround for ruby/irb#50 ruby/reline@5725677d1a * [ruby/irb] Fix newline depth with multiple braces This commit fixes the check_newline_depth_difference method to multiple open braces on one line into account. Before this change we were subtracting from the depth in check_newline_depth_difference on every open brace. This is the right thing to do if the opening and closing brace are on the same line. For example in a method definition we have an opening and closing parentheses we want to add 1 to our depth, and then remove it. ``` def foo() end ``` However this isn't the correct behavior when the brace spans multiple lines. If a brace spans multiple lines we don't want to subtract from check_newline_depth_difference and we want to treat the braces the same way as we do `end` and allow check_corresponding_token_depth to pop the correct depth. Example of bad behavior: ``` def foo() [ ] puts 'bar' end ``` Example of desired behavior: ``` def foo() [ ] puts 'bar' end ``` ruby/irb@7dc8af01e0 * text/readline/test_readline.rb - fix skip on Reline (#2743) TestRelineAsReadline#test_input_metachar passes on MinGW * Add "require 'openstruct'" what is forgotten * [ruby/irb] Fix lib name of OpenStruct ruby/irb@1f3a84ab6b * Add load path and require for ruby/ruby * Rescue EOFError If C-d is pressed before IRB is ready, IRB crashes because EOFError occurs. * Complete indented and quoted string correctly def foo ''.upca[TAB] This will be completed to be: def foo ''.upcase The indent was gone. This commit fixes the bug. * [ruby/irb] Fix crashing when multiple open braces per line ruby/irb#55 If we had put multiple open braces on a line the with no closing brace spaces_of_nest array keeps getting '0' added to it. This means that when we pop off of this array we are saying that we should be in position zero for the next line. This is an issue because we don't always want to be in position 0 after a closing brace. Example: ``` [[[ ] ] ] ``` In the above example the 'spaces_of_nest' array looks like this after the first line is entered: [0,0,0]. We really want to be indented 4 spaces for the 1st closing brace 2 for the 2nd and 0 for the 3rd. i.e. we want it to be: [0,2,4]. We also saw this issue with a heredoc inside of an array. ``` [<<FOO] hello FOO ``` ruby/irb@80c69c8272 * Support history-size in .inputrc correctly * Introduce an abstracted structure about the encoding of Reline The command prompt on Windows always uses Unicode to take input and print output but most Reline implementation depends on Encoding.default_external. This commit introduces an abstracted structure about the encoding of Reline. * Remove an unused setting variable * Use Reline.encoding_system_needs if exists * Add tests for vi_insert and vi_add * Implement vi_insert_at_bol and vi_add_at_eol * [ruby/reline] Implement vi_to_next_char ruby/reline@066ecb0a21 * [ruby/reline] Implement vi_prev_char and vi_to_prev_char ruby/reline@0ad3ee63fa * [ruby/readline-ext] Include ruby/assert.h in ruby/ruby.h so that assertions can be there ruby/readline-ext@4d44c12832 * Stop using minitest dependent methods * Skip a test that uses assert_ruby_status if it doesn't exist * Use omit instead of skip * Check DONT_RUN_RELINE_TEST envvar * [ruby/irb] Add newline_before_multiline_output ruby/irb@9eb1801a66 * [ruby/irb] Fix compatibility with rails before 5.2 Rails before 5.2 added Array#append as an alias to Array#<< , so that it expects only one argument. However ruby-2.5 added Array#append as an alias to Array#push which takes any number of arguments. If irb completion is used in `rails c` (for example "IO.<tab>") it fails with: irb/completion.rb:206:in `<<': wrong number of arguments (given 3, expected 1) (ArgumentError) Using Array#push instead of Array#append fixes compatibility. ruby/irb@5b7bbf9c34 * Reline: Use a more robust detection of MinTTY The previous detection per get_screen_size fails when stdout is passed to a pipe. That is the case when running ruby tests in parallel ("-j" switch). In this case Reline believes that it's running on MinTTY and the tests are running with ANSI IOGate instead of the Windows adapter on MINGW. So parallel test results were different to that of a single process. This commit fixes these differencies. The code is taken from git sources and translated to ruby. NtQueryObject() is replaced by GetFileInformationByHandleEx(), because NtQueryObject() is undocumented and is more difficult to use: https://github.com/git-for-windows/git/blob/c5a03b1e29c69f3f06c8fabd92493edb73469176/compat/winansi.c#L558 * Reline: Fix changed test results due to change to UTF-8 on Windows In commit f8ea2860b0cac1aec79978e6c44168802958e8af the Reline encoding for native windows console was changed to hardcoded UTF-8. This caused failures in reline and readline tests, but they were hidden, because parallel ruby tests incorrectly used Reline::ANSI as IOGate. Tests failures were raised in single process mode, but not with -j switch. This patch corrects encodings on native Windows console. * [ruby/irb] [ruby/irb] Rewrite an expression to detect multiline ruby/irb@ed5cf375a6 ruby/irb@5b7bbf9c34 * [ruby/reline] Implement vi_change_meta ruby/reline@8538e0e10f * Always refer to Reline::IOGate.encoding * Always use UTF-8 for Reline::GeneralIO on Windows * Use test_mode on Reline::History::Test for encoding * [ruby/reline] Support GNOME style Home/End key sequences [Bug #16510] ruby/reline@788f0df845 * [ruby/irb] Add a new easter egg: dancing ruby ruby/irb@e37dc7e58e * [ruby/irb] Exclude useless files from RDoc ruby/irb@8f1ab2400c * [ruby/irb] Exclude useless files from RDoc * Fix inaccuracy in encoding tests These tests assume Encoding.find('locale') == Encoding.find('external') and fail if they are distinct. * [ruby/reline] Fix Reline::Windows#scroll_down I mistook Right and Bottom. ruby/reline@8be401c5f5 * [ruby/reline] Bypass cursor down when a char is rendered at eol on Windows A newline is automatically inserted if a character is rendered at eol on Windows command prompt. ruby/reline@4bfea07e4a * [ruby/reline] Organize special keys escape sequences ruby/reline@41deb1a3d9 * [ruby/readline-ext] Remove unnecessary -I$(top_srcdir) when it's an individual gem ruby/readline-ext@efaca4a5f4 * [ruby/readline-ext] Check TestRelineAsReadline existance ruby/readline-ext@c0a6303168 * [ruby/readline-ext] The ruby/assert.h is adopted by Ruby 2.7 or later ruby/readline-ext@106c31fc1b * Revert "[ruby/readline-ext] Include ruby/assert.h in ruby/ruby.h so that assertions can be there" This reverts commit 425b2064d394639101854c83a061a0918b33b857. This cherry-pick was a mistake. * [ruby/readline-ext] Use require check instead of DONT_RUN_RELINE_TEST env ruby/readline-ext@1df99d1481 * [ruby/readline-ext] Add spec.extensions ruby/readline-ext@8c33abb13c * [ruby/readline-ext] Use rake/extensiokntask to build ruby/readline-ext@b0b5f709bd * Fix readline build dependency * [ruby/irb] Add test_complete_symbol ruby/irb@dbbf086c1f * [ruby/irb] Check doc namespace correctly IRB::InputCompletor::PerfectMatchedProc crashes when doc not found because a variable name was incorrect. ruby/irb@889fd4928f * [ruby/irb] Fix auto indent with closed brace A closed brace in auto-indent shouldn't affect the next brace in the same line, but it behaves like below: p() { } It's a bug. ruby/irb@fbe59e344f * [ruby/irb] Use 0.step instead of (..0).each for Ruby 2.5 ruby/irb@5d628ca40e * Revert "[ruby/irb] Add test_complete_symbol" This reverts commit 3af3431c2c145134996e66f3d8d9ade8ad81bde0. * [ruby/irb] fix reserved words and completion for them ruby/irb@6184b227ad * Add test_complete_symbol The previous version of the test method used a symbol, ":abcdefg" to complete but longer symbols that can be completed are defined by other test methods of other libs. * test/irb/test_completion.rb: suppress a warning: unused literal ignored * [ruby/reline] Use IO#write instead of IO#print IO#print always adds a string of $\ automatically. ruby/reline@a93119c847 * [ruby/irb] Version 1.2.2 ruby/irb@a71753f15a * [ruby/reline] Version 0.1.3 ruby/reline@ea2b182466 * [ruby/irb] Include easter-egg.rb in gemspec `irb` doesn't run because this file isn't included in the gem. ruby/irb@73cda56d25 * [ruby/irb] Version 1.2.3 ruby/irb@dd56e06df5 * support multi-run test for test_readline.rb * [ruby/irb] `yield` outside method definition is a syntax error ruby/irb@dbc7b059c7 * test/readline - allow ENV control of test class creation In ruby/ruby, the tests run on both readline & reline by creating four test classes: ``` TestReadline TestReadlineHistory TestRelineAsReadline TestRelineAsReadlineHistory ``` Reline inports the test files and uses them in its CI. Adding the ENV control allows it to only run the `TestRelineAsReadline` classes. * Omit test_using_quoting_detection_proc_with_multibyte_input temporarily for random order test * support random order test. test_readline: HISTORY should be empty. test_using_quoting_detection_proc: test_using_quoting_detection_proc_with_multibyte_input: Readline.completer_quote_characters= and Readline.completer_word_break_characters= doesn't accept nil, so skip if previous values are nil. * Set Readline.completion_append_character = nil always GNU Readline add a white space when Readline.completion_append_character is not initialized. * Fix a typo [ci skip] * skip test if Reline.completion_proc is nil. Some other tests can set Reline.completion_proc, so if it is nil, simply skip this test. * Reset Reline.point TestRelineAsReadline#test_insert_text expects Readline.point == 0 at the beginning of the test, but a test violate this assumption. * Convert incompatible encoding symbol names * Ignore incompatible convert of symbols * Add workaround for test-bundler failure https://github.com/ruby/actions/runs/500526558?check_suite_focus=true#step:16:127 ``` Failures: 1) Bundler.setup when Bundler is bundled doesn't blow up Failure/Error: expect(err).to be_empty expected `"fatal: not a git repository (or any of the parent directories): .git\nfatal: not a git repository (o...the parent directories): .git\nfatal: not a git repository (or any of the parent directories): .git".empty?` to return true, got false Commands: $ /home/runner/work/actions/actions/snapshot-master/ruby \ -I/home/runner/work/actions/actions/snapshot-master/lib:/home/runner/work/actions/actions/snapshot-master/spec/bundler \ -rsupport/hax -rsupport/artifice/fail \ /home/runner/work/actions/actions/snapshot-master/libexec/bundle install --retry 0 Resolving dependencies... Using bundler 2.1.4 Bundle complete! 1 Gemfile dependency, 1 gem now installed. Use `bundle info [gemname]` to see where a bundled gem is installed. fatal: not a git repository (or any of the parent directories): .git fatal: not a git repository (or any of the parent directories): .git fatal: not a git repository (or any of the parent directories): .git # $? => 0 $ /home/runner/work/actions/actions/snapshot-master/ruby \ -I/home/runner/work/actions/actions/snapshot-master/lib:/home/runner/work/actions/actions/snapshot-master/spec/bundler \ -rsupport/hax -rsupport/artifice/fail \ /home/runner/work/actions/actions/snapshot-master/libexec/bundle exec ruby -e \ require\ \'bundler\'\;\ Bundler.setup fatal: not a git repository (or any of the parent directories): .git fatal: not a git repository (or any of the parent directories): .git fatal: not a git repository (or any of the parent directories): .git # $? => 0 # ./spec/bundler/runtime/setup_spec.rb:1056:in `block (3 levels) in <top (required)>' # ./spec/bundler/spec_helper.rb:111:in `block (3 levels) in <top (required)>' # ./spec/bundler/spec_helper.rb:111:in `block (2 levels) in <top (required)>' # ./spec/bundler/spec_helper.rb:78:in `block (2 levels) in <top (required)>' make: *** [yes-test-bundler] Error 1 ``` * [ruby/irb] Unnamed groups are not captured when named groups are used ruby/irb@0a641a69b0 * [ruby/reline] Work with wrong $/ value correctly ruby/reline@962ebf5a1b * [ruby/irb] Detect multiple lines output simplify The old implementation performance test code: require 'objspace' puts "%.5g MB" % (ObjectSpace.memsize_of_all * 0.001 * 0.001) /\A.*\Z/ !~ ('abc' * 20_000_000) puts "%.5g MB" % (ObjectSpace.memsize_of_all * 0.001 * 0.001) and run `time test.rb`: 2.5868 MB 62.226 MB real 0m1.307s user 0m0.452s sys 0m0.797s The new implementation performance test code: require 'objspace' puts "%.5g MB" % (ObjectSpace.memsize_of_all * 0.001 * 0.001) ('abc' * 20_000_000).include?("\n") puts "%.5g MB" % (ObjectSpace.memsize_of_all * 0.001 * 0.001) and run `time test.rb`: 2.5861 MB 62.226 MB real 0m0.132s user 0m0.088s sys 0m0.042s ruby/irb@40d6610baf * [ruby/reline] Suppress error in case INPUTRC env is empty ruby/reline@bce7e7562b * [ruby/reline] Add yamatanooroti rendering test ruby/reline@f092519525 * [ruby/reline] Rename test suite name of yamatanooroti test ruby/reline@b0f32f5de4 * [ruby/reline] Add a comment why rescue yamatanooroti loading error on the test ruby/reline@2a8061daec * [ruby/irb] Suppress crashing when EncodingError has occurred without lineno ruby/irb@13572d8cdc * [ruby/reline] Suppress error when check ambiguous char width in LANG=C ruby/reline@623dffdd75 * [ruby/io-console] Enable only interrupt bits on `intr: true` ruby/io-console@baaf929041 * [ruby/io-console] bump up to 0.5.4 * [ruby/io-console] Update the minimum requirement of Ruby version ruby/io-console@73e7b6318a * [ruby/io-console] Filter Ruby engine name rather than just /ruby/ This breaks tests using this path on JRuby because the `jruby` executable turns into `jjruby` after the sub. ruby/io-console@e5951aa34c * [ruby/io-console] bump up to 0.5.5 * [ruby/io-console] Prefer keyword arguments ruby/io-console@5facbfc4c8 * [ruby/io-console] [DOC] Improved about `intr:` ruby/io-console@82b630cd79 * [ruby/io-console] Just ignore the extension on other than CRuby ruby/io-console@41b6f09574 * [ruby/io-console] bump up to 0.5.6 Co-authored-by: KOBAYASHI Shuji <[email protected]> Co-authored-by: Ben <[email protected]> Co-authored-by: Yusuke Endoh <[email protected]> Co-authored-by: MSP-Greg <[email protected]> Co-authored-by: Nobuyoshi Nakada <[email protected]> Co-authored-by: Kenta Murata <[email protected]> Co-authored-by: Lars Kanis <[email protected]> Co-authored-by: Lars Kanis <[email protected]> Co-authored-by: Alan Wu <[email protected]> Co-authored-by: Hiroshi SHIBATA <[email protected]> Co-authored-by: Nobuhiro IMAI <[email protected]> Co-authored-by: Nick Lewis <[email protected]> Co-authored-by: S-H-GAMELINKS <[email protected]> Co-authored-by: Koichi Sasada <[email protected]> Co-authored-by: Kazuhiro NISHIYAMA <[email protected]> Co-authored-by: Charles Oliver Nutter <[email protected]>

BenKanouse mentioned this issue Dec 27, 2019

Fix for crashes with complex auto indent #56

Closed

aycabta mentioned this issue Dec 29, 2019

Add tests for RubyLex #57

Merged

BenKanouse mentioned this issue Dec 30, 2019

Fix crashing when multiple braces per line #58

Merged

BenKanouse mentioned this issue Jan 5, 2020

Don't fix indent level inside a heredoc #60

Closed

aycabta closed this as completed Jan 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Crashes with complex autoindent #55

Crashes with complex autoindent #55

aycabta commented Dec 26, 2019

BenKanouse commented Dec 27, 2019

Uh oh!

BenKanouse commented Jan 2, 2020

Uh oh!

aycabta commented Jan 7, 2020

Uh oh!

Crashes with complex autoindent #55

Crashes with complex autoindent #55

Comments

aycabta commented Dec 26, 2019

BenKanouse commented Dec 27, 2019

Uh oh!

BenKanouse commented Jan 2, 2020

Uh oh!

aycabta commented Jan 7, 2020

Uh oh!