ruby-core

Issue #7758 has been updated by Luis Lavena.

Assignee changed from Luis Lavena to Nobuyoshi Nakada

----------------------------------------
Bug #7758: Ruby on Windows crashes when active codepage is codepage 65001 a=
nd outputting unicode character
https://bugs.ruby-lang.org/issues/7758#change-54655

* Author: Josh C
* Status: Assigned
* Priority: Normal
* Assignee: Nobuyoshi Nakada
* ruby -v: ruby 1.9.3p374
* Backport:=20
----------------------------------------
=3Dbegin
I have a script that contains the unicode BOX_DRAWINGS_LIGHT_VERTICAL_AND_R=
IGHT character (http://www.fileformat.info/info/unicode/char/251c/index.htm=
). When the console code page is set to 65001 (utf-8), both ruby 1.9.3 p125=
 and p374 crash, but ruby 1.8.7 p357 does not crash (nor does it output the=
 character correctly). When the code page is set to 437, ruby 1.9.3 (both v=
ersions) correctly output the character, while ruby 1.8.7 outputs the wrong=
 character again.

I don't care about ruby 1.8.7 so much, but I would expect ruby 1.9.3 to out=
put the correct character, especially since the console codepage is set to =
65001 (utf-8).

Here is my script:

 # -*- coding: utf-8 -*-
 puts "a"
 puts "=E2=94=9C"

I verified that the file is correctly encoded:

 C:\test>od.exe -t x1 test65001.rb
 0000000 23 20 2d 2a 2d 20 63 6f 64 69 6e 67 3a 20 75 74
 0000020 66 2d 38 20 2d 2a 2d 0a 70 75 74 73 20 22 61 22
 0000040 0a 70 75 74 73 20 22 e2 94 9c 22 0a 0a

Note the UTF-8 sequence `e2 94 9c`

Here are the codepage settings in the registry:
 C:\test>reg query hklm\SYSTEM\CurrentControlSet\Control\Nls\CodePage /v acp
 HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage
    acp    REG_SZ    1252
 C:\test>reg query hklm\SYSTEM\CurrentControlSet\Control\Nls\CodePage /v oe=
mcp
 HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage
    oemcp    REG_SZ    437
=20
On ruby 1.8.7:

 C:\test>ruby --version
 ruby 1.8.7 (2011-12-28 patchlevel 357) [i386-mingw32]
=20
 C:\test>chcp 437
 Active code page: 437
=20
 C:\test>ruby test65001.rb
 a
 =C3=A2=E2=80=9D=C5=93
=20
 C:\test>chcp 65001
 Active code page: 65001
=20
 C:\test>ruby test65001.rb
 a
 =C3=A2=E2=80=9D=C5=93

On ruby 1.9.3p125:

 C:\test>ruby --version
 ruby 1.9.3p125 (2012-02-16) [i386-mingw32]
=20
 C:\test>chcp 437
 Active code page: 437
=20
 C:\test>ruby test65001.rb
 a
 =C3=83
=20
 C:\test>chcp 65001
 Active code page: 65001
=20
 C:\test>ruby test65001.rb
 a
 =C3=83=C2=A2=C3=A2=E2=82=AC=C2=9D=C3=85=E2=80=9Ctest65001.rb:3: [BUG] rb_s=
ys_fail(<STDOUT>) - errno =3D=3D 0
 ruby 1.9.3p125 (2012-02-16) [i386-mingw32]
=20
 -- Control frame information ---------------------------------------------=
--
 c:0006 p:---- s:0014 b:0014 l:000013 d:000013 CFUNC  :write
 c:0005 p:---- s:0012 b:0012 l:000011 d:000011 CFUNC  :puts
 c:0004 p:---- s:0010 b:0010 l:000009 d:000009 CFUNC  :puts
 c:0003 p:0023 s:0006 b:0006 l:00215c d:0015b4 EVAL   test65001.rb:3
 c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
 c:0001 p:0000 s:0002 b:0002 l:00215c d:00215c TOP
=20
 -- Ruby level backtrace information --------------------------------------=
--
 test65001.rb:3:in `<main>'
 test65001.rb:3:in `puts'
 test65001.rb:3:in `puts'
 test65001.rb:3:in `write'
=20
 -- C level backtrace information -----------------------------------------=
--
 C:\Windows\SysWOW64\ntdll.dll(ZwWaitForSingleObject+0x15) [0x77bbf8b1]
 C:\Windows\syswow64\kernel32.dll(WaitForSingleObjectEx+0x43) [0x772d1194]
 C:\Windows\syswow64\kernel32.dll(WaitForSingleObject+0x12) [0x772d1148]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_vm_bugreport+0xf9) [0x62e5c61d]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_name_err_mesg_new+0x17a) [0x62d3a6fa]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_bug+0x2f) [0x62d3b45b]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_sys_fail+0x163) [0x62d3cfb7]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_io_close+0x171f) [0x62d651cb]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_vm_invoke_proc+0x2fe) [0x62e54586]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_funcall+0x59) [0x62e55035]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_io_puts+0x7e) [0x62d5fb7e]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_thread_mark+0x741) [0x62e49ca1]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_vm_invoke_proc+0x2fe) [0x62e54586]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_io_puts+0x200) [0x62d5fd00]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_thread_mark+0x741) [0x62e49ca1]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_vm_call+0x2b2) [0x62e5855a]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_vm_localjump_error+0x1f66) [0x62e4ed8=
a]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_vm_localjump_error+0x650e) [0x62e5333=
2]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_iseq_eval_main+0x98) [0x62e596f0]
 c:\ruby193\bin\msvcrt-ruby191.dll(rb_check_frozen+0x2a89) [0x62d3fed9]
 c:\ruby193\bin\msvcrt-ruby191.dll(ruby_run_node+0x48) [0x62d41ff4]
 [0x0040136f]
 [0x004010b9]
 [0x00401284]
 C:\Windows\syswow64\kernel32.dll(BaseThreadInitThunk+0x12) [0x772d339a]
 C:\Windows\SysWOW64\ntdll.dll(RtlInitializeExceptionChain+0x63) [0x77bd9ef=
2]
=20
 -- Other runtime information ---------------------------------------------=
--
=20
 * Loaded script: test65001.rb
=20
 * Loaded features:
=20
    0 enumerator.so
    1 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/enc/encdb.so
    2 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/enc/iso_8859_1.so
    3 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/enc/trans/transdb.so
    4 c:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/defaults.rb
    5 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/rbconfig.rb
    6 c:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/deprecate.rb
    7 c:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/exceptions.rb
    8 c:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/defaults/operating_syste=
m.rb
    9 c:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb
   10 c:/ruby193/lib/ruby/site_ruby/1.9.1/rubygems.rb
   11 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/enc/utf_16le.so
   12 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/enc/trans/utf_16_32.so
   13 c:/ruby193/lib/ruby/1.9.1/i386-mingw32/enc/trans/single_byte.so

 [NOTE]
 You may have encountered a bug in the Ruby interpreter or extension librar=
ies.
 Bug reports are welcome.
 For details: http://www.ruby-lang.org/bugreport.html
=20
 This application has requested the Runtime to terminate it in an unusual w=
ay.
 Please contact the application's support team for more information.

On ruby 1.9.3p374:

 c:\test>ruby --version
 ruby 1.9.3p374 (2013-01-15) [i386-mingw32]
=20
 c:\test>chcp 437
 Active code page: 437
=20
 c:\test>ruby test65001.rb
 a
 =C3=83
=20
 c:\test>chcp 65001
 Active code page: 65001
=20
 c:\test>ruby test65001.rb
 a
 =C3=83=C2=A2=C3=A2=E2=82=AC=C2=9D=C3=85=E2=80=9Ctest65001.rb:3: [BUG] rb_s=
ys_fail_str(<STDOUT>) - errno =3D=3D 0
 ruby 1.9.3p374 (2013-01-15) [i386-mingw32]
=20
 -- Control frame information ---------------------------------------------=
--
 c:0006 p:---- s:0014 b:0014 l:000013 d:000013 CFUNC  :write
 c:0005 p:---- s:0012 b:0012 l:000011 d:000011 CFUNC  :puts
 c:0004 p:---- s:0010 b:0010 l:000009 d:000009 CFUNC  :puts
 c:0003 p:0023 s:0006 b:0006 l:001afc d:00059c EVAL   test65001.rb:3
 c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
 c:0001 p:0000 s:0002 b:0002 l:001afc d:001afc TOP
=20
 -- Ruby level backtrace information --------------------------------------=
--
 test65001.rb:3:in `<main>'
 test65001.rb:3:in `puts'
 test65001.rb:3:in `puts'
 test65001.rb:3:in `write'
=20
 -- C level backtrace information -----------------------------------------=
--
 C:\Windows\SysWOW64\ntdll.dll(ZwWaitForSingleObject+0x15) [0x77bbf8b1]
 C:\Windows\syswow64\kernel32.dll(WaitForSingleObjectEx+0x43) [0x772d1194]
 C:\Windows\syswow64\kernel32.dll(WaitForSingleObject+0x12) [0x772d1148]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_bugreport+0xf9) [0x62e5c661]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_name_err_mesg_new+0x17a) [0x62d3a=
81a]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_bug+0x2f) [0x62d3b4f7]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_sys_fail_str+0x15f) [0x62d3d3a3]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_io_flush+0x1336) [0x62d64e22]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_invoke_proc+0x2fe) [0x62e5498a]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_funcall+0x59) [0x62e553e9]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_io_puts+0x78) [0x62d60484]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_mark+0x54d) [0x62e49b85]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_invoke_proc+0x2fe) [0x62e5498a]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_io_puts+0x248) [0x62d60654]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_mark+0x54d) [0x62e49b85]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_call+0x2b2) [0x62e5882e]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_localjump_error+0x1f84) [0x62e=
4f1b0]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_vm_localjump_error+0x6496) [0x62e=
536c2]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_iseq_eval_main+0xd7) [0x62e597ef]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(rb_check_frozen+0x2a85) [0x62d4055d]
 c:\Ruby193p374\bin\msvcrt-ruby191.dll(ruby_run_node+0x48) [0x62d42688]
 [0x0040136f]
 [0x004010b9]
 [0x00401284]
 C:\Windows\syswow64\kernel32.dll(BaseThreadInitThunk+0x12) [0x772d339a]
 C:\Windows\SysWOW64\ntdll.dll(RtlInitializeExceptionChain+0x63) [0x77bd9ef=
2]
=20
 -- Other runtime information ---------------------------------------------=
--
=20
 * Loaded script: test65001.rb
=20
 * Loaded features:

    0 enumerator.so
    1 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/encdb.so
    2 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/iso_8859_1.so
    3 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/trans/transdb.so
    4 c:/Ruby193p374/lib/ruby/site_ruby/1.9.1/rubygems/defaults.rb
    5 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/rbconfig.rb
    6 c:/Ruby193p374/lib/ruby/site_ruby/1.9.1/rubygems/deprecate.rb
    7 c:/Ruby193p374/lib/ruby/site_ruby/1.9.1/rubygems/exceptions.rb
    8 c:/Ruby193p374/lib/ruby/site_ruby/1.9.1/rubygems/defaults/operating_s=
ystem.rb
    9 c:/Ruby193p374/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb
   10 c:/Ruby193p374/lib/ruby/site_ruby/1.9.1/rubygems.rb
   11 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/utf_16le.so
   12 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/trans/utf_16_32.so
   13 c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/trans/single_byte.so

 [NOTE]
 You may have encountered a bug in the Ruby interpreter or extension librar=
ies.
 Bug reports are welcome.
 For details: http://www.ruby-lang.org/bugreport.html

 This application has requested the Runtime to terminate it in an unusual w=
ay.
 Please contact the application's support team for more information.

Based on the line `c:/Ruby193p374/lib/ruby/1.9.1/i386-mingw32/enc/iso_8859_=
1.so` it makes me think ruby is looking at the registry for the active code=
 page (cp1252) instead of calling GetConsoleOutputCP to see which codepage =
it should be using, but that is just a guess.

=3Dend


---Files--------------------------------
win32-bug-7758.diff (2.54 KB)


--=20
https://bugs.ruby-lang.org/