From: "Eregon (Benoit Daloze)" <noreply@...>
Date: 2022-11-21T13:47:31+00:00
Subject: [ruby-core:110834] [Ruby master Bug#19108] Format routines like pack blindly treat a string as ASCII-encoded

Issue #19108 has been updated by Eregon (Benoit Daloze).


Agreed, I think it should be ArgumentError since it's otherwise silently ignoring characters in the pack format string.
A non-verbose warning is better than the current state if ArgumentError is deemed too incompatible.

Here is real case where the silent warning caused confusion for `[1].pack('<L')`: https://github.com/oracle/truffleruby/issues/2791

----------------------------------------
Bug #19108: Format routines like pack blindly treat a string as ASCII-encoded
https://bugs.ruby-lang.org/issues/19108#change-100191

* Author: chrisseaton (Chris Seaton)
* Status: Open
* Priority: Normal
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Format routines like pack and unpack blindly treat a string as ASCII-encoded, even if they aren't ASCII or ASCII-compatible.

I tried to construct code that was misleading using ASCII-incompatible-encodings but couldn't do it in practice (no ASCII-incompatible encodings have a pack directive ASCII byte that is encoded as a printable character.)

But I could demonstrate at least some strange behaviour:

```
p ['foo'].pack('u').encoding # => #<Encoding:US-ASCII>
p ['foo'].pack('u'.encode('UTF-32BE')).encoding # => #<Encoding:ASCII-8BIT>
```

This is because the NUL characters in the second one (which aren't really NUL characters - they're part of the directive characters) explicitly trigger the encoding to change to binary.

There is a warning, but the warning is only for unexpected directives. How about disallowing or warning for non-ascii compatible format strings?



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>