Age | Commit message (Collapse) | Author |
|
https://github.com/ruby/json/commit/f6d6ca3c17
|
|
https://github.com/ruby/json/commit/d54063a790
|
|
Ref: https://github.com/ruby/json/issues/655
The actual buffer is still on the heap, but this saves a pair
of malloc/free.
This helps a lot on micro-benchmarks
Before:
```
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
Oj 531.598k i/100ms
JSON reuse 417.666k i/100ms
Calculating -------------------------------------
Oj 5.735M (± 1.3%) i/s (174.35 ns/i) - 28.706M in 5.005900s
JSON reuse 4.604M (± 1.4%) i/s (217.18 ns/i) - 23.389M in 5.080779s
Comparison:
Oj: 5735475.6 i/s
JSON reuse: 4604380.3 i/s - 1.25x slower
```
After:
```
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
Oj 518.700k i/100ms
JSON reuse 483.370k i/100ms
Calculating -------------------------------------
Oj 5.722M (± 1.8%) i/s (174.76 ns/i) - 29.047M in 5.077823s
JSON reuse 5.278M (± 1.5%) i/s (189.46 ns/i) - 26.585M in 5.038172s
Comparison:
Oj: 5722283.8 i/s
JSON reuse: 5278061.7 i/s - 1.08x slower
```
Bench:
```ruby
require 'benchmark/ips'
require 'oj'
require 'json'
json_encoder = JSON::State.new(JSON.dump_default_options)
test_data = [1, "string", { a: 1, b: 2 }, [3, 4, 5]]
Oj.default_options = Oj.default_options.merge(mode: :compat)
Benchmark.ips do |x|
x.config(time: 5, warmup: 2)
x.report("Oj") do
Oj.dump(test_data)
end
x.report("JSON reuse") do
json_encoder.generate(test_data)
end
x.compare!(order: :baseline)
end
```
https://github.com/ruby/json/commit/72110f7992
|
|
Ref: https://github.com/ruby/json/issues/655
Rather than to write the number backward, and then reverse
the buffer, we can start from the back of the buffer and write
the number in the proper direction.
Before:
```
== Encoding integers (8009 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 8.606k i/100ms
oj 9.598k i/100ms
Calculating -------------------------------------
json 86.059k (± 0.8%) i/s (11.62 μs/i) - 430.300k in 5.000416s
oj 97.409k (± 0.6%) i/s (10.27 μs/i) - 489.498k in 5.025360s
Comparison:
json: 86058.8 i/s
oj: 97408.8 i/s - 1.13x faster
```
After:
```
== Encoding integers (8009 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json (reuse) 9.500k i/100ms
json 9.359k i/100ms
oj 9.722k i/100ms
Calculating -------------------------------------
json (reuse) 96.270k (± 0.4%) i/s (10.39 μs/i) - 484.500k in 5.032777s
json 94.800k (± 2.2%) i/s (10.55 μs/i) - 477.309k in 5.037495s
oj 97.131k (± 0.7%) i/s (10.30 μs/i) - 486.100k in 5.004822s
Comparison:
json (reuse): 96270.1 i/s
oj: 97130.5 i/s - same-ish: difference falls within error
json: 94799.9 i/s - same-ish: difference falls within error
```
https://github.com/ruby/json/commit/0655b58d14
|
|
https://github.com/ruby/stringio/commit/ef03f9368d
|
|
https://github.com/ruby/stringio/commit/a2aab4721c
|
|
In Ruby 2.7 and later, it is the same as `StringValue`.
https://github.com/ruby/stringio/commit/561ea67ea8
|
|
|
|
Prior to 2.7.3, `JSON::Ext::Parser` would only take kwargs.
So if json_pure 2.7.4 is loaded with `json <= 2.7.2` (or stdlib)
it blows up.
Ref: https://github.com/ruby/json/issues/650
Fix: https://github.com/ruby/json/issues/651
https://github.com/ruby/json/commit/4d9dc98817
|
|
Fix: https://github.com/ruby/json/issues/646
Since both `json` and `json_pure` expose the same files, if the
versions don't match, the native extension may be loaded with Ruby
code that don't match and is incompatible.
By doing the `require json/ext/generator/state` from C we ensure
we're at least loading that.
But this is a dirty workaround for the 2.7.x branch, we should
find a better way to fully isolate the two gems.
https://github.com/ruby/json/commit/dfdd4acf36
|
|
Ref: https://github.com/ruby/json/issues/647
Ref: https://github.com/rubygems/rubygems/pull/6490
Older rubygems are executing `extconf.rb` with a broken `$LOAD_PATH`
causing the `json` gem native extension to be loaded with the stdlib
version of the `.rb` files.
This fails with
```
json/common.rb:82:in `initialize': wrong number of arguments (given 1, expected 0) (ArgumentError)
```
Since this is just for `extconf.rb` we can probably just accept that
extra argument and ignore it.
The bug was fixed in rubygems 3.4.9 / 2023-03-20
https://github.com/ruby/json/commit/1f5e849fe0
|
|
https://github.com/ruby/json/commit/7f079b25be
|
|
https://github.com/ruby/json/commit/fb25e94aea
|
|
Fix: https://github.com/ruby/json/issues/437
Before:
```json
{
"foo": {
},
"bar": [
]
}
```
After:
```json
{
"foo": {},
"bar": []
}
```
|
|
|
|
https://github.com/ruby/json/commit/937c8d2e65
|
|
https://github.com/ruby/json/commit/7a3b482013
|
|
https://github.com/ruby/json/commit/a48be35825
|
|
|
|
This regressed since 2.7.2.
https://github.com/ruby/json/commit/35407d6635
|
|
* Otherwise the gem always uses the pure-Ruby backend
as it's missing that file and rescuing the LoadError.
https://github.com/ruby/json/commit/1e2809b0b0
|
|
Fix: https://github.com/ruby/json/pull/625
Declaring the buffer in a sub block cause bugs on some compilers.
https://github.com/ruby/json/commit/90967c9eb0
|
|
Co-authored-by: Jean Boussier <[email protected]>
|
|
https://github.com/ruby/json/commit/b240bde402
Co-authored-by: Jean Boussier <[email protected]>
|
|
https://github.com/ruby/json/commit/d4968d2e48
|
|
https://github.com/ruby/json/commit/42edaf7f17
|
|
Profiling revealed that we were spending lots of time growing the buffer.
Buffer operations is definitely something we want to optimize, but for
this specific benchmark what we're interested in is UTF-8 scanning performance.
Each iteration of the two scaning benchmark were producing 20MB of JSON,
now they only produce 5MB.
Now:
```
== Encoding mostly utf8 (5001001 bytes)
ruby 3.4.0dev (2024-10-18T19:01:45Z master https://github.com/ruby/json/commit/7be9a333ca) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
json 35.000 i/100ms
oj 36.000 i/100ms
rapidjson 10.000 i/100ms
Calculating -------------------------------------
json 359.161 (± 1.4%) i/s (2.78 ms/i) - 1.820k in 5.068542s
oj 359.699 (± 0.6%) i/s (2.78 ms/i) - 1.800k in 5.004291s
rapidjson 99.687 (± 2.0%) i/s (10.03 ms/i) - 500.000 in 5.017321s
Comparison:
json: 359.2 i/s
oj: 359.7 i/s - same-ish: difference falls within error
rapidjson: 99.7 i/s - 3.60x slower
```
https://github.com/ruby/json/commit/1a338532d2
|
|
Since we're looking up the table anyway, we might as well store the
UTF-8 char length in it. For single byte characters that don't need
escaping we store `0`.
This helps on strings with lots of multi-byte characters:
Before:
```
== Encoding mostly utf8 (20004001 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 6.000 i/100ms
oj 10.000 i/100ms
rapidjson 2.000 i/100ms
Calculating -------------------------------------
json 67.978 (± 1.5%) i/s (14.71 ms/i) - 342.000 in 5.033062s
oj 100.876 (± 2.0%) i/s (9.91 ms/i) - 510.000 in 5.058080s
rapidjson 26.389 (± 7.6%) i/s (37.89 ms/i) - 132.000 in 5.027681s
Comparison:
json: 68.0 i/s
oj: 100.9 i/s - 1.48x faster
rapidjson: 26.4 i/s - 2.58x slower
```
After:
```
== Encoding mostly utf8 (20004001 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 7.000 i/100ms
oj 10.000 i/100ms
rapidjson 2.000 i/100ms
Calculating -------------------------------------
json 75.187 (± 2.7%) i/s (13.30 ms/i) - 378.000 in 5.030111s
oj 95.196 (± 2.1%) i/s (10.50 ms/i) - 480.000 in 5.043565s
rapidjson 25.969 (± 3.9%) i/s (38.51 ms/i) - 130.000 in 5.011471s
Comparison:
json: 75.2 i/s
oj: 95.2 i/s - 1.27x faster
rapidjson: 26.0 i/s - 2.90x slower
```
https://github.com/ruby/json/commit/51e2631d1f
|
|
If we assume that even UTF-8 strings are mostly ASCII, we can implement a
fast path for the ASCII parts.
Before:
```
== Encoding mixed utf8 (20012001 bytes)
ruby 3.4.0dev (2024-10-18T15:12:54Z master https://github.com/ruby/json/commit/d1b5c10957) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
json 5.000 i/100ms
oj 9.000 i/100ms
rapidjson 2.000 i/100ms
Calculating -------------------------------------
json 49.403 (± 2.0%) i/s (20.24 ms/i) - 250.000 in 5.062647s
oj 100.120 (± 2.0%) i/s (9.99 ms/i) - 504.000 in 5.035349s
rapidjson 26.404 (± 0.0%) i/s (37.87 ms/i) - 132.000 in 5.001025s
Comparison:
json: 49.4 i/s
oj: 100.1 i/s - 2.03x faster
rapidjson: 26.4 i/s - 1.87x slower
```
After:
```
== Encoding mixed utf8 (20012001 bytes)
ruby 3.4.0dev (2024-10-18T15:12:54Z master https://github.com/ruby/json/commit/d1b5c10957) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
json 10.000 i/100ms
oj 9.000 i/100ms
rapidjson 2.000 i/100ms
Calculating -------------------------------------
json 95.686 (± 2.1%) i/s (10.45 ms/i) - 480.000 in 5.018575s
oj 96.875 (± 2.1%) i/s (10.32 ms/i) - 486.000 in 5.019097s
rapidjson 26.260 (± 3.8%) i/s (38.08 ms/i) - 132.000 in 5.033151s
Comparison:
json: 95.7 i/s
oj: 96.9 i/s - same-ish: difference falls within error
rapidjson: 26.3 i/s - 3.64x slower
```
https://github.com/ruby/json/commit/f8166c2d7f
|
|
Closes: https://github.com/ruby/json/pull/512
https://github.com/ruby/json/commit/d882a45d82
Co-Authored-By: lukeg <[email protected]>
|
|
Fix: https://github.com/ruby/json/issues/534
Only include up to 32 bytes of unparseable the source.
https://github.com/ruby/json/commit/f44995cfb6
|
|
https://github.com/ruby/json/commit/09e1df2643
|
|
https://github.com/ruby/json/commit/1a9af430d2
|
|
Avoid needless hash allocations and such that degrade performance
significantly on micro-benchmarks.
|
|
Fix: https://github.com/ruby/json/issues/419
|
|
unnecessary use of `rb_enc_get()`
(https://github.com/ruby/strscan/pull/108)
- before: #106
## Why?
In `rb_strseq_index()`, the result of `rb_enc_check()` is used.
-
https://github.com/ruby/ruby/blob/6c7209cd3788ceec01e504d99057f9d3b396be84/string.c#L4335-L4368
> enc = rb_enc_check(str, sub);
> return strseq_core(str_ptr, str_ptr_end, str_len, sub_ptr, sub_len,
offset, enc);
-
https://github.com/ruby/ruby/blob/6c7209cd3788ceec01e504d99057f9d3b396be84/string.c#L4309-L4318
```C
strseq_core(const char *str_ptr, const char *str_ptr_end, long str_len,
const char *sub_ptr, long sub_len, long offset, rb_encoding *enc)
{
const char *search_start = str_ptr;
long pos, search_len = str_len - offset;
for (;;) {
const char *t;
pos = rb_memsearch(sub_ptr, sub_len, search_start, search_len, enc);
```
## Benchmark
It shows String as a pattern is 1.24x faster than Regexp as a pattern.
```
$ benchmark-driver benchmark/check_until.yaml
Warming up --------------------------------------
regexp 9.225M i/s - 9.328M times in 1.011068s (108.40ns/i)
regexp_var 9.327M i/s - 9.413M times in 1.009214s (107.21ns/i)
string 9.200M i/s - 9.355M times in 1.016840s (108.70ns/i)
string_var 11.249M i/s - 11.255M times in 1.000578s (88.90ns/i)
Calculating -------------------------------------
regexp 9.565M i/s - 27.676M times in 2.893476s (104.55ns/i)
regexp_var 10.111M i/s - 27.982M times in 2.767496s (98.90ns/i)
string 10.060M i/s - 27.600M times in 2.743465s (99.40ns/i)
string_var 12.519M i/s - 33.746M times in 2.695615s (79.88ns/i)
Comparison:
string_var: 12518707.2 i/s
regexp_var: 10111089.6 i/s - 1.24x slower
string: 10060144.4 i/s - 1.24x slower
regexp: 9565124.4 i/s - 1.31x slower
```
https://github.com/ruby/strscan/commit/ff2d7afa19
|
|
(https://github.com/ruby/strscan/pull/101)
https://github.com/ruby/strscan/commit/d31274f41b
|
|
|
|
(https://github.com/ruby/fiddle/pull/155)
GitHub: fix https://github.com/ruby/fiddle/pull/153
mkmf requires fileutils. JRuby doesn't like it.
https://github.com/ruby/fiddle/commit/df4bd21fc3
|
|
(https://github.com/ruby/fiddle/pull/157)
The GCC warning puts undefined macro names in double quotes while clang
puts them in single quotes.
https://github.com/ruby/fiddle/commit/7dcb40bccb
|
|
FFI backend
(https://github.com/ruby/fiddle/pull/156)
* From https://github.com/ruby/reline/issues/766#issuecomment-2422135968
https://github.com/ruby/fiddle/commit/eea9fd0cc4
|
|
https://github.com/ruby/zlib/commit/5a02eac37f
|
|
Suppress false positive warnings due to a bug in gcc-14 optimizer.
https://github.com/ruby/zlib/commit/eb1cf426c2
|
|
With the introduction of Happy Eyeballs Version 2 to `Socket::tcp`, the following areas have been mainly enhanced:
- How the value specified for `connect_timeout` works
- How Socket.tcp operates with Happy Eyeballs Version 2
A description for the new option `fast_fallback` has been added in https://github.com/ruby/ruby/pull/11813.
Notes:
Merged: https://github.com/ruby/ruby/pull/11920
Merged-By: nobu <[email protected]>
|
|
https://github.com/ruby/json/commit/db9a489ca2
|
|
https://github.com/ruby/json/commit/e2e9936047
|
|
Extracted from: https://github.com/ruby/json/pull/512
Use `rb_hash_lookup2` to check for hash key existence instead
of going through `rb_funcall`.
https://github.com/ruby/json/commit/43835a0d13
Co-Authored-By: lukeg <[email protected]>
|
|
Some changes were missed in the automatic sync.
Notes:
Merged: https://github.com/ruby/ruby/pull/11911
|
|
Returning state->max_nesting is not valid because it's not a Ruby object.
https://github.com/ruby/json/commit/6679ceb
|
|
This optimization is based on a few assumptions:
- Most strings are ASCII only.
- Most strings had their coderange scanned already.
If the above is true, then by checking the string coderange, we can
use a much more streamlined function to encode ASCII strings.
Before:
```
== Encoding twitter.json (466906 bytes)
ruby 3.4.0preview2 (2024-10-07 master 32c733f57b) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
json 140.000 i/100ms
oj 230.000 i/100ms
rapidjson 108.000 i/100ms
Calculating -------------------------------------
json 1.464k (± 1.4%) i/s (682.83 μs/i) - 7.420k in 5.067573s
oj 2.338k (± 1.5%) i/s (427.64 μs/i) - 11.730k in 5.017336s
rapidjson 1.075k (± 1.6%) i/s (930.40 μs/i) - 5.400k in 5.025469s
Comparison:
json: 1464.5 i/s
oj: 2338.4 i/s - 1.60x faster
rapidjson: 1074.8 i/s - 1.36x slower
```
After:
```
== Encoding twitter.json (466906 bytes)
ruby 3.4.0preview2 (2024-10-07 master 32c733f57b) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
json 189.000 i/100ms
oj 228.000 i/100ms
rapidjson 108.000 i/100ms
Calculating -------------------------------------
json 1.903k (± 1.2%) i/s (525.55 μs/i) - 9.639k in 5.066521s
oj 2.306k (± 1.3%) i/s (433.71 μs/i) - 11.628k in 5.044096s
rapidjson 1.069k (± 2.4%) i/s (935.38 μs/i) - 5.400k in 5.053794s
Comparison:
json: 1902.8 i/s
oj: 2305.7 i/s - 1.21x faster
rapidjson: 1069.1 i/s - 1.78x slower
```
|