Commit graph

215 commits

Author SHA1 Message Date
Jean Boussier
f171a263f7
[ruby/json] Fix: generate_json_float to reserve enough memory for large negative floats.
Fix: https://github.com/ruby/json/issues/807

Since https://github.com/ruby/json/pull/800, `fpconv_dtoa` can actually
generate up to 28 chars.

d73ae93d3c
2025-05-26 11:46:12 +09:00
Grant Birkinbine
dc69bebde7
[ruby/json] Update json_encoding_test.rb
0ac54a8161
2025-05-19 11:19:17 +09:00
Cody Horton
3468811ed1
[ruby/json] fix for pretty_generate throwing wrong number of arguments error
8433571dcf
2025-05-19 11:19:17 +09:00
Jean Boussier
cd7495a1d0 [ruby/json] Further improve parsing errors
Report EOF when applicable instead of an empty fragment.

Also stop fragment extraction on first whitespace.

cc1daba860
2025-05-13 14:12:22 +09:00
Jean Boussier
50ef208369 [ruby/json] parser.c: include line and column in error messages
30e35b9ba5
2025-05-13 14:12:22 +09:00
Jean Boussier
af79914002 [ruby/json] Favor decimal notation over scientific notation for floats
e.g.
```
JSON.dump(1746861937.7842371)
```

master:
```
"1.7468619377+9"
```

This branch and older json versions:
```
1746861937.7842371
```

In the end it's shorter, and according to `canada.json` benchmark
performance is the same.

866f72a437
2025-05-12 11:09:11 +09:00
Scott Myron
a3ec53bbb0 [ruby/json] Introduce ARM Neon and SSE2 SIMD.
(https://github.com/ruby/json/pull/743)

See the pull request for the long development history: https://github.com/ruby/json/pull/743

```
== Encoding activitypub.json (52595 bytes)
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
               after     2.913k i/100ms
Calculating -------------------------------------
               after     29.377k (± 2.0%) i/s   (34.04 μs/i) -    148.563k in   5.059169s

Comparison:
              before:    23314.1 i/s
               after:    29377.3 i/s - 1.26x  faster

== Encoding citm_catalog.json (500298 bytes)
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
               after   152.000 i/100ms
Calculating -------------------------------------
               after      1.569k (± 0.8%) i/s  (637.49 μs/i) -      7.904k in   5.039001s

Comparison:
              before:     1485.6 i/s
               after:     1568.7 i/s - 1.06x  faster

== Encoding twitter.json (466906 bytes)
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
               after   309.000 i/100ms
Calculating -------------------------------------
               after      3.115k (± 3.1%) i/s  (321.01 μs/i) -     15.759k in   5.063776s

Comparison:
              before:     2508.3 i/s
               after:     3115.2 i/s - 1.24x  faster
```

49003523da
2025-04-30 08:12:41 +02:00
Jean Boussier
5566a7f740 [ruby/json] Handle non-string keys returning immediate values via to_s
We can't directly call `RBASIC_CLASS` as the return value of
`to_s` may be an immediate.

12dc394d11
2025-04-30 08:12:41 +02:00
Jean Boussier
8fe3fb5d5a [ruby/json] Stop caching the generator state pointer
Fix: https://github.com/ruby/json/issues/790

If we end up calling something that spills the state
on the heap, the pointer we received is outdated and
may be out of sync.

2ffa4ea46b
2025-04-30 08:12:41 +02:00
Jean Boussier
f8720b01d9
[ruby/json] Remove caller location assertions
Fix: https://github.com/ruby/json/pull/781

Tests are ran a bit differently in ruby/ruby
and this assertion breaks.

0d4fc669ed
2025-03-31 09:39:22 +09:00
Hiroshi SHIBATA
a7ab156d94
Revert "Relax assertion pattern for ruby repo"
This reverts commit 09b9468873.
2025-03-31 09:39:22 +09:00
Hiroshi SHIBATA
09b9468873 Relax assertion pattern for ruby repo 2025-03-28 12:44:53 +09:00
Jean Boussier
3bbb52f5c5 [ruby/json] Add missing assert_deprecated_warning
1b16a82980
2025-03-28 12:44:53 +09:00
Jean Boussier
ec171b4075 [ruby/json] Move create_addtions logic in Ruby.
By leveraging the `on_load` callback we can move all this logic
out of the parser. Which mean we no longer have to duplicate
that logic in both parser and that we'll later be able to extract
it entirely from the gem.

f411ddf1ce
2025-03-28 12:44:53 +09:00
Jean Boussier
e8c46f4ca5 [ruby/json] JSON.load invoke the proc callback directly from the parser.
And substitute the return value like `Marshal.load` doesm
which I can only assume was the intent.

This also open the door to re-implement all the `create_addition`
logic in `json/common.rb`.

73d2137fd3
2025-03-28 12:44:53 +09:00
Jean Boussier
756b75f242 [ruby/json] Remove Class#json_creatable? monkey patch.
1ca7efed1f
2025-03-28 12:44:53 +09:00
Jean Boussier
26e55562ce [ruby/json] Cleanup JSON.pretty_generate
01c47a0555
2025-03-28 12:44:53 +09:00
Jean Boussier
2b9a9300ac [ruby/json] Deprecate JSON.fast_generate
6508455d82
2025-03-28 12:44:53 +09:00
Hiroshi SHIBATA
ee85745c61 Removed trailing spaces 2025-03-27 11:37:27 +09:00
eno
d1f3c81258 Faster integer formatting
This commit provides an alternative implementation for a
long → decimal conversion.

The main difference is that it uses an algorithm pulled from
https://github.com/jeaiii/itoa.
The source there is C++, it was converted by hand to C for
inclusion with this gem.
jeaiii's algorithm is covered by the MIT License, see source code.

On addition this version now also generates the string directly into
the fbuffer, foregoing the need to run a separate memory copy.

As a result, I see a speedup of 32% on Apple Silicon M1 for an
integer set of benchmarks.
2025-03-27 11:37:27 +09:00
Jean Boussier
f4529ecbe7 [ruby/json] Deprecate all *_default_options
Globally changing the behavior of the library is a bad idea, as
many different libraries may rely on `json` and may not expect it
and likely never tested that a different default config works for them.

If you need to change the behavior of JSON, it's best to do it only
locally, and not globally.

In addition the new `JSON::Coder` interface is much more suited for
that.

Another reason for the deprecation is that it's impossible to
make `JSON.load` and `JSON.dump` Ractor-safe with such API.

172762c6e4
2025-03-27 11:37:27 +09:00
eno
528c08cc5f [ruby/json] Adjust fpconv to add ".0" to integers
Adds a test case

fix

fa5bdf87cb
2025-03-24 14:35:04 +09:00
Jean Boussier
1d07deb422 [ruby/json] Raise a ParserError on all incomplete unicode escape sequence.
This was the behavior until `2.10.0` unadvertently changed it.

`"\u1"` would raise, but `"\u1zzz"` wouldn't.

7d0637b9e6
2025-03-12 18:02:09 +09:00
Rahim Packir Saibo
e20f1f76f4 [ruby/json] Fix JSON::GeneratorError#detailed_message with Ruby < 3.2
2e015ff839
2025-03-06 18:29:27 +09:00
Jean Boussier
0d62037fc0
[ruby/json] Ensure parser error snippets are valid UTF-8
Fix: https://github.com/ruby/json/issues/755

Error messages now include a snippet of the document
that doesn't parse to help locate the issue, however
the way it was done wasn't UTF-8 aware, and it could
result in exception messages with truncated characters.

It would be nice to go a bit farther and actually support
codepoints, but it's a lot of complexity to do it in C,
perhaps if we move that logic to Ruby given it's not a
performance sensitive codepath.

e144793b72
2025-02-27 13:32:32 +09:00
Jean Boussier
293ad8a4e9
Fix a compatibility issue with MultiJson.dump(obj, pretty: true)
Fix: https://github.com/ruby/json/issues/748

`MultiJson` pass `State#to_h` as options, and the `as_json`
property defaults to `false` but `false` wasn't accepted by
the constructor.
2025-02-12 13:15:01 +09:00
Étienne Barrié
b4bfbcaddc
Optimize Symbol generation in strict mode
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-02-06 16:02:03 +09:00
Étienne Barrié
f865148e19
Fix JSON::Coder to call as_json proc for NaN and Infinity
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-02-06 16:02:03 +09:00
Jean Boussier
98e1c2845a [ruby/json] Refactor convert_UTF8_to_JSON to split searching and escaping code
The goal is to be able to dispatch to more optimized search implementations
without having to duplicate the escaping code.

Somehow, this is a few % faster already:

```
== Encoding activitypub.json (52595 bytes)
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
               after     2.257k i/100ms
Calculating -------------------------------------
               after     22.930k (± 1.3%) i/s   (43.61 μs/i) -    115.107k in   5.020814s

Comparison:
              before:    21604.0 i/s
               after:    22930.1 i/s - 1.06x  faster

== Encoding citm_catalog.json (500298 bytes)
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
               after   137.000 i/100ms
Calculating -------------------------------------
               after      1.397k (± 1.1%) i/s  (715.57 μs/i) -      6.987k in   5.000408s

Comparison:
              before:     1344.4 i/s
               after:     1397.5 i/s - 1.04x  faster

== Encoding twitter.json (466906 bytes)
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
               after   249.000 i/100ms
Calculating -------------------------------------
               after      2.464k (± 1.8%) i/s  (405.81 μs/i) -     12.450k in   5.054131s

Comparison:
              before:     2326.5 i/s
               after:     2464.2 i/s - 1.06x  faster
```

8fb5ae807f
2025-02-03 10:05:25 +09:00
Jean Boussier
50e34fd768
Allow JSON::Fragment to be used even in strict mode 2025-01-28 15:42:20 +09:00
Étienne Barrié
89e316ad06
Introduce JSON::Coder
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-01-28 15:41:47 +09:00
tompng
86b262179d [ruby/json] Reject invalid number: - -.1 -e0
b9bfeecfa9
2025-01-20 14:20:55 +01:00
tompng
525d7a68e4 [ruby/json] Raise parse error on invalid comments
2f57f40467
2025-01-20 14:20:55 +01:00
tompng
c026e44bb5 [ruby/json] Fix parsing incomplete unicode escape "\uaaa"
86c0d4eb7e
2025-01-20 14:20:55 +01:00
Étienne Barrié
e8676cada8 [ruby/json] Introduce JSON::Fragment
9e3500f345

Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2025-01-20 14:20:55 +01:00
Jean Boussier
33708f2dc4 [ruby/json] Fix a regression in the parser with leading /
Ref: https://github.com/ruby/ruby/pull/12598

This could lead to an infinite loop.

f8cfa2696a
2025-01-20 10:31:56 +01:00
Jean Boussier
ef585744c0 Finalize Kevin's handrolled parser.
And get rid of the Ragel parser.

This is 7% faster on activitypub, 15% after on twitter and 11% faster
on citm_catalog.

There might be some more optimization opportunities, I did a quick
optimization pass to fix a regression in string parsing, but other
than that I haven't dug much in performance.
2025-01-20 16:09:00 +09:00
Jean Boussier
91918bb01f [ruby/json] Refactor JSONFixturesTest
c69963fbe5
2025-01-16 09:01:11 +09:00
Jean Boussier
599fbeaffa [ruby/json] Refactor JSON::Ext::Parser to split configuration and parsing state
Ref: https://github.com/ruby/json/pull/718

The existing `Parser` interface is pretty bad, as it forces to
instantiate a new instance for each document.

Instead it's preferable to only take the config and do all the
initialization needed, and then keep the parsing state on the
stack on in ephemeral memory.

This refactor makes the `JSON::Coder` pull request much easier to
implement in a performant way.

c8d5236a92

Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com>
2025-01-14 09:08:02 +01:00
Hiroshi SHIBATA
9f5ca6332b Refactor to omit JSON::GenericObject tests 2025-01-08 17:12:19 +09:00
Jean Boussier
1510d72bec [ruby/json] Fix generate(script_safe: true) to not confuse unrelated characters
Fix: https://github.com/ruby/json/issues/715

The first byte check was missing.

93a7f8717d
2024-12-05 09:16:22 +01:00
Jean Boussier
693a793521 JSON::GeneratorError expose invalid object
Fix: https://github.com/ruby/json/issues/710

Makes it easier to debug why a given tree of objects can't
be dumped as JSON.

Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com>
2024-11-26 15:11:05 +09:00
Jean Boussier
ee0de3fd4e [ruby/json] JSON.dump: write directly into the provided IO
Ref: https://github.com/ruby/json/issues/524

Rather than to buffer everything in memory.

Unfortunately Ruby doesn't provide an API to write into
and IO without first allocating a string, which is a bit
wasteful.

f017af6c0a
2024-11-26 15:11:05 +09:00
Peter Zhu
64827b15e8 [ruby/json] Fix $VERBOSE restore
Manually sync this commit that was missed.

6ae9155
2024-11-11 15:13:24 -05:00
Jean Boussier
1b6a26e23a [ruby/json] JSON.load_file: explictly load the file as UTF-8
Fix: https://github.com/ruby/json/issues/697

This way even if `Encoding.default_external` is set to a weird value
the document will be parsed just fine.

3a8505a8fa
2024-11-11 09:40:11 +09:00
Nobuyoshi Nakada
8254f6492c [ruby/json] Categorize deprecated warning
1acce7aceb
2024-11-06 23:31:30 +01:00
Jean Boussier
4120f2babd [ruby/json] Appease ruby/ruby CI
bc2c970ce4
2024-11-05 18:00:36 +01:00
Jean Boussier
633f65e471 [ruby/json] Strip whitespaces
e85107197b
2024-11-05 18:00:36 +01:00
Jean Boussier
ca8f21ace8 [ruby/json] Resync 2024-11-05 18:00:36 +01:00
Benoit Daloze
e8522f06b5 [ruby/json] Re-enable passing test
82d21f01c5
2024-11-05 18:00:36 +01:00