Commit graph

48 commits

Author SHA1 Message Date
Jean Boussier
1510d72bec [ruby/json] Fix generate(script_safe: true) to not confuse unrelated characters
Fix: https://github.com/ruby/json/issues/715

The first byte check was missing.

93a7f8717d
2024-12-05 09:16:22 +01:00
Jean Boussier
693a793521 JSON::GeneratorError expose invalid object
Fix: https://github.com/ruby/json/issues/710

Makes it easier to debug why a given tree of objects can't
be dumped as JSON.

Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com>
2024-11-26 15:11:05 +09:00
Jean Boussier
ca8f21ace8 [ruby/json] Resync 2024-11-05 18:00:36 +01:00
Jean Boussier
f664e7eaab [ruby/json] Add tests for the behavior of JSON.generate with base types subclasses
Ref: https://github.com/ruby/json/pull/674
Ref: https://github.com/ruby/json/pull/668

The behavior on such case it quite unclear, the goal here is to
figure out whatever was the behavior on Cext version of `json 2.7.0`
and get all implementations to converge.

We can then decide to make them all behave differently if we so wish.

614921dcef
2024-11-05 18:00:36 +01:00
Jean Boussier
ef5565f5d1 JSON.generate: call to_json on String subclasses
Fix: https://github.com/ruby/json/issues/667

This is yet another behavior on which the various implementations
differed, but the C implementation used to call `to_json` on String
subclasses used as keys.

This was optimized out in e125072130229e54a651f7b11d7d5a782ae7fb65
but there is an Active Support test case for it, so it's best to
make all 3 implementation respect this behavior.
2024-11-01 13:04:24 +09:00
Jean Boussier
3782600f0f [ruby/json] Emit warnings when dumping binary strings
Because of it's Ruby 1.8 heritage, the C extension doesn't care
much about strings encoding. We should get stricter over time.

42402fc13f
2024-11-01 13:04:24 +09:00
Jean Boussier
cc2e67a138 Elide Generator::State allocation until a to_json method has to be called
Fix: https://github.com/ruby/json/issues/655

For very small documents, the biggest performance gap with alternatives is
that the API impose that we allocate the `State` object. In a real world app
this doesn't make much of a difference, but when running in a micro-benchmark
this doubles the allocations, causing twice the amount of GC runs, making us
look bad.

However, unless we have to call a `to_json` method, the `State` object isn't
visible, so with some refactoring, we can elude that allocation entirely.

Instead we allocate the State internal struct on the stack, and if we need
to call a `to_json` method, we allocate the `State` and spill the struct on
the heap.

As a result, `JSON.generate` is now as fast as re-using a `State` instance,
as long as only primitives are generated.

Before:
```
== Encoding small mixed (34 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   598.654k i/100ms
                json   400.542k i/100ms
                  oj   533.353k i/100ms
Calculating -------------------------------------
        json (reuse)      6.371M (± 8.6%) i/s  (156.96 ns/i) -     31.729M in   5.059195s
                json      4.120M (± 6.6%) i/s  (242.72 ns/i) -     20.828M in   5.090549s
                  oj      5.622M (± 6.4%) i/s  (177.86 ns/i) -     28.268M in   5.061473s

Comparison:
        json (reuse):  6371126.6 i/s
                  oj:  5622452.0 i/s - same-ish: difference falls within error
                json:  4119991.1 i/s - 1.55x  slower

== Encoding small nested array (121 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   248.125k i/100ms
                json   215.255k i/100ms
                  oj   217.531k i/100ms
Calculating -------------------------------------
        json (reuse)      2.628M (± 6.1%) i/s  (380.55 ns/i) -     13.151M in   5.030281s
                json      2.185M (± 6.7%) i/s  (457.74 ns/i) -     10.978M in   5.057655s
                  oj      2.217M (± 6.7%) i/s  (451.10 ns/i) -     11.094M in   5.044844s

Comparison:
        json (reuse):  2627799.4 i/s
                  oj:  2216824.8 i/s - 1.19x  slower
                json:  2184669.5 i/s - 1.20x  slower

== Encoding small hash (65 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   641.334k i/100ms
                json   322.745k i/100ms
                  oj   642.450k i/100ms
Calculating -------------------------------------
        json (reuse)      7.133M (± 6.5%) i/s  (140.19 ns/i) -     35.915M in   5.068201s
                json      4.615M (± 7.0%) i/s  (216.70 ns/i) -     22.915M in   5.003718s
                  oj      6.912M (± 6.4%) i/s  (144.68 ns/i) -     34.692M in   5.047690s

Comparison:
        json (reuse):  7133123.3 i/s
                  oj:  6911977.1 i/s - same-ish: difference falls within error
                json:  4614696.6 i/s - 1.55x  slower
```

After:

```
== Encoding small mixed (34 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   572.751k i/100ms
                json   457.741k i/100ms
                  oj   512.247k i/100ms
Calculating -------------------------------------
        json (reuse)      6.324M (± 6.9%) i/s  (158.12 ns/i) -     31.501M in   5.023093s
                json      6.263M (± 6.9%) i/s  (159.66 ns/i) -     31.126M in   5.017086s
                  oj      5.569M (± 6.6%) i/s  (179.56 ns/i) -     27.661M in   5.003739s

Comparison:
        json (reuse):  6324183.5 i/s
                json:  6263204.9 i/s - same-ish: difference falls within error
                  oj:  5569049.2 i/s - same-ish: difference falls within error

== Encoding small nested array (121 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   258.505k i/100ms
                json   242.335k i/100ms
                  oj   220.678k i/100ms
Calculating -------------------------------------
        json (reuse)      2.589M (± 9.6%) i/s  (386.17 ns/i) -     12.925M in   5.071853s
                json      2.594M (± 6.6%) i/s  (385.46 ns/i) -     13.086M in   5.083035s
                  oj      2.250M (± 2.3%) i/s  (444.43 ns/i) -     11.255M in   5.004707s

Comparison:
        json (reuse):  2589499.6 i/s
                json:  2594321.0 i/s - same-ish: difference falls within error
                  oj:  2250064.0 i/s - 1.15x  slower

== Encoding small hash (65 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   656.373k i/100ms
                json   644.135k i/100ms
                  oj   650.283k i/100ms
Calculating -------------------------------------
        json (reuse)      7.202M (± 7.1%) i/s  (138.84 ns/i) -     36.101M in   5.051438s
                json      7.278M (± 1.7%) i/s  (137.40 ns/i) -     36.716M in   5.046300s
                  oj      7.036M (± 1.7%) i/s  (142.12 ns/i) -     35.766M in   5.084729s

Comparison:
        json (reuse):  7202447.9 i/s
                json:  7277883.0 i/s - same-ish: difference falls within error
                  oj:  7036115.2 i/s - same-ish: difference falls within error

```
2024-11-01 13:04:24 +09:00
Jean Boussier
b094ee3f23
Handle all formatting configs potentially being nil.
Fix: https://github.com/ruby/json/issues/653

I don't think this was really fully supported in the past, but
it kinda worked with some of the implementations.
2024-10-29 13:25:01 +09:00
Jean Boussier
a5bd0c638a [ruby/json] Workaround rubygems $LOAD_PATH bug
Ref: https://github.com/ruby/json/issues/647
Ref: https://github.com/rubygems/rubygems/pull/6490

Older rubygems are executing `extconf.rb` with a broken `$LOAD_PATH`
causing the `json` gem native extension to be loaded with the stdlib
version of the `.rb` files.

This fails with

```
json/common.rb:82:in `initialize': wrong number of arguments (given 1, expected 0) (ArgumentError)
```

Since this is just for `extconf.rb` we can probably just accept that
extra argument and ignore it.

The bug was fixed in rubygems 3.4.9 / 2023-03-20

1f5e849fe0
2024-10-26 18:44:15 +09:00
Jean Boussier
1045b9f820 [ruby/json] Modernize heredocs
fb25e94aea
2024-10-26 18:44:15 +09:00
Jean Boussier
bfdf02ea72 pretty_generate: don't apply object_nl / array_nl for empty containers
Fix: https://github.com/ruby/json/issues/437

Before:

```json
{
  "foo": {
  },
  "bar": [
  ]
}
```

After:

```json
{
  "foo": {},
  "bar": []
}
```
2024-10-26 18:44:15 +09:00
Jean Boussier
fc9f0cb8c5 [ruby/json] JSON.dump / String#to_json: raise on invalid encoding
This regressed since 2.7.2.

35407d6635
2024-10-26 18:44:15 +09:00
Benoit Daloze
2c6e3bc71e Raise the correct exception in fast_serialize_string
* Related to https://github.com/ruby/json/issues/344
2024-10-26 18:44:15 +09:00
Étienne Barrié
82f7550f65 Use frozen string literals
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2024-10-26 18:44:15 +09:00
Étienne Barrié
11348c583f Use Encoding constants, String#b
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2024-10-26 18:44:15 +09:00
Jean Boussier
8feed977a0 [ruby/json] Assume Encoding is defined
8713aa4812
2024-10-18 11:30:55 +09:00
Hiroshi SHIBATA
8af6606b22
Restore missing test-case from c5a6d80427
Co-authored-by: "Jean Boussier" <byroot@ruby-lang.org>
2024-10-16 11:24:25 +09:00
Hiroshi SHIBATA
fe33475605 Removed trailing space 2024-10-08 14:10:05 +09:00
Jean Boussier
718c4f7e1e JSONPure: String#to_json should raise on invalid encoding
Fix: #344

This matches the ext behavior.
2024-10-08 14:10:05 +09:00
Jean Boussier
8fdd3d0ed6 JSON::Pure fix strict mode
Followup: https://github.com/flori/json/pull/519
Fix: https://github.com/flori/json/issues/584
2024-10-08 14:10:05 +09:00
Étienne Barrié
f4883e7904 [flori/json] Use the compiled extension in test
148afef84c
2024-09-03 11:51:51 +09:00
Hiroshi SHIBATA
abc3d124f7 [flori/json] The modern Ruby uses utf-8 encodings by default
11b31210ac
2023-12-05 12:04:10 +09:00
Takashi Kokubun
e6b35e8a6d [flori/json] Overload kwargs in JSON.dump
936f280f9f
2023-12-05 12:04:08 +09:00
Jean Boussier
a22ed89438 [flori/json] JSON.dump: handle unenclosed hashes regression
Fix: https://github.com/flori/json/issues/553

We can never add keyword arguments to `dump` otherwise
existing code using unenclosed hash will break.

8e0076a3f2
2023-12-05 12:04:07 +09:00
Hiroshi SHIBATA
86045fca24
Manually merged from flori/json
> https://github.com/flori/json/pull/525
  > Rename escape_slash in script_safe and also escape E+2028 and E+2029

  Co-authored-by: Jean Boussier <jean.boussier@gmail.com>

  > https://github.com/flori/json/pull/454
  > Remove unnecessary initialization of create_id in JSON.parse()

  Co-authored-by: Watson <watson1978@gmail.com>
2023-12-01 16:47:06 +09:00
Jean Boussier
0dfeb17296
Rename escape_slash in script_safe and also escape E+2028 and E+2029
It is rather common to directly interpolate JSON string inside
<script> tags in HTML as to provide configuration or parameters to a
script.

However this may lead to XSS vulnerabilities, to prevent that 3
characters need to be escaped:

  - `/` (forward slash)
  - `U+2028` (LINE SEPARATOR)
  - `U+2029` (PARAGRAPH SEPARATOR)

The forward slash need to be escaped to prevent closing the script
tag early, and the other two are valid JSON but invalid Javascript
and can be used to break JS parsing.

Given that the intent of escaping forward slash is the same than escaping
U+2028 and U+2029, I chos to rename and repurpose the existing `escape_slash`
option.
2023-12-01 16:47:06 +09:00
Hiroshi SHIBATA
e42df781d9
[flori/json] define_method is also private at Ruby 2.3 and 2.4
3804f38bf4
2023-10-11 15:45:17 +09:00
Hiroshi SHIBATA
1b19e9198d
[flori/json] remove_method of Module is private at Ruby 2.3 and 2.4
6cbadf6b6e
2023-10-11 15:45:17 +09:00
Hiroshi SHIBATA
c75d54a36c
Fixup 1851824b1c 2023-09-13 16:23:18 +09:00
Hiroshi SHIBATA
1851824b1c [flori/json] alias_method is private on Ruby 2.3 and 2.4
573ef94bc5
2023-09-13 16:01:19 +09:00
Yusuke Endoh
77db0ca6c8 Prevent a warning: method redefined; discarding old included
```
[ 3463/25719] JSONGeneratorTest#test_string_ext_included_calls_super/home/chkbuild/chkbuild/tmp/build/20230905T063003Z/ruby/test/json/json_generator_test.rb:411: warning: method redefined; discarding old included
/home/chkbuild/chkbuild/tmp/build/20230905T063003Z/ruby/test/json/json_generator_test.rb:399: warning: previous definition of included was here
```

20230905T063003Z.log.html.gz
2023-09-05 17:29:53 +09:00
Peter Zhu
91de37c23e Remove --disable-gems in assert_in_out_err
assert_in_out_err adds --disable=gems so we don't need to add
--disable-gems in the args list.
2023-08-28 15:05:19 -04:00
Takashi Kokubun
96a809f621 Use require_relative in JSON tests
to prevent them from conflicting with yarp/test_helper
2023-08-25 15:32:14 -07:00
Nobuyoshi Nakada
9f51810f34 [flori/json] Skip unsupported test on JRuby
7138bf32c7
2023-07-18 12:25:54 +09:00
Ufuk Kayserilioglu
12dfd9d1c9
[flori/json] Call super in included hook
The C extension defines an `included` hook for the
`JSON::Ext::Generator::GeneratorMethods::String` module but neglects to
call `super` in the hook. This can break the functionality of various
other code that rely on the fact that `included` on `Module` will always
be called.

cd8bbe56a3
2023-05-24 09:37:30 +09:00
Hiroshi SHIBATA
edb76e8765 Prepare to release json-2.5.0 2020-12-22 19:44:27 +09:00
Kenta Murata
98cc15ed1e
[json] Stop using prototype objects 2020-12-21 22:10:33 +09:00
Chris Seaton
451836f582
Fix an issue with generate_pretty and empty objects in the Ruby and Java implementations 2020-10-20 21:46:54 +09:00
Kazuhiro NISHIYAMA
ac414139ec
Remove unnecessary executable bit [ci skip] 2020-09-28 23:07:43 +09:00
Karol Bucek
0089854fc5 [test] properly 'skip' test on JRuby
an early return still caused ensure to execute,
setting JSON constant to `nil` for later tests!
2020-09-25 17:28:42 +09:00
Jean Boussier
e1659af372 Add an option to escape forward slash character
Squashed commit of the following:

commit 26d181059989279a79c433cedcd893b4f52e42ee
Author: Francois Chagnon <francois.chagnon@jadedpixel.com>
Date:   Tue Sep 15 21:17:34 2015 +0000

    add config options for escape_slash

commit fa282334051b16df91ca097dd7304b46f3bc7719
Author: Francois Chagnon <francois.chagnon@jadedpixel.com>
Date:   Mon Feb 9 21:09:33 2015 +0000

    add forward slash to escape character
2020-09-25 17:28:42 +09:00
Hiroshi SHIBATA
53e8589c69
Import json-2.3.0 from flori/json 2019-12-12 09:14:09 +09:00
Nobuyoshi Nakada
f83bebdf7a
Skip useless test
`JSONGeneratorTest#test_remove_const_seg` is meaningful only for
the extension library version, but nonsense for pure ruby version.
2019-11-29 11:21:38 +09:00
Aaron Patterson
9026e12f93
Look up constant instead of caching in a global
The global can go bad if the compactor runs, so we need to look up the
constant instead of caching it in a global.
2019-10-17 13:30:09 -07:00
Nobuyoshi Nakada
2003755a2c
[flori/json] Fixed unexpected illegal/malformed utf-8 error
flori/json@c34d01ff6a does not
consider US-ASCII compatible but non-UTF-8 encodings, and causes
an error in RDoc tests.

4f471bf590
2019-10-14 19:54:48 +09:00
hsbt
83735ba29a Merge json-2.0.4.
* https://github.com/flori/json/releases/tag/v2.0.4
  * 09fabeb03e/CHANGES.md (2017-03-23-204)

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58323 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-04-12 00:21:18 +00:00
nobu
7f6410f02d json_generator_test.rb: no Bignum
* test/json/json_generator_test.rb (test_broken_bignum): get rid
  of use of Bignum, obsolete name.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56126 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-09-09 09:27:38 +00:00
hsbt
a7b5d45466 * test/lib/test/unit.rb: added test files with _test suffix for json
upstream.
* test/json: merge original test files from json upstream.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55667 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-07-13 13:27:07 +00:00
Renamed from test/json/test_json_generator.rb (Browse further)