archive/ruby
1
0
Fork 0
mirror of https://github.com/ruby/ruby.git synced 2025-08-25 14:05:02 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 8
91898 commits 30 branches 1174 tags 514 MiB
Commit graph

20692 commits

Author SHA1 Message Date
Peter Zhu
c7708d22c3 Improve coverage of TestProc#test_hash_uniqueness 2024-11-01 10:49:50 -04:00
Peter Zhu
813286762c Add TestProc#test_hash_equal 2024-11-01 10:49:50 -04:00
Peter Zhu
53b3fac6d2 Rename test_hash to test_hash_uniqueness 2024-11-01 10:49:50 -04:00
Peter Zhu
29c480dd6f [Bug #20853] Fix Proc#hash to not change after compaction 
The hash value of a Proc must remain constant after a compaction, otherwise
it may not work as the key in a hash table.
 2024-11-01 10:49:50 -04:00
Jean Boussier
ef5565f5d1 JSON.generate: call to_json on String subclasses 
Fix: https://github.com/ruby/json/issues/667

This is yet another behavior on which the various implementations
differed, but the C implementation used to call `to_json` on String
subclasses used as keys.

This was optimized out in e125072130229e54a651f7b11d7d5a782ae7fb65
but there is an Active Support test case for it, so it's best to
make all 3 implementation respect this behavior.
 2024-11-01 13:04:24 +09:00
Jean Boussier
3782600f0f [ruby/json] Emit warnings when dumping binary strings 
Because of it's Ruby 1.8 heritage, the C extension doesn't care
much about strings encoding. We should get stricter over time.

42402fc13f
 2024-11-01 13:04:24 +09:00
Jean Boussier
f2b8829df0 Deprecate unsafe default options of JSON.load 
[Feature #19528]

Ref: https://bugs.ruby-lang.org/issues/19528

`load` is understood as the default method for serializer kind of libraries, and
the default options of `JSON.load` has caused many security vulnerabilities over the
years.

The plan is to do like YAML/Psych, deprecate these default options and direct
users toward using `JSON.unsafe_load` so at least it's obvious it should be
used against untrusted data.
 2024-11-01 13:04:24 +09:00
Jean Boussier
cc2e67a138 Elide Generator::State allocation until a to_json method has to be called 
Fix: https://github.com/ruby/json/issues/655

For very small documents, the biggest performance gap with alternatives is
that the API impose that we allocate the `State` object. In a real world app
this doesn't make much of a difference, but when running in a micro-benchmark
this doubles the allocations, causing twice the amount of GC runs, making us
look bad.

However, unless we have to call a `to_json` method, the `State` object isn't
visible, so with some refactoring, we can elude that allocation entirely.

Instead we allocate the State internal struct on the stack, and if we need
to call a `to_json` method, we allocate the `State` and spill the struct on
the heap.

As a result, `JSON.generate` is now as fast as re-using a `State` instance,
as long as only primitives are generated.

Before:
```
== Encoding small mixed (34 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   598.654k i/100ms
                json   400.542k i/100ms
                  oj   533.353k i/100ms
Calculating -------------------------------------
        json (reuse)      6.371M (± 8.6%) i/s  (156.96 ns/i) -     31.729M in   5.059195s
                json      4.120M (± 6.6%) i/s  (242.72 ns/i) -     20.828M in   5.090549s
                  oj      5.622M (± 6.4%) i/s  (177.86 ns/i) -     28.268M in   5.061473s

Comparison:
        json (reuse):  6371126.6 i/s
                  oj:  5622452.0 i/s - same-ish: difference falls within error
                json:  4119991.1 i/s - 1.55x  slower

== Encoding small nested array (121 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   248.125k i/100ms
                json   215.255k i/100ms
                  oj   217.531k i/100ms
Calculating -------------------------------------
        json (reuse)      2.628M (± 6.1%) i/s  (380.55 ns/i) -     13.151M in   5.030281s
                json      2.185M (± 6.7%) i/s  (457.74 ns/i) -     10.978M in   5.057655s
                  oj      2.217M (± 6.7%) i/s  (451.10 ns/i) -     11.094M in   5.044844s

Comparison:
        json (reuse):  2627799.4 i/s
                  oj:  2216824.8 i/s - 1.19x  slower
                json:  2184669.5 i/s - 1.20x  slower

== Encoding small hash (65 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   641.334k i/100ms
                json   322.745k i/100ms
                  oj   642.450k i/100ms
Calculating -------------------------------------
        json (reuse)      7.133M (± 6.5%) i/s  (140.19 ns/i) -     35.915M in   5.068201s
                json      4.615M (± 7.0%) i/s  (216.70 ns/i) -     22.915M in   5.003718s
                  oj      6.912M (± 6.4%) i/s  (144.68 ns/i) -     34.692M in   5.047690s

Comparison:
        json (reuse):  7133123.3 i/s
                  oj:  6911977.1 i/s - same-ish: difference falls within error
                json:  4614696.6 i/s - 1.55x  slower
```

After:

```
== Encoding small mixed (34 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   572.751k i/100ms
                json   457.741k i/100ms
                  oj   512.247k i/100ms
Calculating -------------------------------------
        json (reuse)      6.324M (± 6.9%) i/s  (158.12 ns/i) -     31.501M in   5.023093s
                json      6.263M (± 6.9%) i/s  (159.66 ns/i) -     31.126M in   5.017086s
                  oj      5.569M (± 6.6%) i/s  (179.56 ns/i) -     27.661M in   5.003739s

Comparison:
        json (reuse):  6324183.5 i/s
                json:  6263204.9 i/s - same-ish: difference falls within error
                  oj:  5569049.2 i/s - same-ish: difference falls within error

== Encoding small nested array (121 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   258.505k i/100ms
                json   242.335k i/100ms
                  oj   220.678k i/100ms
Calculating -------------------------------------
        json (reuse)      2.589M (± 9.6%) i/s  (386.17 ns/i) -     12.925M in   5.071853s
                json      2.594M (± 6.6%) i/s  (385.46 ns/i) -     13.086M in   5.083035s
                  oj      2.250M (± 2.3%) i/s  (444.43 ns/i) -     11.255M in   5.004707s

Comparison:
        json (reuse):  2589499.6 i/s
                json:  2594321.0 i/s - same-ish: difference falls within error
                  oj:  2250064.0 i/s - 1.15x  slower

== Encoding small hash (65 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
        json (reuse)   656.373k i/100ms
                json   644.135k i/100ms
                  oj   650.283k i/100ms
Calculating -------------------------------------
        json (reuse)      7.202M (± 7.1%) i/s  (138.84 ns/i) -     36.101M in   5.051438s
                json      7.278M (± 1.7%) i/s  (137.40 ns/i) -     36.716M in   5.046300s
                  oj      7.036M (± 1.7%) i/s  (142.12 ns/i) -     35.766M in   5.084729s

Comparison:
        json (reuse):  7202447.9 i/s
                json:  7277883.0 i/s - same-ish: difference falls within error
                  oj:  7036115.2 i/s - same-ish: difference falls within error

```
 2024-11-01 13:04:24 +09:00
Benoit Daloze
88b411464d [ruby/json] Skip test failing on JRuby 
0f0b16b3f5
 2024-11-01 13:04:24 +09:00
Benoit Daloze
eb19156a28 [ruby/json] Add test for parsing broken strings 
850bd077c4
 2024-11-01 13:04:24 +09:00
Jean Boussier
ebfa178b72 [ruby/json] Setup ruby_memcheck 
Hoping it might find the leak reported in https://github.com/ruby/json/issues/460

08635312e5
 2024-11-01 13:04:24 +09:00
Koichi Sasada
783dde2159 alias should not set defined_class for Modules 
`me->defined_class` should be 0 for method entries of
Modules.

This patch checks this condition
and fix https://github.com/ruby/ruby/pull/11965#issuecomment-2448291790
 2024-11-01 11:50:00 +09:00
Hiroshi SHIBATA
a8c32ace45
Removed accidentally sync file 2024-11-01 11:42:29 +09:00
Peter Zhu
843b4f49ee Fix assertion when envval of proc is Qundef 
The following code crashes with assertions enabled because envval could
be Qundef:

    {}.to_proc.dup
 2024-10-31 13:52:24 -04:00
Peter Zhu
66afde9cea Fix indentation in TestProc#test_hash [ci skip] 2024-10-31 10:58:30 -04:00
Kazuki Yamaguchi
27d77a9c73 [ruby/openssl] pkcs7: remove default cipher from PKCS7.encrypt 
Require that users explicitly specify the desired algorithm. In my
opinion, we are not in a position to specify the default cipher.

When OpenSSL::PKCS7.encrypt is given only two arguments, it uses
"RC2-40-CBC" as the symmetric cipher algorithm. 40-bit RC2 is a US
export-grade cipher and considered insecure.

Although this is technically a breaking change, the impact should be
minimal. Even when OpenSSL is compiled with RC2 support and the macro
OPENSSL_NO_RC2 is not defined, it will not actually work on modern
systems because RC2 is part of the legacy provider.

439f456bfa
 2024-10-31 08:31:16 +00:00
Samuel Williams
87fb44dff6
Introduce Fiber Scheduler blocking_region hook. (#11963) 2024-10-31 17:26:37 +13:00
Koichi Sasada
583587dfbf [ruby/error_highlight] use instance_method to get method object 
instead of `method()`.

There is a bug around `define_method`, so this patch is workaround.

4d04537f58
 2024-10-31 00:30:10 +00:00
Charles Oliver Nutter
22abcce704 Only check RubyVM on CRuby 
Blind use of the RubyVM constant here prevents this test from
running on non-CRuby. This patch guards it with RUBY_ENGINE ==
"ruby" to make sure that doesn't happen.
 2024-10-30 15:30:28 -05:00
David Rodríguez
1e1a37220b [rubygems/rubygems] Fix gem update --system leaving old default bundler executables around 
4b81add54c
 2024-10-30 14:22:08 +00:00
Nobuyoshi Nakada
a9f509e6c5 [ruby/error_highlight] Redefine in the module context 
0048bd0285
 2024-10-29 17:40:08 +00:00
Nobuyoshi Nakada
e22d5c2584
Fix method definition owners 2024-10-30 02:02:20 +09:00
Nobuyoshi Nakada
21b3dfa03b [ruby/error_highlight] Suppress smaller max_snippet_width warning 
9cd14c5b9a
 2024-10-29 07:56:32 +00:00
Jean Boussier
b094ee3f23
Handle all formatting configs potentially being nil. 
Fix: https://github.com/ruby/json/issues/653

I don't think this was really fully supported in the past, but
it kinda worked with some of the implementations.
 2024-10-29 13:25:01 +09:00
Jean Boussier
a5bd0c638a [ruby/json] Workaround rubygems $LOAD_PATH bug 
Ref: https://github.com/ruby/json/issues/647
Ref: https://github.com/rubygems/rubygems/pull/6490

Older rubygems are executing `extconf.rb` with a broken `$LOAD_PATH`
causing the `json` gem native extension to be loaded with the stdlib
version of the `.rb` files.

This fails with

```
json/common.rb:82:in `initialize': wrong number of arguments (given 1, expected 0) (ArgumentError)
```

Since this is just for `extconf.rb` we can probably just accept that
extra argument and ignore it.

The bug was fixed in rubygems 3.4.9 / 2023-03-20

1f5e849fe0
 2024-10-26 18:44:15 +09:00
Jean Boussier
3daf16e51f [ruby/json] Cleanup test_helper.rb 
49de571dd8
 2024-10-26 18:44:15 +09:00
Jean Boussier
7314275548 json_pure: fix ractor compatibility 
This actually never worked, because the test was always testing
the ext version from the stdlib, never the pure version nor the
current ext version.
 2024-10-26 18:44:15 +09:00
Jean Boussier
b1d417dc7b [ruby/json] Cleaner .encode / .force_encoding 
cecf04fdfc
 2024-10-26 18:44:15 +09:00
Jean Boussier
1045b9f820 [ruby/json] Modernize heredocs 
fb25e94aea
 2024-10-26 18:44:15 +09:00
Jean Boussier
bfdf02ea72 pretty_generate: don't apply object_nl / array_nl for empty containers 
Fix: https://github.com/ruby/json/issues/437

Before:

```json
{
  "foo": {
  },
  "bar": [
  ]
}
```

After:

```json
{
  "foo": {},
  "bar": []
}
```
 2024-10-26 18:44:15 +09:00
Jean Boussier
fc9f0cb8c5 [ruby/json] JSON.dump / String#to_json: raise on invalid encoding 
This regressed since 2.7.2.

35407d6635
 2024-10-26 18:44:15 +09:00
Benoit Daloze
2c6e3bc71e Raise the correct exception in fast_serialize_string 
* Related to https://github.com/ruby/json/issues/344
 2024-10-26 18:44:15 +09:00
Jean Boussier
70f554efb4 [ruby/json] raise_parse_error: avoid UB 
Fix: https://github.com/ruby/json/pull/625

Declaring the buffer in a sub block cause bugs on some compilers.

90967c9eb0
 2024-10-26 18:44:15 +09:00
Étienne Barrié
44aef5e852 [ruby/json] Drop compatibility for missing Array#permutation (Ruby <= 1.8.6) 
b02091ed44

Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
 2024-10-26 18:44:15 +09:00
Étienne Barrié
82f7550f65 Use frozen string literals 
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
 2024-10-26 18:44:15 +09:00
Étienne Barrié
11348c583f Use Encoding constants, String#b 
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
 2024-10-26 18:44:15 +09:00
Jean Boussier
18cc663aef [ruby/json] Add test coverage for JSON.load with a Proc 
Fix: https://github.com/ruby/json/issues/438

9dd89eaac8
 2024-10-26 18:44:15 +09:00
Jean Boussier
9045258c88 [ruby/json] Limit the size of ParserError exception messages 
Fix: https://github.com/ruby/json/issues/534

Only include up to 32 bytes of unparseable the source.

f44995cfb6
 2024-10-26 18:44:15 +09:00
NAITOH Jun
e61bb75a86 [ruby/strscan] [JRuby] Optimize scan(): Remove duplicate `if 
(restLen() < patternsize()) return context.nil;` checks in
`!headonly`.
(https://github.com/ruby/strscan/pull/110)

- before: #109

## Why?

d31274f41b/ext/jruby/org/jruby/ext/strscan/RubyStringScanner.java (L371-L373)

This means the following :

`if (str.size() - curr < pattern.size()) return context.nil;`

A similar check is made within `StringSupport#index()` within
`!headonly`.

be7815ec02/core/src/main/java/org/jruby/util/StringSupport.java (L1706-L1720)

```Java
    public static int index(ByteList source, ByteList other, int offset, Encoding enc) {
        int sourceLen = source.realSize();
        int sourceBegin = source.begin();
        int otherLen = other.realSize();

        if (otherLen == 0) return offset;
        if (sourceLen - offset < otherLen) return -1;
```

- source = `strBL`
- other = `patternBL`
- offset = `strBeg + curr`

This means the following :
`if (strBL.realSize() - (strBeg + curr) < patternBL.realSize()) return
-1;`

Both checks are the same.

## Benchmark

It shows String as a pattern is 2.40x faster than Regexp as a pattern.

```
$ benchmark-driver benchmark/check_until.yaml
Warming up --------------------------------------
              regexp     7.613M i/s -      7.593M times in 0.997350s (131.35ns/i)
          regexp_var     7.793M i/s -      7.772M times in 0.997364s (128.32ns/i)
              string    13.222M i/s -     13.199M times in 0.998297s (75.63ns/i)
          string_var    15.283M i/s -     15.216M times in 0.995667s (65.43ns/i)
Calculating -------------------------------------
              regexp    10.003M i/s -     22.840M times in 2.283361s (99.97ns/i)
          regexp_var     9.991M i/s -     23.378M times in 2.340019s (100.09ns/i)
              string    23.454M i/s -     39.666M times in 1.691221s (42.64ns/i)
          string_var    23.998M i/s -     45.848M times in 1.910447s (41.67ns/i)

Comparison:
          string_var:  23998466.3 i/s
              string:  23453777.5 i/s - 1.02x  slower
              regexp:  10002809.4 i/s - 2.40x  slower
          regexp_var:   9990580.1 i/s - 2.40x  slower
```

843e931d13
 2024-10-26 18:44:15 +09:00
Benoit Daloze
346085ea94 [ruby/fiddle] Fix Fiddle::Handle.new for a missing library in the 
FFI backend
(https://github.com/ruby/fiddle/pull/156)

* From https://github.com/ruby/reline/issues/766#issuecomment-2422135968

eea9fd0cc4
 2024-10-26 18:44:15 +09:00
tomoya ishida
f1e923631c [ruby/reline] Add completion_append_character test 
(https://github.com/ruby/reline/pull/773)

5f5a0aa78c
 2024-10-25 07:39:42 +00:00
tomoya ishida
979e447d7e [ruby/reline] nonprinting_start and nonprinting_end should be 
removed
(https://github.com/ruby/reline/pull/771)

e36441652a
 2024-10-24 16:36:39 +00:00
Wu
78378cae66 append completion_append_character only when continous completion is … (#764) 
* append completion_append_character only when continous completion is not possible

* refactoring

* remove debug puts
 2024-10-24 14:12:41 +00:00
Ellen Marie Dash
a24cb8ac43 [rubygems/rubygems] [SpecFetcher tests] Use >3 character long fake gem name. 
ce06b8f0d9
 2024-10-24 00:55:31 +00:00
Guilherme Carreiro
0b3d518e81 [ruby/error_highlight] Rename the ErrorHighlight::DefaultFormatter setting to max_snippet_width for clarity 
e13cbd4335
 2024-10-24 00:29:20 +00:00
Ellen Marie Dash
65fd8606a9 [rubygems/rubygems] Add another bail-early condition to suggest_gems_from_name(), with test. 
7bb7c0ac2d
 2024-10-23 20:03:14 +00:00
Guilherme Carreiro
e9ba6c2ea4 [ruby/error_highlight] Adjust truncation, add opt-out mechanism, rename methods, and prepare error highlighting to render on extremely small screens 
c565340958
 2024-10-23 00:58:50 +00:00
Guilherme Carreiro
e7c9dfb3e9 [ruby/error_highlight] Handle very long lines with errors in the middle of the line 
0657bc1afa
 2024-10-23 00:58:50 +00:00
Guilherme Carreiro
5aa8b9e3b5 [ruby/error_highlight] Handle very long lines 
383490a4b4
 2024-10-23 00:58:50 +00:00
Wu
e288604eb3 [ruby/reline] Use IO's encoding instead of Encoding.default_external 
(https://github.com/ruby/reline/pull/765)

* use IO's encoding

* refactoring

* remove unused encoding params

* (for retriggering CI) remove unused encoding params

f09772adab
 2024-10-22 14:43:18 +00:00