Commit graph

8776 commits

Author SHA1 Message Date
Kenta Murata
75f552e973
[ruby/bigdecimal] Fix the precision of the adjusted quotient
8dc8cd339d
2021-12-24 02:28:56 +09:00
Kenta Murata
7b2cfce543
[ruby/bigdecimal] Let BigDecimal_DoDivmod use the same precision calculation as BigDecimal_divide
11cb2c8840
2021-12-24 02:28:55 +09:00
Kenta Murata
e1265c8198
[ruby/bigdecimal] Use larger precision in divide for irrational or recurring results
Just in case for irrational or recurring results, the precision of the
quotient is set to at least more than 2*Float::DIG plus alpha.

[Bug #13754] [Fix GH-94]

99442c75d3
2021-12-24 02:28:55 +09:00
Kenta Murata
8ee8ac6423
[ruby/bigdecimal] Fix trailing zeros handling in rb_uint64_convert_to_BigDecimal
Fix GH-192

eebc98b85a
2021-12-24 02:28:55 +09:00
BurdetteLamar
ed7f4c24d7
[ruby/bigdecimal] Respond to review
f528a0006e
2021-12-24 02:28:54 +09:00
BurdetteLamar
ed8ec5dcb3
[ruby/bigdecimal] Respond to review
4eadcdf0a6
2021-12-24 02:28:54 +09:00
BurdetteLamar
44acab76af
[ruby/bigdecimal] Respond to review
6d69422e37
2021-12-24 02:28:53 +09:00
BurdetteLamar
973e508472
[ruby/bigdecimal] Respond to review for #precision
681cd2d81d
2021-12-24 02:28:53 +09:00
BurdetteLamar
5888d3030d
[ruby/bigdecimal] Enhanced RDoc for bigdecimal.c
3a35f92f8b
2021-12-24 02:28:53 +09:00
BurdetteLamar
01b2ccaa95
[ruby/bigdecimal] Enhanced RDoc for bigdecimal.c
31a7a37426
2021-12-24 02:28:52 +09:00
Jean Boussier
ec478d947f
[ruby/bigdecimal] Fix negative Bignum conversion
Introduced in 4792a917d8

`rb_absint_size` return the number of bytes needed to fit
the absolute integer, but negative integers need the sign, so one more
bit, and potentially one more byte.

0f3d5d0eb7
2021-12-24 02:28:51 +09:00
Olle Jonsson
03507498b6
[ruby/bigdecimal] VpCheckException: improve grammar
I added a space before the parenthesis, too.

159af10b17
2021-12-24 02:28:47 +09:00
Nobuyoshi Nakada
40a1af6151
Install ruby/digest.h when from ext/digest 2021-12-23 20:12:45 +09:00
Kazuki Yamaguchi
ac757b218c [ruby/openssl] pkey: use EVP_PKEY_CTX_new_from_name() on OpenSSL 3.0
Replace EVP_PKEY_CTX_new_id() with the new EVP_PKEY_CTX_new_from_name()
which takes the algorithm name in a string instead of in an NID.

d6535d13d1
2021-12-20 23:42:04 +09:00
Kazuki Yamaguchi
61e426ae05 [ruby/openssl] pkey: assume a pkey always has public key components on OpenSSL 3.0
OpenSSL 3.0's EVP_PKEY_get0() returns NULL for provider-backed pkeys.
This causes segfault because it was supposed to never return NULL
before.

We can't check the existence of public key components in this way on
OpenSSL 3.0. Let's just skip it for now.

ccdb6f7bfa
2021-12-20 23:42:04 +09:00
Kazuki Yamaguchi
0b3482c0e7 [ruby/openssl] ssl: add constants for new SSL_OP_* flags
Add all SSL_OP_* constants defined in OpenSSL 3.0.0 which are not
specific to DTLS.

b1ee2f23b2
2021-12-20 23:42:03 +09:00
Kazuki Yamaguchi
b2fb503dab [ruby/openssl] engine: disable OpenSSL::Engine on OpenSSL 3.0
The entire ENGINE API is deprecated in OpenSSL 3.0 in favor of the new
"Provider" concept.

OpenSSL::Engine will not be defined when compiled with OpenSSL 3.0.
We would need a way to interact with providers from Ruby programs, but
since the concept is completely different from the ENGINE API, it will
not be through the current OpenSSL::Engine interface.

69a27d8de4
2021-12-20 23:42:03 +09:00
Kazuki Yamaguchi
c1a7c6df18 [ruby/openssl] hmac: fix wrong usage of EVP_DigestSignFinal()
According to the manpage, the "siglen" parameter must be initialized
beforehand.

6a60c7b2e7
2021-12-20 23:42:02 +09:00
Kazuki Yamaguchi
8ebf597885 [ruby/openssl] pkey: deprecate PKey#set_* methods
OpenSSL 3.0 made EVP_PKEY immutable. This means we can only have a const
pointer of the low level struct and the following methods can no longer
be provided when linked against OpenSSL 3.0:

 - OpenSSL::PKey::RSA#set_key
 - OpenSSL::PKey::RSA#set_factors
 - OpenSSL::PKey::RSA#set_crt_params
 - OpenSSL::PKey::DSA#set_pqg
 - OpenSSL::PKey::DSA#set_key
 - OpenSSL::PKey::DH#set_pqg
 - OpenSSL::PKey::DH#set_key
 - OpenSSL::PKey::EC#group=
 - OpenSSL::PKey::EC#private_key=
 - OpenSSL::PKey::EC#public_key=

There is no direct replacement for this functionality at the moment.
I plan to introduce a wrapper around EVP_PKEY_fromdata(), which takes
all key components at once to construct an EVP_PKEY.

6848d2d969
2021-12-20 23:42:02 +09:00
Kazuki Yamaguchi
b93ae54258 [ruby/openssl] pkey/ec: deprecate OpenSSL::PKey::EC#generate_key!
OpenSSL::PKey::EC#generate_key! will not work on OpenSSL 3.0 because
keys are made immutable. Users should use OpenSSL::PKey.generate_key
instead.

5e2e66cce8
2021-12-20 23:42:02 +09:00
Kazuki Yamaguchi
0d698be04f [ruby/openssl] pkey/dh: deprecate OpenSSL::PKey::DH#generate_key!
OpenSSL::PKey::DH#generate_key! will not work on OpenSSL 3.0 because
keys are made immutable. Users should use OpenSSL::PKey.generate_key
instead.

8ee6a582c7
2021-12-20 23:42:02 +09:00
Kazuki Yamaguchi
50b90c5fc3 [ruby/openssl] pkey/ec: avoid using EC#public_key= in EC#dh_compute_key
Similarly to DH#compute_key, work around it by constructing a
SubjectPublicKeyInfo. This should be considered as a temporary
implementation.

fc9aabc18d
2021-12-20 23:42:01 +09:00
Kazuki Yamaguchi
dc3f37c6cc [ruby/openssl] pkey/dh: avoid using DH#set_key in DH#compute_key
DH#set_key will not work on OpenSSL 3.0 because keys are immutable.
For now, let's reimplement DH#compute_key by manually constructing a
DER-encoded SubjectPublicKeyInfo structure and feeding it to
OpenSSL::PKey.read.

Eventually, we should implement a new method around EVP_PKEY_fromdata()
and use it instead.

46ca47060c
2021-12-20 23:42:01 +09:00
Kazuki Yamaguchi
df6589e418 [ruby/openssl] pkey: use EVP_PKEY_dup() if available
We can use it to implement OpenSSL::PKey::PKey#initialize_copy. This
should work on all key types, not just DH/DSA/EC/RSA types.

66cd8cbaaf
2021-12-20 23:42:01 +09:00
Kazuki Yamaguchi
c1a36ebfda [ruby/openssl] pkey: allocate EVP_PKEY on #initialize
Allocate an EVP_PKEY when the content is ready: when #initialize
or #initialize_copy is called, rather than when a T_DATA is allocated.
This is more natural because the lower level API has been deprecated
and an EVP_PKEY is becoming the minimum unit of handling keys.

74f6c61756
2021-12-20 23:42:01 +09:00
Kazuki Yamaguchi
02a58fbfd1 [ruby/openssl] pkey: do not check NULL argument in ossl_pkey_new()
Passing NULL to ossl_pkey_new() makes no sense in the first place, and
in fact it is ensured not to be NULL in all cases.

316cb2a41f
2021-12-20 23:42:00 +09:00
Kazuki Yamaguchi
6ef0f272eb [ruby/openssl] pkey: use OSSL_DECODER to load encrypted PEM on OpenSSL 3.0
OpenSSL 3.0 has rewritten routines to load pkeys (PEM_read_bio_* and
d2i_* functions) around the newly introduced OSSL_DECODER API.

This comes with a slight behavior change. They now decrypt and parse
each encountered PEM block, then check the kind of the block. This used
to be the reverse: they checked the PEM header to see the kind, and then
decrypted the content. This means that the password callback may now be
called repeatedly.

Let's use the OSSL_DECODER API directly on OpenSSL 3.0 so that the
return value from the password callback will be reused automatically.

a84ea531bb
2021-12-20 23:42:00 +09:00
Hiroshi SHIBATA
87b968c903
[ruby/psych] psych depends stringio only CRuby
e7bbf26cb2
2021-12-20 18:26:58 +09:00
Hiroshi SHIBATA
706c7a27fa [ruby/psych] Bump version to 4.0.3
75ab76e788
2021-12-20 18:25:50 +09:00
Nobuyoshi Nakada
54f0e63a8c Remove NODE_DASGN_CURR [Feature #18406]
This `NODE` type was used in pre-YARV implementation, to improve
the performance of assignment to dynamic local variable defined at
the innermost scope.  It has no longer any actual difference with
`NODE_DASGN`, except for the node dump.
2021-12-13 12:53:03 +09:00
Nobuyoshi Nakada
e4b35b158a [ruby/cgi] Check integer overflow in long range
https://hackerone.com/reports/1328463

ccaf6027e0
2021-12-12 13:05:15 +09:00
Hiroshi SHIBATA
c41751f10e
[ruby/io-wait] Bump up 0.2.1
c97ab9a212
2021-12-09 19:28:54 +09:00
Yusuke Endoh
17e7219679 ext/ripper/lib/ripper/lexer.rb: Do not deprecate Ripper::Lexer::State#[]
The old code of IRB still uses this method. The warning is noisy on
rails console.
In principle, Ruby 3.1 deprecates nothing, so let's avoid the
deprecation for the while.
I think It is not so hard to continue to maintain it as it is a trivial
shim.

https://github.com/ruby/ruby/pull/5093
2021-12-09 00:30:17 +09:00
Peter Zhu
9f0c6f20c5 [Bug #18382] Fix crash in compaction for ObjectSpace.trace_object_allocations
ObjectSpace.trace_object_allocations can crash when auto-compaction is
enabled.
2021-12-02 13:06:44 -05:00
Nobuyoshi Nakada
524a808d23
Define Ripper::Lexer::Elem#to_s
Alias `#inspect` as `#to_s` also in the new `Ripper::Lexer::Elem`
class, so that `puts Ripper::Lexer.new(code).scan` shows the
attributes.
2021-12-02 18:29:45 +09:00
schneems
8944009be7 Deprecate Lexer::Elem#[] and Lexer::State#[]
Discussed in https://github.com/ruby/ruby/pull/5093#issuecomment-964426481. 

> it would be enough to mimic only [] for almost all cases

This adds back the `Lexer::Elem#[]` and `Lexer::State#[]` and adds deprecation warnings for them.
2021-12-02 15:55:42 +09:00
schneems
3685b5af95 Only iterate Lexer heredoc arrays
The last element in the `@buf` may be either an array or an `Elem`. In the case it is an `Elem` we iterate over every element, when we do not need to. This check guards that case by ensuring that we only iterate over an array of elements.
2021-12-02 15:55:42 +09:00
schneems
3f74eaa7a8 ~1.10x faster Change Ripper.lex structs to classes
## Concept

I am proposing we replace the Struct implementation of data structures inside of ripper with real classes.

This will improve performance and the implementation is not meaningfully more complicated.

## Example

Struct versus class comparison:

```ruby
Elem = Struct.new(:pos, :event, :tok, :state, :message) do
  def initialize(pos, event, tok, state, message = nil)
    super(pos, event, tok, State.new(state), message)
  end

  # ...

  def to_a
    a = super
    a.pop unless a.empty?
    a
  end
end

class ElemClass
  attr_accessor :pos, :event, :tok, :state, :message

  def initialize(pos, event, tok, state, message = nil)
    @pos = pos
    @event = event
    @tok = tok
    @state = State.new(state)
    @message = message
  end

  def to_a
    if @message
      [@pos, @event, @tok, @state, @message]
    else
      [@pos, @event, @tok, @state]
    end
  end
end

# stub state class creation for now
class State; def initialize(val); end; end
```

## MicroBenchmark creation

```ruby
require 'benchmark/ips'
require 'ripper'

pos = [1, 2]
event = :on_nl
tok = "\n".freeze
state = Ripper::EXPR_BEG

Benchmark.ips do |x|
  x.report("struct") { Elem.new(pos, event, tok, state) }
  x.report("class ") { ElemClass.new(pos, event, tok, state) }
  x.compare!
end; nil
```

Gives ~1.2x faster creation:

```
Warming up --------------------------------------
              struct   263.983k i/100ms
              class    303.367k i/100ms
Calculating -------------------------------------
              struct      2.638M (± 5.9%) i/s -     13.199M in   5.023460s
              class       3.171M (± 4.6%) i/s -     16.078M in   5.082369s

Comparison:
              class :  3170690.2 i/s
              struct:  2638493.5 i/s - 1.20x  (± 0.00) slower
```

## MicroBenchmark `to_a` (Called by Ripper.lex for every element)

```ruby
require 'benchmark/ips'
require 'ripper'

pos = [1, 2]
event = :on_nl
tok = "\n".freeze
state = Ripper::EXPR_BEG

struct =  Elem.new(pos, event, tok, state)
from_class = ElemClass.new(pos, event, tok, state)

Benchmark.ips do |x|
  x.report("struct") { struct.to_a }
  x.report("class ") { from_class.to_a }
  x.compare!
end; nil
```

Gives 1.46x faster `to_a`:

```
Warming up --------------------------------------
              struct   612.094k i/100ms
              class    893.233k i/100ms
Calculating -------------------------------------
              struct      6.121M (± 5.4%) i/s -     30.605M in   5.015851s
              class       8.931M (± 7.9%) i/s -     44.662M in   5.039733s

Comparison:
              class :  8930619.0 i/s
              struct:  6121358.9 i/s - 1.46x  (± 0.00) slower
```

## MicroBenchmark data access

```ruby
require 'benchmark/ips'
require 'ripper'

pos = [1, 2]
event = :on_nl
tok = "\n".freeze
state = Ripper::EXPR_BEG

struct =  Elem.new(pos, event, tok, state)
from_class = ElemClass.new(pos, event, tok, state)

Benchmark.ips do |x|
  x.report("struct") { struct.pos[1] }
  x.report("class ") { from_class.pos[1] }
  x.compare!
end; nil
```

Gives ~1.17x faster data access:

```
Warming up --------------------------------------
              struct     1.694M i/100ms
              class      1.868M i/100ms
Calculating -------------------------------------
              struct     16.149M (± 6.8%) i/s -     81.318M in   5.060633s
              class      18.886M (± 2.9%) i/s -     95.262M in   5.048359s

Comparison:
              class : 18885669.6 i/s
              struct: 16149255.8 i/s - 1.17x  (± 0.00) slower
```

## Full benchmark integration of this inside of Ripper.lex

Inside of this repo with this commit

```
$ cd ext/ripper
$ make
$ cat test.rb
file = File.join(__dir__, "../../array.rb")
source = File.read(file)

bench = Benchmark.measure do
  10_000.times.each do
    Ripper.lex(source)
  end
end

puts bench
```

Then execute with and without this change 50 times:

```
rm new.txt
rm old.txt
for i in {0..50}
do
  `ruby -Ilib -rripper -rbenchmark ./test.rb >> new.txt`
  `ruby -rripper -rbenchmark ./test.rb >> old.txt`
done
```

I used derailed benchmarks internals to compare the results:

```
dir = Pathname(".")
branch_info = {}
branch_info["old"]  = { desc: "Struct lex", time: Time.now, file: dir.join("old.txt"), name: "old" }
branch_info["new"]  = { desc: "Class lex", time: Time.now, file: dir.join("new.txt"), name: "new" }
stats = DerailedBenchmarks::StatsFromDir.new(branch_info)
stats.call.banner
```

Which gave us:

```
❤️ ❤️ ❤️  (Statistically Significant) ❤️ ❤️ ❤️

[new] (3.3139 seconds) "Class lex" ref: "new"
  FASTER 🚀🚀🚀 by:
    1.1046x [older/newer]
    9.4700% [(older - newer) / older * 100]
[old] (3.6606 seconds) "Struct lex" ref: "old"

Iterations per sample:
Samples: 51

Test type: Kolmogorov Smirnov
Confidence level: 99.0 %
Is significant? (max > critical): true
D critical: 0.30049534876137013
D max: 0.9607843137254902

Histograms (time ranges are in seconds):

   [new] description:                                        [old] description:
     "Class lex"                                               "Struct lex"
              ┌                                        ┐                ┌                                        ┐
   [3.0, 3.3) ┤▇ 1                                           [3.0, 3.3) ┤ 0
   [3.3, 3.6) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 47       [3.3, 3.6) ┤ 0
   [3.5, 3.8) ┤▇▇ 2                                          [3.5, 3.8) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 46
   [3.8, 4.1) ┤▇ 1                                           [3.8, 4.1) ┤▇▇▇ 4
   [4.0, 4.3) ┤ 0                                            [4.0, 4.3) ┤ 0
   [4.3, 4.6) ┤ 0                                            [4.3, 4.6) ┤▇ 1
              └                                        ┘                └                                        ┘
                         # of runs in range                                        # of runs in range
```

To sum this up, the "new" version of this code (using real classes instead of structs) is 10% faster across 50 runs with a statistical significance confidence level of 99%. Histograms are for visual checksum.
2021-12-02 15:55:42 +09:00
Nobuyoshi Nakada
ae95939784
[ruby/win32ole] Fix typos [ci skip]
8d46bd0c93
2021-11-25 23:53:15 +09:00
Hiroshi SHIBATA
24f8f20cc7
[ruby/win32ole] LICENSE
62fd78078b
2021-11-25 23:53:14 +09:00
Nobuyoshi Nakada
d896746d69
Keep the generated source files when clean [Bug #18363] 2021-11-25 19:16:39 +09:00
Peter Zhu
c51b92c18d [ruby/zlib] [Bug #18358] Fix crash in zlib when in progress
When Zlib::Inflate#inflate or Zlib::Deflate#deflate is called
recursively inside the block, a crash can occur because of an
use-after-free bug.

50fb8a0338
2021-11-24 23:01:41 +09:00
Nobuyoshi Nakada
da34f31ad0 [ruby/cgi] Fix integer overflow
Make use of the check in rb_alloc_tmp_buffer2.

https://hackerone.com/reports/1328463

c728632c1c
2021-11-24 19:58:59 +09:00
Alan Wu
f5d2041138
Avoid assert failure when NULL EC is expected
After 5680c38c75, postponed job APIs now
expect to be called on native threads not managed by Ruby and handles
getting a NULL execution context. However, in debug builds the change
runs into an assertion failure with GET_EC() which asserts that EC is
non-NULL. Avoid the assertion failure by passing `false` for `expect_ec`
instead as the intention is to handle when there is no EC.

Add a test from John Crepezzi and John Hawthorn to exercise this
situation.

See GH-4108
See GH-5094

[Bug #17573]

Co-authored-by: John Hawthorn <john@hawthorn.email>
Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
2021-11-22 19:29:29 -05:00
Nobuyoshi Nakada
ac152b3cac
Update dependencies 2021-11-21 16:21:18 +09:00
Hiroshi SHIBATA
39655aceb2 [ruby/date] Bump version to 3.2.2
21d91c5f4f
2021-11-18 19:39:57 +09:00
Jean Boussier
3ce27552df [ruby/date] Expose Date::VERSION
An almost universal convention for gems is to expose `Namespace::VERSION`
which makes it mcuh easier when debugging etc.

Many gems extracted from ruby don't do this, even though it would be even more
useful because they ship with ruby, so it's less clear which version it is.

fef7ec18d8
2021-11-18 19:39:55 +09:00
Yusuke Endoh
d23b3d9b7d Prevent "already initialized constant Digest::VERSION"
20211117T033003Z.log.html.gz
```
installing default gems from ext:   /home/chkbuild/chkbuild/tmp/build/20211117T033003Z/lib/ruby/gems/3.1.0
/home/chkbuild/chkbuild/tmp/build/20211117T033003Z/ruby/ext/digest/lib/digest/version.rb:4: warning: already initialized constant Digest::VERSION
/home/chkbuild/chkbuild/tmp/build/20211117T033003Z/ruby/.ext/common/digest/version.rb:4: warning: previous definition of VERSION was here
```

This hack is copied from ext/psych/psych.gemspec
2021-11-17 15:08:46 +09:00
Yusuke Endoh
cbb057e677 ext/io/wait/depend: make it work for Ruby 3.1.0-dev
Reverts 2eb3841e9c
because it fails on "update-deps" check in the ruby/ruby CI.

4230891140
```
diff --git a/ext/io/wait/depend b/ext/io/wait/depend
index 7b314b9..449e9fe 100644
--- a/ext/io/wait/depend
+++ b/ext/io/wait/depend
...
```

Maybe now it does not work on Ruby 2.6. This file must be changed for
each Ruby version. I have no good idea to fix this issue.
2021-11-17 11:21:31 +09:00
Jean Boussier
a87c56f820 [ruby/date] check_limit: also handle symbols
376c65942b
2021-11-16 22:51:41 +09:00