Commit graph

9854 commits

Author SHA1 Message Date
Peter Zhu
a500eb9f8c Fix memory leak in Ripper
The following script leaks memory in Ripper:

```ruby
require "ripper"

20.times do
  100_000.times do
    Ripper.parse("")
  end

  puts `ps -o rss= -p #{$$}`
end
```
2023-06-28 09:50:51 -04:00
Jean byroot Boussier
f1313caec1 [ruby/fiddle] Mark Closure, Function and Handle as write barrier
protected
(https://github.com/ruby/fiddle/pull/129)

They don't have a mark function, so they don't need any change.

9bbc732aef

Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2023-06-28 16:36:53 +09:00
Jean byroot Boussier
135d5fa36c [ruby/fiddle] Implement write barriers for Fiddle::Pointer
(https://github.com/ruby/fiddle/pull/127)

Save from having to mark them on every minor.

153c09c99f

Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2023-06-28 16:36:52 +09:00
Sutou Kouhei
572b432e50 [ruby/fiddle] Fix a typo
4c0c5da1a5
2023-06-28 16:36:52 +09:00
Sutou Kouhei
02661f5e9b [ruby/fiddle] Add support for converting "C" (one character string) to char
GitHub: fix https://github.com/ruby/fiddle/pull/96

I wanted to add a test for this but I couldn't find a function that
has a "char" argument in libc...

Reported by kojix2. Thanks!!!

2c863ef8ba
2023-06-28 16:36:51 +09:00
Sutou Kouhei
731d27729b [ruby/fiddle] Add support for more "short" variants
2b22bb9d74
2023-06-28 16:36:50 +09:00
Sutou Kouhei
ebf14d01ba [ruby/fiddle] Add support for "long" variants
GitHub: fix https://github.com/ruby/fiddle/pull/100

Reported by David M. Lary. Thanks!!!

516333dd78
2023-06-28 16:36:50 +09:00
Aaron Patterson
85937f3a0a [ruby/fiddle] Add a helper method for reading/writing memory
(https://github.com/ruby/fiddle/pull/123)

This commit adds two new methods, `Fiddle::Pointer.read` and
`Fiddle::Pointer.write`. Both methods take an address, and will read or
write bytes at that address respectively.

For example we can read from an address without making a Pointer object:

```ruby
Fiddle::Pointer.read(address, 5) # read 5 bytes
```

We can also write to an address without allocating a Pointer object:

```ruby
Fiddle::Pointer.write(address, "bytes") # write 5 bytes
```

This allows us to read / write memory at arbitrary addresses without
instantiating a new `Fiddle::Pointer` object.

Examples where this API would be useful
[1](f03481d28b/lib/tenderjit/fiddle_hacks.rb (L26-L28))
[2](77c8daa2d4/lib/ruby_vm/rjit/c_pointer.rb (L193))
[3](77c8daa2d4/lib/ruby_vm/rjit/c_pointer.rb (L284))

I also added a writer method for the same reasons as the reader.

---------

04238cefed

Co-authored-by: Sutou Kouhei <kou@clear-code.com>
2023-06-28 16:36:49 +09:00
qwyng
e9f62fa5a7 [ruby/openssl] add OpenSSL Provider support
189c167e40
[rhe: tool/update-deps --fix to update ext/openssl/depend]
2023-06-19 17:45:51 +09:00
Kazuki Yamaguchi
af66b9b720 [ruby/openssl] pkey: use unsigned type for bit fields
clang generates a warning:

	../../../../ext/openssl/ossl_pkey.c:326:22: error: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Werror,-Wsingle-bit-bitfield-constant-conversion]
	    arg->interrupted = 1;
	                     ^ ~
	1 error generated.

4b2ba7b022
2023-06-19 01:57:09 +09:00
Kazuki Yamaguchi
397ebbc62d [ruby/openssl] Drop support for Ruby 2.6
Ruby 2.6 has reached EOL on 2022-03. Requiring Ruby 2.7 allows us to
use C99 syntax.

f6ba75e51e
2023-06-19 01:57:09 +09:00
Kazuki Yamaguchi
1d0a087d45 [ruby/openssl] extconf.rb: apply RUBY_OPENSSL_EXT{C,LD}FLAGS after checking features
RUBY_OPENSSL_EXTCFLAGS and RUBY_OPENSSL_EXTLDFLAGS have been added for
the primary purpose of appending custom warning flags during
development and CI.

Since checking programs generated by mkmf may not be completely
warning-free, we don't want to apply -Werror that may be supplied from
those environment variables.

2a95b971d5
2023-06-19 01:57:09 +09:00
Jun Aruga
2a4834057b [ruby/openssl] Workaround: Fix OpenSSL::PKey.read that cannot parse PKey in the FIPS mode.
This commit is a workaround to avoid the error below that the
`OpenSSL::PKey.read` fails with the OpenSSL 3.0 FIPS mode.

```
$ openssl genrsa -out key.pem 4096

$ ruby -e "require 'openssl'; OpenSSL::PKey.read(File.read('key.pem'))"
-e:1:in `read': Could not parse PKey (OpenSSL::PKey::PKeyError)
  from -e:1:in `<main>'
```

The root cause is on the OpenSSL side. The `OSSL_DECODER_CTX_set_selection`
doesn't apply the selection value properly if there are multiple providers, and
a provider (e.g.  "base" provider) handles the decoder implementation, and
another provider (e.g. "fips" provider) handles the keys.

The workaround is to create `OSSL_DECODER_CTX` variable each time without using
the `OSSL_DECODER_CTX_set_selection`.

5ff4a31621
2023-06-19 01:57:09 +09:00
Jean Boussier
1965c09ee5 [ruby/openssl] Implement Write Barrier for all OpenSSL types
The vast majority have no reference so it's just a matter of setting the flags.

For the couple exception, they have very little references so it's
easy.

2c7c6de69e
2023-06-19 01:57:09 +09:00
Jun Aruga
0a84bd6b0b [ruby/openssl] Append flags from environment variables.
According to the `mkmf.rb#init_mkmf`, there are command line options below.

* `--with-cflags` to set the `cflags`
* `--with-ldflags` to set the `ldflags`

For example the following command compiles with the specified flags. Note that
`MAKEFLAGS` is to print the compiler command lines.

```
$ MAKEFLAGS="V=1" \
  bundle exec rake compile -- \
  --with-cflags="-Wundef -Werror" \
  --with-ldflags="-fstack-protector"
```

However, I couldn't find command line options to append the flags. And this
commit is to append the `cflags` and `ldflags` by the environment variables.

```
$ MAKEFLAGS="V=1" \
  RUBY_OPENSSL_EXTCFLAGS="-Wundef -Werror" \
  RUBY_OPENSSL_EXTLDFLAGS="-fstack-protector" \
  bundle exec rake compile
```

b551eb86f6
2023-06-19 01:57:09 +09:00
Nobuyoshi Nakada
31e24a42f3 [ruby/psych] Comment out the code used to expand the file list [ci skip]
715f3bfad0
2023-06-12 12:31:52 +00:00
Nobuyoshi Nakada
70483f6ca4
Add missing dependencies 2023-06-12 19:10:29 +09:00
yui-knk
b481b673d7 [Feature #19719] Universal Parser
Introduce Universal Parser mode for the parser.
This commit includes these changes:

* Introduce `UNIVERSAL_PARSER` macro. All of CRuby related functions
  are passed via `struct rb_parser_config_struct` when this macro is enabled.
* Add CI task with 'cppflags=-DUNIVERSAL_PARSER' for ubuntu.
2023-06-12 18:23:48 +09:00
Nobuyoshi Nakada
bcded49d1a [ruby/psych] Update spec files
ba6fc48591
2023-06-12 07:02:02 +00:00
Samuel Williams
3fe09eba9d
Add deprecations for public struct rb_io members. (#7916)
* Add deprecations for public struct rb_io members.
2023-06-08 20:22:43 +09:00
Hiroshi SHIBATA
d6bddcb013
racc/cparse is extracted from ruby/ruby repository 2023-06-08 18:05:07 +09:00
Hiroshi SHIBATA
d1775aa388 Try to promote racc as bundled gems 2023-06-08 17:25:38 +09:00
Hiroshi SHIBATA
b90683be44 [ruby/psych] Drop to support Ruby 2.4 because the latest version of core_assertions only support Ruby 2.5+
6ec316b7fa
2023-06-08 07:51:44 +00:00
Peter Zhu
7577c101ed
Unify length field for embedded and heap strings (#7908)
* Unify length field for embedded and heap strings

The length field is of the same type and position in RString for both
embedded and heap allocated strings, so we can unify it.

* Remove RSTRING_EMBED_LEN
2023-06-06 10:19:20 -04:00
eileencodes
40f090f433 Revert "Revert "Fix cvar caching when class is cloned""
This reverts commit 10621f7cb9.

This was reverted because the gc integrity build started failing. We
have figured out a fix so I'm reopening the PR.

Original commit message:

Fix cvar caching when class is cloned

The class variable cache that was added in
ruby#4544 changed the behavior of class
variables on cloned classes. As reported when a class is cloned AND a
class variable was set, and the class variable was read from the
original class, reading a class variable from the cloned class would
return the value from the original class.

This was happening because the IC (inline cache) is stored on the ISEQ
which is shared between the original and cloned class, therefore they
share the cache too.

To fix this we are now storing the `cref` in the cache so that we can
check if it's equal to the current `cref`. If it's different we don't
want to read from the cache. If it's the same we do. Cloned classes
don't share the same cref with their original class.

This will need to be backported to 3.1 in addition to 3.2 since the bug
exists in both versions.

We also added a marking function which was missing.

Fixes [Bug #19379]

Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
2023-06-05 11:11:12 -07:00
yui-knk
7b803eafa2 Ripper does not depend on Bison [ci skip]
It also uses Lrama then no dependency on Bison.
2023-06-03 10:34:24 +09:00
Hiroshi SHIBATA
c32a30ec09 [ruby/stringio] Development of 3.0.8 started.
1587d3698e
2023-06-02 08:20:38 +00:00
yui-knk
3a4206c7a1 No need to define "BISON" on extconf.rb
"BISON" is defined in "ext/ripper/depend".
2023-06-02 09:28:30 +09:00
Aaron Patterson
10621f7cb9
Revert "Fix cvar caching when class is cloned"
This reverts commit 77d1b08247.
2023-06-01 14:55:36 -07:00
eileencodes
77d1b08247 Fix cvar caching when class is cloned
The class variable cache that was added in
https://github.com/ruby/ruby/pull/4544 changed the behavior of class
variables on cloned classes. As reported when a class is cloned AND a
class variable was set, and the class variable was read from the
original class, reading a class variable from the cloned class would
return the value from the original class.

This was happening because the IC (inline cache) is stored on the ISEQ
which is shared between the original and cloned class, therefore they
share the cache too.

To fix this we are now storing the `cref` in the cache so that we can
check if it's equal to the current `cref`. If it's different we don't
want to read from the cache. If it's the same we do. Cloned classes
don't share the same cref with their original class.

This will need to be backported to 3.1 in addition to 3.2 since the bug
exists in both versions.

We also added a marking function which was missing.

Fixes [Bug #19379]

Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
2023-06-01 08:52:48 -07:00
Samuel Williams
d5a5333168 [ruby/stringio] Avoid direct struct usage.
(https://github.com/ruby/stringio/pull/54)

We will eventually want to refactor this, but for now this is compatible
enough.
2023-06-01 11:41:19 +00:00
Samuel Williams
64ec1813ac
Restores the changes in io-console without breaking backwards compatibility. (#7882)
This reverts commit 35da41b29b.

This updates `io-console` to avoid accessing the internal details of `rb_io_t`
and instead use `rb_io_descriptor` and `rb_io_path` etc.
2023-06-01 17:43:45 +09:00
Samuel Williams
a218ed5692
Hide the usage of rb_io_t where possible. (#7880)
This retries the compatible parts of the previously reverted PR so we can continue to update related code without breaking backwards compatibility.
2023-06-01 14:23:30 +09:00
Hiroshi SHIBATA
35da41b29b
Revert https://github.com/ruby/io-console/pull/43
This reverts commit 1889133c04 and
  commit 764207e47c.

  Followed up with 85dcc4866d
2023-06-01 09:36:55 +09:00
NARUSE, Yui
85dcc4866d Revert "Hide most of the implementation of struct rb_io. (#6511)"
This reverts commit 18e55fc1e1.

fix [Bug #19704]
https://bugs.ruby-lang.org/issues/19704
This breaks compatibility for extension libraries. Such changes
need a discussion.
2023-06-01 08:43:22 +09:00
Benoit Daloze
30b960ba34 [ruby/io-nonblock] Add TruffleRuby support and add it in CI
* Do not use a C extension on TruffleRuby for these 3 methods.

1b8e52abfa
2023-05-30 12:39:26 +00:00
Benoit Daloze
c7c8e4c96e [ruby/io-nonblock] Use the correct pattern for the fallback of rb_io_descriptor()
* See d1d9aef45c

dadb42422b
2023-05-30 12:39:25 +00:00
Samuel Williams
18e55fc1e1
Hide most of the implementation of struct rb_io. (#6511)
* Add rb_io_path and rb_io_open_descriptor.

* Use rb_io_open_descriptor to create PTY objects

* Rename FMODE_PREP -> FMODE_EXTERNAL and expose it

FMODE_PREP I believe refers to the concept of a "pre-prepared" file, but
FMODE_EXTERNAL is clearer about what the file descriptor represents and
aligns with language in the IO::Buffer module.

* Ensure that rb_io_open_descriptor closes the FD if it fails

If FMODE_EXTERNAL is not set, then it's guaranteed that Ruby will be
responsible for closing your file, eventually, if you pass it to
rb_io_open_descriptor, even if it raises an exception.

* Rename IS_EXTERNAL_FD -> RUBY_IO_EXTERNAL_P

* Expose `rb_io_closed_p`.

* Add `rb_io_mode` to get IO mode.

---------

Co-authored-by: KJ Tsanaktsidis <ktsanaktsidis@zendesk.com>
2023-05-30 10:02:40 +09:00
Benoit Daloze
f61bd3ca80 [ruby/cgi] Add TruffleRuby support and add it in CI
1240fec9c9
2023-05-29 19:33:38 +00:00
Benoit Daloze
610038f3bf [ruby/etc] Use the correct pattern for the fallback of rb_io_descriptor()
* See d1d9aef45c

5185685003
2023-05-29 19:31:55 +00:00
Samuel Williams
98f500d095
[ruby/openssl] Remove usage of IO internals. 2023-05-29 22:01:58 +09:00
Samuel Williams
764207e47c [ruby/io-console] Remove trailing whitespace.
55f1586463
2023-05-29 12:37:49 +00:00
Samuel Williams
1889133c04 [ruby/io-console] Remove usage of IO internals.
(https://github.com/ruby/io-console/pull/43)

4b839a504f

Co-authored-by: Benoit Daloze <eregontp@gmail.com>
2023-05-29 12:30:32 +00:00
Samuel Williams
14abb799c9 [ruby/io-nonblock] Remove usage of IO internals.
(https://github.com/ruby/io-nonblock/pull/11)

caa2b94d19
2023-05-28 08:40:38 +00:00
Samuel Williams
55393645a1 [ruby/io-nonblock] Revert "Remove usage of IO internals."
This reverts commit 9772b6dcb6.

08cabf78e1
2023-05-28 08:01:50 +00:00
Samuel Williams
26dd9c5f0f [ruby/io-nonblock] Remove usage of IO internals.
9772b6dcb6
2023-05-28 07:49:33 +00:00
Samuel Williams
881757c03f [ruby/io-wait] Remove usage of IO internals.
(https://github.com/ruby/io-wait/pull/25)

e5854b46c7
2023-05-28 05:09:02 +00:00
Samuel Williams
00d6674636 [ruby/etc] Remove usage of IO internals.
(https://github.com/ruby/etc/pull/26)

ea15eceb13
2023-05-28 05:08:40 +00:00
Hiroshi SHIBATA
e5bce57e1d [ruby/nkf] Bump up nkf-0.1.3
bc90e2ed39
2023-05-25 09:11:36 +00:00
Hiroshi SHIBATA
3026a2f4e9 [ruby/nkf] Added mission extension configuration
e627a39dff
2023-05-25 09:11:35 +00:00