Commit graph

7790 commits

Author SHA1 Message Date
Jeremy Evans
6f3857f6a7 Support Encoding::Converter newline: :lf and :lf_newline options
Previously, newline: :lf was accepted but ignored.  Where it
should have been used was commented out code that didn't work,
but unlike all other invalid values, using newline: :lf did
not raise an error.

This adds support for newline: :lf and :lf_newline, for consistency
with newline: :cr and :cr_newline.  This is basically the same as
universal_newline, except that it only affects writing and not
reading due to RUBY_ECONV_NEWLINE_DECORATOR_WRITE_MASK.

Add tests for the File.open :newline option while here.

Fixes [Bug #12436]
2022-08-19 20:23:36 -07:00
Nobuyoshi Nakada
1a2f99275b [Bug #18958] format string must be ASCII compatible 2022-08-20 10:06:40 +09:00
Nobuyoshi Nakada
d2483393cb [Bug #18956] Negative codepoints are invalid characters 2022-08-20 10:06:16 +09:00
Nobuyoshi Nakada
1ef49de834 [Bug #18955] format single character for %c 2022-08-20 03:57:13 +09:00
Nobuyoshi Nakada
ce384ef5a9 [Bug #18955] Check length of argument for %c in proper encoding 2022-08-20 03:57:13 +09:00
Nobuyoshi Nakada
2652b0da6b
Harden SizedQueue#push timeout test 2022-08-19 00:16:44 +09:00
Jean Boussier
b0b9f7201a rb_str_resize: Only clear coderange on truncation
If we are expanding the string or only stripping extra capacity
then coderange won't change, so clearing it is wasteful.
2022-08-18 10:09:08 +02:00
Jean Boussier
fe61cad749 Implement SizedQueue#push(timeout: sec)
[Feature #18944]

If both `non_block=true` and `timeout:` are supplied, ArgumentError
is raised.
2022-08-18 10:07:37 +02:00
Peter Zhu
b4daf6e28e Fix flaky test for GC thrashing
GC could be in an intermediate state after creating the objects, so we
should finish GC by running a minor GC.
2022-08-17 10:54:28 -04:00
John Hawthorn
0608a9a086
Optimize Marshal dump/load for large (> 31-bit) FIXNUM (#6229)
* Optimize Marshal dump of large fixnum

Marshal's FIXNUM type only supports 31-bit fixnums, so on 64-bit
platforms the 63-bit fixnums need to be represented in Marshal's
BIGNUM.

Previously this was done by converting to a bugnum and serializing the
bignum object.

This commit avoids allocating the intermediate bignum object, instead
outputting the T_FIXNUM directly to a Marshal bignum. This maintains the
same representation as the previous implementation, including not using
LINKs for these large fixnums (an artifact of the previous
implementation always allocating a new BIGNUM).

This commit also avoids unnecessary st_lookups on immediate values,
which we know will not be in that table.

* Fastpath for loading FIXNUM from Marshal bignum

* Run update-deps
2022-08-15 16:14:12 -07:00
Peter Zhu
0264424d58 Add test for GC thrashing of young object creation
This test will prevent performance regressions like [Bug #18929].
2022-08-15 10:00:03 -04:00
Kouhei Yanagita
24e33b84b5 Remove Numeric#ceildiv 2022-08-12 15:57:52 +09:00
Kouhei Yanagita
4165fd0e76 Add Numeric#ceildiv and Integer#ceildiv 2022-08-12 15:57:52 +09:00
Jeremy Evans
49517b3bb4 Fix inspect for unicode codepoint 0x85
This is an inelegant hack, by manually checking for this specific
code point in rb_str_inspect.  Some testing indicates that this is
the only code point affected.

It's possible a better fix would be inside of lower-level encoding
code, such that rb_enc_isprint would return false and not true for
codepoint 0x85.

Fixes [Bug #16842]
2022-08-11 08:47:29 -07:00
Jeremy Evans
cfb9624460
Fix Array#[] with ArithmeticSequence with negative steps (#5739)
* Fix Array#[] with ArithmeticSequence with negative steps

Previously, Array#[] when called with an ArithmeticSequence
with a negative step did not handle all cases correctly,
especially cases involving infinite ranges, inverted ranges,
and/or exclusive ends.

Fixes [Bug #18247]

* Add Array#slice tests for ArithmeticSequence with negative step to test_array

Add tests of rb_arithmetic_sequence_beg_len_step C-API function.

* Fix ext/-test-/arith_seq/beg_len_step/depend

* Rename local variables

* Fix a variable name

Co-authored-by: Kenta Murata <3959+mrkn@users.noreply.github.com>
2022-08-11 19:16:49 +09:00
Jeremy Evans
ff42e2359b Revert "Add {Method,UnboundMethod}#{public?,private?,protected?}"
This reverts commit 2727815068 and
58dc8bf8f1.

Visibility is an attribute of the method entry in a class, not an
attribute of the Method object.

Fixes [#18729]
Fixes [#18751]
Fixes [#18435]
2022-08-10 13:02:52 -07:00
John Hawthorn
70b60d24b9 Fix inconsistency with opt_aref_with
opt_aref_with is an optimized instruction for accessing a Hash using a
non-frozen string key (ie. from a file without frozen_string_literal).
It attempts to avoid allocating the string, and instead silently using a
frozen string (hash string keys are always fstrings).

Because this is just an optimization, it should be invisible to the
user. However, previously this optimization was could be seen via hashes
with default procs.

For example, previously:

    h = Hash.new { |h, k| k.frozen? }
    str = "foo"
    h[str]   # false
    h["foo"] # true when optimizations enabled

This commit checks that the Hash doesn't have a default proc when using
opt_aref_with.
2022-08-04 14:48:47 -07:00
Jean Boussier
4406cb1bf1 Harden Queue#pop timeout tests
They occasionaly fail with;

```
  FLeaked thread: TestThreadQueue#test_queue_pop_timeout: #<Thread:0x0000000108e38e48 /Users/runner/work/ruby/ruby/src/test/ruby/test_thread_queue.rb:123 sleep>
  .Finished thread: TestThreadQueue#test_deny_pushers: #<Thread:0x0000000108e38e48 /Users/runner/work/ruby/ruby/src/test/ruby/test_thread_queue.rb:123 dead>
  ...
  Retrying...

    1) Failure:
  TestThreadQueue#test_sized_queue_pop_timeout [/Users/runner/work/ruby/ruby/src/test/ruby/test_thread_queue.rb:157]:
  <#<Thread:0x00000001084bc7e8 /Users/runner/work/ruby/ruby/src/test/ruby/test_thread_queue.rb:156 sleep>> expected but was
  <nil>.

    2) Failure:
  TestThreadQueue#test_queue_pop_timeout [/Users/runner/work/ruby/ruby/src/test/ruby/test_thread_queue.rb:124]:
  <#<Thread:0x00000001083ff058 /Users/runner/work/ruby/ruby/src/test/ruby/test_thread_queue.rb:123 sleep>> expected but was
  <nil>.
```

I'm hoping joining for longer should help avoid this.
2022-08-03 10:59:13 +02:00
Jean Boussier
e3aabe93aa Implement Queue#pop(timeout: sec)
[Feature #18774]

As well as `SizedQueue#pop(timeout: sec)`

If both `non_block=true` and `timeout:` are supplied, ArgumentError
is raised.
2022-08-02 11:04:28 +02:00
Nobuyoshi Nakada
ec3f59309e [Bug #17767] Now ENV.clone raises TypeError as well as ENV.dup
One year ago, the former method has been deprecated while the latter
has become an error.  Then the 3.1 released, it is enough time to make
also the former an error.
2022-08-02 16:40:12 +09:00
Akinori MUSHA
1a73a6cdd2
Implement Enumerator::Product and Enumerator.product [Feature #18685] 2022-07-30 20:05:14 +09:00
Matthew Draper
ab08a43ec5
YJIT: Teach getblockparamproxy to handle the no-block case without exiting (#6191)
Teach getblockparamproxy to handle the no-block case without exiting

Co-authored-by: John Hawthorn <john@hawthorn.email>

Co-authored-by: John Hawthorn <john@hawthorn.email>
2022-07-28 11:38:07 -04:00
Hiroshi SHIBATA
3eade59919
Skip randomly failing tests with FreeBSD 12 2022-07-28 19:45:15 +09:00
Nobuyoshi Nakada
f61dd38e5c
Wait the test thread to sleep
Revert "Synchronize the test thread sleep"

This reverts commit 307835fe31.
2022-07-25 18:50:08 +09:00
Nobuyoshi Nakada
307835fe31
Synchronize the test thread sleep 2022-07-25 12:01:09 +09:00
Jeremy Evans
7223c0da15 Do not chomp trailing line separator IO#each with nil separator and chomp
nil separator means no sepator, so chomp should not remove a line
separator.

Partially Fixes [Bug #18770]
2022-07-21 12:55:24 -07:00
Jeremy Evans
203f179ce7 Revert "Do not chomp trailing line separator IO#each with nil separator and chomp"
This reverts commit 04f86ad0b5.

This is causing CI issues, reverting for now.
2022-07-21 08:29:50 -07:00
Jeremy Evans
12ac8971a3 Do not have class/module keywords look up ancestors of Object
Fixes case where Object includes a module that defines a constant,
then using class/module keyword to define the same constant on
Object itself.

Implements [Feature #18832]
2022-07-21 08:28:05 -07:00
Jeremy Evans
04f86ad0b5 Do not chomp trailing line separator IO#each with nil separator and chomp
nil separator means no sepator, so chomp should not remove a line
separator.

Partially Fixes [Bug #18770]
2022-07-21 08:13:40 -07:00
Jeremy Evans
423b41cba7 Make String#each_line work correctly with paragraph separator and chomp
Previously, it was including one newline when chomp was used,
which is inconsistent with IO#each_line behavior. This makes
behavior consistent with IO#each_line, chomping all paragraph
separators (multiple consecutive newlines), but not single
newlines.

Partially Fixes [Bug #18768]
2022-07-21 08:02:32 -07:00
Daniel Colson
32e406d6d3 Ensure _id2ref finds symbols with the correct type
Prior to this commit it was possible to call `ObjectSpace._id2ref` with
an offset static symbol object_id and get back a new, incorrectly tagged
symbol:

```
> sensible_sym = ObjectSpace._id2ref(:a.object_id)
=> :a
> nonsense_sym = ObjectSpace._id2ref(:a.object_id + 40)
=> :a
> sensible_sym == nonsense_sym
=> false
```

`nonsense_sym` ends up tagged with `RUBY_ID_INSTANCE` instead of
`RB_ID_LOCAL`. That means we can do silly things like:

```
> foo = Object.new
> foo.instance_variable_set(:a, 123)
(irb):2:in `instance_variable_set': `a' is not allowed as an instance variable name (NameError)
> foo.instance_variable_set(ObjectSpace._id2ref(:a.object_id + 40), 123)
=> 123
> foo.instance_variables
=> [:a]
```

This was happening because `get_id_entry` ignores the tag bits when
looking up the symbol. So `rb_id2str(symid)` would return a value and
then we'd continue on with the nonsense `symid`.

This commit prevents the situation by checking that the `symid` actually
matches what we get back from `get_id_entry`. Now we get a `RangeError`
for the nonsense id:

```
> ObjectSpace._id2ref(:a.object_id)
=> :a
> ObjectSpace._id2ref(:a.object_id + 40)
(irb):1:in `_id2ref': 0x000000000013f408 is not symbol id value (RangeError)
```

Co-authored-by: John Hawthorn <jhawthorn@github.com>
2022-07-20 10:38:44 -07:00
Noah Gibbs
6140edb5df
Match +YJIT in Ruby desc when testing segv (#6141)
In test_bug_reporter and test_rubyoptions we intentionally
test child processes that cause SEGV. We run them with YJIT
if the parent uses YJIT so that the text description
matches the parent RUBY_DESCRIPTION.
2022-07-20 10:48:58 -04:00
Nobuyoshi Nakada
8f17591435 [Bug #18905] Check symbol name types more strictly 2022-07-20 00:23:38 +09:00
Nobuyoshi Nakada
68903df6f6
[Bug #18922] Normalize time at 24:00:00 UTC 2022-07-18 00:59:27 +09:00
Noah Gibbs
aed1539ec5
YJIT: Add send unit tests (#6143)
Add send unit tests for YJIT
2022-07-15 13:52:47 -04:00
Peter Zhu
7424ea184f Implement Objects on VWA
This commit implements Objects on Variable Width Allocation. This allows
Objects with more ivars to be embedded (i.e. contents directly follow the
object header) which improves performance through better cache locality.
2022-07-15 09:21:07 -04:00
Takashi Kokubun
439d31bc77
MJIT: Merge mjit_worker.c back to mjit.c (#6138)
Since #6006, we no longer avoid executing GC on mjit_worker.c and thus
there's no need to carefully change how we write code whether you're in
mjit.c or mjit_worker.c anymore.
2022-07-14 20:34:46 -07:00
Matt Valentine-House
067a5f1a00 [Feature #18901] Don't run size pool move tests without VWA 2022-07-12 08:50:33 -04:00
Matt Valentine-House
214ed4cbc6 [Feature #18901] Support size pool movement for Arrays
This commit enables Arrays to move between size pools during compaction.
This can occur if the array is mutated such that it would fit in a
different size pool when embedded.

The move is carried out in two stages:

1. The RVALUE is moved to a destination heap during object movement
   phase of compaction
2. The array data is re-embedded and the original buffer free'd if
   required. This happens during the update references step
2022-07-12 08:50:33 -04:00
Matt Valentine-House
6423d32e3b Replace use of double_heap in tests with expand_heap 2022-07-11 09:00:03 -04:00
Takashi Kokubun
fae568edbe
Use NO_JIT_DESCRIPTION only when needed
Apparently 203801566a broke YJIT's CI.
2022-07-10 23:58:19 -07:00
Takashi Kokubun
203801566a
Fix #5872 for MJIT GitHub Actions
If you run tests with RUN_OPTS=--mjit, the test fixes in
https://github.com/ruby/ruby/pull/5872 don't work.
2022-07-10 23:10:36 -07:00
Nobuyoshi Nakada
4a2662ae8b
[Bug #18890] Suppress warnings and fix the message
```
test/ruby/test_parse.rb:1384: warning: assigned but unused variable - obj
test/ruby/test_pattern_matching.rb:1162: warning: unused literal ignored
test/ruby/test_pattern_matching.rb:1165: warning: unused literal ignored
test/ruby/test_pattern_matching.rb:1161: warning: assigned but unused variable - a
test/ruby/test_pattern_matching.rb:1164: warning: assigned but unused variable - b
```

And a newline should be significant here.
2022-07-08 11:31:51 +09:00
Jean Boussier
65ae2bb2e0 Thread#value: handle threads killed by a fork
[Bug #18902]

When a thread is killed because we forked, the `value` if left
to `Qundef`. Returning it woudl crash the VM.
2022-07-07 19:49:25 +02:00
Nobuyoshi Nakada
cd94842922 [Bug #18892] Reset ARGF.lineno after reading shebang 2022-07-07 11:33:07 +09:00
Nobuyoshi Nakada
5ef3c7ea2d
[Bug #18898] Fallback invalid external encoding to the default 2022-07-06 18:08:31 +09:00
Nobuyoshi Nakada
eaeb130b11 [Bug #18890] newline should be insignificant after pattern label 2022-07-06 08:32:36 +09:00
Nobuyoshi Nakada
babdb158d1
Synchronize the test thread not to die before assertions 2022-07-05 13:50:45 +09:00
Nobuyoshi Nakada
982cda9a3e [Bug #18877] Let lex_ctxt not to eat escaped whitespace 2022-06-30 16:31:51 +09:00
Peter Zhu
66eb58d6bd Refactor tests for moving strings with compaction 2022-06-29 15:11:53 -04:00