Commit graph

11460 commits

Author SHA1 Message Date
normal
a226434206 process.c: fix ETXTBUSY from MJIT compiler process
This affects test/ruby/test_process.rb (test_execopt_env_path).

Since MJIT uses vfork+execve in a separate thread, there can be
small window in-between vfork and execve where tmp_script.cmd is
held open by the vforked child.  vfork only pauses the MJIT
thread, not any Ruby Threads, so our call to Process.spawn will
hit ETXTBUSY in that window unless we fork.

main thread                       | MJIT thread
----------------------------------------------------
fd = open(tmp)                    |                |
                                  | vfork for CC   |   CC running
write                             |                | ---------------
fchmod                            |                |  sees "fd" here
close(fd)                         |                |
    Process.spawn called          |                |
vfork  (spawn)|    (new process)  |                |
              | execve => TXTBUSY |                |
              |                   |                | execve (FD_CLOEXEC on fd)
              |                   | vfork returns  |

Holding the waitpid_lock whenever we intend to spawn a process
prevents the MJIT thread from spawning a process while we are
spawning in Ruby-land.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66171 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-03 19:49:54 +00:00
nobu
4d1e32eac6 Revert "Symbol refinements"
Due to assertion failures when VM_CHECK_MODE >= 2.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66155 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-03 08:51:29 +00:00
normal
a063f71a7c webrick: fix tests on Debian sid/unstable with OpenSSL 1.1.1a
OpenSSL complains abour our keys being small and weak :<
Make them big and strong with 2048-bit RSA keys and SHA256 digests

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66152 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-03 04:51:08 +00:00
nobu
b94e344b9d Symbol refinements
* vm_args.c (refine_sym_proc_call): enalbe accidentally disabled
  refinements in Symbol#to_proc.   [Bug #15114]

From: osyo (manga osyo) <manga.osyo@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66151 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-03 04:44:34 +00:00
nobu
1a1dd5ccf4 Revert "Symbol refinements [Bug #15114]"
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66150 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-03 04:44:33 +00:00
nobu
cd5b309575 Symbol refinements [Bug #15114]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66149 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-03 04:34:27 +00:00
nobu
afa685398e Refine RubyVM::AbstractSyntaxTree::Node#type
* ast.c (rb_ast_node_type): simplified to return a Symbol without
  "NODE_" prefix.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66142 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-03 01:06:34 +00:00
nobu
8edd642381 test_pp.rb: show the node when failed
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-03 00:57:06 +00:00
nobu
87e1dd2982 Add RubyVM::AST#pretty_print
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66140 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-03 00:24:38 +00:00
duerst
a96a594f99 solve the genie/zombie/wrestlers bug
enc/unicode.c: - Add U+1F93C (WRESTLERS), U+1F9DE (GENIE), and U+1F9DF
                 to onigenc_unicode_GCB_ranges_E_Base.
               - Add comments with character names.
test/ruby/enc/test_emoji_breaks.rb: Activate tests for genie/zombie/wrestlers.
This closes issue #15343.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66133 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-02 10:07:42 +00:00
mrkn
a7e3516ff1 Fix JSON::Parser against bigdecimal updates
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66127 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-02 05:21:57 +00:00
mrkn
a0e438cd3c Import bigdecimal-1.4.0.pre-20181130a
* 74d25ef..v1.4.0.pre.20181130a

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66124 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-02 05:21:54 +00:00
hsbt
7a46a3b941 Merge rubygems-3.0.0.beta3.
* [GSoC] Multi-factor feature for RubyGems https://github.com/rubygems/rubygems/pull/2369

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66118 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-01 11:01:00 +00:00
samuel
4b99725de9 Avoid GCing dead stack after switching away from a fiber
Fixes <https://bugs.ruby-lang.org/issues/14561> and discussed
<https://bugs.ruby-lang.org/issues/15362>.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66111 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-01 03:49:52 +00:00
normal
8f23f690c1 test/ruby/test_io.rb (test_recycled_fd_close): remove hacks
I think I finally fixed the underlying bug in r65937
("io.c (fptr_finalize_flush): close race leading to EBADF")

I've run this test over 100000 times on a multicore system, now.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66104 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-30 11:44:21 +00:00
normal
dc1ecab084 test/testunit/test_parallel.rb: respect subprocess timeout scale
MJIT tests take longer than 100s on my VM :<

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66101 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-30 03:56:34 +00:00
normal
9d74d402e1 disable non-blocking pipes and sockets by default
There seems to be a compatibility problems with Rails +
Rack::Deflater; so we revert this incompatibility.

This effectively reverts r65922; but keeps the bugfixes to
better support non-blocking sockets and pipes for future use.

[Bug #15356] [Bug #14968]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66093 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-29 20:00:00 +00:00
ko1
a763e3460f clear dst Hash on Hash#replace. [Bug #15358]
* hash.c (linear_copy): solve two issues on `Hash#replace`.
  (1) fix memory leak
      (1-1) don't allocate memory if destination already
            has a memory area.
      (1-2) free destination memory if src is NULL.
  (2) clear transient heap flag if src is NULL. [Bug #15358]


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66091 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-29 08:03:55 +00:00
nobu
4471d4a3e5 time.c: rescue find_timezone when loading
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66080 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-28 14:08:35 +00:00
nobu
2d0833e101 Try with TZInfo and Timezone gems
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66079 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-28 14:08:34 +00:00
nobu
ab73b30539 Time.at in: tz
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66078 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-28 14:08:33 +00:00
nobu
bdfc701b85 Timezone at Time.at
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66077 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-28 14:08:32 +00:00
nobu
337b4b6344 Try Time.find_timezone to convert name to timezone
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66076 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-28 14:08:31 +00:00
nobu
fe08f99e93 Try Time.find_timezone to load
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66075 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-28 14:08:30 +00:00
normal
e9dbafa5c8 test_settracefunc (test_tracepoint_enable_with_target_line): less fragile
Allow us to make other changes to the file and move the method
around without affecting this test.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66067 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-28 04:31:42 +00:00
hsbt
5dbc6583c9 Merge rubygems upstream from 2c499655f2
https://github.com/rubygems/rubygems/pull/2493

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66065 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-28 03:08:14 +00:00
shugo
6ec1720aa3 lib/monitor.rb: avoid race conditions by Thread.handle_interrupt
Suggested by Benoit Daloze.  [ruby-core:88502] [Bug #14998]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66061 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-28 01:51:44 +00:00
aycabta
4beec66757 Merge rdoc-6.1.0.bata2
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66024 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-27 07:02:36 +00:00
duerst
51196cf518 improve messages for test failures
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66010 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 23:30:43 +00:00
svn
27b1553b2f * remove trailing spaces.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66009 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 20:16:16 +00:00
ko1
72e60a0437 TracePoint#enable(target_line:) is supported. [Feature #15289]
* vm_trace.c: `TracePoint#enable(target_line:)` is supported.
  This option enables a hook only at specified target_line.
  target_line should be combination with target and :line event.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66008 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 20:16:14 +00:00
ko1
6e33c16ffd Give up support for cross-callcc set_trace_func.
* cont.c (cont_restore_thread): cause error if trace-status is changed.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66007 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 19:59:08 +00:00
ko1
96990203b7 Support targetting TracePoint [Feature #15289]
* vm_trace.c (rb_tracepoint_enable_for_target): support targetting
  TracePoint. [Feature #15289]

  Tragetting TracePoint is only enabled on specified method, proc
  and so on, example: `tp.enable(target: code)`.

  `code` should be consisted of InstructionSeuqnece (iseq)
  (RubyVM::InstructionSeuqnece.of(code) should not return nil)
  If code is a tree of iseq, TracePoint is enabled on all of
  iseqs in a tree.

  Enabled tragetting TracePoints can not enabled again with
  and without target.

* vm_core.h (rb_iseq_t): introduce `rb_iseq_t::local_hooks`
  to store local hooks.
  `rb_iseq_t::aux::trace_events` is renamed to
  `global_trace_events` to contrast with `local_hooks`.

* vm_core.h (rb_hook_list_t): add `rb_hook_list_t::running`
  to represent how many Threads/Fibers are used this list.
  If this field is 0, nobody using this hooks and we can
  delete it.

  This is why we can remove code from cont.c.

* vm_core.h (rb_vm_t): because of above change, we can eliminate
  `rb_vm_t::trace_running` field.
  Also renamed from `rb_vm_t::event_hooks` to `global_hooks`.

* vm_core.h, vm.c (ruby_vm_event_enabled_global_flags): renamed
  from `ruby_vm_event_enabled_flags.

* vm_core.h, vm.c (ruby_vm_event_local_num): added to count
  enabled targetting TracePoints.

* vm_core.h, vm_trace.c (rb_exec_event_hooks): accepts
  hook list.

* vm_core.h (rb_vm_global_hooks): added for convinience.

* method.h (rb_method_bmethod_t): added to maintain Proc
  and `rb_hook_list_t` for bmethod (defined by define_method).

* prelude.rb (TracePoint#enable): extracet a keyword parameter
  (because it is easy than writing in C).
  It calls `TracePoint#__enable` internal method written in C.

* vm_insnhelper.c (vm_trace): check also iseq->local_hooks.

* vm.c (invoke_bmethod): check def->body.bmethod.hooks.

* vm.c (hook_before_rewind): check iseq->local_hooks
  and def->body.bmethod.hooks before rewind by exception.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66003 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 18:16:39 +00:00
k0kubun
2c9259e6e0 test_jit.rb: skip a test for Windows for now
20558412

removing CloseHandle disallows us to remove the DLL.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 16:13:56 +00:00
k0kubun
08e1705eec process.c: finish MJIT prior to #exec
to prevent ruby from leaving MJIT-related files.

test_jit.rb: add a test to prevent that

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65994 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 13:39:53 +00:00
duerst
0409290ec0 add tests for grapheme clusters using Unicode Emoji test data
Add file test/ruby/enc/test_emoji_breaks.rb to test String#each_grapheme_cluster
test data provided by Unicode (at https://www.unicode.org/Public/emoji/#{EMOJI_VERSION}/).

Lines containing emoji for genies, zombies, and wrestling are ignored
because there seems to be a bug (#15343) in the implementation.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65990 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 09:03:11 +00:00
mame
ebff9dc10e object.c: Deprecate Object#=~ and add NilClass#=~`
Object#=~ always returns nil.  This behavior is not only unuseful but
also troublesome because it may hide a type error.

This change deprecates Object#=~.  For compatibility, NilClass#=~ is
newly introduced.  [Feature #15231]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65989 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 07:55:07 +00:00
mame
e0dd7b4d3a test/rdoc/test_rdoc_rdoc.rb: add dummy finish
RDoc::Options#@exclude is initialized as an empty array.
Then, #finish converts it to a regexp or nil and reassign it to @exclude.
Some methods of RDoc assumes that #finish has been already called.
So, this change forces to assign nil to @exclude.

This type confusion was found during work for removal of `Object#=~`.
[Feature #15231]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 05:46:02 +00:00
naruse
2c652768a8 Normalize month-mday before finding epoch
Especially over the year 2038, 30 Feb and so on may cause odd behavior
on validating found epoch with given year-month-day [Bug #15340]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65974 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-25 16:23:18 +00:00
nobu
87b03e8c3c parser: improve error messages
[Fix GH-2011]

From: Akim Demaille <akim.demaille@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65969 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-25 08:31:40 +00:00
nobu
865313dfac assertions for r65956
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65959 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-24 12:48:56 +00:00
duerst
33afc9e0b9 remove guard against bug #15337, because it is fixed
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65958 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-24 12:33:10 +00:00
duerst
7599b3f6c6 add tests using Unicode test data for grapheme clusters
Add file test/ruby/enc/test_grapheme_breaks.rb to test String#each_grapheme_cluster
and \X extended grapheme cluster matcher in regular expressions against test data
provided by Unicode (ucd/auxiliary/GraphemeBreakTest.txt).

Some lines in the data file are ignored, as follows:
- Lines with a surrogate, because Ruby doesn't handle these
- The case of "\r\n", because there is a bug (#15337) in the implementation

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65955 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-24 12:10:25 +00:00
naruse
7850586af4 Don't use single byte optimization on grapheme clusters
Unicode Text Segmentation considers CRLF as a character. [Bug #15337]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65954 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-24 11:53:19 +00:00
knu
045b0e54d8 Implement Enumerator#+ and Enumerable#chain [Feature #15144]
They return an Enumerator::Chain object which is a subclass of
Enumerator, which represents a chain of enumerables that works as a
single enumerator.

```ruby
e = (1..3).chain([4, 5])
e.to_a #=> [1, 2, 3, 4, 5]

e = (1..3).each + [4, 5]
e.to_a #=> [1, 2, 3, 4, 5]
```

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65949 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-24 08:38:35 +00:00
normal
c0e20037f3 io.c: wait on FD readability w/o GVL reacquisition
Since non-blocking I/O is the default after [Bug #14968],
we will hit it more often and cause more acquisition/release
of GVL to wait on single FD.

This also lets us avoid touching the temporal string locking
as much and lets us clean up some test changes made for
[Bug #14968]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65948 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-24 08:23:26 +00:00
k0kubun
ba9a6f8934 Fix r65926 test_dir.rb for windows (tilde)
[Fix GH-2022]

From: MSP-Greg <greg.mpls@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65943 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-24 00:55:26 +00:00
kazu
16f886328a test/ruby/test_thread.rb: use safe navigation operator.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65934 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-23 10:59:59 +00:00
nobu
32b7457fa4 split each assert_nothing_raised
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65932 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-23 01:57:26 +00:00
naruse
4e1ee66580 Use File.expand_path("~username") instead of ENV["HOME"]
File.expand_path("~username") uses getpwnam() independently from
Dir.home. [Bug #15324]

From: Chris Seaton <chris@chrisseaton.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65926 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-22 10:43:42 +00:00