Commit graph

10514 commits

Author SHA1 Message Date
normal
524e660877 io.c: fix IO.copy_stream on O_APPEND destination on Linux
Linux copy_file_range(2) fails with EBADF if the destination FD
has O_APPEND set.  Preserve existing (Ruby <= 2.4) behavior by
falling back to alternative copy mechanisms if this is the case
(instead of raising Errno::EBADF).

* io.c (nogvl_copy_file_range): do not raise on O_APPEND dst
* test/ruby/test_io.rb (test_copy_stream_append): new test
  [Feature #13867]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60490 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27 18:37:23 +00:00
mame
e35fe8d11b Revert "Revert "Manage AST NODEs out of GC""
This re-introduces r60485.
This reverts commit 5a176b75b1.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60488 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27 16:44:57 +00:00
mame
5a176b75b1 Revert "Manage AST NODEs out of GC"
This reverts commit 620ba74778.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60486 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27 16:02:34 +00:00
mame
620ba74778 Manage AST NODEs out of GC
NODEs in AST are no longer objects managed by GC.  This change will
remove the restriction imposed by the GC.  For example, a NODE can use
more than five words (this is my primary purpose; we want to store the
position data for each NODE, for coverage library), or even a NODE can
have variable length (some kinds of NODEs have unused fields).
To do this, however, we need more work, since Ripper still uses T_NODE
objects managed by the GC.

The life time of NODEs is more obvious than other kinds of objects; they
are created at parsing, and they become disused immediately after
compilation.  This change releases all NODEs by a few `xfree`s after
compilation, so performance will be improved a bit.  In extreme example,
`eval("x=1;" * 10000000)` runs much faster (40 sec. -> 7.8 sec. on my
machine).

The most important part of this change is `ast_t` struct, which has
three contents: (1) NODE buffer (malloc'ed memory), (2) a reference to
the root NODE, and (3) an array that contains objects that must be
marked during parsing (such as literal objects).  Some functions that
had received `NODE*` arguments, must now receive `ast_t*`.

* node.c, node.h: defines `ast_t` struct and related operations.
* gc.c, internal.h: defines `imemo_ast`.
* parse.y: makes `parser_params` struct have a reference to `ast_t`.
  Instead of `rb_node_newnode`, use `rb_ast_newnode` to create a NODE.
* iseq.c, load.c, ruby.c, template/prelude.c.tmpl: modifies some
  functions to handle `ast_t*` instead of `NODE*`.
* test/ruby/test_gc.rb: ad-hoc fix for a failed test.  The test assumes
  GC eden is increased at startup by NODE object creation.  However,
  this change now create no NODE object, so GC eden is not necessarily
  increased.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60485 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27 15:59:02 +00:00
naruse
a6488f04de Revert "Dir.glob with FNM_EXTGLOB is optimized [Feature #13873]"
This reverts commit r60341,r60342,r60344,r60345.
Breaking compabitility of the order of result breaks many tests.
To avoid such effort to fix tests, the order should be kept.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60478 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-27 08:40:40 +00:00
naruse
3acc0ba1d8 Host header should add branckets to IPv6 address [Bug #12642]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60455 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-26 15:29:36 +00:00
ko1
d77305fc9c make root fiber at switching.
* cont.c (fiber_switch): make sure the root fiber object is available
  before the first switching.

* test/ruby/test_fiber.rb: remove "skip".


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60453 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-26 14:38:22 +00:00
ko1
d717dafc0e skip test_fiber_transfer_segv because it cause error with previous commit
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60452 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-26 14:28:43 +00:00
nobu
07f04f468d test_env.rb: fix one-off bug
* test/ruby/test_env.rb (TestEnv#test_win32_blocksize): count the
  terminator byte too.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60439 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-26 07:36:23 +00:00
marcandre
7e3bd6646c TracePoint#enable, #disable: Don't yield any argument. [Bug #14057]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60437 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-26 06:58:09 +00:00
kazu
fc29786926 test/ostruct: Specify frozen_string_literal: true.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60433 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-25 23:45:33 +00:00
usa
8a67b59c31 Get rid of a test error on Windows caused by r60417
* test/testunit/test_parallel.rb (teardown): this code seems to allow
  `Errno::EPIPE` from @worker_in.  in such case, `close` may also raise the same
  exception.
  I'm not confident in this conclusion and still doubt that we should revert
  r60417 instead of this commit or not.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60428 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-25 13:29:27 +00:00
nobu
7da5b32a7d test_io.rb: skip writev test
* test/ruby/test_io.rb (TestIO#test_puts_parallel): skip a test
  needs writev which is not portable.  [Feature #14042]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60424 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-25 12:31:30 +00:00
nobu
237901a41b io.c: warn old write
* io.c (rb_io_puts): warn if write method accepts just one
  argument.  [ruby-core:83529] [Feature #14042]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60423 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-25 12:04:53 +00:00
nobu
635d0822cd io.c: write a newline together
* io.c (rb_io_puts): write a newline together at once for each
  argument.  based on the patch by rohitpaulk (Rohit Kuruvilla) at
  [ruby-core:83508].  [Feature #14042]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60417 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-25 05:44:38 +00:00
nobu
50a99a7679 range.c: use rb_check_funcall
* range.c (rb_range_values): use rb_check_funcall instead of
  calling rb_respond_to then rb_funcall, and allow `begin` and
  `end` to be private as well as other internal conversions.
  [ruby-core:83541] [Bug #14048]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60416 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-25 05:23:54 +00:00
nobu
0ba049256e range.c: check if exclude_end? is defined
* range.c (rb_range_values): should raise TypeError if necessary
  method is not defined, not NoMethodError, when trying to tell if
  the object is a Range and extract info.
  [ruby-core:83541] [Bug #14048]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60411 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-25 01:40:15 +00:00
marcandre
e565c838ce lib/ostruct.rb: Use frozen literals.
Patch adapted from Espartaco Palma. [GH-1714] [Bug #14000]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60406 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-24 18:08:15 +00:00
nobu
177856b8a7 Test for r60396
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60400 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-24 11:24:19 +00:00
ko1
5ee9513a71 Lazy Proc allocation for block parameters
[Feature #14045]

* insns.def (getblockparam, setblockparam): add special access
  instructions for block parameters.
  getblockparam checks VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM and
  if it is not set this instruction creates a Proc object from
  a given blcok and set VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM.
  setblockparam is similar to setlocal, but set
  VM_FRAME_FLAG_MODIFIED_BLOCK_PARAM.

* compile.c: use get/setblockparm instead get/setlocal instructions.
  Note that they are used for method local block parameters (def m(&b)),
  not for block local method parameters (iter{|&b|).

* proc.c (get_local_variable_ptr): creates Proc object for
  Binding#local_variable_get/set.

* safe.c (safe_setter): we need to create Proc objects for postponed
  block parameters when $SAFE is changed.

* vm_args.c (args_setup_block_parameter): used only for block local blcok
  parameters.

* vm_args.c (vm_caller_setup_arg_block): if called with
  VM_CALL_ARGS_BLOCKARG_BLOCKPARAM flag then passed block values should be
  a block handler.

* test/ruby/test_optimization.rb: add tests.

* benchmark/bm_vm1_blockparam*: added.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60397 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-24 11:13:49 +00:00
nobu
30d23ec903 multiple arguments to write
Make write methods of IO-like objects accept multiple arguments,
as well as IO#write.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60383 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-23 14:05:07 +00:00
yui-knk
8bbdbf9e75 Add test cases for branch coverage
* test/coverage/test_coverage.rb (test_branch_coverage_for_while_statement):
  Add test cases for modifier while/until.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-23 13:25:59 +00:00
naruse
9ba147dce6 OpenSSL may show the different error message
d02211c9da (commitcomment-25119729)
From: MSP-Greg <MSP-Greg@users.noreply.github.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60380 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-23 08:28:05 +00:00
nobu
585cbf6535 zlib.c: multiple arguments to write
* ext/zlib/zlib.c (rb_gzwriter_write): accepts multiple arguments.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60379 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-23 07:06:12 +00:00
nobu
aeaeb4b068 stringio.c: write multiple arguments
* ext/stringio/stringio.c (strio_write_m): make StringIO#write
  accept multiple arguments, as well as IO#write.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60377 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-23 06:25:39 +00:00
nobu
08524bc594 io.c: fix infinite retry
* io.c (io_binwritev): fix infinite retry when flushing buffered
  data.  [Feature #9323]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60373 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-23 05:28:12 +00:00
nobu
71a7ef31d7 io.c: fix buffered output
* io.c (io_binwritev): append to buffered data, not overwriting.
  [Feature #9323]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60372 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-23 05:09:35 +00:00
nobu
c0f40369b6 io.c: no restriction
* io.c (io_write_m): remove argc restriction upto IOV_MAX-1.
  [Feature #9323]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60370 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-23 02:25:58 +00:00
kou
5e64882123 rss itunes: fix a bug that <itunes:explicit> value isn't fully supported
Fix GH-1725

<itunes:explicit> accepts "explicit", "yes", "true", "clean", "no" and
"false" as valid values.

Here is the <itunes:explicit>'s description in
https://help.apple.com/itc/podcasts_connect/#/itcb54353390:

> The <itunes:explicit> tag indicates whether your podcast contains
> explicit material. You can specify the following values:
>
>   * Yes | Explicit | True. If you specify yes, explicit, or true,
>     indicating the presence of explicit content, the iTunes Store
>     displays an Explicit parental advisory graphic for your podcast.
>
>   * Clean | No | False. If you specify clean, no, or false, indicating
>     that none of your podcast episodes contain explicit language or
>     adult content, the iTunes Store displays a Clean parental
>     advisory graphic for your podcast.

I don't know whether <itunes:explicit> value is case sensitive or
insensitive. But the current implementation is case insensitive.

Reported by Valerie Woolard Srinivasan. Thanks!!!


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60367 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 16:03:57 +00:00
yui-knk
588ac3ee1f Add test cases for branch coverage
* test/coverage/test_coverage.rb (test_branch_coverage_for_if_statement):
  Add a test case for ternary operator.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60366 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 14:13:50 +00:00
yui-knk
ef29f8db80 Add test cases for branch coverage
* test/coverage/test_coverage.rb (test_branch_coverage_for_if_statement):
  Add test cases for modifier if/unless.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60365 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 13:58:55 +00:00
yui-knk
ecdac92e50 Add test cases for branch coverage
* test/coverage/test_coverage.rb (test_branch_coverage_for_if_statement):
  Add test cases for if/unless without else clauses.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60364 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 13:49:41 +00:00
yui-knk
de482ab143 test/coverage/test_coverage.rb: Use <<~ to ease to calculate column of target codes
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60363 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 13:34:24 +00:00
yui-knk
755dd9f461 Add column numbers to branch coverage
* compile.c (DECL_BRANCH_BASE, ADD_TRACE_BRANCH_COVERAGE): Add
  column to arguments.

* compile.c (compile_if, compile_case, compile_when, compile_loop, iseq_compile_each0):
  Pass column numbers to macros.

* ext/coverage/coverage.c (branch_coverage): Add column numbers to
  a return value.

* test/coverage/test_coverage.rb: Follow-up these changes.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60362 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 13:18:40 +00:00
knu
8c90432af7 Add Set#reset
This method resets the internal state of a set after modification to
existing elements, reindexing and deduplicating them. [Feature #6589]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60360 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 12:25:34 +00:00
hsbt
6693e3e723 Fixed misspelling words.
These are detected by https://github.com/client9/misspell

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60359 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 11:27:06 +00:00
kazu
dfee14d34c Use \A and \z instead of ^ and $
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60358 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 09:16:54 +00:00
yui-knk
15c78e30d6 Fix a target method of a test
* test/ruby/test_string.rb (test_delete): I guess
  this line should be a test case which asserts
  coderange is handled correctly when a result of
  String#delete has non-ASCII characters.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 07:04:00 +00:00
naruse
6888d69747 sort test result
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60344 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 02:19:52 +00:00
glass
3efa7126e5 Make IO#write accept multiple arguments
io.c: make IO#write accept multiple arguments.
it uses writev(2) if possible.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60343 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 02:11:07 +00:00
naruse
b52cdd2b44 sort the test result
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60342 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 02:08:46 +00:00
naruse
3866715356 Dir.glob with FNM_EXTGLOB is optimized [Feature #13873]
The order of resulted array is changed in some cases.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60341 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 02:03:49 +00:00
nobu
f644d3ef67 parse.y: workaround for warnings
* parse.y (mark_lvar_used): enable workaround to suppress unused
  local variables.  [ruby-core:82656] [Bug #13872]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60339 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 01:37:36 +00:00
glass
f1f7b51d73 Add arity check into Hash#flatten
* hash.c (rb_hash_flatten): add arity check

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60335 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-22 00:38:05 +00:00
suke
6f889b50d9 ext/win32ole/win32ole.c (fole_missing): set receiver in NoMethodError.
test/win32ole/test_win32ole.rb: ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60330 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 23:57:10 +00:00
sonots
bc28011f02 Fix typo in comment
* test/ruby/test_transcode.rb: fix typo in comment
  patched by larskanis (Lars Kanis) [GH-1681]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60323 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 23:32:57 +00:00
nobu
80c50308f9 Improve performance of string interpolation
This patch will add pre-allocation in string interpolation.
By this, unecessary capacity resizing is avoided.

For small strings, optimized `rb_str_resurrect` operation is
faster, so pre-allocation is done only when concatenated strings
are large.  `MIN_PRE_ALLOC_SIZE` was decided by experimenting with
local machine (x86_64-apple-darwin 16.5.0, Apple LLVM version
8.1.0 (clang - 802.0.42)).

String interpolation will be faster around 72% when large string is created.

* Before
  ```
  Calculating -------------------------------------
  Large string interpolation
                            1.276M (± 5.9%) i/s -      6.358M in   5.002022s
  Small string interpolation
                            5.156M (± 5.5%) i/s -     25.728M in   5.005731s
  ```

* After
  ```
  Calculating -------------------------------------
  Large string interpolation
                            2.201M (± 5.8%) i/s -     11.063M in   5.043724s
  Small string interpolation
                            5.192M (± 5.7%) i/s -     25.971M in   5.020516s
  ```

* Test code
  ```ruby
  require 'benchmark/ips'

  Benchmark.ips do |x|
    x.report "Large string interpolation" do |t|
      a = "Hellooooooooooooooooooooooooooooooooooooooooooooooooooo"
      b = "Wooooooooooooooooooooooooooooooooooooooooooooooooooorld"

      t.times do
        "#{a}, #{b}!"
      end
    end

    x.report "Small string interpolation" do |t|
      a = "Hello"
      b = "World"

      t.times do
        "#{a}, #{b}!"
      end
    end
  end
  ```

[Fix GH-1626]
From: Nao Minami <south37777@gmail.com>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60320 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 23:21:05 +00:00
rhe
66fd565153 openssl: merge test fix from upstream
Merge a commit from upstream:

	d1cbf6d75280 test/test_ssl_session: skip tests for session_remove_cb

Tests using SSL::SSLContext#session_remove_cb= are now skipped.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60318 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 20:26:26 +00:00
naruse
d02211c9da ignore server side error
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60316 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 16:44:56 +00:00
naruse
b39132e993 Revert "ignore server side error"
This reverts commit r60314.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60315 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-21 16:44:56 +00:00