Commit graph

248 commits

Author SHA1 Message Date
k0kubun
8a15857a7f mjit_compile.c: use local variables for stack
if catch_except_p is FALSE. If catch_except_p is TRUE, stack values
should be on VM's stack when exception is thrown and the JIT-ed frame
is re-executed by VM's exception handler. If it's FALSE, the JIT-ed
frame won't be re-executed and don't need to keep values on VM's stack.

Using local variables allows us to reduce cfp->sp motion. Moving cfp->sp
is needed only for insns whose handles_frame? is false. So it improves
performance.

_mjit_compile_insn.erb: Prepare `stack_size` variable for GET_SP,
STACK_ADDR_FROM_TOP, TOPN macros. Share pc and sp motion partial view.
Use cancel handler created in mjit_compile.c.

_mjit_compile_send.erb: ditto. Also, when iseq->body->catch_except_p is
TRUE, this stops to call mjit_exec directly. I described the reason in
vm_insnhelper.h's comment for EXEC_EC_CFP.

_mjit_compile_pc_and_sp.erb: Shared logic for moving sp and pc. As you
can see from thsi file, when status->local_stack_p is TRUE and
insn.handles_frame? is false, moving sp is skipped. But if
insn.handles_frame? is true, values should be rolled back to VM's stack.
common.mk: add dependency for the file

_mjit_compile_insn_body.erb: Set sp value before canceling JIT on
DISPATCH_ORIGINAL_INSN. Replace GET_SP, STACK_ADDR_FROM_TOP, TOPN macros
for the case ocal_stack_p is TRUE and insn.handles_frame? is false.
In that case, values are not available on VM's stack and those macros
should be replaced.

mjit_compile.inc.erb: updated comments of macros which are supported by
JIT compiler. All references to `cfp->sp` should be replaced and thus
INC_SP, SET_SV, PUSH are no longer supported for now, because they are
not used now.

vm_exec.h: moved EXEC_EC_CFP definition to vm_insnhelper.h because it's
tighly coupled to CALL_METHOD.

vm_insnhelper.h: Have revised EXEC_EC_CFP definition moved from vm_exec.h.
Now it triggers mjit_exec for VM, and has the guard for catch_except_p
on JIT-ed code. See comments for details. CALL_METHOD delegates
triggering mjit_exec to EXEC_EC_CFP.

insns.def: Stopped using EXEC_EC_CFP for the case we don't want to
trigger mjit_exec. Those insns (defineclass, opt_call_c_function) are
not supported by JIT and it's safe to use RESTORE_REGS(), NEXT_INSN().
expandarray is changed to pass GET_SP() to replace the macro in
_mjit_compile_insn_body.erb.

vm_insnhelper.c: change to take sp for the above reason.

[close https://github.com/ruby/ruby/pull/1828]

This patch resurrects the performance which was attached in
[Feature #14235].

* Benchmark

Optcarrot (with configuration for benchmark_driver.gem)
https://github.com/benchmark-driver/optcarrot

$ benchmark-driver benchmark.yml --verbose 1 --rbenv 'before;before+JIT::before,--jit;after;after+JIT::after,--jit' --repeat-count 10
before: ruby 2.6.0dev (2018-03-04 trunk 62652) [x86_64-linux]
before+JIT: ruby 2.6.0dev (2018-03-04 trunk 62652) +JIT [x86_64-linux]
after: ruby 2.6.0dev (2018-03-04 local-variable.. 62652) [x86_64-linux]
last_commit=mjit_compile.c: use local variables for stack
after+JIT: ruby 2.6.0dev (2018-03-04 local-variable.. 62652) +JIT [x86_64-linux]
last_commit=mjit_compile.c: use local variables for stack
Calculating -------------------------------------
                         before  before+JIT       after   after+JIT
           optcarrot     53.552      59.680      53.697      63.358 fps

Comparison:
                        optcarrot
           after+JIT:        63.4 fps
          before+JIT:        59.7 fps - 1.06x  slower
               after:        53.7 fps - 1.18x  slower
              before:        53.6 fps - 1.18x  slower

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62655 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-04 07:04:40 +00:00
k0kubun
b7aae52851 vm.c: add mjit_enable_p flag
to count up total calls properly. Some places (especially CALL_METHOD)
invoke mjit_exec twice for one method call. It would be problematic when
debugging, or possibly it would result in a wrong profiling result.

This commit doesn't have impact for performance:

* Optcarrot benchmark

** before

fps: 59.37757770848619
fps: 56.49998488958699
fps: 59.07900362739362
fps: 58.924749807695996
fps: 57.667905665594894
fps: 57.540021018385254
fps: 59.5518055679647
fps: 55.93831555148311
fps: 57.82685112863262
fps: 59.22391754481736
checksum: 59662

** after

fps: 58.461881158098194
fps: 59.32685183081354
fps: 54.11334310279802
fps: 59.2281560439788
fps: 58.60495705318312
fps: 55.696478648491045
fps: 58.49003452654724
fps: 58.387771929393224
fps: 59.24156772816439
fps: 56.68804731968107
checksum: 59662

* Discourse

Your Results: (note for timings- percentile is first, duration is second in millisecs)

** before (without JIT)

categories_admin:
  50: 16
  75: 17
  90: 24
  99: 37
home_admin:
  50: 20
  75: 20
  90: 24
  99: 42
topic_admin:
  50: 16
  75: 16
  90: 18
  99: 28
categories:
  50: 36
  75: 37
  90: 45
  99: 68
home:
  50: 38
  75: 40
  90: 53
  99: 92
topic:
  50: 14
  75: 15
  90: 17
  99: 26

** after (without JIT)

categories_admin:
  50: 16
  75: 16
  90: 24
  99: 36
home_admin:
  50: 19
  75: 20
  90: 23
  99: 41
topic_admin:
  50: 16
  75: 16
  90: 19
  99: 33
categories:
  50: 35
  75: 36
  90: 44
  99: 61
home:
  50: 38
  75: 40
  90: 52
  99: 101
topic:
  50: 14
  75: 15
  90: 15
  99: 24

** before (with JIT)

categories_admin:
  50: 19
  75: 23
  90: 29
  99: 44
home_admin:
  50: 24
  75: 26
  90: 32
  99: 46
topic_admin:
  50: 20
  75: 22
  90: 27
  99: 44
categories:
  50: 41
  75: 43
  90: 51
  99: 66
home:
  50: 46
  75: 49
  90: 56
  99: 68
topic:
  50: 18
  75: 19
  90: 22
  99: 31

** after (with JIT)

categories_admin:
  50: 18
  75: 21
  90: 28
  99: 42
home_admin:
  50: 23
  75: 25
  90: 31
  99: 51
topic_admin:
  50: 19
  75: 20
  90: 24
  99: 31
categories:
  50: 41
  75: 44
  90: 52
  99: 69
home:
  50: 45
  75: 48
  90: 61
  99: 88
topic:
  50: 19
  75: 20
  90: 24
  99: 33

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62641 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-03 04:07:02 +00:00
k0kubun
3406c5d613 Refactor ERB version checking for keyword arguments
Improving code like r62590. See r62529 for details.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62594 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-27 11:12:23 +00:00
k0kubun
40db89c093 tool/ruby_vm/helpers/dumper.rb: check ERB version
I could not `make` trunk (62585) without this patch.

```
$ make -j4 && make install
        BASERUBY = /home/pocke/.rbenv/shims/ruby --disable=gems
        CC = gcc
        LD = ld
        LDSHARED = gcc -shared
        CFLAGS = -O3 -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wno-tautological-compare -Wno-parentheses-equality -Wno-constant-logical-operand -Wno-self-assign -Wunused-variable -Werror=implicit-int -Werror=pointer-arith -Werror=write-strings -Werror=declaration-after-statement -Werror=implicit-function-declaration -Werror=deprecated-declarations -Werror=misleading-indentation -Wno-overlength-strings -Wno-packed-bitfield-compat -Wsuggest-attribute=noreturn -Wsuggest-attribute=format -Wmissing-noreturn -Wimplicit-fallthrough=0 -Werror=duplicated-cond -Werror=restrict -std=gnu99  -fPIC
        XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector -fno-strict-overflow -DRUBY_DEVEL=1 -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT
        CPPFLAGS =   -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/10.0.0
        DLDFLAGS = -Wl,--compress-debug-sections=zlib -Wl,-soname,libruby.so.2.6  -fstack-protector
        SOLIBS = -lpthread -lgmp -ldl -lcrypt -lm
        LANG = en_GB.UTF-8
        LC_ALL =
        LC_CTYPE =
gcc (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

generating opt_sc.inc
generating optunifs.inc
generating insns.inc
generating insns_info.inc
Traceback (most recent call last):
        6: from ./tool/insns2vm.rb:9:in `<main>'
        5: from ./tool/insns2vm.rb:9:in `each'
        4: from ./tool/insns2vm.rb:10:in `block in <main>'
        3: from /home/pocke/ghq/github.com/ruby/ruby/tool/ruby_vm/helpers/dumper.rb:98:in `generate'
        2: from /home/pocke/ghq/github.com/ruby/ruby/tool/ruby_vm/helpers/dumper.rb:76:in `do_render'
        1: from /home/pocke/.rbenv/versions/trunk/lib/ruby/2.6.0/erb.rb:873:in `result'
/home/pocke/.rbenv/versions/trunk/lib/ruby/2.6.0/erb.rb:869:in `block in result': no implicit conversion of Hash into Integer (TypeError)
make: *** [Makefile:534: opt_sc.inc] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
        6: from ./tool/insns2vm.rb:9:in `<main>'
        5: from ./tool/insns2vm.rb:9:in `each'
        4: from ./tool/insns2vm.rb:10:in `block in <main>'
        3: from /home/pocke/ghq/github.com/ruby/ruby/tool/ruby_vm/helpers/dumper.rb:98:in `generate'
        2: from /home/pocke/ghq/github.com/ruby/ruby/tool/ruby_vm/helpers/dumper.rb:76:in `do_render'
        1: from /home/pocke/.rbenv/versions/trunk/lib/ruby/2.6.0/erb.rb:873:in `result'
/home/pocke/.rbenv/versions/trunk/lib/ruby/2.6.0/erb.rb:869:in `block in result': no implicit conversion of Hash into Integer (TypeError)
make: *** [Makefile:534: optunifs.inc] Error 1
Traceback (most recent call last):
        6: from ./tool/insns2vm.rb:9:in `<main>'
        5: from ./tool/insns2vm.rb:9:in `each'
        4: from ./tool/insns2vm.rb:10:in `block in <main>'
        3: from /home/pocke/ghq/github.com/ruby/ruby/tool/ruby_vm/helpers/dumper.rb:98:in `generate'
        2: from /home/pocke/ghq/github.com/ruby/ruby/tool/ruby_vm/helpers/dumper.rb:76:in `do_render'
        1: from /home/pocke/.rbenv/versions/trunk/lib/ruby/2.6.0/erb.rb:873:in `result'
/home/pocke/.rbenv/versions/trunk/lib/ruby/2.6.0/erb.rb:869:in `block in result': no implicit conversion of Hash into Integer (TypeError)
Traceback (most recent call last):
        6: from ./tool/insns2vm.rb:9:in `<main>'
        5: from ./tool/insns2vm.rb:9:in `each'
        4: from ./tool/insns2vm.rb:10:in `block in <main>'
        3: from /home/pocke/ghq/github.com/ruby/ruby/tool/ruby_vm/helpers/dumper.rb:98:in `generate'
        2: from /home/pocke/ghq/github.com/ruby/ruby/tool/ruby_vm/helpers/dumper.rb:76:in `do_render'
        1: from /home/pocke/.rbenv/versions/trunk/lib/ruby/2.6.0/erb.rb:873:in `result'
/home/pocke/.rbenv/versions/trunk/lib/ruby/2.6.0/erb.rb:869:in `block in result': no implicit conversion of Hash into Integer (TypeError)
make: *** [Makefile:534: insns.inc] Error 1
make: *** [Makefile:534: insns_info.inc] Error 1
```

I guess this issue is same as https://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/62531
So I applied the same change to tool/ruby_vm/helpers/dumper.rb also.

close https://github.com/ruby/ruby/pull/1826

Co-authored-by: Masataka Pocke Kuwabara <kuwabara@pocke.me>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62590 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-26 14:42:39 +00:00
k0kubun
cc777d09f4 erb.rb: deprecate safe_level of ERB.new
Also, as it's in the middle of the list of 4 arguments, 3rd and 4th arguments
(trim_mode, eoutvar) are changed to keyword arguments.
Old ways to specify arguments are deprecated and warned now.

bin/erb: deprecate -S option.

We'll remove all of deprecated ones at Ruby 2.7+.

enc/make_encmake.rb: stopped using deprecated interface
ext/etc/mkconstants.rb: ditto
ext/socket/mkconstants.rb: ditto
sample/ripper/ruby2html.rb: ditto
spec/ruby/library/erb/defmethod/def_erb_method_spec.rb: ditto
spec/ruby/library/erb/new_spec.rb: ditto
test/erb/test_erb.rb: ditto
test/erb/test_erb_command.rb: ditto
tool/generic_erb.rb: ditto
tool/ruby_vm/helpers/dumper.rb: ditto
tool/transcode-tblgen.rb: ditto
lib/rdoc/erbio.rb: ditto
lib/rdoc/generator/darkfish.rb: ditto

[Feature #14256]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62529 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-22 13:28:25 +00:00
k0kubun
ddb65f0b03 mjit_compile.inc.erb: replace opt_key insn
with opt_send_without_block insn if call cache has valid ISeq.
If the receiver is not optimized target of opt_key (i.e. Hash or Array),
it triggers JIT cancel and it would be slow.

This change allows JIT to drop the check for Hash/Array and continue to
execute JIT even if the receiver is not Hash or Array.

See the following benchmark results. It's not improved so much, but it
would be effective when we achieve Ruby method inlining in
_mjit_compile_send.erb.

* Micro benchmark

Given the following bench.rb,

```
class HashWithIndifferentAccess < Hash
  def []=(key, value)
    super(key.to_s, value)
  end

  def [](key)
    super(key.to_s)
  end
end

indhash = HashWithIndifferentAccess.new
indhash[:foo] = 'bar'
key = 'foo'

100000000.times do
  indhash[key]
end
```

** before

```
$ time ./ruby --disable-gems --jit-verbose=1 /tmp/bench.rb
JIT success (31.4ms): block in <main>@/tmp/bench.rb:15 -> /tmp/_ruby_mjit_p18206u0.c
JIT success (669.3ms): []@/tmp/bench.rb:6 -> /tmp/_ruby_mjit_p18206u1.c
Successful MJIT finish
./ruby --disable-gems --jit-verbose=1 /tmp/bench.rb  12.21s user 0.04s system 107% cpu 11.394 total
```

** after

```
$ time ./ruby --disable-gems --jit-verbose=1 /tmp/bench.rb
JIT success (41.0ms): block in <main>@/tmp/bench.rb:15 -> /tmp/_ruby_mjit_p17293u0.c
JIT success (679.0ms): []@/tmp/bench.rb:6 -> /tmp/_ruby_mjit_p17293u1.c
Successful MJIT finish
./ruby --disable-gems --jit-verbose=1 /tmp/bench.rb  11.54s user 0.06s system 108% cpu 10.726 total
```

The execution time is shortened.

* optcarrot benchmark

Optcarrot has no room to be improved by this change. Almost nothing is changed.

fps: 59.54 (before) -> 59.51 (after)

* discourse benchmark

I expected this to be improved a little, but it isn't too.

** before (JIT)

```
categories_admin:
  50: 12
  75: 13
  90: 14
  99: 22
home_admin:
  50: 12
  75: 13
  90: 16
  99: 22
topic_admin:
  50: 12
  75: 13
  90: 15
  99: 21
categories:
  50: 18
  75: 19
  90: 23
  99: 27
home:
  50: 3
  75: 4
  90: 4
  99: 12
topic:
  50: 11
  75: 11
  90: 14
  99: 20
```

** after (JIT)

```
categories_admin:
  50: 12
  75: 12
  90: 16
  99: 24
home_admin:
  50: 12
  75: 12
  90: 14
  99: 21
topic_admin:
  50: 12
  75: 13
  90: 16
  99: 21
categories:
  50: 17
  75: 18
  90: 23
  99: 32
home:
  50: 3
  75: 4
  90: 4
  99: 10
topic:
  50: 11
  75: 12
  90: 13
  99: 20
```

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62398 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-13 15:58:38 +00:00
nobu
9bb7cb7892 use <<- instead of <<~ for older baseruby
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62386 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-12 09:02:30 +00:00
k0kubun
268d9236f6 _mjit_compile_insn_body.erb: refactor
renamed from tool/ruby_vm/views/_mjit_compile_insn_line.erb.
Basically this file should handle everything about macro on JIT.

_mjit_compile_insn.erb: follow the refactoring

common.mk: follow the rename

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62384 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-12 07:27:48 +00:00
k0kubun
f6cebb97e9 _mjit_compile_insn_line.erb: carve out macro expansion
in _mjit_compile_insn.erb to this file. As I'm going to add macro
expansions later, I want to separate such complex things from whole insn
compilation.

_mjit_compile_insn.erb: _mjit_compile_insn_line.erb part was removed.

common.mk: updated build system for them.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62383 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-12 06:54:25 +00:00
k0kubun
dd64636a85 mjit.c: print JIT failure
mjit_compile.inc.erb: show unsupported insn name on --jit-verbose=1 too.
Also, removed osboleted workaround. Now some insn-related functions are
declared with MAYBE_UNUSED.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62382 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-12 05:04:24 +00:00
k0kubun
ebda42fa50 _mjit_compile_send.erb: resolve warnings on clang
/var/folders/tp/g1f_46nj7md_d5_j799rfb140000gn/T//_ruby_mjit_p17396u162.c:2920:143: warning: incompatible integer
      to pointer conversion passing 'long' to parameter of type 'const VALUE *' (aka 'const unsigned long *')
      [-Wint-conversion]
  ...| VM_ENV_FLAG_LOCAL, calling.recv, calling.block_handler, 0x7fc14ceee660, 0x7fc14c870c00, argv + 0, 2, 3);
                                                                               ^~~~~~~~~~~~~~
/Users/k0kubun/.rbenv/versions/ruby-svn/include/ruby-2.6.0/x86_64-darwin16/rb_mjit_min_header-2.6.0.h:21299:21: note:
      passing argument to parameter 'pc' here
       const VALUE *pc,
                    ^
/var/folders/tp/g1f_46nj7md_d5_j799rfb140000gn/T//_ruby_mjit_p17396u162.c:3073:31: warning: incompatible integer
      to pointer conversion passing 'long' to parameter of type 'const rb_iseq_t *' (aka
      'const struct rb_iseq_struct *') [-Wint-conversion]
            vm_push_frame(ec, 0x7fc14c87d290, VM_FRAME_MAGIC_METHOD | VM_ENV_FLAG_LOCAL, calling.recv, ca...
                              ^~~~~~~~~~~~~~
/Users/k0kubun/.rbenv/versions/ruby-svn/include/ruby-2.6.0/x86_64-darwin16/rb_mjit_min_header-2.6.0.h:21294:25: note:
      passing argument to parameter 'iseq' here
       const rb_iseq_t *iseq,
                        ^

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62368 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-11 12:21:13 +00:00
nobu
8d590c99ed _mjit_compile_insn.erb: comment_id
* tool/ruby_vm/views/_mjit_compile_insn.erb: comment ID of
  constant, calling method, and Symbol literal.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-11 02:41:50 +00:00
nobu
4854b14bc3 _mjit_compile_{insn,send}.erb: suppress warnings
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62353 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-11 02:41:49 +00:00
nobu
2b79ab512c _mjit_compile_send.erb: PRI_SERIALT_PREFIX
* tool/ruby_vm/views/_mjit_compile_send.erb: `printf` modifier for
  `rb_serial_t` which may not be `long long`, and '%ll' may not be
  supported.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62351 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-11 00:19:03 +00:00
nobu
0e1a5ece62 mjit_compile.c: original_body_iseq
* mjit_compile.c (mjit_compile): name the original iseq pointer to
  eliminate magic numbers.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62340 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-10 01:28:51 +00:00
k0kubun
288b44328d Export some missing symbols for MJIT
tool/ruby_vm/views/_insn_name_info.erb: on Linux, rb_vm_insn_name_offset
was needed to compile with --jit-debug (Usually --jit-debug requires
more symbols than the situation without --jit-debug because -O2 skips
some functions to compile).

vm.c: when running transform_mjit_header.rb with --jit-wait,
rb_source_location_cstr was repoted to be missing.

string.c: ditto, for rb_str_eql
numeric.c: ditto, for rb_float_eql

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62313 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-08 13:54:37 +00:00
k0kubun
d409017bc6 mjit_compile.c: fix inconsistent indentation
and style of generated code.

I've used 2-space indentation at first but at some moment I started to
use insns.def contents for generated code. So the 4-space indentation
was introduced. But it does no longer make sense.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62255 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-06 14:27:15 +00:00
kazu
5a68bd7f00 Fix a typo [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62228 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-05 12:34:35 +00:00
k0kubun
ed935aa5be mjit_compile.c: merge initial JIT compiler
which has been developed by Takashi Kokubun <takashikkbn@gmail> as
YARV-MJIT. Many of its bugs are fixed by wanabe <s.wanabe@gmail.com>.

This JIT compiler is designed to be a safe migration path to introduce
JIT compiler to MRI. So this commit does not include any bytecode
changes or dynamic instruction modifications, which are done in original
MJIT.

This commit even strips off some aggressive optimizations from
YARV-MJIT, and thus it's slower than YARV-MJIT too. But it's still
fairly faster than Ruby 2.5 in some benchmarks (attached below).

Note that this JIT compiler passes `make test`, `make test-all`, `make
test-spec` without JIT, and even with JIT. Not only it's perfectly safe
with JIT disabled because it does not replace VM instructions unlike
MJIT, but also with JIT enabled it stably runs Ruby applications
including Rails applications.

I'm expecting this version as just "initial" JIT compiler. I have many
optimization ideas which are skipped for initial merging, and you may
easily replace this JIT compiler with a faster one by just replacing
mjit_compile.c. `mjit_compile` interface is designed for the purpose.

common.mk: update dependencies for mjit_compile.c.

internal.h: declare `rb_vm_insn_addr2insn` for MJIT.

vm.c: exclude some definitions if `-DMJIT_HEADER` is provided to
compiler. This avoids to include some functions which take a long time
to compile, e.g. vm_exec_core. Some of the purpose is achieved in
transform_mjit_header.rb (see `IGNORED_FUNCTIONS`) but others are
manually resolved for now. Load mjit_helper.h for MJIT header.
mjit_helper.h: New. This is a file used only by JIT-ed code. I'll
refactor `mjit_call_cfunc` later.
vm_eval.c: add some #ifdef switches to skip compiling some functions
like Init_vm_eval.

win32/mkexports.rb: export thread/ec functions, which are used by MJIT.

include/ruby/defines.h: add MJIT_FUNC_EXPORTED macro alis to clarify
that a function is exported only for MJIT.

array.c: export a function used by MJIT.
bignum.c: ditto.
class.c: ditto.
compile.c: ditto.
error.c: ditto.
gc.c: ditto.
hash.c: ditto.
iseq.c: ditto.
numeric.c: ditto.
object.c: ditto.
proc.c: ditto.
re.c: ditto.
st.c: ditto.
string.c: ditto.
thread.c: ditto.
variable.c: ditto.
vm_backtrace.c: ditto.
vm_insnhelper.c: ditto.
vm_method.c: ditto.

I would like to improve maintainability of function exports, but I
believe this way is acceptable as initial merging if we clarify the
new exports are for MJIT (so that we can use them as TODO list to fix)
and add unit tests to detect unresolved symbols.
I'll add unit tests of JIT compilations in succeeding commits.

Author: Takashi Kokubun <takashikkbn@gmail.com>
Contributor: wanabe <s.wanabe@gmail.com>

Part of [Feature #14235]

---

* Known issues
  * Code generated by gcc is faster than clang. The benchmark may be worse
    in macOS. Following benchmark result is provided by gcc w/ Linux.
  * Performance is decreased when Google Chrome is running
  * JIT can work on MinGW, but it doesn't improve performance at least
    in short running benchmark.
  * Currently it doesn't perform well with Rails. We'll try to fix this
    before release.

---

* Benchmark reslts

Benchmarked with:
Intel 4.0GHz i7-4790K with 16GB memory under x86-64 Ubuntu 8 Cores

- 2.0.0-p0: Ruby 2.0.0-p0
- r62186: Ruby trunk (early 2.6.0), before MJIT changes
- JIT off: On this commit, but without `--jit` option
- JIT on: On this commit, and with `--jit` option

** Optcarrot fps

Benchmark: https://github.com/mame/optcarrot

|         |2.0.0-p0 |r62186   |JIT off  |JIT on   |
|:--------|:--------|:--------|:--------|:--------|
|fps      |37.32    |51.46    |51.31    |58.88    |
|vs 2.0.0 |1.00x    |1.38x    |1.37x    |1.58x    |

** MJIT benchmarks

Benchmark: https://github.com/benchmark-driver/mjit-benchmarks
(Original: https://github.com/vnmakarov/ruby/tree/rtl_mjit_branch/MJIT-benchmarks)

|           |2.0.0-p0 |r62186   |JIT off  |JIT on   |
|:----------|:--------|:--------|:--------|:--------|
|aread      |1.00     |1.09     |1.07     |2.19     |
|aref       |1.00     |1.13     |1.11     |2.22     |
|aset       |1.00     |1.50     |1.45     |2.64     |
|awrite     |1.00     |1.17     |1.13     |2.20     |
|call       |1.00     |1.29     |1.26     |2.02     |
|const2     |1.00     |1.10     |1.10     |2.19     |
|const      |1.00     |1.11     |1.10     |2.19     |
|fannk      |1.00     |1.04     |1.02     |1.00     |
|fib        |1.00     |1.32     |1.31     |1.84     |
|ivread     |1.00     |1.13     |1.12     |2.43     |
|ivwrite    |1.00     |1.23     |1.21     |2.40     |
|mandelbrot |1.00     |1.13     |1.16     |1.28     |
|meteor     |1.00     |2.97     |2.92     |3.17     |
|nbody      |1.00     |1.17     |1.15     |1.49     |
|nest-ntimes|1.00     |1.22     |1.20     |1.39     |
|nest-while |1.00     |1.10     |1.10     |1.37     |
|norm       |1.00     |1.18     |1.16     |1.24     |
|nsvb       |1.00     |1.16     |1.16     |1.17     |
|red-black  |1.00     |1.02     |0.99     |1.12     |
|sieve      |1.00     |1.30     |1.28     |1.62     |
|trees      |1.00     |1.14     |1.13     |1.19     |
|while      |1.00     |1.12     |1.11     |2.41     |

** Discourse's script/bench.rb

Benchmark: https://github.com/discourse/discourse/blob/v1.8.7/script/bench.rb

NOTE: Rails performance was somehow a little degraded with JIT for now.
We should fix this.
(At least I know opt_aref is performing badly in JIT and I have an idea
 to fix it. Please wait for the fix.)

*** JIT off
Your Results: (note for timings- percentile is first, duration is second in millisecs)

categories_admin:
  50: 17
  75: 18
  90: 22
  99: 29
home_admin:
  50: 21
  75: 21
  90: 27
  99: 40
topic_admin:
  50: 17
  75: 18
  90: 22
  99: 32
categories:
  50: 35
  75: 41
  90: 43
  99: 77
home:
  50: 39
  75: 46
  90: 49
  99: 95
topic:
  50: 46
  75: 52
  90: 56
  99: 101

*** JIT on
Your Results: (note for timings- percentile is first, duration is second in millisecs)

categories_admin:
  50: 19
  75: 21
  90: 25
  99: 33
home_admin:
  50: 24
  75: 26
  90: 30
  99: 35
topic_admin:
  50: 19
  75: 20
  90: 25
  99: 30
categories:
  50: 40
  75: 44
  90: 48
  99: 76
home:
  50: 42
  75: 48
  90: 51
  99: 89
topic:
  50: 49
  75: 55
  90: 58
  99: 99

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62197 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-04 11:22:28 +00:00
nobu
d4d671f2b0 bare_instructions.rb: sp_inc is signed
* tool/ruby_vm/models/bare_instructions.rb (predefine_attributes):
  `sp_inc` attribute which may return negative values must be
  signed `rb_snum_t`, to be signed-expanded at type promotion.

* vm_insnhelper.h (ADJ_SP): removed the workaround for platforms
  where rb_num_t is wider than int.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62103 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-30 03:21:52 +00:00
shyouhei
3bc2f5a9b6 redefinition of a typedef is a C11ism
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62090 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-29 08:11:02 +00:00
shyouhei
31ecd18f1a s/CALL_SIMPLE_METHOD/DISPATCH_ORIGINAL_INSN/
Now that DISPATCH_ORIGINAL_INSN is introduced, we can replace
CALL_SIMPLE_METHOD with DISPATCH_ORIGINAL_INSN. These two macros
differ in size very much and results in this big difference in
compiled binary size. This changeset reduces the size of
vm_exec_core from 32,352 bytes to 27,008 bytes on my machine.  As
a result it yields slightly better performance.
Closes [GH-1779].

-----------------------------------------------------------
benchmark results:
minimum results in each 3 measurements.
Execution time (sec)
name    before  after
so_ackermann     0.484  0.454
so_array         0.837  0.779
so_binary_trees  5.928  5.801
so_concatenate   3.473  3.543
so_count_words   0.201  0.222
so_exception     0.255  0.252
so_fannkuch      1.080  1.019
so_fasta         1.459  1.463
so_k_nucleotide  1.218  1.180
so_lists         0.499  0.484
so_mandelbrot    2.189  2.324
so_matrix        0.510  0.496
so_meteor_contest        3.025  2.925
so_nbody         1.319  1.273
so_nested_loop   0.941  0.932
so_nsieve        1.806  1.647
so_nsieve_bits   2.151  2.078
so_object        0.632  0.621
so_partial_sums  1.560  1.632
so_pidigits      1.190  1.183
so_random        0.333  0.353
so_reverse_complement    0.604  0.586
so_sieve         0.521  0.481
so_spectralnorm  1.774  1.722

Speedup ratio: compare with the result of `before' (greater is better)
name    after
so_ackermann    1.065
so_array        1.075
so_binary_trees 1.022
so_concatenate  0.980
so_count_words  0.903
so_exception    1.009
so_fannkuch     1.059
so_fasta        0.997
so_k_nucleotide 1.032
so_lists        1.032
so_mandelbrot   0.942
so_matrix       1.028
so_meteor_contest       1.034
so_nbody        1.036
so_nested_loop  1.009
so_nsieve       1.097
so_nsieve_bits  1.035
so_object       1.018
so_partial_sums 0.956
so_pidigits     1.006
so_random       0.943
so_reverse_complement   1.032
so_sieve        1.083
so_spectralnorm 1.030

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62088 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-29 07:04:50 +00:00
shyouhei
7d4ad74f22 also use sp_inc in vm core
Now that sp_inc attributes are officially provided as inline
functions. Why not use them directly from the vm core, not just
by the compiler. By doing so, it is now possible for us to
optimize stack manipulations. We can now know exactly how many
words of stack space an instruction consumes before it actually
does. This changeset deletes some lines from insns.def because
they are no longer needed.  As a result it reduces the size of
vm_exec_core function from 32,400 bytes to 32,352 bytes on my
machine.

It seems it does not affect performance:

-----------------------------------------------------------
benchmark results:
minimum results in each 3 measurements.
Execution time (sec)
name    before  after
loop_for         1.093  1.061
loop_generator   1.156  1.152
loop_times       0.982  0.974
loop_whileloop   0.549  0.587
loop_whileloop2  0.115  0.121

Speedup ratio: compare with the result of `before' (greater is better)
name    after
loop_for        1.030
loop_generator  1.003
loop_times      1.008
loop_whileloop  0.935
loop_whileloop2 0.949

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62087 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-29 06:56:56 +00:00
shyouhei
9a8b38cacf extensive use of instruction attributes
Instead of using magic numbers, let us define a series of attributes
and use them from the VM core.  Proper function declarations makes
these attributes inlined in most modern compilers.  On my machine
exact same binary is generated with or without this changeset.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62085 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-29 06:47:05 +00:00
k0kubun
fe18a62347 bare_instructions.rb: show class name on inspect
as it's helpful for debugging.

I'm not sure what's the good output for RubyVM::TraceInstructions, so I
left it as it is.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62068 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-27 04:14:29 +00:00
k0kubun
3e458f15c8 insns2vm.rb: add missing word to help
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62066 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-27 02:14:20 +00:00
k0kubun
c8e5796f61 insns2vm.rb: make #generate_parser private
which is only used by #router.

To make it private, we need to stop defining them on top-level.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62065 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-27 02:14:20 +00:00
k0kubun
be27efdb83 dumper.rb: stop getting --destdir option value
via current directory. It's kind of a global state and fragile to
implementation changes in other places, and how the `Dir.getwd` is set
is not obvious from just reading around `RubyVM::Dumper#initialize` if
it depends on the global state.

tool/ruby_vm/controllers/application_controller.rb: explicitly pass
destdir to RubyVM::Dumper.

tool/ruby_vm/scripts/insns2vm.rb: explicitly pass destdir parsed from
optparse.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62064 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-27 01:59:08 +00:00
shyouhei
3234245ae3 move ADD_PC around to optimize PC manipluiations
This commit introduces new attribute handles_flame and if that is
_not_ the case, places ADD_PC right after INC_SP.  This improves
locality of PC manipulations to prevents unnecessary register spill-
outs. As a result, it reduces the size of vm_exec_core from 32,688
bytes to 32,384 bytes on my machine.

Speedup is very faint, but certain.

-----------------------------------------------------------
benchmark results:
minimum results in each 3 measurements.
Execution time (sec)
name    before  after
so_ackermann     0.476  0.464
so_array         0.742  0.728
so_binary_trees  5.493  5.466
so_concatenate   3.619  3.395
so_count_words   0.190  0.184
so_exception     0.249  0.239
so_fannkuch      0.994  0.953
so_fasta         1.369  1.374
so_k_nucleotide  1.111  1.111
so_lists         0.470  0.481
so_mandelbrot    2.059  2.050
so_matrix        0.466  0.465
so_meteor_contest        2.712  2.781
so_nbody         1.154  1.204
so_nested_loop   0.852  0.846
so_nsieve        1.636  1.623
so_nsieve_bits   2.073  2.039
so_object        0.616  0.584
so_partial_sums  1.464  1.481
so_pidigits      1.075  1.082
so_random        0.321  0.317
so_reverse_complement    0.555  0.558
so_sieve         0.495  0.490
so_spectralnorm  1.634  1.627

Speedup ratio: compare with the result of `before' (greater is better)
name    after
so_ackermann    1.025
so_array        1.019
so_binary_trees 1.005
so_concatenate  1.066
so_count_words  1.030
so_exception    1.040
so_fannkuch     1.043
so_fasta        0.996
so_k_nucleotide 1.000
so_lists        0.978
so_mandelbrot   1.004
so_matrix       1.001
so_meteor_contest       0.975
so_nbody        0.959
so_nested_loop  1.007
so_nsieve       1.008
so_nsieve_bits  1.017
so_object       1.056
so_partial_sums 0.989
so_pidigits     0.994
so_random       1.014
so_reverse_complement   0.996
so_sieve        1.010
so_spectralnorm 1.004

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62051 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-26 06:30:58 +00:00
nobu
55af62599a tool/ruby_vm/models/attribute.rb: void for empty arguments
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61838 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-15 05:27:11 +00:00
shyouhei
90cb7d3048 __VA_ARGS__ is a C99ism
give up CALL_ATTRIBUTE macro.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61802 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-13 10:47:33 +00:00
shyouhei
84012cd19f merge revision: 61746
`signed` is required for Rasbian (x86_64).

* tool/ruby_vm/views/_insn_stack_increase.erb: specify `signed` explicitly
  for systems which use `unsigned` for `char` type.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61792 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-12 13:14:53 +00:00
shyouhei
fbacf5faad tool/ruby_vm support for pre-2.0 BASERUBY
This was not requested :) but actually easier than the previous
so I just did it anyway.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61787 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-12 08:38:12 +00:00
shyouhei
8a72c77c79 tool/ruby_vm support for pre-2.1 BASERUBY
as requested by devs, support for BASERUBY prior to 2.1

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61786 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-12 08:38:11 +00:00
shyouhei
069e9ff52c tool/ruby_vm support for pre-2.3 BASERUBY
as requested by devs, support for BASERUBY prior to 2.3

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61785 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-12 08:38:10 +00:00
shyouhei
f6ea376317 delete tool/instruction.rb (2nd try)
Previous commit changed insns.def format. Now is the time for its
generators. In doing so I chose to modernize the system, not just
patch.  My attempt includes

- extensive use of Onigumo regular expressions
- split from one big file (instruction.rb) into separated MVC
- partial view

Also, let me take this opportunity to kill old unused features
such as

- stack caching
- minsns / yasmdata which are never seriously used
- yarvarch document generation (moved to doc/)
- vast majority of unused arguments to insns2vm.rb

This commit generates VM source codes that cleanly compile, and
the generated binary passes tests.  At least for me.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61784 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-12 08:38:09 +00:00
shyouhei
e2b7cb9d32 new insns.def format (2nd try)
- Gave up @j comments
- Room for sp_inc to be a proper grammer element

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61782 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-12 08:38:07 +00:00
shyouhei
5ad95486e6 merge revisions 61753:61750 61747:61740 61737:61728
Revert all the VM generator rewrites; requested by naruse


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61755 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-10 01:53:24 +00:00
hsbt
2340d5e7ea Fixed typos.
* tool/ruby_vm/loaders/opt_operand_def.rb: grammer -> grammar
  * tool/ruby_vm/scripts/insns2vm.rb: beginnig -> beginning

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61752 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-10 01:00:21 +00:00
ko1
39c2149840 signed is required for Rasbian (x86_64).
* tool/ruby_vm/views/_insn_stack_increase.erb: specify `signed` explicitly
  for systems which use `unsigned` for `char` type.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61746 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 16:27:50 +00:00
naruse
0533d9a25e grep_v is ruby 2.3 feature
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61744 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 15:47:53 +00:00
naruse
f5b459c2ed Here document <<~ is Ruby 2.3 feature
baseruby should support wider ruby versions

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61741 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 15:41:10 +00:00
shyouhei
8a233a2b0b insns_info.inc.tmpl: share large tablese
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61736 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 13:30:35 +00:00
shyouhei
e43fec9a6f insns_info.inc: position independent
* template/insns_info.inc.tmpl (insn_name_info): make position
  independent for large strings.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61735 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 13:30:34 +00:00
shyouhei
30bd46a358 comma at the end of enum is a C99ism.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61734 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 13:30:33 +00:00
shyouhei
55317a74f7 delete tool/instruction.rb
Previous commit changed insns.def format. Now is the time for its
generators. In doing so I chose to modernize the system, not just
patch.  My attempt includes

- extensive use of Onigumo regular expressions
- split from one big file (instruction.rb) into separated MVC
- partial view

Also, let me take this opportunity to kill old unused features
such as

- stack caching
- minsns / yasmdata which are never seriously used
- yarvarch document generation (moved to doc/)
- vast majority of unused arguments to insns2vm.rb

This commit generates VM source codes that cleanly compile, and
the generated binary passes tests.  At least for me.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61733 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 13:30:31 +00:00
svn
7d02105e69 * append newline at EOF.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61731 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 13:30:30 +00:00
shyouhei
89df12d849 new insns.def format
- Gave up @j comments
- Room for sp_inc to be a proper grammer element

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61729 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 13:30:28 +00:00