Commit graph

1972 commits

Author SHA1 Message Date
Nobuyoshi Nakada
0b2fc4cf37
extlibs.rb: make symlink only if the target exists 2020-05-23 14:56:13 +09:00
Hiroshi SHIBATA
36f29b598a
Fixed wrong usage rm_rf for cleanup the default gems in sync tool 2020-05-23 07:27:42 +09:00
Nobuyoshi Nakada
2480ef904c
pure_parser.rb: get rid of an error at localed messages 2020-05-20 10:52:32 +09:00
Nobuyoshi Nakada
79292b3088
Make builtin loader sources by Ripper 2020-05-19 12:54:51 +09:00
Takashi Kokubun
b16a2aa938
Reduce code size for rb_class_of
by inlining only hot path.

=== mame/optcarrot ===

$ benchmark-driver -v --rbenv 'before --jit;after --jit' benchmark.yml --repeat-count=24 --output=all
before --jit: ruby 2.8.0dev (2020-05-18T05:21:31Z master 0e5a58b6bf) +JIT [x86_64-linux]
after --jit: ruby 2.8.0dev (2020-05-18T06:12:04Z master 0e3d71a8d1) +JIT [x86_64-linux]
last_commit=Reduce code size for rb_class_of
Calculating -------------------------------------
                                 before --jit           after --jit
Optcarrot Lan_Master.nes    71.62880463568773     70.95730063273503 fps
                            71.73973684273152     71.98447841929851
                            75.03923801841310     75.54262519509039
                            75.16300287174957     77.64029272984344
                            75.16834828625935     78.67861469580785
                            75.17670723726911     78.81879353707393
                            75.67637908020630     79.18188850392886
                            76.19843953215396     79.66484891814478
                            77.28166716118808     79.80278072861037
                            77.38509903325165     80.05859292679696
                            78.12693418455953     80.34624804808006
                            78.73654441746730     80.66326571254345
                            79.25387513454415     80.69760605740196
                            79.44137881689524     81.32053489212245
                            79.50497657368358     81.50250852553751
                            79.62401328582868     82.27544931834611
                            79.79178811723664     82.67455264522741
                            81.20275352937418     82.93857260493297
                            81.57027048640776     83.15019118788184
                            81.63373188649095     83.20728816044721
                            81.93420437766426     83.25027576772972
                            82.05716136357167     83.27072145898173
                            82.21070805525066     83.36008265822194
                            82.56924063784872     83.36112268888493

=== benchmark-driver/sinatra ===

[rps]
before: 13143.49 rps
after: 13505.70 rps

[inlined rb_class_of size]
before: 11.5K
after: 3.8K

(calculated by `dwarftree --die inlined_subroutine --flat --merge --show-size`)
2020-05-17 23:38:19 -07:00
Takashi Kokubun
0e5a58b6bf
Mark some JIT-ed functions as COLDFUNC
Put some functions which are unlikely to be called on JIT in
.text.unlikely section to improve hot code's locality.
2020-05-17 22:21:31 -07:00
Nobuyoshi Nakada
968d6df49f
Added --test-order=nosort option
Run tests in the order given in the command line.
2020-05-17 16:34:06 +09:00
Nobuyoshi Nakada
8c3a60df2a
leakchecker.rb: show test name
When multiple autoclose IO objects are leaked too.
2020-05-16 17:35:09 +09:00
Nobuyoshi Nakada
336119dfc5
extlibs.rb: fixed Downloader.cache_file call and return value
`cache_dir` is an optinal argument but not a keyword argument, and
the return value is a `Pathname`.
2020-05-14 17:19:39 +09:00
Takashi Kokubun
fcd2576290
Stop always inlining not-optimized get/setivar
As we have the optimization in _mjit_compile_ivar.erb, we don't use
these functions if we successfully optimize ivars. Therefore it's
consuming code size where we can't optimize it well.

To decrease code size, we'd better avoid inlining them.

```
$ benchmark-driver -v --rbenv 'before --jit;after --jit' benchmark.yml --repeat-count=12 --output=all
before --jit: ruby 2.8.0dev (2020-05-13T04:49:21Z master 61d451d6ce) +JIT [x86_64-linux]
after --jit: ruby 2.8.0dev (2020-05-13T05:46:59Z master a072d323db) +JIT [x86_64-linux]
Calculating -------------------------------------
                                 before --jit           after --jit
Optcarrot Lan_Master.nes     69.8567095356444      73.0360609399857 fps
                             70.3283101279035      74.6106251902062
                             72.9135395729567      75.2797344839106
                             76.6944690546044      77.5705757485036
                             77.0027971817715      78.2269524040059
                             78.5069033659736      78.4457471014131
                             80.1873218604913      80.1273811685818
                             81.3079566400696      81.7148593303302
                             81.7285827817226      81.7637234555871
                             81.7657097839637      82.4520295422720
                             81.8310114206406      82.7532354379759
                             81.9886488857863      82.8358739858206
```
2020-05-12 22:50:05 -07:00
Hiroshi SHIBATA
5b634afe4c Added test_gems.rb for setup dependencies 2020-05-13 07:54:37 +09:00
Nobuyoshi Nakada
3150b97d32 extlibs.rb: links in extracted directory
Allow to create symbolic links (if available) to share same or
updated files.
2020-05-12 15:57:47 +09:00
Nobuyoshi Nakada
d1748484e8 extlibs.rb: added variable references
Reduce duplicate parts such as package name and version numbers.
2020-05-12 15:57:47 +09:00
卜部昌平
4fca592e8c delete mk_builtin_binary.rb
To generate what is necessary via generic_erb.rb instead.
2020-05-10 16:51:10 +09:00
卜部昌平
191cfcc407 delete mk_call_iseq_optimized.rb
To generate what is necessary via generic_erb.rb instead.
2020-05-10 16:51:10 +09:00
Nobuyoshi Nakada
1181edd54c
rbuninstall.rb: show paths without destdir as well as rbinstall.rb 2020-05-09 13:45:50 +09:00
Nobuyoshi Nakada
e88bb6a2f7
rbuninstall.rb: get rid of errors when prefix ends with "/." 2020-05-09 01:43:53 +09:00
Nobuyoshi Nakada
8830d60dae
rbuninstall.rb: print output record separator instead of $\ 2020-05-09 01:40:40 +09:00
Nobuyoshi Nakada
3333d02357
rbinstall.rb: get rid of installing duplicate files
Should not overwrite ext/rubyvm/lib/forwardable/impl.rb by
lib/forwardable/impl.rb.
2020-05-09 01:22:16 +09:00
Nobuyoshi Nakada
cc3352cc41
rbinstall.rb: record bundled gem files 2020-05-09 00:55:55 +09:00
Hiroshi SHIBATA
1fe2b7f41c Workaround for Minitest5 2020-05-08 07:38:50 +09:00
Kazuhiro NISHIYAMA
22b1e824d7
Check encodings of STDIN,STDOUT,STDERR too 2020-05-08 00:12:48 +09:00
Takashi Kokubun
a5073c053f
Always correct sp on leave cancel
Even if local stack optimization is not used and values are written to
VM stack, the stack pointer itself may not be moved properly. So this
should be always moved on JIT cancellation.

By the way it's hard to write a test for this because if we try to
generate an interrupt, it will be a method call and it consumes the
interrupt by itself on popping a frame.
2020-05-06 20:26:03 -07:00
Nobuyoshi Nakada
545d2ab7d2
leakchecker.rb: try lsof 2020-05-07 02:58:28 +09:00
Nobuyoshi Nakada
de3f725978
Makes nil-safe 2020-05-06 14:09:03 +09:00
Nobuyoshi Nakada
b247ac086e
Ignore FDs kept by system library
`getaddrinfo` on macOS seems keeping FDs to query host names
internally.
2020-05-06 13:47:09 +09:00
Nobuyoshi Nakada
039a8ef786
leakchecker.rb: search /dev/fd too
It is more popular than /proc/self/fd.
2020-05-06 10:09:29 +09:00
Nobuyoshi Nakada
f0b5629c8c
envutil.rb: Also alias to #name 2020-05-05 13:53:44 +09:00
Nobuyoshi Nakada
5c2508060b
Bison 3.5.91 now defaults parse.error to "simple" 2020-05-04 18:10:02 +09:00
Nobuyoshi Nakada
47720e2255
Bison 3.59.1 folds yydestruct function header 2020-05-04 13:35:38 +09:00
Takashi Kokubun
f5ddbba9a2
Include unit id in a function name of an inlined method
I'm trying to make it possible to include all JIT-ed code in a single C
file. This is needed to guarantee uniqueness of all function names
2020-04-30 23:08:13 -07:00
Nobuyoshi Nakada
7c88bb3eba
fetch-bundled_gems.rb: Fixed the position of -c 2020-04-25 13:56:24 +09:00
Nobuyoshi Nakada
f12f9f511d
fetch-bundled_gems.rb: Quiet detached HEAD advice 2020-04-25 13:16:40 +09:00
Hiroshi SHIBATA
c2700379ad
git co is alias of git checkout 2020-04-25 12:15:01 +09:00
Nobuyoshi Nakada
d1f50b9872
Initialize the class variable for Minitest::Unit.current_repeat_count 2020-04-23 10:44:27 +09:00
Nobuyoshi Nakada
2e87488d13
Expand retrying test file names to realpath
As well as worker processes do in Test::Unit::Worker#run.
2020-04-23 10:25:09 +09:00
Nobuyoshi Nakada
3cd3a507e3
[sync_default_gems.rb] Force reset conflict files to be ignored
[ci skip]
2020-04-20 21:27:02 +09:00
Nobuyoshi Nakada
aeb9d11594
Revert irrelevant change [ci skip] 2020-04-20 21:26:55 +09:00
Nobuyoshi Nakada
e571bb8436
Removed phony atomic operations for void* and VALUE 2020-04-20 21:14:04 +09:00
Takashi Kokubun
04e56958e6
Make sure newarraykwsplat accesses a correct index
on stack when local_stack_p is enabled.

This fixes `RB_FL_TEST_RAW:"RB_FL_ABLE(obj)"` assertion failure
on power_assert's test with JIT enabled.
2020-04-18 01:41:50 -07:00
Nobuyoshi Nakada
3152977b31
Rescue EPIPE in _report
Instead of letting callers rescue the error always.
2020-04-17 14:25:20 +09:00
Nobuyoshi Nakada
3cca0d1958 core_assertions.rb: fixed fd leak at timeout 2020-04-17 00:52:29 +09:00
Nobuyoshi Nakada
e231f8e8a8 envutil.rb: not try to close pipes twice 2020-04-17 00:52:29 +09:00
Takashi Kokubun
310ef9f40b
Make vm_call_cfunc_with_frame a fastpath (#3027)
when there's no need to call CALLER_SETUP_ARG and CALLER_REMOVE_EMPTY_KW_SPLAT
(i.e. !rb_splat_or_kwargs_p(ci) && !calling->kw_splat).

Micro benchmark:
```
$ benchmark-driver -v --rbenv 'before;after' benchmark/vm_send_cfunc.yml --repeat-count=4
before: ruby 2.8.0dev (2020-04-13T23:45:05Z master b9d3ceee8f) [x86_64-linux]
after: ruby 2.8.0dev (2020-04-14T00:48:52Z no-splat-fastpath 418d363722) [x86_64-linux]
Calculating -------------------------------------
                         before       after
       vm_send_cfunc    69.585M     88.724M i/s -    100.000M times in 1.437097s 1.127096s

Comparison:
                    vm_send_cfunc
               after:  88723605.2 i/s
              before:  69584737.1 i/s - 1.28x  slower
```

Optcarrot:
```
$ benchmark-driver -v --rbenv 'before;after' benchmark.yml --repeat-count=12 --output=all
before: ruby 2.8.0dev (2020-04-13T23:45:05Z master b9d3ceee8f) [x86_64-linux]
after: ruby 2.8.0dev (2020-04-14T00:48:52Z no-splat-fastpath 418d363722) [x86_64-linux]
Calculating -------------------------------------
                                       before                 after
Optcarrot Lan_Master.nes    50.76119601545175     42.73858236484051 fps
                            50.76388649761503     51.04211379912850
                            50.80930672252514     51.39455790755538
                            50.90236000778749     51.75656936556145
                            51.01744746340430     51.86875277356489
                            51.06495279015112     51.88692482485558
                            51.07785337168974     51.93429603190578
                            51.20163525187862     51.95768145071314
                            51.34671771913112     52.45577266040274
                            51.35918340835583     52.53163888762858
                            51.46641337418146     52.62172484121034
                            51.50835463462257     52.85064021113239
```
2020-04-13 20:32:59 -07:00
Takashi Kokubun
b9d3ceee8f
Unwrap vm_call_cfunc indirection on JIT
for VM_METHOD_TYPE_CFUNC.

This has been known to decrease optcarrot fps:

```
$ benchmark-driver -v --rbenv 'before --jit;after --jit' benchmark.yml --repeat-count=24 --output=all
before --jit: ruby 2.8.0dev (2020-04-13T16:25:13Z master fb40495cd9) +JIT [x86_64-linux]
after --jit: ruby 2.8.0dev (2020-04-13T23:23:11Z mjit-inline-c bdcd06d159) +JIT [x86_64-linux]
Calculating -------------------------------------
                                 before --jit           after --jit
Optcarrot Lan_Master.nes    66.38132676191719     67.41369177299630 fps
                            69.42728743772243     68.90327567263054
                            72.16028300263211     69.62605130880686
                            72.46631319102777     70.48818243767207
                            73.37078877002490     70.79522887347566
                            73.69422431217367     70.99021920193194
                            74.01471487018695     74.69931965402584
                            75.48685183295630     74.86714575949016
                            75.54445264507932     75.97864419721677
                            77.28089738169756     76.48908637569581
                            78.04183397891302     76.54320932488021
                            78.36807984096562     76.59407262898067
                            78.92898762543574     77.31316743361343
                            78.93576483233765     77.97153484180480
                            79.13754917503078     77.98478782102325
                            79.62648945850653     78.02263322726446
                            79.86334213878064     78.26333724045934
                            80.05100635898518     78.60056756355614
                            80.26186843769584     78.91082645644468
                            80.34205717020330     79.01226659142263
                            80.62286066044338     79.32733939423721
                            80.95883033058557     79.63793060542024
                            80.97376819251613     79.73108936622778
                            81.23050939202896     80.18280109433088
```

and I deleted this capability in an early stage of YARV-MJIT development:
0ab130feee

I suspect either of the following things could be the cause:

* Directly calling vm_call_cfunc requires more optimization effort in GCC,
  resulting in 30ms-ish compilation time increase for such methods and
  decreasing the number of methods compiled in a benchmarked period.

* Code size increase => icache miss hit

These hypotheses could be verified by some methodologies. However, I'd
like to introduce this regardless of the result because this blocks
inlining C method's definition.

I may revert this commit when I give up to implement inlining C method
definition, which requires this change.

Microbenchmark-wise, this gives slight performance improvement:

```
$ benchmark-driver -v --rbenv 'before --jit;after --jit' benchmark/mjit_send_cfunc.yml --repeat-count=4
before --jit: ruby 2.8.0dev (2020-04-13T16:25:13Z master fb40495cd9) +JIT [x86_64-linux]
after --jit: ruby 2.8.0dev (2020-04-13T23:23:11Z mjit-inline-c bdcd06d159) +JIT [x86_64-linux]
Calculating -------------------------------------
                     before --jit  after --jit
     mjit_send_cfunc      41.961M      56.489M i/s -    100.000M times in 2.383143s 1.770244s

Comparison:
                  mjit_send_cfunc
         after --jit:  56489372.5 i/s
        before --jit:  41961388.1 i/s - 1.35x  slower
```
2020-04-13 16:45:05 -07:00
Hiroshi SHIBATA
55a5a4bc03
Fixed to initialize ruby-core repository when it has no master branch 2020-04-09 18:23:49 +09:00
Hiroshi SHIBATA
7a26021d1d
Support one repository for updating the latest commits 2020-04-09 16:34:25 +09:00
卜部昌平
9e6e39c351
Merge pull request #2991 from shyouhei/ruby.h
Split ruby.h
2020-04-08 13:28:13 +09:00
Takashi Kokubun
b66d7d9be5
Remove unused variable stack_size
_mjit_compile_send.erb doesn't use _mjit_compile_insn_body.erb
2020-04-06 02:00:23 -07:00
Takashi Kokubun
3194cd36e2
Delay definition of pc_moved_p
to unify the duplicated declarations and to make sure it's not used
until set properly.

Also changed it from legacy TRUE/FALSE to stdbool.
2020-04-06 01:55:18 -07:00