Commit graph

1633 commits

Author SHA1 Message Date
kazu
25c1fd3b90 Reverting all commits from r67479 to r67496 because of CI failures
Because hard to specify commits related to r67479 only.
So please commit again.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67499 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-10 09:15:21 +00:00
nobu
5ae753dfa3 Adjusted styles
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67493 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-10 06:44:41 +00:00
tenderlove
e900bba945 Pin weakmap references
Weak map references can't move because the st_table needs their address
as a key.  But, we also need to remove T_NONE from the map so they
aren't reused.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67485 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-09 23:03:40 +00:00
tenderlove
3ef4db15e9 Adding GC.compact and compacting GC support.
This commit adds the new method `GC.compact` and compacting GC support.
Please see this issue for caveats:

  https://bugs.ruby-lang.org/issues/15626

[Feature #15626]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67479 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-09 20:32:04 +00:00
nobu
b7c4e6d11c gc.c: missing break
* gc.c (obj_memsize_of): T_RATIONAL and T_COMPLEX cannot be an
  imemo.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67464 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-07 11:14:06 +00:00
tenderlove
dd2479bcea Fix more ASAN errors
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67421 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-02 21:59:04 +00:00
tenderlove
be0f09a998 Extract type to sym as a new function
I will use this from another function in compaction

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67420 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-02 21:58:59 +00:00
tenderlove
e377eb0e75 Fix ASAN error
This code was trying to access memory before unpoisoning it.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67417 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-02 19:13:12 +00:00
tenderlove
4283fb42bc Poison / verify the freelist
This commit just adds poisoning around the freelist to help debugging.
Also verify that the freelist only contains T_NONE objects when checking
the heap integrity

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67416 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-02 19:13:07 +00:00
tenderlove
1ad6dde9da Fix ASAN errors when walking the heap
verify_internal_consistency_i and gc_verify_heap_page would walk the
heap, reading data from each slot, but would not unpoison the object
before reading.  This commit unpoisons the slot before reading so that
we won't get ASAN errors

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67408 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-01 23:55:02 +00:00
nobu
56557ec28a [DOC] fix markups [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67337 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-22 11:04:59 +00:00
nobu
d577019746 Early return in gc_mark_children
for types Float, Bignum and Symbol as they do not have references
and singleton classes.

[Fix GH-2091]

From: Lourens Naudé <lourens@bearmetal.eu>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67195 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-03-09 00:00:26 +00:00
naruse
e5347b1c90 Revert "Assert that union sizes match"
This reverts commit r67078.
It breaks armv7l-linux-eabihf and sparc-solaris2.11.
20190215T061708Z.diff.html.gz
20190215T072546Z.diff.html.gz

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67080 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-02-16 15:51:01 +00:00
nobu
387e8967a3 Assert that union sizes match
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67078 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-02-15 05:51:39 +00:00
ko1
91cb5c1a70 fix empty Hash memsize.
* gc.c (obj_memsize_of): ar_table ptr can be NULL (size == 0).


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67054 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-02-12 03:45:15 +00:00
ko1
8ee0a8e91a hide ar_table internals from internal.h.
* internal.h: move ar_table def to hash.c because other files
  don't need to know implementation of ar_table.

* hash.c (rb_hash_ar_table_size): added because gc.c needs to know
  the size_of(ar_table).


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66638 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-29 16:29:44 +00:00
ko1
ff8a775c57 move checking code.
* gc.c (gc_mark_ptr): this check was introduced by accidentaly
  (this is why message is "...", crazy simple) for debugging.
  However, this check is useful because if there is T_NONE
  object here, we can't know which object points to it.
  For debugging reason, I remain this checking code and
  set reasonable error message.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66514 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-23 15:02:17 +00:00
svn
84b49707d6 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66435 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-18 08:11:57 +00:00
ko1
542e63b953 rb_raw_obj_info() support T_MODULE and T_ICLASS.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66434 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-18 08:11:52 +00:00
ko1
e4c79d0d10 rename li_table->ar_table (and related names).
* internal.h: rename the following names:
  * li_table -> ar_table. "li" means linear (from linear search),
    but we use the word "array" (from data layout).
  * RHASH_ARRAY -> RHASH_AR_TABLE. AR_TABLE is more clear.
  * rb_hash_array_* -> rb_hash_ar_table_*.
  * RHASH_TABLE_P() -> RHASH_ST_TABLE_P(). more clear.
  * RHASH_CLEAR() -> RHASH_ST_CLEAR().

* hash.c: rename "linear_" prefix functions to "ar_" prefix.

* hash.c (linear_init_table): rename to ar_alloc_table.

* debug_counter.h: rename obj_hash_array to obj_hash_ar.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66390 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-14 01:10:15 +00:00
nobu
98e65d9d92 Prefer rb_check_arity when 0 or 1 arguments
Especially over checking argc then calling rb_scan_args just to
raise an ArgumentError.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66238 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-12-06 07:49:24 +00:00
shyouhei
0b899a25f5 Remove HEAP_ALIGN_LOG setting in configure.ac for OpenBSD/MirOS
The ruby setting was renamed to HEAP_PAGE_ALIGN_LOG, but the
configure.in (now configure.ac) file was not updated, so the
setting had no effect.  The configure setting is unnecessary
after OpenBSD 5.2 and MirOS has been discontinued (with the last
release being over 10 years ago), so it is better to just remove
the related configure setting.

Fix [Bug #13438]
From: Jeremy Evans <code@jeremyevans.net>



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-29 06:16:31 +00:00
svn
61c8247d55 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66004 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 18:16:54 +00:00
ko1
96990203b7 Support targetting TracePoint [Feature #15289]
* vm_trace.c (rb_tracepoint_enable_for_target): support targetting
  TracePoint. [Feature #15289]

  Tragetting TracePoint is only enabled on specified method, proc
  and so on, example: `tp.enable(target: code)`.

  `code` should be consisted of InstructionSeuqnece (iseq)
  (RubyVM::InstructionSeuqnece.of(code) should not return nil)
  If code is a tree of iseq, TracePoint is enabled on all of
  iseqs in a tree.

  Enabled tragetting TracePoints can not enabled again with
  and without target.

* vm_core.h (rb_iseq_t): introduce `rb_iseq_t::local_hooks`
  to store local hooks.
  `rb_iseq_t::aux::trace_events` is renamed to
  `global_trace_events` to contrast with `local_hooks`.

* vm_core.h (rb_hook_list_t): add `rb_hook_list_t::running`
  to represent how many Threads/Fibers are used this list.
  If this field is 0, nobody using this hooks and we can
  delete it.

  This is why we can remove code from cont.c.

* vm_core.h (rb_vm_t): because of above change, we can eliminate
  `rb_vm_t::trace_running` field.
  Also renamed from `rb_vm_t::event_hooks` to `global_hooks`.

* vm_core.h, vm.c (ruby_vm_event_enabled_global_flags): renamed
  from `ruby_vm_event_enabled_flags.

* vm_core.h, vm.c (ruby_vm_event_local_num): added to count
  enabled targetting TracePoints.

* vm_core.h, vm_trace.c (rb_exec_event_hooks): accepts
  hook list.

* vm_core.h (rb_vm_global_hooks): added for convinience.

* method.h (rb_method_bmethod_t): added to maintain Proc
  and `rb_hook_list_t` for bmethod (defined by define_method).

* prelude.rb (TracePoint#enable): extracet a keyword parameter
  (because it is easy than writing in C).
  It calls `TracePoint#__enable` internal method written in C.

* vm_insnhelper.c (vm_trace): check also iseq->local_hooks.

* vm.c (invoke_bmethod): check def->body.bmethod.hooks.

* vm.c (hook_before_rewind): check iseq->local_hooks
  and def->body.bmethod.hooks before rewind by exception.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@66003 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-26 18:16:39 +00:00
svn
e226772ab6 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65634 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-08 09:46:15 +00:00
shyouhei
39725a4db6 gc.c: avoid integer overflow at process exit
This is rather nitpicking but at the moment the process terminates,
heap_pages_final_slots overflows.

(lldb) bt
* thread #1: tid = 0xc0903, 0x00000001002b3bf7 miniruby`finalize_list(objspace=0x0000000101c09240, zombie=4329149840) + 999 at gc.c:2946, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
  * frame #0: 0x00000001002b3bf7 miniruby`finalize_list(objspace=0x0000000101c09240, zombie=4329149840) + 999 at gc.c:2946
    frame #1: 0x000000010026a69e miniruby`rb_objspace_call_finalizer(objspace=0x0000000101c09240) + 7118 at gc.c:3092
    frame #2: 0x0000000100268ac5 miniruby`rb_gc_call_finalizer_at_exit + 149 at gc.c:3015
    frame #3: 0x00000001002272bc miniruby`ruby_finalize_1 + 156 at eval.c:146
    frame #4: 0x00000001002282b6 miniruby`ruby_cleanup(ex=0) + 4070 at eval.c:238
    frame #5: 0x0000000100228785 miniruby`ruby_run_node(n=0x0000000102060ad8) + 85 at eval.c:317
    frame #6: 0x0000000100000b9c miniruby`main(argc=2, argv=0x00007fff5fbfdc38) + 124 at main.c:42
    frame #7: 0x00007fff9966a5ad libdyld.dylib`start + 1
    frame #8: 0x00007fff9966a5ad libdyld.dylib`start + 1
(lldb) p objspace->heap_pages
(rb_objspace::(anonymous struct)) $0 = {
  sorted = 0x0000000101c070b0
  allocated_pages = 24
  allocatable_pages = 0
  sorted_length = 24
  range = {
    [0] = 0x0000000102020028
    [1] = 0x00000001020dbfd0
  }
  freeable_pages = 0
  final_slots = 0
  deferred_final = 4329149840
}
(lldb)




git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65633 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-08 09:46:09 +00:00
shyouhei
9722ed0e52 suppress GCC warning about unknown sanitizers
See also: 451710726 (L1941)


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65590 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-07 04:56:24 +00:00
shyouhei
4a80c0540f adopt sanitizer API
These APIs are much like <valgrind/memcheck.h>. Use them to
fine-grain annotate the usage of our memory.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65573 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-06 10:06:07 +00:00
shyouhei
cbb6a3a6aa annotate functions to blacklist MSAN
In these functions we are intentionally reading memory address
not owned by us.  These reads should not be diagnosed.

See also [Bug #8680]
See also 451202718



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65564 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-06 05:06:20 +00:00
shyouhei
16f5b18b7a gc.c: move ASAN check to configure
Availability of attributes are checked in configure these days,
rather than compiler macros.  Also
__attribute__((no_address_safety_analysis)) is considered
deprecated in both GCC and Clang.  Use the current best practice
if available.  See also:

https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html
https://clang.llvm.org/docs/AddressSanitizer.html


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65558 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-11-06 02:57:28 +00:00
ko1
c608b2707b add casts.
* gc.c (rb_raw_obj_info): fix type mismatch specified by the following
  build log: 448634481


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65463 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-31 01:36:39 +00:00
svn
d4f1187d31 * remove trailing spaces, expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65455 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 22:12:12 +00:00
ko1
8f675cdd00 support theap for T_HASH. [Feature #14989]
* hash.c, internal.h: support theap for small Hash.
  Introduce RHASH_ARRAY (li_table) besides st_table and small Hash
  (<=8 entries) are managed by an array data structure.
  This array data can be managed by theap.
  If st_table is needed, then converting array data to st_table data.

  For st_table using code, we prepare "stlike" APIs which accepts hash value
  and are very similar to st_ APIs.

  This work is based on the GSoC achievement
  by tacinight <tacingiht@gmail.com> and refined by ko1.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65454 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 22:11:51 +00:00
svn
ca83ed8db6 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65453 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 22:03:48 +00:00
ko1
198ff42258 support theap for T_STRUCT.
* struct.c: members memory can use theap.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65452 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 22:03:42 +00:00
ko1
873d57347f support theap for T_OBJECT.
* variable.c: now instance variable space has theap supports.
  obj_ivar_heap_alloc() tries to acquire memory from theap.

* debug_counter.h: add some counters for theap.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65451 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 22:01:17 +00:00
svn
437392232a * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65450 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 21:54:13 +00:00
ko1
312b105d0e introduce TransientHeap. [Bug #14858]
* transient_heap.c, transient_heap.h: implement TransientHeap (theap).
  theap is designed for Ruby's object system. theap is like Eden heap
  on generational GC terminology. theap allocation is very fast because
  it only needs to bump up pointer and deallocation is also fast because
  we don't do anything. However we need to evacuate (Copy GC terminology)
  if theap memory is long-lived. Evacuation logic is needed for each type.

  See [Bug #14858] for details.

* array.c: Now, theap for T_ARRAY is supported.

  ary_heap_alloc() tries to allocate memory area from theap. If this trial
  sccesses, this array has theap ptr and RARRAY_TRANSIENT_FLAG is turned on.
  We don't need to free theap ptr.

* ruby.h: RARRAY_CONST_PTR() returns malloc'ed memory area. It menas that
  if ary is allocated at theap, force evacuation to malloc'ed memory.
  It makes programs slow, but very compatible with current code because
  theap memory can be evacuated (theap memory will be recycled).

  If you want to get transient heap ptr, use RARRAY_CONST_PTR_TRANSIENT()
  instead of RARRAY_CONST_PTR(). If you can't understand when evacuation
  will occur, use RARRAY_CONST_PTR().

(re-commit of r65444)


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65449 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 21:53:56 +00:00
svn
69b8ffcd5b * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65448 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 21:02:12 +00:00
ko1
7d359f9b69 revert r65444 and r65446 because of commit miss
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65447 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 21:01:55 +00:00
ko1
efe869c0e5 support theap for T_OBJECT.
* variable.c: now instance variable space has theap supports.
  obj_ivar_heap_alloc() tries to acquire memory from theap.

* debug_counter.h: add some counters for theap.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65446 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 20:49:35 +00:00
svn
f926f799e9 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65445 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 20:46:47 +00:00
ko1
90ac549fa6 introduce TransientHeap. [Bug #14858]
* transient_heap.c, transient_heap.h: implement TransientHeap (theap).
  theap is designed for Ruby's object system. theap is like Eden heap
  on generational GC terminology. theap allocation is very fast because
  it only needs to bump up pointer and deallocation is also fast because
  we don't do anything. However we need to evacuate (Copy GC terminology)
  if theap memory is long-lived. Evacuation logic is needed for each type.

  See [Bug #14858] for details.

* array.c: Now, theap for T_ARRAY is supported.

  ary_heap_alloc() tries to allocate memory area from theap. If this trial
  sccesses, this array has theap ptr and RARRAY_TRANSIENT_FLAG is turned on.
  We don't need to free theap ptr.

* ruby.h: RARRAY_CONST_PTR() returns malloc'ed memory area. It menas that
  if ary is allocated at theap, force evacuation to malloc'ed memory.
  It makes programs slow, but very compatible with current code because
  theap memory can be evacuated (theap memory will be recycled).

  If you want to get transient heap ptr, use RARRAY_CONST_PTR_TRANSIENT()
  instead of RARRAY_CONST_PTR(). If you can't understand when evacuation
  will occur, use RARRAY_CONST_PTR().


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65444 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 20:46:24 +00:00
svn
fae6c6bfd8 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65439 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 03:22:09 +00:00
ko1
54726befc3 use RARRAY_AREF() instead of RARRAY_CONST_PTR().
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65438 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-30 03:21:56 +00:00
svn
708142cabd * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65431 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-29 18:00:14 +00:00
ko1
d3edfdc7d9 use RARRAY_AREF() instead of RARRAY_CONST_PTR().
* class.c (rb_keyword_error_new): use RARRAY_AREF() because
  RARRAY_CONST_PTR() can introduce additional overhead in a futre.
  Same fixes for other files.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65430 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-29 18:00:02 +00:00
ko1
85173be41f add new counter about GC.
* debug_counter.h: add `gc_major_oldmalloc`.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65362 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-25 02:23:58 +00:00
kazu
b2b8b4a16e Fix a typo
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65361 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-25 02:07:47 +00:00
ko1
f8dbff557a add new debug_counters for GC.
* debug_counter.h: add new debug counters to count GC.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@65359 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-24 22:17:03 +00:00