Commit graph

124 commits

Author SHA1 Message Date
Jean Boussier
5001cc4716 Optimize ObjectSpace.dump_all
The two main optimization are:
  - buffer writes for improved performance
  - avoid formatting functions when possible

```

|                   |compare-ruby|built-ruby|
|:------------------|-----------:|---------:|
|dump_all_string    |       1.038|   195.925|
|                   |           -|   188.77x|
|dump_all_file      |      33.453|   139.645|
|                   |           -|     4.17x|
|dump_all_dev_null  |      44.030|   278.552|
|                   |           -|     6.33x|
```
2020-09-09 11:11:36 -07:00
Jean Boussier
b49a870414 Add a :since option to dump_all
This is useful to see what a block of code allocated, e.g.

```
GC.start
GC.disable
ObjectSpace.trace_object_allocations do
  # run some code
end
gc_gen = GC.count
allocations = ObjectSpace.dump_all(output: :file, since: gc_gen)
GC.enable
GC.start
retentions = ObjectSpace.dump_all(output: :file, since: gc_gen)
```
2020-09-09 08:05:14 -07:00
Jean Boussier
a74df67244 Fix ObjectSpace.trace_object_allocations_stop to not raise if the tracepoint were not initialized 2020-08-19 08:13:09 -07:00
Aaron Patterson
7226f76255
Reset object allocation trace information
Compaction makes collisions in the weak map more likely.  For now, clear
the tracing information before the test run so we get accurate results
2020-08-18 19:57:45 -07:00
Aaron Patterson
7a58ee9b96 Update references when tracing allocations
The allocation tracing code keeps essentially a weak reference to
objects that have been allocated (storing the allocation information
along with the weak ref).  Compacting the heap would break references in
this weak map, so the wrong values could be returned.

This commit just updates the values in the weak ref in order to fix the
allocation tracing book keeping
2020-08-18 19:10:51 -07:00
Koichi Sasada
bc00e6aa90 Disable debugging test.
```
  1) Failure:
TestObjSpace#test_trace_object_allocations [/tmp/ruby/v3/src/trunk-gc_compact/test/objspace/test_objspace.rb:175]:
<nil> expected but was
<"/tmp/ruby/v3/src/trunk-gc_compact/test/objspace/test_objspace.rb">.
```

@tenderlove will re-enable it soon.
2020-08-18 02:23:12 +09:00
John Hawthorn
971857c332 Fix method name escaping in ObjectSpace.dump
It's possible to define methods with any name, even if the parser
doesn't support it and it can only be used with ex. send.

This fixes an issue where invalid JSON was output from ObjectSpace.dump
when a method name needed escaping.
2020-08-17 09:47:53 -07:00
Nobuyoshi Nakada
27f7b047e0
Also escape DEL code 2020-08-17 22:36:48 +09:00
Nobuyoshi Nakada
7b4b5e0840
Fixed the radix for control chars 2020-08-17 22:30:26 +09:00
Nobuyoshi Nakada
347639ad8a
Removed fragile tests in https://github.com/ruby/ruby/pull/3349 2020-07-23 14:57:05 +09:00
Jean Boussier
6a0cb1d649 Avoid allocating a string when dumping an anonymous module or class 2020-07-23 10:52:30 +09:00
Nobuyoshi Nakada
409b0ec4c3
Suppress unused variable warnings 2019-06-30 11:45:37 +09:00
Nobuyoshi Nakada
c9bbcaccf2
Suppress unused variable warning with RUBYOPT=-w 2019-06-28 12:27:11 +09:00
Nobuyoshi Nakada
ff7f71b288
O_EXCL has no meaning for fdopen
"exclusive access mode is not supported" exception has resulted in
empty "rubyheap-*.json" files after test/objspace/test_objspace.rb.
2019-06-28 12:02:33 +09:00
Nobuyoshi Nakada
bebe8d13f8
Disable frozen-string-literal to disable deduplication
[Bug #14194]
2019-06-23 07:42:11 +09:00
Hiroshi SHIBATA
9eecd7a2fd
Ignore warnings about ambiguous first argument of regexp with assert match. 2019-06-01 14:44:24 +03:00
ko1
74b19e330a fix imemo entry description.
* ext/objspace/objspace.c (count_imemo_objects): `imemo_type_ids`
  should be match with `enum imemo_type` in internal.h and this
  patch fix mismatch.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67058 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-02-13 04:24:59 +00:00
nobu
97fffcadcb ext/objspace/objspace_dump.c: print addresses consistently
The format addresses are printed in are different if you use
`ObjectSpace.dump_all(output: :stdout)` vs.
`ObjectSpace.dump_all(output: :string)` (or `ObjectSpace.dump`) due to
differences in the underlying `vfprintf` implementation.

Use `"%#"PRIxVALUE` to format `VALUE`.

Co-authored-by: Ashe Connor <ashe@kivikakk.ee>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64974 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-09 03:21:57 +00:00
naruse
b9d00698c8 Revert "ext/objspace/objspace_dump.c: print addresses consistently"
This reverts commit r64970.

Visual C++ 12.0 doesn't have PRIxPTR.
Anyway we have our own vfprintf implementation BSD_vfprintf().
If you want to have portable vfprintf, replace it with BSD_vfprintf like
vsnprintf or just use BSD_vfprintf.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64972 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-09 02:01:54 +00:00
tenderlove
c5768e4366 ext/objspace/objspace_dump.c: print addresses consistently
The format addresses are printed in are different if you use
`ObjectSpace.dump_all(output: :stdout)` vs.
`ObjectSpace.dump_all(output: :string)` (or `ObjectSpace.dump`) due to
differences in the underlying `vfprintf` implementation.

Use %"PRIxPTR" instead to be consistent across both.

Co-authored-by: Ashe Connor <ashe@kivikakk.ee>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64970 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-08 23:43:45 +00:00
nobu
5e52f06529 io.c: encoding of ARGF.inplace_mode
* io.c (argf_next_argv): encode inplace mode suffix to the path
  encoding.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60155 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-10-10 12:30:42 +00:00
mame
05561ad75c ext/objspace/objspace.c: remove unneeded code clones.
`setup_hash` have already performed nil check and empty check.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59947 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-17 23:56:37 +00:00
nobu
ae6ebbb3fa fix mortal_dynamic_symbol count
* test/objspace/test_objspace.rb (test_count_symbols): exclude a
  dynamic symbol which has been turned into immortal by
  define_method.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59146 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-06-22 12:53:05 +00:00
ko1
dc44df8a06 try to fix symbol duplication.
* test/objspace/test_objspace.rb (test_count_symbols): make more complex dynamic
  symbols to avoid conflict with other existing static symbols.
  Sometimes this test fails with "mortal_dynamic_symbol=>127".



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59143 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-06-22 08:33:33 +00:00
tenderlove
ef76e3cfb6 Add IMEMO type to heap dump output.
IMEMO objects have many types.  Without this change, we cannot see what
types of IMEMO objects are being used when dumping the heap.  Adding the
type to the IMEMO object will allow us to gather statistics about IMEMO
objects being used.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57486 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-01-31 17:46:51 +00:00
nobu
2c1aa78820 adjust indent [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57262 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-01-04 00:30:45 +00:00
tenderlove
ed2abc4362 Add full option to ObjectSpace.dump_all
The `full` option includes all slots (even `T_NONE`) in the JSON output.
This is to help with debugging heap fragmentation.

Here is an example usage:

```ruby
File.open('heap.json', 'w') do |f|
  ObjectSpace.dump_all(output: f, full: true)
end
```

The `heap.json` file contains all slots, including empty slots.

[Feature #13001] [ruby-core:78468]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57260 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-01-03 22:42:10 +00:00
tenderlove
45e8ccde00 Use JSON lines format for full heap dumps.
This commit changes full heap dumps back to using JSON lines format
(http://jsonlines.org) so that we can process very large heaps without
loading the entire heap in to memory at once.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56174 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-09-16 17:44:45 +00:00
nobu
f54490251a dump to stdout
* test/objspace/test_objspace.rb (test_dump_all): dump to stdout
  instead of a string, get rid of hung up.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55391 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-12 14:20:50 +00:00
nobu
cfb2b14ddc ObjectSpace.dump_all: valid JSON
* ext/objspace/objspace_dump.c: generate valid JSON for dump_all.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55377 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-11 05:04:22 +00:00
naruse
3e92b635fb Add frozen_string_literal: false for all files
When you change this to true, you may need to add more tests.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-16 05:07:31 +00:00
nobu
c92322b537 objspace_dump.c: special const
* ext/objspace/objspace_dump.c: support special constant objects.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52191 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-10-19 06:25:01 +00:00
kazu
4951f3b5e3 fix typos [ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52133 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-10-15 14:37:02 +00:00
nagachika
d0e8610582 * test/objspace/test_objspace.rb
(test_trace_object_allocations_start_stop_clear): clear object
  allocation table first to get rid of erronous detection for obj3.
  [ruby-dev:49095] [Bug #11271]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52128 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-10-14 17:02:56 +00:00
ko1
885d781abc * ext/objspace/objspace.c: add a new method ObjectSpace.count_symbols.
[Feature #11158]
* symbol.c (rb_sym_immortal_count): added to count immortal symbols.
* symbol.h: ditto.
* test/objspace/test_objspace.rb: add a test for this method.
* NEWS: describe about this method.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51654 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-08-21 11:30:24 +00:00
normal
7630957eba iseq.c (iseq_memsize): reimplement for wrapper
* iseq.c (iseq_memsize): reimplement for wrapper
  (param_keyword_size): extracted from iseq_memsize
  (iseqw_mark): new mark function
  (iseqw_data_type): new data type
  (iseqw_new): wrap as iseqw_data_type
  (iseqw_check): adjust for wrapper
  (Init_ISeq): remove iseqw_iseq_key initialization
* test/objspace/test_objspace.rb: new test
  [ruby-core:70344] [Feature #11435]

v2 changes:
- added RUBY_TYPED_WB_PROTECTED and write barrier
- account for rb_call_info_kw_arg_t entries

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51547 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-08-12 12:18:51 +00:00
sorah
cc77375d2d * ext/objspace/objspace_dump.c(dump_object): Return empty JSON object when
passed object is a special const, instead of SEGV.
  Based patch by Kohei Suzuki (eagletmt). [ruby-core:69692] [Bug #11291]

* test/objspace/test_objspace.rb(test_dump_special_consts): Test for above fix.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50982 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-06-20 20:50:20 +00:00
ko1
5a17488189 * test/objspace/test_objspace.rb: relax pattern because uncollectible
flag and marked flag can be false at major GC.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-06-16 08:38:55 +00:00
ko1
6abf7938bf * ext/objspace/objspace.c: add two methods to debug internals.
* ObjectSpace.internal_class_of: return RBASIC_CLASS(obj).
  * ObjectSpace.internal_super_of: return RCLASS_SUPER(cls).
* NEWS: add information about both methods.
* test/objspace/test_objspace.rb: add tests for both methods.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50662 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-05-28 19:40:04 +00:00
ko1
adabc801be * test/objspace/test_objspace.rb: remove debug prints.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50212 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-04-10 10:39:07 +00:00
ko1
7f75a1a810 * ext/objspace/objspace.c: add ObjectSpace.count_imemo_objects method
to count imemo objects for each type.
* test/objspace/test_objspace.rb: add a test.
* NEWS: describe about this addition.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50211 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-04-10 10:38:13 +00:00
nobu
cd322ae533 test_objspace.rb: skip test_count_nodes
* test/objspace/test_objspace.rb (test_count_nodes): NODEs are no
  longer used generally.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50145 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-04-01 06:02:45 +00:00
ko1
5f879f76d7 * test/objspace/test_objspace.rb: flag name was changed
(long_lived -> uncollectible).



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50016 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-03-18 21:30:23 +00:00
nobu
734fa23840 test_objspace.rb: better assertions
* test/objspace/test_objspace.rb: use assertions for better
  failure messages.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49959 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-03-13 05:12:43 +00:00
ko1
85b42fe17b * ext/objspace/objspace.c: ObjectSpace.memsize_of(obj) returns
with sizeof(RVALUE). [Bug #8984]
* gc.c (obj_memsize_of): ditto.
* NEWS: add a NEWS entry.
* test/objspace/test_objspace.rb: catch up this fix.
* test/ruby/test_file_exhaustive.rb: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48846 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-15 08:54:07 +00:00
akr
fb2008a73a * test/lib/envutil.rb: Moved from test/ruby/.
* test/lib/find_executable.rb: Ditto.

* test/lib/memory_status.rb: Ditto.

* test/lib/test/unit.rb: require envutil.

* test/: Don't require envutil in test files.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48409 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-13 16:05:37 +00:00
ko1
123eeb1c1a * gc.c: add incremental GC algorithm. [Feature #10137]
Please refer this ticket for details.
  This change also introduces the following changes.
  * Remove RGENGC_AGE2_PROMOTION and introduce object age (0 to 3).
    Age can be count with FL_PROMOTE0 and FL_PROMOTE1 flags in
    RBasic::flags (2 bit). Age == 3 objects become old objects.
  * WB_PROTECTED flag in RBasic to WB_UNPROTECTED bitmap.
  * LONG_LIVED bitmap to represent living objects while minor GCs
    It specifies (1) Old objects and (2) remembered shady objects.
  * Introduce rb_objspace_t::marked_objects which counts marked
    objects in current marking phase. marking count is needed to
    introduce incremental marking.
  * rename mark related function and sweep related function to
    gc_(marks|sweep)_(start|finish|step|rest|continue).
  * rename rgengc_report() to gc_report().
  * Add obj_info() function to get cstr of object details.
  * Add MEASURE_LINE() macro to measure execution time of specific line.
  * and many small fixes.
* include/ruby/ruby.h: add flag USE_RINCGC.
  Now USE_RINCGC can be set only with USE_RGENGC.
* include/ruby/ruby.h: introduce FL_PROMOTED0 and add FL_PROMOTED1
  to count object age.
* include/ruby/ruby.h: rewrite write barriers for incremental marking.
* debug.c: catch up flag name changes.
* internal.h: add rb_gc_writebarrier_remember() instead of
  rb_gc_writebarrier_remember_promoted().
* array.c (ary_memcpy0): use rb_gc_writebarrier_remember().
* array.c (rb_ary_modify): ditto.
* hash.c (rb_hash_keys): ditto.
* hash.c (rb_hash_values): ditto.
* object.c (init_copy): use rb_copy_wb_protected_attribute() because
  FL_WB_PROTECTED is moved from RBasic::flags.
* test/objspace/test_objspace.rb: catch up ObjectSpace.dump() changes.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47444 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-09-08 04:11:00 +00:00
ktsj
b204fe182f * test/objspace/test_objspace.rb (TestObjSpace#test_dump_uninitialized_file):
remove dependency on json library.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45316 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-03-11 13:23:43 +00:00
charliesome
c8c539a198 ext/objspace: Check fptr before trying to dump FILE object fd
Patch by Scott Francis. Closes GH-562.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45308 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-03-10 21:55:51 +00:00
tmm1
7170baa878 objspace_dump.c: include object's gc flags in dump
* ext/objspace/objspace_dump.c (dump_object): include fstring flag on
  strings. include gc flags (old, remembered, wb_protected) on all objects.
* ext/objspace/objspace_dump.c (Init_objspace_dump): initialize lazy
  IDs before first use.
* gc.c (rb_obj_gc_flags): new function to retrieve object flags
* internal.h (RB_OBJ_GC_FLAGS_MAX): maximum flags allowed for one obj
* test/objspace/test_objspace.rb (test_dump_flags): test for above
* test/objspace/test_objspace.rb (test_trace_object_allocations):
  resolve name before dump (for rb_class_path_cached)

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44105 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-12-10 02:26:09 +00:00