Commit graph

89 commits

Author SHA1 Message Date
Kasumi Hanazuki
8aac19d598 io_buffer: Reimplement dcompact for IO::Buffer
The `source` field in IO::Buffer can have a String or an IO::Buffer
object, if not nil.

- When the `source` is a String object. The `base` field points to the
  memory location of the String content, which can be embedded in
  RSTRING, and in that case, GC compaction can move the memory region
  along with the String object.

  Thus, IO::Buffer needs to pin the `source` object to prevent `base`
  pointer from becoming invalid.

- When the `source` is an IO::Buffer, then `base` is a pointer to a
  malloced or mmapped memory region, managed by the source IO::Buffer.
  In this case, we don't need to pin the source IO::Buffer object,
  since the referred memory region won't get moved by GC.

Closes: [Bug #21210]
2025-06-17 07:57:02 +02:00
Kasumi Hanazuki
4cc58c3a6f Revert "Mark rb_io_buffer_type references declaratively"
This reverts commit 6012145299.
2025-06-17 07:57:02 +02:00
Benoit Daloze
2956573b09 Add test for IO::Buffer.for(frozen_string) {} and omit rb_str_{,un}locktmp in that case 2025-06-16 22:59:10 +02:00
Samuel Williams
73c9d6ccaa
Allow IO#close to interrupt IO operations on fibers using fiber_interrupt hook. (#12839) 2025-05-23 14:55:05 +09:00
Peter Zhu
6d70847e27 Make free, memsize functions of IO::Buffer static
They are not used outside of io_buffer.c, so we can make them static.
2025-03-31 16:37:44 -04:00
Matt Valentine-House
6012145299 Mark rb_io_buffer_type references declaratively
allowing them to support compaction and be moved.
2025-03-31 17:10:37 +01:00
Peter Zhu
7341a4fc07 [Bug #20933] Fix IO::Buffer overlap calculation
The allocated buffers may be consecutive memory addresses. This will mean
that `b->base == a->base + a->size` even though `a` and `b` are separate
buffers.
2024-12-09 11:52:02 -05:00
Kunshan Wang
8ae7c22972 Annotate anonymous mmap
Use PR_SET_VMA_ANON_NAME to set human-readable names for anonymous
virtual memory areas mapped by `mmap()` when compiled and run on Linux
5.17 or higher.  This makes it convenient for developers to debug mmap.
2024-11-21 13:48:05 -05:00
Samuel Williams
3c0b09ac9e
Allow io_buffer_memmove to release the GVL for large buffers. (#12021)
[Feature #20902]
2024-11-20 21:27:16 +13:00
Kasumi Hanazuki
35bf660337
io_buffer.c: Allow copies between overlapping buffers with #copy and #set_string (#11640)
The current implementation of `IO::Buffer#copy` and `#set_string` has
an undefined behavior when the source and destination memory overlaps,
due to the underlying use of the `memcpy` C function.

This patch guarantees the methods to be safe even when copying between
overlapping buffers by replacing `memcpy` with `memmove`,

Fixes: [Bug #20745]
2024-11-06 20:55:39 +00:00
Samuel Williams
c50298d7d4
Introduce rb_io_blocking_region which takes struct rb_io argument. (#11795)
This does not change any actual behaviour, but provides a choke point for blocking IO operations.

* Update `IO::Buffer` to use `rb_io_blocking_region`.

* Update `File` to use `rb_io_blocking_region`.

* Update `IO` to use `rb_io_blocking_region`.
2024-10-05 15:10:12 +13:00
Samuel Williams
75c4ac3395
Update IO::Buffer documentation. (#11737)
Update documentation.
2024-10-03 15:33:48 +13:00
Nobuyoshi Nakada
35e124832e
[Bug #20755] Frozen string should not be writable via IO::Buffer 2024-10-01 18:46:35 +09:00
Nobuyoshi Nakada
637067440f
[Bug #20752] Slice of readonly IO::Buffer also should be readonly 2024-09-30 20:39:14 +09:00
TOMITA Masahiro
48ebd77e59 Fix comment: Buffer.for(string) without block returns readonly buffer 2024-05-19 23:24:50 +09:00
TOMITA Masahiro
cb43540d53 Fix IO::Buffer comment 2024-05-19 23:24:50 +09:00
Nobuyoshi Nakada
d6564615e7
Fix format specifier for DWORD 2024-05-09 10:11:19 +09:00
Samuel Williams
c5cf4d4e12
Improve behavioural consistency of unallocated (zero length) IO::Buffer. (#9532)
This makes the behaviour of IO::Buffer.new(0) and IO::Buffer.new.slice(0, 0) consistent.

Fixes https://bugs.ruby-lang.org/issues/19542 and https://bugs.ruby-lang.org/issues/18805.
2024-01-15 10:47:13 +13:00
Samuel Williams
e5a4f757bd
Fix Window private file mapping unlink EACCES issue. (#9358)
* Don't return early.

* Add missing `mapping` assignment.

* Make debug logs conditional.
2023-12-27 00:36:56 +13:00
Samuel Williams
260bf60e52
Correctly release the underlying file mapping. (#9340)
* Avoiding using `Tempfile` which was retaining the file preventing it from unlinking.
2023-12-25 14:20:53 +13:00
Samuel Williams
37753f163e
IO::Buffer improvements and documentation. (#9329)
* Restore experimental warnings.

* Documentation and code structure improvements.

* Improved validation of flags, clarified documentation of argument handling.

* Remove inconsistent use of `Example:` and add example to `null?`.

* Expose `IO::Buffer#private?` and add test.
2023-12-25 02:03:36 +13:00
Peter Zhu
8e6f63df47 Implement IO::Buffer on VWA 2023-12-19 09:13:20 -05:00
Nobuyoshi Nakada
835f1461f7
[DOC] Add document of IO::Buffer#hexdump 2023-12-18 20:17:44 +09:00
Peter Zhu
c5e3d6da9c Implement Write Barriers on IO::Buffer
Benchmark:

```
require "benchmark"

puts(Benchmark.measure do
  ary = 1_000_000.times.map { IO::Buffer.for("") }

  10.times { GC.start(full_mark: false) }
end)
```

Before:

```
14.330119   0.051497  14.381616 ( 14.445106)
```

After:

```
7.481152   0.040166   7.521318 (  7.535209)
```
2023-12-14 12:22:38 -05:00
Peter Zhu
2350c79462 Use xfree for IO::Buffer
Since IO::Buffer is allocated using TypedData_Make_Struct, it must use
xfree to free the buffer otherwise it will cause more major GC to run.

Example:

```
10.times do
  1_000_000.times { IO::Buffer.new(0) }

  puts "oldmalloc_increase_bytes: #{GC.stat(:oldmalloc_increase_bytes)}, major_gc_count: #{GC.stat(:major_gc_count)}"
end
```

Before:

```
oldmalloc_increase_bytes: 14904176, major_gc_count: 3
oldmalloc_increase_bytes: 2399424, major_gc_count: 5
oldmalloc_increase_bytes: 5204640, major_gc_count: 6
oldmalloc_increase_bytes: 2199936, major_gc_count: 7
oldmalloc_increase_bytes: 34199936, major_gc_count: 7
oldmalloc_increase_bytes: 24223360, major_gc_count: 8
oldmalloc_increase_bytes: 5967616, major_gc_count: 9
oldmalloc_increase_bytes: 37967616, major_gc_count: 9
oldmalloc_increase_bytes: 9689792, major_gc_count: 10
oldmalloc_increase_bytes: 41689792, major_gc_count: 10
```

After:

```
oldmalloc_increase_bytes: 117392, major_gc_count: 2
oldmalloc_increase_bytes: 26128, major_gc_count: 2
oldmalloc_increase_bytes: 71600, major_gc_count: 2
oldmalloc_increase_bytes: 117072, major_gc_count: 2
oldmalloc_increase_bytes: 17296, major_gc_count: 2
oldmalloc_increase_bytes: 62768, major_gc_count: 2
oldmalloc_increase_bytes: 108240, major_gc_count: 2
oldmalloc_increase_bytes: 153712, major_gc_count: 2
oldmalloc_increase_bytes: 53936, major_gc_count: 2
oldmalloc_increase_bytes: 99408, major_gc_count: 2
```
2023-12-12 18:47:58 -05:00
Samuel Williams
00467699a9
Don't warn generally for IO::Buffer, only on specific code paths e.g. map. (#9131) 2023-12-06 13:42:45 +13:00
Samuel Williams
a095740fed
Fix io_buffer_get_string default length computation. (#8427)
* Fix `io_buffer_get_string` default length computation.

When an offset bigger than the size is given, the resulting length will be
computed incorrectly. Raise an argument error in this case.

* Validate all arguments.
2023-09-14 20:37:43 +12:00
Nobuyoshi Nakada
19346c2336
[Bug #19754] Make IO::Buffer#get_string check offset range (#8016) 2023-09-13 09:45:26 +12:00
Samuel Williams
f0b43597ff
[DOC] Improved documentation. (#8319) 2023-08-29 23:53:28 +12:00
Samuel Williams
bf1bc5362e
Improve read/write/pread/pwrite consistency. (#7860)
* Documentation consistency.

* Improve consistency of `pread`/`pwrite` implementation when given length.

* Remove HAVE_PREAD / HAVE_PWRITE - it is no longer optional.
2023-05-27 18:48:47 +09:00
Samuel Williams
135a0d26a1
Improvements to IO::Buffer read/write/pread/pwrite. (#7826)
- Fix IO::Buffer `read`/`write` to use a minimum length.
2023-05-24 10:17:35 +09:00
Samuel Williams
bd786e7896
Fix mutation on shared strings. (#7837) 2023-05-22 12:58:17 +09:00
Samuel Williams
bcb16eb1e5
Rename data -> buffer for better readability. (#7836) 2023-05-22 12:27:20 +09:00
Samuel Williams
648870b5c5
Support IO#pread / IO#pwrite using fiber scheduler. (#7594)
* Skip test if non-blocking file IO is not supported.
2023-03-31 00:48:55 +13:00
Samuel Williams
466aa8010f
Fix incorrect usage of rb_fiber_scheduler_io_(p)(read|write). (#7593) 2023-03-25 18:36:27 +13:00
Kasumi Hanazuki
09295ea796
IO::Buffer#resize: Free internal buffer if new size is zero (#7569)
`#resize(0)` on an IO::Buffer with internal buffer allocated will
result in calling `realloc(data->base, 0)`. The behavior of `realloc`
with size = 0 is implementation-defined (glibc frees the object
and returns NULL, while BSDs return an inaccessible object). And
thus such usage is deprecated in standard C (upcoming C23 will make it
UB).

To avoid this problem, just `free`s the memory when the new size is zero.
2023-03-25 12:12:23 +13:00
John Bampton
2f7270c681
Fix spelling (#7389) 2023-02-27 09:56:06 -08:00
Nobuyoshi Nakada
ef00c6da88
Adjust else style to be consistent in each files [ci skip] 2023-02-26 13:20:43 +09:00
Samuel Williams
d0031db9b3
Prefer RB_NUM2LONG for string length. (#7379) 2023-02-25 23:04:10 +13:00
Samuel Williams
57bc3f2f46
Add IO::Buffer.string for efficient string creation. (#7364) 2023-02-25 18:40:26 +13:00
Nobuyoshi Nakada
3b567eb491
[Bug #19459] Remove unnecessary always-true checks (#7362)
`length` is a required argument for `IO::Buffer#read` and
`IO::Buffer#write` methods, and `argc` is already checked with
`rb_check_arity`.  Also fix the call-seq of `IO::Buffer#read`.
2023-02-23 18:40:13 +13:00
Victor Shepelev
c3c116f6a6
[DOC] Document new methods of IO::Buffer and Fiber::Scheduler (#7016)
Co-authored-by: Samuel Williams <samuel.williams@oriontransfer.co.nz>
2022-12-24 11:48:58 +13:00
Samuel Williams
11ad9a42d7
Fix missing handling of offset argument in IO::Buffer pread and pwrite. (#7012) 2022-12-24 07:52:10 +13:00
Nobuyoshi Nakada
f28e79caaa
Use consistent style [ci skip] 2022-12-02 23:46:21 +09:00
Yudai Takada
29e6d97517
Fix typos (#6775)
* s/Innteger/Integer/

* s/diretory/directory/

* s/Bufer/Buffer/

* s/defalt/default/

* s/covearge/coverage/
2022-11-20 21:07:18 -08:00
S-H-GAMELINKS
1f4f6c9832 Using UNDEF_P macro 2022-11-16 18:58:33 +09:00
Yusuke Nakamura
5344618cb7 [DOC] Fix IO::Buffer#slice rdoc position
Before this change, rdoc shows empty in 'slice' method section
2022-11-03 14:40:17 -04:00
Nobuyoshi Nakada
287eac5e8e Fix format specifiers for size_t 2022-10-26 18:57:26 +09:00
Samuel Williams
fc3137ef54
Add support for anonymous shared IO buffers. (#6580) 2022-10-19 18:53:38 +13:00
Samuel Williams
ced1d17280
Improvements to IO::Buffer implementation and documentation. (#6525) 2022-10-12 12:59:05 +13:00