(https://github.com/ruby/zlib/pull/55)
zstream_discard_input was encoding and character-aware when given input is user-provided, so this discards `len` chars instead of `len` bytes.
Also Zlib.gunzip explains in its rdoc that it is equivalent with the following code, but this doesn't fail for UTF-8 String.
```ruby
string = %w[1f8b0800c28000000003cb48cdc9c9070086a6103605000000].pack("H*").force_encoding('UTF-8')
sio = StringIO.new(string)
p gz.read #=> "hello"
gz&.close
p Zlib.gunzip(string) #=> Zlib::DataError
```
Reported and discovered by eagletmt at https://twitter.com/eagletmt/status/1689692467929694209c5e58bc62a
When Zlib::Inflate#inflate or Zlib::Deflate#deflate is called
recursively inside the block, a crash can occur because of an
use-after-free bug.
50fb8a0338
See also: https://github.com/ruby/csv/issues/117#issuecomment-933289373
How to reproduce with x.csv.gz in the issue comment:
Zlib::GzipReader.open("x.csv.gz") do |rio|
rio.gets(nil, 1024)
while line = rio.gets(nil, 8192)
raise line unless line.valid_encoding?
end
end
Reported by Dimitrij Denissenko. Thanks!!!
b1f182e98f
I'm not sure whether this handles all multithreaded use cases,
but this handles the example that crashes almost immediately
and does 10,000,000 total deflates using 100 separate threads.
To prevent the tests from taking forever, the committed test
for this uses only 10,000 deflates across 10 separate threads,
which still causes a segfault in the previous implementation
almost immediately.
Fixes [Bug #17803]
4b1023b3f2
If a buffer keyword argument is given, it is used as the buffer,
instead of creating new strings. This can result in significantly
lower memory usage during inflation.
Implements #19dac9a9b57d
This commit deletes
{IO,ARGF,StringIO,Zib::GZipReader}#{bytes,chars,lines,codepoints}, which
have been deprecated since c47c095b97.
Note that String also has those methods. They are neither depreacted
nor deleted because they are not aliases of counterpart each_something.
* cast to suppress C4267 warnings; no possible loss of data as
following the comparison.
* shift base address to suppress LNK4281; although /DYNAMICBASE is
preferable, not sure from which version of link.exe supports it.
Most gzip tools support concatenated gz streams in a gz file. This
offers a way to handle such gz files in Ruby.
Fixes [Bug #9790]
Fixes [Bug #11180]
Fixes [Bug #14804]
e2ce56de7d
