Commit graph

72 commits

Author SHA1 Message Date
Jean Boussier
0dfeb17296
Rename escape_slash in script_safe and also escape E+2028 and E+2029
It is rather common to directly interpolate JSON string inside
<script> tags in HTML as to provide configuration or parameters to a
script.

However this may lead to XSS vulnerabilities, to prevent that 3
characters need to be escaped:

  - `/` (forward slash)
  - `U+2028` (LINE SEPARATOR)
  - `U+2029` (PARAGRAPH SEPARATOR)

The forward slash need to be escaped to prevent closing the script
tag early, and the other two are valid JSON but invalid Javascript
and can be used to break JS parsing.

Given that the intent of escaping forward slash is the same than escaping
U+2028 and U+2029, I chos to rename and repurpose the existing `escape_slash`
option.
2023-12-01 16:47:06 +09:00
Ufuk Kayserilioglu
12dfd9d1c9
[flori/json] Call super in included hook
The C extension defines an `included` hook for the
`JSON::Ext::Generator::GeneratorMethods::String` module but neglects to
call `super` in the hook. This can break the functionality of various
other code that rely on the fact that `included` on `Module` will always
be called.

cd8bbe56a3
2023-05-24 09:37:30 +09:00
Jean Boussier
66b52f046f [flori/json] Stop including the parser source __LINE__ in exceptions
It makes testing for JSON errors very tedious. You either have
to use a Regexp or to regularly update all your assertions
when JSON is upgraded.

de9eb1d28e
2022-07-29 19:10:10 +09:00
Aaron Patterson
8ef30bcc04
Fix GC compatibility: Don't stash encodings in global constants
This value should either be pinned, or looked up when needed at runtime.
Without pinning, the GC may move the encoding object, and that could
cause a crash.

In this case it is easier to find the value at runtime, and there is no
performance penalty (as Ruby caches encoding indexes).  We can shorten
the code, be compaction friendly, and incur no performance penalty.
2021-02-01 12:20:34 -08:00
Kenta Murata
14d7d1df25
[json] Make json Ractor safe 2020-12-21 22:10:43 +09:00
Kenta Murata
98cc15ed1e
[json] Stop using prototype objects 2020-12-21 22:10:33 +09:00
Jean Boussier
e1659af372 Add an option to escape forward slash character
Squashed commit of the following:

commit 26d181059989279a79c433cedcd893b4f52e42ee
Author: Francois Chagnon <francois.chagnon@jadedpixel.com>
Date:   Tue Sep 15 21:17:34 2015 +0000

    add config options for escape_slash

commit fa282334051b16df91ca097dd7304b46f3bc7719
Author: Francois Chagnon <francois.chagnon@jadedpixel.com>
Date:   Mon Feb 9 21:09:33 2015 +0000

    add forward slash to escape character
2020-09-25 17:28:42 +09:00
Marc-Andre Lafortune
26041da2fb
[flori/json] Typo fix
26c1769969
2020-07-01 18:47:51 +09:00
zverok
7f1e3a7b7c [flori/json] Add :nodoc: for GeneratorMethods
2f3f44c180
2020-01-06 15:13:50 +09:00
Aaron Patterson
1d564acedc
Remove unused constant.
This constant isn't used, so lets remove it.
2019-10-17 13:35:26 -07:00
Aaron Patterson
9026e12f93
Look up constant instead of caching in a global
The global can go bad if the compactor runs, so we need to look up the
constant instead of caching it in a global.
2019-10-17 13:30:09 -07:00
Sho Hashimoto
308bbb4e10
[flori/json] Add ascii_only option to JSON::Ext::Generator::State.new.
0e99a9aac5
2019-10-14 19:54:49 +09:00
Watson
98a9445db9
[flori/json] Add shortcut converting to String
In where to convert Hash key to String for json, this patch will add shortcut for String/Symbol in Hash key.

```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
                json    65.000  i/100ms
Calculating -------------------------------------
                json    659.576  (± 1.5%) i/s -      3.315k in   5.027127s
```

```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
                json    78.000  i/100ms
Calculating -------------------------------------
                json    789.781  (± 2.7%) i/s -      3.978k in   5.041043s
```

```
require 'json'
require 'benchmark/ips'

obj = []

1000.times do |i|
  obj << {
    "id" => i,
    :age => 42,
  }
end

Benchmark.ips do |x|
  x.report "json" do |iter|
    count = 0
    while count < iter
      JSON.generate(obj)
      count += 1
    end
  end
end
```

38c0f6dbe4
2019-10-14 19:54:49 +09:00
Watson
a2f9c38a71
[flori/json] Convert Hash object using rb_hash_foreach()
To convert Hash convert, this part was using following pseudo code

```
obj.keys.each do |key|
  value = obj[key]
  ...
end
```

and `rb_funcall()` was called for `obj.keys`.
It might be slightly heavy to call the Ruby method.
This patch will iterate to convert Hash object about key/value using `rb_hash_foreach()` Ruby API instead of `rb_funcall()`.

```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
                json    55.000  i/100ms
Calculating -------------------------------------
                json    558.501  (± 1.1%) i/s -      2.805k in   5.022986s
```

```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
                json    65.000  i/100ms
Calculating -------------------------------------
                json    659.576  (± 1.5%) i/s -      3.315k in   5.027127s
```

```
require 'json'
require 'benchmark/ips'

obj = []

1000.times do |i|
  obj << {
    "id" => i,
    :age => 42,
  }
end

Benchmark.ips do |x|
  x.report "json" do |iter|
    count = 0
    while count < iter
      JSON.generate(obj)
      count += 1
    end
  end
end
```

a73323dc5e
2019-10-14 19:54:49 +09:00
Nobuyoshi Nakada
2003755a2c
[flori/json] Fixed unexpected illegal/malformed utf-8 error
flori/json@c34d01ff6a does not
consider US-ASCII compatible but non-UTF-8 encodings, and causes
an error in RDoc tests.

4f471bf590
2019-10-14 19:54:48 +09:00
Watson
d7fa7e2c86
[flori/json] Convert string encoding to UTF-8 only when needed
## Before
```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
                json   129.000  i/100ms
Calculating -------------------------------------
                json      1.300k (± 2.3%) i/s -      6.579k in   5.064656s
```

## After
```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
                json   189.000  i/100ms
Calculating -------------------------------------
                json      1.964k (± 3.3%) i/s -      9.828k in   5.011237s
```

## Code
```
require 'json'
require 'benchmark/ips'

obj = []

1000.times do |i|
  obj << {
    "id" => i,
    :age => 42,
  }
end

Benchmark.ips do |x|
  x.report "json" do |iter|
    count = 0
    while count < iter
      JSON.generate(obj)
      count += 1
    end
  end
end
```

c34d01ff6a
2019-10-14 19:54:48 +09:00
Watson
40724d7d10
[flori/json] Convert String encoding using rb_str_encode()
`rb_funcall` might be slightly heavy to call the Ruby method.
This patch will convert String encoding using `rb_str_encode()` instead of `rb_funcall()`.

## Before
```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
                json    78.000  i/100ms
Calculating -------------------------------------
                json    789.781  (± 2.7%) i/s -      3.978k in   5.041043s
```

## After
```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
                json   129.000  i/100ms
Calculating -------------------------------------
                json      1.300k (± 2.3%) i/s -      6.579k in   5.064656s
```

## Code
```
require 'json'
require 'benchmark/ips'

obj = []

1000.times do |i|
  obj << {
    "id" => i,
    :age => 42,
  }
end

Benchmark.ips do |x|
  x.report "json" do |iter|
    count = 0
    while count < iter
      JSON.generate(obj)
      count += 1
    end
  end
end
```

9ae6d2969c
2019-10-14 19:54:48 +09:00
Watson
641136c4af
[flori/json] Does not check whether illegal utf-8 if string has ascii only.
## Before
```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
                json    25.000  i/100ms
Calculating -------------------------------------
                json    250.478  (± 4.8%) i/s -      1.250k in   5.002238s
```

## After
```
$ ruby bench_json_generate.rb
Warming up --------------------------------------
                json    32.000  i/100ms
Calculating -------------------------------------
                json    360.652  (± 3.6%) i/s -      1.824k in   5.064511s
```

## Test code
```
require 'json'
require 'benchmark/ips'

obj = []

1000.times do |i|
  obj << {
    :string => "x" * 100,
    :utf8 => "あ" * 100
  }
end

Benchmark.ips do |x|
  x.report "json" do |iter|
    count = 0
    while count < iter
      JSON.generate(obj)
      count += 1
    end
  end
end
```

91a24ecac3
2019-10-14 19:54:48 +09:00
Nobuyoshi Nakada
2a8be8ec33
Suppress uninitialized instance variable warnings 2019-07-09 08:31:27 +09:00
tenderlove
91793b8967 Add GC.compact again.
🙏

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67620 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-20 01:19:47 +00:00
tenderlove
744e5df715 Reverting compaction for now
For some reason symbols (or classes) are being overridden in trunk

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67598 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-17 09:41:41 +00:00
tenderlove
3c55b643ae Adding GC.compact and compacting GC support.
This commit adds the new method `GC.compact` and compacting GC support.
Please see this issue for caveats:

  https://bugs.ruby-lang.org/issues/15626

[Feature #15626]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-17 03:17:25 +00:00
kazu
25c1fd3b90 Reverting all commits from r67479 to r67496 because of CI failures
Because hard to specify commits related to r67479 only.
So please commit again.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67499 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-10 09:15:21 +00:00
tenderlove
e385c159da Add error globals to mark list so they don't move
JSON gem is referencing constants defined in Ruby then keeping a
reference as a global.  We need to register these globals so they stay
pinned.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67483 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-04-09 21:13:32 +00:00
hsbt
6604e1b7cd Merge json-2.2.0 from flori/json.
https://github.com/flori/json/releases/tag/v2.2.0

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67127 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2019-02-23 03:43:58 +00:00
nobu
c8094ff154 no ID cache in Init functions
Init functions are called only once, cache is useless.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62429 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-16 08:39:48 +00:00
hsbt
83735ba29a Merge json-2.0.4.
* https://github.com/flori/json/releases/tag/v2.0.4
  * 09fabeb03e/CHANGES.md (2017-03-23-204)

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58323 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-04-12 00:21:18 +00:00
hsbt
11a94f2a36 * ext/json/*, test/json/json_parser_test.rb: Update json-2.0.2.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55784 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-08-01 03:16:30 +00:00
hsbt
1130658738 * ext/json/*, test/json/*: Update json-2.0.1.
Changes of 2.0.0: f679ebd0c6/CHANGES.md (2015-09-11-200)
  Changes of 2.0.1: f679ebd0c6/CHANGES.md (2016-07-01-201)
  [Feature #12542][ruby-dev:49706][fix GH-1395]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-07-05 11:49:39 +00:00
nobu
c071c05229 Integer unification macro
* include/ruby/ruby.h (RUBY_INTEGER_UNIFICATION): macro to tell if
  Integer is integrated.  [ruby-core:75718][Bug #12427]
* include/ruby/backward.h, internal.h (rb_cFixnum, rb_cBignum):
  fallback to rb_cInteger.
* bignum.c, numeric.c, ext/json/generator/generator.{c,h}: use the
  macro.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55394 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-13 11:34:56 +00:00
akr
f9727c12cc [Feature #12005] Unify Fixnum and Bignum into Integer
* [Feature #12005] Unify Fixnum and Bignum into Integer

* include/ruby/ruby.h (rb_class_of): Return rb_cInteger for fixnums.

* insns.def (INTEGER_REDEFINED_OP_FLAG): Unified from
  FIXNUM_REDEFINED_OP_FLAG and BIGNUM_REDEFINED_OP_FLAG.

* vm_core.h: Ditto.

* vm_insnhelper.c (opt_eq_func): Use INTEGER_REDEFINED_OP_FLAG instead
  of FIXNUM_REDEFINED_OP_FLAG.

* vm.c (vm_redefinition_check_flag): Use rb_cInteger instead of
  rb_cFixnum and rb_cBignum.
  (C): Use Integer instead of Fixnum and Bignum.

* numeric.c (fix_succ): Removed.
  (Init_Numeric): Define Fixnum as Integer.

* bignum.c (bignew): Use rb_cInteger instead of Rb_cBignum.
  (rb_int_coerce): replaced from rb_big_coerce and return fixnums
  as-is.
  (Init_Bignum): Define Bignum as Integer.
  Don't define ===.

* error.c (builtin_class_name): Return "Integer" for fixnums.

* sprintf.c (ruby__sfvextra): Use rb_cInteger instead of rb_cFixnum.

* ext/-test-/testutil: New directory to test.
  Currently it provides utilities for fixnum and bignum.

* ext/json/generator/generator.c: Define mInteger_to_json.

* lib/mathn.rb (Fixnum#/): Redefinition removed.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55024 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-17 06:53:48 +00:00
nobu
4d059bf9f5 generator.c: allocate structs with wrapper
* ext/json/generator/generator.c (cState_s_allocate): allocate
  structs with making new wrapper objects and get rid of potential
  memory leak.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50661 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-05-28 07:17:55 +00:00
hsbt
2e4f0af00f * ext/json/*, test/json/*: Reverted r50231. Because it's not works with
cross-compile environment.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50267 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-04-12 08:36:37 +00:00
hsbt
cfaddc2a32 * ext/json/*, test/json/*, defs/default_gems: Gemify JSON library.
[fix GH-867][Feature #11057]
* test/ruby/test_extlibs.rb: removed json gem from existence extentions.
* gems/bundled_gems: added json gem into bundled gem.
* lib/rdoc/rubygems_hook.rb: ignored no json environment.
* lib/rubygems/test_case.rb, test/rubygems/*: ditto.
* lib/rdoc/test_case.rb, test/rdoc/*: ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50231 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-04-11 11:14:36 +00:00
hsbt
f5219fee63 * ext/json: merge upstream from flori/json
change usage of TypedData. [Feature #10739][ruby-core:67564]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49582 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-02-13 05:19:12 +00:00
nobu
e5c516c89a generator.c: fix infinite recursion
* ext/json/generator/generator.c (generate_json): get rid of
  unnecessary recursive calls which can cause infinite recursion.
  T_STRING may not have rb_cString.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49562 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-02-11 02:01:36 +00:00
hsbt
8f37674ffd * ext/json, test/json: merge JSON HEAD(259dee6)
separate imprementation of Typed_Data macro.
  https://github.com/flori/json/compare/v1.8.1...v1.8.2

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49236 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-01-13 12:08:28 +00:00
hsbt
4a481ad400 * ext/json, test/json: merge JSON HEAD(17fe8e7)
https://github.com/flori/json/compare/v1.8.1...17fe8e7

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49051 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-28 22:29:11 +00:00
nobu
5547719573 ext/json: for ancient backward compatibilities
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49050 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-28 17:03:36 +00:00
nobu
d29ff24793 json: backward compatibilities
* ext/json/generator/generator.c (JSON_Generator_State_type): add
  #ifdef for backward compatibility.
* ext/json/parser/parser.rl (JSON_Parser_type): ditto.
* ext/json/generator/generator.h (ZALLOC): add fallback definition.
* ext/json/parser/parser.h (ZALLOC): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@49038 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-27 11:12:58 +00:00
nobu
e580a631be use 0 for reserved
use 0 for rb_data_type_t::reserved instead of NULL, since its type
may be changed in the future and possibly not a pointer type.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48662 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-12-01 06:38:04 +00:00
nobu
bdfc2e2942 json/generator: typed data
* ext/json/generator/generator.c (JSON_Generator_State_type): turn
  into typed data.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47797 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-04 23:33:09 +00:00
nobu
5540c1de81 ext: protoize no-arguments functions
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47792 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-10-04 23:09:32 +00:00
nobu
1f29e8e2ba class.c: do nothing if copying self
* class.c (rb_mod_init_copy): do nothing if copying self.
  [ruby-dev:47989] [Bug #9535]
* hash.c (rb_hash_initialize_copy): ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45076 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-02-21 11:42:03 +00:00
nobu
cfa2a21062 ext/json: backward compatibilities
* ext/json/fbuffer/fbuffer.h, ext/json/generator/generator.c: add
  macros for backward compatibilities.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44615 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-01-15 08:16:46 +00:00
naruse
14cab32596 * ext/json: merge JSON 1.8.1.
002ac2771c...e09ffc0d7d
* Remove Rubinius exception since transcoding should be working now.
* Fix https://github.com/flori/json/issues/162 reported by Marc-Andre
  Lafortune <github_rocks@marc-andre.ca>. Thanks!
* Applied patches by Yui NARUSE <naruse@airemix.jp> to suppress
  warning with -Wchar-subscripts and better validate UTF-8 strings.
* Applied patch by ginriki@github to remove unnecessary if.
* Add load/dump interface to JSON::GenericObject to make
  serialize :some_attribute, JSON::GenericObject
  work in Rails active models for convenient
  SomeModel#some_attribute.foo.bar access to serialised JSON data.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43731 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-11-20 02:55:08 +00:00
nobu
3f78d84661 Eliminate less-than-zero checks for unsigned variables
* ext/bigdecimal/bigdecimal.c, ext/digest/md5/md5.c,
  ext/json/fbuffer/fbuffer.h, ext/json/generator/generator.c:
  Eliminate less-than-zero checks for unsigned variables.
  According to section 4.1.5 of C89 standard, size_t is an unsigned
  type.  These checks were found with 'cppcheck' static analysis tool.
  [ruby-core:57117] [Feature #8890]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42920 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-09-12 13:37:11 +00:00
naruse
ce573f3166 * ext/json/generator/generator.c (isArrayOrObject): cast char to
unsigned char. [Bug #8378]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40609 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-05-08 05:00:02 +00:00
zzak
3a7a773aeb * ext/socket/option.c: Document synonymous methods, by windwiny [GH-277]
* ext/stringio/stringio.c: ditto
* ext/io/wait/wait.c: ditto
* ext/gdbm/gdbm.c: ditto
* ext/dl/cfunc.c: ditto
* ext/zlib/zlib.c: ditto
* ext/win32ole/win32ole.c: ditto
* ext/dbm/dbm.c: ditto
* ext/json/generator/generator.c: ditto
* ext/date/date_core.c: ditto


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40319 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-04-16 03:25:50 +00:00
naruse
062d2ee6f7 * ext/json: merge JSON 1.7.7.
This includes security fix. [CVE-2013-0269]
  d0a62f3ced
  https://groups.google.com/d/topic/rubyonrails-security/4_YvCpLzL58/discussion

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@39208 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-02-12 03:05:45 +00:00