Commit graph

57 commits

Author SHA1 Message Date
Kevin Menard
2a509787cb [ruby/prism] Track whether a Symbol should have its encoding changed from the source encoding.
Ruby sets a Symbol literal's encoding to US-ASCII if the symbols consists only of US ASCII code points. Character escapes can also lead a Symbol to have a different encoding than its source's encoding.

f315660b31
2024-01-26 20:15:19 +00:00
Kevin Newton
242dc537f9 [ruby/prism] Exclude encodings that are dynamic
6749146c0e
2024-01-11 19:14:25 +00:00
Kevin Newton
82f18baa21 [ruby/prism] Provide flags for changing encodings
e838eaff6f
2023-12-06 14:23:38 -05:00
Kevin Newton
19a321bff0 [ruby/prism] CESU encoding
2d5b9c2b3c
2023-11-30 21:37:56 -05:00
Kevin Newton
ca26e0e34b [ruby/prism] EUC-TW encoding
edfb54f039
2023-11-30 21:37:56 -05:00
Kevin Newton
32249c2cf1 [ruby/prism] GB18030 encoding
ca3ab7ec89
2023-11-30 21:37:56 -05:00
Kevin Newton
a9162a44c5 [ruby/prism] Emacs MULE encodings
4c06b6c42e
2023-11-30 21:37:56 -05:00
Kevin Newton
10d3897e13 [PRISM] Big5 encodings 2023-11-30 21:37:56 -05:00
Kevin Newton
700e172a50 [ruby/prism] EUC-KR encodings
ba5218385a
2023-11-30 21:37:56 -05:00
Kevin Newton
ddaa073058 [ruby/prism] Support other EUC-JP encodings
d040337ce9
2023-11-30 21:37:56 -05:00
Kevin Newton
7b5bb978fb [PRISM] Alias CP51932 to EUC-JP 2023-11-30 21:37:56 -05:00
Kevin Newton
9ba92327f2 [PRISM] Consolidate SJIS encodings 2023-11-30 21:37:56 -05:00
Kevin Newton
219c3c1c09 [ruby/prism] Add other UTF8 encodings
709fb6e09f
2023-11-30 21:37:56 -05:00
Matt Boldt
9fc40d2b26 [ruby/prism] Add MacJapanese encoding
MacJapanese (also aliased as MacJapan) is a modified Shift_JIS
encoding, but is implemented identically in Ruby

9e0a097699
2023-11-29 12:08:15 -05:00
Dhaval
9fada99cb2 [ruby/prism] added CP950 encoding
9c2d1cf4ba
2023-11-29 11:15:50 -05:00
heyogrady
e3ef05a434 [ruby/prism] Add CP949 encoding
9e78dfdf69
2023-11-22 09:12:35 -05:00
Patrick O'Grady
e9ccceab11 [ruby/prism] Add KOI8-U encoding
(https://github.com/ruby/prism/pull/1906)

* Add test for KOI8-U

* Rename koi8 char_width function

- Rename function for use with any KOI8-based encoding

* Add KOI8-U encoding

* Add encoding to encoding.md

6cad4552f7
2023-11-22 13:21:15 +00:00
Thiago Araujo
91685c01d3 [ruby/prism] feat: adds encoding for TIS-620
(https://github.com/ruby/prism/pull/1901)

2c308e6697
2023-11-21 17:12:00 +00:00
Ryan Garver
e16ff17374 Rename the big5-hkscs stuff to something more generic and add UAO sharing common code.
Merge the Big5 extensions into pm_big5.c
2023-11-20 21:18:31 -05:00
Lynne Ashminov
307a5cb20b [ruby/prism] adds encodings for ibm869
(https://github.com/ruby/prism/pull/1886)

41462400b7
2023-11-20 20:06:14 +00:00
Maple Ong
9fbdb26f06 [ruby/prism] Add and test ibm863 encoding
(https://github.com/ruby/prism/pull/1853)

* Add and test ibm863

* Remove dup encoding and add alias

* Update test/prism/encoding_test.rb

Co-authored-by: Kevin Newton <kddnewton@gmail.com>

* Readd bitfield table lol

---------

4cd756d7ff

Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2023-11-20 20:06:00 +00:00
David Wessman
f9fb05f9d0 [ruby/prism] feat: Adds macCroatian encoding
(https://github.com/ruby/prism/pull/1880)

* feat: Adds macCroatian encoding

- Based on:
https://en.wikipedia.org/wiki/Mac_OS_Croatian_encoding
https://www.unicode.org/Public/MAPPINGS/VENDORS/APPLE/CROATIAN.TXT

Co-authored-by: Josefine Rost <nijrost@gmail.com>

* Use output from bin/encodings and adds to docs/encoding.md

---------

019a82d8f3

Co-authored-by: Josefine Rost <nijrost@gmail.com>
2023-11-20 16:07:19 +00:00
Derek Moore
701b0650de [ruby/prism] feat: add encoding for IBM865
(https://github.com/ruby/prism/pull/1884)

* feat: add encoding for IBM865

* style: fix incorrect autoformat

14c6ae0182
2023-11-20 02:02:55 +00:00
Syed Faraaz Ahmad
5fea1d08bd [ruby/prism] feat: add encoding for ibm866
(https://github.com/ruby/prism/pull/1864)

Add encoding for ibm866

---------

1a96cc71f7

Co-authored-by: Kevin Newton <kddnewton@gmail.com>
2023-11-19 01:05:41 +00:00
Orhan Toy
5d2d0962cc [ruby/prism] Add GB1988 encoding
78d3fa7172
2023-11-19 01:05:11 +00:00
Orhan Toy
71a30b7596 [ruby/prism] Add macCyrillic encoding
220b40921a
2023-11-19 01:03:19 +00:00
Ryan Garver
cd4316a51f [ruby/prism] Big5 HKSCS encoding
3ca9823eb4
2023-11-18 13:52:17 -05:00
Mike Dalton
bbf14bbba6 [ruby/prism] Add IBM864 encoding
Fixes https://github.com/ruby/prism/pull/1868
Related #1843

abc136dfc9
2023-11-18 18:47:15 +00:00
Thomas Marshall
a6f9e6add2 [ruby/prism] Add macCentEuro encoding
ff95edbd99
2023-11-18 18:41:03 +00:00
Peter Cai
585fdfe1f5 [ruby/prism] add Windows-874 encoding
0670dd3b9a
2023-11-17 20:06:48 +00:00
Kevin Newton
229f6e5bb4 [ruby/prism] Update spacing in encoding_test.rb
56508c2201
2023-11-17 20:06:28 +00:00
Haldun Bayhantopcu
50b7b927a3 [ruby/prism] Add macThai
f654058f50
2023-11-17 20:06:27 +00:00
Haldun Bayhantopcu
0a081a33eb [ruby/prism] Add macRoman
42b20ee399
2023-11-17 20:06:16 +00:00
Kevin Newton
db4303f953 [ruby/prism] Never test locale encoding
f0f057b055
2023-11-17 15:28:08 +00:00
Kevin Newton
c2f2090da6 [ruby/prism] Do not test locale encoding on windows
8f40536431
2023-11-17 14:50:31 +00:00
Haldun Bayhantopcu
4a26a65e42 [ruby/prism] Add macTurkish
2232d4b6a0
2023-11-17 14:32:40 +00:00
Kevin Newton
3bc41f4f0b [ruby/prism] Add macGreek encoding
c36d3fc647
2023-11-16 18:00:08 +00:00
Kevin Newton
ce85355931 [ruby/prism] Only run encoding tests on CRuby
a63acc0629
2023-11-16 17:39:40 +00:00
Kevin Newton
4b2915f0b9 [ruby/prism] Add macRomania encoding
bb73801cf4
2023-11-16 17:39:40 +00:00
Kevin Newton
b753e16945 [ruby/prism] IBM862 encoding
151698bf86
2023-11-16 17:39:38 +00:00
Kevin Newton
7dba4f424d [ruby/prism] IBM861 encoding
6d2b59384a
2023-11-16 17:39:37 +00:00
Kevin Newton
91af9e8d93 [ruby/prism] IBM860 encoding
aa6163d77a
2023-11-16 17:39:37 +00:00
Kevin Newton
a7df025c3f [ruby/prism] IBM857 encoding
8c9b580f84
2023-11-16 17:39:36 +00:00
Kevin Newton
076d0957b9 [ruby/prism] IBM855 encoding
9354ad1848
2023-11-16 17:39:36 +00:00
Kevin Newton
b0a188655d [ruby/prism] IBM852 encoding
45251fcbf1
2023-11-16 17:39:35 +00:00
Kevin Newton
ef748f353b [ruby/prism] IBM775 encoding
65175641b1
2023-11-16 17:39:34 +00:00
Kevin Newton
ae3c346746 [ruby/prism] IBM-737 encoding
af1a665939
2023-11-16 17:39:33 +00:00
Kevin Newton
8c44d69b9f [ruby/prism] IBM720 encoding
fc1f6ea3af
2023-11-16 17:39:33 +00:00
Kevin Newton
ba937eee0b [ruby/prism] IBM437 encoding
d60329eeb5
2023-11-16 17:39:32 +00:00
Kevin Newton
f60b974393 [ruby/prism] Add cp855 encoding
8069d143f5
2023-11-16 17:39:32 +00:00