code unit sequences to integers so as to store the entity
maps. Code in traverse_for_entities assumed little
endian. Fixed.
(in practice, due to the absence of unicode and entity
mappings for multi-byte encodings -- except UTF-8 --, this
doesn't matter, so the relevant code was commented out for
performance reasons).
- Dramatic improvements on the performance of html_entity_decode and htmlspecialchars_decode, as the
string is now traversed only once. Speedups of 20 to 25 times with Windows release builds and a
~250 characters string (for 2nd and subsequent calls).
- Consistent behavior on html_entity_decode. For instance, the entity in "&<" would be decoded,
but not "&é". Not anymore. The code path for "basic" and non-basic entities is now mostly
shared.
- Code of html_entity_decode and htmlspecialchars_decode is now shared.
- [DOC] More consistent behavior of htmlspecialchars_decode. Instead of translating only <, >,
&, ", ' and ', now e.g. ", ', ', ', etc. are also decoded.
- [DOC] Previous translation of unicode code points in numerical entities was seriously broken. When
the code points for some character were not the same in unicode and the target encoding, the
behavior could be an erroneous translation (e.g. 0x80-0xA0 in win-1252) or no translation at all.
Added unicode translation tables for all single-byte encodings. Entities are not translated for
multi-byte entities, except for ASCII characters whose code points are shared. We could add
the huge translation tables (several thousand elements) for those encodings in the future.
- Fixed numerical entities that after # had text accepted by strcol being accepted.
- Much more commented and well-structured code...
- Tests for get_html_translation_table()) are broken. I stared fixing the tests, but then I realized
it was completely helpless because get_html_translation_table() is broken by not handling
multi-byte characters correctly.
htmlspecialchars() to skip multibyte sequences intead of returning an
empty string (as iconv's //IGNORE). These functions will still never
return an invalid or incomplete multibyte sequence.
Fixes#43896
