php-src/ext/mbstring/tests/mb_strtolower_basic.phpt
Alex Dowad 39b46a5398 Implement Unicode conditional casing rules for Greek letter sigma
The capital Greek letter sigma (Σ) should be lowercased as σ except
when it appears at the end of a word; in that case, it should be
lowercased as the special form ς.

This rule is included in the Unicode data file SpecialCasing.txt.
The condition for applying the rule is called "Final_Sigma" and is
defined in Unicode technical report 21. The rule is:

• For the special casing form to apply, the capital letter sigma must
  be preceded by 0 or more "case-ignorable" characters, preceded by
  at least 1 "cased" character.
• Further, capital sigma must NOT be followed by 0 or more
  case-ignorable characters and then at least 1 cased character.

"Case-ignorable" characters include certain punctuation marks, like
the apostrophe, as well as various accent marks. There are actually
close to 500 different case-ignorable characters, including accent marks
from Cyrillic, Hebrew, Armenian, Arabic, Syriac, Bengali, Gujarati,
Telugu, Tibetan, and many other alphabets. This category also includes
zero-width spaces, codepoints which indicate RTL/LTR text direction,
certain musical symbols, etc.

Since the rule involves scanning over "0 or more" of such
case-ignorable characters, it may be necessary to scan arbitrarily far
to the left and right of capital sigma to determine whether the special
lowercase form should be used or not. However, since we are trying to
be both memory-efficient and CPU-efficient, this implementation limits
how far to the left we will scan. Generally, we scan up to 63 characters
to the left looking for a "cased" character, but not more.

When scanning to the right, we go up to the end of the string if
necessary, even if it means scanning over thousands of characters.

Anyways, it is almost impossible to imagine that natural text will
include "words" with more than 63 successive apostrophes (for example)
followed by a capital sigma.

Closes GH-8096.
2023-01-12 17:41:11 +02:00

94 lines
3.3 KiB
PHP
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

--TEST--
Test mb_strtolower() function : basic functionality
--EXTENSIONS--
mbstring
--FILE--
<?php
/*
* Test basic functionality of mb_strtolower
*/
echo "*** Testing mb_strtolower() : basic functionality***\n";
$ascii_lower = 'abcdefghijklmnopqrstuvwxyz';
$ascii_upper = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
$greek_lower = base64_decode('zrHOss6zzrTOtc62zrfOuM65zrrOu868zr3Ovs6/z4DPgc+Dz4TPhc+Gz4fPiM+J');
$greek_upper = base64_decode('zpHOks6TzpTOlc6WzpfOmM6ZzprOm86czp3Ons6fzqDOoc6jzqTOpc6mzqfOqM6p');
echo "\n-- ASCII String --\n";
$ascii = mb_strtolower($ascii_upper);
var_dump($ascii);
if($ascii == $ascii_lower) {
echo "Correctly converted\n";
} else {
echo "Incorrectly converted\n";
}
echo "\n-- Multibyte String --\n";
$mb = mb_strtolower($greek_upper, 'UTF-8');
var_dump(base64_encode($mb));
if ($mb == $greek_lower) {
echo "Correctly converted\n";
} else {
echo "Incorrectly converted\n";
}
echo "\n-- Greek letter sigma --\n";
var_dump(mb_strtolower("Σ", 'UTF-8'));
var_dump(mb_strtolower("", 'UTF-8'));
var_dump(mb_strtolower("aΣb", 'UTF-8'));
var_dump(mb_strtolower("aΣ b", 'UTF-8'));
var_dump(mb_strtolower(" ΣΣΣΣ ", 'UTF-8'));
// Apostrophe, full stop, colon, etc. are "case-ignorable"
// When checking whether capital sigma is at the end of a word or not, we skip over
// any number of case-ignorable characters, both when scanning back and when scanning forward
var_dump(mb_strtolower("", 'UTF-8'));
var_dump(mb_strtolower("ab'Σ", 'UTF-8'));
var_dump(mb_strtolower("Σ'", 'UTF-8'));
var_dump(mb_strtolower("Σ'a", 'UTF-8'));
var_dump(mb_strtolower("a'Σ'a", 'UTF-8'));
// We scan back by at least 63 characters when necessary,
// but there is no guarantee that we will scan back further than that
var_dump(mb_strtolower('a' . str_repeat('.', 63) . "Σ", 'UTF-8'));
var_dump(mb_strtolower('a' . str_repeat('.', 64) . "Σ", 'UTF-8')); // Context-sensitive casing doesn't work here!
// When scanning forward to confirm if capital sigma is at the end of a word or not,
// there is no limit as to how far we will scan
var_dump(mb_strtolower("abcΣ" . str_repeat('.', 64) . ' abc', 'UTF-8'));
var_dump(mb_strtolower("abcΣ" . str_repeat('.', 64) . 'a abc', 'UTF-8'));
var_dump(mb_strtolower("abcΣ" . str_repeat('.', 256) . ' abc', 'UTF-8'));
echo "Done";
?>
--EXPECT--
*** Testing mb_strtolower() : basic functionality***
-- ASCII String --
string(26) "abcdefghijklmnopqrstuvwxyz"
Correctly converted
-- Multibyte String --
string(64) "zrHOss6zzrTOtc62zrfOuM65zrrOu868zr3Ovs6/z4DPgc+Dz4TPhc+Gz4fPiM+J"
Correctly converted
-- Greek letter sigma --
string(2) "σ"
string(3) "aς"
string(4) "aσb"
string(5) "aς b"
string(10) " σσσς "
string(3) "'σ"
string(5) "ab'ς"
string(3) "σ'"
string(4) "σ'a"
string(6) "a'σ'a"
string(66) "a...............................................................ς"
string(67) "a................................................................σ"
string(73) "abcς................................................................ abc"
string(74) "abcσ................................................................a abc"
string(265) "abcς................................................................................................................................................................................................................................................................ abc"
Done