Open for a PR including a testcase?

I tried this test on the code and tests still pass. Could this be a locale-sensitive behavior?
Can you provide us with a reproducer?

--- a/src/Symfony/Component/DomCrawler/Tests/AbstractCrawlerTest.php
+++ b/src/Symfony/Component/DomCrawler/Tests/AbstractCrawlerTest.php
@@ -265,6 +265,9 @@ abstract class AbstractCrawlerTest extends TestCase
         $crawler = $this->createTestCrawler()->filterXPath('//p');
         $this->assertSame('Elsa <3', $crawler->text(null, true), '->text(null, true) returns the text with normalized whitespace');
         $this->assertNotSame('Elsa <3', $crawler->text(null, false));
+
+        $document = $this->createCrawler('<?xml version="1.0" encoding="utf-8" standalone="yes"?><node>mą</node>');
+        $this->assertSame('mą', $document->text(null, true));
     }
 

Hmm... You're right. I cannot reproduce the error within a testcase in this repository.
I'll investigate and try to create a working test case.

@nicolas-grekas Yes, it's a behavior, that appears when setlocale is called before creating the Crawler:

setlocale(LC_CTYPE, null);
$document = $this->createCrawler('<?xml version="1.0" encoding="utf-8" standalone="yes"?><node>mą</node>');
$this->assertSame('mą', $document->text(null, true));

I know it's incorrect to call setlocale with null as second parameter 😉 The call is somewhere deep within the CMS used (TYPO3) and only happens in a specific context, when a method is called before the configuration is available:
https://github.com/TYPO3/typo3/blob/10.4/typo3/sysext/core/Classes/Utility/PathUtility.php#L165-L172

The u-modifier within the regular expression in the text() method fixes this bug.

What's your system locale? Eg what does setlocale(LC_CTYPE, '0'); return for you?

setlocale(LC_CTYPE, '0') returns the string C.

I'm sorry I'm unable to reproduce.
We cannot blindly add the u modifier as that might 1. break parsing non-UTF8 html 2. trim more "spaces" than required.

Hm, very strange. I am currently using the following test-file (without symfony or any other dependencies):

<?php

setlocale(LC_CTYPE, null);
$text = 'mą';
$replaced = preg_replace('/(?:\s{2,}+|[^\S ])/', ' ', $text);
var_dump($replaced);
print_r(array_map('dechex', array_map('ord', preg_split('//', $replaced))));

I am using macOS 12.4 with PHP 8.1.5 (installed using Homebrew). The following command returns the incorrect string:

$ LC_ALL=en_US.UTF-8 php -f test.php

string(3) "m� "
Array
(
    [0] => 0
    [1] => 6d
    [2] => c4
    [3] => 20
    [4] => 0
)

The following command returns the correct code:

$ LC_ALL=C php -f test.php
(or even invalid LC)
$ LC_ALL=invalid php -f test.php

string(3) "mą"
Array
(
    [0] => 0
    [1] => 6d
    [2] => c4
    [3] => 85
    [4] => 0
)

I was not able to reproduce the buggy behavior on another system (I tried a couple linux/unix servers which all worked fine). But I was able to reproduce on another Mac. Which system are you using?

I'm on Linux. We could try replacing those \s by an explicit character class.
But this is still strange and this creates a doubt for the other \s in the code 😮‍💨

It's not the \s character class, but the \S class:

<?php

setlocale(LC_CTYPE, null);
$text = 'ą';
echo 'Original: ' . bin2hex($text) . "\n";

$replaced = preg_replace('/[^\S ]/', ' ', $text);

echo 'Replaced: ' . bin2hex($replaced) . "\n";

This outputs

Original: c485
Replaced: c420

This means, the "character" 85 get's replaced by 20 (a space). Maybe preg_replace sees the 85 as two chars. One \S and one space.

Looks like pcre is using ctype to decide what is a space. Out of curiosity, what's the output of this script for you?
Also, what's the output of locale on your machine? I'm trying to figure in which locale x85 is a space.

Anyway, fixed in #47175

The script outputs the following:

9
a
b
c
d
20
85
a0

locale on Terminal outputs the following:

LANG="de_DE.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL="en_US.UTF-8"

I tested your commit and it works fine 👍 thanks :)

I rechecked my PHP version. Initially I said that I am using PHP 8.1.5. But the CLI used PHP 7.4.30.
I've got three different PHP versions installed:
PHP 8.1.8 and 8.0.21 work fine. (I've installed updates, that's why it's no longer 8.1.5)
PHP 7.4.30 is still buggy.

Maybe this is a strange PHP macOS Bug?

Anyways: Your commit fixes it for 7.4.30 and does not break anything in 8.1 or 8.0 for me :)

Thanks for all the insights. Seeing 85 there is unexpected to me but that goes beyond Symfony's responsibility. At least here we're done :)