Can you please add a test for this? It should run on windows only.

Can you please add a test for this? It should run on windows only.

Is there a group in PHPUnit to run only on Windows (like @group windows)? Else, how I can do it? I can also check if the function sapi_windows_cp_get exists...

If the function exists only on windows, it should be enough yes.
If you aren't sure, you can use a '\\' === DIRECTORY_SEPARATOR additional check for both code and tests, see e.g.

Ok thank you! I'm writing it.

It's difficult to test it because the stream used for QuestionHelper tests is the PHP memory php://memory. However, this issue is only present in the console with php://stdin.
Does anyone have any idea?

You can try phpt test case similar as https://github.com/symfony/symfony/blob/80d1f449831fbceefd481544c82f8244759eda4d/src/Symfony/Component/Console/Tests/phpt/uses_stdin_as_interactive_input.phpt

You can try phpt test case similar as https://github.com/symfony/symfony/blob/80d1f449831fbceefd481544c82f8244759eda4d/src/Symfony/Component/Console/Tests/phpt/uses_stdin_as_interactive_input.phpt

This solution does not work 😞

Can you try the following test locally? Without your patch at first, in order to make sure it breaks under your setup.

--TEST--
QuestionHelper should parse special chars on windows input.
--STDIN--
à é
--FILE--
<?php

use Symfony\Component\Console\Input\ArgvInput;
use Symfony\Component\Console\Output\ConsoleOutput;
use Symfony\Component\Console\Helper\QuestionHelper;
use Symfony\Component\Console\Question\Question;

$vendor = __DIR__;
while (!file_exists($vendor.'/vendor')) {
    $vendor = dirname($vendor);
}
require $vendor.'/vendor/autoload.php';

$output = new ConsoleOutput();
$output->write((new QuestionHelper())->ask(new ArgvInput(), $output, new Question("Special chars\n")));
exit(0);
?>
--EXPECTF--
Special chars
à é

Can you try the following test locally? Without your patch at first, in order to make sure it breaks under your setup.

--TEST--
QuestionHelper should parse special chars on windows input.
--STDIN--
à é
--FILE--
<?php

use Symfony\Component\Console\Input\ArgvInput;
use Symfony\Component\Console\Output\ConsoleOutput;
use Symfony\Component\Console\Helper\QuestionHelper;
use Symfony\Component\Console\Question\Question;

$vendor = __DIR__;
while (!file_exists($vendor.'/vendor')) {
    $vendor = dirname($vendor);
}
require $vendor.'/vendor/autoload.php';

$output = new ConsoleOutput();
$output->write((new QuestionHelper())->ask(new ArgvInput(), $output, new Question("Special chars\n")));
exit(0);
?>
--EXPECTF--
Special chars
à é

No more results... I think PHPUnit simulate and standard input but does not take all behaviors of the fgets function.

CP1252 allows more diacritics and is already the default on my French VM. CP850 would be an alternate possibility.

Can you try and take 1252 if possible, 850 otherwise?

Oh, note that codepage 65001 is UTF-8, it would be even better. Can you give it a try?

CP1252 allows more diacritics and is already the default on my French VM. CP850 would be an alternate possibility.

Can you try and take 1252 if possible, 850 otherwise?

Oh, note that codepage 65001 is UTF-8, it would be even better. Can you give it a try?

Only 1252 and 850 work (and 437 of course). I think the better code page is 1252 because it provides a better characters support.

Does the function return false is your case with 65001?
If yes, we could set(65001) ?: set(1252) maybe?

Does the function return false is your case with 65001?
If yes, we could set(65001) ?: set(1252) maybe?

No it returns true but replaces à by nothing or \.

Thanks for fixing this bug @YaFou.

I'm facing this issue again in 5.2.x-dev, i'm using windows 10, before i ask question, all utf8 chars are wrong.

I'm facing this issue again in 5.2.x-dev, i'm using windows 10, before i ask question, all utf8 chars are wrong.

Thank you for your feedback @lpj145. Can you provide a screenshot from your terminal? Are you on Windows?

With this change, the value returned by the helper on Windows is a binary string. Take this example command:

<?php

namespace App\Command;

use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Console\Question\Question;

class TestCommand extends Command
{
    protected static $defaultName = 'app:test';

    protected function execute(InputInterface $input, OutputInterface $output)
    {
        $helper = $this->getHelper('question');
        $question = new Question('What? ');

        dump($helper->ask($input, $output, $question));

        return Command::SUCCESS;
    }
}

This is the output for the input martínez:

C:\Develop\htdocs\testing\symfony>php bin\console app:test
What? martínez
^ b"martínez"

If I remove the call to sapi_windows_cp_get() the result is this:

C:\Develop\htdocs\testing\symfony>php bin\console app:test
What? martínez
^ "mart\x00nez"

This issue was initially reported here, but after some tests I think the issue is caused by this change. I tested it using three terminal emulators: CMD, PowerPhell and Git Bash:

I have a similar issue on Windows 10, Symfony 5.2.6, Hungarian OS environment and chars like áíűőüöúóé.

After first ansfer output gets wrong charset even if I don't use any special characters in questions or answers. And it's not just the output, all the inner encoding settings become strange, e.g. An exception has been thrown during the rendering of a template ("Invalid UTF-8 string.") exception during execution.

Works:

$command = $this->getApplication()->find('generate');

$generateArguments = [
    'project' => 'local',
];

return $command->run(new ArrayInput($generateArguments), $output);

Error:

$helper = $this->getHelper('question');

$projectQuestion = new Question("Projekt: </> \n", 'local');
$project = $helper->ask($input, $output, $projectQuestion);

$command = $this->getApplication()->find('generate');

$generateArguments = [
    'project' => $project,
];

return $command->run(new ArrayInput($generateArguments), $output);