Skip to content

QuestionHelper input breaks UTF-8 characters #37495

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Bast1onCZ opened this issue Jul 5, 2020 · 3 comments · Fixed by #41113
Closed

QuestionHelper input breaks UTF-8 characters #37495

Bast1onCZ opened this issue Jul 5, 2020 · 3 comments · Fixed by #41113

Comments

@Bast1onCZ
Copy link

Bast1onCZ commented Jul 5, 2020

Symfony version(s) affected: 5.1.0

Description
When an UTF-8 character is confirmed in a console as a response to QuestionHelper's question, it gets replaced by '\' or ''.
E.g. 'Čeština' > 'e\tina'. My OS is Windows 10.

How to reproduce

// inside Command::execute

$response = $questionHelper->ask($input, $output,
    new Question("Write something containing UTF-8 characters")
);
// response contains UTF-8 characters replaced by '\' or removed
@chalasr
Copy link
Member

chalasr commented Jul 5, 2020

On which OS/Terminal are you experiencing this?

@chalasr chalasr added the Console label Jul 5, 2020
@Bast1onCZ
Copy link
Author

Bast1onCZ commented Jul 6, 2020

On which OS/Terminal are you experiencing this?

It's Windows 10.

I'd like also note, that when passing UTF-8 characters to commands as arguments, it works properly, so there must be something wrong with question utilities

@chalasr
Copy link
Member

chalasr commented Jul 6, 2020

Looks like it is the same problem as reported in #37278, should be fixed by #37385.

chalasr added a commit that referenced this issue Jul 6, 2020
This PR was merged into the 3.4 branch.

Discussion
----------

[Console] Fixes question input encoding on Windows

| Q             | A
| ------------- | ---
| Branch?       | 3.4 <!-- see below -->
| Bug fix?      | yes
| New feature?  | no <!-- please update src/**/CHANGELOG.md files -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tickets       | Fix #35842, Fix #36324, Fix #37495 and Fix #37278 <!-- prefix each issue number with "Fix #", if any -->
| License       | MIT
| Doc PR        | no <!-- required for new features -->
<!--
Replace this notice by a short README for your feature/bugfix. This will help people
understand your PR and can be used as a start for the documentation.

Additionally (see https://symfony.com/releases):
 - Always add tests and ensure they pass.
 - Never break backward compatibility (see https://symfony.com/bc).
 - Bug fixes must be submitted against the lowest maintained branch where they apply
   (lowest branches are regularly merged to upper ones so they get the fixes too.)
 - Features and deprecations must be submitted against branch master.
-->

To ask a question to a user, the [QuestionHelper](https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Console/Helper/QuestionHelper.php) use [`fgets`](https://www.php.net/manual/en/function.fgets.php). However, special characters are not supported on Windows with this function (like accents: `é`, `à`, `ö`). The solution is to set a special encoding with [`sapi_windows_cp_get`](https://www.php.net/manual/en/function.sapi-windows-cp-get).

> Before
```php
$stream = fopen('php://stdin', 'r');
$input = fgets($stream);
echo $input;

// input: "Bonjour à tous"
// output: 'Bonjour \ tous" or "Bonjour   tous"
```

> After
```php
// Check if the function exists because it only exists on Windows
if(function_exists('sapi_windows_cp_set')) {
    sapi_windows_cp_get(437);
}

$stream = fopen('php://stdin', 'r');
$input = fgets($stream);
echo $input;

// input: "Bonjour à tous"
// output: 'Bonjour à tous"
```

*Thanks to @bnjmnfnk for the solution 😉*

Commits
-------

4288df4 [Console] Fixes question input encoding on Windows
@chalasr chalasr closed this as completed Jul 6, 2020
nicolas-grekas added a commit that referenced this issue May 7, 2021
This PR was submitted for the 5.x branch but it was squashed and merged into the 5.2 branch instead.

Discussion
----------

[Console] Fix Windows code page support

| Q             | A
| ------------- | ---
| Branch?       | 5.2
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Tickets       | Fix #37385, Fix #35842, Fix #36324, Fix #37495, Fix #37278
| License       | MIT

Corrects previous fixes that dealt with the mojibake problem on Windows where an OEM code page was applied to an input string and then messed with PHP.internal_encoding setting used by the script. This caused strings with different encodings to be displayed on the console output.

Commits
-------

be68682 [Console] Fix Windows code page support
nicolas-grekas added a commit that referenced this issue May 13, 2021
This PR was merged into the 4.4 branch.

Discussion
----------

[Console] Fix Windows code page support

| Q             | A
| ------------- | ---
| Branch?       | 4.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Tickets       | Fix #37385, Fix #35842, Fix #36324, Fix #37495, Fix #37278
| License       | MIT

Corrects mojibake problem on Windows where an OEM code page was applied to an input string and then messed with PHP.internal_encoding setting used by the script. This caused strings with different encodings to be displayed on the console output.

Commits
-------

4145278 [Console] Fix Windows code page support
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants