Skip to content

[Process][Console] deprecated defining commands as strings #27821

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 9, 2018

Conversation

nicolas-grekas
Copy link
Member

@nicolas-grekas nicolas-grekas commented Jul 3, 2018

Q A
Branch? master
Bug fix? no
New feature? yes
BC breaks? no
Deprecations? yes
Tests pass? yes
Fixed tickets #27796
License MIT
Doc PR -
  • Added the Process::fromShellCommandline() static constructor to define shell command-lines
  • Allowed passing commands as array($process, 'ENV_VAR' => 'value') to ProcessHelper::run()
  • Deprecated passing commands as strings when creating a Process instance.
  • Deprecated the Process::setCommandline() and the PhpProcess::setPhpBinary() methods.
  • Deprecated passing a command as a string to ProcessHelper::run(), pass it the command as an array of arguments instead.
  • Made the ProcessHelper class final

@stof
Copy link
Member

stof commented Jul 3, 2018

this does not running things like git fetch origin && git checkout v1.2.1 anymore, right ? What is the expected way to run any chaining (&&, ||, |, etc...) ?

@stof
Copy link
Member

stof commented Jul 3, 2018

and for anyone asking, && can be used for cross-platform commands, so this does not even contradicts the cross-platform support (and composer uses that a lot if you need an example)

UPGRADE-4.2.md Outdated
* Deprecated the `setHorizontalBorderChar()` method in favor of the `setDefaultCrossingChars()` method in `TableStyle`.
* Deprecated the `getHorizontalBorderChar()` method in favor of the `getBorderChars()` method in `TableStyle`.
* Deprecated the `setVerticalBorderChar()` method in favor of the `setVerticalBorderChars()` method in `TableStyle`.
* Deprecated the `getVerticalBorderChar()` method in favor of the `getBorderChars()` method in `TableStyle`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

already part of UPGRADE-4.1, to revert?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reverted

@nicolas-grekas
Copy link
Member Author

nicolas-grekas commented Jul 3, 2018

git fetch origin && git checkout v1.2.1

$process = Process::fromShellCommandline('git fetch origin && git checkout v1.2.1');

@Toflar
Copy link
Contributor

Toflar commented Jul 4, 2018

Cool, thanks for working on that! I think it clearly reduces the chance of an accidental mistake. Now you can only pass an array and you have to explicitly ask the Process component for shell command lines using wihtShell(). In my opinion this is way better DX than the current "I accept both, array or string" interface.

UPGRADE-4.2.md Outdated
-------

* Deprecated passing a command as a string to `ProcessHelper::run()`,
pass it the command as an array of arguments instead.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pass the command

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also add a word about the new withShell() method (as passing an array of arguments is not always possible, perhaps mention the cases where using withShell() is "valid"?).

-------

* Deprecated the `Process::setCommandline()` and the `PhpProcess::setPhpBinary()` methods.
* Deprecated passing commands as strings when creating a `Process` instance.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

@@ -21,6 +21,18 @@ Console
* Removed the `getHorizontalBorderChar()` method in favor of the `getBorderChars()` method in `TableStyle`.
* Removed the `setVerticalBorderChar()` method in favor of the `setVerticalBorderChars()` method in `TableStyle`.
* Removed the `getVerticalBorderChar()` method in favor of the `getBorderChars()` method in `TableStyle`.
* The `ProcessHelper::run()` method takes the command as an array of arguments.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

4.2.0
-----

* allowed passing commands as `array($process, 'ENV_VAR' => 'value')` to `ProcessHelper::run()`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

@@ -330,7 +330,7 @@ private function resolveCommands($value)
throw new \LogicException('Resolving commands requires the Symfony Process component.');
}

$process = new Process('echo '.$matches[0]);
$process = \method_exists(Process::class, 'withShell') ? Process::withShell('echo '.$matches[0]) : new Process('echo '.$matches[0]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about adding a comment to not forget simplifying this piece of code again in 5.0?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'll have many places to clean up like this when preparing 5.0, we will spot it with the usual grep method_exists

@@ -163,6 +167,23 @@ public function __construct($commandline, string $cwd = null, array $env = null,
$this->pty = false;
}

/**
* @param string $command The command line to pass to the shell of the OS
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a description explaining the "valid" use cases for using this method.

@javiereguiluz
Copy link
Member

I don't like the ::withShell() name much. It's not easy to understand or explain. Even the proposed explanation doesn't say much: "::withShell() defines shell command-lines"

Also, what's the reasoning behind the deprecation of Process::setCommandline() and PhpProcess::setPhpBinary()? Thanks!

@Toflar
Copy link
Contributor

Toflar commented Jul 5, 2018

I don't like the ::withShell() name much. It's not easy to understand or explain. Even the proposed explanation doesn't say much: "::withShell() defines shell command-lines"

I agree with @javiereguiluz here. If you're not into processes that much it's hard to understand. Not sure how to name it though. Maybe we should start by trying to define the difference in 1 sentence. That could help us to come up with a good name?

@nicolas-grekas
Copy link
Member Author

Maybe take some inspiration from symfony/symfony-docs#9988 recently?

what's the reasoning behind the deprecation of Process::setCommandline() and PhpProcess::setPhpBinary

They are abstraction leaks. A process should not be mutable for these settings. They completely defeat the as array() argument (new Process([])->setCommandline('...') as a workaround is not go)

@Toflar
Copy link
Contributor

Toflar commented Jul 5, 2018

Process::createShellSpecific()?

@javiereguiluz
Copy link
Member

javiereguiluz commented Jul 5, 2018

If we don't want to explain all the internal details of using these "shell commands", maybe we can rename ::withShell() as ::asString() (or ::fromString(), etc.) ?

$result = (new Process(['ls', '-al']))->getOutput();

$result = (Process::asString('ls -al'))->getOutput();

@Toflar
Copy link
Contributor

Toflar commented Jul 5, 2018

This would render the whole idea kind of useless. Because then we can also just keep everything as is and accept both, an array and a string. People will just update and ask themselves why the constructor is now an array and just modify and use ::asString() to get rid of the deprecation message. Then we still did not achieve that the devs understand the difference and choose the right thing for the right purpose.

@javiereguiluz
Copy link
Member

@Toflar can you please explain to me the difference between them, or the consequences of using this shell things vs the other thing? Imagine that I'm not a tech person, so the explanation is a simple as possible. That way we'll be able to find the right name. Thanks!

@Toflar
Copy link
Contributor

Toflar commented Jul 5, 2018

I'm struggling with finding a good name myself. I basically raised this issue because @nicolas-grekas tweetet about the difference saying something like "did you know you can pass process arguments as an array?". I checked his linked docs PR (symfony/symfony-docs#9988) and I just noticed that the docs are nice but there's room for improvement code-wise. Right now, the Process component does just happily accept both, an array or a string as constructor argument. When you pass it as a string, the arguments are not escaped and passed on to the shell as is. This means you as a developer can benefit from additional features such as stream redirections but you are also responsible to use the correct syntax and escape everything needed yourself which is dependent on the underlying OS. So e.g. if you develop on Windows and deploy to Linux, your command might fail.
If you pass it as an array though, the Process component escapes everything correctly according to the OS you are currently working on.

And my argument was that in 90% of all cases, the commands are likely rather simple and don't need any special shell feature. Or in other words: 90% of all our devs want to pass an array not a string. I just feel like that with this PR we can protect devs from some unforeseen compatibility issues which is something worth working towards imho 😄

@nicolas-grekas
Copy link
Member Author

nicolas-grekas commented Jul 5, 2018

@javiereguiluz asString is a very confusing name because it hides the fact that there are major differences between the array definition and the "withShell" one: we do not want to hide the fact that there is a shell in between. "string" is only the tip of the iceberg, there is a world of differences.

The fact that the shell is there is the reason why we can use e.g. > in these strings to express stream redirection, but it's a regular argument in the array syntax. That's also the reason why signaling doesn't work as one would expect most of the time. Doc PR symfony/symfony-docs#9988 removes sentences stating "Due to some limitations in PHP [...] you may have to prefix your commands with exec". That's a very misleading statement: it has nothing to do to PHP but related to the fact that commands are run by a shell. Most comments on http://php.net/proc_open also are about how confusing it is to run commands via a shell.

So, when using the array syntax, conceptually, you do not use a shell. Escaping becomes a foreign concept (and thus saying is "there is no need to escape" builds on no ground - there is nothing to escape for or against.)
As soon as you use a shell, you enter another world, where it becomes your responsibility to write the command in a language the shell understands. Since this is a language, it has a grammar, and thus a way to escape strings so that their content cannot be confused with any meaningful tokens in this language. That's where escaping comes into play.

PS: even when defining command in the shell's language, people should still NOT escape strings themselves. This is very very similar to prepared statements: you do NOT want to escape your SQL parameters yourself because that's actually impossible to achieve in a portable and secure way. Instead, you use placeholders in prepared statements and bind the values when executing. That's exactly what ppl should do for commands also, and what https://github.com/symfony/symfony-docs/pull/9988/files#diff-2e6951f193acece85d23e18380939b51R122 explains.

I hope this help understand the reasons for this PR.

@javiereguiluz
Copy link
Member

javiereguiluz commented Jul 5, 2018

@Toflar @nicolas-grekas your comments were perfectly clear. Thanks! I propose other names then:

$result = (new Process(['ls', '-al']))->getOutput();

$result = (Process::raw('ls -al'))->getOutput();
$result = (Process::rawCommand('ls -al'))->getOutput();
$result = (Process::unescaped('ls -al'))->getOutput();
$result = (Process::unescapedCommand('ls -al'))->getOutput();

@nicolas-grekas
Copy link
Member Author

Process::fromShellCommandline('...'): we don't care if it's longer when it's not the recommended way, and telling there is a shell there is the missing hint that creates so much confusion right now.

@Toflar
Copy link
Contributor

Toflar commented Jul 5, 2018

It's not really about escaping then, so I'd go with Process::fromShellCommandline('...') and for the docs I think there should be some more examples probably.

@javiereguiluz
Copy link
Member

I don't care about the method name length. My concern is that ::fromShellCommandline() is not self-explanatory. You need to know a lot about this to be able o understand it without reading some doc.

@nicolas-grekas
Copy link
Member Author

nicolas-grekas commented Jul 5, 2018

IMHO, fromShellCommandline is the most self-explanatory proposal so far. And yes, we don't want ppl to jump using this without reading some docs. That's not the recommended way for many reasons (see above).

@@ -44,9 +46,22 @@ public function run(OutputInterface $output, $cmd, $error = null, callable $call
$formatter = $this->getHelperSet()->get('debug_formatter');

if ($cmd instanceof Process) {
$process = $cmd;
} else {
$cmd = array($cmd);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why wrapping the Process object in an array, to unwrap it after that ? this looks really weird.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to normalize the data structure and simplify the following lines

$process = new Process($cmd);
$cmd = array();
} elseif (($cmd[0] ?? null) instanceof Process) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is no adding support for passing an array with a Process as first argument (and other stuff after that). does it actually make sense ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See PR description:

Allowed passing commands as array($process, 'ENV_VAR' => 'value') to ProcessHelper::run()

@nicolas-grekas
Copy link
Member Author

Thanks @fabpot, comments addressed.

* @param string $command The command line to pass to the shell of the OS
* @param string|null $cwd The working directory or null to use the working dir of the current PHP process
* @param array|null $env The environment variables or null to use the same environment as the current PHP process
* @param mixed|null $input The input as stream resource, scalar or \Traversable, or null for no input
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

null is part of mixed :)

@fabpot
Copy link
Member

fabpot commented Jul 9, 2018

Thank you @nicolas-grekas.

@fabpot fabpot merged commit 8895bc1 into symfony:master Jul 9, 2018
fabpot added a commit that referenced this pull request Jul 9, 2018
…ings (nicolas-grekas)

This PR was merged into the 4.2-dev branch.

Discussion
----------

[Process][Console] deprecated defining commands as strings

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | yes
| Tests pass?   | yes
| Fixed tickets | #27796
| License       | MIT
| Doc PR        | -

 * Added the `Process::fromShellCommandline()` static constructor to define shell command-lines
 * Allowed passing commands as `array($process, 'ENV_VAR' => 'value')` to `ProcessHelper::run()`
 * Deprecated passing commands as strings when creating a `Process` instance.
 * Deprecated the `Process::setCommandline()` and the `PhpProcess::setPhpBinary()` methods.
 * Deprecated passing a command as a string to `ProcessHelper::run()`, pass it the command as an array of arguments instead.
 * Made the `ProcessHelper` class final

Commits
-------

8895bc1 [Process][Console] deprecated defining commands as strings
@nicolas-grekas nicolas-grekas deleted the proc-array branch July 22, 2018 20:25
@nicolas-grekas nicolas-grekas modified the milestones: next, 4.2 Nov 1, 2018
This was referenced Nov 3, 2018
javiereguiluz added a commit to symfony/symfony-docs that referenced this pull request Nov 14, 2018
This PR was merged into the master branch.

Discussion
----------

Update doc for Process v4.2

Follows symfony/symfony#27821
Fixes #10299

Commits
-------

3a0a450 Update doc for Process v4.2
fabpot added a commit that referenced this pull request Sep 10, 2023
…tion (derrabus)

This PR was merged into the 6.4 branch.

Discussion
----------

[Console] Remove `Process::fromShellCommandline()` detection

| Q             | A
| ------------- | ---
| Branch?       | 6.4
| Bug fix?      | no
| New feature?  | no
| Deprecations? | no
| Tickets       | N/A
| License       | MIT
| Doc PR        | N/A

This method exists since #27821 (Symfony 4.2)

Commits
-------

12ee32d [Console] Remove Process::fromShellCommandline() detection
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants