Skip to content

[HttpClient] Prevent "Fatal error" in data collector #46700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 29, 2022
Merged

[HttpClient] Prevent "Fatal error" in data collector #46700

merged 1 commit into from
Jun 29, 2022

Conversation

fmata
Copy link
Contributor

@fmata fmata commented Jun 17, 2022

Q A
Branch? 6.1
Bug fix? yes
New feature? no
Deprecations? no
Tickets Fix #46604
License MIT

Since "copy as cURL" has been added in Profiler for HttpClient usage, when payloads exceed the size allowed by escapeshellarg() we have a Fatal error 😞

The right allowed size depends on OS and is not leaked in PHP land so I prefer to be conservative and allow the lower usual value as explained in PHP source code https://github.com/php/php-src/blob/9458f5f2c8a8e3d6c65cc181747a5a75654b7c6e/ext/standard/exec.c#L77

I only added check where I think it's relevant.

*/
private function argMaxLengthIsSafe(string $payload): bool
{
return strlen($payload) <= 4096;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4096 bytes is super small, this must be a mistake since you mention 4Mo in the comment
please also move the method at the bottom of the class and add a leading slash to \strlen

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the PHP codebase, the size seems to be in bytes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please also move the method at the bottom of the class and add a leading slash to \strlen

yes sir 👍, I thought php-cs-fixer would have pointed this out to me

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fabbot (aka php-cs-fixer) is on vacation I think these days, it doesn't show up in the CI reports since a few weeks :P

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the PHP codebase, the size seems to be in bytes.

OK but then we cannot go by this limit, it's way too low I think.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On my machine, I can go up to escapeshellarg(str_repeat('-', 2097149));.
What about others?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, as seen in the PHP source code, the value of cmd_max_len can be set in a bunch of different ways during compilation. So it can indeed be higher on some systems. But this is not exposed to userland.

anyway, disabling the copy as curl feature in the profile is better than doing a fatal error when running the HTTP request

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stof because I messed up 🤦

@nicolas-grekas on my Ubuntu 22.04 (same on Debian 11)

$ getconf ARG_MAX
2097152

Copy link
Member

@nicolas-grekas nicolas-grekas Jun 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So apparently on Windows we can't go above 8kio, and on linux we need to set a reasonably portable limit. Here is a suggestion, going a bit below the limit to account for escaping if that matters?
return \strlen($payload) < ('\\' === \DIRECTORY_SEPARATOR ? 8100 : 256000);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it ty, done.

@fmata
Copy link
Contributor Author

fmata commented Jun 28, 2022

@nicolas-grekas what is missing in this PR to be merged & released ? Last 6.1 patch was released 2 days ago 😞 We are facing fatal errors in dev environment on each request using HttpClient (we use it to upload PDF files).

Copy link
Member

@nicolas-grekas nicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It felt off my radar before the release sorry.

@nicolas-grekas
Copy link
Member

Thank you @fmata.

@nicolas-grekas nicolas-grekas merged commit c4d1c57 into symfony:6.1 Jun 29, 2022
@fmata fmata deleted the httpclient-escapeshellarg branch July 10, 2022 08:08
@fabpot fabpot mentioned this pull request Jul 29, 2022
@DavidBadura
Copy link
Contributor

FYI: Unfortunately, ARG_MAX varies from system to system and the hardcoded limit would have to be lowered even further for alpine. I created a follow-up ticket for it with more insides: #49693

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants