Skip to content

[HttpClient] CurlHttpClient forces unsupported chunked transfer encoding for HTTP/2 requests #54516

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
michaelhue opened this issue Apr 7, 2024 · 0 comments

Comments

@michaelhue
Copy link
Contributor

Symfony version(s) affected

7.0

Description

When sending a FormData request (and possibly other chunkable body types), CurlHttpClient will set a Transfer-Encoding: chunked header:

if (!isset($options['normalized_headers']['transfer-encoding'])) {
$curlopts[\CURLOPT_HTTPHEADER][] = 'Transfer-Encoding:'.(isset($curlopts[\CURLOPT_INFILESIZE]) ? '' : ' chunked');
}

  • The HTTP/2 protocol does not support chunked transfer encoding.
  • Normally curl would handle the correct format itself based on the HTTP protocol version of the server.
  • But when the Transfer-Encoding header is set to chunked explicitly, curl always uses chunked transfer encoding for the body.
  • This results in requests to HTTP/2 server with invalid content.

For example, given a request with FormData ['foo' => 'bar'], an HTTP/1 server receives this:

Headers {
  accept: "*/*",
  "accept-encoding": "gzip",
  "content-type": "multipart/form-data; boundary=1It362u0",
  host: "localhost:8000",
  "transfer-encoding": "chunked",
  "user-agent": "Symfony HttpClient (Curl)"
}
FormData { foo: "bar" }

While an HTTP/2 server sees this:

Headers {
  accept: "*/*",
  "accept-encoding": "gzip",
  "content-type": "multipart/form-data; boundary=Mm4P-FZG",
  "user-agent": "Symfony HttpClient (Curl)"
}
FormData { foo: "2\r\n\r\n\r\n3\r\nbar\r\n2\r\n\r\n\r\ne" }

curl does omit the transfer-encoding header, but keeps the encoding for the body.

How to reproduce

Repo for reproducing the issue:
https://github.com/michaelhue/symfony-curl-http2-chunked-issue

The test source code can be found here:
https://github.com/michaelhue/symfony-curl-http2-chunked-issue/blob/main/test.php

Possible Solution

It seems to me that the following lines can be removed completely, which will fix the issue:

if (!isset($options['normalized_headers']['transfer-encoding'])) {
$curlopts[\CURLOPT_HTTPHEADER][] = 'Transfer-Encoding:'.(isset($curlopts[\CURLOPT_INFILESIZE]) ? '' : ' chunked');
}

However my understanding of curl and the Symfony internals is limited and I can't anticipate the potential fallout from this change.

I'd be happy to create a pull request with this solution, if deemed appropriate.

Additional Context

This problem was discovered while trying to debug this issue: #54491

@michaelhue michaelhue added the Bug label Apr 7, 2024
nicolas-grekas added a commit that referenced this issue Apr 8, 2024
This PR was squashed before being merged into the 5.4 branch.

Discussion
----------

[HttpClient] Let curl handle transfer encoding

| Q             | A
| ------------- | ---
| Branch?       | 5.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Issues        | Fix #54516, Fix #54491
| License       | MIT

Removes the `Transfer-Encoding: chunked` header that is added for chunked request bodies. This will allow curl to handle the transfer encoding and set appropriate headers based on the HTTP protocol version it negotiated with the server, since HTTP/2 does not support chunked transfer encoding.

Commits
-------

69a7867 [HttpClient] Let curl handle transfer encoding
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants