Skip to content

[HttpClient] Fix buffering after calling AsyncContext::passthru() #47879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

lubo13
Copy link
Contributor

@lubo13 lubo13 commented Oct 16, 2022

Q A
Branch? 5.4
Bug fix? yes
New feature? no
Deprecations? no
Tickets -
License MIT
Doc PR -

Hi, guys, I think there is an issue with the RetryableHttpClient, let me explain to you what I've seen.

  1. Request is executed into AsyncResponse::__construct

  2. The initializer Closure executes AsyncResponse::passthru, there the AsyncContext and a new stream are created

  3. The next step is yielding from passthruStream and there the stream is checked - $r->stream->valid(). Here the stream is not a valid iterator, because the last retry attempt has already been made and the generator is closed.

  4. This leads the response body stream to be unseekable because the $r->openBuffer() is not called. The content is left with a value null and when AsyncResponse->toStream() is called from Psr18Client::110 the content with value null is bound to the content of StreamWrapper.

  5. After that this newly generated resource is passed to the Psr17Factory->createStreamFromResource() and the resulted (unseekable body) stream is passed to the $response->withBody($body) onto Psr18Client::121

  6. The result is that the Response body stream is not seekable and when I execute two times (string) $response->getBody() after the first attempt the body is an empty string.

Let me know what you think.

Thank you.

@carsonbot
Copy link

Hey!

I see that this is your first PR. That is great! Welcome!

Symfony has a contribution guide which I suggest you to read.

In short:

  • Always add tests
  • Keep backward compatibility (see https://symfony.com/bc).
  • Bug fixes must be submitted against the lowest maintained branch where they apply (see https://symfony.com/releases)
  • Features and deprecations must be submitted against the 6.2 branch.

Review the GitHub status checks of your pull request and try to solve the reported issues. If some tests are failing, try to see if they are failing because of this change.

When two Symfony core team members approve this change, it will be merged and you will become an official Symfony contributor!
If this PR is merged in a lower version branch, it will be merged up to all maintained branches within a few days.

I am going to sit back now and wait for the reviews.

Cheers!

Carsonbot

@carsonbot carsonbot changed the title [Bug]RetryableHttpClient - unseekable body stream RetryableHttpClient - unseekable body stream Oct 16, 2022
@nicolas-grekas
Copy link
Member

The attached patch doesn't look correct to me because it'd mean yielding a $chunk that should be skipped, since it belongs to a response that should be retried instead.

Can you reproduce your scenario in a test case so that we can figure out another fix maybe?

@lubo13
Copy link
Contributor Author

lubo13 commented Oct 17, 2022

Yes, sure it's not the correct one, it's just a dirty hack to try it out if we open the buffer do we receive the seekable stream?
I will try to reproduce it with tests.
But meanwhile, do you have anything in mind is this an expected behaviour?

@nicolas-grekas
Copy link
Member

Not yet sorry. The test case would help a lot!

@lubo13
Copy link
Contributor Author

lubo13 commented Oct 17, 2022

@nicolas-grekas the test has been added, I will waiting for your response
Thank you

@nicolas-grekas nicolas-grekas changed the base branch from 6.2 to 5.4 October 18, 2022 12:03
@carsonbot carsonbot changed the title RetryableHttpClient - unseekable body stream [HttpClient] RetryableHttpClient - unseekable body stream Oct 18, 2022
@nicolas-grekas nicolas-grekas force-pushed the retryable-http-client-unseekable-stream branch from 8cc9e10 to b1d03d1 Compare October 18, 2022 12:04
@nicolas-grekas nicolas-grekas changed the title [HttpClient] RetryableHttpClient - unseekable body stream [HttpClient] Fix buffering of after calling AsyncContext::passthru() Oct 18, 2022
Copy link
Member

@nicolas-grekas nicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I added the fix to the PR)

@nicolas-grekas nicolas-grekas changed the title [HttpClient] Fix buffering of after calling AsyncContext::passthru() [HttpClient] Fix buffering after calling AsyncContext::passthru() Oct 18, 2022
@nicolas-grekas nicolas-grekas force-pushed the retryable-http-client-unseekable-stream branch from b1d03d1 to b559f80 Compare October 18, 2022 12:08
@nicolas-grekas nicolas-grekas removed this from the 6.2 milestone Oct 18, 2022
@nicolas-grekas nicolas-grekas added this to the 5.4 milestone Oct 18, 2022
@nicolas-grekas
Copy link
Member

Thank you @lubo13.

@nicolas-grekas nicolas-grekas merged commit d50fbf6 into symfony:5.4 Oct 18, 2022
@lubo13 lubo13 deleted the retryable-http-client-unseekable-stream branch October 18, 2022 17:00
@lubo13
Copy link
Contributor Author

lubo13 commented Oct 18, 2022

@nicolas-grekas Thank you, could we apply this fix to 6.1 with a new bugfix version? Sorry, but I could find the tag, if you already have created such please share it.

@nicolas-grekas
Copy link
Member

It's already merged up. Releases happen at the end of the month.

@lubo13
Copy link
Contributor Author

lubo13 commented Oct 18, 2022

Okay, Thank you

This was referenced Oct 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants