Skip to content

[HttpClient] Make CachingHttpClient compatible with RFC 9111 #59576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 53 commits into
base: 7.4
Choose a base branch
from

Conversation

Lctrs
Copy link
Contributor

@Lctrs Lctrs commented Jan 21, 2025

Q A
Branch? 7.3
Bug fix? no
New feature? yes
Deprecations? yes
Issues Fix #36858 Fix #49937 Fix #41843
License MIT

https://www.rfc-editor.org/rfc/rfc9111.html

@carsonbot

This comment has been minimized.

@Lctrs Lctrs marked this pull request as ready for review January 21, 2025 23:54
@carsonbot carsonbot added this to the 7.3 milestone Jan 21, 2025
@stof
Copy link
Member

stof commented Jan 22, 2025

Can we bring this to CachingHttpClient (with a BC layer) instead of introducing a new class that is the good implementation of caching but with a name that is much less clear ?
Also, having the RFC id in the class name is a bad idea to me, as it would be a pain if a new RFC gets released to replace the current one (which happened multiple times for RFCs related to HTTP)

@Lctrs Lctrs force-pushed the rfc9111-http-client branch from 62db76a to 5b52dd2 Compare February 18, 2025 20:59
@Lctrs Lctrs changed the title [HttpClient] Add an RFC 9111 compliant client [HttpClient] Make CachingHttpClientCompatible with RFC 9111 Feb 18, 2025
@Lctrs Lctrs changed the title [HttpClient] Make CachingHttpClientCompatible with RFC 9111 [HttpClient] Make CachingHttpClient compatible with RFC 9111 Feb 18, 2025
@Lctrs
Copy link
Contributor Author

Lctrs commented Feb 19, 2025

UPDATE:

  • CI failure on PHP 8.5 is unrelated
  • Psalm failure seems to be a false positive to me
  • Tests pass locally on low deps, but not on github actions. I need to look at it.
  • I don't know how to please fabbot. Should I not make any change to the upgrade file ?

Otherwise, PR is ready on my side. Ready for another reviews.

@Lctrs Lctrs force-pushed the rfc9111-http-client branch 2 times, most recently from b7a0bb4 to 12bf66f Compare February 20, 2025 16:16
@Lctrs Lctrs requested a review from stof February 21, 2025 08:45
@Lctrs Lctrs force-pushed the rfc9111-http-client branch from e93f3b2 to d0be468 Compare February 22, 2025 23:19
@OskarStark OskarStark changed the title [HttpClient] Make CachingHttpClient compatible with RFC 9111 [HttpClient] Make CachingHttpClient compatible with RFC 9111 Feb 24, 2025
@Lctrs Lctrs force-pushed the rfc9111-http-client branch 3 times, most recently from b9df14d to 0405dc5 Compare February 28, 2025 10:51
@Lctrs Lctrs force-pushed the rfc9111-http-client branch from b13e679 to 7c66ffd Compare April 16, 2025 14:09
$varyFields = $this->cache->get($varyKey, static fn (): array => []);

$metadataKey = self::getMetadataKey($requestHash, $options['normalized_headers'], $varyFields);
$cachedData = $this->cache->get($metadataKey, static fn (): null => null);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to save the null value in the backed on a miss?
if not, we should use the $save argument passed by ref to the closure:

Suggested change
$cachedData = $this->cache->get($metadataKey, static fn (): null => null);
$cachedData = $this->cache->get($metadataKey, fn ($item, &$save) => $save = false;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(the same concern should be consider everywhere get() is used)

Copy link
Member

@fabpot fabpot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does the code behave when there is 2 concurrent requests for the same URL where one request is creating the cache while the other tries to get the cache? At the bare minimum, maybe we should lock the key?
Also, maybe the implementation should take into account the fact that we could implement stale-while-revalidate support later on.

@nicolas-grekas
Copy link
Member

Not sure about locking as a first step, maybe just some pointer would be enough: while writing, we use some random key name, and once the cache is populated, we update some other pointer key, this one predictably derived from the request. When the cache is stale or empty, that'd mean writing many concurrent streams to the cache backend for the same request, until one finished and wins of the others. No ideal, but at least correct from a behavioral perspective.
The next step would be to wrap everything in one may $cache->get() callback, which would the act as the lock unit, possibly capable of serving something while revalidating and/or waiting/streaming while the concurrent write thread is doing its job. I'd be fine doing this next step in a follow up PR if that's too involving for this one.

@fabpot fabpot modified the milestones: 7.3, 7.4 May 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants