Skip to content

[HttpClient] Make CachingHttpClient compatible with RFC 9111 #59576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 53 commits into
base: 7.4
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
867ee2a
[HttpClient] Add an RFC 9111 compliant client
Lctrs Jan 21, 2025
d8b4992
rename and doc
Lctrs Jan 22, 2025
492b4ab
remove configurable status codes and methods + doc constructor
Lctrs Jan 22, 2025
4dc0157
phpdoc
Lctrs Jan 22, 2025
5462d60
into cachinghttpclient with bc layer
Lctrs Jan 22, 2025
ea6004f
cs
Lctrs Jan 22, 2025
6a2ee73
encode vary field value to avoid collision
Lctrs Jan 22, 2025
dc8069f
require-dev symfony/cache
Lctrs Jan 22, 2025
c06169d
fixing legacy tests
Lctrs Jan 22, 2025
5339c8f
fix legacy tests 2
Lctrs Feb 18, 2025
033f9d8
throw exception when chunk cache item not found
Lctrs Feb 18, 2025
ffedf61
fix check if response is cacheable
Lctrs Feb 18, 2025
c5b0404
more fixes and tests
Lctrs Feb 18, 2025
71f2e3f
cs fixes
Lctrs Feb 18, 2025
54e9fb2
replace ttl by maxTtl (should be clearer) + add missing docs + fix tests
Lctrs Feb 18, 2025
4079e2e
fix lowest tests
Lctrs Feb 18, 2025
22d5691
add changelogs
Lctrs Feb 18, 2025
fb273c9
own cache pool
Lctrs Feb 19, 2025
061118e
added -> add
Lctrs Feb 19, 2025
9c7fe49
$store -> $cache
Lctrs Feb 19, 2025
59cbd40
private legacy
Lctrs Feb 19, 2025
288a2a2
cleanup phpdocs
Lctrs Feb 19, 2025
04439cd
freshness enum
Lctrs Feb 19, 2025
aeddd21
fix stream issues
Lctrs Feb 19, 2025
ea96bf2
more phpdoc fix
Lctrs Feb 19, 2025
a2969ba
cs fix
Lctrs Feb 19, 2025
a52a36b
fix cache definition
Lctrs Feb 20, 2025
47d5422
put caching client between retry and throttling and invalidate cache …
Lctrs Feb 20, 2025
03b782a
also clock mock symfony cache namespace
Lctrs Feb 20, 2025
c949f8a
bcb: also return async response
Lctrs Feb 20, 2025
7f888ac
fix stream and add tests
Lctrs Feb 20, 2025
e6b3756
extend TransportException
Lctrs Feb 21, 2025
e5260d3
tests: in memory cache adapter
Lctrs Feb 21, 2025
454339a
order UPGRADE-7.3.md
Lctrs Feb 28, 2025
df72186
ensure positive integer for max_ttl option
Lctrs Feb 28, 2025
1ab157d
reword CHANGELOG.md
Lctrs Feb 28, 2025
6d4f0d6
remove empty() usage
Lctrs Feb 28, 2025
1b0a30a
evaluateCachedFreshness -> evaluateCacheFreshness
Lctrs Feb 28, 2025
73311bc
remove dev deps on symfony/filesystem
Lctrs Feb 28, 2025
6b1dfda
add more tests
Lctrs Mar 12, 2025
463b71b
vary asterisk prevents caching
Lctrs Mar 12, 2025
0ebc780
exclude non cacheable headers from cache
Lctrs Mar 12, 2025
d1886eb
remove wrongly cacheable status codes
Lctrs Mar 12, 2025
e15142c
implement heuristic caching
Lctrs Mar 12, 2025
bb9b0ba
Update src/Symfony/Component/HttpClient/CachingHttpClient.php
Lctrs Mar 31, 2025
342fedc
Update src/Symfony/Component/HttpClient/CachingHttpClient.php
Lctrs Mar 31, 2025
e16d16f
Update src/Symfony/Component/HttpClient/CachingHttpClient.php
Lctrs Mar 31, 2025
bf285c0
Update src/Symfony/Component/HttpClient/CachingHttpClient.php
Lctrs Mar 31, 2025
8c9871d
switch to a sha256 based hash
Lctrs Mar 31, 2025
54b9fde
turn some vars by ref to static ones
Lctrs Mar 31, 2025
4c1a097
cs
Lctrs Mar 31, 2025
df3f803
more static methods
Lctrs Mar 31, 2025
7c66ffd
switch to TagAwareCacheInterface
Lctrs Apr 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
switch to TagAwareCacheInterface
  • Loading branch information
Lctrs committed Apr 16, 2025
commit 7c66ffd82f2fc9dedd07fd4ebe629ed6a34bbec3
85 changes: 41 additions & 44 deletions src/Symfony/Component/HttpClient/CachingHttpClient.php
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@

namespace Symfony\Component\HttpClient;

use Symfony\Component\Cache\Adapter\TagAwareAdapterInterface;
use Symfony\Component\HttpClient\Caching\Freshness;
use Symfony\Component\HttpClient\Chunk\ErrorChunk;
use Symfony\Component\HttpClient\Exception\ChunkCacheItemNotFoundException;
Expand All @@ -23,6 +22,8 @@
use Symfony\Component\HttpKernel\HttpCache\HttpCache;
use Symfony\Component\HttpKernel\HttpCache\StoreInterface;
use Symfony\Component\HttpKernel\HttpClientKernel;
use Symfony\Contracts\Cache\ItemInterface;
use Symfony\Contracts\Cache\TagAwareCacheInterface;
use Symfony\Contracts\HttpClient\ChunkInterface;
use Symfony\Contracts\HttpClient\HttpClientInterface;
use Symfony\Contracts\HttpClient\ResponseInterface;
Expand Down Expand Up @@ -81,7 +82,7 @@
*/
private const MAX_HEURISTIC_FRESHNESS_TTL = 86400;

private TagAwareAdapterInterface|HttpCache $cache;
private TagAwareCacheInterface|HttpCache $cache;
private array $defaultOptions = self::OPTIONS_DEFAULTS;

/**
Expand All @@ -94,13 +95,13 @@
*/
public function __construct(
private HttpClientInterface $client,
TagAwareAdapterInterface|StoreInterface $cache,
TagAwareCacheInterface|StoreInterface $cache,
array $defaultOptions = [],
private readonly bool $sharedCache = true,
private readonly ?int $maxTtl = null,
) {
if ($cache instanceof StoreInterface) {
trigger_deprecation('symfony/http-client', '7.3', 'Passing a "%s" as constructor\'s 2nd argument of "%s" is deprecated, "%s" expected.', StoreInterface::class, __CLASS__, TagAwareAdapterInterface::class);
trigger_deprecation('symfony/http-client', '7.3', 'Passing a "%s" as constructor\'s 2nd argument of "%s" is deprecated, "%s" expected.', StoreInterface::class, __CLASS__, TagAwareCacheInterface::class);

if (!class_exists(HttpClientKernel::class)) {
throw new \LogicException(\sprintf('Using "%s" requires the HttpKernel component, try running "composer require symfony/http-kernel".', __CLASS__));
Expand Down Expand Up @@ -149,19 +150,17 @@

$requestHash = self::hash($method.$fullUrl);
$varyKey = "vary_{$requestHash}";
$varyItem = $this->cache->getItem($varyKey);
$varyFields = $varyItem->isHit() ? $varyItem->get() : [];
$varyFields = $this->cache->get($varyKey, static fn (): array => []);

$metadataKey = self::getMetadataKey($requestHash, $options['normalized_headers'], $varyFields);
$metadataItem = $this->cache->getItem($metadataKey);
$cachedData = $metadataItem->isHit() ? $metadataItem->get() : null;
$cachedData = $this->cache->get($metadataKey, static fn (): null => null);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to save the null value in the backed on a miss?
if not, we should use the $save argument passed by ref to the closure:

Suggested change
$cachedData = $this->cache->get($metadataKey, static fn (): null => null);
$cachedData = $this->cache->get($metadataKey, fn ($item, &$save) => $save = false;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(the same concern should be consider everywhere get() is used)


$freshness = null;
if (\is_array($cachedData)) {
$freshness = $this->evaluateCacheFreshness($cachedData);

Check failure on line 160 in src/Symfony/Component/HttpClient/CachingHttpClient.php

View workflow job for this annotation

GitHub Actions / Psalm

NoValue

src/Symfony/Component/HttpClient/CachingHttpClient.php:160:56: NoValue: All possible types for this argument were invalidated - This may be dead code (see https://psalm.dev/179)

if (Freshness::Fresh === $freshness) {
return $this->createResponseFromCache($metadataKey, $cachedData, $method, $url, $options, $fullUrlTag);

Check failure on line 163 in src/Symfony/Component/HttpClient/CachingHttpClient.php

View workflow job for this annotation

GitHub Actions / Psalm

InvalidReturnStatement

src/Symfony/Component/HttpClient/CachingHttpClient.php:163:24: InvalidReturnStatement: The inferred type 'Symfony\Component\HttpClient\Response\MockResponse' does not match the declared return type 'Symfony\Component\HttpClient\Response\AsyncResponse' for Symfony\Component\HttpClient\CachingHttpClient::request (see https://psalm.dev/128)

Check failure on line 163 in src/Symfony/Component/HttpClient/CachingHttpClient.php

View workflow job for this annotation

GitHub Actions / Psalm

NoValue

src/Symfony/Component/HttpClient/CachingHttpClient.php:163:69: NoValue: All possible types for this argument were invalidated - This may be dead code (see https://psalm.dev/179)
}

if (isset($cachedData['headers']['etag'])) {
Expand All @@ -185,9 +184,8 @@
$expiresAt,
$fullUrlTag,
$requestHash,
$varyItem,
$varyKey,
&$metadataKey,
$metadataItem,
$cachedData,
$freshness,
$url,
Expand All @@ -202,7 +200,7 @@
// avoid throwing exception in ErrorChunk#__destruct()
$chunk instanceof ErrorChunk && $chunk->didThrow(true);
$context->passthru();
$context->replaceResponse($this->createResponseFromCache($metadataKey, $cachedData, $method, $url, $options, $fullUrlTag));

Check failure on line 203 in src/Symfony/Component/HttpClient/CachingHttpClient.php

View workflow job for this annotation

GitHub Actions / Psalm

NullArgument

src/Symfony/Component/HttpClient/CachingHttpClient.php:203:96: NullArgument: Argument 2 of Symfony\Component\HttpClient\CachingHttpClient::createResponseFromCache cannot be null, null value provided to parameter with type array{chunks_count: int, headers: array<string, array<array-key, string>|string>, initial_age: int, status_code: int, stored_at: int} (see https://psalm.dev/057)

return;
}
Expand Down Expand Up @@ -230,16 +228,19 @@
if (304 === $statusCode && null !== $freshness) {
$maxAge = $this->determineMaxAge($headers, $cacheControl);

$cachedData['expires_at'] = self::calculateExpiresAt($maxAge);
$cachedData['stored_at'] = time();
$cachedData['initial_age'] = (int) ($headers['age'][0] ?? 0);
$cachedData['headers'] = array_merge($cachedData['headers'], array_diff_key($headers, self::EXCLUDED_HEADERS));
$this->cache->get($metadataKey, static function (ItemInterface $item) use ($headers, $maxAge, $cachedData, $expiresAt, $fullUrlTag): array {
$item->expiresAt($expiresAt)->tag($fullUrlTag);

Check failure on line 232 in src/Symfony/Component/HttpClient/CachingHttpClient.php

View workflow job for this annotation

GitHub Actions / Psalm

UndefinedInterfaceMethod

src/Symfony/Component/HttpClient/CachingHttpClient.php:232:59: UndefinedInterfaceMethod: Method Psr\Cache\CacheItemInterface::tag does not exist (see https://psalm.dev/181)

$metadataItem->set($cachedData)->expiresAt($expiresAt);
$this->cache->save($metadataItem);
$cachedData['expires_at'] = self::calculateExpiresAt($maxAge);
$cachedData['stored_at'] = time();
$cachedData['initial_age'] = (int) ($headers['age'][0] ?? 0);
$cachedData['headers'] = array_merge($cachedData['headers'], array_diff_key($headers, self::EXCLUDED_HEADERS));

Check failure on line 237 in src/Symfony/Component/HttpClient/CachingHttpClient.php

View workflow job for this annotation

GitHub Actions / Psalm

InvalidArrayOffset

src/Symfony/Component/HttpClient/CachingHttpClient.php:237:66: InvalidArrayOffset: Cannot access value on variable $cachedData using offset value of 'headers', expecting 'expires_at', 'stored_at' or 'initial_age' (see https://psalm.dev/115)

return $cachedData;
}, \INF);

$context->passthru();
$context->replaceResponse($this->createResponseFromCache($metadataKey, $cachedData, $method, $url, $options, $fullUrlTag));

Check failure on line 243 in src/Symfony/Component/HttpClient/CachingHttpClient.php

View workflow job for this annotation

GitHub Actions / Psalm

NullArgument

src/Symfony/Component/HttpClient/CachingHttpClient.php:243:96: NullArgument: Argument 2 of Symfony\Component\HttpClient\CachingHttpClient::createResponseFromCache cannot be null, null value provided to parameter with type array{chunks_count: int, headers: array<string, array<array-key, string>|string>, initial_age: int, status_code: int, stored_at: int} (see https://psalm.dev/057)

return;
}
Expand All @@ -247,7 +248,7 @@
if ($statusCode >= 500 && $statusCode < 600) {
if (Freshness::StaleButUsable === $freshness) {
$context->passthru();
$context->replaceResponse($this->createResponseFromCache($metadataKey, $cachedData, $method, $url, $options, $fullUrlTag));

Check failure on line 251 in src/Symfony/Component/HttpClient/CachingHttpClient.php

View workflow job for this annotation

GitHub Actions / Psalm

NullArgument

src/Symfony/Component/HttpClient/CachingHttpClient.php:251:100: NullArgument: Argument 2 of Symfony\Component\HttpClient\CachingHttpClient::createResponseFromCache cannot be null, null value provided to parameter with type array{chunks_count: int, headers: array<string, array<array-key, string>|string>, initial_age: int, status_code: int, stored_at: int} (see https://psalm.dev/057)

return;
}
Expand Down Expand Up @@ -292,25 +293,26 @@
}

if ($chunk->isLast()) {
$this->cache->saveDeferred($varyItem->set($varyFields)->tag($fullUrlTag)->expiresAt($expiresAt));
$this->cache->get($varyKey, static function (ItemInterface $item) use ($varyFields, $expiresAt, $fullUrlTag): array {
$item->tag($fullUrlTag)->expiresAt($expiresAt);

Check failure on line 297 in src/Symfony/Component/HttpClient/CachingHttpClient.php

View workflow job for this annotation

GitHub Actions / Psalm

UndefinedInterfaceMethod

src/Symfony/Component/HttpClient/CachingHttpClient.php:297:32: UndefinedInterfaceMethod: Method Psr\Cache\CacheItemInterface::tag does not exist (see https://psalm.dev/181)

return $varyFields;
}, \INF);

$maxAge = $this->determineMaxAge($headers, $cacheControl);

$this->cache->saveDeferred(
$this->cache->getItem($metadataKey)
->tag($fullUrlTag)
->set([
'status_code' => $context->getStatusCode(),
'headers' => array_diff_key($headers, self::EXCLUDED_HEADERS),
'initial_age' => (int) ($headers['age'][0] ?? 0),
'stored_at' => time(),
'expires_at' => self::calculateExpiresAt($maxAge),
'chunks_count' => $chunkIndex,
])
->expiresAt($expiresAt)
);

$this->cache->commit();
$this->cache->get($metadataKey, static function (ItemInterface $item) use ($context, $headers, $maxAge, $expiresAt, $fullUrlTag, $chunkIndex): array {
$item->tag($fullUrlTag)->expiresAt($expiresAt);

return [
'status_code' => $context->getStatusCode(),
'headers' => array_diff_key($headers, self::EXCLUDED_HEADERS),
'initial_age' => (int) ($headers['age'][0] ?? 0),
'stored_at' => time(),
'expires_at' => self::calculateExpiresAt($maxAge),
'chunks_count' => $chunkIndex,
];
}, \INF);

yield $chunk;

Expand All @@ -319,12 +321,11 @@

++$chunkIndex;
$chunkKey = "{$metadataKey}_chunk_{$chunkIndex}";
$chunkItem = $this->cache->getItem($chunkKey)
->tag($fullUrlTag)
->set($chunk->getContent())
->expiresAt($expiresAt);
$this->cache->get($chunkKey, static function (ItemInterface $item) use ($expiresAt, $fullUrlTag, $chunk): string {
$item->tag($fullUrlTag)->expiresAt($expiresAt);

$this->cache->save($chunkItem);
return $chunk->getContent();
}, \INF);

yield $chunk;
}
Expand Down Expand Up @@ -684,15 +685,11 @@
new MockResponse(
(function () use ($key, $cachedData, $fullUrlTag): \Generator {
for ($i = 0; $i <= $cachedData['chunks_count']; ++$i) {
$chunkItem = $this->cache->getItem("{$key}_chunk_{$i}");

if (!$chunkItem->isHit()) {
yield $this->cache->get("{$key}_chunk_{$i}", function (ItemInterface $item) use ($fullUrlTag): never {
$this->cache->invalidateTags([$fullUrlTag]);

throw new ChunkCacheItemNotFoundException(\sprintf('Missing cache item for chunk with key "%s". This indicates an internal cache inconsistency.', $chunkItem->getKey()));
}

yield $chunkItem->get();
throw new ChunkCacheItemNotFoundException(\sprintf('Missing cache item for chunk with key "%s". This indicates an internal cache inconsistency.', $item->getKey()));
}, 0);
}
})(),
['http_code' => $cachedData['status_code'], 'response_headers' => ['age' => $cachedData['initial_age'] + (time() - $cachedData['stored_at'])] + $cachedData['headers']]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ class LegacyCachingHttpClientTest extends TestCase

public function testRequestHeaders()
{
$this->expectDeprecation('Since symfony/http-client 7.3: Passing a "Symfony\Component\HttpKernel\HttpCache\StoreInterface" as constructor\'s 2nd argument of "Symfony\Component\HttpClient\CachingHttpClient" is deprecated, "Symfony\Component\Cache\Adapter\TagAwareAdapterInterface" expected.');
$this->expectDeprecation('Since symfony/http-client 7.3: Passing a "Symfony\Component\HttpKernel\HttpCache\StoreInterface" as constructor\'s 2nd argument of "Symfony\Component\HttpClient\CachingHttpClient" is deprecated, "Symfony\Contracts\Cache\TagAwareCacheInterface" expected.');

$options = [
'headers' => [
Expand All @@ -51,7 +51,7 @@ public function testRequestHeaders()

public function testDoesNotEvaluateResponseBody()
{
$this->expectDeprecation('Since symfony/http-client 7.3: Passing a "Symfony\Component\HttpKernel\HttpCache\StoreInterface" as constructor\'s 2nd argument of "Symfony\Component\HttpClient\CachingHttpClient" is deprecated, "Symfony\Component\Cache\Adapter\TagAwareAdapterInterface" expected.');
$this->expectDeprecation('Since symfony/http-client 7.3: Passing a "Symfony\Component\HttpKernel\HttpCache\StoreInterface" as constructor\'s 2nd argument of "Symfony\Component\HttpClient\CachingHttpClient" is deprecated, "Symfony\Contracts\Cache\TagAwareCacheInterface" expected.');

$body = file_get_contents(__DIR__.'/Fixtures/assertion_failure.php');
$response = $this->runRequest(new MockResponse($body, ['response_headers' => ['X-Body-Eval' => true]]));
Expand All @@ -63,7 +63,7 @@ public function testDoesNotEvaluateResponseBody()

public function testDoesNotIncludeFile()
{
$this->expectDeprecation('Since symfony/http-client 7.3: Passing a "Symfony\Component\HttpKernel\HttpCache\StoreInterface" as constructor\'s 2nd argument of "Symfony\Component\HttpClient\CachingHttpClient" is deprecated, "Symfony\Component\Cache\Adapter\TagAwareAdapterInterface" expected.');
$this->expectDeprecation('Since symfony/http-client 7.3: Passing a "Symfony\Component\HttpKernel\HttpCache\StoreInterface" as constructor\'s 2nd argument of "Symfony\Component\HttpClient\CachingHttpClient" is deprecated, "Symfony\Contracts\Cache\TagAwareCacheInterface" expected.');

$file = __DIR__.'/Fixtures/assertion_failure.php';

Expand All @@ -82,7 +82,7 @@ public function testDoesNotIncludeFile()

public function testDoesNotReadFile()
{
$this->expectDeprecation('Since symfony/http-client 7.3: Passing a "Symfony\Component\HttpKernel\HttpCache\StoreInterface" as constructor\'s 2nd argument of "Symfony\Component\HttpClient\CachingHttpClient" is deprecated, "Symfony\Component\Cache\Adapter\TagAwareAdapterInterface" expected.');
$this->expectDeprecation('Since symfony/http-client 7.3: Passing a "Symfony\Component\HttpKernel\HttpCache\StoreInterface" as constructor\'s 2nd argument of "Symfony\Component\HttpClient\CachingHttpClient" is deprecated, "Symfony\Contracts\Cache\TagAwareCacheInterface" expected.');

$file = __DIR__.'/Fixtures/assertion_failure.php';

Expand All @@ -99,7 +99,7 @@ public function testDoesNotReadFile()

public function testRemovesXContentDigest()
{
$this->expectDeprecation('Since symfony/http-client 7.3: Passing a "Symfony\Component\HttpKernel\HttpCache\StoreInterface" as constructor\'s 2nd argument of "Symfony\Component\HttpClient\CachingHttpClient" is deprecated, "Symfony\Component\Cache\Adapter\TagAwareAdapterInterface" expected.');
$this->expectDeprecation('Since symfony/http-client 7.3: Passing a "Symfony\Component\HttpKernel\HttpCache\StoreInterface" as constructor\'s 2nd argument of "Symfony\Component\HttpClient\CachingHttpClient" is deprecated, "Symfony\Contracts\Cache\TagAwareCacheInterface" expected.');

$response = $this->runRequest(new MockResponse(
'test', [
Expand Down
Loading