Skip to content

[Cache] Purging takes forever with new FilesystemTagAwareAdapter #32701

Closed
@Toflar

Description

@Toflar

Symfony version(s) affected: 4.3.*

Description

With the new optimized FilesystemTagAwareAdapter tags are now stored as symlinks.
See #30370.

Now when you have a considerable number of cache entries or especially tags, $cache->prune() will take forever because there's no special prune() implementation in that new adapter. It just uses the prune() of the FilesystemTrait which loops over all the files and is doing an fopen() (https://github.com/symfony/symfony/blob/4.4/src/Symfony/Component/Cache/Traits/FilesystemTrait.php#L31).

So imagine you have 10 000 cache entries and 100 000 tags, that's going to take quite a while.
I'm not even sure if tags can expire? So maybe we should implement prune() in such a way that it excludes the tags folder?

How to reproduce

  1. mkdir test && cd test
  2. composer require symfony/cache symfony/filesystem
  3. mkdir cache
  4. Add the following test.php and run it using php test.php:
<?php

use Symfony\Component\Cache\Adapter\FilesystemTagAwareAdapter;
use Symfony\Contracts\Cache\ItemInterface;

require_once 'vendor/autoload.php';

$cache = new FilesystemTagAwareAdapter('', 0, __DIR__ . '/cache');

for ($i = 0;$i < 10000; $i++) {
    $value = $cache->get(bin2hex(random_bytes(4)), function (ItemInterface $item) {
        $item->expiresAfter(3600);

        for ($j = 0;$j < 10; $j++) {
            $item->tag(bin2hex(random_bytes(4)));
        }

        return 'foobar';
    });
}

echo 'Generated entries and tags. Now pruning...' . "\n";

$cache->prune(); // now you should be feeling how long it takes

Possible Solution

Maybe just exclude the tags folder from pruning. But if we can, we should also clean up tags that do not point to any valid cache entry anymore so we don't have any orphan tag symlinks.

Gonna cc @andrerom here because he implemented the new optimized adapter which is otherwise working perfectly so thanks again! 😄

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions