Skip to content

[Cache] Add hierarchical invalidation with ContextAwareAdapterInterface #19521

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

nicolas-grekas
Copy link
Member

@nicolas-grekas nicolas-grekas commented Aug 3, 2016

Q A
Branch? master
Bug fix? no
New feature? yes
BC breaks? no
Deprecations? no
Tests pass? yes
Fixed tickets -
License MIT
Doc PR symfony/symfony-docs#6858

This PR adds a public interface to do hierachical invalidation. By using the new withContext() method, one is able to create a contextualized set of cache keys. Invalidating is as easy as changing the context identifier provided as argument.
Contexts can be chained to create subcontexts inside contexts.
All subcontexts share the same adapter implementation + connection state.
Clearing a parent clears all subcontexts.

This feature is already implemented by e.g. Drupal 8, and it plays nice with tags: all forks share the same tags space.

@nicolas-grekas
Copy link
Member Author

For reference, the intent of this PR is to provide a basic block that could allow building something like Drupal Contexts:
https://www.drupal.org/developing/api/8/cache/contexts

return $this->namespace.$key;
}
if (strlen($id = $this->namespace.$key) > $this->maxIdLength) {
$id = $this->namespace.substr(str_replace('/', '-', base64_encode(md5($key, true))), 0, -2);
Copy link
Member Author

@nicolas-grekas nicolas-grekas Aug 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Namespace prefix excluded, this produces a string of length 22 (thus the 22/23 in the code)
(the last two cut chars are padding =)

@nicolas-grekas nicolas-grekas force-pushed the cache-fork branch 6 times, most recently from 4e02558 to ec4957b Compare August 7, 2016 15:07
nicolas-grekas added a commit that referenced this pull request Aug 7, 2016
…s-grekas)

This PR was merged into the 3.1 branch.

Discussion
----------

[Cache] Use SCAN instead of KEYS with Redis >= 2.8

| Q             | A
| ------------- | ---
| Branch?       | 3.1
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | -
| License       | MIT
| Doc PR        | -

With #19521 coming,  clearing cache keys by prefix is going to be used a lot more often.
Time to fix Redis cache clearing.

Commits
-------

aadeb11 [Cache] Use SCAN instead of KEYS with Redis >= 2.8
@nicolas-grekas nicolas-grekas force-pushed the cache-fork branch 3 times, most recently from 5fa8d1f to 024ea79 Compare August 7, 2016 16:33
@nicolas-grekas
Copy link
Member Author

@andrerom or anyone else, any comment here?

@andrerom
Copy link
Contributor

andrerom commented Sep 2, 2016

So this can for instance be used by HttpCache so it can handle very by logic right?

@nicolas-grekas
Copy link
Member Author

nicolas-grekas commented Sep 2, 2016

Yes for sure. https://www.drupal.org/developing/api/8/cache/contexts is a real use case of this feat. Maybe one should first read this page then help me find the right words to explain the feat. ? :)
This is hierarchical invalidation done from the outside, (the inside being prefixing keys explicitly e.g. ->getItem('foo/bar'))

@andrerom
Copy link
Contributor

andrerom commented Sep 2, 2016

well I looked at that page, and couldn't help but think most of these use cases (incl Http Cache) could be solved by reflecting context in key, as invalidation is done with tags it won't complicate invalidation, and lookup will need to know the context of what they are looking for anyway.

But I'm probably overlooking something.

@nicolas-grekas
Copy link
Member Author

nicolas-grekas commented Sep 2, 2016

could be solved by reflecting context in key

true, and this feat. is exactly here for decoupling this: context computation and keys.
You can create the context outside, then give a regular but contextualized adapter to another service that shouldn't care about the context it's working in.

@nicolas-grekas nicolas-grekas changed the title [Cache] Add namespace based invalidation with ForkableAdapterInterface [Cache] Add hierarchical invalidation with ContextAwareAdapterInterface Sep 14, 2016
@nicolas-grekas
Copy link
Member Author

nicolas-grekas commented Sep 14, 2016

Vocabulary changed: the interface is now called ContextAwareAdapterInterface and its method withContext().

@nicolas-grekas
Copy link
Member Author

Any comment here? Any vote @symfony/deciders?

@nicolas-grekas
Copy link
Member Author

So, thinking a bit more about this: should calling deleteItem on a parent also clear the item recursively? Looks like it could be useful. Then what about the other psr6 methods? Dunno. Which means this isn't ready :)
Closing

@nicolas-grekas nicolas-grekas deleted the cache-fork branch September 23, 2016 18:51
fabpot added a commit that referenced this pull request Sep 24, 2016
…ant using hashing (nicolas-grekas)

This PR was merged into the 3.2-dev branch.

Discussion
----------

[Cache] Handle arbitrary key length when the backend cant using hashing

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | -
| License       | MIT
| Doc PR        | -

Saving some bits from #19521 :) Already awaited by PdoAdapter which defines the property.

Commits
-------

11f448f [Cache] Handle arbitrary key length when the backend cant using hashing
@LionsAd
Copy link

LionsAd commented Oct 5, 2016

The reason why there was confusion is that you need a 2-step cache_get() process - or a cache collector like 'Vary' if you want to use that.

It gets very useful once you have a sub-item use contexts that are not known to the parent.

It basically solves the problem for complex sub-items, which the parent can't query for its contexts and which even might not know its contexts until computation time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants