-
-
Notifications
You must be signed in to change notification settings - Fork 9.6k
[Cache] Improve RedisTagAwareAdapter invalidation logic & requirements #33461
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Cache] Improve RedisTagAwareAdapter invalidation logic & requirements #33461
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's cool!
A hypothetical future step could be to schedule the cleanups using Messenger :)
Some setbacks here, clients are not so complete:
These can potentially be fixed by instead using SMEMBERS + skipping SSCAN/SMEMBERS on tagId if RENAME returned 0 (not found in this case). More of a Server issue:
This one I'm unsure of, should we try to make sure we RENAME within same node? Suggestions @nicolas-grekas / @berezuev? |
Can't we rename to a key that will be guaranteed on the same node, using the special syntax they support for this? |
Do you happen to remember where that is described? Thought about that too, but didn't know where to look. |
Sure, see https://redis.io/topics/cluster-spec, the section about "hash tags" |
is it possible to have |
This comment has been minimized.
This comment has been minimized.
The alternative could be to talk directly to the corresponding node in the cluster. That's possible AFAIK. |
This comment has been minimized.
This comment has been minimized.
I think there is a function for that :) |
e264119
to
e2c430b
Compare
RedisCluster failure was fixed in be04079. I did not find a simple way to deal with AppVeyor failure seems unrelated (HttpClient\Tests\NativeHttpClientTest::testNotATimeout), so rebased and moving this to review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please rebase, you've been caught by fabbot :)
cbc72b7
to
f9e47ba
Compare
last commit 2017... |
Another reason to throw and encourage using phpredis... |
Mergeable? |
IMHO Yes, we can do the improvements as a follow up, it will anyway need some investigation and testing. But it's entirely up to you two. |
I'd prefer doing it in this PR, depending on your availability to do it :) |
db80b9d
to
b41526e
Compare
Hello, I just push-forced this here: Should fix the last comment. |
e28a53e
to
28948ee
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 Looks good to me 👍
Further improvements here depends on future improvements in Predis if anyone is up for the tasks:
- Exception on rename on missing key (should align with phpredis: false response when missing)
- Lack of api to find slot of key when using RedisCluster class (should also be added to ClusterInterface, based on implementation in PredisCluster class)
28948ee
to
093a3b4
Compare
Actually, I just discovered this is already possible! PR updated to use pipelining.
that's a bit more involving, as that'd require implementing the logic to react to |
67d569d
to
5ece961
Compare
5ece961
to
3d38c58
Compare
Thank you @andrerom. |
…c & requirements (andrerom) This PR was merged into the 4.4 branch. Discussion ---------- [Cache] Improve RedisTagAwareAdapter invalidation logic & requirements | Q | A | ------------- | --- | Branch? | 4.4 | Bug fix? | yes, _and improvment_ | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | License | MIT | Doc PR | Changes logic of invalidation in RedisTagAwareAdapter in order to: - Delete the tag key on invalidation => _avoiding possible left behind empty tag keys that Redis is not allowed to evict, gradually consuming more and more memory_ Positive side effects of no longer using sPOP: - Lowered requirements to Redis 2.8, and no specific version constraint for phpredis - Lift limitation of 2 billion keys per tag _(Now only limited by Redis Set datatype: 4 billion)_ Commits ------- 3d38c58 [Cache] Improve RedisTagAwareAdapter invalidation logic & requirements
Changes logic of invalidation in RedisTagAwareAdapter in order to:
Positive side effects of no longer using sPOP: