Skip to content

[Cache] Fix invalidating tags on Redis <5 #43158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 26, 2021

Conversation

wouterj
Copy link
Member

@wouterj wouterj commented Sep 24, 2021

Q A
Branch? 5.3
Bug fix? yes
New feature? no
Deprecations? no
Tickets Fix #42126
License MIT
Doc PR -

https://redis.io/commands/eval#replicating-commands-instead-of-scripts

Note: starting with Redis 5, the replication method described in this section (scripts effects replication) is the default and does not need to be explicitly enabled.

Starting with Redis 3.2, it is possible to select an alternative replication method. Instead of replicating whole scripts, we can just replicate single write commands generated by the script. We call this script effects replication.

To enable script effects replication you need to issue the following Lua command before the script performs a write:

redis.replicate_commands()

@nicolas-grekas
Copy link
Member

<5 instead of <3? What about doDeleteYieldTags()?

@wouterj wouterj changed the title [Cache] Fix invalidating tags on Redis <3 [Cache] Fix invalidating tags on Redis <5 Sep 24, 2021
@wouterj wouterj force-pushed the issue-42126/redis3-replication branch from 7539aea to 64784d2 Compare September 24, 2021 08:51
@wouterj
Copy link
Member Author

wouterj commented Sep 24, 2021

What about doDeleteYieldTags()?

This new replication method is only required when the commands are non deterministic. It seems like the doDeleteYieldTags() script is deterministic. For completeness, I've started a redis:3 container locally and ran the RedisTagAwareAdapterTest integration tests with this container. They all passed with the changes in this PR.

I've also updated the PR to handle any exception better. Currently, users get a PHP Error: "Cannot use object of type RedisException as array". After this PR, they'll get the actual redis exception.

@wouterj wouterj force-pushed the issue-42126/redis3-replication branch from 64784d2 to 61e00cd Compare September 24, 2021 08:54
@wouterj wouterj changed the base branch from 5.3 to 4.4 September 24, 2021 08:55
@wouterj wouterj modified the milestones: 5.3, 4.4 Sep 24, 2021
@wouterj wouterj force-pushed the issue-42126/redis3-replication branch from 61e00cd to dbe0cf7 Compare September 24, 2021 09:01
@nicolas-grekas
Copy link
Member

For my understanding, when did you encounter RedisException being returned? Is that a dev-time only issue or can this happen in regular operations? I'm asking because the cache adapters are not supposed to fail when there is an issue.

@wouterj
Copy link
Member Author

wouterj commented Sep 24, 2021

We had this occur when invalidating caches in production.

I'm not sure what would be the best way forward. If there is an error, nothing is invalidated. So simply logging the exception and retuning would probably not be correct either.

@nicolas-grekas
Copy link
Member

nicolas-grekas commented Sep 24, 2021

Do you know what the RedisException was about?

@wouterj
Copy link
Member Author

wouterj commented Sep 25, 2021

Do you know what the RedisException was about?

Yeah, it was about the error in the lua script that is fixed in this PR:

ERR Error running script (call to f_cea3ea3b59f1b3411febbbeb5f94de8a695cec73): @user_script:19: @user_script: 19: Write commands not allowed after non deterministic commands. Call redis.replicate_commands() at the start of your script in order to switch to single commands replication mode.

So in theory, you will not get this exception. But I don't know anything about Redis, so I guess maybe there is something else in this script that breaks for e.g. Redis 4 or Redis 6 in the future?

@nicolas-grekas
Copy link
Member

Can we call CacheItem::log when we have the exception somehow?

@jderusse
Copy link
Member

So in theory, you will not get this exception. But I don't know anything about Redis, so I guess maybe there is something else in this script that breaks for e.g. Redis 4 or Redis 6 in the future?

method SSCAN is not deterministic (because SETs are unordered), that's why, in Redis 4, you have to tell Redis to replicate commands.

more info https://redis.io/commands/eval#replicating-commands-instead-of-scripts and https://redis.io/commands/EVAL#scripts-with-deterministic-writes

@wouterj wouterj force-pushed the issue-42126/redis3-replication branch from dbe0cf7 to eb09827 Compare September 25, 2021 11:08
@wouterj
Copy link
Member Author

wouterj commented Sep 25, 2021

Can we call CacheItem::log when we have the exception somehow?

Oh, didn't know about this helper method. I've updated the PR with your suggestions: any exception is now logged and not thrown, and doInvalidateTags() nows returns false when there was an exception.

Copy link
Member

@nicolas-grekas nicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(once psalm failure is fixed)

@wouterj wouterj force-pushed the issue-42126/redis3-replication branch from eb09827 to fda7e7e Compare September 25, 2021 11:31
@fabpot
Copy link
Member

fabpot commented Sep 26, 2021

Thank you @wouterj.

@fabpot fabpot merged commit 5c5fd0c into symfony:4.4 Sep 26, 2021
@wouterj wouterj deleted the issue-42126/redis3-replication branch September 26, 2021 16:37
This was referenced Sep 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Cache] invalidating tags breaks after updating to symfony/cache 5.3.3 with Redis versions below 5.x
6 participants