-
-
Notifications
You must be signed in to change notification settings - Fork 9.6k
[HttpKernel] Use flock() for HttpCache's lock files #19300
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Appveyor tests fail because HttpCacheTestCase tries to clear directories and lacks permissions to unlink LockHandler's lockfiles. |
@nicolas-grekas you seem to be the master of tests here. How would you approach this? |
On Windows, a file can't be unlinked until all handlers to it have been closed. |
Not sure this is the right approach: a real lock file will create another race condition unless the lock file is not unlinked (the race being when a lock is acquired but another process unlinks the just locked file). |
@nicolas-grekas Not sure I got your above comment right 😦 My intention was to use In fact Regarding the Windows tests, for some reason Do you have any idea how I could debug this? I don't have a Windows machine with PHP set up anywhere near... :-( |
We need to be sure of that. It will leave standalone files forever, and their number can grow large. Does it really not matter? If we assume yes (which need confirmation), then maybe use flock directly here? The LockHandler only adds boilerplate to me in this case. |
I am using This PR places the lock files in the same directory as the cache entry in question. So I'd say yes, this can double the amount of files present in the
Now I get that: You cannot safely
My assumption was that all it does was to handle the edge-cases and platform inconsistencies of using file locking, tested and packed up behind a simple API. Don't you think I'd have to reinvent all this if I refrain from using |
Then what about locking the cache file itself instead of creating a dedicated lock file? Using flock directly also should be better, there are no special inconsistencies to deal with in LockHandler |
Note to self: flock() also supports shared "reader" locks, might be an additional advantage here. |
Looks like a matter of correctly implementing it to me and not fall into the dangerous traps, isn't it? |
Locking was only applied when an existing cache entry was updated, so no need for shared (read) locks. |
See proposal at nicolas-grekas#5 |
👍 :) |
👍 |
Thank you @mpdude. |
This PR was merged into the 2.7 branch. Discussion ---------- [HttpKernel] Use flock() for HttpCache's lock files | Q | A | ------------- | --- | Branch? | 2.7 | Bug fix? | yes | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | no | Fixed tickets | #16777, #15813 and #16312 are also related | License | MIT | Doc PR | When a PHP process crashes or terminates (maybe the OOM killer kicks in or other bad things ™️ happen) while the `HttpCache` holds a `.lck` file, that lock file may not get `unlink()`ed. The result is that other requests trying to access this cache entry will see a few seconds delay while waiting for the lock; they will eventually continue but send 503 status codes along with the response. The sudden buildup of PHP processes caused by the additional delay may cause further problems (sudden load increase). As `LockHandler` is using `flock()`-based locking, locks should be released by the OS when the PHP process terminates. I wrote this as bugfix against 2.7 because every once in a while I encounter situations (not always reproducible) where `.lock` files are left over and keep the cache locked. Commits ------- 2668edd [HttpKernel] Use flock() for HttpCache's lock files
When a PHP process crashes or terminates (maybe the OOM killer kicks in or other bad things ™️ happen) while the
HttpCache
holds a.lck
file, that lock file may not getunlink()
ed.The result is that other requests trying to access this cache entry will see a few seconds delay while waiting for the lock; they will eventually continue but send 503 status codes along with the response. The sudden buildup of PHP processes caused by the additional delay may cause further problems (sudden load increase).
As
LockHandler
is usingflock()
-based locking, locks should be released by the OS when the PHP process terminates.I wrote this as bugfix against 2.7 because every once in a while I encounter situations (not always reproducible) where
.lock
files are left over and keep the cache locked.