-
-
Notifications
You must be signed in to change notification settings - Fork 9.6k
[Cache] Simple Memcached Adapter #20858
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Cache] Simple Memcached Adapter #20858
Conversation
c5ed119
to
88b9ab9
Compare
protected $skippedTests = array( | ||
//'testExpiration' => 'Testing expiration slows down the test suite', | ||
//'testHasItemReturnsFalseWhenDeferredItemIsExpired' => 'Testing expiration slows down the test suite', | ||
//'testDefaultLifeTime' => 'Testing expiration slows down the test suite', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These commented-out lines should be removed prior to merge.
e0d09f5
to
740bce0
Compare
|
||
public static function setupBeforeClass() | ||
{ | ||
if (!extension_loaded('memcached') || !version_compare(phpversion('memcached'), '2.1.0', '>')) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only 2.2.x and 3.x are supported. Version 2.1.x causes symbol error in extension when requesting last result code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See https://travis-ci.org/symfony/symfony/jobs/182877309#L2410 for example of the above described issue. Only applicable to memcached
version 2.1.0
in Travis PHP 5.5
environment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To the unaware reader (like me a few minutes ago), this reads as if 2.1.1 is fine. But there is no 2.1.x, x>0
Could the check be ">=2.2.0"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw, this check could be moved to a public static function isSupported()
on the adapter (see ApcuAdapter for similar concern)
|
||
private function getIdsByPrefix($namespace) | ||
{ | ||
if (false === $ids = $this->client->getAllKeys()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was wondering: how does this method play with Memcached::OPT_PREFIX_KEY
? Does it ignore it (ie return all keys, even those that don't start with the prefix) or does it filter by prefix?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From the docs:
Memcached::getAllKeys()
queries each memcache server and retrieves an array of all keys stored on them at that point in time [...]
So it sounds like it ignores that setting, but I haven't used Memcached::OPT_PREFIX_KEY
myself, so I don't think we'd find a conclusive answer without delving into the C code for the extension.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI: Remember, I don't have a combination of libmemcached/ext-memcached that actually supports this method so testing this behavior is...difficult. I'll see if I can't compile a working combination using phpenv later tonight.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should drop the use of getAllKeys and reduce doClear to return $this->client->flush();
: you can't make it work, and it comes with many notices saying this can kill kittens.
*/ | ||
protected function doFetch(array $ids) | ||
{ | ||
foreach ($this->client->getMulti($ids) as $id => $val) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can't we just return $this->client->getMulti($ids);
?
*/ | ||
protected function doHave($id) | ||
{ | ||
return $this->client->get($id) !== false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since false
is a valid value, this can't work, isn't it?
I guess this should be
$this->client->get($id);
return $this->client->getResultCode() === \Memcached::RES_SUCCESS;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is an error on my part: intended for that to be an OR (||). The idea is to avoid the additional call to $this->client->getResultCode()
if the $this->client->get($id)
isn't false
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is what I meant to have there (and what I'd recommend we use):
protected function doHave($id)
{
return $this->client->get($id) !== false
|| $this->client->getResultCode() !== \Memcached::RES_NOTFOUND;
}
Kind of a micro-optimization, but worth avoiding an additional function call if unnecessary, IMHO.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated per above comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Checking against === RES_SUCCESS looks required to me, !== opens for all the other codes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, that change does make sense. I missed the change from !== RES_NOTFOUND
to === RES_SUCCESS
in your comment. Updated in latest push.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're almost good for the first step :)
use Symfony\Component\Cache\Adapter\MemcachedAdapter; | ||
|
||
/** | ||
* @author Rob Frawley 2nd <rmf@src.run> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we usualy don't add author on tests (even if test some have, but they could be removed)
|
||
public static function setupBeforeClass() | ||
{ | ||
if (!extension_loaded('memcached') || !version_compare(phpversion('memcached'), '2.1.0', '>')) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw, this check could be moved to a public static function isSupported()
on the adapter (see ApcuAdapter for similar concern)
} | ||
|
||
/** | ||
* @group memcachedAdapter |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we don't use groups like this
/** | ||
* @group memcachedAdapter | ||
*/ | ||
public function testAdapterAndClientCreationAndServers() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not sure this test is useful: it tests only test methods/behavior, I'd remove it
740bce0
to
1143944
Compare
@nicolas-grekas New pushed code with changes per your comments (and in some cases, my responses). Also, un-commented test skips at this point as prior tests have shown Otherwise, a bit of cleanup in the test class as well. Should be good to go @nicolas-grekas! Let me know if I can do anything else to get this merge ready. |
3b2676d
to
778d5e6
Compare
@nicolas-grekas I am unable to find any combination of libmemcached and ext-memcached that result in a non-false return for How do you feel about replacing this class's private function getIdsByPrefix($namespace)
{
if (false === ($ids = $this->client->getAllKeys())
&& false === ($ids = $this->getAllKeysMemcache())) {
return false;
}
return array_filter((array) $ids, function ($id) use ($namespace) {
return 0 === strpos($id, $namespace);
});
}
private function getAllKeysMemcache()
{
static $client = null;
if (false === $client
|| (null === $client
&& false === $client = $this->createMemcacheClient())) {
return false;
}
$ids = array();
foreach ($client->getExtendedStats('slabs') as $group) {
foreach ($group as $slabId => $metadata) {
if (!is_array($metadata)) {
continue;
}
foreach ($client->getExtendedStats('cachedump', (int)$slabId, 1000) as $slabIds) {
if (is_array($slabIds)) {
$ids = array_merge($ids, array_keys($slabIds));
}
}
}
}
return $ids;
}
private function createMemcacheClient()
{
if (!extension_loaded('memcache')) {
return false;
}
$client = new \Memcache();
foreach ($this->client->getServerList() as $srv) {
$client->addServer(array_shift($srv), array_shift($srv));
}
return $client;
} This ensures the check for the Note: Mixing these extensions in terms of their values is dangerous, due to serialization differences, but in terms of simply returning keys is harmless. Due to the fact that Basic logic required for this implementation: When
|
I think this is going too far, that we should give up and just call flush... :) |
$this->client = $client; | ||
} | ||
|
||
public static function isSupported() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should be moved as first method in the class (same as e.g. ApcuAdapter)
778d5e6
to
64f3cce
Compare
Ha; fair enough. This should be good to go in that case. Updated per most recent comments. |
return $isCleared; | ||
} | ||
|
||
private function getIdsByPrefix($namespace) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ping @robfrawley: :)
protected function doClear($namespace)
{
return $this->client->flush();
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a commit with https://github.com/symfony/symfony/pull/20858/files#diff-e01499fab5ad7f5e36517d157bc13e2fR96 to check if any Travis environment actually has a functional combination of the extension and library. After checking https://travis-ci.org/symfony/symfony/builds/183319461, if none actually utilize this, then I'm with you 100%.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nope. None call the extra logic: removed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removing that also raises coverage to 100%, so it's a solid move in my book.
0673846
to
94e8e88
Compare
@nicolas-grekas Should I make the same |
{ | ||
$toDelete = count($ids); | ||
foreach ((array) $this->client->deleteMulti($ids) as $result) { | ||
if (true === $result || \Memcached::RES_NOTFOUND === $result) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so, only one remaining comment here: in which case can $result
be true
? when the cast is in effect and only then (look at the doc, I'd say yes?)
Shouldn't the implem be this one?
if (true === $results = $this->client->deleteMulti($ids)) {
return true;
}
$toDelete = count($ids);
foreach ($results as $result) {
if (\Memcached::RES_SUCCESS === $result || \Memcached::RES_NOTFOUND === $result) {
--$toDelete;
}
}
return 0 === $toDelete;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Return of $this->client->deleteMulti($ids)
is actually always bool[]
now that we have isSupported() require >=2.2.0
.
So the cast can be removed now, to simply have
$toDelete = count($ids);
foreach ($this->client->deleteMulti($ids) as $result) {
if (true === $result || \Memcached::RES_NOTFOUND === $result) {
--$toDelete;
}
}
return 0 === $toDelete;
That was to support <2.2.0
extension versions that would return a single bool
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Scratch that, you're right. The check can now simply be:
$toDelete = count($ids);
foreach ($this->client->deleteMulti($ids) as $result) {
if (\Memcached::RES_SUCCESS === $result || \Memcached::RES_NOTFOUND === $result) {
--$toDelete;
}
}
return 0 === $toDelete;
The situation where it returns bool
is behind us with the version requirement in isSupported()
.
94e8e88
to
12de2ae
Compare
Let's go :) |
Thank you @robfrawley. |
This PR was merged into the 3.3-dev branch. Discussion ---------- [Cache] Simple Memcached Adapter | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | yes | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | | License | MIT | Doc PR | | Related PRs | #20863, ~~#20752~~ Commits ------- 12de2ae memcached cache pool adapter
…ctoring (robfrawley, javiereguiluz) This PR was merged into the master branch. Discussion ---------- [Cache] Memcached Adapter Docs and Cache Pool Docs Refactoring Documentation for symfony/symfony#20863 and symfony/symfony#20858. PR symfony/symfony#20863 is still being written/finalized, so this is a WIP following that PR at this time. Resolves #7256. Commits ------- 7db929c corrected spelling/grammar c49d427 cleanup all cache adapter documentation, extend redis docs e3eff5b Reviewed the entire article d4292b7 refactor cache pool documentation and add memcached adapter docs
#20752