-
-
Notifications
You must be signed in to change notification settings - Fork 9.6k
[Cache] Advanced Memcached Adapter #20863
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Cache] Advanced Memcached Adapter #20863
Conversation
@nicolas-grekas @stof This PR vastly cuts down on the server/option logic from prior proposals. It builds on #20858 and replaces the original proposal #20752. Removed support for |
8d1ff7c
to
2da89b5
Compare
This has been updated with any changes made against #20858 |
|
||
private function setOption($opt, $val) | ||
{ | ||
$toRestore = error_reporting(~E_ALL); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why changing the error reporting ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To return a false instead of an E_WARNING on invalid option passed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, IMO, the warning would be much easier to debug, as it explains what is wrong. So I'm -1 on this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See https://github.com/symfony/symfony/pull/20863/files#diff-ed071d63655500d46a7c7866cdb74cddR193 for test checking a false return on error instead of an E_WARNING. Is this something you don't think we should do?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nicolas-grekas @stof Is the consensus to allow the E_WARNING and not handle a bool return here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
HHVM simply ignores invalid options silently anyway, so the options are:
- catch error and return bool on PHP
- always return true on HHVM
OR if we don't catch the E_WARNING
- throw E_WARNING on PHP
- again, silent on HHVM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OR, another option; continue catching the E_WARNNG but transform into an exception, which IMHO, may be the most developer friendly option? Does the cache component require the symfony polyfills? So we'd have access to get_last_error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, we could just let the error handler do it, without doing any special handling for E_WARNING happening inside setOption
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed error suppression and amended test to check for E_WARNING in latest push.
2da89b5
to
2f3aeed
Compare
* | ||
* @return bool | ||
*/ | ||
public function setup(array $dsns = array(), array $opts = array()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we should expose such public API
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was a middle-ground approach to allow a simple string DSN for create
as @nicolas-grekas wanted, but also allow for an array of DSNs using a calls
during service construction to this method.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As well, without this method, all the logic from setup(), setOption(), resolveOption(), addServer(), parseServerDSN(), and isServerInClientPool() would need to be shoved into create(). I vastly prefer small, simple methods to a behemoth like RedisAdapter::createConnection(), IMHO.
* @param string[] $dsns | ||
* @param mixed[] $opts | ||
* | ||
* @return bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Returning a boolean does not make sense here:
- handling errors by boolean return values is not user-friendly (no message explaining the issue)
- it is ignored by
create()
anyway, meaning you don't have error handling at all
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Returns $this
now in latest push.
return array_values($srv); | ||
} | ||
|
||
protected function isServerInClientPool($host, $port) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why protected ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oversight from when this was part of a shared method for \Memcached
and \Memcache
implementations. No longer the case, obviously. Fixed in latest push.
protected function isServerInClientPool($host, $port) | ||
{ | ||
return (bool) array_filter($this->client->getServerList(), function ($srv) use ($host, $port) { | ||
return $host === array_shift($srv) && $port === array_shift($srv); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should avoid shifting the array. Mutating an array for reading it is a bad idea: it forces PHP to copy it (not that bad here as it has only a few items, but very bad in general
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Gone
|
||
protected function isServerInClientPool($host, $port) | ||
{ | ||
return (bool) array_filter($this->client->getServerList(), function ($srv) use ($host, $port) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
using array_filter is a bad idea: it forces to compare all servers even if the first one is a match, meaning that you force to always reach the worse case (when there is no match, we are indeed forced to compare them all)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed to foreach in latest push.
public static function setupBeforeClass() | ||
{ | ||
if (!MemcachedAdapter::isSupported()) { | ||
self::markTestSkipped('Extension memcached >2.1.0 required.'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the min version is 2.2.0 in the code, not 2.1.0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
>2.1.0
and >=2.2.0
are the same thing, but I follow the logic of clarifying this. 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not the same. 2.2.0RC1
is > 2.1.0
but is not >= 2.2.0
(the same would be true for 2.1.1
if there was such a release)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Clarified in latest push.
|
||
public static function tearDownAfterClass() | ||
{ | ||
self::$client->flush(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shouldn't it be flushed between tests instead of at the end of the testcase, to ensure test isolation ?
I would even go further and avoid making the client static, ensuring we don't reuse it between tests at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is how the Redis adapter handles it, but yeah; makes sense. Changed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The base test class already calls clear on the pool, that's why it's not needed, isn't it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like you're right! Reverted to:
public static function setupBeforeClass()
{
if (!MemcachedAdapter::isSupported()) {
self::markTestSkipped('Extension memcached >=2.2.0 required.');
}
static::$client = static::getClientFromAdapter(MemcachedAdapter::create(
static::defaultClientServer(),
static::defaultClientOptions()
));
parent::setupBeforeClass();
}
return new MemcachedAdapter(self::$client, str_replace('\\', '.', __CLASS__), $defaultLifetime); | ||
} | ||
|
||
public function testCreateAdaptor() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shouldn't it be adapter
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup ;-) But that test is actually gone now.
3947b38
to
f780643
Compare
Latest commits bring in line with latest changes to #20858 (will squash shortly) |
c46a207
to
762e10d
Compare
Removed all instanced of |
762e10d
to
2c83d2c
Compare
Squashed commits. All changes from #20858 implemented in this PR. |
904aa74
to
b2b131c
Compare
This PR was merged into the 3.3-dev branch. Discussion ---------- [Cache] Simple Memcached Adapter | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | yes | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | | License | MIT | Doc PR | | Related PRs | #20863, ~~#20752~~ Commits ------- 12de2ae memcached cache pool adapter
Rebase needed |
b2b131c
to
43a4559
Compare
Done! |
43a4559
to
25590a9
Compare
Let's go for string or array of DSN. |
e6baecc
to
23fbf08
Compare
Removed the other interface methods, except for |
23fbf08
to
fa4def0
Compare
Yes, please remove AbstractClient and the interface. Please also make sure that the createClient (or createConnection) returns a Memcached instance. |
fa4def0
to
d35b594
Compare
Makes sense (re: \Memcached return). Interface/abstract removed as well! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We only need one single new public method in the whole PR :)
/** | ||
* @author Rob Frawley 2nd <rmf@src.run> | ||
*/ | ||
class MemcachedClient |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This class should be removed: creating a Memcached layer is not the responsibility of the Cache component. But by having this class, people will use it as such. An alternative could be to mark it as internal, but I'm not sure this makes more sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Marked @internal
and final
for now. I really, really, really want to avoid shoving all this logic into a single createConnection
method within the Adapter itself...
*/ | ||
class MemcachedClient | ||
{ | ||
private static $serverDefaults = array( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we don't need server defaults in the cache component (we may have in FrameworkBundle later on, but that's another topic.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These are the distribution defaults for an installed memcached server with no user configuration applied. Additionally, this doesn't actually add a default server if no DSN is passed. At a minimum, the user still needs to pass memcached:
as the DSN. An empty array or empty string will not add a server.
|
||
private static $optionDefaults = array( | ||
'compression' => true, | ||
'libketama_compatible' => true, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you think about enabling binary mode by default?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Almost added it in my first pass actually, but decided not to be presumptuous. Since you feel similarly, I say yes.
)); | ||
} | ||
|
||
private function resolveServerReturn($server) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about username+ password for sasl?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed for both host and socket DSNs:
memcached://user:password@127.0.0.1
memcached://user:password@/var/local/run/memcached.socket
$this->client->flushBuffers(); | ||
} | ||
|
||
parent::__destruct(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't the parent be called before?
public function __destruct() | ||
{ | ||
if (!$this->client->isPersistent() && method_exists($this->client, 'flushBuffers')) { | ||
$this->client->flushBuffers(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not done automatically when the connection is destructed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TBH, this method isn't even documented yet, found it when looking through the extension source to determine available options for the documentation PR. It seems like this is supposed to be called if the new buffer_writes
option is enabled. But I'd have to take a closer look at the source of the extension to confirm if/when it is called by the extension itself. As far as I can tell via a first pass, it is only exposing the function from the library itself and not utilizing it within the extension otherwise.
d35b594
to
315bdb5
Compare
Also, made the |
444f9fc
to
1fc814d
Compare
Documentation PR submitted (assuming this is merged as-is) as symfony/symfony-docs#7265: view the resulting output on platform.sh. |
6e780c3
to
acea4ad
Compare
Why is appveyor ignoring |
acea4ad
to
c55532c
Compare
@nicolas-grekas Do you have any idea why AppVeyor is erroring at seemingly random places in the test suite: on line 259 and on line 999? Also, what more needs to be done prior to merging this? |
Continued in #21108 |
… to MemcachedAdapter (nicolas-grekas, robfrawley) This PR was merged into the 3.3-dev branch. Discussion ---------- [Cache] Add DSN, createClient & better error reporting to MemcachedAdapter | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | yes | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | - | License | MIT | Doc PR | symfony/symfony-docs#7265 Replaces #20863 ping @robfrawley: would you mind opening a doc PR for this? Commits ------- 87030b4 [cache] Add tests for MemcachedAdapter::createClient() e109438 [Cache] Add DSN, createClient & better error reporting to MemcachedAdapter
…ctoring (robfrawley, javiereguiluz) This PR was merged into the master branch. Discussion ---------- [Cache] Memcached Adapter Docs and Cache Pool Docs Refactoring Documentation for symfony/symfony#20863 and symfony/symfony#20858. PR symfony/symfony#20863 is still being written/finalized, so this is a WIP following that PR at this time. Resolves #7256. Commits ------- 7db929c corrected spelling/grammar c49d427 cleanup all cache adapter documentation, extend redis docs e3eff5b Reviewed the entire article d4292b7 refactor cache pool documentation and add memcached adapter docs
#20752Expansion of simple adapter from #20858. Replacement for #20752.