@asierguti Thank you for the patch! Will you mind adding some corresponding test cases to the existing test suite to demonstrate how it actually works? Thanks!

Well, I have a couple of scripts in python.

The idea is to create a simple HTTP server which sends an HTTP answer with the current timestamp in the body.

The second script will just try to retrieve a particular URL, then wait a number of seconds, and try to grab the same URL once again. It will check if it was due to recache or not, and it will check the timestamp in the body.

I will upload both scripts shortly.

@asierguti Okay, I've had a closer look and got your idea. Basically you want to validate the response from the srcache_fetch subrequest by looking at the cached Expires response header. I agree it is useful when the cache storage backend does not support expiring itself (though using Python here is not really efficient, maybe you should just talk to elliptics with a bit of Lua using ngx_lua module's cosocket API).

Before your patch can be merged, please adjust the coding style to be consistent with the existing code base and preferably with test cases in your patch for the existing test scaffold. (BTW, I'm not interested in your Python scripts and I don't want to include any Python code in the existing test suite that is driven completely by Perl. Please look at the existing test cases for examples).

@asierguti Also, I think it's a good idea to introduce a srcache_fetch_validate_expires on|off directive for this new feature. It will be a waste of CPU cycles for those cache storage backends that do support expiring (like memcached and redis).

And for the automatic "Expires" header insertion part, I must say I don't like it. It is just totally wrong to always blindly override the original "Expires" response header. Though it is what you want for your particular use case but it will break others' setup.

A better approach for Expires response header insertion for your use case is to use ngx_lua module's header_filter_by_lua and the existing $srcache_expires variable exposed by ngx_srcache to insert the "Expires" response header in case of a cache miss yourself, which is much more flexible.

Thanks a lot for your feedback.

First of all, I will try to translate all our tests to perl. It will take
some time, since I've never programmed in perl. Hopefully, I will have it
completed by this weekend.

Regarding your suggestions, I didn't quite get what you said about
srcache_fetch_validate_expires
on|off.How is this feature supposed to work? Having our frontend servers
check the expiration dates of all the data in our backend storage servers
will increase the traffic network. Network is more important than CPU
cycles, at least for us.

Now, regarding your suggestion about LUA. To be honest, I thought about it
before. The thing is that we want to have flexibility to adjust the
expiration time according to the response code we get from our real backend
server (not the cache storage backend). For example, if we get 200, we
would like to cache the data for a week, but if we get 301, we would like
to cache it only for a day. That's why $srcache_expires
was not suitable for us. If you have any suggestion, I will be glad to
consider it.

Cheers.

On 19 March 2014 23:16, Yichun Zhang notifications@github.com wrote:

@asierguti https://github.com/asierguti Also, I think it's a good idea
to introduce a srcache_fetch_validate_expires on|off directive for this
new feature. It will be a waste of CPU cycles for those cache storage
backends that do support expiring (like memcached and redis).

And for the automatic "Expires" header insertion part, I must say I don't
like it. It is just totally wrong to always blindly override the original
"Expires" response header. Though it is what you want for your particular
use case but it will break others' setup.

A better approach for Expires response header insertion for your use case
is to use ngx_lua module's header_filter_by_lua and the existing

$srcache_expires variable exposed by ngx_srcache to insert the "Expires"
response header in case of a cache miss yourself, which is much more
flexible.

Reply to this email directly or view it on GitHubhttps://github.com//pull/26#issuecomment-38094715
.

Hi,

I don't know if you saw my previous comment. In case you missed it, or I did something wrong posting it, here it goes once again.

Thanks a lot for your feedback.

First of all, I will try to translate all our tests to perl. I'm in the process of doing so. I just have one small question. I would like to get in a variable the current time when testing (request part in the test), and compare it to the response. How could I do it? I can get the current time in a variable, but I don't seem to find how I can extract the "Expires" header, and compare it to the previous data.

Regarding your suggestions, I didn't quite get what you said about rcache_fetch_validate_expires on|off.How is this feature supposed to work? Having our frontend servers check the expiration dates of all the data in our backend storage servers will increase the traffic network. Network is more important than CPU cycles, at least for us.

Now, regarding your suggestion about LUA. To be honest, I thought about it before. The thing is that we want to have flexibility to adjust the expiration time according to the response code we get from our real backend server (not the cache storage backend). For example, if we get 200, we would like to cache the data for a week, but if we get 301, we would like to cache it only for a day. That's why $srcache_expires was not suitable for us. If you have any suggestion, I will be glad to consider it.

Looking forward to hearing from you.

Cheers.

@asierguti Sorry, I'm currently busy with the company work. I'll get back to you as soon as I can manage :) Thanks!