Skip to content

Minor cleanups in HttpCache, especially centralize Date header fixup #22033

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from
Closed

Minor cleanups in HttpCache, especially centralize Date header fixup #22033

wants to merge 6 commits into from

Conversation

mpdude
Copy link
Contributor

@mpdude mpdude commented Mar 17, 2017

Q A
Branch? 2.8
Bug fix? yes
New feature? no
BC breaks? no
Deprecations? no
Tests pass? yes
Fixed tickets
License MIT
Doc PR

I am trying to hunt some issues where the HTTP cache does not revalidate responses (or does not use a cached response?) in the edge case of having s-maxage = 0 and possibly a backend that's slow to respond.

I don't see yet what exactly my problem is there, but it must somehow be related with the Date header handling and/or age calculations in the HTTPCache.

As a first step, I'd like to suggest this cleanup in HTTPCache. Especially, it would be helpful if we could make sure that there is just one place where we set the Date header if the backend does not send one (?). This makes sure we always have this header to base later age calculations on it.

mpdude added 6 commits March 17, 2017 11:30
This makes use of the fact that all requests to the backend go through this method.

Backends should provide a Date header indicating the time at which the response
was generated. But if (for whatever reason) they don't, we'll keep the time
at which the response was received as the response Date.

To-Do: Review the RFC; it probably suggests using the time at which the request
was initiated, not the time the response was received.
@Nek-
Copy link
Contributor

Nek- commented Mar 17, 2017

Thanks for your contribution. 😄

Is it possible to add a test case that exposes this issue?

@mpdude
Copy link
Contributor Author

mpdude commented Mar 17, 2017

I will add that once I have found/understood it.

For now, this PR is intended as a refactoring that helps (at least me) to better follow execution paths through the HttpCache.

@mpdude
Copy link
Contributor Author

mpdude commented Mar 17, 2017

#19390 added a check in the HttpCache::store() method to make sure we have a Date header. As all requests go through forward(), to me this seems to be the better place.

@mpdude
Copy link
Contributor Author

mpdude commented Mar 17, 2017

@Nek- the test in #22043 comes close

@fabpot
Copy link
Member

fabpot commented Mar 22, 2017

merging into master as this is not really a bug fix

@fabpot
Copy link
Member

fabpot commented Mar 22, 2017

Thank you @mpdude.

@fabpot fabpot closed this Mar 22, 2017
fabpot added a commit that referenced this pull request Mar 22, 2017
…header fixup (mpdude)

This PR was submitted for the 2.8 branch but it was merged into the 3.3-dev branch instead (closes #22033).

Discussion
----------

Minor cleanups in HttpCache, especially centralize Date header fixup

| Q             | A
| ------------- | ---
| Branch?       | 2.8
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets |
| License       | MIT
| Doc PR        |

I am trying to hunt some issues where the HTTP cache does not revalidate responses (or does not use a cached response?) in the edge case of having `s-maxage = 0` and possibly a backend that's slow to respond.

I don't see yet what exactly my problem is there, but it must somehow be related with the `Date` header handling and/or age calculations in the `HTTPCache`.

As a first step, I'd like to suggest this cleanup in `HTTPCache`. Especially, it would be helpful if we could make sure that there is just one place where we set the `Date` header if the backend does not send one (?). This makes sure we always have this header to base later age calculations on it.

Commits
-------

30639c6 Minor cleanups in HttpCache, especially centralize Date header fixup
fabpot added a commit that referenced this pull request Mar 22, 2017
… (first?) test for it (mpdude)

This PR was squashed before being merged into the 3.3-dev branch (closes #22043).

Discussion
----------

Refactor stale-while-revalidate code in HttpCache, add a (first?) test for it

| Q             | A
| ------------- | ---
| Branch?       | 2.8
| Bug fix?      | no
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets |
| License       | MIT
| Doc PR        |

I came up with this while trying to hunt a production bug related to handling of stale cache entries under the condition of a busy backend (also see #22033).

It's just a refactoring to make the code more readable plus a new test.

Commits
-------

b14057c Refactor stale-while-revalidate code in HttpCache, add a (first?) test for it
@mpdude mpdude deleted the http-cache-cleanup-date branch March 23, 2017 05:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants