Skip to content

[HttpCache] Add warning that Apache 2.4 mod_deflate changes ETag #12644

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mpdude opened this issue Nov 16, 2019 · 3 comments
Closed

[HttpCache] Add warning that Apache 2.4 mod_deflate changes ETag #12644

mpdude opened this issue Nov 16, 2019 · 3 comments
Labels
Milestone

Comments

@mpdude
Copy link
Contributor

mpdude commented Nov 16, 2019

In Apache 2.4, when mod_deflate compresses a response body, it will also modify the ETag header (if present) and append -gzip to it. This is to comply with HTTP specs and make sure that different representations of the resource also have different ETags.

Here's the relevant parts of results for two curl requests to a simple controller that just sets ETag: some-etag on the response. One is without compression, the other one enables it.

#> curl -I -X GET 'http://my.app/test'
HTTP/1.1 200 OK
Date: Sat, 16 Nov 2019 16:54:59 GMT
Server: Apache
ETag: "some-etag"    <--- HERE
Last-Modified: Fri, 15 Nov 2019 23:00:00 GMT
Vary: Accept-Encoding
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

versus

#> curl -I -X GET 'http://my.app/test' --compressed
HTTP/1.1 200 OK
Date: Sat, 16 Nov 2019 16:56:25 GMT
Server: Apache
ETag: "some-etag-gzip"    <--- HERE
Last-Modified: Fri, 15 Nov 2019 23:00:00 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 10208
Content-Type: text/html; charset=UTF-8

From the client's perspective, "some-etag-gzip" is the opaque ETag value which will be included in revalidation requests.

Since Apache/mod_deflate fails to revert the change upon such requests (and probably cannot reasonably do so, since it cannot make assumptions about the meaning of the ETag), the modified value will be the one reaching PHP and Symfony. This effectively breaks ETag-based validation as outlined in the documentation. The controller code won't be short-cut and a 200 response will be generated instead of 304.

This behavior has been reported as a bug in Apache back in 2008. From that report, it seems mod_brotli faces the same challenge.

A configuration option DeflateAlterETag has been added in Apache 2.5 to either turn off ETag modification (possibly causing other issues?) or to remove ETags altogether from compressed responses. For mod_brotli, a BrotliAlterETag switch is available in Apache 2.4 already.

A possible workaround suggested is to add the following to the web server configuration:

RequestHeader edit "If-None-Match" '^"((.*)-(gzip|br))"$' '"$1", "$2"'

This will turn request headers like If-None-Match: "some-etag-gzip" into If-None-Match: "some-etag-gzip", "some-etag". I can confirm that this passes Symfony's \Symfony\Component\HttpFoundation\Response::isNotModified() as one would expect.

My suggestion is to add a warning/heads-up notice at https://symfony.com/doc/current/http_cache/validation.html#validation-with-the-etag-header. This could include the workaround configuration (if we want to endorse it) and/or point to this issue.

@javiereguiluz
Copy link
Member

@mpdude I'm speechless about your issue report ... because it couldn't be better 😍 . It's just perfect 👏 ... so we took inspiration from it to fix this problem in #12796. Thanks a lot for this amazing contribution!

@mpdude
Copy link
Contributor Author

mpdude commented Dec 10, 2019

Thank you @javiereguiluz and the others involved, you did the actual work

@FurriousFox
Copy link

hmm, but now you got the issue that if you send a request with if-none-match: "0-5debc62fd30dd-br" it'll respond with etag: "0-5debc62fd30dd" (which is missing the -br part)

something like this could solve it though

SetEnvIf If-None-Match '^"((.*)-(gzip))"$' gzip
SetEnvIf If-None-Match '^"((.*)-(br))"$' br

RequestHeader edit "If-None-Match" '^"((.*)-(gzip|br))"$' '"$1", "$2"'

Header edit "ETag" '^"(.*)"$' '"$1-gzip"' env=gzip
Header edit "ETag" '^"(.*)"$' '"$1-br"' env=br

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants