Skip to content

Varnish cookbook session cookie handling #4628

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 30, 2015
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
explain how to work with cookies and sessions when caching
  • Loading branch information
dbu committed Jan 18, 2015
commit c88ad327d4f7a0b59cd20f2c222d40026e654826
2 changes: 2 additions & 0 deletions book/http_cache.rst
Original file line number Diff line number Diff line change
Expand Up @@ -383,6 +383,8 @@ This has two very reasonable consequences:
blog post). Caching them would prevent certain requests from hitting and
mutating your application.

.. _http-cache-defaults:

Caching Rules and Defaults
~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
82 changes: 82 additions & 0 deletions cookbook/cache/varnish.rst
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,87 @@ If the ``X-Forwarded-Port`` header is not set correctly, Symfony will append
the port where the PHP application is running when generating absolute URLs,
e.g. ``http://example.com:8080/my/path``.

Session Cookies and Caching
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this section is the only addition compared to #4627

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not dropping the word "Session", it's for all Cookies.

---------------------------

By default, a sane caching proxy does not cache anything when a request is sent
with :ref:`cookies or a basic authentication header<http-cache-introduction>`.
This is because the content of the page is supposed to depend on the cookie
value or authentication header.

If you know for sure that the backend never uses sessions or basic
authentication, have varnish remove the corresponding header from requests to
prevent clients from bypassing the cache. In practice, you will need sessions
at least for some parts of the site, e.g. when using forms with
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may we could go a step in this direction?

even if you drop the Cookie Header using Varnish, on the client side you can access them using JavaScript. For example you could extract the username from a cookie and replace the markup to display a custom welcome message.
In practice some parts of the site aren't cacheable for all users and sometimes it's not possible to solve this issues using JavaScript. For example consider you have a Form with a CSRF Protection. In this situation every user needs to get a unique token for the CSRF Protection. Here you can decide not the cache the whole Page. An alternative would be not to cache an ESI Request that is delivering the part of the Form that is responsible for the CSRF protection or decide to cache based on a user specific cookie. Using the Vary Header you can make sure that Varnish is caching based on another Header. Using Vary in combination with the Cookie Header allows you to fill up the Varnish cache with custom elements for every different cookie value. if the cookie value depends on the user, you are able to cache custom pages in varnish. Often there are a bunch of strategies with different advantages and disadvantages. Choosing the right caching strategy depdends on you use case, your Infrastructure and your Team.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dbu may go could use some parts of https://github.com/symfony/symfony-docs/pull/4628/files#r22298163 ? i just want to make sure that the reader get the idea that there a a bunch of caching strategies. filling up the varnish with user specific data is probably the worst one :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't think this cookbook should talk about general strategies to sort out cacheable from user content. there is the book chapter on caching that explains ESI. the topic of moving small things to javascript cookies is not specific to varnish but to caching in general and should rather go into #4661 or next to that. there is also #4141 which would cover the CSRF part of the topic.

ideally, both #4661 and #4141 would be wrapped up and merged, then this section would simply summarize and link information. if you want to add something to #4661 or next to it about moving information into javascript cookies that can be ignored by varnish, that would be useful information too.

#4141 is rather old however, so maybe i should just shorten this section and remove things that are too complicated / dangerous here.

:ref:`CSRF Protection <forms-csrf>`. In this situation, make sure to only
start a session when actually needed, and clear the session when it is no
longer needed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this PR is merged, we should add a hint to the cookbook entry: https://github.com/symfony/symfony-docs/pull/4772/files


.. todo link to cookbook/session/avoid_session_start once https://github.com/symfony/symfony-docs/pull/4661 is merged
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in case #4661 is merged after this PR


Cookies can also be created in Javascript and used only in the frontend, e.g.
Google analytics. These cookies do not matter for the backend and should not
affect the caching decision. Configure your Varnish cache to
`clean the cookies header`_. Unless you changed the PHP configuration, your session
cookie has the name PHPSESSID:

.. code-block:: varnish4

sub vcl_recv {
if (req.http.Cookie) {
set req.http.Cookie = ";" + req.http.Cookie;
set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
set req.http.Cookie = regsuball(req.http.Cookie, ";(PHPSESSID)=", "; \1=");
set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");

if (req.http.Cookie == "") {
remove req.http.Cookie;
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what's the result of this? Is this removing the PHPSESSID cookie only before the request is made to the backend? If so, will this now mean that all the pages on my site cannot start a session (whether they're being cached or not?).

Thanks - I'm asking stupid questions because I'm not a pro on this, and I know we are very close to making this very easy and useful :).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, the only cookie that will be retained is PHPSESSID. this is only about caching when there is no session. (together with what i removed, it would also be relevant to make Vary: Cookie actually useful as the cookie header becomes static with this cleanup).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i added a sentence above to hopefully explain what the idea is, and two comments that should make this more understandable


If only small parts of your application depend on cookies (e.g. you display the
username in the header), you can use :ref:`ESI <edge-side-includes>` for those
fragments. Configure Varnish to store and look up requests in its cache even if
Cookies are present in the request:

.. code-block:: varnish4

sub vcl_recv() {
if (req.http.Cookie) {
/* Force cache lookup for requests with cookies */
return (lookup);
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In practical terms, what should this mean to the user? I think it means that you're telling Varnish to yes be ok with caching requests that contain cookies. And so, as a user, you must be very careful to not rely on the session for any parts of your page where you return caching headers. If you violate this rule, then you will cache something with - for example - a user's username in it.

Is this about right? I want to walk the user through this as much as possible. For me, it seems that you should be (as you elude to here) identifying which portions of your page need the session, isolating them into non-cached ESI fragments, then caching the whole page (or doing a complete opposite strategy where you isolate non-session-needing heavy pieces, then don't cache the whole page).

The point is, this is non-trivial, so perhaps we need to walk them through a bit more tutorial-styled.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I still mean everything I said, but most of what I said should be in the book. I still think we can add a few more details, like those I have in my first paragraph.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i tried to make this more explicit, and also added a warning at the end of the section.


You need to make sure that your backend correctly sets the ``Vary`` header to
tell which responses vary on the cookie and which are the same::

public function loginAction()
{
// ...
$response->setVary('Cookie');
// ...
}

Only set the ``Vary: Cookie`` header on actions that actually depend on whether
the user is logged in, but not on any other actions.

.. caution::

Be sure to test your setup. If you do not ``Vary`` content that depends on
the session, users will see content from somebody else. If you ``Vary`` too
much, the Varnish cache will be filled with duplicate content for every
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i dont think we should suggest varying based on cookies. this is just useable if you vary on flags (Terms, Users Age, ...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i disagree. assume i have one url /_fragment/user/toolbar that renders the user name into every page. the rest of the content does not vary on cookie, but this one url does. and its embedded with ESI into every page. then we would want to cache this url as a variant for every single user to avoid reloading it all the time. once you log out, your session will be destroyed, meaning the session cookie changes and thus you don't see your username anymore.

obviously, you need to clean the cookie string for this to work, you don't want any js cookies irrelevant for the backend in here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could also store the username in a cookie / localstorage or something else to print it. from my point of view storing user specific content in the reverse proxy can be a good idea but most time you should try to avoid it. people could think it's a good idea to mess up the cache with a Vary on the Cookie. This issue is hard to find for newbies because most times, if they test they use the same cookie. Why not creating a advanced cookbook where we explain how to cache based on Cookies (Javascript), Cookies + Vary (Varnish), Localstorage, ....

user, rendering the cache pointless as cache hits will become rare.

.. tip::

If content is not different for every user, but depends on the roles of a
user, a solution is to separate the cache per group. This pattern is
implemented and explained by the FOSHttpCacheBundle_ under the name
*User Context*.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about linking directly to the relevant page of the bundle documentation ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea. linked.


Ensure Consistent Caching Behaviour
-----------------------------------

Expand Down Expand Up @@ -169,6 +250,7 @@ proxy before it has expired, it adds complexity to your caching setup.
.. _`Varnish`: https://www.varnish-cache.org
.. _`Edge Architecture`: http://www.w3.org/TR/edge-arch
.. _`GZIP and Varnish`: https://www.varnish-cache.org/docs/3.0/phk/gzip.html
.. _`Clean the cookies header`: https://www.varnish-cache.org/trac/wiki/VCLExampleRemovingSomeCookies
.. _`Surrogate-Capability Header`: http://www.w3.org/TR/edge-arch
.. _`cache invalidation`: http://tools.ietf.org/html/rfc2616#section-13.10
.. _`FOSHttpCacheBundle`: http://foshttpcachebundle.readthedocs.org/