Skip to content

Rewrite the varnish cookbook article #4627

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 22, 2014
Merged

Rewrite the varnish cookbook article #4627

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9ea3671
Rewrite the varnish cookbook article
dbu Dec 11, 2014
dc9430a
drop varnish 2 example from varnish cookbook article
dbu Dec 19, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,12 +101,10 @@
lexers['php-annotations'] = PhpLexer(startinline=True)
lexers['php-standalone'] = PhpLexer(startinline=True)
lexers['php-symfony'] = PhpLexer(startinline=True)
lexers['varnish2'] = CLexer()
lexers['varnish3'] = CLexer()
lexers['varnish4'] = CLexer()

config_block = {
'varnish2': 'Varnish 2',
'varnish3': 'Varnish 3',
'varnish4': 'Varnish 4'
}
Expand Down
268 changes: 83 additions & 185 deletions cookbook/cache/varnish.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,27 +7,91 @@ How to Use Varnish to Speed up my Website
Because Symfony's cache uses the standard HTTP cache headers, the
:ref:`symfony-gateway-cache` can easily be replaced with any other reverse
proxy. `Varnish`_ is a powerful, open-source, HTTP accelerator capable of serving
cached content quickly and including support for :ref:`Edge Side Includes <edge-side-includes>`.
cached content fast and including support for :ref:`Edge Side Includes <edge-side-includes>`.

Trusting Reverse Proxies
------------------------
.. index::
single: Varnish; configuration

Make Symfony Trust the Reverse Proxy
------------------------------------

For ESI to work correctly and for the :ref:`X-FORWARDED <varnish-x-forwarded-headers>`
headers to be used, you need to configure Varnish as a
:doc:`trusted proxy </cookbook/request/load_balancer_reverse_proxy>`.

.. index::
single: Varnish; configuration
.. _varnish-x-forwarded-headers:

Routing and X-FORWARDED Headers
-------------------------------

To ensure that the Symfony Router generates URLs correctly with Varnish,
a ``X-Forwarded-Port`` header must be present for Symfony to use the
correct port number.

Configuration
-------------
This port depends on your setup. Lets say that external connections come in
on the default HTTP port 80. For HTTPS connections, there is another proxy
(as Varnish does not do HTTPS itself) on the default HTTPS port 443 that
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

‘system’ is a bit vague: are you referring to an Apache or Nginx proxy here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. or haproxy or some load balancer system that does the SSL termination. i don't really want to go into the details here as its out of scope, apart from the fact that the port has to be communicated. i am assuming that people doing this will know how to do it... do you have an idea how to make this better without digressing into server setup too much?

handles the SSL termination and forwards the requests as HTTP requests to
Varnish with a ``X-Forwarded-Proto`` header. In this case, you need to add
the following configuration snippet:

As seen previously, Symfony is smart enough to detect whether it talks to a
reverse proxy that understands ESI or not. It works out of the box when you
use the Symfony reverse proxy, but you need a special configuration to make
it work with Varnish. Thankfully, Symfony relies on yet another standard
written by Akamai (`Edge Architecture`_), so the configuration tips in this
chapter can be useful even if you don't use Symfony.
.. code-block:: varnish4
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we only show varnish4?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we use varnish4 code highlighting and do not show tabs at all, because
all versions look the same .


sub vcl_recv {
if (req.http.X-Forwarded-Proto == "https" ) {
set req.http.X-Forwarded-Port = "443";
} else {
set req.http.X-Forwarded-Port = "80";
}
}

.. note::

Remember to configure :ref:`framework.trusted_proxies <reference-framework-trusted-proxies>`
in the Symfony configuration so that Varnish is seen as a trusted proxy
and the ``X-Forwarded-*`` headers are used.

Varnish automatically forwards the IP as ``X-Forwarded-For`` and leaves
the ``X-Forwarded-Proto`` header in the request. If you do not configure
Varnish as trusted proxy, Symfony will see all requests as coming through
insecure HTTP connections from the Varnish host instead of the real client.

If the ``X-Forwarded-Port`` header is not set correctly, Symfony will append
the port where the PHP application is running when generating absolute URLs,
e.g. ``http://example.com:8080/my/path``.

Ensure Consistent Caching Behaviour
-----------------------------------

Varnish uses the cache headers sent by your application to determine how
to cache content. However, versions prior to Varnish 4 did not respect
``Cache-Control: no-cache``. To ensure consistent behaviour, use the following
configuration if you are still using Varnish 3:

.. configuration-block::

.. code-block:: varnish3

sub vcl_fetch {
/* By default, Varnish3 ignores Cache-Control: no-cache and private
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i split up the cache handling and ESI as they have nothing to do with each other

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, we should recommend HTTP/1.1 (Cache-Control) over HTTP/1.0 (Pragma).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did that. dropped the whole pragma topic.

https://www.varnish-cache.org/docs/3.0/tutorial/increasing_your_hitrate.html#cache-control
*/
if (beresp.http.Cache-Control ~ "no-cache" ||
beresp.http.Cache-Control ~ "private"
) {
return (hit_for_pass);
}
}

Enable Edge Side Includes (ESI)
-------------------------------

As explained in the :ref:`Edge Side Includes section<edge-side-includes>`,
Symfony detects whether it talks to a reverse proxy that understands ESI or
not. When you use the Symfony reverse proxy, you don't need to do anything.
But to make Varnish instead of Symfony resolve the ESI tags, you need some
configuration in Varnish. Symfony uses the ``Surrogate-Capability`` header
from the `Edge Architecture`_ described by Akamai.

.. note::

Expand Down Expand Up @@ -58,22 +122,12 @@ Symfony adds automatically:

.. code-block:: varnish4

/* (https://www.varnish-cache.org/docs/4.0/whats-new/upgrading.html#req-not-available-in-vcl-backend-response) */
sub vcl_backend_response {
// Check for ESI acknowledgement and remove Surrogate-Control header
if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
unset beresp.http.Surrogate-Control;
set beresp.do_esi = true;
}
/* By default Varnish ignores Pragma: nocache
(https://www.varnish-cache.org/docs/4.0/users-guide/increasing-your-hitrate.html#cache-control)
so in order avoid caching it has to be done explicitly */
if (beresp.http.Pragma ~ "no-cache") {
// https://www.varnish-cache.org/docs/4.0/whats-new/upgrading.html#hit-for-pass-objects-are-created-using-beresp-uncacheable
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}
}

.. code-block:: varnish3
Expand All @@ -84,38 +138,13 @@ Symfony adds automatically:
unset beresp.http.Surrogate-Control;
set beresp.do_esi = true;
}
/* By default Varnish ignores Cache-Control: nocache
(https://www.varnish-cache.org/docs/3.0/tutorial/increasing_your_hitrate.html#cache-control),
so in order avoid caching it has to be done explicitly */
if (beresp.http.Pragma ~ "no-cache" ||
beresp.http.Cache-Control ~ "no-cache" ||
beresp.http.Cache-Control ~ "private") {
return (hit_for_pass);
}
}

.. code-block:: varnish2

sub vcl_fetch {
// Check for ESI acknowledgement and remove Surrogate-Control header
if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
unset beresp.http.Surrogate-Control;
esi;
}
/* By default Varnish ignores Cache-Control: nocache
so in order avoid caching it has to be done explicitly */
if (beresp.http.Pragma ~ "no-cache" ||
beresp.http.Cache-Control ~ "no-cache" ||
beresp.http.Cache-Control ~ "private") {
return (hit_for_pass);
}
}

.. caution::
.. tip::

Compression with ESI was not supported in Varnish until version 3.0
(read `GZIP and Varnish`_). If you're not using Varnish 3.0, put a web
server in front of Varnish to perform the compression.
If you followed the advice about ensuring a consistent caching
behaviour, those vcl functions already exist. Just append the code
to the end of the function, they won't interfere with each other.

.. index::
single: Varnish; Invalidation
Expand All @@ -134,139 +163,8 @@ proxy before it has expired, it adds complexity to your caching setup.
invalidation by helping you to organize your caching and
invalidation setup.

Varnish can be configured to accept a special HTTP ``PURGE`` method
that will invalidate the cache for a given resource:

.. code-block:: varnish4

/*
Connect to the backend server
on the local machine on port 8080
*/
backend default {
.host = "127.0.0.1";
.port = "8080";
}

sub vcl_recv {
/*
Varnish default behavior doesn't support PURGE.
Match the PURGE request and immediately do a cache lookup,
otherwise Varnish will directly pipe the request to the backend
and bypass the cache
*/
if (req.request == "PURGE") {
return(lookup);
}
}

sub vcl_hit {
// Match PURGE request
if (req.request == "PURGE") {
// Force object expiration for Varnish < 3.0
set obj.ttl = 0s;
// Do an actual purge for Varnish >= 3.0
// purge;
error 200 "Purged";
}
}

sub vcl_miss {
/*
Match the PURGE request and
indicate the request wasn't stored in cache.
*/
if (req.request == "PURGE") {
error 404 "Not purged";
}
}

.. caution::

You must protect the ``PURGE`` HTTP method somehow to avoid random people
purging your cached data. You can do this by setting up an access list:

.. code-block:: varnish4

/*
Connect to the backend server
on the local machine on port 8080
*/
backend default {
.host = "127.0.0.1";
.port = "8080";
}

// ACL's can contain IP's, subnets and hostnames
acl purge {
"localhost";
"192.168.55.0"/24;
}

sub vcl_recv {
// Match PURGE request to avoid cache bypassing
if (req.request == "PURGE") {
// Match client IP to the ACL
if (!client.ip ~ purge) {
// Deny access
error 405 "Not allowed.";
}
// Perform a cache lookup
return(lookup);
}
}

sub vcl_hit {
// Match PURGE request
if (req.request == "PURGE") {
// Force object expiration for Varnish < 3.0
set obj.ttl = 0s;
// Do an actual purge for Varnish >= 3.0
// purge;
error 200 "Purged";
}
}

sub vcl_miss {
// Match PURGE request
if (req.request == "PURGE") {
// Indicate that the object isn't stored in cache
error 404 "Not purged";
}
}

.. _varnish-x-forwarded-headers:

Routing and X-FORWARDED Headers
-------------------------------

To ensure that the Symfony Router generates URLs correctly with Varnish,
proper ```X-Forwarded``` headers must be added so that Symfony is aware of
the original port number of the request. Exactly how this is done depends
on your setup. As a simple example, Varnish and your web server are on the
same machine and that Varnish is listening on one port (e.g. 80) and Apache
on another (e.g. 8080). In this situation, Varnish should add the ``X-Forwarded-Port``
header so that the Symfony application knows that the original port number
is 80 and not 8080.

If this header weren't set properly, Symfony may append ``8080`` when generating
absolute URLs:

.. code-block:: varnish4

sub vcl_recv {
if (req.http.X-Forwarded-Proto == "https" ) {
set req.http.X-Forwarded-Port = "443";
} else {
set req.http.X-Forwarded-Port = "80";
}
}

.. note::

Remember to configure :ref:`framework.trusted_proxies <reference-framework-trusted-proxies>`
in the Symfony configuration so that Varnish is seen as a trusted proxy
and the ``X-Forwarded-`` headers are used.
The documentation of the `FOSHttpCacheBundle`_ explains how to configure
Varnish and other reverse proxies for cache invalidation.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it ok to just throw out the purge information here and reference FOSHttpCacheBundle instead?

the x-forwarded-proto moved to the top as its actually always required to get a correct setup

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with that. I would make FOSHttpCacheBundle a link here too though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i did not link because its linked in the paragraph just above. but does
not hurt i guess, added the link.


.. _`Varnish`: https://www.varnish-cache.org
.. _`Edge Architecture`: http://www.w3.org/TR/edge-arch
Expand Down