@jmartinsnexity Try locally fix from: #28144

Thank you @stloyd but the #28144 fix doesn't change anything for my problem.

After reviewing some of last issues, I think my problem might be related to the #28225 or #28226

Please confirm #28241 fixes your issue.

Hello @nicolas-grekas, i'm sorry to tell that the #28241 does not fix the issue.
I may have misslead you by saying that my issue could be related with #cve-2018-14774.

As I couldn't reproduce it locally, I set up a new environnement on production server in order to do some debug.
Here is what I found :
The Exception is raised by Component/HttpKernel/EventListener/FragmentListener line 90.
This occurs because on line 86 signer->check() return false, which is caused by a different URI hash result in Component/HttpKernel/UriSigner.

If the MasterRequest is GET, the subRequests are signed by HTTPS url and checked by HTTPS url : it works.
If the MasterRequest is POST, the subRequests are signed by HTTPS url but checked by HTTP url : it fails.

When it fails, it means that in Component/HttpFoundation/Request, the function isSecure returns false.
After reading the function, I think the problem might be linked with the value returned by "$this->getTrustedValues(self::HEADER_CLIENT_PROTO)"

In v3.3.17, $this->getTrustedValues(self::HEADER_CLIENT_PROTO) returns ["https"].
In v3.3.18, it returns [].

@jmartinsnexity would you be able to create a small app that reproduces the issue with e.g. Symfony 2.8 or 3.4 (3.3 is not maintained anymore) when behind your varnish setup? It'd like to clone it and run it locally. Thanks.

Alternatively, could you try reverting this patch partially and figure out if why change breaks your case? Trying to debug the internal state changes around it would be awesome if you can.

@nicolas-grekas
I found the main difference in SubRequest headers between 3.3.17 and 3.3.18 when MasterRequest is POST.

Let say my browser IP is 1.1.1.1.
v3.3.17 headers in SubRequest :
X_FORWARDED_FOR : "1.1.1.1, 127.0.0.1"
X_FORWARDED_PROTO : "https"

v3.3.18 headers in SubRequest :
X_FORWARDED_FOR : "127.0.0.1"
X_FORWARDED_PROTO : NULL

I guess the new SubRequestHandler removes the 'X_FORWARDED_PROTO' header for Cached SubRequests even if the connection is secure.

I'll try to go into deeper debug to find out what lines break my case.

When the method of the masterRequest is POST, the REMOTE_ADDR in the SubRequest is 127.0.0.1 and not the IP of the LoadBalancer.

Is it possible that in SubRequestHandler line 44 "IpUtils::checkIp($remoteAddr, $trustedProxies)" considers 127.0.0.1 as unsecure, and removes X_FORWARDED_* headers on line 46 ?

Are you sure you defined your trusted proxies correctly, ie the list contains the ip of varnish as seen from PHP?

Can you spot any different in the incoming master request except POST?

I'm running out of ideas so a reproducer would be very welcome.

@nicolas-grekas
Please consider this my last chance before developing a whole new app to reproduce my case.
Thank you for your time and attention.

I came out with a solution, let me explain, you'll know much better than me if it's good. Everything is about Symfony/Component/HttpKernel/HttpCache/SubRequestHandler.php.

As mentioned earlier, i didn't understand why 127.0.0.1 is not set as trusted proxy before the handle function removes untrusted values, so I tried to pull up 5 lines (from line 79-83 to 42-46) :

        // ensure 127.0.0.1 is set as trusted proxy
        if (!IpUtils::checkIp('127.0.0.1', $trustedProxies)) {
            Request::setTrustedProxies(array_merge($trustedProxies, array('127.0.0.1')), Request::getTrustedHeaderSet());
        }

Then, in the "// remove untrusted values" block, I replaced "$trustedProxies" by "Request::getTrustedProxies()", as it's been modified just before.

This modification preserves the trusted "X_FORWARDED_PROTO" request header, so the signed subrequest hash is correct when checking it.
All my render_esi twig blocks are now rendered, even behind my Load Balancer, on POST request.

Full modified handle function :

    public static function handle(HttpKernelInterface $kernel, Request $request, $type, $catch)
    {
        // save global state related to trusted headers and proxies
        $trustedProxies = Request::getTrustedProxies();
        $trustedHeaderSet = Request::getTrustedHeaderSet();
        $trustedHeaders = array(
            Request::HEADER_FORWARDED => Request::getTrustedHeaderName(Request::HEADER_FORWARDED, false),
            Request::HEADER_X_FORWARDED_FOR => Request::getTrustedHeaderName(Request::HEADER_X_FORWARDED_FOR, false),
            Request::HEADER_X_FORWARDED_HOST => Request::getTrustedHeaderName(Request::HEADER_X_FORWARDED_HOST, false),
            Request::HEADER_X_FORWARDED_PROTO => Request::getTrustedHeaderName(Request::HEADER_X_FORWARDED_PROTO, false),
            Request::HEADER_X_FORWARDED_PORT => Request::getTrustedHeaderName(Request::HEADER_X_FORWARDED_PORT, false),
        );

        // ensure 127.0.0.1 is set as trusted proxy
        if (!IpUtils::checkIp('127.0.0.1', $trustedProxies)) {
            Request::setTrustedProxies(array_merge($trustedProxies, array('127.0.0.1')), Request::getTrustedHeaderSet());
        }
        
        // remove untrusted values
        $remoteAddr = $request->server->get('REMOTE_ADDR');
        if (!IpUtils::checkIp($remoteAddr, Request::getTrustedProxies())) {
            foreach (array_filter($trustedHeaders) as $name) {
                $request->headers->remove($name);
            }
        }

        // compute trusted values, taking any trusted proxies into account
        $trustedIps = array();
        $trustedValues = array();
        foreach (array_reverse($request->getClientIps()) as $ip) {
            $trustedIps[] = $ip;
            $trustedValues[] = sprintf('for="%s"', $ip);
        }
        if ($ip !== $remoteAddr) {
            $trustedIps[] = $remoteAddr;
            $trustedValues[] = sprintf('for="%s"', $remoteAddr);
        }

        // set trusted values, reusing as much as possible the global trusted settings
        if ($name = $trustedHeaders[Request::HEADER_FORWARDED]) {
            $trustedValues[0] .= sprintf(';host="%s";proto=%s', $request->getHttpHost(), $request->getScheme());
            $request->headers->set($name, implode(', ', $trustedValues));
        }
        if ($name = $trustedHeaders[Request::HEADER_X_FORWARDED_FOR]) {
            $request->headers->set($name, implode(', ', $trustedIps));
        }
        if (!$name && !$trustedHeaders[Request::HEADER_FORWARDED]) {
            Request::setTrustedProxies($trustedProxies, $trustedHeaderSet | Request::HEADER_X_FORWARDED_FOR);
            $request->headers->set(Request::getTrustedHeaderName(Request::HEADER_X_FORWARDED_FOR, false), implode(', ', $trustedIps));
        }

        // fix the client IP address by setting it to 127.0.0.1,
        // which is the core responsibility of this method
        $request->server->set('REMOTE_ADDR', '127.0.0.1');

        try {
            return $kernel->handle($request, $type, $catch);
        } finally {
            // restore global state
            Request::setTrustedProxies($trustedProxies, $trustedHeaderSet);
        }
    }

This patch turns 127.0.0.1 as an always trusted proxy. I'm not sure why we should do this...

I just merged #28241 into 3.4. Could you update to 3.4@dev before trying to reproduce (and check again that 3.4@dev still has the issue?) Please also double check your trusted proxies list.

Thank you @nicolas-grekas, the problem is solved in latest 3.4 version !

Awesome, thanks for checking.