Description
We have a large Symfony app, which struggles to clear cache in low memory environments (platform.sh). It ends up swapping and a cache warmup can take minutes.
I've tried updating to php7, which has improved our app massively, but has some interesting side-effects to clearing the cache.
I've added some crude debug to show the memory consumption and time taken during this process.
The debug output format is TIME: CURRENT MEMORY PEAK MEMORY: EVENT DESCRIPTION
$ php app/console cache:clear --env=prod
08:33:44: 8MB 10MB: pre loading kernel cache
08:33:44: 8MB 10MB: post loading kernel cache
08:33:44: 8MB 10MB: pre initializing bundles
08:33:44: 10MB 10MB: post initializing bundles
08:33:44: 10MB 10MB: pre initializing container
08:33:44: 10MB 10MB: pre cache require
08:33:44: 20MB 20MB: pre setting continaer locally
08:33:44: 20MB 20MB: pre warming up container
08:33:44: 20MB 20MB: post initializing container
08:33:44: 20MB 20MB: pre initializing bundles-in-container
08:33:44: 20MB 20MB: post initializing bundles-in-container
08:33:46: 34MB 34MB: pre command->run()
Clearing the cache for the prod environment with debug false
08:33:46: 34MB 34MB: Clearing outdated warmup directory
08:33:46: 36MB 36MB: warming up cache
08:33:46: 36MB 36MB: removed warmup dir
08:33:46: 36MB 36MB: pre getting temp kernel
08:33:46: 36MB 36MB: pre booting temp kernel AppKerne_
08:33:46: 36MB 36MB: pre loading kernel cache
08:33:46: 36MB 36MB: post loading kernel cache
08:33:46: 36MB 36MB: pre initializing bundles
08:33:46: 36MB 36MB: post initializing bundles
08:33:46: 36MB 36MB: pre initializing container
08:33:46: 36MB 36MB: pre buildig container
08:33:46: 40MB 40MB: pre compiling container
08:33:49: 58MB 58MB: pre dump container
08:33:49: 58MB 58MB: post phpdumper
08:33:49: 58MB 58MB: pre set proxy dumper
08:33:49: 58MB 58MB: pre dumper dump
08:33:49: 62MB 65MB: pre stripping contents
08:33:49: 146MB 166MB: pre writing cache
08:33:49: 146MB 166MB: post writing cache
08:33:49: 144MB 166MB: pre cache require
08:33:49: 150MB 166MB: pre setting continaer locally
08:33:49: 150MB 166MB: pre warming up container
08:33:49: 150MB 166MB: pre warming up: Symfony\Bridge\Doctrine\CacheWarmer\ProxyCacheWarmer
08:33:50: 150MB 166MB: post warming up: Symfony\Bridge\Doctrine\CacheWarmer\ProxyCacheWarmer
08:33:50: 150MB 166MB: pre warming up: Symfony\Bridge\Doctrine\CacheWarmer\ProxyCacheWarmer
08:33:50: 150MB 166MB: post warming up: Symfony\Bridge\Doctrine\CacheWarmer\ProxyCacheWarmer
08:33:50: 150MB 166MB: post initializing container
08:33:50: 150MB 166MB: pre initializing bundles-in-container
08:33:50: 150MB 166MB: post initializing bundles-in-container
08:33:50: 150MB 166MB: post booting temp kernel
08:33:50: 150MB 166MB: enabled optional warmers
08:33:50: 150MB 166MB: pre warming up: Symfony\Bundle\FrameworkBundle\CacheWarmer\TemplatePathsCacheWarmer
08:33:50: 150MB 166MB: post warming up: Symfony\Bundle\FrameworkBundle\CacheWarmer\TemplatePathsCacheWarmer
08:33:50: 150MB 166MB: pre warming up: Symfony\Bundle\AsseticBundle\CacheWarmer\AssetManagerCacheWarmer
08:34:39: 174MB 174MB: post warming up: Symfony\Bundle\AsseticBundle\CacheWarmer\AssetManagerCacheWarmer
08:34:39: 174MB 174MB: pre warming up: Symfony\Bridge\Doctrine\CacheWarmer\ProxyCacheWarmer
08:34:39: 174MB 174MB: post warming up: Symfony\Bridge\Doctrine\CacheWarmer\ProxyCacheWarmer
08:34:39: 174MB 174MB: pre warming up: Symfony\Bridge\Doctrine\CacheWarmer\ProxyCacheWarmer
08:34:39: 174MB 174MB: post warming up: Symfony\Bridge\Doctrine\CacheWarmer\ProxyCacheWarmer
08:34:39: 174MB 174MB: pre warming up: Symfony\Bundle\FrameworkBundle\CacheWarmer\TranslationsCacheWarmer
08:34:41: 180MB 180MB: post warming up: Symfony\Bundle\FrameworkBundle\CacheWarmer\TranslationsCacheWarmer
08:34:41: 180MB 180MB: pre warming up: Symfony\Bundle\FrameworkBundle\CacheWarmer\RouterCacheWarmer
08:34:43: 184MB 186MB: post warming up: Symfony\Bundle\FrameworkBundle\CacheWarmer\RouterCacheWarmer
08:34:43: 184MB 186MB: pre warming up: Symfony\Bundle\TwigBundle\CacheWarmer\TemplateCacheCacheWarmer
08:37:04: 284MB 284MB: post warming up: Symfony\Bundle\TwigBundle\CacheWarmer\TemplateCacheCacheWarmer
08:37:04: 284MB 284MB: done warmup
08:37:04: 284MB 284MB: pre fixing meta references
08:37:04: 284MB 284MB: post fixing meta references
08:37:04: 284MB 284MB: pre fixing cached file references
08:37:04: 284MB 288MB: post fixing cached file references
08:37:04: 284MB 288MB: pre fixing container references
08:37:04: 286MB 290MB: post fixing container references
08:37:04: 286MB 290MB: pre deleting temp kernel file
08:37:04: 286MB 290MB: post deleting temp kernel file
08:37:04: 284MB 290MB: removing old cache dir
08:37:04: 290MB 290MB: Done
08:37:04: 290MB 290MB: post command->run()
08:37:04: 290MB 290MB: pre terminateEvent
08:37:04: 290MB 290MB: post terminateEvent
There's a couple of interesting things here, but the one I wanted to ask about is here:
08:33:49: 62MB 65MB: pre stripping contents
08:33:49: 146MB 166MB: pre writing cache
The debug I added was to Kernel:
if (!$this->debug) {
$this->println("pre stripping contents");
$content = static::stripComments($content);
}
$this->println("pre writing cache");
There is an ~80mb permanent growth in memory consumption, which is strange and does not happen on php5, which uses much more memory, but releases it entirely after this method call.
I tried a few things locally to see if there was anything about the way that it was called that could impact this, but I couldn't make any improvement.
Any ideas what's happening here? It could easily be a php7 issue, just wondering if there's a Symfony solution.
The other issue here is the crazy amount of time spent in TemplatePaths and TemplateCache warming (also nowhere near as long on php5), I'll try to dig a bit further into this one locally though...