-
-
Notifications
You must be signed in to change notification settings - Fork 9.1k
Description
Bug report
Currently asset modules are stored in webpack filesystem cache, seemingly multiple times per asset, increasing the cache size significantly.
What is the current behavior?
Asset modules are cached in cache packs, seems both by the asset parser and then when dumping the compilation generated assets. In our large production build, webpack cache takes ~17GB.
We added the following patch in IdleFileCachePlugin.js and the cache size dropped to 1.8GB and cached build times improved.
diff --git a/lib/cache/IdleFileCachePlugin.js b/lib/cache/IdleFileCachePlugin.js
index 3ac59121bafafbdafae3a7dbe0db04242fb41352..fa67d0c0d64935dd4d53e650945a083521d4d748 100644
--- a/lib/cache/IdleFileCachePlugin.js
+++ b/lib/cache/IdleFileCachePlugin.js
@@ -57,6 +57,23 @@ class IdleFileCachePlugin {
compiler.cache.hooks.store.tap(
{ name: "IdleFileCachePlugin", stage: Cache.STAGE_DISK },
(identifier, etag, data) => {
+ if (identifier.includes("|asset/")) {
+ // avoid storing big binary blobs in cache
+ // we also skip anything directly related to them (like generated code) since that is invalidated every time because the file itself isn't in cache.
+ return;
+ }
+ if (identifier.startsWith("Compilation/assets|")) {
+ // this skips storing generated complete assets in cache, since the asset resources are not cached, these files get invalidated anyway
+ return;
+ }
If the current behavior is a bug, please provide the steps to reproduce.
Enable filesystem cache, configure asset/resource modules.
What is the expected behavior?
Asset module binary data is not stored in cache. Perhaps add a size limit to allow caching (just an example) 2KB assets, but nothing bigger. Reading the asset from disk directly would be as fast as reading it from the cache.
Other relevant information:
webpack version: 5.97.1
Node.js version: 23.8.0
Operating System: Win11
Additional tools: