Skip to content

Asset module binary files are stored in webpack cache #19359

@Knagis

Description

@Knagis

Bug report

Currently asset modules are stored in webpack filesystem cache, seemingly multiple times per asset, increasing the cache size significantly.

What is the current behavior?

Asset modules are cached in cache packs, seems both by the asset parser and then when dumping the compilation generated assets. In our large production build, webpack cache takes ~17GB.

We added the following patch in IdleFileCachePlugin.js and the cache size dropped to 1.8GB and cached build times improved.

diff --git a/lib/cache/IdleFileCachePlugin.js b/lib/cache/IdleFileCachePlugin.js
index 3ac59121bafafbdafae3a7dbe0db04242fb41352..fa67d0c0d64935dd4d53e650945a083521d4d748 100644
--- a/lib/cache/IdleFileCachePlugin.js
+++ b/lib/cache/IdleFileCachePlugin.js
@@ -57,6 +57,23 @@ class IdleFileCachePlugin {
 		compiler.cache.hooks.store.tap(
 			{ name: "IdleFileCachePlugin", stage: Cache.STAGE_DISK },
 			(identifier, etag, data) => {
+				if (identifier.includes("|asset/")) {
+					// avoid storing big binary blobs in cache
+					// we also skip anything directly related to them (like generated code) since that is invalidated every time because the file itself isn't in cache. 
+					return;
+				}
+				if (identifier.startsWith("Compilation/assets|")) {
+					// this skips storing generated complete assets in cache, since the asset resources are not cached, these files get invalidated anyway
+					return;
+				}

If the current behavior is a bug, please provide the steps to reproduce.

Enable filesystem cache, configure asset/resource modules.

What is the expected behavior?

Asset module binary data is not stored in cache. Perhaps add a size limit to allow caching (just an example) 2KB assets, but nothing bigger. Reading the asset from disk directly would be as fast as reading it from the cache.

Other relevant information:
webpack version: 5.97.1
Node.js version: 23.8.0
Operating System: Win11
Additional tools:

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions