Skip to content

Fix shelljs compatibility with esbuild bundler #1161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

7PH
Copy link

@7PH 7PH commented Jun 8, 2024

Summary

Fixes #1160

  • Add explicit file extension to dynamic command import.
  • This call to require() is dynamic and does not add the file extension.
  • Esbuild resolves files with their explicit file extension (eg './src/cat.js').
  • Esbuild automatically adds .js extension in imports when not present but can't modify dynamic imports. For dynamic imports, file extension needs to be explicitly specified prior to bundling.

Test

Here is a script to reproduce (to be run in the repository root):

# Prepare
rm -rf test-esbuild && mkdir test-esbuild && cd test-esbuild
npm init --yes
echo "console.log(require('../shell.js').pwd().stdout && '\n OK');" > index.js
npm i -D esbuild

# Run
npx esbuild --platform=node --bundle index.js --outfile=out.js; node out.js

In the current master branch of this repository:

  out.js  230.5kb

⚡ Done in 10ms
/.../shelljs/test-esbuild/out.js:5
  throw new Error("Module not found in bundle: " + path);
  ^

Error: Module not found in bundle: ./src/cat
    at /.../

Node.js v20.3.1

In this branch

  out.js  230.5kb

⚡ Done in 9ms

 OK

The bundle size doesn't change because all deps are already in the bundle, it's a resolution issue, as discussed in #1160

- shelljs#1160
- Add explicit file extension to dynamic command import.
- This call to require() is dynamic and does not add the file extension.
- Esbuild resolves files with their explicit file extension (eg './src/cat.js').
- Esbuild automatically adds .js extension in imports when not present but can't modify dynamic imports. For dynamic imports, file extension needs to be explicitly specified prior to bundling.
@7PH 7PH changed the title fix: fix shelljs compatibility with esbuild bundler Fix shelljs compatibility with esbuild bundler Jun 8, 2024
@nfischer
Copy link
Member

Have you tested this change on other bundlers? I'm not familiar with the current state of bundlers in the community. Is esbuild a popular one?

@7PH
Copy link
Author

7PH commented Jun 15, 2024

Have you tested this change on other bundlers?

I've tested:

  • esbuild (37.5K stars, 31M weekly downloads)
  • rollup (24.9K stars, 22.3M weekly downloads)
  • @vercel/ncc (9K stars, 0.4M weekly downloads)

For esbuild, details are in the issue #1160

rollup

With rollup it doesn't work ouf of the box, but you can use a commonjs plugin and tweak its options so that these imports are treated dynamically (dynamicRequireTargets). This has minor security implications (Note: In extreme cases, this feature may result in some paths being rendered as absolute in the final bundle). This change doesn't affect rollup.

Reproducer
// rollup.config.js
var commonjs = require('@rollup/plugin-commonjs');
var nodeResolve = require('@rollup/plugin-node-resolve').nodeResolve;

module.exports = {
  input: 'index.js',
  output: {
    file: 'out.js',
    format: 'cjs',
  },
  plugins: [commonjs({
    dynamicRequireTargets: [
      'node_modules/shelljs/src/*.js',
    ],
  }), nodeResolve()],
};

// index.js
var shelljs = require('shelljs');

console.log(shelljs.pwd().stdout && '\n OK');

// bash
//$ npx rollup -c rollup.config.js

image

ncc

With ncc it works out of the box because ncc assumes the require could be called without the .js extension as well. (for our use-case, ncc also creates a separate exec-child.js file which isn't suitable for us). This change doesn't affect ncc.

Reproducer
npx ncc build index.js -o out
node out/index.js

image

@7PH
Copy link
Author

7PH commented Jul 29, 2024

Hello @nfischer , is there anything I can do to help make this change go live?

@jonsherrard
Copy link

Would also love to get this in.shelljs is very useful for github actions

@AlexanderLill
Copy link

AlexanderLill commented Sep 4, 2024

I am also highly interested in this, shelljs would be even more useful if it could be bundled! @nfischer
Currently also facing the ./src/cat issue which is so often reported in the issues of this project.

@AlexanderLill
Copy link

AlexanderLill commented Sep 4, 2024

I tried the fix proposed here, but it did not work in my setup with esbuild. However, just manually writing the require statements works for me (even though it is not so elegant):

AlexanderLill#1

@qarlosalberto
Copy link

I have the same issue. It's very common to use esbuild to bundle VSCode extensions. Is there any plan to merge it? Thanks!

@7PH 7PH mentioned this pull request Oct 29, 2024
@nfischer
Copy link
Member

I'm willing to revisit this, however I need someone to show me how to test this change to make sure it actually works. I also see #1119 was raised for a similar issue, so I'm not sure which is the better approach.

@7PH
Copy link
Author

7PH commented Mar 5, 2025

Hello @nfischer , thanks for answering!

I'm not sure which is the better approach.

As a general rule of thumb, I say dynamic imports is something to avoid because that will always mess with build-time tree shaking. Bundlers can cope with this but this relies on blindly putting all the file content in the bundle.

TBH moving to static imports would be the preferred approach if it was my project, but I also think my change is fine if you want to keep the dynamic import.

however I need someone to show me how to test this change to make sure it actually works

For testing, I've put there a reproducer for ESBuild, do you want a reproducer for other bundlers?

@nfischer
Copy link
Member

nfischer commented Mar 8, 2025

Closing this in favor of #1119 and #1194

@nfischer nfischer closed this Mar 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dynamic require() without .js extension breaks bundling (esbuild)
5 participants