Skip to content

[BUG] Can not import .wasm (by @prisma/client/wasm, and other wasm-pack generated) #139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mizchi opened this issue Nov 21, 2024 · 11 comments
Labels
bug Something isn't working triage

Comments

@mizchi
Copy link

mizchi commented Nov 21, 2024

Describe the bug

Opennext and @prisma/client/wasm do not work by import *.wasm.

Prisma does not work on opennext.

$ npm install prisma @prisma/client -D
$ npx prisma --init
# ... edit to use @prisma/client/wasm
$ npx next build
$ npx cloudflare # it works yet
$ wrangler deploy # fail at wrangler's  esbuild
   Creating an optimized production build ...
  Compiled successfully
  Linting and checking validity of types    
  Collecting page data    
Error occurred prerendering page "/". Read more: https://nextjs.org/docs/messages/prerender-error
PrismaClientKnownRequestError: 
Invalid `prisma.post.findMany()` invocation:


Unknown file extension ".wasm" for /home/mizchi/mizchi/next-memo/node_modules/.pnpm/@prisma+client@5.22.0_prisma@5.22.0/node_modules/.prisma/client/query_engine_bg.wasm
    at dr.handleRequestError (/home/mizchi/mizchi/next-memo/node_modules/.pnpm/@prisma+client@5.22.0_prisma@5.22.0/node_modules/@prisma/client/runtime/wasm.js:21:7567)
    at dr.handleAndLogRequestError (/home/mizchi/mizchi/next-memo/node_modules/.pnpm/@prisma+client@5.22.0_prisma@5.22.0/node_modules/@prisma/client/runtime/wasm.js:21:6875)
    at dr.request (/home/mizchi/mizchi/next-memo/node_modules/.pnpm/@prisma+client@5.22.0_prisma@5.22.0/node_modules/@prisma/client/runtime/wasm.js:21:6565)
    at async u (/home/mizchi/mizchi/next-memo/node_modules/.pnpm/@prisma+client@5.22.0_prisma@5.22.0/node_modules/@prisma/client/runtime/wasm.js:30:9560)
    at async I (/home/mizchi/mizchi/next-memo/.next/server/app/page.js:1:39579)
Export encountered an error on /page: /, exiting the build.
  Static worker exited with code: 1 and signal: null
node:internal/errors:983
  const err = new Error(message);

Vite and webpack can handle it but wrangler's esbuild & workerd can not

Steps to reproduce

  • setup prisma
  • import @prisma/client/wasm
import { PrismaClient } from "@prisma/client/wasm";
import { PrismaD1 } from "@prisma/adapter-d1";
import { getCloudflareContext } from "@opennextjs/cloudflare";

export interface Env {
  DB: D1Database;
}

export async function getPrismaClient() {
  try {
    const ctx: any = await getCloudflareContext();
    return new PrismaClient({ adapter: new PrismaD1(ctx.env.DB) });
  } catch (error) {
    console.error("Error getting Prisma client", error);
    throw error;
  }
}

npx wrangler deploy

Expected behavior

@prisma/client works on opennext.

@opennextjs/cloudflare version

0.2.1

Node.js version

22.x

Wrangler version

3.88.0

next info output

Operating System:
  Platform: linux
  Arch: x64
  Version: #1 SMP Tue Nov 5 00:21:55 UTC 2024
  Available memory (MB): 64163
  Available CPU cores: 16
Binaries:
  Node: 22.9.0
  npm: 10.8.3
  Yarn: N/A
  pnpm: 9.9.0
Relevant Packages:
  next: 15.0.3 // Latest available version is detected (15.0.3).
  eslint-config-next: 15.0.3
  react: 18.3.1
  react-dom: 18.3.1
  typescript: 5.6.3
Next.js Config:
  output: N/A


### Additional context

This `.wasm`  import style is generated by Rust's `wasm-pack`
@mizchi mizchi added bug Something isn't working triage labels Nov 21, 2024
@mizchi mizchi changed the title [BUG] Can not import .wasm (by @prisma/client) [BUG] Can not import .wasm (by @prisma/client/wasm) Nov 21, 2024
@mizchi mizchi changed the title [BUG] Can not import .wasm (by @prisma/client/wasm) [BUG] Can not import .wasm (by @prisma/client/wasm, and other wasm-pack generated) Nov 21, 2024
@wmadden
Copy link

wmadden commented Nov 29, 2024

Hi @mizchi, it looks like your issue is caused by wrangler which by default will delete any .wasm files when you publish.

Others have solved this problem by adding configuration to wrangler.toml - read more about it here.

@vicb
Copy link
Contributor

vicb commented Nov 29, 2024

Good point, can you try adding find_additional_modules=true in your wrangler.toml?
It will add the the .wasm files as additional modules.

@mizchi
Copy link
Author

mizchi commented Dec 3, 2024

thx @vicb @wmadden

I used those links to verify this by rewriting node_moudules directly at hand.

First I needed to examine the prisma code generation and runtime steps.

  • Code generation process
    • Running npx prisma generate executes @prisma/client/generator-build/index.js by prisma's CLI, and @prisma/client/.prisma/client/* generates Assemble the client code.
    • Copy @prisma/client/.prisma/client/query_engine_wasm.bg according to Prisma's generator(postgres/mysql/sqlite)
  • Runtime
    • import {PrismaClient} from @prisma/client/wasm resolves @prisma/client/.prisma/client/wasm.js(cjs)
    • Initialize query_engine_bg.wasm as binary by fs.readFileSync with return new WebAssembly.Module(queryEngineWasmFileBytes).

https://github.com/prisma/prisma/blob/b6fe641510af621b30eb183076d98b61cdfe9c3d/packages/client/src/generation/utils/buildGetQueryEngineWasmModule.ts#L13-L21

I found some problems here.

  • According to this document workers need to initialize wasm in global scope before the fetch handler
  • prisma/client resolves "./wasm.js" as cjs, so imports in global scope cannot resolve it
  • The wrangler wasm_modules cannot be used in ESM mode. But the output of opennext always expects ESM
  • Webpack: opennext pre-processes next as webpack first, so .wasm cannot be resolved at this stage. you need to delay resolving .wasm until wrangler esbuild.

This is my trial and error. It hasn't completely worked yet, but it worked on hand to the point where it bypasses the initialization of wasm.

First, in next.config.js, prevent .wasm from loading.

    config.plugins.push(
      new webpack.IgnorePlugin({
        contextRegExp: /\\.wasm$/,
        resourceRegExp: /\\.wasm$/, }
      })
    );

At prisma/client/wasm runtime, modify the code generator to resolve from global variables.

diff --git a/generator-build/index.js b/generator-build/index.js
index f0adad1914bb059450d36d8df69acd57f9c95750..056bf25d709c0ca12f0b748d58f2156f54fadec9 100644
--- a/generator-build/index.js
+++ b/generator-build/index.js
@@ -7753,10 +7753,7 @@ function buildQueryEngineWasmModule(wasm, copyEngine, runtimeNameJs) {
     return `config.engineWasm = {
       getRuntime: () => require('. /query_engine_bg.js'),.
       getQueryEngineWasmModule: async () => {
- const queryEngineWasmFilePath = require('path').join(config.dirname, 'query_engine_bg.wasm')
- const queryEngineWasmFileBytes = require('fs').readFileSync(queryEngineWasmFilePath)
-fs'.      
- return new WebAssembly.Module(queryEngineWasmFileBytes)
+ return __PRISMA_BINARY;
       }
     }`;
   }
@@ -7764,9 +7761,7 @@ function buildQueryEngineWasmModule(wasm, copyEngine, runtimeNameJs) {
     return `config.engineWasm = {
   getRuntime: () => require('. /query_engine_bg.js'),.
   getQueryEngineWasmModule: async () => {
- const loader = (await import('#wasm-engine-loader')).default
- const engine = (await loader).default
- return engine 
+ return __PRISMA_BINARY;
   }
 }`;
   }

With this in place, copy . /query_engine_bg.wasm directly down into .workers-next and insert the following code into the .workers-next/index.mjs entry point for workers.

import wasm from ". /query_engine_bg.wasm";
globalThis.__PRISMA_BINARY = await WebAssembly.instantiate(wasm);
// ...original

This should now work, at least on the Workers runtime.

However, we could not verify that this actually works. The reason is that it fails in the prerender phase of the Next App Router. The prerender executes .next directly in node.js, so on the contrary, it can no longer resolve the __PRISMA_BINARY in this patch.

There may be a better way. I will experiment some more.

@vicb
Copy link
Contributor

vicb commented Dec 3, 2024

Thanks for the detailed analysis!

@Yesterday17
Copy link

Yesterday17 commented Feb 18, 2025

I managed to use and deploy my app using prisma by following the steps above, but with a slightly different change:

import wasm from "./query_engine_bg.wasm";
globalThis.__PRISMA_BINARY = wasm // <-- Prisma would initialize it with correct imports

And prisma will do the WebAssembly.instantiate step in its code.

@mizchi
Copy link
Author

mizchi commented Mar 28, 2025

I deal with the issue on a regular basis.

Prisma itself is currently being rewritten from Rust to TypeScript.

https://www.prisma.io/blog/rust-to-typescript-update-boosting-prisma-orm-performance

This will likely circumvent this issue.

We're enabling an experimental client based on benchmarks.

https://github.com/prisma/query-compiler-benchmarks/tree/main/benches/prisma-pg-qc

Setup

DATABASE_URL="..."
PRISMA_UNSTABLE_CLIENT_ENGINE_TYPE=true

schema.prisma

datasource db {
  provider = "postgres"
  url      = env("DATABASE_URL")
}

generator client {
  provider        = "prisma-client-js"
  previewFeatures = ["driverAdapters"]
  engineType      = "client"
}

PRISMA_UNSTABLE_CLIENT_ENGINE_TYPE=true and engineType = "client" are impotant.

Run with postgres on node.js

import { PrismaClient } from "@prisma/client";
import { PrismaPg } from "@prisma/adapter-pg";
import pg from "pg";

const connectionString = process.env.DATABASE_URL ?? "";
const pool = new pg.Pool({ connectionString });
const adapter = new PrismaPg(pool);
const prisma = new PrismaClient({ adapter });
await prisma.$connect();

try {
  const users = await prisma.user.findMany();
  console.log("users", users);
} catch (error) {
  console.error("error", error);
} finally {
  await prisma.$disconnect().catch(console.error);
  process.exit(0);
}

This runtime allowed me to avoid using wasm, at least for postgres on localhost docker.

However, it didn't work with anything other than postgres.

I haven't tried it yet, but it's likely to work on opennext if you use prisma/adapte-pg-worker.

@mizchi
Copy link
Author

mizchi commented May 4, 2025

Prisma (no-rust, 6.7.0 preview) works with OpenNext!

https://github.com/prisma/prisma/releases/tag/6.7.0

It uses with queryCompiler and driverAdapters.

generator client {
  provider = "prisma-client-js"
  previewFeatures = ["queryCompiler", "driverAdapters"]
  output   = "../src/generated/prisma"
}

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

// sample model
model User {
  id        String   @id @default(cuid())
  name      String?
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
}

Generate prisma/client without runtime engines.

$ npx prisma generate --no-engine
$ npx prisma migrate dev

(I tried with https://www.prisma.io/postgres)

// src/app/page.tsx
import { PrismaClient } from "../generated/prisma";
const prisma = new PrismaClient();

export default async function Page() {
  const user = await prisma.user.findFirst({});
  return (
    <div>
      <pre>{JSON.stringify(user, null, 2)}</pre>
    </div>
  );
}

This doesn't solve the issue of wasm-pack generating code that can't be loaded, but it seems to solve the problem with prisma.

@mizchi
Copy link
Author

mizchi commented May 6, 2025

I reported prisma/client/runtime/library does not work on workerd but it relates opennext bundle process about import.meta.url

prisma/prisma#27098

To summarize briefly, @prisma/client/runtime/library does not work in the workerd environment and expands fileURLToPath(import.meta.url) to fileURLToPath(undefined), which causes an error.

@conico974
Copy link
Collaborator

@mizchi This should be fixed in 1.0.0

Also see the docs for prisma here https://opennext.js.org/cloudflare/howtos/db#prisma-orm

@mizchi
Copy link
Author

mizchi commented May 6, 2025

@conico974

I overlooked that prisma documentation. Thanks!

I tried v1.0.0 and the import.meta.url issue seems to be resolved.

Unfortunately, it seems that I can't use that method when I enable queryCompiler.

generator client {
  provider = "prisma-client-js"
  previewFeatures = ["queryCompiler", "driverAdapters"]
  // output   = "../src/generated/prisma"
}
$ npx prisma generate --no-engine
Warning: You did not specify an output path for your `generator` in schema.prisma. This behavior is deprecated and will no longer be supported in Prisma 7.0.0. To learn more visit https://pris.ly/cli/output-path

I failed to call prisma.user.findMany({}) in this case.

It seems that specifying output will be required in 7.0.0, but the experimental queryCompiler may already require it.

@conico974
Copy link
Collaborator

@mizchi
Have you tried without the experimental queryCompiler ? What error do you get ?
I've tested with what's in the docs and it works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Development

No branches or pull requests

5 participants