Skip to content

Contributing guide: nix-shell doesn't install all tools #4854

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mtojek opened this issue Nov 2, 2022 · 5 comments
Closed

Contributing guide: nix-shell doesn't install all tools #4854

mtojek opened this issue Nov 2, 2022 · 5 comments
Assignees
@matifali
Labels
s3 Bugs that confuse, annoy, or are purely cosmetic

Comments

@mtojek
Copy link
Member

mtojek commented Nov 2, 2022

OS: Monterey 12.3

I was following the CONTRIBUTING guide and decided to use nix-shell. Unfortunately, the method wasn't perfect as I spent some time on solving pkg-config related issues:

Show console dump 
error /Users/mtojek/code/coder/site/node_modules/canvas: Command failed.
Exit code: 1
Command: node-pre-gyp install --fallback-to-build --update-binary
Arguments:
Directory: /Users/mtojek/code/coder/site/node_modules/canvas
Output:
node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using node-pre-gyp@1.0.10
node-pre-gyp info using node@16.16.0 | darwin | arm64
node-pre-gyp http GET https://github.com/Automattic/node-canvas/releases/download/v2.10.0/canvas-v2.10.0-node-v93-darwin-unknown-arm64.tar.gz
node-pre-gyp ERR! install response status 404 Not Found on https://github.com/Automattic/node-canvas/releases/download/v2.10.0/canvas-v2.10.0-node-v93-darwin-unknown-arm64.tar.gz
node-pre-gyp WARN Pre-built binaries not installable for canvas@2.10.0 and node@16.16.0 (node-v93 ABI, unknown) (falling back to source compile with node-gyp)
node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/Automattic/node-canvas/releases/download/v2.10.0/canvas-v2.10.0-node-v93-darwin-unknown-arm64.tar.gz
gyp info it worked if it ends with ok
gyp info using node-gyp@9.0.0
gyp info using node@16.16.0 | darwin | arm64
gyp info ok
gyp info it worked if it ends with ok
gyp info using node-gyp@9.0.0
gyp info using node@16.16.0 | darwin | arm64
gyp info find Python using Python version 3.8.9 found at "/Library/Developer/CommandLineTools/usr/bin/python3"
gyp http GET https://nodejs.org/download/release/v16.16.0/node-v16.16.0-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v16.16.0/node-v16.16.0-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v16.16.0/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v16.16.0/SHASUMS256.txt
gyp info spawn /Library/Developer/CommandLineTools/usr/bin/python3
gyp info spawn args [
gyp info spawn args   '/nix/store/c0hvn0hnxppdrvk3iamz3cfk50jhvl8y-nodejs-16.16.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/mtojek/code/coder/site/node_modules/canvas/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/nix/store/c0hvn0hnxppdrvk3iamz3cfk50jhvl8y-nodejs-16.16.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/mtojek/Library/Caches/node-gyp/16.16.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/mtojek/Library/Caches/node-gyp/16.16.0',
gyp info spawn args   '-Dnode_gyp_dir=/nix/store/c0hvn0hnxppdrvk3iamz3cfk50jhvl8y-nodejs-16.16.0/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/mtojek/Library/Caches/node-gyp/16.16.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/mtojek/code/coder/site/node_modules/canvas',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
/bin/sh: pkg-config: command not found
gyp: Call to 'pkg-config pixman-1 --libs' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/nix/store/c0hvn0hnxppdrvk3iamz3cfk50jhvl8y-nodejs-16.16.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:261:16)
gyp ERR! stack     at ChildProcess.emit (node:events:527:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! System Darwin 21.4.0
gyp ERR! command "/nix/store/c0hvn0hnxppdrvk3iamz3cfk50jhvl8y-nodejs-16.16.0/bin/node" "/nix/store/c0hvn0hnxppdrvk3iamz3cfk50jhvl8y-nodejs-16.16.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--update-binary" "--module=/Users/mtojek/code/coder/site/node_modules/canvas/build/Release/canvas.node" "--module_name=canvas" "--module_path=/Users/mtojek/code/coder/site/node_modules/canvas/build/Release" "--napi_version=8" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v93"
gyp ERR! cwd /Users/mtojek/code/coder/site/node_modules/canvas
gyp ERR! node -v v16.16.0
gyp ERR! node-gyp -v v9.0.0
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute '/nix/store/c0hvn0hnxppdrvk3iamz3cfk50jhvl8y-nodejs-16.16.0/bin/node /nix/store/c0hvn0hnxppdrvk3iamz3cfk50jhvl8y-nodejs-16.16.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --update-binary --module=/Users/mtojek/code/coder/site/node_modules/canvas/build/Release/canvas.node --module_name=canvas --module_path=/Users/mtojek/code/coder/site/node_modules/canvas/build/Release --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v93' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/mtojek/code/coder/site/node_modules/@mapbox/node-pre-gyp/lib/util/compile.js:89:23)
node-pre-gyp ERR! stack     at ChildProcess.emit (node:events:527:28)
node-pre-gyp ERR! stack     at maybeClose (node:internal/child_process:1092:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5)
node-pre-gyp ERR! System Darwin 21.4.0
node-pre-gyp ERR! command "/nix/store/c0hvn0hnxppdrvk3iamz3cfk50jhvl8y-nodejs-16.16.0/bin/node" "/Users/mtojek/code/coder/site/node_modules/canvas/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build" "--update-binary"
node-pre-gyp ERR! cwd /Users/mtojek/code/coder/site/node_modules/canvas
node-pre-gyp ERR! node -v v16.16.0
node-pre-gyp ERR! node-pre-gyp -v v1.0.10
node-pre-gyp ERR! not ok

I encountered a few similar issues which I solved with:

brew install pkgconfig
brew install pango
brew install cairo
brew install pandocomatic
brew install pangocairo

According to the CONTRIBUTING page, these tools should be installed while bootstrapping the dev environment with nix-shell? Manual steps are advised only in case a user doesn't want to install the nix-shell.
@ghuntley ghuntley self-assigned this Nov 4, 2022
@ghuntley ghuntley added the bug label Nov 11, 2022
@bpmct bpmct added the s3 Bugs that confuse, annoy, or are purely cosmetic label Feb 9, 2023
@github-actions
Copy link

This issue is becoming stale. In order to keep the tracker readable and actionable, I'm going close to this issue in 7 days if there isn't more activity.

@github-actions github-actions bot added the stale This issue is like stale bread. label May 11, 2023
@matifali
Copy link
Member

Hi @mtojek, if you can list the extra steps needed, I can close this by updating the Contributing docs.

@mtojek
Copy link
Member Author

mtojek commented May 15, 2023

Extra steps were only the ones in the issue description (brew install ...). AFAIR nothing else was required.

@matifali
Copy link
Member

ok. I will test and send a PR. Thanks :)

@matifali matifali assigned matifali and unassigned matifali May 15, 2023
@github-actions github-actions bot removed the stale This issue is like stale bread. label May 16, 2023
This was referenced May 25, 2023
Merged
Closed
@matifali
Copy link
Member

matifali commented Jun 27, 2023 •
edited
Loading

@mtojek Should we close this as completed? I just tested nix-shell, and it works fine.
Edit: I am not able to run ./scripts/develop.sh successfully.

Error 
+ yarn install
yarn install v1.22.19
[1/4] Resolving packages...
success Already up-to-date.
$ yarn typegen
yarn run v1.22.19
$ xstate typegen 'src/**/*.ts'
It is highly recommended to set `predictableActionArguments` to `true` when using `createMachine`. https://xstate.js.org/docs/guides/actions.html
Warning: Compound state node 'templateVersion.done' has no initial state.
src/xServices/users/usersXService.ts - success
src/xServices/templateVersionEditor/templateVersionEditorXService.ts - success
src/xServices/groups/createGroupXService.ts - success
src/xServices/template/searchUsersAndGroupsXService.ts - success
src/xServices/starterTemplates/starterTemplateXService.ts - success
src/xServices/auth/authXService.ts - success
src/xServices/groups/groupsXService.ts - success
src/xServices/template/templateVariablesXService.ts - success
src/xServices/groups/editGroupXService.ts - success
src/xServices/deploymentStats/deploymentStatsMachine.ts - success
src/xServices/workspaceSchedule/workspaceScheduleBannerXService.ts - success
src/xServices/deploymentConfig/deploymentConfigMachine.ts - success
src/xServices/template/templateACLXService.ts - success
src/xServices/auth/authMethodsXService.ts - success
src/xServices/terminal/terminalXService.ts - success
src/xServices/userSecuritySettings/userSecuritySettingsXService.ts - success
src/xServices/updateCheck/updateCheckXService.ts - success
src/xServices/groups/groupXService.ts - success
src/xServices/entitlements/entitlementsXService.ts - success
src/xServices/setup/setupXService.ts - success
src/xServices/starterTemplates/starterTemplatesXService.ts - success
src/xServices/experiments/experimentsMachine.ts - success
src/xServices/templateVersion/templateVersionXService.ts - success
src/xServices/users/createUserXService.ts - success
src/xServices/workspaceBuild/workspaceBuildXService.ts - success
src/xServices/workspaceSchedule/workspaceScheduleXService.ts - success
src/xServices/sshKey/sshKeyXService.ts - success
src/xServices/roles/siteRolesXService.ts - success
src/xServices/createTemplate/createTemplateXService.ts - success
src/xServices/buildInfo/buildInfoXService.ts - success
src/xServices/workspaceAgentLogs/workspaceAgentLogsXService.ts - success
src/xServices/appearance/appearanceXService.ts - success
src/xServices/portForward/portForwardXService.ts - success
src/xServices/pagination/paginationXService.ts - success
src/xServices/users/searchUserXService.ts - success
src/xServices/templates/templatesXService.ts - success
src/xServices/workspace/workspaceXService.ts - success
src/xServices/quotas/quotasXService.ts - success
src/xServices/createWorkspace/createWorkspaceXService.ts - success
Done in 7.06s.
Done in 7.49s.
yarn run v1.22.19
$ NODE_ENV=production yarn vite build
$ /mnt/c/Users/matif/github/coder/coder/site/node_modules/.bin/vite build
vite v4.3.9 building for production...
transforming (1) index.html
node:internal/event_target:1012
  process.nextTick(() => { throw err; });
                           ^
Error: ENOENT: no such file or directory, uv_cwd
    at process.cwd (node:internal/main/worker_thread:123:21)
    at MessagePort.<anonymous> (node:internal/main/worker_thread:138:50)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:737:20)
    at exports.emitMessage (node:internal/per_context/messageport:23:28)
Emitted 'error' event on Worker instance at:
    at [kOnErrorMessage] (node:internal/worker:300:10)
    at [kOnMessage] (node:internal/worker:311:37)
    at MessagePort.<anonymous> (node:internal/worker:212:57)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:737:20)
    at exports.emitMessage (node:internal/per_context/messageport:23:28)
    at [kOnExit] (node:internal/worker:278:5)
    at Worker.<computed>.onexit (node:internal/worker:209:20) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'uv_cwd'
}

Node.js v18.16.1
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
make: *** [Makefile:359: site/out/index.html] Error 1
make: *** Waiting for unfinished jobs....

Edit 2: after running make clean and then ./scripts/develop.sh seems to fail differently

Details 
generate
+ yarn install
yarn install v1.22.19
[1/4] Resolving packages...
success Already up-to-date.
$ yarn typegen
yarn run v1.22.19
$ xstate typegen 'src/**/*.ts'
go: downloading golang.org/x/tools v0.10.0
It is highly recommended to set `predictableActionArguments` to `true` when using `createMachine`. https://xstate.js.org/docs/guides/actions.html
Warning: Compound state node 'templateVersion.done' has no initial state.
src/xServices/groups/groupsXService.ts - success
src/xServices/workspaceAgentLogs/workspaceAgentLogsXService.ts - success
src/xServices/users/createUserXService.ts - success
src/xServices/template/templateVariablesXService.ts - success
src/xServices/users/searchUserXService.ts - success
src/xServices/groups/createGroupXService.ts - success
src/xServices/users/usersXService.ts - success
src/xServices/template/searchUsersAndGroupsXService.ts - success
src/xServices/buildInfo/buildInfoXService.ts - success
src/xServices/groups/editGroupXService.ts - success
src/xServices/template/templateACLXService.ts - success
src/xServices/groups/groupXService.ts - success
src/xServices/templateVersionEditor/templateVersionEditorXService.ts - success
src/xServices/deploymentConfig/deploymentConfigMachine.ts - success
src/xServices/userSecuritySettings/userSecuritySettingsXService.ts - success
src/xServices/entitlements/entitlementsXService.ts - success
src/xServices/portForward/portForwardXService.ts - success
src/xServices/sshKey/sshKeyXService.ts - success
src/xServices/updateCheck/updateCheckXService.ts - success
src/xServices/templateVersion/templateVersionXService.ts - success
src/xServices/createWorkspace/createWorkspaceXService.ts - success
src/xServices/appearance/appearanceXService.ts - success
src/xServices/createTemplate/createTemplateXService.ts - success
src/xServices/setup/setupXService.ts - success
src/xServices/workspace/workspaceXService.ts - success
src/xServices/templates/templatesXService.ts - success
src/xServices/starterTemplates/starterTemplateXService.ts - success
src/xServices/auth/authXService.ts - success
src/xServices/workspaceSchedule/workspaceScheduleXService.ts - success
src/xServices/terminal/terminalXService.ts - success
src/xServices/auth/authMethodsXService.ts - success
src/xServices/workspaceSchedule/workspaceScheduleBannerXService.ts - success
src/xServices/workspaceBuild/workspaceBuildXService.ts - success
src/xServices/roles/siteRolesXService.ts - success
src/xServices/experiments/experimentsMachine.ts - success
src/xServices/deploymentStats/deploymentStatsMachine.ts - success
src/xServices/quotas/quotasXService.ts - success
src/xServices/pagination/paginationXService.ts - success
src/xServices/starterTemplates/starterTemplatesXService.ts - success
Done in 9.81s.
Done in 10.21s.
yarn run v1.22.19
$ NODE_ENV=production yarn vite build
$ /mnt/c/Users/matif/github/coder/coder/site/node_modules/.bin/vite build
vite v4.3.9 building for production...
transforming (78) node_modules/@mui/material/styles/withStyles.js/mnt/c/Users/matif/github/coder/coder/build/coder-slim_0.24.1-devel+70cd87140_linux_amd64
transforming (87) src/components/RuntimeErrorState/RuntimeErrorState.tsx/mnt/c/Users/matif/github/coder/coder/build/coder-slim_0.24.1-devel+70cd87140_windows_amd64.exe
/mnt/c/Users/matif/github/coder/coder/build/coder-slim_0.24.1-devel+70cd87140_darwin_amd64
transforming (99) vite/preload-helper/mnt/c/Users/matif/github/coder/coder/build/coder-slim_0.24.1-devel+70cd87140_linux_armv7
/mnt/c/Users/matif/github/coder/coder/build/coder-slim_0.24.1-devel+70cd87140_linux_arm64
/mnt/c/Users/matif/github/coder/coder/build/coder-slim_0.24.1-devel+70cd87140_darwin_arm64
transforming (187) node_modules/@mui/utils/esm/formatMuiErrorMessage.js/mnt/c/Users/matif/github/coder/coder/build/coder-slim_0.24.1-devel+70cd87140_windows_arm64.exe
transforming (294) src/components/SettingsAccountForm/SettingsAccountForm.tsx/mnt/c/Users/matif/github/coder/coder/site/out/bin /mnt/c/Users/matif/github/coder/coder
transforming (424) src/pages/AuditPage/AuditPageView.tsx78f8b71d897eb215c846b0adff70bb9c338d02c4 *coder-darwin-amd64
d64f832a7b465bfad4a90a28f02d8ca41ffd16b5 *coder-darwin-arm64
d4a1fcaddf157f3da8d7ee18b70cfc7397ad7b74 *coder-linux-amd64
f93aba936cb83f0e9d17dbfb6064993fab4e932d *coder-linux-arm64
433aa2172a37295ab4b5f043a74b75ffc185c771 *coder-linux-armv7
a780ca65fab06f3d5a668011b5d4854b6c17e92e *coder-windows-amd64.exe
74141141f6f252e9177602899686bba98c1d0c1a *coder-windows-arm64.exe
/mnt/c/Users/matif/github/coder/coder
/mnt/c/Users/matif/github/coder/coder/site/out/bin /mnt/c/Users/matif/github/coder/coder
transforming (730) node_modules/react-markdown/index.js/mnt/c/Users/matif/github/coder/coder
transforming (812) node_modules/@tanstack/query-core/build/lib/removable.mjsbuild/coder-slim_0.24.1-devel+70cd87140.tar : 24.10%   (   407 MiB =>   98.1 MiB, build/coder-slim_0.24.1-devel+70cd87140.tar.zst)
✓ 5158 modules transformed.
rendering chunks (10)...
<--- Last few GCs --->

[8581:0x4a68160]   206626 ms: Scavenge 2016.0 (2073.1) -> 2015.7 (2083.9) MB, 4.3 / 0.0 ms  (average mu = 0.691, current mu = 0.165) allocation failure;
[8581:0x4a68160]   206632 ms: Scavenge 2022.5 (2083.9) -> 2022.6 (2084.1) MB, 4.6 / 0.0 ms  (average mu = 0.691, current mu = 0.165) allocation failure;
[8581:0x4a68160]   206641 ms: Scavenge 2023.4 (2084.1) -> 2022.6 (2106.6) MB, 8.8 / 0.0 ms  (average mu = 0.691, current mu = 0.165) allocation failure;


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xaabe68 node::Abort() [/nix/store/p0f8i04zwf1dd66n2qkazk5x0fbsy7mp-nodejs-18.16.1/bin/node]
 2: 0x9a0821  [/nix/store/p0f8i04zwf1dd66n2qkazk5x0fbsy7mp-nodejs-18.16.1/bin/node]
 3: 0xcb8aa0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/nix/store/p0f8i04zwf1dd66n2qkazk5x0fbsy7mp-nodejs-18.16.1/bin/node]
 4: 0xcb8e6b v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/nix/store/p0f8i04zwf1dd66n2qkazk5x0fbsy7mp-nodejs-18.16.1/bin/node]
 5: 0xe9dba5  [/nix/store/p0f8i04zwf1dd66n2qkazk5x0fbsy7mp-nodejs-18.16.1/bin/node]
 6: 0xeb1fd9 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/nix/store/p0f8i04zwf1dd66n2qkazk5x0fbsy7mp-nodejs-18.16.1/bin/node]
 7: 0xe8e77a v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/nix/store/p0f8i04zwf1dd66n2qkazk5x0fbsy7mp-nodejs-18.16.1/bin/node]
 8: 0xe8fb24 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/nix/store/p0f8i04zwf1dd66n2qkazk5x0fbsy7mp-nodejs-18.16.1/bin/node]
 9: 0xe7022d v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/nix/store/p0f8i04zwf1dd66n2qkazk5x0fbsy7mp-nodejs-18.16.1/bin/node]
10: 0x12609fb v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/nix/store/p0f8i04zwf1dd66n2qkazk5x0fbsy7mp-nodejs-18.16.1/bin/node]
11: 0x16aa039  [/nix/store/p0f8i04zwf1dd66n2qkazk5x0fbsy7mp-nodejs-18.16.1/bin/node]
Aborted
error Command failed with exit code 134.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 134.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
make: *** [Makefile:359: site/out/index.html] Error 134

Edit 3: Setting export NODE_OPTIONS="--max-old-space-size=4096" fixed the issue, and I successfully built the coder.

@matifali matifali self-assigned this Jun 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@matifali matifali

Labels
s3 Bugs that confuse, annoy, or are purely cosmetic
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@ghuntley @ammario @matifali @mtojek @bpmct