Install Script stalls indefinitely, shows logs after terminating process #103
Comments
|
At a glance, it seems like you're using a non-M1 mac, is that correct? If so, we don't provide pre-built packages for that (for macOS, we only provide it for M1), so in this case, npm install is trying to build librdkafka from scratch, which is not only time consuming, but a bunch of dependencies are required for that. I can suggest doing this:
Let me know if that helps, or if you're getting issues. |
|
That worked! Thank you very much. |
|
Im still getting an error when i do npm install. I followed the steps but i get this error now. npm verbose stack Error: command failed
npm verbose stack at promiseSpawn (/Users/davidqiu/.volta/tools/image/npm/10.8.1/node_modules/@npmcli/promise-spawn/lib/index.js:22:22)
npm verbose stack at spawnWithShell (/Users/davidqiu/.volta/tools/image/npm/10.8.1/node_modules/@npmcli/promise-spawn/lib/index.js:124:10)
npm verbose stack at promiseSpawn (/Users/davidqiu/.volta/tools/image/npm/10.8.1/node_modules/@npmcli/promise-spawn/lib/index.js:12:12)
npm verbose stack at runScriptPkg (/Users/davidqiu/.volta/tools/image/npm/10.8.1/node_modules/@npmcli/run-script/lib/run-script-pkg.js:77:13)
npm verbose stack at runScript (/Users/davidqiu/.volta/tools/image/npm/10.8.1/node_modules/@npmcli/run-script/lib/run-script.js:9:12)
npm verbose stack at /Users/davidqiu/.volta/tools/image/npm/10.8.1/node_modules/@npmcli/arborist/lib/arborist/rebuild.js:329:17
npm verbose stack at run (/Users/davidqiu/.volta/tools/image/npm/10.8.1/node_modules/promise-call-limit/dist/commonjs/index.js:67:22)
npm verbose stack at /Users/davidqiu/.volta/tools/image/npm/10.8.1/node_modules/promise-call-limit/dist/commonjs/index.js:84:9
npm verbose stack at new Promise (<anonymous>)
npm verbose stack at callLimit (/Users/davidqiu/.volta/tools/image/npm/10.8.1/node_modules/promise-call-limit/dist/commonjs/index.js:35:69)
npm verbose pkgid @confluentinc/kafka-javascript@0.3.0
npm error code 1.../@confluentinc/kafka-javascript install$ node-pre-gyp install --fallback-to-build
│ node-pre-gyp info it worked if it ends with ok
│ node-pre-gyp info using node-pre-gyp@1.0.11
│ node-pre-gyp info using node@20.16.0 | darwin | x64
│ node-pre-gyp info check checked for "/node_modules/.pnpm/@confluentinc+kafka-javascript@0.2.1/node_modules/@co…
│ node-pre-gyp http GET https://github.com/confluentinc/confluent-kafka-javascript/releases/download/v0.2.1/confluent-kafka-javascript-v0.2.1-node-v115-darwin-unknown-x64.tar.gz
│ node-pre-gyp ERR! install response status 404 Not Found on https://github.com/confluentinc/confluent-kafka-javascript/releases/download/v0.2.1/confluent-kafka-javascript-v0.2.1-…
│ node-pre-gyp WARN Pre-built binaries not installable for @confluentinc/kafka-javascript@v0.2.1 and node@20.16.0 (node-v115 ABI, unknown) (falling back to source compile with nod…
│ node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/confluentinc/confluent-kafka-javascript/releases/download/v0.2.1/confluent-kafka-javascript-v0.2.…
│ gyp info it worked if it ends with ok
│ gyp info using node-gyp@10.2.0
│ gyp info using node@20.16.0 | darwin | x64
│ gyp info ok
│ gyp info it worked if it ends with ok
│ gyp info using node-gyp@10.2.0
│ gyp info using node@20.16.0 | darwin | x64
│ gyp info find Python using Python version 3.11.3 found at "/usr/local/opt/python@3.11/bin/python3.11"
│ gyp info spawn /usr/local/opt/python@3.11/bin/python3.11
│ gyp info spawn args [
│ gyp info spawn args '/Users/davidqiu/.volta/tools/image/packages/pnpm/lib/node_modules/pnpm/dist/node_modules/node-gyp/gyp/gyp_main.py',
│ gyp info spawn args 'binding.gyp',
│ gyp info spawn args '-f',
│ gyp info spawn args 'make',
│ gyp info spawn args '-I',
│ gyp info spawn args '/node_modules/.pnpm/@confluentinc+kafka-javascript@0.2.1/node_modules/@confluentinc/kafka…
│ gyp info spawn args '-I',
│ gyp info spawn args '/Users/davidqiu/.volta/tools/image/packages/pnpm/lib/node_modules/pnpm/dist/node_modules/node-gyp/addon.gypi',
│ gyp info spawn args '-I',
│ gyp info spawn args '/Users/davidqiu/Library/Caches/node-gyp/20.16.0/include/node/common.gypi',
│ gyp info spawn args '-Dlibrary=shared_library',
│ gyp info spawn args '-Dvisibility=default',
│ gyp info spawn args '-Dnode_root_dir=/Users/davidqiu/Library/Caches/node-gyp/20.16.0',
│ gyp info spawn args '-Dnode_gyp_dir=/Users/davidqiu/.volta/tools/image/packages/pnpm/lib/node_modules/pnpm/dist/node_modules/node-gyp',
│ gyp info spawn args '-Dnode_lib_file=/Users/davidqiu/Library/Caches/node-gyp/20.16.0/<(target_arch)/node.lib',
│ gyp info spawn args '-Dmodule_root_dir=/node_modules/.pnpm/@confluentinc+kafka-javascript@0.2.1/node_modules/@…
│ gyp info spawn args '-Dnode_engine=v8',
│ gyp info spawn args '--depth=.',
│ gyp info spawn args '--no-parallel',
│ gyp info spawn args '--generator-output',
│ gyp info spawn args 'build',
│ gyp info spawn args '-Goutput_dir=.'
│ gyp info spawn args ]
│ gyp info ok
│ gyp info it worked if it ends with ok
│ gyp info using node-gyp@10.2.0
│ gyp info using node@20.16.0 | darwin | x64
│ gyp info spawn make
│ gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
│ CXX(target) Release/obj.target/confluent-kafka-javascript/src/binding.o
│ In file included from ../src/binding.cc:13:
│ /node_modules/.pnpm/@confluentinc+kafka-javascript@0.2.1/node_modules/@confluentinc/kafka-javascript/src/bindi…
│ #include "rdkafkacpp.h" // NOLINT
│ ^~~~~~~~~~~~~~
│ 1 error generated.
│ make: *** [Release/obj.target/confluent-kafka-javascript/src/binding.o] Error 1
│ gyp ERR! build error
│ gyp ERR! stack Error: `make` failed with exit code: 2
│ gyp ERR! stack at ChildProcess.<anonymous> (/Users/davidqiu/.volta/tools/image/packages/pnpm/lib/node_modules/pnpm/dist/node_modules/node-gyp/lib/build.js:216:23)
│ gyp ERR! System Darwin 22.6.0
│ gyp ERR! command "/Users/davidqiu/.volta/tools/image/node/20.16.0/bin/node" "/Users/davidqiu/.volta/tools/image/packages/pnpm/lib/node_modules/pnpm/dist/node_modules/node-gyp/bi…
│ gyp ERR! cwd /node_modules/.pnpm/@confluentinc+kafka-javascript@0.2.1/node_modules/@confluentinc/kafka-javascr…
│ gyp ERR! node -v v20.16.0
│ gyp ERR! node-gyp -v v10.2.0
│ gyp ERR! not ok
│ node-pre-gyp ERR! build error
│ node-pre-gyp ERR! stack Error: Failed to execute '/Users/davidqiu/.volta/tools/image/node/20.16.0/bin/node /Users/davidqiu/.volta/tools/image/packages/pnpm/lib/node_modules/pnpm…
│ node-pre-gyp ERR! stack at ChildProcess.<anonymous> (/node_modules/.pnpm/@mapbox+node-pre-gyp@1.0.11/node_…
│ node-pre-gyp ERR! stack at ChildProcess.emit (node:events:519:28)
│ node-pre-gyp ERR! stack at maybeClose (node:internal/child_process:1105:16)
│ node-pre-gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:305:5)
│ node-pre-gyp ERR! System Darwin 22.6.0
│ node-pre-gyp ERR! command "/Users/davidqiu/.volta/tools/image/node/20.16.0/bin/node" "/node_modules/.pnpm/@map…
│ node-pre-gyp ERR! cwd /node_modules/.pnpm/@confluentinc+kafka-javascript@0.2.1/node_modules/@confluentinc/kafk…
│ node-pre-gyp ERR! node -v v20.16.0
│ node-pre-gyp ERR! node-pre-gyp -v v1.0.11
│ node-pre-gyp ERR! not ok
│ Failed to execute '/Users/davidqiu/.volta/tools/image/node/20.16.0/bin/node /Users/davidqiu/.volta/tools/image/packages/pnpm/lib/node_modules/pnpm/dist/node_modules/node-gyp/bin…
└─ Failed in 3.5s at /node_modules/.pnpm/@confluentinc+kafka-javascript@0.2.1/node_modules/@confluentinc/kafka-javascript |
@milindl any ideas on the reason why its failing now. I followed your steps and set the env variables too but im now getting the above error. |
|
At a very basic glance it seems like it can't find the header files for librdkafka. Could you try doing the following steps to verify this? Make a program in any directory, check.c #include <stdio.h>
#include <string.h>
#include <librdkafka/rdkafka.h>
int main() {
rd_kafka_conf_t *conf;
char buf[512];
size_t sz = sizeof(buf);
rd_kafka_conf_res_t res;
printf("Version string: %s\n", rd_kafka_version_str());
conf = rd_kafka_conf_new();
res = rd_kafka_conf_get(conf, "builtin.features", buf, &sz);
if (res != RD_KAFKA_CONF_OK) {
printf("ERROR: conf_get failed: %d\n", res);
return 1;
}
printf("builtin.features: %s\n", buf);
return 0;
}And try compiling and running the program (you're using mac too right? I think clang should be present) Does that work without any further issues? |
|
@milindl thanks for helping to debug the apple silicon hiccups with running rdkafka. I've installed The problem is that the
Theoretically, moving those into the ENVs CKJS_LINKING=dynamic BUILD_LIBRDKAFKA=0 LIBRARY_PATH=$(brew --prefix librdkafka)/lib CPATH=$(brew --prefix librdkafka)/include pnpm add '@confluentinc/kafka-javascript'☝️ but that still fails for me with the same error (not finding the librdkafka headers). EDIT: Also I just realized that it's complaining about Which made me realize that this build expects you include the specific librdkafka dir - not just the general include path. So BUILD_LIBRDKAFKA=0 LIBRARY_PATH=$(brew --prefix)/lib CPATH=$(brew --prefix)/include/librdkafka pnpm add '@confluentinc/kafka-javascript'Was what it was after. 🚀 |
|
Thanks for documenting your process @wspurgin , that's very informative for other users. What node version and library version are you using? With the latest one, we've added pre-built binaries for both Intel and Apple Silicon Macs, so ideally you should be able to avoid doing the build entirely. |
|
@milindl - here were my specs
|
So i did this to get it installed. |
|
Hey @dqiubread , thanks for documenting your steps too. Did you not need to set CKJS_LINKING=dynamic as well? |
I did not need to set CKJS_LINKING. My issue was on an intel mac |
|
closing this as the questions seem to be resolved, another ticket can be opened for future questions |
Environment Information
Steps to Reproduce
package.json.pnpm installin theconfluent-kafka-javascript Configuration Settings
Only added to package.json.
Additional context
We are using PNPM and Turbo for our monorepo. When I run
pnpm installin the workspace root, it stalls indefinitely on "Running install script.Upon Ctrl+C to terminate the install, it shows these logs all at once:
Logs
It appears to fail on
building dependency libcrypto...when i hitdeps/librdkafka/config.hThe text was updated successfully, but these errors were encountered: