When running under Meteor: archived threads in combination with wasm not supported

[sethwklein]

We're hoping to use BLAKE3 for content hashing due to its awesome speed. The server is in Meteor and we're getting an error that appears to be very low level.

Reproduction:

# install meteor
# I wrote this reproduction using 1.9, but I believe my colleague has tried
# beta versions.
nvm use 12
meteor create --react simple-todos-2
cd simple-todos-2
meteor npm install blake3
meteor
^C
# apologies for being old school. ed isn't usable, but it is scriptable!
ed server/main.js
%p
/function
-1
i
var blake3 = require('blake3');
.
%p
wq
meteor
# observe crash:
W20200303-14:22:16.239(-5)? (STDERR)
W20200303-14:22:16.288(-5)? (STDERR)
W20200303-14:22:16.288(-5)? (STDERR) #
W20200303-14:22:16.288(-5)? (STDERR) # Fatal error in , line 0
W20200303-14:22:16.289(-5)? (STDERR) # archived threads in combination with wasm not supported
W20200303-14:22:16.289(-5)? (STDERR) #
W20200303-14:22:16.289(-5)? (STDERR) #
W20200303-14:22:16.290(-5)? (STDERR) #
W20200303-14:22:16.290(-5)? (STDERR) #FailureMessage Object: 0x102e79240

This issuse seems to be related: nodejs/node#29767 .

It suggests --no-wasm-code-gc, but that is already in use:

ps aux | grep no-wasm-code-gc
sk               23007  21.5  3.4 17398548 281708 s011  S+    2:27PM   0:15.17 /Users/sk/.meteor/packages/meteor-tool/.1.9.0.z44zal++os.osx.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.osx.x86_64/dev_bundle/bin/node --max-old-space-size=4096 --no-wasm-code-gc /Users/sk/.meteor/packages/meteor-tool/.1.9.0.z44zal++os.osx.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.osx.x86_64/tools/index.js

The error message appears to come from here: https://chromium.googlesource.com/v8/v8/+/master/src/wasm/wasm-engine.cc#106

which contains this comment:

Archived threads are rarely used, and not combined with Wasm at the
moment. Implement this and test it properly once we have a use case for
that.

[connor4312]

That is... interesting. I'm not familiar with archived threads, a quick google indicates they're an internal V8 structure. Do you have suggestions for a fix here?

[sethwklein]

I wonder if it's something the Rust to JavaScript compilation is using. I'm also curious why the server side is using WASM at all. There's a ton I don't know here, but that's a couple things that stand out.

[connor4312]

We use WASM from Node.js if we can't get valid bindings. This is the install script. I don't think this problem is related to Rust->WASM, it seems to be more in how the wasm is run than anything in the wasm bytecode itself. I'm also not familiar at all with meteor so. I'll try to see if I can figure something out this weekend.

[sethwklein]

So if I'm understanding correctly, it's likely that Meteor is messing up the blake3 install so it doesn't fetch the appropriate bindings, resulting in use of a WASM version. I suppose the WASM version should work and so there's an issue there, but my assumption is that native is faster than WASM, so I'm mostly interested in getting the native bindings used.

[sethwklein]

FWIW, I expect to be looking into how Meteor might be messing up the install on Monday.

[connor4312]

It looks like the missing native support was because the latest version was missing a file. I've updated the publish script to make sure to generate it each time. It will be fixed in 2.1.3.

[sethwklein]

With the new version, I'm still seeing the same error when running under Meteor. The really odd thing is that the log shows native bindings being retrieved:

--- sk@silver:~/test-blake3-meteor-2/simple-todos-2 $
meteor npm install --loglevel verbose blake3
npm info it worked if it ends with ok         
npm verb cli [
npm verb cli   '/Users/sk/.meteor/packages/meteor-tool/.1.9.0.z44zal++os.osx.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.osx.x86_64/dev_bundle/bin/node',
npm verb cli   '/Users/sk/.meteor/packages/meteor-tool/1.9.0/mt-os.osx.x86_64/dev_bundle/bin/npm',
npm verb cli   'install',
npm verb cli   '--loglevel',
npm verb cli   'verbose',
npm verb cli   'blake3'
npm verb cli ]
npm info using npm@6.13.4
npm info using node@v12.14.0
npm verb npm-session b2bada0285d832f4
npm http fetch GET 304 https://registry.npmjs.org/blake3 619ms (from cache)
npm timing stage:loadCurrentTree Completed in 860ms
npm timing stage:loadIdealTree:cloneCurrentTree Completed in 2ms
npm timing stage:loadIdealTree:loadShrinkwrap Completed in 111ms
npm timing stage:loadIdealTree:loadAllDepsIntoIdealTree Completed in 125ms
npm timing stage:loadIdealTree Completed in 282ms
npm timing stage:generateActionsToTake Completed in 16ms
npm verb correctMkdir /Users/sk/.npm/_locks correctMkdir not in flight; initializing
npm verb lock using /Users/sk/.npm/_locks/staging-6c3c1fa12c69a1a7.lock for /Users/sk/test-blake3-meteor-2/simple-todos-2/node_modules/.staging
npm timing audit submit Completed in 262ms
npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/audits/quick 263ms
npm timing audit body Completed in 1ms
npm timing action:extract Completed in 352ms
npm timing action:finalize Completed in 5ms
npm timing action:refresh-package-json Completed in 7ms
npm info lifecycle blake3@2.1.3~preinstall: blake3@2.1.3
npm timing action:preinstall Completed in 3ms
npm info linkStuff blake3@2.1.3
npm timing action:build Completed in 2ms
npm info lifecycle blake3@2.1.3~install: blake3@2.1.3

> blake3@2.1.3 install /Users/sk/test-blake3-meteor-2/simple-todos-2/node_modules/blake3
> node -e "try{require('./dist/build/install')}catch(e){}"

Retrieving native BLAKE3 bindings for Node v12 on darwin...
BLAKE3 bindings retrieved
npm verb lifecycle blake3@2.1.3~install: unsafe-perm in lifecycle true
npm verb lifecycle blake3@2.1.3~install: PATH: /Users/sk/.meteor/packages/meteor-tool/.1.9.0.z44zal++os.osx.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.osx.x86_64/dev_bundle/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/Users/sk/test-blake3-meteor-2/simple-todos-2/node_modules/blake3/node_modules/.bin:/Users/sk/test-blake3-meteor-2/simple-todos-2/node_modules/.bin:/Users/sk/.meteor/packages/meteor-tool/1.9.0/mt-os.osx.x86_64/dev_bundle/bin:/Users/sk/.meteor/packages/meteor-tool/1.9.0/mt-os.osx.x86_64:/Users/sk/.meteor/packages/meteor-tool/1.9.0/mt-os.osx.x86_64/dev_bundle/lib/node_modules/.bin:/Users/sk/.nvm/versions/node/v12.14.0/bin:/Users/sk/.cargo/bin:/Users/sk/.rbenv/shims:/usr/local/heroku/bin:/Users/sk/opt/vim7/bin:/Users/sk/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/share/dotnet:~/.dotnet/tools:/usr/local/go/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/Xamarin Workbooks.app/Contents/SharedSupport/path-bin:/Users/sk/go/bin:/Users/sk/cache/go/bin:/Users/sk/.cargo/bin:/usr/local/ant/bin:/Users/sk/Library/Android/sdk/platform-tools:/Users/sk/Library/Android/sdk/tools:/Users/sk/Library/Android/sdk/tools/bin
npm verb lifecycle blake3@2.1.3~install: CWD: /Users/sk/test-blake3-meteor-2/simple-todos-2/node_modules/blake3
npm timing action:install Completed in 1106ms
npm info lifecycle blake3@2.1.3~postinstall: blake3@2.1.3
npm timing action:postinstall Completed in 1ms
npm verb unlock done using /Users/sk/.npm/_locks/staging-6c3c1fa12c69a1a7.lock for /Users/sk/test-blake3-meteor-2/simple-todos-2/node_modules/.staging
npm timing stage:executeActions Completed in 1506ms
npm timing stage:rollbackFailedOptional Completed in 0ms
npm timing stage:runTopLevelLifecycles Completed in 2719ms
npm verb saving [ { name: 'blake3', spec: '^2.1.3', save: 'dependencies' } ]
npm info lifecycle undefined~preshrinkwrap: undefined
npm info lifecycle undefined~shrinkwrap: undefined
npm info lifecycle undefined~postshrinkwrap: undefined
+ blake3@2.1.3
added 1 package from 1 contributor and audited 639 packages in 2.822s
found 0 vulnerabilities

npm verb exit [ 0, true ]
npm timing npm Completed in 3216ms
npm info ok 
--- sk@silver:~/test-blake3-meteor-2/simple-todos-2 $

The error is the same.

Is there some way for the fallback to be triggered at runtime even if it wasn't triggered at install time?

[connor4312]

We do point the browser and module entrypoints in the package.json to the webassembly version. Meteor may be picking up one of those.

Sorry I haven't looked into it more yet. Busy the last week moving to a new place. Should have more time this coming weekend.

[sethwklein]

With all the best humor, I'll consider your apology appropriate and accept it only when I'm paying you :D

I probably need to spend the afternoon pushing other things forward, but might look into it later in the week. Otherwise it'll happen when you get time :)

[sethwklein]

Just FYI, we're having a contractor look into this. He's not sure he has the skills for it, but he's going to look into it and see if it's something he can get done in a reasonable number of hours.

[wvanooijen92]

Hi, I am the contractor. I have looked into the issue for a bit and found this error is thrown as soon as it uses WASM related code.

I am not sure it is the reference to WASM in the browser or module entry points. When I change export * from './node'; to export * from './node-native'; in index.ts:16 (main entry), the first load executes without problems, then after a few seconds the same error appears. I guess this is the fallback
@connor4312 mentions, but I could not confirm (lots of new stuff to learn). What do you think?

In the past thread related issues in Meteor have proven to be tedious. Though, for our project, we would really like to use your awesome package. Do you think it would be possible to publish the native code only (and disable WASM completely, import { blake } from 'blake3/native';)?

I wonder if this is related to a collision of node-fibers and wasm. There may be a hint in this comment if anyone is interested in solving it.

Meanwhile I will discuss options internally.

[sethwklein]

We ended up moving the hashing into a microservice since this problem isn't seen under plain Node. And then I reimplemented that in Go since it's easier to do all the non-hashing parts in Go than in JavaScript, but that's another story.