npx crashes on postinstall script

What I Wanted to Do

Use npx to run a npm package that has a postinstall script.

e.g npx gatsby new mysite or npx now

I have also built a minimal reproduction which is used below: npx my-demo-postinstall-plugin@1.0.0

I would expect it to install the npm package and run all lifecycle scripts including postinstall.

What Happened Instead

root@e63aa92f975b:/# npx my-demo-postinstall-plugin@1.0.0
internal/modules/cjs/loader.js:670
    throw err;
    ^

Error: Cannot find module '/root/.npm/_npx/7/lib/node_modules/my-demo-postinstall-plugin/postinstall.js'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:668:15)
    at Function.Module._load (internal/modules/cjs/loader.js:591:27)
    at Function.Module.runMain (internal/modules/cjs/loader.js:877:12)
    at internal/main/run_main_module.js:21:11
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! my-demo-postinstall-plugin@1.0.0 postinstall: `node postinstall.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the my-demo-postinstall-plugin@1.0.0 postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2019-08-07T20_06_02_024Z-debug.log
Install for my-demo-postinstall-plugin@1.0.0 failed with code 1

Reproduction Steps

npx my-demo-postinstall-plugin@1.0.0
It’s easy to test in a fresh docker container but it should work locally too (though on some cases globally installed package allows this to work). Hence I’ve been testing this in a fresh docker container.

The most simple way to reproduce this is: docker run -it --rm node:10 npx my-demo-postinstall-plugin@1.0.0

And the code for my demo package is at: https://gist.github.com/jamo/27edc1f235882ae8d28e37184ee3ed92

Details

It appears the command that npx is running is

/usr/local/bin/node /usr/local/lib/node_modules/npm/bin/npm-cli.js install my-demo-postinstall-plugin@latest --global --prefix /root/.npm/370 --loglevel error --json

And running this command alone fails.

However, if I change the prefix path to e.g. /foo/bar (from the /root/.npm/$pid) the command succeeds.

npm-debug.log:

0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node',
1 verbose cli   '/usr/local/lib/node_modules/npm/bin/npm-cli.js',
1 verbose cli   'install',
1 verbose cli   'my-demo-postinstall-plugin@1.0.0',
1 verbose cli   '--global',
1 verbose cli   '--prefix',
1 verbose cli   '/root/.npm/_npx/7',
1 verbose cli   '--loglevel',
1 verbose cli   'error',
1 verbose cli   '--json' ]
2 info using npm@6.9.0
3 info using node@v10.16.1
4 verbose npm-session e982fe16ea9cca44
5 silly install loadCurrentTree
6 silly install readGlobalPackageData
7 http fetch GET 200 https://registry.npmjs.org/my-demo-postinstall-plugin 136ms
8 silly pacote version manifest for my-demo-postinstall-plugin@1.0.0 fetched in 146ms
9 timing stage:loadCurrentTree Completed in 156ms
10 silly install loadIdealTree
11 silly install cloneCurrentTreeToIdealTree
12 timing stage:loadIdealTree:cloneCurrentTree Completed in 0ms
13 silly install loadShrinkwrap
14 timing stage:loadIdealTree:loadShrinkwrap Completed in 1ms
15 silly install loadAllDepsIntoIdealTree
16 silly resolveWithNewModule my-demo-postinstall-plugin@1.0.0 checking installable status
17 timing stage:loadIdealTree:loadAllDepsIntoIdealTree Completed in 5ms
18 timing stage:loadIdealTree Completed in 7ms
19 silly currentTree lib
20 silly idealTree lib
20 silly idealTree `-- my-demo-postinstall-plugin@1.0.0
21 silly install generateActionsToTake
22 timing stage:generateActionsToTake Completed in 4ms
23 silly diffTrees action count 1
24 silly diffTrees add my-demo-postinstall-plugin@1.0.0
25 silly decomposeActions action count 8
26 silly decomposeActions fetch my-demo-postinstall-plugin@1.0.0
27 silly decomposeActions extract my-demo-postinstall-plugin@1.0.0
28 silly decomposeActions preinstall my-demo-postinstall-plugin@1.0.0
29 silly decomposeActions build my-demo-postinstall-plugin@1.0.0
30 silly decomposeActions install my-demo-postinstall-plugin@1.0.0
31 silly decomposeActions postinstall my-demo-postinstall-plugin@1.0.0
32 silly decomposeActions finalize my-demo-postinstall-plugin@1.0.0
33 silly decomposeActions refresh-package-json my-demo-postinstall-plugin@1.0.0
34 silly install executeActions
35 silly doSerial global-install 8
36 verbose correctMkdir /root/.npm/_locks correctMkdir not in flight; initializing
37 verbose makeDirectory /root/.npm/_locks creation not in flight; initializing
38 silly makeDirectory /root/.npm/_locks uid: 0 gid: 0
39 verbose lock using /root/.npm/_locks/staging-7245e04c7ab2b5c7.lock for /root/.npm/_npx/7/lib/node_modules/.staging
40 silly doParallel extract 1
41 silly extract my-demo-postinstall-plugin@1.0.0
42 silly tarball trying my-demo-postinstall-plugin@1.0.0 by hash: sha512-E62QD1khOB9bcUWQlDvbIBecRRqh875kVHO0Ch9f6hypSRQcGXcP0oSDIQ50esETnqyPRO36wMtShTauBdpmgw==
43 silly tarball no local data for my-demo-postinstall-plugin@1.0.0. Extracting by manifest.
44 http fetch GET 200 https://registry.npmjs.org/my-demo-postinstall-plugin/-/my-demo-postinstall-plugin-1.0.0.tgz 27ms
45 silly extract my-demo-postinstall-plugin@1.0.0 extracted to /root/.npm/_npx/7/lib/node_modules/.staging/my-demo-postinstall-plugin-8f0ce4b4 (34ms)
46 timing action:extract Completed in 36ms
47 silly doReverseSerial unbuild 8
48 silly doSerial remove 8
49 silly doSerial move 8
50 silly doSerial finalize 8
51 silly finalize /root/.npm/_npx/7/lib/node_modules/my-demo-postinstall-plugin
52 timing action:finalize Completed in 3ms
53 silly doParallel refresh-package-json 1
54 silly refresh-package-json /root/.npm/_npx/7/lib/node_modules/my-demo-postinstall-plugin
55 timing action:refresh-package-json Completed in 6ms
56 silly doParallel preinstall 1
57 silly preinstall my-demo-postinstall-plugin@1.0.0
58 info lifecycle my-demo-postinstall-plugin@1.0.0~preinstall: my-demo-postinstall-plugin@1.0.0
59 timing action:preinstall Completed in 1ms
60 silly doSerial build 8
61 silly build my-demo-postinstall-plugin@1.0.0
62 info linkStuff my-demo-postinstall-plugin@1.0.0
63 silly linkStuff my-demo-postinstall-plugin@1.0.0 has /root/.npm/_npx/7/lib/node_modules as its parent node_modules
64 silly linkStuff my-demo-postinstall-plugin@1.0.0 is part of a global install
65 silly linkStuff my-demo-postinstall-plugin@1.0.0 is installed into a global node_modules
66 silly linkStuff my-demo-postinstall-plugin@1.0.0 is installed into the top-level global node_modules
67 verbose linkBins [ { 'my-demo-postinstall-plugin': 'index.js' },
67 verbose linkBins   '/root/.npm/_npx/7/bin',
67 verbose linkBins   true ]
68 timing action:build Completed in 4ms
69 silly doSerial global-link 8
70 silly doParallel update-linked 0
71 silly doSerial install 8
72 silly install my-demo-postinstall-plugin@1.0.0
73 info lifecycle my-demo-postinstall-plugin@1.0.0~install: my-demo-postinstall-plugin@1.0.0
74 timing action:install Completed in 0ms
75 silly doSerial postinstall 8
76 silly postinstall my-demo-postinstall-plugin@1.0.0
77 info lifecycle my-demo-postinstall-plugin@1.0.0~postinstall: my-demo-postinstall-plugin@1.0.0
78 verbose lifecycle my-demo-postinstall-plugin@1.0.0~postinstall: unsafe-perm in lifecycle false
79 verbose lifecycle my-demo-postinstall-plugin@1.0.0~postinstall: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/root/.npm/_npx/7/lib/node_modules/my-demo-postinstall-plugin/node_modules/.bin:/root/.npm/_npx/7/lib/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
80 verbose lifecycle my-demo-postinstall-plugin@1.0.0~postinstall: CWD: /root/.npm/_npx/7/lib/node_modules/my-demo-postinstall-plugin
81 silly lifecycle my-demo-postinstall-plugin@1.0.0~postinstall: Args: [ '-c', 'node postinstall.js' ]
82 silly lifecycle my-demo-postinstall-plugin@1.0.0~postinstall: Returned: code: 1  signal: null
83 info lifecycle my-demo-postinstall-plugin@1.0.0~postinstall: Failed to exec postinstall script
84 timing action:postinstall Completed in 43ms
85 verbose unlock done using /root/.npm/_locks/staging-7245e04c7ab2b5c7.lock for /root/.npm/_npx/7/lib/node_modules/.staging
86 timing stage:rollbackFailedOptional Completed in 2ms
87 timing stage:runTopLevelLifecycles Completed in 268ms
88 verbose stack Error: my-demo-postinstall-plugin@1.0.0 postinstall: `node postinstall.js`
88 verbose stack Exit status 1
88 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:301:16)
88 verbose stack     at EventEmitter.emit (events.js:198:13)
88 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
88 verbose stack     at ChildProcess.emit (events.js:198:13)
88 verbose stack     at maybeClose (internal/child_process.js:982:16)
88 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
89 verbose pkgid my-demo-postinstall-plugin@1.0.0
90 verbose cwd /
91 verbose Linux 4.15.0-55-generic
92 verbose argv "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/bin/npm-cli.js" "install" "my-demo-postinstall-plugin@1.0.0" "--global" "--prefix" "/root/.npm/_npx/7" "--loglevel" "error" "--json"
93 verbose node v10.16.1
94 verbose npm  v6.9.0
95 error code ELIFECYCLE
96 error errno 1
97 error my-demo-postinstall-plugin@1.0.0 postinstall: `node postinstall.js`
97 error Exit status 1
98 error Failed at the my-demo-postinstall-plugin@1.0.0 postinstall script.
98 error This is probably not a problem with npm. There is likely additional logging output above.
99 verbose exit [ 1, true ]

Platform Info

$ npm --versions
{ npm: '6.9.0',
  ares: '1.15.0',
  brotli: '1.0.7',
  cldr: '35.1',
  http_parser: '2.8.0',
  icu: '64.2',
  modules: '64',
  napi: '4',
  nghttp2: '1.34.0',
  node: '10.16.1',
  openssl: '1.1.1c',
  tz: '2019a',
  unicode: '12.1',
  uv: '1.28.0',
  v8: '6.8.275.32-node.54',
  zlib: '1.2.11' }
$ node -p process.platform
linux