npm Community Forum (Archive)

The npm community forum has been discontinued.

To discuss usage of npm, visit the GitHub Support Community.

NPM 6.8.0-next.1 to 6.8.0 can perform incomplete package installs with Git / GitHub URLs

What I Wanted to Do

I wanted to install a package from a Git source. (I tried the git+ssh and github protocols.) I expected this to result in a complete copy of the repository contents (give or take the odd ignored file or metadata file).

What Happened Instead

The install worked, and no errors were reported, but the installed package under node_modules was lacking a folder that exists in the Git repository, and wasn’t excluded via .gitignore, .npmignore, nor implicitly via package.json#files.

Reproduction Steps

Original repro using everyday NPM commands:

  1. Make sure to have NPM version 6.8.0-next.1, 6.8.0-next.2, or 6.8.0 installed. (Earlier and later versions don’t exhibit this issue.)

    $ npm i -g npm@6.8.0
  2. Create an empty directory with only the following package.json inside it:

      "private": true,
      "dependencies": {
        "@stakx/npm-install-incomplete-demo": "github:stakx/npm-install-incomplete-demo"
  3. Run npm install.

  4. Look for the node_modules/@stakx/npm-install-incomplete-demo/src/core folder. It should be there (if you cross-check with the GitHub repository), but you likely won’t see it.

Alternative repro (more technical):

Another way of seeing the same problem is with npm pack:

 $ npm i -g npm@6.8.0 && npm pack github:stakx/npm-install-incomplete-demo
 npm notice === Tarball Contents ===
 npm notice 153B package.json
 npm notice 0    src/something-else/.gitkeep

 $ npm i -g npm@6.7.0 && npm pack github:stakx/npm-install-incomplete-demo
 npm notice === Tarball Contents ===
 npm notice 153B package.json
+npm notice 0    src/core/.gitkeep
 npm notice 0    src/something-else/.gitkeep


I’ve been experimenting and debugging a little and ended up in npm/node_modules/npm/lib/pack.js, function packFromPackage, where a .tgz tarball is produced by pacote.tarball.toFile. This produces an incomplete tarball. If I perform the exact same function call in my own Node script, a complete tarball is produced. (I also ruled out cacache as the error source by changing opts.cache to an empty directory.)

My current thinking is that the NPM versions indicated above use a bad combination of dependencies.

For completeness’ sake, I’ve published the same repository as a proper NPM package (see Installing it from there works as it should.

Note that this problem appears to be solved under 6.9.0-next.0. I am posting this bug report anyway to raise awareness of this, in case the bug returns in future versions.

Platform Info

$ npm --versions
{ npm: '6.8.0',
  ares: '1.15.0',
  cldr: '34.0',
  http_parser: '2.8.0',
  icu: '63.1',
  llhttp: '1.0.1',
  modules: '67',
  napi: '3',
  nghttp2: '1.34.0',
  node: '11.4.0',
  openssl: '1.1.0j',
  tz: '2018e',
  unicode: '11.0',
  uv: '1.24.0',
  v8: '',
  zlib: '1.2.11' }

$ node -p process.platform

You identified the crucial issue, folders named core being left out by pack. This got introduced in 6.8.0 and will be fixed in 6.9.0: