NPM 6.8.0-next.1 to 6.8.0 can perform incomplete package installs with Git / GitHub URLs

What I Wanted to Do

I wanted to install a package from a Git source. (I tried the git+ssh and github protocols.) I expected this to result in a complete copy of the repository contents (give or take the odd ignored file or metadata file).

What Happened Instead

The install worked, and no errors were reported, but the installed package under node_modules was lacking a folder that exists in the Git repository, and wasn’t excluded via .gitignore, .npmignore, nor implicitly via package.json#files.

Reproduction Steps

Original repro using everyday NPM commands:

  1. Make sure to have NPM version 6.8.0-next.1, 6.8.0-next.2, or 6.8.0 installed. (Earlier and later versions don’t exhibit this issue.)

    $ npm i -g npm@6.8.0
    
  2. Create an empty directory with only the following package.json inside it:

    {
      "private": true,
      "dependencies": {
        "@stakx/npm-install-incomplete-demo": "github:stakx/npm-install-incomplete-demo"
      }
    }
    
  3. Run npm install.

  4. Look for the node_modules/@stakx/npm-install-incomplete-demo/src/core folder. It should be there (if you cross-check with the GitHub repository), but you likely won’t see it.

Alternative repro (more technical):

Another way of seeing the same problem is with npm pack:

 $ npm i -g npm@6.8.0 && npm pack github:stakx/npm-install-incomplete-demo
 ...
 npm notice === Tarball Contents ===
 npm notice 153B package.json
 npm notice 0    src/something-else/.gitkeep
 ...

 $ npm i -g npm@6.7.0 && npm pack github:stakx/npm-install-incomplete-demo
 ...
 npm notice === Tarball Contents ===
 npm notice 153B package.json
+npm notice 0    src/core/.gitkeep
 npm notice 0    src/something-else/.gitkeep
 ...

Details

I’ve been experimenting and debugging a little and ended up in npm/node_modules/npm/lib/pack.js, function packFromPackage, where a .tgz tarball is produced by pacote.tarball.toFile. This produces an incomplete tarball. If I perform the exact same function call in my own Node script, a complete tarball is produced. (I also ruled out cacache as the error source by changing opts.cache to an empty directory.)

My current thinking is that the NPM versions indicated above use a bad combination of dependencies.

For completeness’ sake, I’ve published the same repository as a proper NPM package (see npmjs.com). Installing it from there works as it should.

Note that this problem appears to be solved under 6.9.0-next.0. I am posting this bug report anyway to raise awareness of this, in case the bug returns in future versions.

Platform Info

$ npm --versions
{ npm: '6.8.0',
  ares: '1.15.0',
  cldr: '34.0',
  http_parser: '2.8.0',
  icu: '63.1',
  llhttp: '1.0.1',
  modules: '67',
  napi: '3',
  nghttp2: '1.34.0',
  node: '11.4.0',
  openssl: '1.1.0j',
  tz: '2018e',
  unicode: '11.0',
  uv: '1.24.0',
  v8: '7.0.276.38-node.13',
  zlib: '1.2.11' }

$ node -p process.platform
win32

You identified the crucial issue, folders named core being left out by pack. This got introduced in 6.8.0 and will be fixed in 6.9.0:

2 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.