Using npm ci does not run prepare script for git modules

priority:medium
triaged
cli
help-wanted

(Eric Dahlseng) #1

What I Wanted to Do

Ran npm ci in a package with a git module, which had a prepare script specified.

What Happened Instead

The prepare script was not run.

Reproduction Steps

Install a git module with a prepare script: npm install git+ssh....
Run npm ci.

Details

Without the ability for npm ci to run prepare scripts on install, it makes it very hard to utilize the benefits of this continuous integration when using git modules.

Platform Info

$ npm --versions
{  npm: '6.1.0',
  ares: '1.10.1-DEV',
  cldr: '32.0',
  http_parser: '2.8.0',
  icu: '60.1',
  modules: '57',
  napi: '3',
  nghttp2: '1.29.0',
  node: '8.11.2',
  openssl: '1.0.2o',
  tz: '2017c',
  unicode: '10.0',
  uv: '1.19.1',
  v8: '6.2.414.54',
  zlib: '1.2.11' }
$ node -p process.platform
darwin

(Kat Marchán) #2

Oh huh. I thought that would be working. Not really sure what’s going on, but I would love some help with this. I assume something’s going wrong in libcipm but it might also be in lib/pack.js in the npm repo.


(Eric Dahlseng) #3

@zkat thanks for pointing out possible locations for the issue. I’ll take a closer look and see if I can narrow down what’s going on.


(Eric Dahlseng) #4

tl;dr

I think I’ve found out what’s going on. Indeed, looks like a bug in libcipm. @zkat, I’ll submit a pull request with a fix shortly I’ve submitted a pull request.

Reproduction notes

I found that the NPM cache was influencing the ability to reproduce this bug (if the package was installed on a new machine with npm install for the first time, then subsequent npm ci would use the cached package, which had already run the prepare script). On machines that installed the package for the first time with npm ci, the prepare script was never run.

The issue can be reproduced in npm@6.2.0 with the following steps:

  • npm init
  • npm install <git module with prepare script>
  • npm cache clean --force
  • npm ci

Root cause

Within the NPM CLI, the pacote config returned from lib/config/pacote.js contains a dirPacker key whose value points to the packGitDep function from lib/pack.js. Under a normal npm install, everything works as expected, but for npm ci, the value for dirPacker is undefined. The reason for this is due to a bug with libcipm, under lib/extract.js. The child() function within this file accepts an opts parameter, and overwrites the returned pacote configuration’s dirPacker with the value from the opts parameter. When opts.dirPacker is undefined, the pacote configuration for dirPacker is overwritten with undefined.

Proposed solution

Within cipm, lib/extract.js, child() function, if opts.dirPacker is undefined, don’t overwrite the configuration returned from toPacote. I’ve confirmed that making this change has solved the issue. I’ll submit a pull request shortly and will update this post when the pull request is up. Pull request submitted.

Tests

Is there a good way to add tests that can prevent regressions against this in the future? I don’t see a good way to add tests to libcipm, but maybe I’m missing something.


(Eric Dahlseng) #5

Update: The PR has been merged. Waiting for the fix to be included in an npm CLI release. Will update here once that happens!


(Eric Dahlseng) #6

Update:


(Eric Dahlseng) #7

npm v6.4.1 was released today, and I’ve just confirmed that after upgrading npm ci works as expected with git modules when following the steps I listed above!


(Johnny Robeson) #8

I have 6.4.1 installed but i’m seeing still seeing this with npm ci and npm install. Does that make it a different issue?


(Eric Dahlseng) #9

@jrobeson have you cleared your NPM cache (npm cache clean --force)? If npm ci was used on a version prior to 6.4.1, NPM may have cached a version of the package that hasn’t run the prepare script.


(Johnny Robeson) #10

@edahlseng yes I did


(Eric Dahlseng) #11

@jrobeson If that’s the case, then this may be a different bug. I’ve been using the 6.4.1 extensively over the past couple of weeks and have not seen any issues with the use cases that I’m exercising npm with.