package-lock.json changes from one `npm install` to the next

cli
priority:medium
triaged

(Dane Powell) #1

What I Wanted to Do

If I run npm install repeatedly for a given package.json / package-lock.json file on the same machine, I expect package-lock.json to not be modified from one run to the next.

What Happened Instead

When I run npm install repeatedly for a given package.json / package-lock.json file on the same machine, package-lock.json changes from one run to the next depending on the state of the machine.

On a clean install with no existing node_modules folder, a lot of the packages contain the key "optional": true

Subsequently, when node_modules already exists and I run npm install , the "optional": true keys get removed.

This generates a ton of noise and commit diffs.

Reproduction Steps

Via kevinburkeomg in the upstream issue, these steps will reproduce this class of issue, but the actual contents of the package-lock file that are changing are a little different. In my private projects I was seeing the optional keys changing, but when I run the steps below I see dependency orders changing. Maybe the same root cause, who knows :slight_smile:

Follow these steps:

  1. Clone the repository
  2. Run “npm install” to populate node_modules . Nothing should change in the git diff
  3. Run “npm install” again and observe that the package-lock.json file changes, the reverse of the diff shown here: kevinburkeomg/npm-inconsistency@18b1a21

Details

This is the only potentially relevant error I see:

14 verbose stack Error: bs4@0.1.0 build: `gulp`
14 verbose stack Exit status 1
14 verbose stack     at EventEmitter.<anonymous> (/usr/lib/node_modules/npm/lib/utils/lifecycle.js:255:16)
14 verbose stack     at emitTwo (events.js:106:13)
14 verbose stack     at EventEmitter.emit (events.js:191:7)
14 verbose stack     at ChildProcess.<anonymous> (/usr/lib/node_modules/npm/lib/utils/spawn.js:40:14)
14 verbose stack     at emitTwo (events.js:106:13)
14 verbose stack     at ChildProcess.emit (events.js:191:7)
14 verbose stack     at maybeClose (internal/child_process.js:920:16)
14 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:230:5)
15 verbose pkgid bs4@0.1.0

More details from original issue: https://github.com/npm/npm/issues/20926

Platform Info

$ npm --versions
{ npm: '6.1.0',
  ares: '1.10.1-DEV',
  cldr: '32.0',
  http_parser: '2.8.0',
  icu: '60.1',
  modules: '57',
  napi: '3',
  nghttp2: '1.29.0',
  node: '8.11.2',
  openssl: '1.0.2o',
  tz: '2017c',
  unicode: '10.0',
  uv: '1.19.1',
  v8: '6.2.414.54',
  zlib: '1.2.11' }
$ node -p process.platform
linux

(Kat Marchán) #2

Triage notes: From original github issue, this seems to be a repro: https://github.com/npm/npm/issues/20926#issuecomment-397372131 Haven’t tried it myself, though.

Also, this is probably one of the issues that package-lock.json keeps changing between platforms and runs is having trouble with.


(Brian Olore) #3

I can confirm that problem in the original github issue is still present in npm v6.3 & v6.4.

FWIW - in other projects, I’ve also seen @danepowell’s "optional": true bug, but don’t have a reproducible use case


package-lock.json is not complete on first run for some modules.
(Dane Powell) #4

Thanks, I added the repro steps from the upstream issue to my description. However, the specific bug that they produce is slightly different from what I saw. It might still be the same root cause, I think it’s worth investigating regardless.


(Rhys Arkins) #5

I am also seeing the same problem and would like to point out that it also applies to --package-lock-only at times too. e.g. an automated process updates the lock file with --package-lock-only and optional lines are added, then someone runs npm install and see them removed again. Although it’s possible to debug this diff to see it’s mostly contained to optional lines only, the effect is it erodes trust in the lock file if people just get used to seeing it changing all the time and learn to ignore it.


(Stef) #6

This issue is preventing my team from committing the lockfile into VC for the reasons @rarkins mentioned. We are using Renovate for automated package updates, which adds optional:true lines to the lock, and then running npm install locally removes these lines (macOS 10.13, npm 6.4.1).

After each Renovate run, these changes would either have to be reverted (after every install) or committed into VC. As @rarkins says, this erodes trust in the lockfile and somewhat defeats the point of automated package updates.


(Yuri Kanivetsky) #8

Let us separate optional: true issue from the one demonstrated by this post. They doesn’t seem to be related.

You can find more details on optional: true bug in the following thread, with explanation and possible fix.