package-lock.json keeps changing between platforms and runs

cli
priority:medium
triaged

(Thomas Grainger) #1

Hi, this is a cross-post from https://github.com/npm/npm 1 since the issue in question does not seem to have made its way here yet.

The issue is: https://github.com/npm/npm/issues/17722 14

To summarize, the way package-locking handles optional dependencies results in inconsistencies in the lock-file across different platforms, which adds unexpected churn and results in some dependencies not being locked if the lock-file was generated on the “wrong” platform.

Sorry for not following the reporting template, but everything is kind of already said in the original issue.

What I Wanted to Do

npm install with a stable package-lock.json

What Happened Instead

package-lock.json is different between platforms and sometimes between runs

Reproduction Steps

loads of examples in that github bug

Details

Loads of examples in that github bug

Platform Info

$ npm --versions
{ npm: '6.2.0',
  ares: '1.14.0',
  cldr: '33.1',
  http_parser: '2.8.0',
  icu: '62.1',
  modules: '64',
  napi: '3',
  nghttp2: '1.32.0',
  node: '10.8.0',
  openssl: '1.1.0h',
  tz: '2018e',
  unicode: '11.0',
  uv: '1.22.0',
  v8: '6.7.288.49-node.19',
  zlib: '1.2.11' }

$ node -p process.platform
linux

package-lock.json and optional packages
package-lock.json changes from one `npm install` to the next
(Thomas Grainger) #2

Reformatted from package-lock.json and optional packages


(Thomas Grainger) #3

I think this is also related: https://github.com/npm/npm/issues/18103

I’m opening this issue because:

  • npm is crashing.
  • npm is producing an incorrect install.
  • npm is doing something I don’t understand.
  • Other ( see below for feature requests ):

What’s going wrong?

doc/files/npm-package-locks.md should be updated according to the changes introduced in npm 5.1.0 by means of #16866.

In that document it says:

The presence of a package lock changes the installation behaviour such that:

  1. The module tree described by the package lock is reproduced. This means reproducing the structure described in the file, using the specific files referenced in “resolved” if available, falling back to normal package resolution using “version” if one isn’t.

This holds no longer true since npm 5.1.0, because now the generated module tree is a combined result of both package.json and package-lock.json . (Example: package.json specifies some package with version ^1.1.0 ; package-lock.json had locked it with version 1.1.4 ; but actually, the package is already available with version 1.1.9 . In this case npm i resolves the package to 1.1.9 and overwrites the lockfile accordingly, hence ignoring the information in the lock file.)

  1. The documentation should be corrected and clarified, so that users can understand the behaviour of the lock file. (Maybe other places are affected as well, that I don’t know of.)
  2. It would be helpful if the documentation would point out the preferred way to perform reproducible builds, as this is not obvious anymore. This is especially problematic, because there is already some confusion for people who are trying to achieve deterministic build behaviour, e.g. on CI platforms.

(Thomas Grainger) #4

@zkat can we get a response to this rather than moving it to #support ?


(Felix Becker) #5

This is a big problem for automatic dependency update tools like Renovate: https://github.com/renovatebot/renovate/issues/2294
Essentially the update is not fully automatic, because after every update, I have to fix the lockfile manually again.


(Thomas Grainger) #6

using npm ci isn’t really an option - because we want to detect cases where a developer changed package.json but didn’t generate the package-lock.json or generated it on a different platform with different optional deps eg MacOS


(Felix Becker) #7

I set up a minimal reproduction case in a GitHub repository: felixfbecker/npm-changing-lockfile-repro