npm Community Forum (Archive)

The npm community forum has been discontinued.

To discuss usage of npm, visit the GitHub Support Community.

package-lock.json keeps changing between platforms and runs

Hi, this is a cross-post from 1 since the issue in question does not seem to have made its way here yet.

The issue is: 14

To summarize, the way package-locking handles optional dependencies results in inconsistencies in the lock-file across different platforms, which adds unexpected churn and results in some dependencies not being locked if the lock-file was generated on the “wrong” platform.

Sorry for not following the reporting template, but everything is kind of already said in the original issue.

What I Wanted to Do

npm install with a stable package-lock.json

What Happened Instead

package-lock.json is different between platforms and sometimes between runs

Reproduction Steps

loads of examples in that github bug


Loads of examples in that github bug

Platform Info

$ npm --versions
{ npm: '6.2.0',
  ares: '1.14.0',
  cldr: '33.1',
  http_parser: '2.8.0',
  icu: '62.1',
  modules: '64',
  napi: '3',
  nghttp2: '1.32.0',
  node: '10.8.0',
  openssl: '1.1.0h',
  tz: '2018e',
  unicode: '11.0',
  uv: '1.22.0',
  v8: '',
  zlib: '1.2.11' }

$ node -p process.platform

Reformatted from package-lock.json and optional packages

I think this is also related:

I’m opening this issue because:

What’s going wrong?

doc/files/ should be updated according to the changes introduced in npm 5.1.0 by means of #16866.

In that document it says:

The presence of a package lock changes the installation behaviour such that:

  1. The module tree described by the package lock is reproduced. This means reproducing the structure described in the file, using the specific files referenced in “resolved” if available, falling back to normal package resolution using “version” if one isn’t.

This holds no longer true since npm 5.1.0, because now the generated module tree is a combined result of both package.json and package-lock.json . (Example: package.json specifies some package with version ^1.1.0 ; package-lock.json had locked it with version 1.1.4 ; but actually, the package is already available with version 1.1.9 . In this case npm i resolves the package to 1.1.9 and overwrites the lockfile accordingly, hence ignoring the information in the lock file.)

  1. The documentation should be corrected and clarified, so that users can understand the behaviour of the lock file. (Maybe other places are affected as well, that I don’t know of.)
  2. It would be helpful if the documentation would point out the preferred way to perform reproducible builds, as this is not obvious anymore. This is especially problematic, because there is already some confusion for people who are trying to achieve deterministic build behaviour, e.g. on CI platforms.

@zkat can we get a response to this rather than moving it to #support ?

This is a big problem for automatic dependency update tools like Renovate:
Essentially the update is not fully automatic, because after every update, I have to fix the lockfile manually again.

using npm ci isn’t really an option - because we want to detect cases where a developer changed package.json but didn’t generate the package-lock.json or generated it on a different platform with different optional deps eg MacOS

I set up a minimal reproduction case in a GitHub repository: felixfbecker/npm-changing-lockfile-repro

I can reproduce bug with changed optional: true flags event on single machine without change platform/os.

$ ls

$ npm install
$ ls
node_modules  package.json  package-lock.json

$ git add .
$ git commit "install"
$ rm -rf node_modules
$ npm install
$ git status
changed package-lock.json

$ git diff
# lot of added optional: true lines
# okay, let commit it

$ git commit -am "change in lockfile"
$ rm -rf node_modules
$ npm install
$ git status
changed package-lock.json
$ git diff
# lot of added optional: true, but in other packages
# okay, commit it again
$ git commit -am "lockfiles, again"
$ rm -rf node_modules
$ npm install
$ git status
no changes # finally

You can get my package.json dependencies here for local reproduce:

UPD: also, sometimes “optional: true” lines gonna to be removed, but I didn’t caught actusl case.

UPD2: just make npm i somepackage removes “optional: true” from some packages.

Is there any plans to fix it?

We do have the same problem here. I applied this fix and it helps, but there are still some machines that produce different package-lock.json files. The differences can be on:

Special note on the https <-> http. This one is actually strange because for example yargs 11.1.0 can be in HTTPS on some machine, which I don’t understand because when I read, I can see that the referenced tarball for this version is http and not https. I must be missing something here… And of course npm config get registry outputs

PS : we all have node 8.11.4 or 8.12, and NPM 6.4.1.

I think we have some progress on the optional and dev part in this thread. That fix fixes at least one of the problems, I’m not sure if there are any other. As for HTTPS <-> HTTP, I believe that’s covered here.

I have another related problem, which I don’t think has been mentioned yet.

I define a custom registry in my .npmrc file, that proxies to the primary npm registry for most packages, and hosts our internal organization packages at another URL:

... registry auth stuff here...

I find that package-lock.json inconsistently includes either the “correct” registry vs the default registry:

grep -R "" package-lock.json | wc -l

grep -R "" package-lock.json  | wc -l

You might be tempted to posit that the references are from before the .npmrc was configured, but no, you’d be wrong: npm i regularly switches references from TO for no reason at all.

Currently I am using npm 6.4.1.

I continue to see all 3 of these changes. I’m on node 10.12.0, npm 6.4.1

The fix to pin all dev environments to specific versions isn’t ideal. Would be good to get a resolution to this.


The http/https issue is being tracked here:

And the optional/dev issue can be tracked here:

We have run into these issues too. We develop on Mac and then AWS builds from GitHub and so see changes with the optional flag and sometimes the http/https changes too. (Node v8.11.4 and NPM v6.4.1) Our team would love to see a solution to these issues.

Have confirmed the http/https issue is fixed in npm 6.6.0-next.0

However the following issues continue:

Hoping the fix for these will land in the 6.6.0 release.