NPM Install <package> overrides/deletes Gitlab repositories listed in package.json

What I Wanted to Do

I’m working in an Ionic project. I ran npm install to install a new plugin.

What Happened Instead

npm installed the new plugin but deleted every git dependency hosted in our private gitlab repository. I had to manually run npm install for each one of the deleted dependencies listed in package.json to restore the deleted packages. We have to do this again and again.

Reproduction Steps

  1. Configure a private gitlab repository and upload a test package.
  2. Create a new npm test project and have your package.json look like this:
"dependencies": {
   ...
   "problematic-dependency": "git+https://lab.mycompany.com/blah/blah/myrepo.git#sometag",
   "cordova-plugin-network-information": "git+https://github.com/apache/cordova-plugin-network-information.git", // can be any other public github dependency
}
  1. Run npm install to install everything for the first time.
  2. Run npm install some-new-dependency to install a new dependency of your choice (doesn’t matter if it is not of git type, any package hosted in the public npm registry will do). You will notice that problematic-dependency has been deleted from the node_modules folder even though it still appears in the package.json. Any other public github dependency remains however, in our case cordova-plugin-network-information is still in place.

Details

This is probably a variation of issue 19394 which apaprently was fixed in v5.7. We however reproduced this new bug in v5.7, as we found the original bug in github and advised the team to upgrade to at least v5.7. We believe the bug was only fixed for github dependencies (public or private) or maybe it was only fixed for public git dependencies hosted anywhere.

Platform Info

Reproduced in Linux and mac.

$ npm --versions
{ ourApp: '1.0.0',
  npm: '5.7.0',
  ares: '1.13.0',
  cldr: '32.0',
  http_parser: '2.7.0',
  icu: '60.1',
  modules: '59',
  nghttp2: '1.29.0',
  node: '9.4.0',
  openssl: '1.0.2n',
  tz: '2017c',
  unicode: '10.0',
  uv: '1.18.0',
  v8: '6.2.414.46-node.17',
  zlib: '1.2.11' }

$ node -p process.platform
linux

I can’t reproduce this in 6.10.2. Can you update to the latest npm and see if it’s still a problem?

I can reproduce this using the latest version 6.10.2. “npm install” will remove any packages that are not listed as dependencies in package.json.

  1. Install any package with npm install --no-save my-package
  2. Install any other package with npm install
  3. Observe that the first package was removed when installing the second

What’s the name field in the package.json file in the test package? It should be "name": "problematic-dependency" in this case.


EDIT: scratch that, it seems to work fine for me in npm 6.10.2, regardless of the name in the git dep’s package.json file.

@koga73 This is by design. Extraneous packages (ie, packages in node_modules that are not listed anywhere as dependencies) are removed by npm install. You could make the argument that it’s unexpected or undesirable intent, but it is at least a different issue, and working as intended.

If there isn’t already a request for this I can open one. Previous versions of npm didn’t have this issue. Now if you use npm link, install local dependencies or anything that doesn’t save the dependency to package.json it gets removed next time npm install is ran.

This is rarely going to be the case. npm install by default runs as if you typed the --save option since v5. you need to explicitly type --no-save to not save it. In my case neither me or my team mates installed anything with that option.

@isaacs Yep, tried with 6.10.2 and it doesn’t delete gitlab dependencies anymore. This is my versions now:

{ 
  npm: '6.10.2',
  ares: '1.13.0',
  cldr: '33.0',
  http_parser: '2.8.0',
  icu: '61.1',
  modules: '59',
  napi: '3',
  nghttp2: '1.32.0',
  node: '9.11.2',
  openssl: '1.0.2o',
  tz: '2018c',
  unicode: '10.0',
  uv: '1.19.2',
  v8: '6.2.414.46-node.23',
  zlib: '1.2.11'
}

The bug must have been fixed somewhere in between versions 5.7 and 6.10.2. I have to say v6.10.2 was too new for us to have. Our development environment for this project was created last year and at that time the latest npm version was 5.6. We even packaged it in virtual machines and never updated any dependency or tool unless needed. Some devs however had a newer environment in their macs however they also suffered the deletion so maybe they still didn’t have a npm version new enough. I guess nobody ever updates npm :smile: