devDependencies are installed if global flag used and shrinkwrap file is present

What I Wanted to Do

Wanted to install a module where npm-shrinkwrap.json is present, and normal and dev dependencies are both defined. In that case what I’ve expected was npm install --global --production only installs production dependencies (as it is the case when there is not shrinkwrap file).

This belief is based on the npm help, as the behaviour of the global flag should work, though that does not mention npm-shrinkwrap.json specifically.

In global mode (ie, with -g or --global appended to the command), it installs the current package context (ie, the current working directory) as a global package. By default, npm install will install all modules listed as dependencies in npm help 5 package.json.
With the --production flag (or when the NODE_ENV environment variable is set to production), npm will not install modules listed in devDependencies. NOTE: The --production flag has no particular meaning when adding a dependency to a project.

What Happened Instead

  1. with --global --production and shrinkwrap, the devDependencies are installed as well :heavy_multiplication_x:
  2. with --global --production --no-shrinkwrap only the prod dependencies are installed :heavy_check_mark:
  3. with --production and shrinkwrap (that is no --global flag, otherwise the same as 1.) only the production dependencies are installed :heavy_check_mark:

Reproduction Steps

Set up a simple package.json with 1 dependency and 1 devDependency:

{
  "name": "npmtest",
  "version": "0.0.1",
  "description": "Testing npm-shrinkwrap",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "jquery": "^3.4.1"
  },
  "devDependencies": {
    "lodash": "^4.17.15"
  }
}

then:

  • npm install to install both dependencies
  • npm shrinkwrap to generate the shrinkwrap file, and this was the content, just to cross-check:
{
  "name": "npmtest",
  "version": "0.0.1",
  "lockfileVersion": 1,
  "requires": true,
  "dependencies": {
    "jquery": {
      "version": "3.4.1",
      "resolved": "https://registry.npmjs.org/jquery/-/jquery-3.4.1.tgz",
      "integrity": "sha512-36+AdBzCL+y6qjw5Tx7HgzeGCzC81MDDgaUP8ld2zhx58HdqXGoBd+tHdrBMiyjGQs0Hxs/MLZTu/eHNJJuWPw=="
    },
    "lodash": {
      "version": "4.17.15",
      "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.15.tgz",
      "integrity": "sha512-8xOcRHvCjnocdS5cpwXQXVzmmh5e5+saE2QGoeQmbKmRS6J3VQppPOIt0MnmE+4xlZoumy0GPG0D0MVIQbNA1A==",
      "dev": true
    }
  }
}

The reproduction is then (node set up with nvm):

a) global flag and shrinkwrap:

~/prog/npmtest> rm -rf node_modules 
~/prog/npmtest> npm install --global --production
+ npmtest@0.0.1
added 2 packages from 3 contributors and updated 1 package in 0.397s
~/prog/npmtest> ls node_modules 
jquery  lodash

installs also the devDependencies.

b) global flag and no-shrinkwrap flag

~/prog/npmtest> rm -rf node_modules              
~/prog/npmtest> npm install --global --production --no-shrinkwrap
+ npmtest@0.0.1
added 1 package from 1 contributor and updated 1 package in 0.368s
~/prog/npmtest> ls node_modules                              
jquery

correct packages.

c) no global flag, and have shrinkwrap:

~/prog/npmtest> npm install --production                
npm WARN npmtest@0.0.1 No repository field.

added 1 package from 1 contributor and audited 2 packages in 0.421s
found 0 vulnerabilities

~/prog/npmtest> ls node_modules     
jquery

also installs the right packages.

d) global flag but no shrinkwrap:

~/prog/npmtest> rm npm-shrinkwrap.json
~/prog/npmtest> rm -rf node_modules   
~/prog/npmtest> npm install --global --production                
+ npmtest@0.0.1
added 1 package from 1 contributor and updated 1 package in 0.344s
~/prog/npmtest> ls node_modules 
jquery

also installs the right packages.

Details

A debug log generated with npm install --global --install --timing with a shrinkwrap:

2019-07-23T13_19_29_786Z-debug.log (7.6 KB)

Platform Info

$ npm --versions
{
  npmtest: '0.0.1',
  npm: '6.10.1',
  ares: '1.15.0',
  brotli: '1.0.7',
  cldr: '35.1',
  http_parser: '2.8.0',
  icu: '64.2',
  llhttp: '1.1.4',
  modules: '72',
  napi: '4',
  nghttp2: '1.38.0',
  node: '12.6.0',
  openssl: '1.1.1c',
  tz: '2019a',
  unicode: '12.1',
  uv: '1.30.1',
  v8: '7.5.288.22-node.14',
  zlib: '1.2.11'
}
$ node -p process.platform
linux

Found the possible issue and opened a PR, any feedback is appreciated!

Scratch this above for the time being, the changes fixed my simple test cases, but broke the tests, note to self, never skip npm test before sending a PR. Sorry about it.

Thus the original issue still stands, and looking into it.

The test I think could be used for this (and failing on the current release) is adding into test/tap/install-cli-only-shrinkwrap.js this code below:

test('\'npm install --only=production --global\' should only install dependencies', function (t) {
  cleanup()
  setup()
  common.npm(['install', '--only=production', '--global'], EXEC_OPTS, function (err, code, stdout, stderr) {
    if (err) throw err
    t.comment(stdout.trim())
    t.comment(stderr.trim())
    t.is(code, 0, 'npm install did not raise error code')
    t.ok(
      existsSync(
        path.resolve(pkg, 'node_modules/dependency/package.json')
      ),
      'dependency was installed'
    )
    t.notOk(
      existsSync(path.resolve(pkg, 'node_modules/dev-dependency/package.json')),
      'devDependency was NOT installed'
    )
    t.end()
  })
})

(which is an expansion of another test already in that file, without the --global flag, which behaves correctly).

The test above then throw for now:

test/tap/install-cli-only-shrinkwrap.js ............. 13/14 7s
  'npm install --only=production --global' should only install dependencies
  not ok devDependency was NOT installed
    at:
      line: 140
      column: 7
      file: test/tap/install-cli-only-shrinkwrap.js
      type: global
    stack: |
      test/tap/install-cli-only-shrinkwrap.js:140:7
      f (node_modules/once/once.js:25:25)
      ChildProcess.<anonymous> (test/common-tap.js:127:5)

Digging deeper, the issue seems to be, that when using global and a shrinkwrap, the given dev dependency ends up with an empty requiredBy list, and if development packages are excluded, due the check for “is it really dev package all the way?” fails.

  • If no shrinkwrap is used, the dev dependency correctly has the project being installed listed as requiredBy
  • if no global is set (while keeping shrinkwrap), ditto the requiredBy is added

Thus my previous fix attemt, while it was likely on the correct path, should be reversed: instead of throwing away the package (not installing it) if it’s not required by any other package (which breaks other use cases), instead the thing is to correctly set the requiredBy value. Just not really clear yet how that ends up being set, and why is it not here… :confused: