`npm ls` produces false negative missing dependencies when `node_modules` dir is symlinked

What I Wanted to Do

Check if packages are installed correctly (no missing dependencies). I expected npm ls to give no false negatives.

What Happened Instead

It gave a lot of false negatives.

Also, after second npm i structure of node_modules dir changed, and ng can’t find packages, loaders.

Reproduction Steps

$ git clone https://github.com/x-yuri/issues -b npm-symlinked-node-modules npm-symlinked-node-modules
$ cd npm-symlinked-node-modules

$ tree
.
β”œβ”€β”€ current
β”‚   β”œβ”€β”€ node_modules -> ../shared/node_modules
β”‚   β”œβ”€β”€ package.json
β”‚   └── package-lock.json
└── shared
    └── node_modules

$ cd current
$ cat package.json
{
  "dependencies": {
    "resolve": "^1.9.0"
  }
}

$ npm i
npm WARN current No description
npm WARN current No repository field.
npm WARN current No license field.

added 2 packages from 2 contributors and audited 2 packages in 0.535s
found 0 vulnerabilities

$ npm ls
/home/yuri/prj/issues/current
β”œβ”€β”€ path-parse@1.0.6 extraneous
└─┬ resolve@1.9.0
  └── UNMET DEPENDENCY path-parse@^1.0.6

npm ERR! extraneous: path-parse@1.0.6 /home/yuri/prj/issues/current/node_modules/path-parse
npm ERR! missing: path-parse@^1.0.6, required by resolve@1.9.0

$ npm i
npm WARN current No description
npm WARN current No repository field.
npm WARN current No license field.

moved 1 package and audited 2 packages in 0.559s
found 0 vulnerabilities

$ npm ls
/home/yuri/prj/issues/current
└─┬ resolve@1.9.0
  └── path-parse@1.0.6

$ git diff
diff --git a/current/package-lock.json b/current/package-lock.json
index a7551f0..07a0808 100644
--- a/current/package-lock.json
+++ b/current/package-lock.json
@@ -2,17 +2,19 @@
   "requires": true,
   "lockfileVersion": 1,
   "dependencies": {
-    "path-parse": {
-      "version": "1.0.6",
-      "resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.6.tgz",
-      "integrity": "sha512-GSmOT2EbHrINBf9SR7CDELwlJ8AENk3Qn7OikK4nFYAu3Ote2+JYNVvkpAEQm3/TLNEJFD/xZJjzyxg3KBWOzw=="
-    },
     "resolve": {
       "version": "1.9.0",
       "resolved": "https://registry.npmjs.org/resolve/-/resolve-1.9.0.tgz",
       "integrity": "sha512-TZNye00tI67lwYvzxCxHGjwTNlUV70io54/Ed4j6PscB8xVfuBJpRenI/o6dVk0cY0PYTY27AgCoGGxRnYuItQ==",
       "requires": {
         "path-parse": "^1.0.6"
+      },
+      "dependencies": {
+        "path-parse": {
+          "version": "1.0.6",
+          "resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.6.tgz",
+          "integrity": "sha512-GSmOT2EbHrINBf9SR7CDELwlJ8AENk3Qn7OikK4nFYAu3Ote2+JYNVvkpAEQm3/TLNEJFD/xZJjzyxg3KBWOzw=="
+        }
       }
     }
   }

Link to the corresponding branch of a GitHub repository.

With more complex package.json running npm i won’t solve the issue. Take for example the following package.json:

{
  "dependencies": {
    "@angular-devkit/build-angular": "^0.11.4",
    "@angular/compiler-cli": "7.x",
    "@angular/compiler": "7.x",
    "ajv-keywords": "3.1.0"
  }
}

ajv-keywords’ peer dependency on ajv is resolved due to deduplication. But with symlinked node_modules dir, after running npm i twice, node_modules contains just: @angular-devkit, @angular, ajv-keywords. As such the peer dependency is no longer met. Although it was met after first npm i.

Details

I believe it’s common to have the following directory structure on the server (at least among Ruby and PHP developers):

site
β”œβ”€β”€ current -> releases/3
β”œβ”€β”€ shared
└─┬ releases
  β”œβ”€β”€ 1
  β”œβ”€β”€ 2
  └── 3

And generally, all sort of packages are shared between releases by symlinking to shared dir. And it would be great to share nodejs packages as well:

β”œβ”€β”¬ shared
β”‚ └── node_modules
└─┬ releases
  └─┬ 3
    └── node_modules -> ../../shared/node_modules

If not for this issue.

npm seems to intentionally not follow symlinks. As a result, it can’t find the package and considers it extraneous. On second npm i run it deletes the supposedly extraneous package, and settles with path-parse inside resolve's node_modules dir.

There’s a related issue. But the thing is that not only do packages’ locations change but also npm ls produces misleading output.

The workarounds are:

  1. Ignore it and run npm ls with NODE_PRESERVE_SYMLINKS=1 when needed.
  2. Run npm i twice.
  3. Don’t share node_modules dir.

Platform Info

$ npm --versions
{ npm: '6.9.0',
  ares: '1.15.0',
  brotli: '1.0.7',
  cldr: '35.1',
  http_parser: '2.8.0',
  icu: '64.2',
  llhttp: '1.1.1',
  modules: '67',
  napi: '4',
  nghttp2: '1.36.0',
  node: '11.15.0',
  openssl: '1.1.1b',
  tz: '2019a',
  unicode: '12.1',
  uv: '1.28.0',
  v8: '7.0.276.38-node.19',
  zlib: '1.2.11' }

$ node -p process.platform
linux