npm Community Forum (Archive)

The npm community forum has been discontinued.

To discuss usage of npm, visit the GitHub Support Community.

npm install for package with local dependency fails

While I can neither confirm or deny if the original issue has been resolved, I can confirm that on npm v6.9.0, the issue persists if the dependency has a symlink in its path.

8771 verbose stack Error: ENOENT: no such file or directory, rename '/Users/sohrab/workspaces/myworkspace/myapp/server/node_modules/.staging/@company/fastify-openapi-cfdde382/node_modules/@babel/cli' -> '/Users/sohrab/workspaces/myworkspace/myapp/server/node_modules/.staging/@babel/cli-e2c46704'
8772 verbose cwd /Users/sohrab/workspaces/myworkspace/myapp/server
8773 verbose Darwin 18.5.0
8774 verbose argv "/Users/sohrab/.nvm/versions/node/v10.15.3/bin/node" "/Users/sohrab/.nvm/versions/node/v10.15.3/bin/npm" "update" "nyc" "mocha"
8775 verbose node v10.15.3
8776 verbose npm  v6.9.0
8777 error path /Users/sohrab/workspaces/myworkspace/myapp/server/node_modules/.staging/@company/fastify-openapi-cfdde382/node_modules/@babel/cli
8778 error code ENOENT
8779 error errno -2
8780 error syscall rename
8781 error enoent ENOENT: no such file or directory, rename '/Users/sohrab/workspaces/myworkspace/myapp/server/node_modules/.staging/@company/fastify-openapi-cfdde382/node_modules/@babel/cli' -> '/Users/sohrab/workspaces/myworkspace/myapp/server/node_modules/.staging/@babel/cli-e2c46704'
8782 error enoent This is related to npm not being able to find a file.
8783 verbose exit [ -2, true ]

When using file: dependency to a symlink directory:

....
"dependencies": {
    "@company/fastify-openapi": "file:../fastify-openapi",
...

My current workaround is to keep deleting package-lock.json.


https://github.com/npm/cli/pull/40


What I Wanted to Do

Install a package that has local dependencies (i.e. “file:…”).

What Happened Instead

I get an error message that transitive dependency of the local dependency cannot be installed:

ENOENT: no such file or directory, rename ‘/XYZ/npm-install-local-dependency-bug/examples/node_modules/.staging/npm-install-local-dependency-bug-f307b53e/node_modules/acorn’ -> ‘/XYZ/npm-install-local-dependency-bug/examples/node_modules/.staging/acorn-af0fc6c4’

Reproduction Steps

$ git clone https://github.com/jhecking/npm-install-local-dependency-bug.git .
Cloning into '.'...
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 7 (delta 2), reused 7 (delta 2), pack-reused 0
Unpacking objects: 100% (7/7), done.

$ cat package.json
{
  "name": "npm-install-local-dependency-bug",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "standard": "^11.0"
  }
}

$ npm install
npm WARN npm-install-local-dependency-bug@1.0.0 No description
npm WARN npm-install-local-dependency-bug@1.0.0 No repository field.

added 214 packages from 173 contributors and audited 374 packages in 2.304s
found 0 vulnerabilities

$ cd examples/

$ cat package.json
{
  "name": "examples",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "npm-install-local-dependency-bug": "file:.."
  }
}

$ npm install
npm WARN examples@1.0.0 No description
npm WARN examples@1.0.0 No repository field.

npm ERR! path /XYZ/npm-install-local-dependency-bug/examples/node_modules/.staging/npm-install-local-dependency-bug-f307b53e/node_modules/acorn
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall rename
npm ERR! enoent ENOENT: no such file or directory, rename '/XYZ/npm-install-local-dependency-bug/examples/node_modules/.staging/npm-install-local-dependency-bug-f307b53e/node_modules/acorn' -> '/XYZ/npm-install-local-dependency-bug/examples/node_modules/.staging/acorn-af0fc6c4'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/jhecking/.npm/_logs/2018-07-20T04_01_29_488Z-debug.log

Details

2018-07-20T04_01_29_488Z-debug.log (21.0 KB)

Platform Info

$ npm --versions
{ examples: '1.0.0',
  npm: '6.2.0',
  ares: '1.14.0',
  cldr: '33.0',
  http_parser: '2.8.0',
  icu: '61.1',
  modules: '64',
  napi: '3',
  nghttp2: '1.32.0',
  node: '10.6.0',
  openssl: '1.1.0h',
  tz: '2018c',
  unicode: '10.0',
  uv: '1.21.0',
  v8: '6.7.288.46-node.13',
  zlib: '1.2.11' }
$ node -p process.platform
darwin


I mean, I know what the commit does, ofc, but the error is interesting…

So this class of ENOENT error occurs when we think we have a bundled dependency and we’re trying to unbundle it before continuing the install.

One of the quirks of file: type dependencies where we’re linking to a location outside of the location they’re being installed to is that they bring their own dependency tree, much like bundled dependencies. That you’re getting this implies that it’s getting confused and improperly trying to treat a linked dependency like a bundled dependency. Though in this case it has to do with a combination of links, bundles and transitive dependencies, which is apparently how our test suite overlooked it.


To reproduce using serverless-artillery we can run:

git clone --depth=50 --branch=refactor-modularize-faas https://github.com/Nordstrom/serverless-artillery.git Nordstrom/serverless-artillery
cd Nordstrom/serverless-artillery/
npm ci

Ran last against the commit 4c6e921b76eae4edf08698c8b565b4f663a129a0


From that changelog, this one entry stands out to me:

0dffa9c2a 609d6f6e1 08f81aa94 f8b76e076 6d609822d 59d080a22 Restore the ability to bundle dependencies that are uninstallable from the registry. This also eliminates needless registry lookups for bundled dependencies.

Fixed a bug where attempting to install a dependency that is bundled inside another module without reinstalling that module would result in ENOENT errors. (@iarna)

It has to do with bundled dependencies, which the acorn package is in this case. And because it’s supposed to address ENOENT errors, which is exactly the kind of error that is now occurring. Might be a red herring, off course.


We’ve also found this in every published 6.x release. Ensuring package-lock.json isn’t present solves the issue but creates a challenge in our travis-ci builds.


Would this issue cause the same issue reported here: https://github.com/npm/npm/issues/18352 Long story short, we have a bundledDependencies entry for some-module and in our source tree, we have node_modules/some-module with the appropriate package.json and version. The only recourse right now is to use npm@4.


This might be the same bug and has a reproducible example: ENOENT error when the same package appears in local dependencies


(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)


This is still happening in npm 6.11.3. The only “solution” I found was removing package-lock.json.


Ok, I believe I have a fix for this. Transitive dependencies of the linked module that had been kept at least one level deep in the linked module would result in having the fact they were (deep) inside a symlinks lost when that tree was being reconstituted from a a lock-file.

All that being said, I think we shouldn’t be storing the children if linked deps that exist outside of our own tree in our lockfile at all, so I’ve patched that as well. I’ll put this in canary soonish so you all can give it a go.


One more observation that might or might not be helpful to pin down the exact issue:

If I delete the package-lock.json file in the examples subdirectory, prior to running npm install in that directory, then the command succeeds.


You wanna do a git bisect on that? :slight_smile:


Have you tried the current npm@next, currently npm@6.6.0-next.0? It seems to fix the issue for me (after deleting node_modules and package-lock.json once).


Any update on this issue (or above PR)? Would love for this to land. Let me know if I can help at all!


Here is the result of running git bisect:

$ git bisect start v6.2.0 v5.9.0-next.0
Bisecting: a merge base must be tested
[b1d3b2b33b71beb202ff612b12f7da623152737a] doc: update changelog for npm@5.9.0

$ git bisect run ../bisect_test
running ../bisect_test
Bisecting: 114 revisions left to test after this (roughly 7 steps)
[89102c0d995c3d707ff2b56995a97a1610f8b532] shrinkwrap: Prefer computed resolved from dep tree
running ../bisect_test
Bisecting: 56 revisions left to test after this (roughly 6 steps)
[0573d91e57c068635a3ad4187b9792afd7b5e22f] cacache@11.0.1
running ../bisect_test
Bisecting: 28 revisions left to test after this (roughly 5 steps)
[ff6b31f775f532bb8748e8ef85911ffb35a8c646] meta: Update npm's lockfile to ranges from versions
running ../bisect_test
Bisecting: 14 revisions left to test after this (roughly 4 steps)
[f9de7ef3a1089ceb2610cd27bbd4b4bc2979c4de] version: Let version succeed when `package-lock.json` is gitignored
running ../bisect_test
Bisecting: 7 revisions left to test after this (roughly 3 steps)
[471ee1c5b58631fe2e936e32480f3f5ed6438536] install/deps: fix bug where optional status not saved to package-lock
running ../bisect_test
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[0da38b7b4aff0464c60ad12e0253fd389efd5086] test: Ensure optional status is saved to lock file
running ../bisect_test
Bisecting: 1 revision left to test after this (roughly 1 step)
[9dea95e319169647bea967e732ae4c8212608f53] is-only-optional: Read optional status from shrinkwrap when we have to
running ../bisect_test
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[3888d20517593095038caa3cff68d697ae8769a8] inflate-shrinkwrap: Synthetic link deps get the right realpath
running ../bisect_test
3888d20517593095038caa3cff68d697ae8769a8 is the first bad commit
commit 3888d20517593095038caa3cff68d697ae8769a8
Author: Rebecca Turner <me@re-becca.org>
Date:   Wed Apr 11 17:51:06 2018 -0700

    inflate-shrinkwrap: Synthetic link deps get the right realpath

:040000 040000 aa2c50ece1903e2ecfd4bae6b5d439f11381511d 0c429da2ebace27245b47bd0210f032223150de9 M	lib
bisect run success

I’m not going to pretend I understand what that commit does, so someone else will probably have to take it from here. :slight_smile:


Thanks @iarna! I gave your iarna/enoent-on-link-up branch a try and it does fix the problem for me!


I have same issue.

Current workarounds:

My motivation for using npm + file: is that it work with dependabot.
But, itself(npm + file:) does not work correctly.


I’m using 6.6.0-next.1 and I still can’t install local dependencies.
Here is a minimal repo showing the problem: https://github.com/baptistemarchand/npm-local-path-bug-repro
I have two packages A and B.
A depends on B, using a local path (file:../).
B depends on left-pad (sorry :blush: ).

.
├── a
│   ├── package-lock.json
│   └── package.json
└── b
    └── package.json

When I run npm install in package A it succeeds (there are no errors), the symlink to package B is created in node_modules (as expected) but the dependencies of B are not installed. The tree now looks like this :

..
├── a
│   ├── node_modules
│   │   └── b -> ../../b
│   ├── package-lock.json
│   └── package.json
└── b
    └── package.json

If I use npm ci instead of npm install it works as expected, producing the following tree:

.
├── a
│   ├── node_modules
│   │   └── b -> ../../b
│   ├── package-lock.json
│   └── package.json
└── b
    ├── node_modules
    │   └── left-pad
    │       ├── ....
    └── package.json

It also installs correctly if I remove the package-lock.json in package A.

Am I right to think that this is caused by the same bug discussed above?

Thank you for your help.


Neither PR appears to resolve this, unless I’ve misunderstood how to use them. I’d rate the probability of my misunderstanding to be ~0.7. I cloned the repo, switched to the branch, and ran node .../npm/cli ci.


(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)


I am seeing the same issue with npm v6.2.0


(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)


I am still hitting this error on Travis builds when using file: dependencies, and always hit it locally on subsequent npm installs as well. Looking around it seems like this has been an issue for over a year – is this slated to be fixed?


@jhecking @eamodio @johnrjj @jtwebman @erikerikson Let’s vote for this issue to draw more attention from NPM team! They seem to have recently added voting functionality - it’s next to the title of the issue. Sorry for spamming :)


Any news on a fix for this? I too can reproduce this.


Just adding a link to the “first bad commit” identified by git bisect:
https://github.com/npm/cli/commit/3888d20517593095038caa3cff68d697ae8769a8

Hopefully @iarna has some idea what that commit does and why it might cause this problem?


I opened another PR with tests for it as the old one seems to have stalled.

https://github.com/npm/cli/pull/86


Note: This seems to be a regression since npm@5.10.0-next.0. I can’t repro it with npm@5.9.0-next.0 or anything reasonably earlier.

I’d appreciate some help if someone was interested in looking at the changelog for 5.10.0-next.0 and see if anything jumps out at them. (I’m just triaging right now).


I’m currently using webpack resolve alias as a workaround which works quite well: https://webpack.js.org/configuration/resolve/


Hi! I have a similar problem (I think).

I have a project A that depends of a package B using local path (file:…). And package B has also some locals dependencies on packages C and D also with local path (file:…) something like:

├── a
    └── package.json
         └── "b": "file:packages/b.tgz"
             └── package.json
                └── "c": "file:modules/c.tgz"
                └── "d": "file:modules/d.tgz"

I’ve tried removing all: package-lock.json, node_modules, clear npm cache, and nothing seems to work. I keep geting this error:

3037 verbose stack Error: ENOENT: no such file or directory, stat 'C:\xampp\htdocs\test-module\packages\b.tgz\modules\c.tgz'
3038 verbose cwd C:\xampp\htdocs\test-module
3039 verbose Windows_NT 10.0.17134
3040 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\abcd\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "install"
3041 verbose node v8.12.0
3042 verbose npm  v6.9.0
3043 error path C:\xampp\htdocs\test-module\packages\b.tgz\modules\c.tgz
3044 error code ENOENT
3045 error errno -4058
3046 error syscall stat
3047 error enoent ENOENT: no such file or directory, stat 'C:\xampp\htdocs\test-module\packages\b.tgz\modules\c.tgz'
3048 error enoent This is related to npm not being able to find a file.
3049 verbose exit [ -4058, true ]

Any help?
Thanks :slight_smile: