npm ci fail to local packages

(Felipe Gonçalves Marques) #1

What I Wanted to Do

I have a project organized into two modules: ‘parent’ and ‘child’, where the child depends
on the parent project and the parent depends on a library (fs-extra). I’m using npm@6.9.

The folder structure is the following:

-src
----parent
----child

In child, I run npm install ../parent, which work fine.
I was hopping for npm ci to work fine as well.

What Happened Instead

Instead, it gave me the following error:

npm info it worked if it ends with ok
npm verb cli [ '/Users/felipe.marques/.nvm/versions/node/v10.14.1/bin/node',
npm verb cli   '/Users/felipe.marques/.nvm/versions/node/v10.14.1/bin/npm',
npm verb cli   'ci',
npm verb cli   '-ddd' ]
npm info using npm@6.9.0
npm info using node@v10.14.1
npm verb npm-session 7c09017a04b5df73
npm info prepare initializing installer
npm verb prepare starting workers
npm verb prepare installation prefix: /Users/felipe.marques/Desktop/Projects/so/child
npm verb prepare using package-lock.json
npm verb checkLock verifying package-lock data
npm verb teardown shutting down workers.
npm info teardown Done in 0s
npm verb stack Error: fs-extra not accessible from parent
npm verb stack     at reqAddr (/Users/felipe.marques/.nvm/versions/node/v10.14.1/lib/node_modules/npm/node_modules/npm-logical-tree/index.js:159:17)
npm verb stack     at Object.keys.forEach.name (/Users/felipe.marques/.nvm/versions/node/v10.14.1/lib/node_modules/npm/node_modules/npm-logical-tree/index.js:129:22)
npm verb stack     at Array.forEach (<anonymous>)
npm verb stack     at addChild (/Users/felipe.marques/.nvm/versions/node/v10.14.1/lib/node_modules/npm/node_modules/npm-logical-tree/index.js:128:40)
npm verb stack     at Array.from.forEach.name (/Users/felipe.marques/.nvm/versions/node/v10.14.1/lib/node_modules/npm/node_modules/npm-logical-tree/index.js:113:5)
npm verb stack     at Array.forEach (<anonymous>)
npm verb stack     at lockTree (/Users/felipe.marques/.nvm/versions/node/v10.14.1/lib/node_modules/npm/node_modules/npm-logical-tree/index.js:107:5)
npm verb stack     at then.then.then.then (/Users/felipe.marques/.nvm/versions/node/v10.14.1/lib/node_modules/npm/node_modules/libcipm/index.js:163:21)
npm verb cwd /Users/felipe.marques/Desktop/Projects/so/child
npm verb Darwin 18.2.0
npm verb argv "/Users/felipe.marques/.nvm/versions/node/v10.14.1/bin/node" "/Users/felipe.marques/.nvm/versions/node/v10.14.1/bin/npm" "ci" "-ddd"
npm verb node v10.14.1
npm verb npm  v6.9.0
npm ERR! fs-extra not accessible from parent
npm verb exit [ 1, true ]
npm timing npm Completed in 682ms

Reproduction Steps

mkdir project
cd project
mkdir child
mkdir parent
cd parent
npm create
npm install fs-extra
cd ../child
npm create
npm install ../parent
npm ci

Details

This problem happens in npm@6.9, although it does not happen for npm@6.4.
Comparing the package-lock.json for each npm version, I notice the following differences:
In version npm@6.4, the entry for the parent package is:

    "parent": {
      "version": "file:../parent",
      "requires": {
        "fs-extra": "^7.0.1"
      },
      "dependencies": {
        "fs-extra": {
          "version": "7.0.1",
          "bundled": true,
          "requires": {
            "graceful-fs": "^4.1.2",
            "jsonfile": "^4.0.0",
            "universalify": "^0.1.0"
          }
        },
        "graceful-fs": {
          "version": "4.1.15",
          "bundled": true
        },
        "jsonfile": {
          "version": "4.0.0",
          "bundled": true,
          "requires": {
            "graceful-fs": "^4.1.6"
          }
        },
        "universalify": {
          "version": "0.1.2",
          "bundled": true
        }
      }
    }

While in the npm@6.9, it is:

    "parent": {
      "version": "file:../parent",
      "requires": {
        "fs-extra": "^7.0.1"
      }
    }

From the stack, I also find the following code in npm-logical-tree:

function reqAddr (pkgLock, name, fromAddr) {
  const lockNode = atAddr(pkgLock, fromAddr)
  const child = (lockNode.dependencies || {})[name]

  if (child) {
    return `${fromAddr}:${name}`
  } else {
    const parts = fromAddr.split(':')
    while (parts.length) {
      parts.pop()
      const joined = parts.join(':')
      const parent = atAddr(pkgLock, joined)
      if (parent) {
        const child = (parent.dependencies || {})[name]
        if (child) {
          return `${joined}${parts.length ? ':' : ''}${name}`
        }
      }
    }
    const err = new Error(`${name} not accessible from ${fromAddr}`)
    err.pkgLock = pkgLock
    err.target = name
    err.from = fromAddr
    throw err
  }
}

Where the lockNode in the parententry, so in version 6.9, the missing of dependencies field is responsible for the error.

I don’t know why this change in the package-lock.json format for local dependencies, this is why I
didn’t provide a patch to fix this. But if someone could explain me the structural change and how
this bug should be solved, I would love to send a PR.

Platform Info

npm versions: 6.9 and 6.4
node version: 10.14.1

$ npm --versions
<!-- paste output here -->
$ node -p process.platform
<!-- paste output here -->
6.8.0: `npm ci` fails with local dependency
(Pirobox) #2

I asked a question on stack overflow relative to this issue.

After further analysis, I discovered that command npm ci, in the scenario described here with locally referencing projects, works correctly with npm 6.7, but starts breaking with version 6.8 and successive versions.

(Lars Willighagen) #3

Looks like

You are correct, the lock file is missing a dependencies field. Without it, npm ci cannot find some of the required packages in the package tree, and so fails to verify the lock.

(Felipe Gonçalves Marques) #4

Hi, Lars, thanks for the reply.

So it should generate the dependencies field?
I didn’t start to dig in into the code, because I was not sure if this is a bug or a decision to remove this field from the file.

1 Like
(Joel Marcotte) #5

I’d like to know this as well. For the sake of me I can’t find documentation on what is the intended behaviour for package-lock wrt file: dependencies.

I suspect the issue cropped up from this commit https://github.com/npm/cli/commit/d5137091dd695a2980f7ade85fdc56b2421ff677 but I don’t understand why removing dependencies was required and/or intended for the fix it was trying to make.

(Joel Marcotte) #6

You are correct, the lock file is missing a dependencies field.

Can you tell us whether this was a deliberate outcome or not? I’m wondering if I should look into fixing this in npm or libcipm.

1 Like