npm ci fails with transitive local packages

triaged
cli
priority:low

(Auke van Leeuwen) #1

What I Wanted to Do

I’m using a bit of a mono-repo layout with multiple modules that have a close relation to each other. I wanted to install the packages from the package-lock.json using npm ci in the CI build to create reproducible builds. I expected all dependencies to be installed correctly.

I have the following structure:

β”œβ”€β”€ common
β”‚   └── node_modules
β”‚       └── npmcitest-utils -> ../../utils
β”œβ”€β”€ module1
β”‚   └── node_modules
β”‚       β”œβ”€β”€ npmcitest-common -> ../../common
β”‚       └── npmcitest-utils -> ../../utils
└── utils

All of these β€˜modules’ (utils, common, module1) are separate npm modules with their own package.json. I can npm install all of these modules fine with

for i in utils common module1; do 
  (
    echo -e "\n\nBuilding $i"
    cd $i
    rm -rf node_modules
    npm install
  )
done

# all good here

However after the package-lock.json files are generated I cannot run npm ci:

for i in utils common module1; do
  (
    echo -e "\n\nBuilding $i"
    cd $i
    rm -rf node_modules
    npm ci
  )
done


Building utils
added 0 packages in 0.036s (3.3 KB)
 

Building common
added 1 packages in 0.028s


Building module1
npm ERR! path /private/tmp/npmcitest/module1/node_modules/npmcitest-common/node_modules/npmcitest-utils/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/private/tmp/npmcitest/module1/node_modules/npmcitest-common/node_modules/npmcitest-utils/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/avanleeuwen/.npm/_logs/2018-12-13T12_22_51_711Z-debug.log

Log: 2018-12-13T12_22_51_711Z-debug.log

What Happened Instead

npm ci fails (see above). I know the following symbolic links:

tree -a
.
β”œβ”€β”€ common
β”‚   β”œβ”€β”€ node_modules
β”‚   β”‚   └── npmcitest-utils -> ../../../../utils
β”‚   β”œβ”€β”€ package-lock.json
β”‚   └── package.json
β”œβ”€β”€ module1
β”‚   β”œβ”€β”€ node_modules
β”‚   β”‚   β”œβ”€β”€ npmcitest-common -> ../../common
β”‚   β”‚   └── npmcitest-utils -> ../../utils
β”‚   β”œβ”€β”€ package-lock.json
β”‚   └── package.json
└── utils
    β”œβ”€β”€ package-lock.json
    └── package.json

7 directories, 7 files

Of which the top one is incorrect.

Reproduction Steps

  1. Clone https://github.com/aukevanleeuwen/npmcitest
  2. Checkout the test1 branch
  3. Run the commands above

Details

I also have a different error message when I include other dependencies. When I include other dependencies I get the following error:

  1. Checkout the test2 branch
  2. Run the commands above
npm ERR! path /private/tmp/npmcitest/module1/node_modules/npmcitest-common/node_modules/npmcitest-utils
npm ERR! code EEXIST
npm ERR! errno -17
npm ERR! syscall mkdir
npm ERR! EEXIST: file already exists, mkdir '/private/tmp/npmcitest/module1/node_modules/npmcitest-common/node_modules/npmcitest-utils'
npm ERR! File exists: /private/tmp/npmcitest/module1/node_modules/npmcitest-common/node_modules/npmcitest-utils
npm ERR! Move it away, and try again.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/avanleeuwen/.npm/_logs/2018-12-13T12_58_31_135Z-debug.log

Platform Info

$ npm --versions
{ npm: '6.5.0',
  ares: '1.15.0',
  cldr: '34.0',
  http_parser: '2.8.0',
  icu: '63.1',
  llhttp: '1.0.1',
  modules: '67',
  napi: '3',
  nghttp2: '1.34.0',
  node: '11.4.0',
  openssl: '1.1.0j',
  tz: '2018e',
  unicode: '11.0',
  uv: '1.24.0',
  v8: '7.0.276.38-node.13',
  zlib: '1.2.11' }
$ node -p process.platform
darwin

(Auke van Leeuwen) #2

I could not add another attachment (as a first time user) here, so uploading it again here for the test2 case:

2018-12-13T12_58_31_135Z-debug.log (9.0 KB)


(Auke van Leeuwen) #3

I just now see that it will β€˜autoclose’ after 3 months. Anything I can do to get this fixed?


(Lars Willighagen) #4

I have this bookmarked and was planning to pick this up when I have the time. However, there’s a bunch of personal stuff that has higher priority for me at the moment. You can, of course, start debugging and/or submit a fix yourself, but other than that, if there’s no-one picking this up earlier then I am, no, not anything I can think of.