npm Community Forum (Archive)

The npm community forum has been discontinued.

To discuss usage of npm, visit the GitHub Support Community.

Running npm ci without deleting old node_modules folder #20104

Hi, we would like to use the new npm ci command in our Jenkins build but we cannot delete the node_modules folder.

I found a description of the issue we encounter in the following archived Github issue : https://github.com/npm/npm/issues/20104.

We have an other use case where the whole deletion of the node_modules directory is problematic.

To speed-up our npm install executions in Jenkins we map a tmpfs Docker volume to the node_modules folder.

Our configuration is similar to the following (Jenkins starts a Docker container and then npm ci is executed inside it) :

docker run -t -d -u 500:500 --tmpfs=/project_directory/node_modules:exec,gid=500,uid=500 -v /home/ec2-user/efs/.npm:/.npm ... node:11

With this mapping our npm install commands are very very fast because all file copies are done completely in memory.

But because the tmpfs volume is created at container start it cannot be deleted by npm ci after.

To solve our problem it would require one of the following

What do you think about that ?

Thanks,

Baptiste


I would take a patch that does rm -rf node_modules/**/* as the new default for npm ci, tbh. It’s a thing that’s been mentioned before.


Hi @zkat and thanks for your reactivity on this !

Ok, I think I found the mentioned you talked about, here are the links I found to add a little more context for others who would need this feature too :

Thanks,

Baptiste


Hi, ususre if there is any traction on this. This has been opened since November last year and I see some github issues as old as March 2018.
We have a similar issue where we mount node_modules in docker as an external volume and due to this can’t use npm ci. I think npm ci should definitely delete contents of node_modules rather than the directory itself.


It would be great to have this. Our CI system (concourse) mounts the node_modules cache as a volume, hence we cannot simply delete the entire directory.


This is a problem for our team’s containerised builds too - any chance we could merge @zkat patch:

thanks


I’m managing CI servers with a large number js projects and i’m looking to optimize the duration and the IO spent on npm install, which represents a significant part of the total amount of the build.

I’ve been looking into npm ci as a replacement of npm install as soon as i heard of it, but was disappointed by the fact that npm ci deletes the whole node_modules, which makes it less efficient than npm install when a node_modules set up by a previous CI build is available.

I also just discovered this cipm and give it a try. Here’s the results of the tests I made on one js project installing 1200 modules. I measured the real user time, cpu time, system cpu time, and read and written MB by the process:

Main points:

The fact that cipm, as npm ci, deletes the whole node_modules, has a huge impact on duration and IO spent.
The best combination for now would be to use cipm during the initial build when there’s no node_modules, and then use npm install.
But i think cipm could be faster than npm install if it did not delete node_modules.

Would it be possible to add a parameter to not delete node_modules?
As most of the files there won’t change, there must be a smarter way to do this than deleting and reinstalling everything.

Also filled in cipm, but not much activity there : https://github.com/npm/libcipm/issues/3


I agree, this would be useful.
I also found this issue: npm/npm#20104
But the npm/npm repo has been archived…
I don’t think the history/interest of that issue should be lost, though. Would like to see this --no-delete option in the near future.


Hi! Just wanted to mention this is currently being reviewed by the CLI team and will likely be merged soon!


What I do is that I have the node_modues folder being cached based on the checksum of the package-lock.json; then whenever the lock file changes, I run npm ci and save a new cache based on the new checksum of the lock file. When it doesn’t change, I just restore the cache and skip installing dependencies. In this setup, since I didn’t restore the cache if the lock file changes, when I run npm ci there would be no node_modules folder to delete.