What I Wanted to Do
Get consistent results for scoped package versions from the npm registry.
What Happened Instead
Recently published scoped packages can sometimes “disappear” temporarily soon after they first appear, leading to
npm install errors for projects that have already updated to the latest.
It cannot be reproduced simply. I see this happening a lot as part of a dependency update bot/service I run over thousands of repositories. It only happens for scoped dependencies and not non-scoped.
My service creates PRs when it detects new versions of npm packages available for a repository, and it also uses an npm follower to get them pretty quickly. Sometimes the new version is absent from the registry response in a subsequent check (e.g. an hour later) and that results in either:
- The previous PR being closed automatically, as it seems no longer applicable, OR
- A new “rollback” PR being necessary if the previous PR was already merged
It’s almost always scoped packages that this happens for, which makes it highly unlikely that it’s a problem with my service/cache. If my client had a mistaken cache then you would expect to see 90%+ of the roll back PRs being non-scoped as a proportion of total, whereas they are 99% scoped.
An example of this happening on 2018-07-16 was with the
@babel packages. In one example I looked at, at 18:03 UTC
7.0.0-beta.54 were detected, and at 19:39 UTC a result was returned showing
7.0.0-beta.53 as “latest”. At 20:10 UTC I triggered another check where
7.0.0-beta.54 was returned again.
This is causing big problems for npm users, because subsequent
npm install commands can fail. Having it happen to scoped packages is particularly unfortunate because these are often “internal” packages and users do want to merge them immediately - sometimes automatically - after tests pass, so that downstream packages can use them immediately. Holding back updates of scoped packages for ~2 hours to work around this problem is not feasible.
I don’t expect this to be an easy one to be solved but I think it’s imperative considering that private scoped modules is npm’s main revenue stream and it seems to not be working reliably. Maybe certain npmjs caches are not updating correctly so it’s based on chance?
This is not using the
npm CLI tool - queries are done directly using registry REST API.