Some packages have dist.tarball as http and not https

Note that modern versions of the CLI can automatically resolve git conflicts on the pkglock for you with npm i --package-lock-only. Additionally, you can replace all instances of http: with https: by hand and have it work just fine.

While there’s no security risks regarding https or https, I wouldn’t say this is just an aesthetic issue. Our team alone spends multiple hours a week working around this bug. I have not seen any workaround that allows our developers to ignore the package-lock files. Junior developers are tripped up by this, and we have to explain them that this is a bug in npm, and that they should ignore these changes. Juniors easily commit these files, and this bug puts an extra burden on them and their PR reviewers / seniors.

Low priority bugs like this one (that keep on hanging around) are the exact reason yarn was able to gain so much popularity in a short time. I’m all for npm as the standard, but please reconsider the importance of tiny bugs that hit developers on a day to day basis.

6 Likes

This is definitely not just an aesthetic issue. This can and does lead to lost productivity in discerning real changes to package-lock.json vs the superfluous http-https flip-flopping when reviewing PRs for instance.

7 Likes

npm install is litterally the only command we need to behave 100% with no bug, the rest: npx, npm ci, npm audit, those are new features, but npm install should our only true God

and I don’t want to switch to other package manager… we have better problem to solve than to deal with npm vs yarn :grimacing: :pray:

Hi everyone, we’re discussing this internally so I want to give you a summary.

The changing http/https urls you’re seeing recorded come from inconsistent data in the registry. I don’t have an ETA on that fix, but someone else will chime in if they can. It’s important to note that the actual request for that url is being converted to https when appropriate, so this is only an issue of the data string being recorded to your package-lock.json.

The CLI team would like to figure out a way we can smooth this over so you only see the http/https url prefix you had in the file previously – the behavior on the backend will remain consistent. If we can figure out a straightforward way to accomplish this, a community member could submit a PR.

We’ll update when we have more. Thanks for your patience!

3 Likes

I’ve got an idea:

https://github.com/zkat/pacote/pull/167

1 Like

I really like @larsgw’s idea. As internally https is always used, it would be great if npm would just always output https urls, no matter what the registry returns. I know this is usually not npm’s responsibility as it can work with multiple registries. But this would release the burden from the registry team, and allow them to solve it within their own timeframe

On the other hand, I’m still confused why different machines get different urls from the registry. Load balanced machines/endpoints that serve different urls from corrupted caches? Registry responses that are stored locally in the npm cache on the machine?

And @zkat: Thanks for pointing me to --package-lock-only, that will allow us to solve this for our juniors! This is exactly the other way around, it only updates the package-lock. But I want to do an install without updating the package-lock, essentially npm ci, without removing the node_modules folder. But that does not seem to exist. Also, npm ci is not an option each time we start our project because we have a huge set of dependencies: over 20 secs of installation when removing the node_modules folder.

1 Like

Does --no-package-lock work (I believe that’s how Boolean flags work)?

Nope, that would completely ignore the package-lock file during install and you would end up with the most recent packages. More or less the same thing that happens when one removes the package-lock file.

If anyone’s interested, I made a workaround by adding the following to my package.json scripts (same idea as above, but works on all platforms)

	     "postshrinkwrap": "replace --silent 'http://' 'https://' ./package-lock.json",

Make sure to npm install --save-dev replace

4 Likes

Wow, @coderkevin! I’m so glad I had this issue 2 hours after you commented. That saved me a lot of time, thanks!

1 Like

Great suggestion Kevin, thank you!

using native find command
"postshrinkwrap": "find . -name node_modules -prune -o -name package-lock.json -exec sed -i 's/http:\/\//https:\/\//g' {} +"

I want to update with a case, and ask for a workaround since it is aesthetic. If there is a functionality lose you can only conisider it as minor if there is a strict workaround.

Here is the details:

We are using nexus as a corporate repository and http protocol is disabled. So considering https support of npm we enable proxy without direct access. But due to invalid entries in npm metadata nexus cannot download tarballs, we cannot have even initial build to have package-lock.json

Regards.

4 Likes

Hey all, the rewrite job that was fixing this issue finished running. I recommend you fix any remaining issues you have by hand, and it should not repeat itself going forward (barring cached versions – use --prefer-online to make sure you’re not getting anything stale).

So I consider this issue resolved now. We likely won’t be making CLI-side changes related to this, so we should be all set.

10 Likes

I have updated to the latest version of NPM (6.5.0), force cleaned caches and run the install command with --prefer-online. Still, some of the URLs in package lock got converted to “http” protocol.

Am I doing something wrong?

Remove the node_modules/ folder, package-lock.json and run ‘npm install --prefer-online’ again, it worked for me.

1 Like

Please remove node_modules as well (along with doing everything else).

If this continues, please tell me what specific packages you experienced this with and I’ll take a look.

1 Like

should I be using this on every install now?