Some packages have dist.tarball as http and not https

registry

(John Hunter) #21

Any indication when this is going to be resolved? Manually adjusting the lock file (or using a script to do it) is not a long term solution. With contributors on different npm versions we get constant flip-flopping on the committed lock file.


(ELLIOTTCABLE) #22

Same here, @johnhunter; I’m wondering if this is a different issue? The primary issue above seems to be deterministic; that is, certain packages should, apparently, always be either http or https? But in our team, we have constant diff-noise of this form:

 "minimist": {
   "version": "1.2.0",
-  "resolved": "http://registry.npmjs.org/minimist/-/minimist-1.2.0.tgz",
+  "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.0.tgz",
   "integrity": "sha1-o1AIsg9BOD7sH7kU9M1d95omQoQ="
},

Note that the version-number nor integrity have changed; this is the same version.

(Was there a recent change to this behaviour? One of the clients downgrading these same packages to http is at npm 6.3.0; one constantly committing re-upgrades to https is at npm 6.4.1.)


(Pavlo Zhukov) #23

Removing node_modules directory, mentioned here, helps me to resolve the problem


(Phil Léger) #24

I wish npm would go back to github issues like everyone else… googling issues gets you to the npm/npm repo and then you have to search for hours in here… the bug discussion don’t show if a resolution has been found or not, I don’t like the experience…

so has this been fixed? this issues has been opened since May 2018 https://github.com/npm/npm/issues/20719

thank you…


(Phil Léger) #25

I can confirm this happens on npm 6.4.1 on windows


(Phil Léger) #26

When I run the same command on my mac, save version of npm and node, it stays https …

** edit nevermind, the problem is still there on MacOS npm version 6.4.1

there are also changes to the dev and optional properties… not sure what they are for.


(Piotr Katolik) #27

I am having the same issue comparing package-lock.json files between mac os and linux origin.


(Wei-Cheng Pan) #29

Just remove package-lock.json from VCS.
It causes more trouble than it solves.

If someone insists to have a lock file, just dd if=/dev/urandom of=package-lock.json. npm does not generate it deterministically anyway.


(Phil Léger) #30

Removing it is not an option, I rely on npm ci to make sure I’m shipping exact versions of dependencies, I’m not sure why you are suggesting this.

Thanks.


(Guido Bouman) #31

@zkat I love all of the work being done on npm, great improvements.

Yet 3 months since your comment that the fix would be easy and would roll out eventually to everyone, we’re still stuck with flipflopping http & https in package-lock.json files. This happens between developers that have identical versions of node & npm installed.

Is there a proposed resolution already? (Maybe we need to clear npm local machine caches?)

While I love npx, this issue is something that hits our teams on a day to day basis, while I only use npx once every month, at maximum. May I ask what the priorities are on these issues that hit nearly all developers that work together with other developers on projects?


(Kat Marchán) #32

This is pretty low priority because it’s largely an aesthetic issue. You’re not making requests to http unless your npm config get registry says http:. I have no ETA for you.


(Phil Léger) #33

???

it’s creating git conflicts between Mac users, Windows users and sometimes CI, whenever a npm install is run

why would this be aesthetic? this is a productivity issue


(Gavin Aiken) #34

This may be considered low priority at npm, but I agree with Phil - it’s really irritating for users. Churn on package-lock files every time someone runs npm i causes unnecessary noise on commits unless developers are careful and fix the http / https changes. I’d ask for you to reconsider the priority on this and get it escalated. Even as a low priority issue, for it to be affecting the community for more than 6 months doesn’t seem great.


(Kat Marchán) #35

Note that modern versions of the CLI can automatically resolve git conflicts on the pkglock for you with npm i --package-lock-only. Additionally, you can replace all instances of http: with https: by hand and have it work just fine.


(Guido Bouman) #36

While there’s no security risks regarding https or https, I wouldn’t say this is just an aesthetic issue. Our team alone spends multiple hours a week working around this bug. I have not seen any workaround that allows our developers to ignore the package-lock files. Junior developers are tripped up by this, and we have to explain them that this is a bug in npm, and that they should ignore these changes. Juniors easily commit these files, and this bug puts an extra burden on them and their PR reviewers / seniors.

Low priority bugs like this one (that keep on hanging around) are the exact reason yarn was able to gain so much popularity in a short time. I’m all for npm as the standard, but please reconsider the importance of tiny bugs that hit developers on a day to day basis.


(Greg Ferreri) #37

This is definitely not just an aesthetic issue. This can and does lead to lost productivity in discerning real changes to package-lock.json vs the superfluous http-https flip-flopping when reviewing PRs for instance.


(Phil Léger) #38

npm install is litterally the only command we need to behave 100% with no bug, the rest: npx, npm ci, npm audit, those are new features, but npm install should our only true God

and I don’t want to switch to other package manager… we have better problem to solve than to deal with npm vs yarn :grimacing: :pray:


(Audrey Eschright) #39

Hi everyone, we’re discussing this internally so I want to give you a summary.

The changing http/https urls you’re seeing recorded come from inconsistent data in the registry. I don’t have an ETA on that fix, but someone else will chime in if they can. It’s important to note that the actual request for that url is being converted to https when appropriate, so this is only an issue of the data string being recorded to your package-lock.json.

The CLI team would like to figure out a way we can smooth this over so you only see the http/https url prefix you had in the file previously – the behavior on the backend will remain consistent. If we can figure out a straightforward way to accomplish this, a community member could submit a PR.

We’ll update when we have more. Thanks for your patience!


(Lars Willighagen) #40

I’ve got an idea:


(Guido Bouman) #41

I really like @larsgw’s idea. As internally https is always used, it would be great if npm would just always output https urls, no matter what the registry returns. I know this is usually not npm’s responsibility as it can work with multiple registries. But this would release the burden from the registry team, and allow them to solve it within their own timeframe

On the other hand, I’m still confused why different machines get different urls from the registry. Load balanced machines/endpoints that serve different urls from corrupted caches? Registry responses that are stored locally in the npm cache on the machine?

And @zkat: Thanks for pointing me to --package-lock-only, that will allow us to solve this for our juniors! This is exactly the other way around, it only updates the package-lock. But I want to do an install without updating the package-lock, essentially npm ci, without removing the node_modules folder. But that does not seem to exist. Also, npm ci is not an option each time we start our project because we have a huge set of dependencies: over 20 secs of installation when removing the node_modules folder.