npm install downgrading resolved packages from https to http registry in package-lock.json

registry
priority:low
triaged

(Stefan Maric) #1

What I Wanted to Do

npm install to generate a fresh package-lock.json with https urls only.

What Happened Instead

Some packages have their resolved field pointing to the http registry.

Reproduction Steps

mkdir httptest
cd httptest
npm init -y
# Some packages that were resolved to http registry on a project
npm install babel-plugin-syntax-object-rest-spread buffer@4.9.1 chalk@1.1.3 external-editor@2.2.0 minimist mkdirp
# Just a control package to show it is not for all packages
npm install lodash
# Display http urls
cat package-lock.json | grep 'http://'
# Display https ones just for control
cat package-lock.json | grep 'https://'

I get the following output:

Wrote to /home/sam/Code/httptest/package.json:

{
  "name": "httptest",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}


npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN httptest@1.0.0 No description
npm WARN httptest@1.0.0 No repository field.

+ babel-plugin-syntax-object-rest-spread@6.13.0
+ mkdirp@0.5.1
+ minimist@1.2.0
+ external-editor@2.2.0
+ buffer@4.9.1
+ chalk@1.1.3
added 21 packages in 0.953s
npm WARN httptest@1.0.0 No description
npm WARN httptest@1.0.0 No repository field.

+ lodash@4.17.10
added 1 package in 0.517s
      "resolved": "http://registry.npmjs.org/babel-plugin-syntax-object-rest-spread/-/babel-plugin-syntax-object-rest-spread-6.13.0.tgz",
      "resolved": "http://registry.npmjs.org/buffer/-/buffer-4.9.1.tgz",
      "resolved": "http://registry.npmjs.org/chalk/-/chalk-1.1.3.tgz",
      "resolved": "http://registry.npmjs.org/external-editor/-/external-editor-2.2.0.tgz",
      "resolved": "http://registry.npmjs.org/minimist/-/minimist-1.2.0.tgz",
      "resolved": "http://registry.npmjs.org/mkdirp/-/mkdirp-0.5.1.tgz",
          "resolved": "http://registry.npmjs.org/minimist/-/minimist-0.0.8.tgz",

      "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-2.1.1.tgz",
      "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-2.2.1.tgz",
      "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.3.0.tgz",
      "resolved": "https://registry.npmjs.org/chardet/-/chardet-0.4.2.tgz",
      "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz",
      "resolved": "https://registry.npmjs.org/has-ansi/-/has-ansi-2.0.0.tgz",
      "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.24.tgz",
      "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.1.12.tgz",
      "resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz",
      "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.10.tgz",
      "resolved": "https://registry.npmjs.org/os-tmpdir/-/os-tmpdir-1.0.2.tgz",
      "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
      "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-3.0.1.tgz",
      "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-2.0.0.tgz",
      "resolved": "https://registry.npmjs.org/tmp/-/tmp-0.0.33.tgz",

Details

It seems to happen with specific packages, specific versions (thus I specified some version above), and after a specific point in time. I recently created a fresh package-lock.json in a project, and the exact same version of those packages were resolved to the https registry at that time (using same node and npm version, and using same package.json).

Platform Info

$ npm --versions
{ httptest: '1.0.0',
  npm: '5.6.0',
  ares: '1.10.1-DEV',
  cldr: '32.0',
  http_parser: '2.8.0',
  icu: '60.1',
  modules: '57',
  napi: '3',
  nghttp2: '1.32.0',
  node: '8.11.4',
  openssl: '1.0.2p',
  tz: '2017c',
  unicode: '10.0',
  uv: '1.19.1',
  v8: '6.2.414.54',
  zlib: '1.2.11' }

$ node -p process.platform
linux

But I was also able to replicate with latest node 10.9.0 and npm 6.2.0 and npm 6.4.1.


Tarball URLs on registry randomly switch from HTTPS to HTTP
Some packages have dist.tarball as http and not https
package-lock.json keeps changing between platforms and runs
(Jan Lehnardt) #2

we’re seeing the same on various combinations of node and npm, any pointers here would be appreciated.


(Elia Maino) #3

There’s a similar issue opened on GitHub since a while (20 mar): https://github.com/npm/npm/issues/20106

Some people had the problem solved by performing the steps described in https://github.com/npm/npm/issues/20106#issuecomment-383770315 (not me anyway)

It would be definitely nice to have an update on this.


(Stefan Maric) #4

For those looking to workaround it, I’ve been simply replacing the URLs after an npm install that creates/updates the package-lock (and running npm i again) and haven’t experienced issues so far:

sed -i -e 's/http:\/\//https:\/\//g' package-lock.json

But indeed this behavior is odd, creates security concerns, and seems somehow related to the registry and not to the npm CLI itself.


(David Cannady) #5

The same issue has been plaguing my projects as well. This is a security issue and should be considered critical in terms of bug priority.


(Luis Lobo Borobia) #6

I have been seeing the same behavior.


(Amith George) #7

I faced the same issue today. I can confirm that after performing the steps listed in that comment, package-lock.json didn’t have any changes.

Just for future reference, I am pasting the steps as listed in the comment

$ rm -rf node_modules/
$ npm cache clean --force
(Revert the changes in your package-lock.json file)
$ npm i

(Jayden Seric) #8

I tweeted about this issue after noticing it the other day: https://twitter.com/jaydenseric/status/1038939403974897664

npm cache clean --force and a fresh install does not fix the issue.

  • Node.js v10.10.0
  • npm v6.4.1

(Stefan Maric) #9

Searching around, I noticed this is a duplicated: Some packages have dist.tarball as http and not https

Indeed this is an issue with the registry itself and applies to any npm CLI version.

Maybe unrelated, but I also noticed the registry responds with diff content-types for diff packages:

$ curl --silent --head https://registry.npmjs.com/buffer | grep -i content-type
content-type: application/octet-stream

$ curl --silent --head https://registry.npmjs.com/lodash | grep -i content-type
content-type: application/json

So maybe it is a faulty instance behind a load balancer the source of trouble.

PD: Just saw @nexdrew just linked this thread there.


(Will Morgan) #10

I tried this with NPM v6.4.1 and it’s still an issue. I work around it by just doing a grep replace http: to https:. Strangely when I re-run an npm install, it still tried to change a random(?) package’s URL from https back to http. In my case, it was Bluebird.


(Marian Schubert) #11

I’m having same problem.

$ npm --version
6.4.1


(Alf Eaton) #12

It is weird that different versions of a package appear to switch between http and https:

> npm view react-scripts@* dist.tarball
react-scripts@0.0.0 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.0.0.tgz'
react-scripts@0.1.0 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.1.0.tgz'
react-scripts@0.2.0 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.2.0.tgz'
react-scripts@0.2.1 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.2.1.tgz'
react-scripts@0.2.2 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.2.2.tgz'
react-scripts@0.2.3 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.2.3.tgz'
react-scripts@0.3.0 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.3.0.tgz'
react-scripts@0.3.1 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.3.1.tgz'
react-scripts@0.4.0 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.4.0.tgz'
react-scripts@0.4.1 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.4.1.tgz'
react-scripts@0.4.2 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.4.2.tgz'
react-scripts@0.4.3 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.4.3.tgz'
react-scripts@0.5.0 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.5.0.tgz'
react-scripts@0.5.1 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.5.1.tgz'
react-scripts@0.6.0 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.6.0.tgz'
react-scripts@0.6.1 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.6.1.tgz'
react-scripts@0.7.0 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.7.0.tgz'
react-scripts@0.8.0 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.8.0.tgz'
react-scripts@0.8.1 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.8.1.tgz'
react-scripts@0.8.2 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.8.2.tgz'
react-scripts@0.8.3 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.8.3.tgz'
react-scripts@0.8.4 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.8.4.tgz'
react-scripts@0.8.5 'http://registry.npmjs.org/react-scripts/-/react-scripts-0.8.5.tgz'
react-scripts@0.9.0 'https://registry.npmjs.org/react-scripts/-/react-scripts-0.9.0.tgz'
react-scripts@0.9.1 'https://registry.npmjs.org/react-scripts/-/react-scripts-0.9.1.tgz'
react-scripts@0.9.2 'https://registry.npmjs.org/react-scripts/-/react-scripts-0.9.2.tgz'
react-scripts@0.9.3 'https://registry.npmjs.org/react-scripts/-/react-scripts-0.9.3.tgz'
react-scripts@0.9.4 'https://registry.npmjs.org/react-scripts/-/react-scripts-0.9.4.tgz'
react-scripts@0.9.5 'https://registry.npmjs.org/react-scripts/-/react-scripts-0.9.5.tgz'
react-scripts@1.0.0 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.0.tgz'
react-scripts@1.0.1 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.1.tgz'
react-scripts@1.0.2 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.2.tgz'
react-scripts@1.0.3 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.3.tgz'
react-scripts@1.0.4 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.4.tgz'
react-scripts@1.0.5 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.5.tgz'
react-scripts@1.0.6 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.6.tgz'
react-scripts@1.0.7 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.7.tgz'
react-scripts@1.0.8 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.8.tgz'
react-scripts@1.0.9 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.9.tgz'
react-scripts@1.0.10 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.10.tgz'
react-scripts@1.0.11 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.11.tgz'
react-scripts@1.0.12 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.12.tgz'
react-scripts@1.0.13 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.13.tgz'
react-scripts@1.0.14 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.14.tgz'
react-scripts@1.0.15 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.15.tgz'
react-scripts@1.0.16 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.16.tgz'
react-scripts@1.0.17 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.0.17.tgz'
react-scripts@1.1.0 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.1.0.tgz'
react-scripts@1.1.1 'http://registry.npmjs.org/react-scripts/-/react-scripts-1.1.1.tgz'
react-scripts@1.1.2 'http://registry.npmjs.org/react-scripts/-/react-scripts-1.1.2.tgz'
react-scripts@1.1.3 'http://registry.npmjs.org/react-scripts/-/react-scripts-1.1.3.tgz'
react-scripts@1.1.4 'http://registry.npmjs.org/react-scripts/-/react-scripts-1.1.4.tgz'
react-scripts@1.1.5 'https://registry.npmjs.org/react-scripts/-/react-scripts-1.1.5.tgz'
react-scripts@2.0.0 'https://registry.npmjs.org/react-scripts/-/react-scripts-2.0.0.tgz'
react-scripts@2.0.1 'https://registry.npmjs.org/react-scripts/-/react-scripts-2.0.1.tgz'
react-scripts@2.0.2 'https://registry.npmjs.org/react-scripts/-/react-scripts-2.0.2.tgz'

(Alex At Cascade) #13

Seeing the same thing, and finding it very concerning.


(St John Peaster) #14

Same thing happened to me.

I just deleted my node_modules folder then did a fresh install - and thankfully that fixed it.

npm -v
6.4.1


(John Hunter) #15

Same here, but the fix didn’t work for me :(
npm 6.4.1


(Kat Marchán) #16

This is a duplicate of Some packages have dist.tarball as http and not https. For future reference, please note that you can safely sed -i this back to https until it’s resolved, since it’s a registry-side issue.


(Kat Marchán) #17