npm install downgrading resolved packages from https to http registry in package-lock.json

registry
priority:low
triaged

(Stefan Maric) #1

What I Wanted to Do

npm install to generate a fresh package-lock.json with https urls only.

What Happened Instead

Some packages have their resolved field pointing to the http registry.

Reproduction Steps

mkdir httptest
cd httptest
npm init -y
# Some packages that were resolved to http registry on a project
npm install babel-plugin-syntax-object-rest-spread buffer@4.9.1 chalk@1.1.3 external-editor@2.2.0 minimist mkdirp
# Just a control package to show it is not for all packages
npm install lodash
# Display http urls
cat package-lock.json | grep 'http://'
# Display https ones just for control
cat package-lock.json | grep 'https://'

I get the following output:

Wrote to /home/sam/Code/httptest/package.json:

{
  "name": "httptest",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}


npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN httptest@1.0.0 No description
npm WARN httptest@1.0.0 No repository field.

+ babel-plugin-syntax-object-rest-spread@6.13.0
+ mkdirp@0.5.1
+ minimist@1.2.0
+ external-editor@2.2.0
+ buffer@4.9.1
+ chalk@1.1.3
added 21 packages in 0.953s
npm WARN httptest@1.0.0 No description
npm WARN httptest@1.0.0 No repository field.

+ lodash@4.17.10
added 1 package in 0.517s
      "resolved": "http://registry.npmjs.org/babel-plugin-syntax-object-rest-spread/-/babel-plugin-syntax-object-rest-spread-6.13.0.tgz",
      "resolved": "http://registry.npmjs.org/buffer/-/buffer-4.9.1.tgz",
      "resolved": "http://registry.npmjs.org/chalk/-/chalk-1.1.3.tgz",
      "resolved": "http://registry.npmjs.org/external-editor/-/external-editor-2.2.0.tgz",
      "resolved": "http://registry.npmjs.org/minimist/-/minimist-1.2.0.tgz",
      "resolved": "http://registry.npmjs.org/mkdirp/-/mkdirp-0.5.1.tgz",
          "resolved": "http://registry.npmjs.org/minimist/-/minimist-0.0.8.tgz",

      "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-2.1.1.tgz",
      "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-2.2.1.tgz",
      "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.3.0.tgz",
      "resolved": "https://registry.npmjs.org/chardet/-/chardet-0.4.2.tgz",
      "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz",
      "resolved": "https://registry.npmjs.org/has-ansi/-/has-ansi-2.0.0.tgz",
      "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.24.tgz",
      "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.1.12.tgz",
      "resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz",
      "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.10.tgz",
      "resolved": "https://registry.npmjs.org/os-tmpdir/-/os-tmpdir-1.0.2.tgz",
      "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
      "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-3.0.1.tgz",
      "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-2.0.0.tgz",
      "resolved": "https://registry.npmjs.org/tmp/-/tmp-0.0.33.tgz",

Details

It seems to happen with specific packages, specific versions (thus I specified some version above), and after a specific point in time. I recently created a fresh package-lock.json in a project, and the exact same version of those packages were resolved to the https registry at that time (using same node and npm version, and using same package.json).

Platform Info

$ npm --versions
{ httptest: '1.0.0',
  npm: '5.6.0',
  ares: '1.10.1-DEV',
  cldr: '32.0',
  http_parser: '2.8.0',
  icu: '60.1',
  modules: '57',
  napi: '3',
  nghttp2: '1.32.0',
  node: '8.11.4',
  openssl: '1.0.2p',
  tz: '2017c',
  unicode: '10.0',
  uv: '1.19.1',
  v8: '6.2.414.54',
  zlib: '1.2.11' }

$ node -p process.platform
linux

But I was also able to replicate with latest node 10.9.0 and npm 6.2.0 and npm 6.4.1.


Tarball URLs on registry randomly switch from HTTPS to HTTP
Some packages have dist.tarball as http and not https
(Jan Lehnardt) #2

we’re seeing the same on various combinations of node and npm, any pointers here would be appreciated.


(Elia Maino) #3

There’s a similar issue opened on GitHub since a while (20 mar): https://github.com/npm/npm/issues/20106

Some people had the problem solved by performing the steps described in https://github.com/npm/npm/issues/20106#issuecomment-383770315 (not me anyway)

It would be definitely nice to have an update on this.


(Stefan Maric) #4

For those looking to workaround it, I’ve been simply replacing the URLs after an npm install that creates/updates the package-lock (and running npm i again) and haven’t experienced issues so far:

sed -i -e 's/http:\/\//https:\/\//g' package-lock.json

But indeed this behavior is odd, creates security concerns, and seems somehow related to the registry and not to the npm CLI itself.


(David Cannady) #5

The same issue has been plaguing my projects as well. This is a security issue and should be considered critical in terms of bug priority.


(Luis Lobo Borobia) #6

I have been seeing the same behavior.


(Amith George) #7

I faced the same issue today. I can confirm that after performing the steps listed in that comment, package-lock.json didn’t have any changes.

Just for future reference, I am pasting the steps as listed in the comment

$ rm -rf node_modules/
$ npm cache clean --force
(Revert the changes in your package-lock.json file)
$ npm i

(Jayden Seric) #8

I tweeted about this issue after noticing it the other day: https://twitter.com/jaydenseric/status/1038939403974897664

npm cache clean --force and a fresh install does not fix the issue.

  • Node.js v10.10.0
  • npm v6.4.1

(Stefan Maric) #9

Searching around, I noticed this is a duplicated: Some packages have dist.tarball as http and not https

Indeed this is an issue with the registry itself and applies to any npm CLI version.

Maybe unrelated, but I also noticed the registry responds with diff content-types for diff packages:

$ curl --silent --head https://registry.npmjs.com/buffer | grep -i content-type
content-type: application/octet-stream

$ curl --silent --head https://registry.npmjs.com/lodash | grep -i content-type
content-type: application/json

So maybe it is a faulty instance behind a load balancer the source of trouble.

PD: Just saw @nexdrew just linked this thread there.