npm audit error messaging update for 401s

triaged
cli
priority:medium

(Simon Kurtz) #1

What I Wanted to Do

We use Microsoft Azure DevOps (formerly Visual Studio Team Services) as a third party npm registry with an npm upstream. Periodically, I check whether npm audit is supported, but DevOps still does not have an endpoint set up for that. I have submitted a Developer Community suggestion which has garnered moderate support but has not yet been picked up or commented on by Microsoft.

What Happened Instead

Running npm audit while pointed to the DevOps registry yields this result:

npm ERR! code E401
npm ERR! Unable to authenticate, your authentication token seems to be invalid.
npm ERR! To correct this please trying logging in again with:
npm ERR!     npm login

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\skurtz\AppData\Roaming\npm-cache\_logs\2018-12-11T15_05_34_810Z-debug.log

From the looks of it, the audit command does not pass any authentication credentials (no Bearer token), which I suspect is done, so that audit can run as unencumbered as possible. DevOps, however, receives an unauthenticated request and returns a 401. That makes sense from Microsoft’s perspective as they don’t support that endpoint, so no authentication exclusions specifically for audit exist.

Prior to npm 6.5.0, audit showed an error message that read Your configured registry <registry URL> does not support audit requests. That used to show for all HTTP errors >= 400. I see that Rebecca changed it to === 404 || >= 500 in commit 5702175.

While I can agree with that change, it does leave DevOps and perhaps other 3rd party registry users a tad bit confused as the issue is not one of authentication but of support of the audit feature.

What I would like to see would be a bit more detail that indicates that this may not necessarily be authentication related, although it may be a bit of a stretch for npm to bridge that gap here. I will submit a PR to take a stab at this and see if it gains any traction. Update 12/17: Link to PR.

Reproduction Steps

Point to a registry on Azure DevOps and run npm audit. I’m sorry I cannot be more specific or helpful here as our registry is private.

Details

0 info it worked if it ends with ok
1 verbose cli [ 'C:\\Program Files\\nodejs\\node.exe',
1 verbose cli   'C:\\Users\\skurtz\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli   'audit' ]
2 info using npm@6.5.0
3 info using node@v11.4.0
4 verbose npm-session 300374f1549a7b48
5 timing audit compress Completed in 19ms
6 info audit Submitting payload of 70493 bytes
7 http fetch POST 401 https://<redacted>.pkgs.visualstudio.com/_packaging/<redacted>Npm/npm/registry/-/npm/v1/security/audits 463ms
8 verbose stack Error: Unable to authenticate, need: Bearer
8 verbose stack     at res.buffer.catch.then.body (C:\Users\skurtz\AppData\Roaming\npm\node_modules\npm\node_modules\npm-registry-fetch\check-response.js:89:17)
8 verbose stack     at process.internalTickCallback (internal/process/next_tick.js:77:7)
9 verbose statusCode 401
10 verbose cwd C:\Dev\<redacted>
11 verbose Windows_NT 6.1.7601
12 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\skurtz\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "audit"
13 verbose node v11.4.0
14 verbose npm  v6.5.0
15 error code E401
16 error Unable to authenticate, your authentication token seems to be invalid.
17 error To correct this please trying logging in again with:
17 error     npm login
18 verbose exit [ 1, true ]

Platform Info

$ npm --versions
6.5.0
$ node -p process.platform
v11.4.0

Release: npm@6.6.0-next.1
Release: npm@6.6.0
(Kat Marchán) #2

This should be fixed in 6.6.0-next.0. Can you try again with npm@next?


(Simon Kurtz) #3

Great to hear from you, Kat!

I just gave it a go but see the same output:

skurtz@SKURTZ-DEVLT C:\Dev\_sandbox\npm-audit-test
$ npm init -y
Wrote to C:\Dev\_sandbox\npm-audit-test\package.json:

{
  "name": "npm-audit-test",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}



skurtz@SKURTZ-DEVLT C:\Dev\_sandbox\npm-audit-test
$ npm i del
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN npm-audit-test@1.0.0 No description
npm WARN npm-audit-test@1.0.0 No repository field.

+ del@3.0.0
added 26 packages from 6 contributors in 8.339s

skurtz@SKURTZ-DEVLT C:\Dev\_sandbox\npm-audit-test
$ npm -v
6.6.0-next.0

skurtz@SKURTZ-DEVLT C:\Dev\_sandbox\npm-audit-test
$ npm audit
npm ERR! code E401
npm ERR! Unable to authenticate, your authentication token seems to be invalid.
npm ERR! To correct this please trying logging in again with:
npm ERR!     npm login

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\skurtz\AppData\Roaming\npm-cache\_logs\2018-12-17T18_05_50_854Z-debug.log

skurtz@SKURTZ-DEVLT C:\Dev\_sandbox\npm-audit-test

Here’s the output from the debug.log:

0 info it worked if it ends with ok
1 verbose cli [ 'C:\\Program Files\\nodejs\\node.exe',
1 verbose cli   'C:\\Users\\skurtz\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli   'audit' ]
2 info using npm@6.6.0-next.0
3 info using node@v11.4.0
4 verbose npm-session 7d3245d03d12e5d7
5 http fetch POST 401 https://<redacted>.pkgs.visualstudio.com/_packaging/<redacted>Npm/npm/registry/-/npm/v1/security/audits 312ms
6 verbose stack Error: Unable to authenticate, need: Bearer
6 verbose stack     at res.buffer.catch.then.body (C:\Users\skurtz\AppData\Roaming\npm\node_modules\npm\node_modules\npm-registry-fetch\check-response.js:94:17)
6 verbose stack     at process.internalTickCallback (internal/process/next_tick.js:77:7)
7 verbose statusCode 401
8 verbose pkgid audits
9 verbose cwd C:\Dev\_sandbox\npm-audit-test
10 verbose Windows_NT 6.1.7601
11 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\skurtz\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "audit"
12 verbose node v11.4.0
13 verbose npm  v6.6.0-next.0
14 error code E401
15 error Unable to authenticate, your authentication token seems to be invalid.
16 error To correct this please trying logging in again with:
16 error     npm login
17 verbose exit [ 1, true ]

(Kat Marchán) #4

:thinking::thinking::thinking: thanks for trying it out. Not sure why it’s happening but I’ll look into it!


(Simon Kurtz) #5

You’re welcome. Does this PR help?


(Kat Marchán) #6

I think so! Thank you!


(Kat Marchán) #7

Hey @simonua. I looked into it a bit more, and I don’t understand why you’re not getting auth info. It definitely looks like it should be sending it. Is there any way you can check that it’s not your server yielding spurious 401s, in this case? I want to make sure this works well for you and I need some help debugging your case.

Edit: nevermind, found a bug


(Simon Kurtz) #8

Hey @zkat, thanks for looking. I’ll be happy to test any changes you may want to make.


(Kat Marchán) #9

@simonua I just pushed a canary with what I believe is the fix. Can you try npx npmc@latest audit and see if you still run into this?


(Simon Kurtz) #10

Using npx npmc@latest audit (6.5.0-canary2) result:

skurtz@SKURTZ-DEVLT C:\Dev\_sandbox\npm-audit-test
$ npx npmc@latest audit
npx: installed 401 in 22.253s
npm notice CANARY npmc is experimental software. If you find an issue, please file it in the main npm repository, and call out that you were using npmc.
npm ERR! code E401
npm ERR! Unable to authenticate, your authentication token seems to be invalid.
npm ERR! To correct this please trying logging in again with:
npm ERR!     npm login

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\skurtz\AppData\Roaming\npm-cache\_logs\2018-12-18T20_25_20_301Z-debug.log
0 info it worked if it ends with ok
1 verbose cli [ 'C:\\Program Files\\nodejs\\node.exe',
1 verbose cli   'C:\\Users\\skurtz\\AppData\\Roaming\\npm-cache\\_npx\\27096\\node_modules\\npmc\\bin\\npm-cli.js',
1 verbose cli   'audit' ]
2 info using npm@6.5.0-canary.2
3 info using node@v11.4.0
4 notice CANARY npmc is experimental software. If you find an issue, please file it in the main npm repository, and call out that you were using npmc.
5 verbose npm-session 1db647756fc79dc1
6 http fetch POST 401 https://<redacted>.pkgs.visualstudio.com/_packaging/<redacted>Npm/npm/registry/-/npm/v1/security/audits 225ms
7 verbose stack Error: Unable to authenticate, need: Bearer
7 verbose stack     at res.buffer.catch.then.body (C:\Users\skurtz\AppData\Roaming\npm-cache\_npx\27096\node_modules\npmc\node_modules\npm-registry-fetch\check-response.js:94:17)
7 verbose stack     at process.internalTickCallback (internal/process/next_tick.js:77:7)
8 verbose statusCode 401
9 verbose pkgid audits
10 verbose cwd C:\Dev\_sandbox\npm-audit-test
11 verbose Windows_NT 6.1.7601
12 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\skurtz\\AppData\\Roaming\\npm-cache\\_npx\\27096\\node_modules\\npmc\\bin\\npm-cli.js" "audit"
13 verbose node v11.4.0
14 verbose npm  v6.5.0-canary.2
15 error code E401
16 error Unable to authenticate, your authentication token seems to be invalid.
17 error To correct this please trying logging in again with:
17 error     npm login
18 verbose exit [ 1, true ]

(Kat Marchán) #11

sigh. I’ll get back to you, then. Not sure what’s going on anymore. But I did find a legit bug, at least.


(Simon Kurtz) #12

No worries! I appreciate you looking. I know you’re busy with tink, too.

Tangential question: Is it too presumptuous to automatically issue an npm audit request against the main npm registry in the event of an audit failure against the configured registry? I could see that answer be yes very easily but I don’t know that this has been asked before and just wanted to see what npm’s take was.


(Kat Marchán) #13

We don’t want to do that because it can potentially leak private information to the public registry, and I think a lot of people would be unhappy about that.

Do you mind if I DM you to get some debugging info for what you’re running into?


(Simon Kurtz) #14

Thanks for that explanation. That does make pretty obvious sense.

DM me any time, please. Happy to help.


(Kat Marchán) #15

Update: the original issue is legitimate and has been fixed with npm@6.6.0-next.0. Further errors were due to an alternate registry returning 401s for unknown endpoints (such as the audit endpoint), and thus confusing the CLI. So, this matter is resolved! Thanks @simonua for your time to work through this issue and figure out what was going on.


(Simon Kurtz) #16

Thank you! Appreciate ya, Kat!


(Kat Marchán) #17

p.s. https://github.com/npm/cli/pull/128 I made this PR anyway, which I think better covers all the failures.


(Simon Kurtz) #18

I verified that the messaging in 6.6.0-next.1 successfully addresses this bug. Thank you, Kat, Rebecca, and Audrey!