Can I query NPM for all advisory information?


(Adam) #1

Running npm audit gets me most of the data I need, but I’d also like to get the reported, published, and accessed dates for an advisory. This information is available through the website, but if it’s possible to get it through an API call, or using a couchdb query, I’d prefer to do that. Unfortunately I’m not too familiar with CouchDB yet, so have had trouble figuring out whether it’s possible to do on registry.npmjs.com/registry.

Basically what I want to do is run npm audit to get a list of advisories, then query NPM to get the reported, published, and access dates.


(Lars Willighagen) #2

You can run

npm audit --json

to receive more detailed and machine-readable information. Example output for jquery@2:

{
  "actions": [
    {
      "action": "install",
      "module": "jquery",
      "target": "3.3.1",
      "isMajor": true,
      "resolves": [
        {
          "id": 328,
          "path": "jquery",
          "dev": false,
          "optional": false,
          "bundled": false
        }
      ]
    }
  ],
  "advisories": {
    "328": {
      "findings": [
        {
          "version": "2.2.4",
          "paths": [
            "jquery"
          ],
          "dev": false,
          "optional": false,
          "bundled": false
        }
      ],
      "id": 328,
      "created": "2017-03-20T21:50:28.000Z",
      "updated": "2018-03-26T21:14:32.509Z",
      "deleted": null,
      "title": "Cross-Site Scripting (XSS)",
      "found_by": {
        "name": "Egor Homakov"
      },
      "reported_by": {
        "name": "Egor Homakov"
      },
      "module_name": "jquery",
      "cves": [
        "CVE-2017-16012"
      ],
      "vulnerable_versions": ">=1.4.0 <=1.11.3 || >=1.12.4 <=2.2.4",
      "patched_versions": ">=3.0.0",
      "overview": "Affected versions of `jquery` interpret `text/javascript` responses from cross-origin ajax requests, and automatically execute the contents in `jQuery.globalEval`, even when the ajax request doesn't contain the `dataType` option.",
      "recommendation": "Update to version 3.0.0 or later.",
      "references": "[Issue #2432](https://github.com/jquery/jquery/issues/2432)\n[Commit #b078a62](https://github.com/jquery/jquery/commit/b078a62013782c7424a4a61a240c23c4c0b42614)\n[PR #2588](https://github.com/jquery/jquery/pull/2588)",
      "access": "public",
      "severity": "high",
      "cwe": "CWE-725",
      "metadata": {
        "module_type": "Browser.Library",
        "exploitability": 3,
        "affected_components": ""
      },
      "url": "https://nodesecurity.io/advisories/328"
    }
  },
  "muted": [],
  "metadata": {
    "vulnerabilities": {
      "info": 0,
      "low": 0,
      "moderate": 0,
      "high": 1,
      "critical": 0
    },
    "dependencies": 1,
    "devDependencies": 0,
    "optionalDependencies": 0,
    "totalDependencies": 1
  },
  "runId": "9196ca16-d2bc-4463-acc9-7e057ab02cca"
}

(Adam) #3

Thanks, that was my first step and is how I’m getting the list of advisory IDs I care about. But the only timestamps I can see are the “created_at” and “updated_at” fields, but I’d really like to have the “reported at” and “published at” fields shown in the advisory timeline (for example, Advisory published: Mar 21st, 2017, Reported: Mar 20th, 2017 from https://www.npmjs.com/advisories/328). The delay between reporting and publication is one of the things I’m interested in.


(Lars Willighagen) #4

Right, sorry. I saw the date fields and forgot to check if those were actually the ones you were looking for. I do think created corresponds to “Reported”, but I’m not sure about that either now.


(Lars Willighagen) #5

You can request it by requesting the webpage but with X-Spiferack: 1, but that’s a bit of a hack and I don’t know if they want people to do that.

curl -H 'X-Spiferack: 1' https://www.npmjs.com/advisories/696
{
  "advisoryData": {
    "id": 696,
    "created": "2018-08-30T03:53:51.510Z",
    "updated": "2018-08-30T04:04:26.815Z",
    ...
    "formatted": {
      ...
      "created": "Aug 30th, 3:53:51 am",
      "updated": "Aug 30th, 4:04:26 am"
    }
  },
  "events": [
    {
      "id": 1249,
      "advisory_id": 696,
      "created": "2018-08-30T03:59:51.303Z",
      "type": "access",
      "message": "Advisory made public",
      "formatted": {
        "created": "Aug 30th, 2018"
      }
    },
    {
      "id": 1247,
      "advisory_id": 696,
      "created": "2018-08-30T03:58:25.477Z",
      "type": "published",
      "message": "Advisory published on loopback.io",
      "formatted": {
        "created": "Aug 30th, 2018"
      }
    },
    {
      "id": 1245,
      "advisory_id": 696,
      "created": "2018-08-30T03:54:32.831Z",
      "type": "reported",
      "message": "Initial report by NelsonBrandao",
      "formatted": {
        "created": "Aug 30th, 2018"
      }
    }
  ],
  ...
}

(Adam) #6

Oooh, that’s nice. My activities are low volume for now so I think that will work. Can you tell me how you figured that out?

I’m not going to mark the problem as solved just yet in case someone knows of a different way to do it that might work better long term, but I’ll come back and do that soon.


(Lars Willighagen) #7

Looking at how the npm website gets it information (to see if they were using some endpoint), I remembered their JavaScript navigation. When navigating to a different page they request the content in JSON form. Then it was a matter of figuring out which parts of the (XMLHttpRequest) request makes it return JSON instead of a full page, and the answer was quite simple, luckily, and not some base64-encoded authorization token cookies.


(Adam) #8

I heard back from NPM support and they said this was a good solution. Appreciate the help!


(system) #9

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.