npm CLI: --json flag does not handle output from package scripts correctly

What I Wanted to Do

I wanted to run: npm install --json and receive back a valid JSON object.

What Happened Instead

When I run this command and npm installs a package that spams output messages to the console, those output messages are NOT wrapped into the JSON object. Rather, they are simply dumped in front of it.

This means that any tool trying to parse the output of the install command will fail because the output is, in fact, NOT valid JSON.

Reproduction Steps

  1. Create a standard package.json file that depends on a package that spams the console after installing. core-js-pure@3.1.3 is a great example.

  2. In the folder with the file from (1), run: npm install --json

  3. Observe that the output has spammy messages before the JSON object, which means that any tool attempting to parse the output as JSON will fail.

What I See

Here is the truncated output I see:

> core-js-pure@3.1.3 postinstall /some/file/path
> node scripts/postinstall || echo "ignore"

e[96mThank you for using core-js (e[94m https://github.com/zloirock/core-js e[96m) for polyfilling JavaScript standard library!e[0m[96mThe project needs your help! Please consider supporting of core-js on Open Collective or Patreon: e[0m
e[96m>e[94m https://opencollective.com/core-js e[0m
e[96m>e[94m https://www.patreon.com/zloirock e[0m e[96mAlso, the author of core-js (e[94m https://github.com/zloirock e[96m) is looking for a good job -)e[0m

{
  "added": [
{
  "action": "add",
  "name": "lodash",
  "version": "4.17.11",
  "path": "/some/file/path/node_modules/@babel/core/node_modules/lodash"
},
[...]  // Remainder of JSON output truncated for brevity

The Fix

  1. Any spam/output from package.json scripts must be properly embedded in the JSON output object. It cannot simply be dumped in front of the JSON.

  2. Relying on --quiet or --silent is not a fix because those flags will prevent npm from emitting wanted information about errors and warnings.

  3. The temporary workaround I’m using is to take the output data from npm, convert it to a string, drop everything before the first { character and then attempt to parse the remainder of the string as JSON. This is obviously incredibly fragile and will fail whenever a package author spams the console with a string that contains {.

Note: If you’re using the --json flag, you’re doing so because you want the output processed by some build tool or app; not by a human. The current behavior completely breaks the point of having --json. The contract when using the --json flag must be that ALL output is embedded in a JSON object. Otherwise, the flag is useless.

Platform Info

$ npm --versions
{ npm: '6.8.0',
  ares: '1.14.0',
  cldr: '33.1',
  http_parser: '2.8.0',
  icu: '62.1',
  modules: '64',
  napi: '3',
  nghttp2: '1.34.0',
  node: '10.14.1',
  openssl: '1.1.0j',
  tz: '2018e',
  unicode: '11.0',
  uv: '1.23.2',
  v8: '6.8.275.32-node.36',
  zlib: '1.2.11' }

$ node -p process.platform
darwin

Idea

If the work to correct this behavior is too extensive, an alternate approach exists:

When npm is going to dump output in front of the JSON object, prepend the object with a standard token for which implementors can scan. Example:

>Spammy message from package author
>If you liked my package, buy me a Ferarri!
>This package is deprecated; don't use it

***BEGINJSONOUTPUT***
{
 "added": {...}
 ...
}
***ENDJSONOUTPUT***

The ending token is not necessary if there is no situation in which npm will print anything after the JSON object.

At this point in time I don’t think this is considered a bug as such, according to the documentation. But your comments are still valid. :-)

https://docs.npmjs.com/misc/config.html#json

This feature is currently experimental, and the output data structures for many commands is either not implemented in JSON yet, or subject to change. Only the output from npm ls --json and npm search --json are currently valid.

Yea. I did see that, but it’s been there for a while now and since the --json flag is returning JSON output with other commands, it’s at least partially implemented for install.

Whoever is working on the implementation for the remaining commands should be aware of this behavior, so I didn’t know where else to file the issue.