postpack hook has no access to the generated .tgz for inspection

What I Wanted to Do

I wrote a postpack bash hook to validate that the generated .tgz installs and runs properly, to prevent publishing an invalid package (eg : missing dependancy, error in .npmignore…).

When running “npm pack”, my script works well, although I grab on the generated .tgz using a pretty sophisticated command

packed_module=$(npm list --depth=0 | sed -n '1 s/^@//p' | cut -d " "     -f1 | sed 's/[\/@]/-/g').tgz

What Happened Instead

When running “npm publish”, the .tgz is not generated at the same location. From examining source code and pausing in my script, I discovered that it is generated in a temp directory and named “package.tgz”.

There is no way for my script to find it (in a clean and safe way at least)

Reproduction Steps

in your package.json, in “scripts”, add “postpack”:“read -p ‘return to continue’ DUMMY”
When the script is paused, you can check that .tgz is not in the project directory. A temp folder named /tmp/npm-XXXXX-XXXXXX/tmp/fromDir-XXXXXXXX contains a package.tgz file.

Details

Explanation is simple :
in “publish.js”, function publishFromDirectory() invokes **pack.packDirectory()" with a target to a “package.tgz” in temporary folder.
in “pack.js”, function pack_() invoked packDirectory() with target set to ```const target = `{name}-{mani.version}.tgz````

Possible solutions

The simplest would be to put the path to the .tgz in an environment variable, which could be named something like “npm_lifecycle_postpack_archive”. It would solve the problem and also make it simpler to write a postpack hook.

Other solutions could be to add a command line to npm publish, to set explicitly where to generate the archive (and write the postpack hook accordingly).

Platform Info

$ npm --versions
{
  '@olivr70/test2': '0.0.1',
  npm: '6.10.0',
  ares: '1.15.0',
  brotli: '1.0.7',
  cldr: '35.1',
  http_parser: '2.8.0',
  icu: '64.2',
  llhttp: '1.1.4',
  modules: '72',
  napi: '4',
  nghttp2: '1.38.0',
  node: '12.5.0',
  openssl: '1.1.1c',
  tz: '2019a',
  unicode: '12.1',
  uv: '1.29.1',
  v8: '7.5.288.22-node.14',
  zlib: '1.2.11'
}
$ node -p process.platform
linux

Adding npm_lifecycle_postpack_archive to the environment seems reasonable, but I’d worry that doing anything to change that archive might result in breaking the integrity checksums. Maybe if there’s a postpack script, we copy it into cacache’s tmp folder, and provide that? Or just make sure to re-compute checksums after the script and before uploading for the publish.

Patch welcome. I do think this belongs in #ideas rather than #bugs, since you’re describing new functionality rather than something not working as documented.

Hello,

I put into bug because the postpack script cannot work the same way whether its invoked on “npm pack” or “npm publish”, and also because in my mind “postpack” implied accessing the result of the previous step (not just logging…). But this would be more a spec bug than a technical bug.

I only thought of my need, which is does not imply modifying the archive. I am not sure there is a need for updating the archive, but who knows ?

By the way I imagine a tampered with archive would be rejected by the platform when received no ?

I think it is a strong requirement that “postpack” should work the same when invoke both ways, and modifications should be made in pack.js only.

I can see 3 possibilities :

  1. consider that postpack should not modify the archive : before invoking the hook, make a temporary copy of the archive (including when running npm pack).
  2. consider that postpack should not modify the archive : after invoking the hook, verify the archives checksum locally and fail if they are not valid
  3. allow modification of the archive : then the checksums should be recomputed after postpack

This decision should be made by the npm core team in my mind.

I would not rely on the registry rejecting a tarball with a mismatched integrity checksum. The npm, Inc.
public registry definitely will, but I don’t know about all the various private registry implementations out there. And in any case, it’d be setting a user up for an error.

I am not sure there is a need for updating the archive, but who knows ?

I could imagine pulling in binary assets and stuffing them in the archive, or updating docs, or filtering various stuff out for policy or security reasons, or even attaching an author signature to the end. But regardless of the validity or wisdom of using a postpack script for these cases, if my years in the npm game have taught me one thing, it’s “if you allow it, someone will come to depend on it”.

Any of the three possibilities you list would be an acceptable way to avoid that footgun, imo. I’d choose 3 if it’s easy, or 1 if it isn’t. (Perhaps option 1 could also print a warning if the shasum changes, like “hey, you modified this thing, but that’s not supported, so I’m throwing away your modifications, just fyi”.)

This topic is related to issue #19543 on the archived repo npm/npm.