"Write after end" when installing packages with 5.8 and later


(Kat Marchán) #1

What I Wanted to Do

Do a regular install with the latest npm, with an empty cache.

What Happened Instead

❯ rm -rf /tmp/empty-cache && rm -rf node_modules && npx npmc install --cache /tmp/empty-cache
npx: installed 763 in 28.723s
      throw new Error('write after end')

Error: write after end
    at MiniPass.write (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/minipass/index.js:81:13)
rest of stack trace
    at ReadEntry.entry.on (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/pacote/lib/extract-stream.js:30:21)
    at ReadEntry.emit (events.js:185:15)
    at ReadEntry.emit (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/node_modules/minipass/index.js:296:25)
    at ReadEntry.[maybeEmitEnd] (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/node_modules/minipass/index.js:249:12)
    at ReadEntry.end (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/node_modules/minipass/index.js:162:27)
    at Unpack.[consumeBody] (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/lib/parse.js:210:13)
    at Unpack.[consumeChunkSub] (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/lib/parse.js:391:40)
    at Unpack.[consumeChunk] (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/lib/parse.js:360:30)
    at Unzip.(anonymous function).on.chunk (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/lib/parse.js:291:59)
    at Unzip.emit (events.js:180:13)
    at Unzip.emit (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/node_modules/minipass/index.js:296:25)
    at Unzip.write (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/node_modules/minipass/index.js:99:17)
    at Unzip.write (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/node_modules/minizlib/index.js:284:29)
    at Unpack.write (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/lib/parse.js:307:19)
    at PassThrough.ondata (_stream_readable.js:651:20)
      throw new Error('write after end')

Reproduction Steps

This bug only happens on certain networks, on certain machines:

  1. Get a regular repository with a bunch of deps.
  2. rm -rf node_modules
  3. npm install --cache /tmp/empty-cache
  4. :boom:


This was reproduced on 5.8.0 and any later versions of npm. Node version(s) unknown.

See also: https://github.com/npm/node-tar/issues/180

(Kat Marchán) #2

There’s a patch to minipass now that should prevent double-ends altogether (along with a root cause fix). It should get pulled in before the next release, but it’ll need to be tested to confirm the original poster isn’t experiencing it anymore (they can check next week, because it happens on the office network).

(Joshua Chaitin Pollak) #3

I will check Monday!

(Kat Marchán) #4

I’ve merged the relevant patches into release-next and pushed out a new canary. I believe this issue is now fixed. I look forward to the result of y’all’s testing!

I’ve also updated what seemed to be the busiest issue reporting this bug. I’m looking forward to having this place be the bug tracker. It’s so hard to figure out wtf people are actually running into. :weary:

(Kat Marchán) #5

Looks like that wasn’t enough. There’s another patch over at https://github.com/npm/node-tar/pull/182 that’s supposed to fix it on node 10 but I’m legit confused because minipass shouldn’t be able to emit twice anymore.

(Kat Marchán) #6

This is finally fixed for real, and confirmed, and will be released as part of npm@6.1.0 this week. (Note: the fix will land for npm ci with the first release after this one. Oops! npm i is ok tho)

You can try the patch out in the meantime by using the latest canary for your installs (npx npmc@latest install).

(Kat Marchán) #8

npm@6.1.0 is now latest. This fixes this issue in npm install. The fix will be available for users of npm ci in the following release, and is already fixed in our release branch.

(Kat Marchán) #9

(Kat Marchán) #10

I’m gonna leave this open until it’s been confirmed with the next release.