`npm pack` includes items from `.git` folder if there is a branch called `readme` (regression in 6.9.0)

What I Wanted to Do

$ git branch readme
$ npm pack

What Happened Instead

14:01 $ npm pack
[..snip..]
npm notice === Tarball Contents === 
[..snip..]
npm notice 169B   .git/logs/refs/heads/readme   
npm notice 41B    .git/refs/heads/readme        
[..snip..]

Reproduction Steps

(same as “What I wanted to do”)

Platform Info

14:02 $ npm --versions
{ '@my-namespace/my-package: '4.16.0',
  npm: '6.9.0',
  ares: '1.15.0',
  cldr: '33.1',
  http_parser: '2.8.0',
  icu: '62.1',
  modules: '64',
  napi: '3',
  nghttp2: '1.34.0',
  node: '10.15.3',
  openssl: '1.1.0j',
  tz: '2018e',
  unicode: '11.0',
  uv: '1.23.2',
  v8: '6.8.275.32-node.51',
  zlib: '1.2.11' }
14:03 $ node -p process.platform
darwin

This is causing issues if such a package is installed as a dep, and then npm prune --production is run, as it starts to complain about EISGIT.

This regression happens as well when installing from a github URL.

Steps to reproduce:

npx npm@6.9.0 install github:particle-iot/zeromq.js
ls node_modules/zeromq/.git # index (should be "No such file or directory"

This problem also occurs if there is a filename with readme in .git/hooks. I noticed this after publishing a package where I cloned the repo with GitKraken. GitKraken always creates a README.sample in the hooks directory. Later calling npm install in the parent project failed with an error message like this:

Error: npm: Command failed with exit code 1 Error output:
npm ERR! path /path/to/package/in/node_modules
npm ERR! code EISGIT
npm ERR! git /path/to/package/in/node_modules: Appears to be a git repo or submodule.
npm ERR! git     /path/to/package/in/node_modules
npm ERR! git Refusing to remove it. Update manually,
npm ERR! git or move it out of the way first.

I also found an issue in the old npm github repo which was never addressed:

This seems to be an issue with npm in general. Something is causing .git to be published, without pack and without a readme branch or anything. There was an issue reported here https://github.com/shipshapecode/ember-shepherd/issues/319

@rwwagner90
I was not able to reproduce this problem with ember-shepherd (on Mac):

$ npm --version
6.9.0
$ git clone git@github.com:shipshapecode/ember-shepherd.git
...
$ cd ember-shepherd
$ npm install
...
$ npm run build
...
$ npm pack
npm notice 
npm notice 📦  ember-shepherd@6.0.1
npm notice === Tarball Contents === 
npm notice 3.6kB  package.json                          
npm notice 18.9kB CHANGELOG.md                          
npm notice 59.2kB ember-shepherd-6.0.1.tgz              
npm notice 18.7kB HISTORY.md                            
npm notice 436B   index.js                              
npm notice 1.1kB  LICENSE.md                            
npm notice 2.3kB  README.md                             
npm notice 12.7kB addon/services/tour.js                
npm notice 465B   addon/utils/attachTo.js               
npm notice 977B   addon/utils/buttons.js                
npm notice 372B   addon/utils/dom.js                    
npm notice 56B    app/services/tour.js                  
npm notice 849B   config/deploy.js                      
npm notice 90B    config/environment.js                 
npm notice 472B   fastboot/instance-initializers/tour.js
npm notice === Tarball Details === 
npm notice name:          ember-shepherd                          
npm notice version:       6.0.1                                   
npm notice filename:      ember-shepherd-6.0.1.tgz                
npm notice package size:  76.5 kB                                 
npm notice unpacked size: 120.3 kB                                
npm notice shasum:        9dd4367fc152babd7c496b8b2f8fd1ef024e89ff
npm notice integrity:     sha512-GRAO+scrMKKth[...]ULDV/YrvKK/lA==
npm notice total files:   15                                      
npm notice 
ember-shepherd-6.0.1.tgz

(For reference, there were changes to ignore handling in npm 6.8.0 and 6.9.0: npm pack leaving out files (6.8.0 only)

I released a patch version of ember-shepherd to remove the .git directory. If you install version 6.0.0 of ember-shepherd, you will see the .git directory.

I had the same thing come up when I had a remote called readme in my .git/ folder.

Adding .git to .npmignore causes the .git/ folder to no longer be included in npm pack/npm publish.

I think the problem is the default whitelist has a higher priority than the internal blacklist, which I don’t think should be true.

Based on https://docs.npmjs.com/misc/developers#keeping-files-out-of-your-package, I think the same problem would apply to any branches, remotes, tags, etc. named readme, changelog, license, or package.json.