`npm link` should not put links to binaries in <prefix>/bin

(Alain Kalker) #1

What I Wanted to Do

When in the top directory of a local project, when I used npm link to link a directory containing some other NPM package into the current project, I expected the other package to be linked into the current project’s node_modules directory, with no other side effects.
In particular: any binaries provided by the linked package should be made available in the node_modules/.bin directory, as if the package was installed using npm install.

What Happened Instead

While the other package did get linked into my project’s node_modules directory, and its binaries were available in my project’s node_modules/.bin directory, symlinks to them were also put into <prefix>/bin (as if I had done an npm install -g in the linked package’s top directory). This caused any files in <prefix>/bin which happened to have the same filenames as these binaries to get clobbered.

Reproduction Steps

$ # Skip the following line if `npm` is already set up for user-global installs
$ npm config set prefix ~/.local
$ # Skip the following line if `<prefix>/bin` is already on the $PATH
$ export PATH=$(npm config get prefix)/bin:$PATH
$ # Git clone an npm package to be linked into our local project
$ git clone https://github.com/piuccio/cowsay.git
[...git output...]
$ Create a local `cowsay` script for testing, verify that it works
$ cat > ~/.local/bin/cowsay <<EOF
> #!/bin/sh
> echo "Moo!"
> EOF
$ chmod +x ~/.local/bin/cowsay
$ hash -r
$ cowsay "Hello!"
Moo!
$ # Start a local project
$ mkdir blub
$ cd blub
$ npm init
$ # fill in info, add dependencies, etc.
$ # Try and link package `cowsay` into current project
$ npm link ../cowsay
[...npm output...]
/home/miki/.local/bin/cowthink -> /home/miki/.local/lib/node_modules/cowsay/cli.js
/home/miki/.local/bin/cowsay -> /home/miki/.local/lib/node_modules/cowsay/cli.js
/home/miki/.local/lib/node_modules/cowsay -> /home/miki/vcs/git/node-modules/cowsay
/home/miki/vcs/git/node-modules/blub/node_modules/cowsay -> /home/miki/.local/lib/node_modules/cowsay -> /home/miki/vcs/git/node-modules/cowsay
$ # Test if original `cowsay` still works
$ cowsay "Hello!"
 ________
< Hello! >
 --------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||
$ # So the original `<prefix>/bin/cowsay` got clobbered. Too bad...
$ # Check if binaries from the linked package are available in the expected place
$ node_modules/.bin/cowsay "Hello!"
 ________
< Hello! >
 --------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Details

Platform Info

$ npm --versions
{ blub: '1.0.0',
  npm: '6.9.0',
  ares: '1.15.0',
  brotli: '1.0.7',
  cldr: '34.0',
  http_parser: '2.9.1',
  icu: '63.1',
  llhttp: '1.1.1',
  modules: '67',
  napi: '4',
  nghttp2: '1.37.0',
  node: '11.14.0',
  openssl: '1.1.1b',
  tz: '2018e',
  unicode: '11.0',
  uv: '1.28.0',
  v8: '7.0.276.38-node.18',
  zlib: '1.2.11' }
$ node -p process.platform
linux
(John Gee) #2

npm link works a little differently than you are assuming. Linking is actually a two step process including the global packages location.

npm link ../cowsay is a shortcut for

cd ../cowsay
npm link
cd -
npm link cowsay

This is documented in npm help link and at https://docs.npmjs.com/cli/link

(Alain Kalker) #3

@shadowspawn I’m aware of the two-step linking process (actually in the example in my original post I use the shortcut version npm link ../cowsay instead).
I’m not having any issues with the linking itself, the issue is with npm link installing things in <prefix>/bin (as if instead of npm link I had done an npm install -g in the linked package’s top directory), clobbering any files in <prefix>/bin with the same filename which were already there. There is no need for npm link to install any binaries globally, they should (and are) already made available in the link destination project’s node_modules/.bin directory.
I’ve edited my original post to hopefully make this a bit clearer.
BTW: Why was this moved to Support? I’m not asking for help, I’m reporting a problem.

(Alain Kalker) #4

One final example of why I strongly believe that current npm link's behaviour is a problem:

Suppose a user has 2 npm packages (or 2 different versions of the same package) checked out from Git, each in its own directory: a_provider and b_provider. Let’s suppose that both of these packages provide a binary, foo. The foo binaries from both packages, although they have the same name, perform entirely different tasks.

Let’s suppose now that the user wants to create 2 projects: a_consumer, in which s/he links the package a_provider, and another project: b_consumer, into which s/he links the package b_provider. The user would now end up with a global binary <prefix>/foo, which happens to be linked to b_provider's foo binary, because b_provider was linked last. In other words, regardless of whether the user wanted this global binary in the first place, the order in which packages a_provider and b_provider are linked now dictates which global foo binary remains.

This is not acceptable: when the user wanted to use one of the two foo's globally, s/he would do a npm install -g in either one of the provider packages. The correct way to distinguish between the 2 foo's is to use either a_consumer/node_modules/.bin/foo or b_consumer/node_modules/.bin/foo.

I don’t think I can make it any clearer than this.

(John Gee) #5

Thanks for the explanation. I’ll reply later about link behaviour, and in brief why I moved topic out of #bugs for now (and apologies, I usually put in a comment at least saying I did that).

(John Gee) #6

I added a whole story for future reference: Why did you move my topic from #bugs?

(John Gee) #7

From the documentation:

First, npm link in a package folder will create a symlink in the global folder {prefix}/lib/node_modules/<package> that links to the package where the npm link command was executed. (see npm-config for the value of prefix ). It will also link any bins in the package to {prefix}/bin/{name} .

There are further examples on the page making it clear that the global install is intended.

For my typical use case, this is a useful behaviour! I have a few CLI packages that are intended for global install. I link the version I am developing so I can use and test it just as if I had installed the development version globally. No need to reinstall with each code change, the version I am developing is being used via the link.

The recommended way of avoiding packages conflicting is to install them locally rather than globally. This does not include npm link though, as you have discovered.

So if you want an npm link like behaviour that allows active development without the intermediate global-like install, I suggest that goes in #ideas.

If you want advice on how to achieve your workflow, you could perhaps expand on why you need to link rather than install a_provider and b_provider.

I hope that helps.