Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

git+ssh package install does not seem to work #513

Closed
Fishrock123 opened this issue Oct 5, 2016 · 103 comments
Closed

git+ssh package install does not seem to work #513

Fishrock123 opened this issue Oct 5, 2016 · 103 comments
Assignees
Labels

Comments

@Fishrock123
Copy link

Fishrock123 commented Oct 5, 2016

OP's Note: if you are also having this EXACT problem, please upvote this WITHOUT commenting.


Do you want to request a feature or report a bug?

bug

What is the current behavior?

yarn install v0.14.0
info No lockfile found.
[1/4] 🔍  Resolving packages...
error Couldn't find package "<package>" on the "npm" registry.

If the current behavior is a bug, please provide the steps to reproduce.

"devDependencies": {
    "license-builder": "git+ssh://git@github.com/fishrock123/<package>.git",
}

What is the expected behavior?

typical install

Please mention your node.js, yarn and operating system version.

Node.js: v6.6.1-pre
Yarn: v0.14.0 (master)
OS: OSX 10.10.5

@Fishrock123 Fishrock123 changed the title git+ssh package identify does not seem to work git+ssh package install does not seem to work Oct 5, 2016
@sebmck sebmck added the cat-bug label Oct 5, 2016
@sebmck sebmck self-assigned this Oct 5, 2016
@sebmck
Copy link
Contributor

sebmck commented Oct 5, 2016

Do you have a repro that I can use verbatim? Having trouble reproducing this.

@Fishrock123
Copy link
Author

No, sorry.

It wasn't actually from my personal github though. It's "git+ssh://git@github.com/<org>/<package>.git"

The repo is private and I have read/write access. (This accesses it via a SSH key registered on my github account)

Is there additional log output I can get somehow for you?

@Fishrock123
Copy link
Author

I will mention this happens with more than one such identifier.

@danharper
Copy link
Contributor

Minimum repro:

{
  "name": "x",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "devDependencies": {
      "eslint-config-radweb": "git+https://git@github.com/radweb/eslint-config-radweb.git"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

The package isn't on the registry, if that makes a difference.

@BBB
Copy link

BBB commented Oct 11, 2016

This also errors when specifying a git tag (which npm allows).

Example snippet:

...
  "react-quill": "git+https://git@github.com/alexkrolick/react-quill.git#v2.0.1",
...

@adambowles
Copy link

I also get this #621

@JamesHenry
Copy link

Just adding in case subtlety different/requires an additional solution to @BBB case of git tag:

I am trying to install a specific commit hash with git+ssh. NPM default client supports this.

@BBB
Copy link

BBB commented Oct 11, 2016

Looks like #573, #633 and #639 are related

@mko
Copy link

mko commented Oct 11, 2016

For context, from NPM's docs:

npm install <git remote url>:

Installs the package from the hosted git provider, cloning it with git. First it tries via the https (git with github) and if that fails, via ssh.

<protocol>://[<user>[:<password>]@]<hostname>[:<port>][:][/]<path>[#<commit-ish>]

<protocol> is one of git, git+ssh, git+http, git+https, or git+file. If no <commit-ish> is specified, then master is used.

It should also be noted that <commit-ish> is a pretty wide array of resolvable values.

A commit object or an object that can be recursively dereferenced to a commit object. The following are all commit-ishes: a commit object, a tag object that points to a commit object, a tag object that points to a tag object that points to a commit object, etc.

Note: These git remote url installs also should be ensured to work across both public and private git server instances using SSH keys for server authentication, not just GitHub/GitLab/etc. You could imagine a scenario where a company uses a local git server in-house for all of their internally managed dependencies (or even private GitHub repos accessed via SSH). As of right now, yarn is not set up to accommodate these relatively common use cases.

Easiest way to set up a repro case would be to try to install a package from a private GitHub repository using Yarn.

@milosivanovic
Copy link

milosivanovic commented Oct 11, 2016

If you specify the following source string:

"devDependencies": {
    "license-builder": "ssh://github.com/<user>/<package>",
}

then it does attempt to clone the stated repository over SSH, but fails with "Permission denied (publickey)" because GitHub expects clients to login as the git user, and in this case the local account user was used by default.

When you do specify git@ to force git to login as git to GitHub, then it fails with the usual:

error Couldn't find any versions for <package> that matches ssh://git@github.com/<user>/<package>.

So, it almost works, but doesn't support specifying the username. If your local account user happened to be called git, then it would actually succeed.

@milosivanovic
Copy link

milosivanovic commented Oct 11, 2016

If you add the following to your ~/.ssh/config file:

Host github.com
        User git

you can force all logins to github.com via SSH to use the user git by default, and this makes yarn able to clone from private repositories when using the ssh://github.com/<user>/<package> source format.

@131
Copy link

131 commented Oct 11, 2016

This is a strong no-go for us, using git referenced repos (pointing our local gitlab EE instance) is a strong part of our workflow 😢
Also it's very usefull for forking and pointing to "before merge and npm publish" packages (e.g. http-proxy ...)

@ntucker
Copy link

ntucker commented Oct 11, 2016

@milosivanovic

If you add the following to your ~/.ssh/config file:

But my config already has my ssh auth credentials for github so I can access the private repos. This workaround only works for public repos, right?

@kblcuk
Copy link

kblcuk commented Oct 11, 2016

@milosivanovic @ntucker actually this worked in my particular case; I didn't had any ssh config file to start with.

@ntucker
Copy link

ntucker commented Oct 11, 2016

@kblcuk ah, well @milosivanovic was going around to other issues and claiming his workaround worked for those cases, so I figured this was the issue for the general problems with git urls.

@milosivanovic
Copy link

milosivanovic commented Oct 11, 2016

@ntucker the stated workaround is for private repositories. If you already have a Host github.com entry in ~/.ssh/config, then add User git to that entry and yarn will be able to clone when you specify a source string such as ssh://github.com/<user>/<package>, i.e. without "git+" and without any user specified.

@ntucker
Copy link

ntucker commented Oct 11, 2016

@milosivanovic How does github know my username to do ssh auth then?

@milosivanovic
Copy link

milosivanovic commented Oct 11, 2016

@ntucker when communicating over SSH, GitHub expects you to log in to their servers as user "git". If you try to log in with your usual GitHub username, authentication will fail. GitHub over SSH differentiates you by your public key, not username. Reference: https://help.github.com/articles/testing-your-ssh-connection/

@silverwind
Copy link

silverwind commented Oct 11, 2016

Just thought to mention because many are probably not aware of this feature. For GitHub, you can also depend on the tarball URL which works fine with yarn. Installs faster too.

https://github.com/user/repo/tarball/branch

@brokenalarms
Copy link

brokenalarms commented Oct 11, 2016

@milosivanovic unfortunately this workaround doesn't work for our internal urls in the format:
git+ssh://git@source.com:team-name/repo.git

If you change the initial repo to the format ssh://source.com/team-name/repo.git...

...then it will work for the initial one...but then of course all the other internal dependencies that the first internal dependency points to breaks it, since they're all in that format.

Without going through and changing all the URLs on all our repos and dependencies to the workaround format (then having to deal with it not working on npm normally), we're a bit blocked on this too.

As @131 pointed out, this is a major way teams use npm internally (that I'm aware of).

Looks great besides that though!

@milosivanovic
Copy link

@brokenalarms the ssh://host.com/user/repo format is fully backwards-compatible with npm (as long as the expected user is specified in the SSH config file), but that's a fair point nonetheless.

@alanhogan
Copy link

#1784 requests a new release. Please just leave a thumbs-up reaction!

@sateffen
Copy link

sateffen commented Nov 15, 2016

My issue describes a problem, that is similar, even though not the same. I looked through the code a little, and found this interesting part, which is actually used on every git url:

static cleanUrl(url): string {
    return url.replace(/^git\+/, '');
}

Soooo.... Can anyone tell me, what's the reason for removing the git+ for each git url passed to yarn? I don't see a real reason, and the code is lacking documentation, so maybe someone could explain the intention :)

@BryanCrotaz
Copy link

#1816 may well fix this - take a look at the code changes - it certainly fixes the problem with colon after the domain

@jlouazel
Copy link

jlouazel commented Nov 16, 2016

The problem seems to be fixed on yarn v0.17.0. I was able to get one of my private Github repository on a specific version.

@devcer
Copy link

devcer commented Nov 16, 2016

Is this issue fixed? I m trying to migrate my project from npm to yarn but still facing this issue with yarn@0.17.2 !

@jlouazel
Copy link

jlouazel commented Nov 16, 2016

@viswanathamsantosh Seems to work on my side

image

@devcer
Copy link

devcer commented Nov 16, 2016

looks like this is fixed here #971 !replacing colon( : ) with slash( / ) actually worked. :')

@sateffen
Copy link

Replacing the colon with a slash doesn't work in my case :( git+ssh://git@private... still gets cut down to ssh://git@private...

@martynchamberlin
Copy link

Yes I'm still having this problem too, even with version 0.17.2 of Yarn. The git+ part gets removed and I end up with:

Permission denied (publickey).
fatal: Could not read from remote repository.

Given that it's working for some people though makes me confused. Any idea what we're doing wrong?

@qballer
Copy link

qballer commented Nov 16, 2016

place this in ~/.ssh/config

Host github.com
        User git

@martynchamberlin
Copy link

Yes! That fixed it for me. Thanks.

It'd be nice though if cleanUrl were not getting run, so we could have this directly in the URL instead. Having to change a configuration file requires a devops change where we could otherwise have version control handling it. Not sure what the thought process behind this was...?

@DominicBoettger
Copy link

Same problem here. Urls of private repositories are not working as before (npm).

git+ssh://git@github.com:ORG/repo.git should work as it's needed to be compatible with npm during the migration phase...

@martynchamberlin
Copy link

@DominicBoettger Once you've added User git to your ~/.ssh/config you'll also want to change that colon to a forward slash. In my experience today, Yarn doesn't play well with the colon yet.

git+ssh://github.com/ORG/repo.git

@sarus
Copy link

sarus commented Nov 17, 2016

dependencies of the form

git+ssh://git@bitbucket.org:myuser/repo.git#v1.0.0",

do not work for me with the latest yarn 017.2. The error is:

ssh: Could not resolve hostname bitbucket.org:myuser: Name or service not known

Haven't tested the work arounds yet but hoping yarn will eventually support the same syntax as NPM. Should this be a new issue or is this still applicable to this issue?

@BryanCrotaz
Copy link

@sarus PR #1816 will fix this

@DominicBoettger
Copy link

Please merge PR #1816 and publish a new version 👍

@vsantosu
Copy link

vsantosu commented Dec 4, 2016

Merge? Someone? NPM IS DRIVING ME NUTS!! Please merge and release :(

@vichle
Copy link

vichle commented Dec 5, 2016

This issue remains in v0.18.0.

Calling yarn install in a clean project, without node_modules and yarn.lock file works. Calling it again straight after will result in the "Could not resolve hostname" error.

I discovered that removing the yarn.lock file works, so I assume that there is either something wrong in the lock file or the way that yarn clones when reading from a lock file.

Hope this helps!

@regou
Copy link

regou commented Dec 16, 2016

This issue is the very one which stops many developers to use yarn.
We need take this issue seriously

@vsantosu
Copy link

@regou totally agree man... This is the only reason I cannot use Yarn...

@danielkcz
Copy link

People, instead of this constant yammering that nobody works on this, just watch that mentioned PR #1816 and you will see they are trying to merge it...

@BryanCrotaz
Copy link

Please everyone we have a fix for this in #1816 that Flavio and I spent around ten days on.

However a different set of tests fails depending on where the tests are run.

Please run the tests on your machine and report on #1816 what results you got and what your OS and node version is

@torifat
Copy link
Member

torifat commented Dec 16, 2016

Thanks @FredyC & @BryanCrotaz for point everyone to #1816. I'm locking this thread for the time being.

@yarnpkg yarnpkg locked and limited conversation to collaborators Dec 16, 2016
@bestander
Copy link
Member

Fixed via #2384

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests