New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to connect to private repository #11
Comments
We get our data from github
|
If you are trying to get data from a private repository, you can only do so if you've got your .ssh keys set up properly. See here how to do that: https://help.github.com/articles/generating-ssh-keys However, you will only get git data and not github data (i.e. issues). |
Yes that makes sense, thanks for the friendly reminder. Any way you know that we can get issue data into the splunkgit app as is? Plans to mod your python scripts? FYI….we’re messing around with the v3 api now for some reporting, and we might extend what you’ve done here for the splunk app (the visualization is nice). |
I have created the ssh rsa keys and successfully cloned the repository locally. How do I configure splunkgit app in the splunkgit.conf to get git data for private repos? And has the python scripts been modified to get github data (issues)? This would make the SplunkGit a great tool for reporting and metrics at a glance. Thanks! |
@atul-tewari you should be able to get git data for private repositories by using the ssh link to the private repository, if you've set up your ssh keys correctly. You will, however, only get git data and not github data and issue tracking. For that we will have to implement support for githubs v3 api, as @sometheycallme was mentioning. I might consider looking at this when I have the time. @sometheycallme I totally missed your comment, and as I just mentioned, you're not able to get the issue data as the app is right now. v3 api would have to be implemented and if you do decide to do that, it'd be awesome :) FYI: I'm fixing some of the charts for Splunk 5.x and I'm also indexing commit messages now, but the messages doesn't have any visualizations yet. |
No worries. we completed our recent 5.x upgrade. we will incorporate what you have done, and any updates I'll certainly share them.
you got it. we've decided to take a stab at it. not likely to start until after the holiday season later in january due to some current workload, but it is up there in priority. We will most likely use this as a foundation: http://developer.github.com/v3/issues/events/ Good example of what it might look like https://plugins.atlassian.com/plugins/com.atlassian.jirawallboard.atlassian-wallboard-plugin https://github.com/archiloque/rest-client http://www.intridea.com/blog/2008/4/1/ruby-github-simple-access-to-the-github-api |
Thank you so much Petterik! And just to be sure, the private and public key files have to exist in the $HOME/.ssh directory correct? Is there a configuration setting that I have to set to point to the private/public RSA keys in addition to the $HOME/.ssh directory ? I was able to connect using these keys successfully through "SmartGit" and create a local clone of the private repositories. However, from within splunk, I was unable to connect to GitHub using ssh. Thanks! |
Awesome @sometheycallme! @atul-tewari Yes, the keys should live in Another FYI: I released a new version with some small fixes. I'm not sure what version you've been using of Splunkgit all this time, but you might want to check it out. |
After some further investigation, I had to drop the ssh rsa key pass phrase to make it work. Thanks for your timely help. We are very keen on using Splunkgit and would love to see GIT issues retrieved through the v3 API. After I got this to work, I upgraded to 1.3.0.1 SplunkGit. In the earlier version and the new one, I get the following error in Splunkd.log for all the private repos I connected to. ERROR ExecProcessor - message from "$HOME/infrastructure-tool/splunk/etc/apps/splunkgit/bin/fetch_git_repo_data.sh" git: $HOME/infrastructure-tool/splunk/lib/libz.so.1: no version information available (required by git). Git data is still retrieved, though. Let me know if you have come across this error. Thanks again! |
Petterik, I can no longer see any data come across other than number of forks and number of watchers. All other searches do not display any data. It does not get all the authors, coders, and file types like it did earlier. Is this version a little different from 1.2.1? 1.2.1 version seemed to get all data except of course GIT issues. Are there more config changes I have to make? I enabled jQuery-hackathon as well in inputs.conf 12-19-2012 20:42:13.872 -0600 ERROR SearchResults - Failed to remove "$HOME/infrastructure-tool/splunk/etc/users/admin/splunkgit/history/u16552355.csv.tmp1": No such file or directory Thanks! |
@atul-tewari Never seen that "libz.so.1: no version information available (required by git)." Error before. A quick google search suggests that you should update your git version. The other errors are from enabling Splunkgit hasn't changed much from 1.2.1 to 1.3.0.1, and if everything except github issues was working for you with 1.2.1, I might even suggest to use that version, until I've made 1.3.x more stable. |
@sometheycallme and @petterik any luck with this yet? 🎱 |
@petterik and @atul-tewari (since you got it to work somewhat), could you post a step-by-step guide how one gets Splunk to work with local Hopefully your answer(s) will help other too... Cheers! |
We are actually back to the drawing board at the moment. We've been able to successfully pull information from the github api (currently pulling json for private repo notifications) And messing with passing a "time" parameter to the api query. Next up is parsing the json output. (looking at spath, and splunkbase at the moment.) @petterik has done quite an extensive job with python and consuming local git log data for a mirrored repo. We're actually after something a bit different. When we have some working field extractions, and / or something to play with, I'll put it up on github under the https://github.com/organizations/phlowy organization. Sorry for the slow work, but it's difficult to find time to hack away. |
The ruby script we are playing with (you'll need rkj/gh github api wrapper) |
http://docs.splunk.com/Documentation/Splunk/5.0.2/SearchReference/Spath
|
update: we were able to successfully pull some test JSON from a private github repository (and organization) using "notifications" Then using spath - worked like a charm. automatically detected array items and pulled them out for us. We will continue with this today, and try to post some rudimentary example of a script (or that's at least our goal). cc: @jcwx |
Bump - any more movement, we'd like to use this with private repos. |
There's not much movement in this project. I can take a look at it after the new year!
On Dec 22, 2014, at 8:18 PM, Corey Frang <notifications@github.commailto:notifications@github.com> wrote: Bump - any more movement, we'd like to use this with private repos. — |
@petterik
Hello @petterik
I've just installed the splunkgit app on our internal splunk instance, and while it seems to work okay connecting to public repositories, when I provide a read-only account git url it doesn't seem to like this.
I'm perusing the code base to see if you are able to support private repositories or not, although the wiki seemed to mention support of both..
cc @jumanjiman
The text was updated successfully, but these errors were encountered: