Add --preserve-host flag and GHORG_PRESERVE_HOSTNAME Environment Variable

[rrrix]

Feature Request

Please add a flag --preserve-host, similar to --preserve-dir.

Also, an environment variable GHORG_PRESERVE_HOSTNAME, similar to GHORG_PRESERVE_DIRECTORY_STRUCTURE.

This should result in the behavior described below.

Examples

Generic

Given the variables:

SCM_HOST=git.example.com
SCM_URL=https://${SCM_HOST}/

The command:

ghorg clone --preserve-host  \
        --scm=$SCM \
        --base-url=$SCM_URL \
        $ORG

would clone to: ${GHORG_ABSOLUTE_PATH_TO_CLONE_TO}/${SCM_HOST}/${ORG}

GitHub Example:

ghorg clone --scm=github --preserve-host kubernetes would clone to ${GHORG_ABSOLUTE_PATH_TO_CLONE_TO}/github.com/kubernetes.

GitLab Example:

ghorg clone --scm=gitlab --preserve-host gitlab-org would clone to ${GHORG_ABSOLUTE_PATH_TO_CLONE_TO}/gitlab.com/gitlab-org

GitLab Self-Hosted Example:

ghorg clone --scm=gitlab --preserve-host --base-url=https://my.gitlab.instance.example.com/ some-gitlab-org would clone to ${GHORG_ABSOLUTE_PATH_TO_CLONE_TO}/my.gitlab.instance.example.com/some-gitlab-org

BitBucket Example:

ghorg clone --scm=gitlab --preserve-host --base-url=https://my.bitbucket.instance.example.com/ some-bitbucket-group would clone to ${GHORG_ABSOLUTE_PATH_TO_CLONE_TO}/my.bitbucket.instance.example.com/some-bitbucket-group

Describe alternatives you've considered

I currently use of the --path option to manually point my clones to ~/ghorg/{scm-hostname}/${org-name}, but this makes using ghorg reclone a little too verbose for my liking, as the reclone.yml file does not support environment variables (e.g. $HOME).

Using vmware^{1, 2} and saltstack^{3, 4} as examples, which have both GitLab.com and GitHub.com organizations, if I could set GHORG_PRESERVE_HOSTNAME in my ~/.config/ghorg/conf.yaml, I could then simplify my reclone.yaml as:

# ~/.config/ghorg/reclone.yaml

# GHORG_PRESERVE_HOSTNAME: True in ~/.config/ghorg/conf.yaml

## VMWare ## 
# clone to ~/ghorg/github.com/vmware
vmware-github:
  cmd: "ghorg clone vmware --scm=github"

# clone to ~/ghorg/gitlab.com/vmware
vmware-gitlab:
  cmd: "ghorg clone vmware --scm-gitlab"

## Saltstack ##
# clone to ~/ghorg/github.com/saltstack
saltstack-github:
  cmd: "ghorg clone saltstack --scm=github"

# clone to ~/ghorg/gitlab.com/saltstack
saltstack-gitlab:
  cmd: "ghorg clone saltstack --scm=gitlab"

It currently looks like:

`~/.config/ghorg/reclone.yaml` with absolute paths

# ~/.config/ghorg/reclone.yaml

# GHORG_PRESERVE_HOSTNAME: True in ~/.config/ghorg/conf.yaml

## VMWare ## 
# clone to ~/ghorg/github.com/vmware
vmware-github:
  cmd: "ghorg clone vmware --scm=github --path /Users/me/ghorg/github.com"

# clone to ~/ghorg/gitlab.com/vmware
vmware-gitlab:
  cmd: "ghorg clone vmware --scm-gitlab --path /Users/me/ghorg/gitlab.com"

## Saltstack ##
# clone to ~/ghorg/github.com/saltstack
saltstack-github:
  cmd: "ghorg clone saltstack --scm=github --path /Users/me/ghorg/github.com"

# clone to ~/ghorg/gitlab.com/saltstack
saltstack-gitlab:
  cmd: "ghorg clone saltstack --scm=gitlab --path /Users/me/ghorg/gitlab.com"

Not to mention self-hosted GitLab, BitBitbucket, and more...

• Debian self-hosted: https://salsa.debian.org/debian
• Debian GitHub.com: https://github.com/debian
• Alpine Linux self-hosted: https://gitlab.alpinelinux.org/alpine
• Atlassian Bitbucket.org: https://bitbucket.org/atlassianlabs/workspace/projects/

Additional Context

I use ghorg frequently, it's awesome!! As part of my role as a Security Operations Software Engineer, I'm constantly looking to ghorg (and frequently combine it with tools like ag) to perform software reviews, analysis and other activities. The killer feature is the ability to clone entire orgs across multiple SCM's, Cloud-Hosted or Self-Hosted.

Before ghorg, I did this all by hand with custom Python tooling that I had to build and maintain. Thanks @gabrie30!

[gabrie30]

Thanks for raising this thoughtful issue @rrrix! Great to hear how you're using ghorg too!

I do see some value in this feature. However, I'm not sure when I'll have to time to get around to it.

In the meantime, what I would suggest trying is the --output-dir flag. It's not exactly what you want however it might make things a little easier on you. For example with the Debian clones you could do.

ghorg clone debian --output-dir=debaian-cloud
ghorg clone debian --base-url=https://salsa.debian.org --ouptput-dir=debian-self-hosted

This will put both inside your ghorg dir but you'll be able to tell them apart much easier.