Support for private neworking?

[flowinh2o]

Howdy folks. I have been evaluating some of the features of skypilot for my company and really like what it can do so far! Maybe it's already possible and I missed it, but it would be nice if you could specify existing private networks. For example in our case we have internal infrastructure such as Weights and Biases and gitlab repos that we would like to be able to connect to from the cloud compute environment without using ssh tunnels or exposing the system to the public.

[concretevitamin]

Hi @flowinh2o, great to hear!

For accessing Weights and Biases and gitlab repos, what secrets do they require?

One possible idea is to put those secrets into a private bucket, and then in each YAML you can mount those buckets into /secrets. This way any cluster's setup/run sections have access to them.

[flowinh2o]

Hi @concretevitamin. Thanks for the suggestion. For this discussion I am mainly curious about the network connectivity to privately hosted resources that the cloud based clusters need access to rather than the authentication. The authentication would obviously need to be addressed once we get connectivity as well but think that is a separate topic.

[concretevitamin]

We would love to address this in depth. Do you have more concrete requirements to share about this?

BTW, if you're on AWS, we have private VPC support. See #1693 (comment). You can use the following config to request all SkyPilot clusters be launched in your private VPC. Then, if this private VPC has set up connectivity to "privately hosted resources", it should work.

aws:
  # The VPC to use in each region.
  #
  # If this is set, any region without a unique VPC with this name will not be
  # able to launch SkyPilot nodes. SkyPilot's failover will still properly
  # function to look for such an eligible region.
  #
  # Optional; default: None (SkyPilot will use the default VPC in each region).
  # vpc_name: <name>

  # Set to true to use private IPs to communicate between the local client and
  # any SkyPilot nodes. This requires the networking stack is properly set up.
  # Specifically, setting this flag means SkyPilot will only use subnets that
  # satisfy both of the following to launch nodes:
  #  - subnets with name tags containing the substring "private"
  #  - subnets that are configured to not assign public IPs (the
  #    `map_public_ip_on_launch` attribute is False)
  #
  # This flag is typically set together with 'vpc_name' above and
  # 'auth.ssh_proxy_command'.
  #
  # Optional; default: False.
  # use_internal_ips: False

  # If set, this is passed as the '-o ProxyCommand' option for any SSH
  # connections (including rsync) used to communicate between the local client
  # and any SkyPilot nodes. This option is not used between SkyPilot nodes,
  # since they may not have such a proxy set up.
  #
  # Useful for using a jump server to communicate with SkyPilot nodes hosted in
  # private subnets without public IPs.
  #
  # Optional; default: None.
  # ssh_proxy_command: ssh -W %h:%p -i ~/.ssh/sky-key -o StrictHostKeyChecking=no ssh-user@1.2.3.4

[flowinh2o]

Great to see there is already support for this on aws! Anything similar for gcp? For our use case we have existing on-prem infrastructure and tooling that we would like to leverage during cloud bursting over private networks.

[concretevitamin]

we have existing on-prem infrastructure and tooling that we would like to leverage during cloud bursting over private networks.

We haven't dealt with private networks between on-prem and GCP. Please feel free to PM me on Slack or email so we can learn more on what's actually required.

[flowinh2o]

We would like to connect back to an on premise instance of Weights and Biases and gitlab hosted internally.

[concretevitamin]

I think this may be best handled outside of SkyPilot first. A casual search:

• https://cloud.google.com/network-connectivity/docs/vpn/concepts/overview
• https://cloud.google.com/community/tutorials/using-ha-vpn-with-fortigate

If this ends up working, SkyPilot should just work.