Skip to content

Releases: skypilot-org/skypilot

SkyPilot v0.5.0

27 Feb 04:36
@Michaelvll Michaelvll
v0.5.0
1e4e871
Compare
Choose a tag to compare
SkyPilot v0.5.0 Latest
Latest

SkyPilot v0.5.0: SkyServe, New Provisioner, LLMs, Kubernetes, and More Clouds

We are excited to release SkyPilot v0.5.0, where we introduce a significant amount of new features and enhancements, including:

  • SkyPilot Serving
  • New provisioner
  • LLM recipes for the latest open models and engines
  • Kubernetes support improvement
  • 4 new clouds (contributed by the cloud providers!)

and more!

Release Highlights

New Features

  • Multiple candidate resources: SkyPilot now supports multiple candidate resources for a single task (using multiple accelerators, any_of or ordered in resources), allowing users to significantly enlarge the resource pool and get higher availability.
  • New Provisioner: Provisioner gets a new implementation, which is 2x faster and more reliable for supported clouds. Support launching clusters with more than 100 nodes. Dependency requirements for clouds are also significantly reduced.
  • Disk Tier: Introducing best disk tier for the best performance and cost, so you can choose the best disk for any cloud. (#2434)
  • Allow 2x spot jobs to be run concurrently
  • Mount storage back after cluster restart

SkyServe

SkyServe is a serving system on top of SkyPilot that deploys and scales any HTTP services across one or more regions or clouds, with autoscaling, load balancing, and more.

  • Introducing SkyServe: deploy and scale your AI models across multiple regions or clouds. (#2458)
  • Autoscaler: Request rate based autoscaling policy. (#2868, #2878)
  • Autoscaler: Support scaling to 0 when no requests (#2938)
  • Rolling update: Support rolling update for existing services (#2935, #3057)

Other Enhancements

New LLM Recipes

Kubernetes

Kubernetes support received a number of New Features and Enhancements.

  • Multi-node support for Kubernetes (#2609, #3019)
  • Open ports support for Kubernetes (#2588, #2713, #2997, #3200)
  • Support Coreweave label for GPUs in Kubernetes (Coreweave support under development) (#2650)
  • Starting a kubernetes GPU cluster locally with sky local up (#2890)
  • Custom Image Support for Kubernetes Instances (#2729, #3019, #3210)
  • New provisioner for kubernets for better performance and robustneess (#3019)
  • Supporting Kubernetes cluster launched with k3s and Rancher (#3148)

Other Enhancements

More Clouds

SkyPilot now supports 13 cloud providers, including 4 new provider-contributed clouds: VMWare vSphere, RunPod, Fluidstack and Cudo Compute.

Clouds

AWS

New Features

  • New provisioner for AWS: >2x faster for multi-node provisioning and more reliable for cluster launching. (#1702, #2719, #2792)
  • Support for AWS Trainium accelerator (#2690)
  • Support null for proxy command to filter regions (#2756)
  • Support CUDA 12.1 with default image updates (#2788)
  • Job scheduling on Inferentia and Trainium (#2969, #2798)
  • Allow specifying security_group (#3133)

Enhancements

  • Make public / private subnet selection robust (#2867)
  • Avoid hanging for restarting an instance in STOPPING state (#2998)
  • Remove sunset instance types (#2610)
  • Add docs for custom VPC support (#2776)

Fixes

  • Fix conda installation on AWS default image (#3206)
  • Robustify the custom image support (#3216)
  • Fix subnet selection for AWS and autodown for spot instances (#2921)
  • Fix minimal permission for AWS (#2978)
  • Improve opening ports for AWS (#2716)
  • Autstop with new provisioner (#2719)

GCP

New Features

  • Security: Custom VPC support for GCP. (#2764, #2772, #2854, #2944)
  • Security: Support private IP with proxy jump on GCP. (#2819)
  • New provisioner: Adopted new provisioner for GCP with >2x faster and more robust provisioning (#2681, #2719, #2943)
  • Automatically use reserved instances from multiple reserved pools (#2836, #2681)
  • Support L4 accelerator for GCP (#2724)
  • Allow stopping spot clusters on GCP (#2877)

Enhancements

  • Allow stopping VM with local SSD (#2587)
  • Update default runtime version for TPU node (#2601, #2602)
  • Handling transient error during launching GCP clusters (#2669)
  • Update GCSFuse version to 1.3.0 for GCS storage mount (#2887)
  • Set TPU VM the default option for TPU accelerators (#1758)
  • Ignore missing gcp credentials for latest gcloud and avoid duplicating credentials (#3028, #3172, #3234)

Fixes

  • Fix custom docker image support (#3218)
  • Fix minimal roles required for GCP (#2704)
  • Robustify the catalog fetching (#3141)
  • Fix ports on TPU VM and cluster launched before 0.4.0 (#2641)
  • Fix backward compatibility issue with GCP clusters (#2604)
  • Fix --disk-size for Custom Machine Images (#2718)
  • Update catalog fetcher with more options (#2562)
  • Assign GCP VMs with service account (#2972)
  • Fix machine image support (#3030, #3236)
  • Fix error handling for failed provisioning (#2852)
  • Leave out TPU v5 in catalog as it is not supported (#2656)
  • Fix GCP minimal permission (#2947, #2770, #2761)

Azure

Enhancements

  • Make ports openning more robust (#2649, #2891, #3084)
  • Additional arguments for Azure catalog fetcher and support H100 (#2561, #2844, #2847)
  • Support CUDA 12.1 with default image updates (#2468)
  • Support spot instances on Azure (#2871)

Fixes

  • Fix custom docker image support (#3218)
  • UX: Fix Azure disk tier explicitly shown in resources str (#3064)
  • Fix status query for Azure (#3015)

SCP

  • Fix SCP error raised in sky check (#3038)

CLI & Core interfaces

New Features

  • Multi-node jobs fail fast fast for single node failure (#3081)
  • Add configurations for not uploading credentials (#2904)
  • Adding sky status --endpoints CLI (#3199)
  • Support more characters in cluster name (#3130)
  • Show all regions and more accurate price in sky show-gpus (#2583, #2892, #2933, #2946, #3083, #3149, #3113)
  • Allow infering cloud from region or zone (#2632)
  • Add --commit and --version for sky CLI (#2720, #2731, #2733)

Enhancements

  • Robustify runtime initialization on remote cluster (#3132)
  • Better error message for YAML parsing (#3040)
  • Smarter GPU name completion (#3014)
  • Speed up retry until up by not doing exponential backoff (#2821)
  • Add schema validation for config (#2645)
  • Allow --disk-tier none override (#2906)
  • sky check improvement (#3174, #3212, #3160)
  • Better logging for CLIs (#2535, #2691, #2728, #3139, #3175)

Fixes

  • Fix permissi...
Read more

Contributors

thams, concretevitamin, and 32 other contributors
Assets 2

SkyPilot v0.4.1

29 Oct 21:05
@Michaelvll Michaelvll
v0.4.1
bfedf45
Compare
Choose a tag to compare
SkyPilot v0.4.1

This is a patch release to ship bug fixes faster to our users! This release includes many feature updates and bug fixes, including the new provisioner for AWS, fixing OOM and credential issues for long-running spot jobs, and some additional improvements.

Detailed changelog coming up in v0.5!

Assets 2

SkyPilot v0.4.0

19 Sep 16:57
@romilbhardwaj romilbhardwaj
v0.4.0
c324612
Compare
Choose a tag to compare
SkyPilot v0.4.0

SkyPilot v0.4.0: Kubernetes, native containers, ports and new clouds

We are excited to release SkyPilot v0.4.0, which brings a host of new features and improvements, including Kubernetes support, native container support, ability to open ports, and more.

Release Highlights

New Features

  • Kubernetes support: SkyPilot tasks and clusters can now run on Kubernetes clusters, including on-prem and cloud hosted deployments (GKE, EKS).
    • If you have a working kubeconfig, simply run sky check and sky launch --cloud kubernetes to run your task on Kubernetes.
    • If desired, tasks can also failover to the cloud when the Kubernetes cluster does not have enough resources. The same SkyPilot YAMLs and CLI works seamlessly across Kubernetes and clouds.
  • Opening ports on clusters: Open ports on your clusters with the ports field. These ports are publicly accessible and can be used for hosting LLM inference endpoints, Jupyter notebooks, web servers, Tensorboard, and other services.
  • Native container support: If your task uses docker containers, SkyPilot's setup and run commands can now directly be executed in that container. This allows you to wrap your environment in a container and run it on any cloud with SkyPilot.
  • Reservation support: This release adds support for GCP reservations. SkyPilot will now prioritize using your reservations on the cloud to save costs and get higher availability.
  • New Managed Spot Features

New LLM Recipes

More Clouds

SkyPilot now supports 8 clouds, including community contributed support for two new clouds:

SkyPilot now also supports IBM COS buckets (#1966).

Core and UX Improvements

  • Faster failover: 30x faster failover with our new quota optimization which checks if quotas are available before launching a cluster (Supported on GCP, AWS).
  • Easily get VM IPs: The new --ip flag for sky status returns the public IP address of the cluster (e.g., sky status --ip mycluster). Use this to access services such as LLM inference endpoints, jupyter notebooks and more.
  • Improved scriptability: SkyPilot YAMLs and CLI are more scriptable than ever - file_mounts can be dynamically defined with environment variables (docs, example), environment variables can be set through a dotenv file with the new --env-file flag (#2296).
  • Core optimizations: Multi-node clusters stop 4x faster (#2199), sky status updates for stopped clusters are 10x faster (#2288), and the job queue is more memory efficient (#1636).
  • Nightly releases: We now release nightly versions of SkyPilot. To get the cutting edge of SkyPilot without installing from source, run pip install skypilot-nightly (#1446)

Deprecation

  • SkyPilot On-prem is now deprecated and Kubernetes will be the recommended mode of running SkyPilot on on-prem clusters.

Below is a detailed list of changes.

Managed Spot

New Features

  • Spot pipeline support: automatically handles a pipeline of spot jobs. (#1982)
  • Spot dashboard is now available with sky spot dashboard: you can now see all your spot jobs in GUI (#2103, #2136)
  • Spot callback - users can now run custom code when spot job status changes (#2106, #2364)
  • Resource configuration of the spot controller can now be customized (docs, #2040)

Enhancements

  • SkyPilot now shows the spot job's resources and estimated cost before confirmation (#2524)
  • Switch to eager failover recovery policy for better spot lifetime (#2234)
  • Reduce the logging for launching spot controller (#2056)

Fixes

CLI & YAML interfaces

New Features

  • Users can now use environment variables to dynamically define file_mounts (docs, #2146)
  • sky status can now show the head IP of the cluster with -a or --ip flags (#2305, #2563)
  • sky down/stop/start defaults to a unique cluster if it exists and sky cancel without cluster cancels the latest task (#2325)

Enhancement

Fixes

  • Fixed the order of VMs in optimizer table when --cpus is provided (#2037)
  • Better handling when sky launch is interrupted (#2206, #2252)

Backend

New Features

  • Users can now open ports for their clusters with the ports field (docs, #2210, #2477)
  • Docker support in image_id - tasks can now be run inside docker containers (docs, #1910)
  • Users can now clone a cluster from an existing cluster's disk with the --clone-disk-from flag (#2098)
  • Users can now launch their own ray cluster on a SkyPilot cluster (#2020)

Enhancements

  • 30x faster failover for AWS and GCP when quotas are not available (#1953, #2187, #2313)
  • Faster sky launch by caching cluster IP address (#2400)
  • Job queue is now more resource efficient, with significant memory consumption reduction on remote cluster (#1636)
  • Cluster names no longer map directly to cloud cluster names. Instead, they are mapped to a unique cluster name on the cloud. This helps with isolation across users sharing cloud accounts. (#2403)
  • More efficient and robust stopping/termination for AWS (#2121)
  • sky status --refresh for STOPPED cluster is 10x faster (#2079)
  • Empty YAML fields are now allowed (#1890)

Fixes

Storage

New Features

  • IBM COS is now supported (#1966)
  • sky spot launch will now exclude files from .gitignore (#2018)

Enhancements

  • Deletion is now parallelized for faster deletion (#2058)
  • UX improvements for sky storage CLI (#2063, #2177)
  • GCS bucket mounting now uses gcsfuse v1.0.1 (#2470)

Fixes

Dependencies

  • Avoid buggy grpcio versions (#2055)
  • Pydantic is pinned to <2.0 (#2157)
  • PyYAML is pinned to >3.13, != 5.4.* to avoid issues with Cython 3 (#2256, #2514)
  • Ray <= 2.6.3 is supported on local machines (#2401)
  • pycryptodome, oauth2client are no longer required (#2515)

Clouds

AWS

  • H100 GPUs are now supported (#2323)
  • New docs for AWS cloud administrator about advanced login option (SSO and account switching) (#1888)
  • Insufficient permission is now handled gracefully (#2415, #2456)
  • Fixed a bug where existing AWS cluster would end up in INIT state after changing identity (#2442)
  • Fix fetching AZ when describe zones permission does not exist in all regions (#2463)

GCP

Read more

Contributors

tobi, pushmatrix, and 32 other contributors
Assets 2

SkyPilot v0.3.3

17 Jul 23:38
@romilbhardwaj romilbhardwaj
v0.3.3
f42c032
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
SkyPilot v0.3.3

This patch release brings many bug fixes and features, including new mechanics for stop/down, callbacks for spot jobs and a critical dependency fix for PyYAML after the release of cython 3.

Detailed changelog coming up in v0.4!

Assets 2

SkyPilot v0.3.2

01 Jul 05:21
@Michaelvll Michaelvll
v0.3.2
3b3e633
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
SkyPilot v0.3.2

This is a patch release to ship bug fixes faster to our users! This release includes many feature updates and bug fixes, including the pedantic dependency issue, disk cloning, file mounts, and cloud-specific improvements.

Detailed changelog coming up in v0.4!

Assets 2

SkyPilot v0.3.1

04 Jun 17:29
@concretevitamin concretevitamin
v0.3.1
344a70b
Compare
Choose a tag to compare
SkyPilot v0.3.1

This is a patch release to ship several important enhancements and bug fixes:

Enhancements

  • On-demand H100 GPU from Lambda is supported! sky launch --gpus h100
    • To use it, remove any previous Lambda catalog: rm -rf ~/.sky/catalogs/v5/lambda
  • Managed spot: make job cancellation during failover more robust to mitigate a rare FAILED_SETUP error (#1998)

Fixes

  • Provisioner / Backend
    • Fix provision failover encountering FileNotFoundError (#2005)
    • Fix user-level ray cluster causing SkyPilot cluster to be in INIT state (#2020)
  • Logging
    • Fix certain logs of multi-node jobs not being streamed due to Ray 2.4 log dedup (#2026)
    • Fix logs being created in current pwd $PWD/~/sky_logs in some cases (#2009)
  • Managed spot
    • Fix sky spot launch --retry-until-up to make it actually retry until up (#2004)
  • Storage
    • Fix a rare storage cloud check error if sky check has never been called (#2017)
  • On-prem
    • Fix detecting A5000 and A6000 GPUs (#2023)

Full Changelog: v0.3.0...v0.3.1

Assets 2

SkyPilot v0.3.0

30 May 17:29
@concretevitamin concretevitamin
v0.3.0
3deccb2
Compare
Choose a tag to compare
SkyPilot v0.3.0

SkyPilot v0.3.0: LLM Support, New Clouds, Enhanced Production-Readiness

We are excited to release SkyPilot v0.3, the most significant release thus far in the project's history.

v0.3 focuses on:

  • LLM support (Vicuna, LLaMA)
  • New clouds (Lambda Cloud; IBM; Cloudflare R2)
  • Enhanced production readiness

See the release blog post for a deep-dive into highlights.

Release notes below are as compared to v0.2 (full changelog).

Release Highlights

  • LLM support
    • Vicuna LLM chatbot trained using SkyPilot for $300 on spot instances!
    • Serve your own LLaMA LLM chatbot on any cloud: full example, blog, repo
    • Significantly expanded GPU availability by leveraging the widest selection of clouds (see below)
  • More clouds, more choices: delivering the highest GPU availability & cost savings
    • Lambda Cloud is now supported!
      • This brings high-end GPUs at lower costs to SkyPilot. (#1557, #1838)
      • Simply run sky check to set it up. Docs here.
    • IBM Cloud is now supported!
      • This brings the first hyperscaler cloud after AWS/GCP/Azure to SkyPilot. (#1598)
    • Cloudflare R2 object store is now supported!
  • Managed Spot is made significantly more robust via a host of fixes/enhancements.
  • Cluster leakage prevention and detection are significantly improved.
  • CLI/API & Backend shipped many new features:
    • sky cost-report; fine-grained optimizer; user identity; AWS SSO; private IP-only VPCs; Ray runtime is decoupled from user's Ray clusters; ...

CLI/API

New Features

  • New CLI sky cost-report: show the estimated cost of launched clusters (#1301, #1621, #1780, #1680, #1788)
    • Experimental: Costs for clusters with auto{stop,down} scheduled may not be accurate.
  • New resource filtering support in sky launch / YAML resources: field
    • Add vCPUs --cpus support #1622
    • Add memory --memory support #1746
    • Add disk performance tier --disk-tier support #1812
  • Add --detach-setup and --detach-run to sky launch #1379
  • Add --retry-until-up, --region, --zone, and --idle-minutes-to-autostop for interactive nodes #1297
  • Add autodown (#1217, #1254)
  • Support calling sky status/sky.status() on specific clusters #1568
  • Support --region in sky show-gpus #1187
  • Support passing AMIs for different regions in image_id field under resources #1384

Enhancements

  • Improvements to sky show-gpus
    • Show DEVICE_MEMORY for AWS & Lambda #1825
    • Support querying specific number/type of devices sky show-gpus <gpu>:<num> (same syntax as sky launch --gpus) #1924
  • Check image existence and its size can fit in OS disk #1508
  • Make sky down -p bypass identity mismatch errors. #1892

Fixes

  • Make repeated sky {cpu,gpu,tpu}node commands correctly reuse existing cluster if possible #1787
  • Fix errors from empty 'resources' field in YAML. #1816
  • Make autostop more robust for AWS custom images that by default export 2 credential env vars (#1880, #1894, #1946)

Managed spot

New Features

  • Latest in-progress spot jobs are shown in sky status (#1270, #1467, #1691)
  • Detailed reasons for failed spot jobs are exposed in sky spot queue -a (#1655)

Enhancements

  • Make sky spot launch default -r/--retry-until-up to True. #1781
  • Make job termination/cancellation significantly more robust (#1433, #1745)
  • Catch "pre-launch" errors early (e.g., invalid cluster names, no cloud access) to avoid unnecessary retries (#1714)
  • sky start on the spot controller resets the default autostop #1453
  • sky spot queue displays job states with colors (#1473)
  • sky spot queue no longer shows a cached (and possibly stale) version of the jobs (#1742)
  • Disallow sky down on spot controller when in-progress spot jobs exist #1667
  • New state FAILED_SETUP for spot jobs that fail during setup (#1479)
  • New state CANCELLING for spot jobs that are being cancelled (#1785)
  • Keep env var SKYPILOT_JOB_ID the same for all recoveries of the same job #1400

Fixes

TPU

Robustness is enhanced for TPUs in various modes: VMs, pods, spot (#1500, #1279, #1359, #1483, #1562, ...).

Provisioner

Enhancements

  • Cluster leakage prevention is significantly improved!
    • Skip Ray's launch hash check, which caused many leakage (#1671)
    • Launch existing cluster in the same zone to avoid leakage (#1700)
    • Existing cluster's cluster YAML will keep certain fields unchanged across re-launch (#1235, #1251)
    • Fix leakage of existing cluster when failed to start #1497
  • Disable unattended-upgrade (nondeterministic APT lock) on cluster start
    • Previously, apt install ... in setup may non-deterministically fail due to APT lock being held by background unattended upgrades
    • Now: for AWS cloud-init ensures unattended-upgrade is disabled at boot (#1949, #1954); for other clouds we kill the processes (#1347)
  • Generate valid cluster names when username has invalid characters #1526

Fixes

Storage

New Features

  • Cloudflare R2 is now supported! #1736
    • R2 is an S3-compatible object store with zero egress fee.
    • To use it, see setup docs and usage docs.
  • Support multiple paths in the source of a storage mount, e.g., source: [~/mydir/myfile.txt, ~/datasets] #1311 #1677

Enhancements

Fixes

  • Fix sky storage delete for externally deleted buckets #1875
  • Disallow single files for upload to Storage #1231
  • fix rsync for paths with spaces #1190

Backend

New Features

  • New feature: Fine-grained optimizer
    • Optimizing & provisioning retries at the granularity of regions/zones #975
    • In other words, SkyPilot now automatically recognizes and optimizes across the cost differences between zones (e.g., AWS zones have different prices for the same spot instance type) or regions
  • New feature: User identity is associated with each cluster (#1513, #1550, #1809)
    • Identities are e.g., different AWS profiles / GCP projects
    • With this, users are free to switch across identities, and SkyPilot will properly protect each cluster

Enhancements

  • Ray runtime on SkyPilot clusters is upgraded to v2.4.0 (#1734)
    • All existing clusters are automatically upgraded on its next sky launch/start
    • Local client's ray requirement is updated to ray[default]>=2.2.0,<=2.4.0 to fix some dependency conflicts with click/grpcio/protobuf
  • Ray cluster used by the SkyPilot runt...
Read more

Contributors

turian, stephenbalaban, and 21 other contributors
Assets 2

SkyPilot v0.2.5

20 Mar 19:19
@romilbhardwaj romilbhardwaj
v0.2.5
84dbfd3
Compare
Choose a tag to compare
SkyPilot v0.2.5

Another patch release to ship bug fixes faster to our users! This release includes many fixes, including those for managed spot and cloud specific improvements.

Detailed changelog coming up in v0.3!

Assets 2

SkyPilot v0.2.4

06 Feb 06:43
@romilbhardwaj romilbhardwaj
v0.2.4
2a0b9ba
Compare
Choose a tag to compare
SkyPilot v0.2.4

This patch release brings more bug fixes, including fixes for cloud-specific networking and VPC configuration and managed spot.

Detailed changelog coming up in v0.3!

Assets 2

SkyPilot v0.2.3

27 Jan 01:06
@romilbhardwaj romilbhardwaj
v0.2.3
2511c87
Compare
Choose a tag to compare
SkyPilot v0.2.3

What's Changed

This is a patch release with lots of bug fixes across the board, including many cloud-specific networking and VPC fixes.

Stay tuned for a detailed changelog coming up in v0.3!

Assets 2