Support for new serverless compute

[gaoshihang]

Describe the feature

Databricks just released the new Serverless Compute at May 1st, I think it will help us a lot on the pipeline, to avoid slow spin-up time of All-purpose/job cluster. So, can we add support for this type of Compute?

Describe alternatives you've considered

Additional context

https://docs.databricks.com/en/workflows/jobs/run-serverless-jobs.html

Who will this benefit?

I think all kinds of job can get benefit from this, because we don't need to control the cluster by ourselves.

Are you interested in contributing this feature?

Yes, but don't know how to do.