Perspectives for development interfaces

[ccrvlh]

Hi all.

I'm starting to learn about MicroVMs (mainly Firecracker and CloudHypervisor) having worked mainly with LXD and Docker. So this is all pretty new to me (and challenging to be honest).

The lack of a wrapper on top of those two tools is challenging for a newbie. Not having something like lxd list or docker run makes for a less then ideal developer experience. In that sense, flitlock is looking great. I haven't had much experience with containerd, but by reading the docs I understand it is the backbone for metadata and image management, while flintlock interacts with firecracker to launch VMs.

As is, there's still the need to build the requests manually. I've seen fl as well, but I wonder what are the plans going forward.
My current use case is not really related to k8s (I understand that's flintlock focus for now), but rather to manage VMs for several applications, anything from workflows to web servers. If I were to think of the ideal interface, I would be thinking of a CLI, somewhat similar to the management capabilities that docker/lxd offer, and a SDK to be able to do things programmatically without having to manually send shell commands (which gives a lot of headaches with parsing and interpreting things).

I was wondering what are the plans moving forward for flintlock, whether there'll be some development in the direction of a flintctl with a Go SDK, and also if this will move towards a generic MicroVM manager vs a K8S cluster management tool. (I saw @richardcase demo on a live with Alex Ellis and things are really impressive indeed!)

I would be happy to contribute to this as well.
Thanks a lot for sharing, really appreciate the work you guys have been doing.

[richardcase]

Hey @lowercase00 👋

Although flintlock was created for the use case of running Kubernetes it actually knows nothing of Kubernetes 😄 That was a conscious design decision so that we didn't bake any assumptions into it. Flintlock can be used for anything that a VM can be used for...so its pretty flexible.

We designed flintlock to be API-driven, as we wanted an easy way to create/delete vms and provide a standard way to integrate into other tooling/orchestration (cluster api in our use case). As flintlock is basically a daemon, after you make an API request it will store the spec of the microvm (using containerd) and then the reconciler in flintlock picks this up and then does all the interaction with containerd (for volume images) and Firecracker to manage the lifecycle of the VM. This has the added benefit that if the host reboots, the vms should be re-created. The idea is that you install a flintlock on any host where you want to run microVMs.

fl was the first experiment in providing a cli experience for flintlock and it can be used to create/delete microvms on demand. It also has some helpers around metadata if you would like to use something like cloud-init. fl basically just calls the gRPC endpoint of a flintlock instance. We do have an open issue (#108) to provide a API client package that would make calling the gRPC endpoint easier.....but we haven't started this.

[richardcase]

Just to give you a heads-up, fl was a quick experiment....so it's not been implemented in an ideal way 😉 It could certainly benefit from a good refactor and moving into the main flintlock repo.

Ah yes, I should also've said the Firecracker slack 😄

[ccrvlh]

Got it, I'll keep that in mind. I've seen https://github.com/warehouse-13/hammertime as well, which seems similar do fl in objective?

[richardcase]

Yes, the difference being with hammertime you supply a request file and fl is more flag driven

[Callisto13]

I've seen https://github.com/warehouse-13/hammertime as well, which seems similar do fl in objective?

In my eyes hammertime and fl fill different use-cases, so i don't see them having much in common for long, and you will use either depending on what you need.

Hammertime is intended to be a very simple, no bells or whistles "one thing at a time" with the response simply dumped back so you can manipulate with jq or something. Basically a more user friendly curl, and it wont be developed further than that stripped back use-case. I made it for quick testing and simple manipulation of resources. fl aims to fulfil a more rounded UX, being to microvms what docker is to containers.

[ccrvlh]

Thanks @Callisto13 , that makes sense.
I was looking at both docker and lxc (which I find it has a great interface and ecosystem). And though of something like this:

flintctl build: should there be a way to build MicroVM images/FS? Not sure how that would look like. Maybe a build command could receive a Dockerfile and output a FS + Kernel to be consumed by flintlock?
flintctl create: this creates (but doesn’t run a MicroVMs)?
flintctl events: should the server keep track of events?
flintctl exec should the tool allow for a shortcut to SSH into the instances? Or maybe something similar to vagrant like flintctl ssh ?
flintctl export would that make sense? Exporting the FS as tar? Or maybe snapshot would already deal with this?
flintctl images would list the available images/fs
flintctl import receives a TAR and make things ready to be used as a firecracker VM?
flintctl inspect look at the instance metadata?
flintctl kill kills the VM
flintctl login/logout would it make sense to connect to a remote registry? Would that be a regular OCI compliant registry, or maybe would also allow for FS + Kernel type of tar?
flintctl logs get the instance logs
flintctl pause find a way to pause the VM? If I remember FC won’t allow for a proper pause, but could turnoff and reconnect to the storage
flintctl list list all VMs (somewhat like lxc list)
flintctl rename can maybe rename or give new ID to VM
flintctl launch create and deploy an instance
flintctl update updates the tool itself
flintctl start/stop related to existing VM
flintctl delete delete everything related to a VM
flintctl network manages the network
flintctl storage manages the storage
flintctl snapshot snapshots an image to tar

(I sent that to @richardcase as well). So this is just me thinking out loud (and with little to no experience with microvms, so purely thinking of things I might want to use) and I may be way off, so would be great to hear some thoughts from you guys on the desired interface.