Hosting KVM Disks on MooseFS?

[Lindsay-Mathieson]

How's that looking these days? either RAW or QCOW2. Primarily Windows VM's, running on proxmox

[tianon]

I run a QCOW2-backed QEMU VM on mine, and I only really see issues with it when I restart the MooseFS master (upgrades, maintenance, etc) and the filesystem metadata goes unavailable for the very short period of the restart.

I've been meaning to play with mfsbdev for a while now because I'm guessing it probably has better behavior than simply having a file sitting on my mfsmount instance referenced by QEMU, but haven't had a chance yet (and my container-based deployment of both MooseFS and QEMU each makes that a little harder for me to use). 😅

[Lindsay-Mathieson]

I've been meaning to play with mfsbdev

That would be interesting, though I can't quite get my head around the syntax required for qemu to use it

[tianon]

I finally got a chance to look at it again, and I don't think mfsbdev really solves much for the QEMU use case as-is, so I filed #570 😄

However, maybe the block devices set up and managed by mfsbdev are more stable or performant than having QEMU use a file straight off a standard MooseFS mount -- probably something worth testing but not something I'm set up to test very well right now. 😇

[onlyjob]

See also #188.

[Lindsay-Mathieson]

See also #188.

I'm using proxmox, you can mark storage directories as "shared" in it, allowing you to live migrate VM's between nodes - if the directory actually is shared, I used it quite successfully with LizardFS.

[rusticek]

I was thinking about qcow2 vs raw. As long as moosefs is cow (copy on write), having another cow layout (qcow2) on top of that would not cause unnecessary overhead? Thus possibly lower performance etc.?

[chogata]

MooseFS is not copy-on-write in general - the only thing that behaves like that is the snapshot. So unless you will use snapshots for backups of your kvm disk image, you will not have a double cow :)

[rusticek]

Oh you are definetelly right, i think of mfs as copy on write system because I use snapshots, but in general it is not cow as you said.

[chitvan1988]

Sorry to Hijack the Thread. Few Queries:
1)How to connect Moosefs storage to libvirt as in what should be the Storage Pool type (Refer: https://libvirt.org/storage.html )
2) Is Live Migration doable in KVM using Moofefs as backend.
3) Live Migration is achieved via virsh commands only or is there any cloud like wrapper being used (like CloudStack,opennebulla, openstack ,cockpit(VM Manager), XCP-ng or any other etc)
I have already gone through #188. But some workaround required.

[chitvan1988]

@chogata any ideas/inputs please?

[chogata]

@xandrus Do you have any input? This is your area of expertise :)

[xandrus]

Hi!
OK so things look like this:

1. At the moment MooseFS has no native support in libvirt, so you have to use a standard "dir" as storage pool type.

# On all libvirt/KVM servers storage pool should have the same name:
virsh pool-define-as moosefs dir - - - - "/mnt/moosefs"

2. It is possible, but you have to use unsafe option during migration. Something like this:

 virsh migrate --unsafe --live myVM qemu+ssh://kvm.destination.host/system

3. If you want to use “cloud like wrapper”, the best option is to use XCP-NG, as it has native support for MooseFS client. Check out this link: https://xcp-ng.org/docs/storage.html#storage-types
   In addition, it is very easy to integrate MooseFS with Proxmox. Basically, add a directory with the “Shared” option selected:

Personally, I feel that even if there is no native driver, there is always the possibility of integrating MooseFS with other “cloud-style” solutions such as oVirt, CloudStack, OpenNebula. If you can install a MooseFS client, you can use it with your hypervisor.

[zunder1990]

There is this https://github.com/Zorlin/pve-moosefs

[chitvan1988]

Many thanks for insights