Shared memory between hypervisor and Baremetal VM

[maskedarray]

Hi,

I was wondering how would one implement a shared memory region between hypervisor and VM?

I am basically trying to profile a few things with bao and baremetal VM and my code would work out quite well if I can share an object between hypervisor and Baremetal VM. Any suggestions would be helpful.

Thanks.

[josecm]

Hey! My first question here is: do you want to map an already existing guest region in the hypervisor or allocate a new one?

In case you just want to map an existing region, you can use the following function:

vaddr_t mem_map_cpy(struct addr_space *ass, struct addr_space *asd, vaddr_t vas,
                vaddr_t vad, size_t num_pages)

It allows you to copy a mapping on the source address space ass starting at virtual address vas of size num_pages * 4K to the destination address space asd at virtual address vad.

The hypervisor currently makes use of this function in vm_install_image to temporarily map the guest image's physical pages and copy the image to it:

...
    vaddr_t dst_va = mem_map_cpy(&vm->as, &cpu()->as, vm->config->image.base_addr,
                                INVALID_VA, img_num_pages);
    memcpy((void*)dst_va, (void*)src_va, vm->config->image.size);
....

As in other mapping functions in Bao, by passing INVALID_VA as the vad, the function will choose a free virtual address in the destination address space and return it.

Let me know if this helps.

[maskedarray]

Thanks for the detailed and prompt response. Assuming the source is hypervisor and destination is Guest, does this work before the guest is initialized or after? How would the hypervisor decide a free virtual address available at Guest AS? Isn't guest supposed to tell which virtual addresses are available?

[josecm]

@maskedarray sorry for the delayed answer

Assuming the source is hypervisor and destination is Guest, does this work before the guest is initialized or after?

Any point after vm_init or at the end of that function would be fine.

How would the hypervisor decide a free virtual address available at Guest AS? Isn't guest supposed to tell which virtual addresses are available?

The hypervisor manages the guest's second-stage page tables, so it knows what addresses are free in the guest's physical address space.
If you want the guest to decide where in its physical address space this shared memory region will be mapped, maybe you could add a hypercall for that effect.

[maskedarray]

I see. I am still looking into how the whole memory management and coloring is working under Bao. I have a crazy idea that I need to try :)

[josecm]

looking into how the whole memory management and coloring is working under Bao

Bao keeps track of allocated pages in a "page pool" struct with a bitmap where each bit represents a page. There is a page pool for each of the memory regions present on the platform description struct. The colors assigned to a given address space are represented using a "colors" mask, where if a bit is set, that color is assigned to that address space. When allocating physical pages, if that address space is not assigned all available colors, the allocation falls back to the pp_alloc_clr, which will allocate n pages that are contiguous with respect to the assigned colors. The number of colors and of pages a color block takes is calculated in cache_calc_colors and stored in COLOR_NUM and COLOR_SIZE, respectively. It essentially boils down to COLOR_NUM = llc_way_size / PAGE_SIZE and COLOR_SIZE = llc_way_size / COLOR_NUM. There is a small detail: Bao does not consider partitioning private caches (i.e., in most platforms, the first-level cache), so the COLOR_SIZE must be at least the size of the first-level cache. In platforms like zynqmp this brings down the number of colors from 16 to 8.

I have a crazy idea that I need to try :)

Looking forward to see that!