python,stdlib: Support added for multiple disk boards

[amatabsc]

This commit introduces the initial support for using multiple disks on different gem5 boards. Currently, only the necessary infrastructure has been added to support multiple disks on the board without implementing full functionality.
Procedure for adding additional functionality:

#Your _add_disk_to_board and _finalize_workload from your_board.py should be something like this:
@overrides(KernelDiskWorkload)
    def _add_disk_to_board(self, disk_image: AbstractResource, is_root: bool): 
        image = CowDiskImage(
            child=RawDiskImage(read_only=True), read_only=False
        )

        image.child.image_file = disk_image.get_local_path()

        if is_root:
            self.disk.vio.image = image

        else:
            idx = len(self._disks)
            new_disk_pio_addr = 0x10009000 + 0x1000 * len(self._disks)
            new_disk = RiscvMmioVirtIO(
                vio=VirtIOBlock(),
                interrupt_id=0x8,
                pio_size=0x1000,
                pio_addr=new_disk_pio_addr,
            )

            new_disk.vio.image = image

            setattr(self, f"disk{idx}", new_disk)
            disk = getattr(self, f"disk{idx}")

            self._disks.append(disk)

    @overrides(KernelDiskWorkload)
    def _finalize_workload(self): #You should create this new method
        # Connect the additional disks added to the board
        self._off_chip_devices += self._disks
        self._setup_io_devices()
        self._setup_pma()

        self.generate_device_tree(m5.options.outdir)
        self.workload.dtb_filename = os.path.join(
            m5.options.outdir, "device.dtb"
        )

        self.workload.dtb_addr = 0x87E00000

cc: @aarmejach
Change-Id: I21d30660bb4dc126ce418f3252f0feb65b41ee0c

[BobbyRBruce]

Am I correct in thinking this is still a WIP/Draft? I'm wondering how much feedback you want here for now.

I'm in strong support of this endevour. In general I think there's a lot we can do to improve the KernelDiskWorkload class. Notably how the device path is determined (I really have no good answers for this, it's a mess right now).

[amatabsc]

Am I correct in thinking this is still a WIP/Draft? I'm wondering how much feedback you want here for now.

I'm in strong support of this endevour. In general I think there's a lot we can do to improve the KernelDiskWorkload class. Notably how the device path is determined (I really have no good answers for this, it's a mess right now).

This board changes are fully supported in riscv. We are already using this changes locally, but I was not sure if you wanted to modify any existing board to use it. Although, I have added the changes in the riscv_matched board as an example. Let me know if I should proceed in a different way.

[BobbyRBruce]

Am I correct in thinking this is still a WIP/Draft? I'm wondering how much feedback you want here for now.
I'm in strong support of this endevour. In general I think there's a lot we can do to improve the KernelDiskWorkload class. Notably how the device path is determined (I really have no good answers for this, it's a mess right now).

This board changes are fully supported in riscv. We are already using this changes locally, but I was not sure if you wanted to modify any existing board to use it. Although, I have added the changes in the riscv_matched board as an example. Let me know if I should proceed in a different way.

Am I correct in thinking this is still a WIP/Draft? I'm wondering how much feedback you want here for now.
I'm in strong support of this endevour. In general I think there's a lot we can do to improve the KernelDiskWorkload class. Notably how the device path is determined (I really have no good answers for this, it's a mess right now).

This board changes are fully supported in riscv. We are already using this changes locally, but I was not sure if you wanted to modify any existing board to use it. Although, I have added the changes in the riscv_matched board as an example. Let me know if I should proceed in a different way.

I've made some suggested changes on the PR via my review.I think it'd be good to split this change up into 1 commit which makes the necessary changes to the the KernelDiskWorkload class and one additional commit for each board updated to utilize this multiple disk image feature. I don't think updating every board to use this is necessary for this PR to be merged (see my review comments for more context on how to implement this).

I would ask for some tests to be included to verify the multiple disk image feature for the boards you submit actually work. I can help you with that: you'd need to provide me with a config script which utilizes multiple disk images on a board and some way to verify that the images have been loaded. This can wait a bit until all the changes have been implemented but should be thought about.

[amatabsc]

Am I correct in thinking this is still a WIP/Draft? I'm wondering how much feedback you want here for now.
I'm in strong support of this endevour. In general I think there's a lot we can do to improve the KernelDiskWorkload class. Notably how the device path is determined (I really have no good answers for this, it's a mess right now).

This board changes are fully supported in riscv. We are already using this changes locally, but I was not sure if you wanted to modify any existing board to use it. Although, I have added the changes in the riscv_matched board as an example. Let me know if I should proceed in a different way.

Am I correct in thinking this is still a WIP/Draft? I'm wondering how much feedback you want here for now.
I'm in strong support of this endevour. In general I think there's a lot we can do to improve the KernelDiskWorkload class. Notably how the device path is determined (I really have no good answers for this, it's a mess right now).

This board changes are fully supported in riscv. We are already using this changes locally, but I was not sure if you wanted to modify any existing board to use it. Although, I have added the changes in the riscv_matched board as an example. Let me know if I should proceed in a different way.

I've made some suggested changes on the PR via my review.I think it'd be good to split this change up into 1 commit which makes the necessary changes to the the KernelDiskWorkload class and one additional commit for each board updated to utilize this multiple disk image feature. I don't think updating every board to use this is necessary for this PR to be merged (see my review comments for more context on how to implement this).

I would ask for some tests to be included to verify the multiple disk image feature for the boards you submit actually work. I can help you with that: you'd need to provide me with a config script which utilizes multiple disk images on a board and some way to verify that the images have been loaded. This can wait a bit until all the changes have been implemented but should be thought about.

I've prepared the following script, have in mind that the rootfs.img and the benchmarks.img should have your own route.

import argparse

from gem5.isas import ISA
from gem5.prebuilt.riscvmatched.riscvmatched_board import RISCVMatchedBoard
from gem5.resources.resource import (
    obtain_resource,
    DiskImageResource,
)
from gem5.simulate.simulator import Simulator
from gem5.utils.requires import requires

requires(isa_required=ISA.RISCV)

parser = argparse.ArgumentParser(
    description="A script which uses the RISCVMatchedBoard in FS mode."
)

parser.add_argument(
    "-i",
    "--to-init",
    action="store_true",
    help="Exit the simulation after the Linux Kernel boot.",
)

args = parser.parse_args()

# instantiate the riscv matched board with default parameters
board = RISCVMatchedBoard(
    clk_freq="1.2GHz",
    l2_size="2MB",
    is_fs=True,
)
additional_disk_images = ["rootfs.img",
                          "benchmarks.img"
                         ]
disk_array = []
for i in range(len(additional_disk_images)):
    if i == 0:
        disk_array.append(
            DiskImageResource(
                local_path=additional_disk_images[i], root_partition="1"
            )
        )
    else:
        disk_array.append(
            DiskImageResource(
                local_path=additional_disk_images[i],
            )
        )
kernel=obtain_resource("riscv-bootloader-vmlinux-5.10")
board.set_kernel_disk_workload(
    kernel=kernel,
    disk_image=disk_array[0],
    additional_disk_images=disk_array[1:],
)

simulator = Simulator(board=board)
simulator.run()

[BobbyRBruce]

@amatabsc : I've added two commits to this PR. The first is just minor formatting changes (nothing functional). The other adds a test for booting multiple disks on the RISCVMatched board. This test is simple. The same disk image is mounted twice and when the system is finished booting, lsblk is run and the output checked to ensure both disks are present.

I'm going to rerun this test to double-check this works (RISCVMatched takes a long time to boot, so bare with me...)/

[BobbyRBruce]

I've found getting test to verify this features is harder than I expected. I'm going to play around a bit with this configuration manually just to confirm this works but I may have to submit this without a test for now.