[2024] AArch64 support #1088
[2024] AArch64 support #1088
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
Thanks very much for this, it looks awesome! I haven't been through it all, I've most concentrated on the arm layer at the moment (and I haven't made my way through all of that). The comments pointing to the location of useful documentation is massively useful and very much appreciated! 5:D There's a few little bits that can be cleaned up but on the whole it looks perfectly in line with the rest of volatility, so good work! It'll take me a while to find time to go through more of it, and check the various references to fully get what's going on.
I'm trying to figure out the split between mapping and _mapping, it looks like _mapping returns every chunk, and mapping coalesces them together is that about right? I think that's probably for the best, but it feels like it makes for bulky code somehow. Really excited to see where this codes (and find some Windows-on-arm samples to try it against too!). 5:D
| kaslr_shift = init_task_address - cls.virtual_to_physical_address( | ||
| init_task_json_address | ||
| ) | ||
| if layer_class == arm.AArch64: |
There was a problem hiding this comment.
Rather than changing the API, it should be possible to derive this by getting the layer using layer_name and then doing an if on the class of the layer. The if statement feels a little ugly though, since it'll need support for every layer that comes about. It might be worth separating this out into a class that can be attached to each class, so that a new layer class can be introduced without tinkering with this code in the future?
There was a problem hiding this comment.
As the whole find_aslr function worked out of the box, without the need to use the virtual_to_physical_address function, I did this quick fix to avoid repeating the whole function (separating in find_aslr_aarch64 and find_aslr_intel).
But if in the future we have to add some other specific AArch64 code to find_aslr (e.g. handle some retrocompatibility), we will eventually do a clear separation. So I guess we could do it directly even if the code will be 99% the same ?
| ) | ||
| self._base_layer = self.config["memory_layer"] | ||
| # self._swap_layers = [] # TODO | ||
| self._page_map_offset = self.config["page_map_offset"] |
There was a problem hiding this comment.
There are two pages maps as part of AArch64, aren't there? Which one should people provide here? Might be better to use the terminology used by the architecture here, rather than reusing values used elsewhere...
There was a problem hiding this comment.
In the Linux Stacker, we are instantiating the kernel Layer. We pass the kernel DTB (Translation Table Base 1 Base Address) to the AArch64 layer. There is in fact a unique page map for the whole kernel, stored in ARM register TTBR1, and a page map for the user land stored in TTBR0. TTBR0 is not constant, and changes on every process context switch. What this means, is that TTBR0 is changed to the value of the current process DTB (which happens continuously).
For me, there are only two possible callers to this layer :
- LinuxStacker (kernel, only once)
- "Processes"
People should not have to instantiate the kernel layer twice, but only user space layers (programs) ? In another comment I detailed how I implemented a check to facilitate this.
volatility3/framework/layers/arm.py
Outdated
| # self._swap_layers = [] # TODO | ||
| self._page_map_offset = self.config["page_map_offset"] | ||
| self._tcr_el1_tnsz = self.config["tcr_el1_tnsz"] | ||
| self._page_size = self.config["page_size"] |
There was a problem hiding this comment.
Is this part of the architecture, or is it dynamic, or is it something pre-defined?
There was a problem hiding this comment.
_page_map_offsetis the DTB (either kernel or specific to a process). For the kernel, it can be calculated thanks to the existing Intel implementation. For the process, I don't know exactly where, but it is stored in thetask_structdirectly if I'm not mistaken._tcr_el1_tnsz: The size offset of the memory region addressed by TTBR1_EL1. The region size is 2(64-T1SZ) bytes -> See page 7081 of the doc. It is defined in the kernel config, but we cannot predict it (it is calculated in LinuxStacker)._page_size: Page size (4, 16 or 64 KB), also defined in the kernel config, but we cannot predict it (it is calculated in LinuxStacker).
| self._tcr_el1_tnsz = self.config["tcr_el1_tnsz"] | ||
| self._page_size = self.config["page_size"] | ||
|
|
||
| # Context : TTB0 (user) or TTB1 (kernel) |
There was a problem hiding this comment.
So this layer will only cope with one of user or kernel? That might be complex to handle in the future?
There was a problem hiding this comment.
This layer needs to be instantiated one time for the kernel, and as many times as there are processes. End users should not have to instantiate the kernel layer twice, and instantiating a process layer is basically the same as the Intel logic (I think mostly of malfind, which I used to validate the processes layers translations).
volatility3/framework/layers/arm.py
Outdated
| 49 | ||
| if self._ttbs_granules[ttb] in [4, 16] and self._is_52bits[ttb] | ||
| else 47, | ||
| self._ttb_lookups_descriptors[ttb][-1][1], |
There was a problem hiding this comment.
It would be handy to have some descriptions of what the array looks like somewhere (maybe in the docstring for _determine_tbbs_lookup_descriptors), otherwise it's difficult to tell what the last element would contain.
There was a problem hiding this comment.
Noted, I will take a look !
volatility3/framework/layers/arm.py
Outdated
| ) | ||
| level += 1 | ||
|
|
||
| if AARCH64_TRANSLATION_DEBUGGING: |
There was a problem hiding this comment.
Again, probably better to have a togglable instance variable, rather than a global module variable.
volatility3/framework/layers/arm.py
Outdated
| name="memory_layer", optional=False | ||
| ), | ||
| requirements.IntRequirement(name="page_map_offset", optional=False), | ||
| requirements.IntRequirement(name="page_map_offset_kernel", optional=False), |
There was a problem hiding this comment.
This... doesn't look like it's used anywhere, and anything related to the kernel shouldn't be a requirement of the layer to operate, it should be optional, because the architecture should be able to map with or without a kernel?
There was a problem hiding this comment.
"page_map_offset_kernel" is instantied once by the LinuxStacker, when creating the kernel layer. This implies that any other caller will try to stack in user space, without passing "page_map_offset_kernel" but only "page_map_offset" (it is the same naming as Intel, to avoid rewriting everything). Thanks to how the framework is designed, the class will still be able to access "page_map_offset_kernel", which indicates what is the current context (am I being instantiated for user space or for kernel space ?). See :
self._virtual_addr_space = (
self._page_map_offset == self.config["page_map_offset_kernel"]
)self._virtual_addr_space = 0 -> we are in user space/context (TTB0)
self._virtual_addr_space = 1 -> we are in kernel space/context (TTB1)
Additionally, we need to keep track of the context, because of this part :
# Check if requested address belongs to the context virtual memory space
if ttb_selector != self._virtual_addr_space:
raise exceptions.InvalidAddressException(
layer_name=self.name,
invalid_address=virtual_offset,
)Making the page_size or tcr_el1_tnsz options optional isn't viable, because they are constant for their whole spaces. In my sense, calculating everything once and for all in the LinuxStacker, allows compatibility with existing LinuxIntel APIs and facilitates higher layers stacking. As pointed out, Volatility will keep track of the previous config options for layers stacking on top of the AArch64 one.
volatility3/framework/layers/arm.py
Outdated
| ), | ||
| requirements.IntRequirement(name="page_map_offset", optional=False), | ||
| requirements.IntRequirement(name="page_map_offset_kernel", optional=False), | ||
| requirements.IntRequirement(name="tcr_el1_tnsz", optional=False), |
There was a problem hiding this comment.
You can actually provide a description field with requirements, which might be quite handy here?
There was a problem hiding this comment.
Noted, I will take a look !
volatility3/framework/layers/arm.py
Outdated
| requirements.TranslationLayerRequirement( | ||
| name="memory_layer", optional=False | ||
| ), | ||
| requirements.IntRequirement(name="page_map_offset", optional=False), |
There was a problem hiding this comment.
Might also be good to provide a description here (as noted later, Requirement classes can take a description attribute. Just some clarification on exactly what this would be for arm, since it's not quite as obvious as with intel....
There was a problem hiding this comment.
Noted, I will take a look !
volatility3/framework/layers/arm.py
Outdated
| def maximum_address(cls) -> int: | ||
| return (1 << cls._maxvirtaddr) - 1 | ||
|
|
||
| def __canonicalize(self, addr: int) -> int: |
There was a problem hiding this comment.
These two don't appear to get called from anywhere and are strictly private? Are they necessary? If they can't be depended on in children of this class, then they don't seem all that useful?
There was a problem hiding this comment.
Inspired from : https://github.com/volatilityfoundation/volatility3/blob/develop/volatility3/framework/layers/intel.py#L93 and https://github.com/volatilityfoundation/volatility3/blob/develop/volatility3/framework/layers/intel.py#L119.
I realize now that I messed up the visibility of canonicalize, and will fix it. I did not investigated which higher level code was likely to use these functions, but it definetely crashed when they were missing.
|
Hi @ikelos, thanks for your quick replies ! I will check every of your comment right now, as I have some spare time. Regarding your general concern regarding |
|
Hehehe, yeah, I figured it might be. That's not too sweet the intel layer isn't overly complicated but it's fine for now. If you find any ways of simplifying it we can back port then to intel too... 5;D |
|
Here is what a typical layer instantiation debug looks like : Kernel layer instantiated by DEBUG volatility3.framework.automagic.linux: Linux ASLR shift values determined: physical -ffffffbfc7e00000 virtual 172a600000
DEBUG volatility3.framework.layers.arm: Base layer : Elf64Layer
DEBUG volatility3.framework.layers.arm: Virtual address space : kernel
DEBUG volatility3.framework.layers.arm: Virtual addresses space range : ('0xffffffc000000000', '0xffffffffffffffff')
DEBUG volatility3.framework.layers.arm: Page size : 4
DEBUG volatility3.framework.layers.arm: T1SZ : 25
DEBUG volatility3.framework.layers.arm: Page map offset : 0x41963000
DEBUG volatility3.framework.layers.arm: Translation mappings : [(38, 30), (29, 21), (20, 12)]
DEBUG volatility3.framework.automagic.linux: Kernel DTB was found at: 0x41963000
DEBUG volatility3.framework.automagic.linux: AArch64 image found
DEBUG volatility3.framework.layers.arm: Base layer : memory_layer
DEBUG volatility3.framework.layers.arm: Virtual address space : kernel
DEBUG volatility3.framework.layers.arm: Virtual addresses space range : ('0xffffffc000000000', '0xffffffffffffffff')
DEBUG volatility3.framework.layers.arm: Page size : 4
DEBUG volatility3.framework.layers.arm: T1SZ : 25
DEBUG volatility3.framework.layers.arm: Page map offset : 0x41963000
DEBUG volatility3.framework.layers.arm: Translation mappings : [(38, 30), (29, 21), (20, 12)]Processes layers instantiated by DEBUG volatility3.framework.layers.arm: Base layer : memory_layer
DEBUG volatility3.framework.layers.arm: Virtual address space : user
DEBUG volatility3.framework.layers.arm: Virtual addresses space range : ('0x0', '0x3fffffffff')
DEBUG volatility3.framework.layers.arm: Page size : 4
DEBUG volatility3.framework.layers.arm: T0SZ : 25
DEBUG volatility3.framework.layers.arm: Page map offset : 0x47266000
DEBUG volatility3.framework.layers.arm: Translation mappings : [(38, 30), (29, 21), (20, 12)]
DEBUG volatility3.framework.layers.arm: Base layer : memory_layer
DEBUG volatility3.framework.layers.arm: Virtual address space : user
DEBUG volatility3.framework.layers.arm: Virtual addresses space range : ('0x0', '0x3fffffffff')
DEBUG volatility3.framework.layers.arm: Page size : 4
DEBUG volatility3.framework.layers.arm: T0SZ : 25
DEBUG volatility3.framework.layers.arm: Page map offset : 0x47928000
DEBUG volatility3.framework.layers.arm: Translation mappings : [(38, 30), (29, 21), (20, 12)]Check out memory samples here [7 days] :
|
| return self._page_is_dirty(self._translate_entry(offset)[2]) | ||
|
|
||
| @staticmethod | ||
| def _page_is_dirty(entry: int) -> bool: |
There was a problem hiding this comment.
Dirty state is hardware managed only > Armv8.1-A. It is software managed by Linux (bit 55), so we can use it reliably and for all Arm versions.
However, we need to know how Windows manages it (add a class WindowsMixIn(AArch64) which overwrites _page_is_dirty), or directly use the hardware managed version for any OS, but it won't work for memory dumps < Armv8.1-A.
[1], see D8.4.6, page 5877 and [1], see D8.4.6, page 5877 and https://developer.arm.com/documentation/102376/0200/Access-Flag/Dirty-state
|
|
||
| # Never stack on top of an intel layer | ||
| # Never stack on top of a linux layer |
There was a problem hiding this comment.
To improve this, why not add an explicit layer property, like _is_top_layer inside volatility3/framework/layers/intel.py#Intel and volatility3/framework/layers/arm.py#AArch64 and check with following :
if getattr(layer, "_is_top_layer") and layer._is_top_layer:
return NoneIf we keep the current implementation, we have to change the Linux, Windows and Mac stacker for each new architecture layer.
There was a problem hiding this comment.
Part of the reason is virtualization, it's possible to have an arm layer inside an intel layer (and it's certainly possible to have an intel layer on top an intel layer). You're right, it's not really a scalable solution (hence the FIXME right beneath this line), but it's also not a trivial attribute...
There was a problem hiding this comment.
Which stacker is supposed to do this (LinuxStacker, WindowsIntelStacker and MacIntelStacker blocks it), for example if a VM managed by qemu-system-aarch64 on an Intel host was running when the memory was dumped ?
Will a "VM (qemu) layer" be available from the globals context.layers variable too :
- FileLayer
- LimeLayer
- IntelLayer
- Memory Layer
- QemuLayer (not sure about this one)
- IntelLayer
- Memory Layer
- IntelLayer
- IntelLayer
- LimeLayer
There was a problem hiding this comment.
I don't think there is anything automatic in core that would do that, but it would be nice to have.
There has been this issue from a while ago that talks about that kind of thing.
#464
There was a problem hiding this comment.
Thanks for pointing it out :) So in the current state, as it is not implemented, we will continue to strictly refuse stacking on top of Linux and AArch64. Adding an explicit flag on those two might be a temporary and more scalable solution ?
I leave this as a side note, for a potentiel reader in the future interested in AArch64 hypervisor execution mode (and what it might imply, if treating a layer from the hypervisor point of view) : https://developer.arm.com/documentation/102412/0103/Privilege-and-Exception-levels/Exception-levels
|
@Abyss-W4tcher I am trying your branch 👀 Dump: Symbol creation: Updated the pslist plugin adding AArch64 and run it: OUTPUT: Any suggestion? 👅 |
|
Hi @garanews, could we discuss about it in Slack DMs, to avoid filling the PR with comments ? I'll post a summary here if we get this fixed :) edit : The branch wasn't correctly merged. Be sure to do |
Waiting the merge of volatilityfoundation/volatility3#1088
Hi 👋,
This PR provides AArch64 integration to the Volatility3 framework, as well as a design rework in the current linux stacker.
Implementation follows ARM official documentation, and includes essential APIs for higher level code (plugins etc.).
You can follow the roadmap here :
My ressources :
Testing :
Linux version 6.1.21Linux version 3.18.94Linux version 6.1.0-rpi8-rpi-v8Linux version 5.10.157-android13-4-00003-g830b023b88f3-dirtyLinux version 4.9.201-tegraLinux version 6.1.36Thanks to everyone who took the time to test this PR on their devices !
Unstable plugins :
linux.bash:sections have no sizeon Android related samples.bashby default, as it's not GNU/Linux running on the system . Either we need a dedicated plugin (.ash_history?), or a check to tell users why the plugin isn't working.linux.lsof:dentrymembers can be NULL, and this behaviour isn't handled (more informations in the dedicated Slack thread)Please note that this is still experimental, testing is still going on. If you want to try this PR out, here are the steps :
git checkout aarch64-support)dwarf2jsonIf you have any real-life samples to provide, or if you encounter any error, feel free to comment below !