-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Require safe transmutability traits for buffers #79
base: main
Are you sure you want to change the base?
Conversation
I hadn't considered the issue of sending structs with padding - may have to switch to require something equivalent to |
I'm looking to see if we can get a full "0.11" release for |
cdb857e
to
6c9764e
Compare
I removed the public |
6c9764e
to
fe67a0b
Compare
That release seems to have happened in the mean time. I think the public dependency would actually be a better choice as interoperability w.r.t. these key traits could be automatic, e.g. other crates could implement this for their data types and we would automatically benefit. Ecosystem service effects and all that... |
I see two concerns here:
(which I personally use ATM for transmitting complex structs). My current understanding would be that
But we would probably want people to opt in explicitly here, e.g. via a wrapper like |
fe67a0b
to
b173c6e
Compare
…safe to construct from bytes. Buffer is now only implemented for types that implement ToBytes. BufferMut is now only implemented for types that implement FromAnyBytes.
b173c6e
to
3dcbd5a
Compare
`EquivalenceFromAnyBytes` indicates a type's Equivalence implementation doesn't map any fields that are unable to contain arbitrary bit representations. This is to mainly guard the "bool" type, allowing it to be sent, but not received..
This change adds a new `mpi::Bool` type that is representationally valid for all bit representations, but provides member functions for doing runtime checks that the contained value is a representationally valid bool. Examples updated to use `mpi::Bool` where it makes sense.
On further thought/discussion, it wasn't obvious, actually, that those traits are the best match. Essentially, we want a trait that only requires that the fields mapped by
Good idea - done.
Yeah, you're absolutely right. I changed the name of the trait to |
To address the issue with |
@AndrewGaspar One thing I am wondering about here is usability. More specifically whether it is truly worth it to split the write and read sides of the equivalence traits. Is it really sufficiently common to have different sources for the sender and receiver processes as to significantly ease development of the write side of communication relation? Personally, I think just folding the |
I 100% get your concern and I'd like to eliminate it altogether. Here's where I'm stuck.
Here's the problem: if we roll Maybe if I'm understanding you correctly, the proposed alternative is to implement |
I see.
I think I would rather look at this from the other end: If Of course, this would be a highly intrusive change to the API. There could also be a lot of friction if my model code contains e.g. a But in the end, this would probably penalise users with types that would derive both One last question on my mind is whether there is a trait in |
Ah, sorry, this is non-sense, as |
Was just about to say this - exactly. That's why I opted not to use it. |
Take one last look? |
One more idea I had concerning usability: Maybe we should make This would also suggest to make the derive macro for
instead of
One downside I see would be being unable to manually implement |
Yeah, I had considered doing exactly what you're describing, but decided not to for the exact downsides you described. I also wasn't sure if automatically implementing multiple traits for a single derive was considered good practice. I don't know, I think I'm still leaning towards keeping them as separate derives because I think that's the maximally flexible option. |
Alright. I'd say we keep it as-is then and collect some experience actually using it. We are at 0.6 instead of 1.6 after all. |
af7d7cd
to
84136c9
Compare
src/transmute.rs
Outdated
pub use super::EquivalenceFromAnyBytes; | ||
} | ||
|
||
/// Any type whose `Equivalence` implementation maps only fields that can be composed of an |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am sorry that it never seems to stop, but one more question came to my mind on this: If one has a struct with three fields, say one is a bool
, and an Equivalence
that maps only the other two which are EquivalenceFromAnyBytes
, then this could implement EquivalenceFromAnyBytes
as it fulfills what is written here, but that would also mean that I can receive a partially formed instance via MPI (as the bool field is not mapped and hence not initialised). On the other hand, wrapping it received value in MaybeUninit
makes this is workable Equivalence
implementation, one just has to initialise the bool
field manually after receiving and before calling assume_init
.
Hence I think we should reword this in a more operational way, i.e. what we want is that the Equivalence
impl of the type in question produces a fully initialised value from any bit pattern received via MPI.
I think this would also suggest adjusting the naming a bit to something like Full...
, Complete...
, Safe...
or UnconditionalEquivalence`?
I also wonder about the derive macro for EquivalenceFromAnyBytes
: The above problem even exists if the unmapped field is say u32
as while this TriviallyTransmutable
, it would still be unmapped and hence uninitialised when receiving the type via MPI (it is not insta-UB, just UB I guess). So just checking the type of all fields is not sufficient if we do not know if all fields are actually mapped.
For me, this again suggests making this a super trait of Equivalence
and also making the derive macro for it derive both traits because this would make sure that all fields are mapped and the check if those field types are themselves EquivalenceFromAnyBytes
sound.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the other hand, wrapping it received value in MaybeUninit makes this is workable Equivalence implementation, one just has to initialise the bool field manually after receiving and before calling assume_init.
You don't even really need MaybeUninit
. In pure safe code, you can just initialize the receive buffer like normal before hand and there won't be any undefined behavior.
Hence I think we should reword this in a more operational way, i.e. what we want is that the Equivalence impl of the type in question produces a fully initialised value from any bit pattern received via MPI.
I don't think that's necessary. As long as you're correctly initializing your receive buffer, including unmapped fields, then there is no issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As long as you're correctly initializing your receive buffer, including unmapped fields, then there is no issue.
This only applies to methods where I pass an already initialised value in, which is then "merely updated". But things like Source::receive
and Source::receive_vec
produce their return types Msg: Equivalence
using only the data passed in via MPI.
These functions would also need updated trait bounds if I understand things correctly as adding BufferMut: EquivalenceFromAnyBytes
does not affect them as they work using with_uninitialized2
internally.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Crap, good point. But I don't think we can guard against too-small messages, though, even if we made the trait stricter. Let me test this on my system to see...
In the cases what if we just required either implementing Default
for your type, or otherwise wrapping it in MaybeUninit
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But I don't think we can guard against too-small messages, though, even if we made the trait stricter. Let me test this on my system to see...
Nvm, OpenMPI will crash if the message is too small to fill the receive buffer. Let me see with mpich, too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand it looks suspicious but I don't think there is anything inherently wrong with sending MaybeUninit? Of course, the receiver should not call assume_init if the value sent was not fully initialised, but I don't think MPI would care.
That's not right - it would allow you to read uninitialized memory without an unsafe block.
let mut data = [0xFFu8; 1024];
comm.unpack_into(
comm.pack(&[MaybeUninit::<u8>::uninit(); 1024][..])
.as_slice(),
&mut data[..],
0,
);
println!("{:?}", data);
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, indeed. When the data really is uninitialised it read without an unsafe
block.
Hhhmmm, so EquivalenceFromAnyBytes
cannot be a sub trait of Equivalence
because we want MaybeUninit<T>: EquivalenceFromAnyBytes
if T: Equivalence
but we do not want MaybeUninit<T>: Equivalence
. But even this does not really help us as we still we need to have separate receive_uninit<Msg>() -> MaybeUninit<Msg>
API to express that Msg: Equivalence
and MaybeUninit<Msg>: EquivalenceFromAnyBytes
, but this would also mean that we do not need MaybeUninit: EquivalenceFromAnyBytesat all as this is implicit in the
receive_unit` signature, i.e. separate entry points for uninitialised messages make the blanket impl redundant.
Should we split this into three traits Equivalence
, SafeToSend
and SafeToReceive
where MaybeUninit<T>
would be SafeToReceive
but not SafeToSend
? This does not really sound better than the separate API entry points, does it? Especially since the API entry points for initialised messages could be forwarding implementations like
fn receive<Msg>(&self) -> Msg where Msg: EquivalenceFromAnyBytes { unsafe { self.receive_uninit().assume_init() } }
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I would still argue to make EquivalenceFromAnyBytes
a sub trait of Equivalence
even if we need handle MaybeUninit
separately.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
((Just so this will never end (really sorry for this being so complicated): I think at this point EquivalenceFromAnyBytes
really is Equivalence + TriviallyTransmutable
so we could reuse the trait from safe-transmute
.))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AndrewGaspar I tried making progress into any of the above directions and laziness pushed me towards introducing an additional trait which matches the two situations where are interested in instead of increasing the API surface, i.e.
pub unsafe trait SafeToReceive {
type Inner: Equivalence;
}
unsafe impl<T> SafeToReceive for T
where
T: Equivalence + EquivalenceFromAnyBytes,
{
type Inner = T;
}
unsafe impl<T> SafeToReceive for MaybeUninit<T>
where
T: Equivalence,
{
type Inner = T;
}
c.f. https://github.com/adamreichold/rsmpi/commit/2e228673db0ebb5398ee96d68c789894f5887f83
Do you think this would be a viable approach to move this change forward?
(I still think we might want to go for a sub trait or even TriviallyTransmutable
, but by now, I think it would be wiser to postpone any such changes to separate issues/pull requests.)
EquivalenceFromAnyBytes now requires that the type can be safely and fully initialized from a properly sized, arbitrary MPI message. This also fixes a bunch of soundness holes in routines that construct and return a receive value. Removes the transmute module and folds into the datatype module now that it's fairly empty.
Use derive(EquivalenceUnsafe) to derive just Equivalence
} | ||
(msg, status) | ||
|
||
let count = status.count(&datatype).expect( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just for readability, couldn't you move the checking of status
and count
(meaning the panicking essentially) inside the closure passed to with_uninitialized2
and thereby avoid manually juggling the MaybeUninit
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I had a think'o here: status
is MaybeUninit
itself and we need its assume_init
to be called before making the check. That sad, I wonder if with_uninitialized
helps here at all as it somewhat obscures where assume_init
is called? Maybe we need a receive helper to encapsulate this repeating two step check?
@adamreichold Sorry I haven't checked in on this in awhile - I've been going through a long interview process with a few different companies, and haven't had much free time. Thankfully, I'm finally done with all that, and I may even get to write Rust full time in my new job! 🤞 I'm going to try getting this done when I leave my current job next week - I have four weeks off before I start my next job. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@adamreichold @AndrewGaspar 👋 What do you think should be done with this PR? I'll admit to being kinda worried about what it sets out to do, given that MPI says mismatched types are invalid (even though implementations don't check). Based on @jtronge's work on our paper (https://dl.acm.org/doi/fullHtml/10.1145/3615318.3615328), I'd rather create a safe way to check for type matching, such as creating lightweight typed communicators where type matching can be validated by nonblocking collectives or metadata piggy-backed on the first message (or possibly MPI extensions, which the paper show can have small overhead). Perhaps these efforts are complementary, but I'm still not sure how to think about this PR. (I'm not asking for any development time, just your opinion about what effort would be fruitful.)
/// Allow datatypes with Equivalence to be used in buffer packing functions | ||
unsafe impl<T> Equivalence for MaybeUninit<T> | ||
where | ||
T: Equivalence, | ||
{ | ||
type Out = T::Out; | ||
|
||
fn equivalent_datatype() -> Self::Out { | ||
T::equivalent_datatype() | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is a problem since it's UB to send from an uninitialized buffer. I think MaybeUninit should be used with an interface that returns a reference to the message as received (e.g., as a slice of the MaybeUninit
receive buffer), as discussed here. #128 (comment)
// All subfields are `FromAnyBytes`. | ||
unsafe impl EquivalenceFromAnyBytes for TupleType {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe we need the stronger statement that there is no padding. For example, consider where bytemuck requires NoUninit
versus Pod
. I think for a symmetric interface, we basically need Pod
and perhaps we should have rsmpi recognize bytemuck traits rather than making our own traits (which have fuzzy semantics at present).
@@ -48,25 +60,39 @@ unsafe impl Equivalence for ComplexDatatype { | |||
} | |||
} | |||
|
|||
// All subfields are `FromAnyBytes`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we also need that there are no padding/uninit parts, which is a hard thing to test visually, but the bytemuck derive macro does check.
I added two new traits inspired by this Pre-RFC:
ToBytes
andFromAnyBytes
.ToBytes
can be safely applied to any type with deterministic layout that has no internal padding (and therefore has no information disclosure potential).FromAnyBytes
can be safely applied to any type where all potential bit patterns of its components are valid instances of the type.This change makes the default
Buffer
implementation requireToBytes
, and theBufferMut
implementation requireFromAnyBytes
. This fixes issues related to receiving invalid values intobool
.This fixes #54.