Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Require safe transmutability traits for buffers #79

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

AndrewGaspar
Copy link
Contributor

@AndrewGaspar AndrewGaspar commented Dec 8, 2019

I added two new traits inspired by this Pre-RFC: ToBytes and FromAnyBytes. ToBytes can be safely applied to any type with deterministic layout that has no internal padding (and therefore has no information disclosure potential). FromAnyBytes can be safely applied to any type where all potential bit patterns of its components are valid instances of the type.

This change makes the default Buffer implementation require ToBytes, and the BufferMut implementation require FromAnyBytes. This fixes issues related to receiving invalid values into bool.

This fixes #54.

@AndrewGaspar
Copy link
Contributor Author

@AndrewGaspar
Copy link
Contributor Author

I hadn't considered the issue of sending structs with padding - may have to switch to require something equivalent to ToBytes for send buffers.

@AndrewGaspar AndrewGaspar changed the title Require TriviallyTransmutable for receive buffers WIP: Require TriviallyTransmutable for receive buffers Dec 9, 2019
@AndrewGaspar
Copy link
Contributor Author

I'm looking to see if we can get a full "0.11" release for safe_transmute. nabijaczleweli/safe-transmute-rs#62

@AndrewGaspar AndrewGaspar changed the title WIP: Require TriviallyTransmutable for receive buffers Require safe transmutability traits for buffers Dec 17, 2019
@AndrewGaspar
Copy link
Contributor Author

I removed the public safe-transmute dependency (though still use it in an example) and added the desired traits myself. I updated the MR text to reflect this.

@adamreichold
Copy link
Contributor

I removed the public safe-transmute dependency (though still use it in an example) and added the desired traits myself. I updated the MR text to reflect this.

That release seems to have happened in the mean time. I think the public dependency would actually be a better choice as interoperability w.r.t. these key traits could be automatic, e.g. other crates could implement this for their data types and we would automatically benefit. Ecosystem service effects and all that...

@adamreichold
Copy link
Contributor

adamreichold commented Sep 25, 2020

I see two concerns here:

  1. I would like to see MaybeUninit<M> included in the fold so that I can still receive arbitrary data types which are not safe to transmute even though I will have to make the assertion of correct communication by calling the unsafe assume_init in my code.

  2. I am really not sure how this would interact with e.g. using UserData::structured: Having such an implemention for example based on #[derive(Equivalence)] should be fully sufficient to send structures over the wire even if they are not safely transmutable as the MPI implementation will only access valid data at specific offsets with correct type information. FromAnyBytes seems to be necessary only if one using an equivalence like

fn equivalent_datatype() -> Self::Out {
  UserDatatype::contiguous(mem::size_of::<Self>(), &u8::equivalent_datatype());
}

(which I personally use ATM for transmitting complex structs). My current understanding would be that FromAnyBytes would not be an additional trait bound (in addition to Equivalence) but could rather provide a blanket impl of Equivalence like

unsafe impl<T> Equivalence for T where T: FromAnyBytes + ToBytes {
  type Out = UserDatatype;

  fn equivalent_datatype() -> Self::Out {
    UserDatatype::contiguous(mem::size_of::<Self>(), &u8::equivalent_datatype());
  }
}

But we would probably want people to opt in explicitly here, e.g. via a wrapper like EquivalentToBytes(&mut T)?

…safe to construct from bytes.

Buffer is now only implemented for types that implement ToBytes.
BufferMut is now only implemented for types that implement FromAnyBytes.
`EquivalenceFromAnyBytes` indicates a type's Equivalence implementation
doesn't map any fields that are unable to contain arbitrary bit
representations. This is to mainly guard the "bool" type, allowing it
to be sent, but not received..
This change adds a new `mpi::Bool` type that is representationally valid
for all bit representations, but provides member functions for doing
runtime checks that the contained value is a representationally valid
bool.

Examples updated to use `mpi::Bool` where it makes sense.
@AndrewGaspar
Copy link
Contributor Author

That release seems to have happened in the mean time. I think the public dependency would actually be a better choice as interoperability w.r.t. these key traits could be automatic, e.g. other crates could implement this for their data types and we would automatically benefit. Ecosystem service effects and all that...

On further thought/discussion, it wasn't obvious, actually, that those traits are the best match. Essentially, we want a trait that only requires that the fields mapped by Equivalence can receive any bytes. Deriving Equivalence already requires the mapped fields to be valid objects in Rust.

I would like to see MaybeUninit included in the fold so that I can still receive arbitrary data types which are not safe to transmute

Good idea - done.

I am really not sure how this would interact with e.g. using UserData::structured: Having such an implemention for example based on #[derive(Equivalence)] should be fully sufficient to send structures over the wire even if they are not safely transmutable

Yeah, you're absolutely right. I changed the name of the trait to EquivalenceFromAnyBytes to indicate that only those fields mapped by Equivalence require this attribute.

@AndrewGaspar
Copy link
Contributor Author

AndrewGaspar commented Nov 16, 2020

To address the issue with bool, I added a new type mpi::Bool that can receive any bit pattern, but has methods for unwrapping the value and checking validity. So you can still send bool, but only receive mpi::Bool. mpi::Bool is mapped as MPI_C_BOOL, just like bool, so it's valid as far as MPI is concerned to do this mapping.

README.md Outdated Show resolved Hide resolved
build.rs Outdated Show resolved Hide resolved
Cargo.toml Outdated Show resolved Hide resolved
@adamreichold
Copy link
Contributor

@AndrewGaspar One thing I am wondering about here is usability. More specifically whether it is truly worth it to split the write and read sides of the equivalence traits. Is it really sufficiently common to have different sources for the sender and receiver processes as to significantly ease development of the write side of communication relation?

Personally, I think just folding the EquivalenceFromAnyBytes into the Equivalence trait would be simpler to understand and to explain without really making it more difficult to adhere to these requirements. Especially since the blanket implementation for MaybeUninit provides a generic escape hatch.

@AndrewGaspar
Copy link
Contributor Author

I 100% get your concern and I'd like to eliminate it altogether. Here's where I'm stuck.

  1. Equivalence designates an MPI_Datatype that safely maps the type. If MPI required type-checking, we'd be done here.
  2. EquivalenceFromAnyBytes says: No matter what message you throw at me, my Equivalence implementation can safely (in Rust terms) map it into my representation
  3. However, if you can't do that, that's OK: you can still wrap your type in MaybeUninit. As long as it implements Equivalence, it's still safe to receive into.

Here's the problem: if we roll EquivalenceFromAnyBytes up into Equivalence, then we can't require MaybeUninit<T> in the cases where it's not safe to write arbitrary bytes into T.

Maybe if I'm understanding you correctly, the proposed alternative is to implement Equivalence on e.g. MaybeUninit<bool> instead? Though if that's the case, maybe we just only implement it on mpi::Bool.

@adamreichold
Copy link
Contributor

adamreichold commented Nov 17, 2020

Here's the problem: if we roll EquivalenceFromAnyBytes up into Equivalence, then we can't require MaybeUninit in the cases where it's not safe to write arbitrary bytes into T.

I see.

Maybe if I'm understanding you correctly, the proposed alternative is to implement Equivalence on e.g. MaybeUninit instead? Though if that's the case, maybe we just only implement it on mpi::Bool.

I think I would rather look at this from the other end: If T: Equivalence, the write API consumes &T whereas all read API produce MaybeUninit<T>. For types that are trivially transmutable, going from MaybeUninit<T> to T would be safe.

Of course, this would be a highly intrusive change to the API. There could also be a lot of friction if my model code contains e.g. a Vec<T> from which I send and into which I also want to receive, making it necessary to transmute &mut Vec<T> to &mut Vec<MaybeUninit<T>> even though that is always safe. (I am also not sure if safe-transmute does have the necessary functions to go from T to MaybeUninit<T> and back in safe way.)

But in the end, this would probably penalise users with types that would derive both Equivalence and EquivalenceFromAnyBytes now. So while I don't really feel comfortable about it, I agree that the current proposal looks like the one with the lowest friction for the most common use case.

One last question on my mind is whether there is a trait in safe-transmute that we could use instead of EquivalenceFromAnyBytes? It does feel a bit like reinventing TriviallyTransmutable?

@adamreichold
Copy link
Contributor

adamreichold commented Nov 17, 2020

One last question on my mind is whether there is a trait in safe-transmute that we could use instead of EquivalenceFromAnyBytes? It does feel a bit like reinventing TriviallyTransmutable?

Ah, sorry, this is non-sense, as EquivalenceFromAnyBytes has a less strict contract, it means only what is touched by my Equivalence impl is FromAnyBytes, not the whole type. Sorry for the confusion!

@AndrewGaspar
Copy link
Contributor Author

Ah, sorry, this is non-sense, as EquivalenceFromAnyBytes has a less strict contract, it means only what is touch by my Equivalence impl is FromAnyBytes, not the whole type. Sorry for the confusion!

Was just about to say this - exactly. That's why I opted not to use it.

@AndrewGaspar
Copy link
Contributor Author

Take one last look?

@adamreichold
Copy link
Contributor

Take one last look?

One more idea I had concerning usability: Maybe we should make EquivalenceFromAnyBytes a sub trait of Equivalence as it really only makes sense for a type that actually implements Equivalence in the first place?

This would also suggest to make the derive macro for EquivalenceFromAnyBytes derive both impls so that I can just write

#[derive(EquivalenceFromAnyBytes)]

instead of

#[derive(Equivalence, EquivalenceFromAnyBytes)]

One downside I see would be being unable to manually implement Equivalence but derive EquivalenceFromAnyBytes, but then again manually implementing EquivalenceFromAnyBytes on top of manually implementing Equivalence might be the better trade-off if the common case is simplified?

build.rs Outdated Show resolved Hide resolved
src/checked_bool.rs Outdated Show resolved Hide resolved
@AndrewGaspar
Copy link
Contributor Author

Take one last look?

One more idea I had concerning usability: Maybe we should make EquivalenceFromAnyBytes a sub trait of Equivalence as it really only makes sense for a type that actually implements Equivalence in the first place?

This would also suggest to make the derive macro for EquivalenceFromAnyBytes derive both impls so that I can just write

#[derive(EquivalenceFromAnyBytes)]

instead of

#[derive(Equivalence, EquivalenceFromAnyBytes)]

One downside I see would be being unable to manually implement Equivalence but derive EquivalenceFromAnyBytes, but then again manually implementing EquivalenceFromAnyBytes on top of manually implementing Equivalence might be the better trade-off if the common case is simplified?

Yeah, I had considered doing exactly what you're describing, but decided not to for the exact downsides you described. I also wasn't sure if automatically implementing multiple traits for a single derive was considered good practice.

I don't know, I think I'm still leaning towards keeping them as separate derives because I think that's the maximally flexible option.

@adamreichold
Copy link
Contributor

I don't know, I think I'm still leaning towards keeping them as separate derives because I think that's the maximally flexible option.

Alright. I'd say we keep it as-is then and collect some experience actually using it. We are at 0.6 instead of 1.6 after all.

src/transmute.rs Outdated
pub use super::EquivalenceFromAnyBytes;
}

/// Any type whose `Equivalence` implementation maps only fields that can be composed of an
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sorry that it never seems to stop, but one more question came to my mind on this: If one has a struct with three fields, say one is a bool, and an Equivalence that maps only the other two which are EquivalenceFromAnyBytes, then this could implement EquivalenceFromAnyBytes as it fulfills what is written here, but that would also mean that I can receive a partially formed instance via MPI (as the bool field is not mapped and hence not initialised). On the other hand, wrapping it received value in MaybeUninit makes this is workable Equivalence implementation, one just has to initialise the bool field manually after receiving and before calling assume_init.

Hence I think we should reword this in a more operational way, i.e. what we want is that the Equivalence impl of the type in question produces a fully initialised value from any bit pattern received via MPI.

I think this would also suggest adjusting the naming a bit to something like Full..., Complete..., Safe... or UnconditionalEquivalence`?

I also wonder about the derive macro for EquivalenceFromAnyBytes: The above problem even exists if the unmapped field is say u32 as while this TriviallyTransmutable, it would still be unmapped and hence uninitialised when receiving the type via MPI (it is not insta-UB, just UB I guess). So just checking the type of all fields is not sufficient if we do not know if all fields are actually mapped.

For me, this again suggests making this a super trait of Equivalence and also making the derive macro for it derive both traits because this would make sure that all fields are mapped and the check if those field types are themselves EquivalenceFromAnyBytes sound.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the other hand, wrapping it received value in MaybeUninit makes this is workable Equivalence implementation, one just has to initialise the bool field manually after receiving and before calling assume_init.

You don't even really need MaybeUninit. In pure safe code, you can just initialize the receive buffer like normal before hand and there won't be any undefined behavior.

Hence I think we should reword this in a more operational way, i.e. what we want is that the Equivalence impl of the type in question produces a fully initialised value from any bit pattern received via MPI.

I don't think that's necessary. As long as you're correctly initializing your receive buffer, including unmapped fields, then there is no issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as you're correctly initializing your receive buffer, including unmapped fields, then there is no issue.

This only applies to methods where I pass an already initialised value in, which is then "merely updated". But things like Source::receive and Source::receive_vec produce their return types Msg: Equivalence using only the data passed in via MPI.

These functions would also need updated trait bounds if I understand things correctly as adding BufferMut: EquivalenceFromAnyBytes does not affect them as they work using with_uninitialized2 internally.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Crap, good point. But I don't think we can guard against too-small messages, though, even if we made the trait stricter. Let me test this on my system to see...

In the cases what if we just required either implementing Default for your type, or otherwise wrapping it in MaybeUninit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I don't think we can guard against too-small messages, though, even if we made the trait stricter. Let me test this on my system to see...

Nvm, OpenMPI will crash if the message is too small to fill the receive buffer. Let me see with mpich, too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand it looks suspicious but I don't think there is anything inherently wrong with sending MaybeUninit? Of course, the receiver should not call assume_init if the value sent was not fully initialised, but I don't think MPI would care.

That's not right - it would allow you to read uninitialized memory without an unsafe block.

    let mut data = [0xFFu8; 1024];
    comm.unpack_into(
        comm.pack(&[MaybeUninit::<u8>::uninit(); 1024][..])
            .as_slice(),
        &mut data[..],
        0,
    );

    println!("{:?}", data);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, indeed. When the data really is uninitialised it read without an unsafe block.

Hhhmmm, so EquivalenceFromAnyBytes cannot be a sub trait of Equivalence because we want MaybeUninit<T>: EquivalenceFromAnyBytes if T: Equivalence but we do not want MaybeUninit<T>: Equivalence. But even this does not really help us as we still we need to have separate receive_uninit<Msg>() -> MaybeUninit<Msg> API to express that Msg: Equivalence and MaybeUninit<Msg>: EquivalenceFromAnyBytes, but this would also mean that we do not need MaybeUninit: EquivalenceFromAnyBytesat all as this is implicit in thereceive_unit` signature, i.e. separate entry points for uninitialised messages make the blanket impl redundant.

Should we split this into three traits Equivalence, SafeToSend and SafeToReceive where MaybeUninit<T> would be SafeToReceive but not SafeToSend? This does not really sound better than the separate API entry points, does it? Especially since the API entry points for initialised messages could be forwarding implementations like

fn receive<Msg>(&self) -> Msg where Msg: EquivalenceFromAnyBytes { unsafe { self.receive_uninit().assume_init() } }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I would still argue to make EquivalenceFromAnyBytes a sub trait of Equivalence even if we need handle MaybeUninit separately.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

((Just so this will never end (really sorry for this being so complicated): I think at this point EquivalenceFromAnyBytes really is Equivalence + TriviallyTransmutable so we could reuse the trait from safe-transmute.))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AndrewGaspar I tried making progress into any of the above directions and laziness pushed me towards introducing an additional trait which matches the two situations where are interested in instead of increasing the API surface, i.e.

pub unsafe trait SafeToReceive {
    type Inner: Equivalence;
}

unsafe impl<T> SafeToReceive for T
where
    T: Equivalence + EquivalenceFromAnyBytes,
{
    type Inner = T;
}

unsafe impl<T> SafeToReceive for MaybeUninit<T>
where
    T: Equivalence,
{
    type Inner = T;
}

c.f. https://github.com/adamreichold/rsmpi/commit/2e228673db0ebb5398ee96d68c789894f5887f83

Do you think this would be a viable approach to move this change forward?

(I still think we might want to go for a sub trait or even TriviallyTransmutable, but by now, I think it would be wiser to postpone any such changes to separate issues/pull requests.)

EquivalenceFromAnyBytes now requires that the type can be safely and
fully initialized from a properly sized, arbitrary MPI message.

This also fixes a bunch of soundness holes in routines that construct
and return a receive value.

Removes the transmute module and folds into the datatype module now that
it's fairly empty.
Use derive(EquivalenceUnsafe) to derive just Equivalence
}
(msg, status)

let count = status.count(&datatype).expect(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for readability, couldn't you move the checking of status and count (meaning the panicking essentially) inside the closure passed to with_uninitialized2 and thereby avoid manually juggling the MaybeUninit?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I had a think'o here: status is MaybeUninit itself and we need its assume_init to be called before making the check. That sad, I wonder if with_uninitialized helps here at all as it somewhat obscures where assume_init is called? Maybe we need a receive helper to encapsulate this repeating two step check?

@AndrewGaspar
Copy link
Contributor Author

@adamreichold Sorry I haven't checked in on this in awhile - I've been going through a long interview process with a few different companies, and haven't had much free time. Thankfully, I'm finally done with all that, and I may even get to write Rust full time in my new job! 🤞 I'm going to try getting this done when I leave my current job next week - I have four weeks off before I start my next job.

Copy link
Contributor

@jedbrown jedbrown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adamreichold @AndrewGaspar 👋 What do you think should be done with this PR? I'll admit to being kinda worried about what it sets out to do, given that MPI says mismatched types are invalid (even though implementations don't check). Based on @jtronge's work on our paper (https://dl.acm.org/doi/fullHtml/10.1145/3615318.3615328), I'd rather create a safe way to check for type matching, such as creating lightweight typed communicators where type matching can be validated by nonblocking collectives or metadata piggy-backed on the first message (or possibly MPI extensions, which the paper show can have small overhead). Perhaps these efforts are complementary, but I'm still not sure how to think about this PR. (I'm not asking for any development time, just your opinion about what effort would be fruitful.)

Comment on lines +272 to +282
/// Allow datatypes with Equivalence to be used in buffer packing functions
unsafe impl<T> Equivalence for MaybeUninit<T>
where
T: Equivalence,
{
type Out = T::Out;

fn equivalent_datatype() -> Self::Out {
T::equivalent_datatype()
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a problem since it's UB to send from an uninitialized buffer. I think MaybeUninit should be used with an interface that returns a reference to the message as received (e.g., as a slice of the MaybeUninit receive buffer), as discussed here. #128 (comment)

Comment on lines +23 to +24
// All subfields are `FromAnyBytes`.
unsafe impl EquivalenceFromAnyBytes for TupleType {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we need the stronger statement that there is no padding. For example, consider where bytemuck requires NoUninit versus Pod. I think for a symmetric interface, we basically need Pod and perhaps we should have rsmpi recognize bytemuck traits rather than making our own traits (which have fuzzy semantics at present).

@@ -48,25 +60,39 @@ unsafe impl Equivalence for ComplexDatatype {
}
}

// All subfields are `FromAnyBytes`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we also need that there are no padding/uninit parts, which is a hard thing to test visually, but the bytemuck derive macro does check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Require unsafe for receive operations on types with invalid bit patterns, like bool
3 participants