Improving ergonomics of sending arrays/vectors

[EdmundGoodman]

Motivation

Currently, arrays and vectors of data cannot be represented by the ReceiveFuture type, as they do not implement the Equivalence trait. This means that functions like immediate_receive cannot be used with them.

I am working on porting a High-Performance Computing benchmark from C++ to Rust, which involves sending and receiving arrays or vectors of data. Looking through the documentation, there are various ways to do this, such as using receive_vec or immediate_receive_into. However, it would be useful if functions like immediate_receive also supported these types.

Example

I would like to be able to write code like this:

use mpi::traits::*;

fn main() {
    let universe = mpi::initialize().unwrap();
    let world = universe.world();

    let to_send = [1, 2, 3];
    let future = world.any_process().immediate_receive::<[i32; 3usize]>();
    world.this_process().send(&to_send);
    let (msg, status) = future.get();

    println!("{:?} = {:?}", to_send, msg);
    println!("{:?}", status);
}

However, this fails as it requires [T; D] to implement the Equivalence trait.

Attempted solution

I attempted to create a fix which I could PR to resolve this by adding an implementation to datatype.rs:

unsafe impl<T, const D: usize> Equivalence for [T; D]
where
    T: Equivalence,
{
    type Out = UserDatatype;

    fn equivalent_datatype() -> Self::Out {
        UserDatatype::contiguous(D as i32, &T::equivalent_datatype())
    }
}

Unfortunately, this results in conflicting implementations for all other traits on [T; D] in datatype.rs (AsDatatype, Collection, ...), and I am not entirely sure why all these other traits can co-exist, but Equivalence conflicts with all of them.

I am very happy to have a further look at this and PR to fix it, but would appreciate some advice on the direction to work in, as I am slightly stuck on where to go from here.

[jtronge]

It looks like you might be able to fix the error by removing all [T; D] implementations for AsDatatype, Collection, etc. These are getting implemented twice since there's already a blanket implementation that does this. For example, with Buffer the trait is getting implemented twice by these lines:

• unsafe impl<T> Buffer for T where T: Equivalence {}
• unsafe impl<T, const D: usize> Buffer for [T; D] where T: Equivalence {}

Removing the second one should get rid of the error.

I'm not sure if this might break other things though.

As a work around for current rsmpi, you could use a wrapper struct around an array that derives Equivalence:

#[derive(Equivalence)]
struct Wrapper {
    data: [i32; 3],
}

There are more examples of that in examples/struct.rs.

[EdmundGoodman]

Awesome, thanks for the advice! Removing those blanketed implementations seemed to fix things.

I have made a PR for this (#176 ) which passes all the integration tests, along with some additional ones for the new functionality. I just realised that cargo test does not run all the integration test examples - will run run-examples.sh and attempt to diagnose any issues that present themselves there.

I am also interested in looking at implementing similar support for the Equivalence trait on Rust Vec types as mentioned in the initial issue, but I think that this might be less trivial as their length cannot be known at compile time, so I decided to try arrays first. As before, I'd appreciate any advice on whether you think it is completely impossible/directions to look in, but otherwise I'll just have a bit of a play with it and see if I can get anything working 😄.

[jedbrown]

I intend to make a custom test harness so that cargo test can run the integration tests, but for now, one needs ci/run-examples.sh --all-features.

I don't think we can do the semantic you ask for with Vec. One can use Vec::with_capacity or reserve if they are passing a Vec for a receive_into operation, but there's no place to put that information if using plain receive methods.

[EdmundGoodman]

Hi,

Having had a bit more of a stare at the code, I think I agree that neither of the semantics I suggested would be very easy to implement, so I will probably close this issue unless there is any objection.

I managed to get something working for the use case I was looking at, but I needed to trawl through a lot of docs/integration test examples to do so. Because of this, I was wondering if it would be helpful for me to make a PR to add some more written explanations, perhaps as rustdoc for the integrations tests which could then be included in the github.io/docs.rs websites - if so, I'll have a look at doing that in the next couple of weeks when I have time.

[jedbrown]

A PR to improve docs would absolutely be welcome! Note that we now use -Zrustdoc-scrape-examples (#173; but haven't published a new version since adding that). With nightly, you can cargo doc to see those snippets. That might have helped you, or offer a way in which to document things.