Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update UTransport interface specification #134

Conversation

sophokles73
Copy link
Contributor

Changed registerListener method signature to support
matching on message source and sink addresses by means
of UUri patterns.

This change allows UTransport to be used by higher level
APIs for implementing all message exchange patterns
supported by uProtocol.

Fixes #73

@sophokles73 sophokles73 added documentation Improvements or additions to documentation enhancement New feature or request labels May 6, 2024
Copy link
Contributor

@stevenhartley stevenhartley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comments


| result
| Future<Void, link:../basics/error_model.adoc[UStatus]>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The send API is synchronous and returns not a future but the actual UStatus.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why I would get back a UStatus instead of Void if sending the message succeeded? FMPOV this would result in a pretty clunky developer experience which would require me to always explicitly check for the UCode to be 0, or am I mistaken?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clunky yes but simpler than checking if it is void or UStatus etc...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO this

try {
  transport.send(myMessage);
} catch (SendException e) {
  log.error(e);
}

is much preferable over this

var result = transport.send(myMessage);
if (result.isError()) {
  log.error("failed to send message: {}", result.getCode());
} else {
  // proceed
}

This is also true for Rust and I guess every other language that has an exception mechanism. Using that programming language's exception mechanism seems very desirable to me ...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps a bit of my Rust bias creeping in here, but we have language-level mechanisms to make discriminating success-vs-failure cases in Result<T, E>.

Modeled with explicit success and failure at the type level:

fn register_listener(
  sourceFilter: UUri,
  sinkFilter: UUri?,
  listener: UListener&
) -> Result<(), UStatus>;

allows us to easily:

let registration_result = client.register_listener(source_filter, sink_filter, listener);

// use pattern matching
match registration_result {
  Err(ustatus) => { /* handle error case with access to `code` and `message` */ }
  Ok(()) => { /* handle success */ }
}

// use if-let pattern
if let Err(ustatus) = registration_result {
  /* handle just the error condition inside of ustatus */
}

Modeled without explicit success and failure at the type level:

fn register_listener(
  sourceFilter: UUri,
  sinkFilter: UUri?,
  listener: UListener&
) -> UStatus;

we then need to:

let registration_result = client.register_listener(source_filter, sink_filter, listener);

// unpack the UCode and message to then match on and use both
let (code, message) = (registration_result.get_code(), registration_result.get_message());
match code {
  UCode::OK => { /* handle the success case */ }
   _ => { /* handle the remaining cases of `code` and also use `message` */ }
}

// unable to use if-let, must fall back to an unpack and if condition
let (code, message) = (registration_result.get_code(), registration_result.get_message());
if code != UCode::OK {
  /* handle the cases where not a success using `code` and `message` */
}

Could we ask for other language implementors to weigh in here with what this would look like?

----
send(
IN message: UMessage&,
OUT result: Future<Void, UStatus>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The result is the return value of the send() API and it is not a future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is in the Rust library :-)
I wonder if we should actually bother with sync vs async in this spec at all or not leave this up to the language library developers instead.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Transport I think we need to be explicit as we said historically that send() returning that the message has been delivered to the "next hop". It is only the transport layer acknowledgement and not the communication layer (ex. rpc or pub/sub) acknowledgement.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Transport I think we need to be explicit as we said historically that send() returning that the message has been delivered to the "next hop". It is only the transport layer acknowledgement and not the communication layer (ex. rpc or pub/sub) acknowledgement.

And I think that this is the important thing to make clear. Whether an implementation implements the delivery to the next hop using a blocking/synchronous call or a non-blocking/asynchronous call does not really make a difference regarding the delivery semantics/guarantees, does it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whether it uses blocking or not underneath uTransport, the API itself has to be consistent, otherwise we cannot build a generic uStreamer that behaves the same regardless of the underlining transport implementation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, we do need consistency for ability to plug into a uStreamer.

Could we have all the methods of a UTransport be asynchronous, giving a bit more control over it to the implementor?

For the record, the up-streamer-rust implementation in PR uses the up-rust UTransport definition which has all async methods.

IN sourceFilter: UUri,
IN sinkFilter: UUri?,
IN listener: UListener&,
OUT result: Future<Void, UStatus>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not future, synchronous API

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, in the Rust library, this returns a Future in order to take advantage of async transport client libraries.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

per previous message, the sending of the message is synchronous, not the delivery to the end destination.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need the send be synchronous though? If it's asynchronous, this allows greater flexibility for UTransport implementors.

up-l1/README.adoc Outdated Show resolved Hide resolved
receive(
IN sourceFilter: UUri,
IN sinkFilter: UUri?,
OUT result: Future<UMessage&, UStatus>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above about Future


=== Receive()
=== Receive
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The receive API is a non-blocking synchronous API that returns immediately if there are no messages to consume. I also noticed that I didn't explicitly state that this API is optional to implement only if the transport utilizes pull delivery method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, makes sense. Conversely, does this also mean that the un-/register_listener methods are optional to implement only if the transport utilizes push delivery?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes true


|===

* *MUST* return `NOT_FOUND` if there are no messages for the given topic
[oft-sid="req~utransport-register-listener~1"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to discuss adding this metadata to the rest of the specs, was this one added to test out the links and then if we agree we would add them to all requirements? How do we link this in our code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding these might indeed be a little premature. Ideally, they would serve as OpenFastTrace specification item IDs which could then be included in source code, tests etc to provide for traceability across all artifacts. I haven't yet tested this, I have to admit. In fact, OFT does not (yet) support AsciiDoc files. However, from what I can tell, it should not be too hard to add this to OpenFastTrace.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love that we can do this one a single file (ex. the uri.adoc) and see if this can work end-2-end before we start adding the tags everywhere and then have to remove/change them later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is what I have in mind as well. I have spent some time investigating OpenFastTrace's extension API today. Looks very doable ...

up-l2/rpcclient.adoc Show resolved Hide resolved
up-l1/README.adoc Outdated Show resolved Hide resolved
@sophokles73
Copy link
Contributor Author

@stevenhartley @PLeVasseur I have pushed changes and have amended your comments ...

@matthewd0123
Copy link

Question/comment. Can/should we also align the error messages returned as well? Good that we're aligning the UCodes, but would ask if it's worth it to align the UStatus messages as well between implementations. Benefit would be consistency, but not sure if it's too much micromanagement.

@sophokles73
Copy link
Contributor Author

Question/comment. Can/should we also align the error messages returned as well? Good that we're aligning the UCodes, but would ask if it's worth it to align the UStatus messages as well between implementations. Benefit would be consistency, but not sure if it's too much micromanagement.

FMPOV relying on error message format/content is an antipattern that leads to code that is hard to maintain and will produce hard to find bugs during runtime. So, no, I do not think that we should standardize error messages ...

| result
| Future<Void>
| A succeeded future if the message has been sent successfully. This means that the underlying messaging infrastructure has accepted the message for delivery. It does *not* (necessarily) mean that the message has reached its destination (yet). +
Otherwise, a failed future which contains a `UCode` indicating the reason for the failure.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this:

Otherwise, a failed future which contains a `UCode` indicating the reason for the failure.

mean that the send signature should read:

send(message: UMessage) : Future<Void, UCode>

?

I'm just unfamiliar with how we would represent that bit of mechanism into the send() signature I suppose.

Edit: I see this could apply to a few other function signatures in this file as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Java, for example, the CompletableFuture<T> class is parameterized with the type of successful completion only because the Future will always be failed with an Exception. Rust is the only language I know of where any type can be used to represent the erroneous outcome.
FMPOV it should be sufficient to define that the Future will be failed with something that contains the relevant information (the UCode in this case). In the Java case this probably means that the Exception contains a UCode typed field. In Rust it might mean that the error type is UCode. Does that make sense?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the context. Makes sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should, instead of using the Future return type, just declare the concrete return type (T) and then also declare an Exception being thrown. Would that help to make it easier to understand? We could then also add some general wording to a section called Implementation Hints or similar, where we advise implementers to use non-blocking async calls. WDYT? @PLeVasseur @stevenhartley

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like your idea of having a tips & tricks section for implementers in which to put some of this advice. I still think having some level of flexibility even at uP-L1 for languages to write it the most reasonable way makes sense, which is where such a section could bring further value IMHO.

up-l1/README.adoc Outdated Show resolved Hide resolved
Implementations

* *MAY* fail invocations with a `UCode::UNIMPLEMENTED` if the transport does not support the _push_ <<_delivery_method>>. In that case, the <<_unregisterlistener>> method *MUST* also fail accordingly.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something about the _ gets mangled or something...?

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's weird, the vscode preview correctly renders the links and it is the standard AsciiDoc format. Anyway, I will replace it with the (linked to) section title.

up-l1/README.adoc Outdated Show resolved Hide resolved
up-l1/README.adoc Outdated Show resolved Hide resolved
IN sinkFilter: UUri?,
OUT result: Future<UMessage&, UStatus>
)
receive(sourceFilter: UUri, sinkFilter: UUri [0..1]) : Future<UMessage>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a thought -- I think for most folks, they will actually want to block here.

Maybe we can have a short snippet like --

If you wish to treat this function as synchronous, you may use your language of choice's mechanism for doing so. For example, Java's CompleteableFuture::complete() or Rust's .await.

Dunno. Too hand-holdy?

@stevenhartley -- sanity check me on the Java point 😆

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a thought -- I think for most folks, they will actually want to block here.

FMPOV the important thing about receive is, that it should fail immediately if no message is available, i.e. it should not wait for a message becoming available.

It is not really about whether a particular implementation actually uses async or sync calls. However, IMHO a UTransport implementation should follow a consistent approach across all methods, i.e. either sync or async. If client code needs to be able to deal with async methods anyway, because all of the other methods are async, then I do not see much value in this particular method being the only one defined as sync.

In general, when it comes to IO operations (and UTransport is as close to IO as it gets), using async non-blocking calls has been proven to make systems more performant, scalable and less resource consuming. This also matches my personal experience with other (communication infrastructure) projects ...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FMPOV the important thing about receive is, that it should fail immediately if no message is available, i.e. it should not wait for a message becoming available.

Totally agree!

However, IMHO a UTransport implementation should follow a consistent approach across all methods, i.e. either sync or async.

Agree here as well.


My point was less for an implementer of a uTransport, but more for a user of uTransport that you may often want to simply .await on the async call, because the receive call should either have a message available or not and fail.

Hope that makes sense.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please view this as non-blocking for merge. It's a minor point that could be discussed later; more a tip to help a developer, but may just as easily fit in with example code we may write later.

Changed registerListener method signature to support
matching on message source and sink addresses by means
of UUri patterns.

This change allows UTransport to be used by higher level
APIs for implementing all message exchange patterns
supported by uProtocol.

Fixes eclipse-uprotocol#73
describe send method semantics in more detail
@sophokles73 sophokles73 force-pushed the support_filtering_on_source_and_sink_address branch from 03d5483 to 88b17df Compare May 16, 2024 13:56
@sophokles73
Copy link
Contributor Author

@stevenhartley @PLeVasseur I have pushed changes as discussed in the call.
Would you mind taking another look?

@tamarafischer I have tried to add some wording to the expected outcome of the send() method. WDYT?

@PLeVasseur
Copy link
Contributor

Would you mind taking another look?

Changes to the section we discussed on the call look good to me!

Copy link
Contributor

@stevenhartley stevenhartley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor changes needed but consider it approved from me once the changes are done. Thanks @sophokles73 !

up-l1/README.adoc Outdated Show resolved Hide resolved
up-l2/rpcclient.adoc Show resolved Hide resolved
upclient.adoc Outdated Show resolved Hide resolved
upclient.adoc Show resolved Hide resolved
Co-authored-by: Steven Hartley <steven@orangepeel.ca>
Co-authored-by: Steven Hartley <steven@orangepeel.ca>
@sophokles73
Copy link
Contributor Author

@stevenhartley I have accepted your proposed changes but in order to merge the PR, GitHub still requires your explicit approval ...

Copy link
Contributor

@stevenhartley stevenhartley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sophokles73 sophokles73 merged commit 8e5b4d8 into eclipse-uprotocol:main May 17, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Does UTransport::register_listener match on source or sink address?
4 participants