Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is using Aeron stable with RSocket? #653

Open
CodeMason opened this issue Jun 13, 2019 · 10 comments · May be fixed by #972
Open

Is using Aeron stable with RSocket? #653

CodeMason opened this issue Jun 13, 2019 · 10 comments · May be fixed by #972
Assignees
Labels
Milestone

Comments

@CodeMason
Copy link

The readme states that Aeron is available for use, however from looking at the issues and at stackoverflow, the comment I wouldn't recommend using the Aeron implementation currently. bothers me.

So before spending the engineering effort to use RSocket + Aeron, and for the sake of clarity, can we use Aeron right now (0.12.2-RC4) with rsocket-java?

@robertroeser
Copy link
Member

Aeron+RSocket isn't production worthy right now - I have an an update but we haven't really found the need to use Aeron right now. What's your use-case?

@CodeMason
Copy link
Author

We're using aeron to push huge amounts of data between servers, and are interested in RSocket features.

@ronenhamias
Copy link

@CodeMason
as part of our research to implement reactor aeron we where also interested to implement rsocket aeron. https://github.com/scalecube/rsocket-transport-aeron

one issue we found with integrating Rsocket with Aeron is the use of netty byte buffers currently there is no way in rsocket implementation to introduce an alternative byte buffer.
this restriction forces to copy the byte buffers from aeron to netty and causes performance impact.

please check
https://github.com/scalecube/reactor-aeron

@OlegDokuka
Copy link
Member

Hey @ronenhamias!

The main reason for coping with BB is to have the power of refcounting and subsequent ability to deal with multithreading (which is less possible for Aeron buffers (correct me if I'm wrong)). Thus, I'm curious how do you solve that in rector-aeron and how do you cope with multithreading, the asynchronous reactor in your case?

Regards,
Oleh

@ronenhamias
Copy link

ronenhamias commented Nov 14, 2019

Hey @OlegDokuka! :)

the reasons are clear and netty is great probably for most of use cases. netty BB is a powerful thing. on the other hand aeron does not know anything about netty BB so if someone see a need/have an aeron use-case then the restriction of using netty BB with argona BB causes additional copy just for the sake of fitting aeron to rsocket abstractions. in such case this someone will not enjoy the benefits of aeron and his exceptions will not be met as aeron implies a very specific way of doing things.
in such case i think its better to use just plain aeron or reactor-aeron or giving up on aeron so it really depends on the usecase and requirements.

Regards,
Ronen.

@ebfhub
Copy link

ebfhub commented Feb 3, 2020

I might be wrong, but I think you can do this.
Note that the UnsafeBuffer should belong to the thread calling offer(), and can be reused after offer() returns.

        static final UnsafeBuffer buffer = new UnsafeBuffer();
        if (bb.hasArray()) {
            buffer.wrap(bb.array(),0,len);
        } else if (bb.hasMemoryAddress()){
            buffer.wrap(bb.memoryAddress(),len);
        } else {
            ...??... throw
        }

       pub.offer(buffer,0,len);

And something like this for reading:

ByteBufWrappingDirectBuffer .java

@OlegDokuka
Copy link
Member

@ebfhub do you have any samples of usage of mentioned BB?

I'm really curious to see how it is used in any project to fully understand whether we can adopt that as well with no subsequent issues related to asynchronous propagation of the given instance

Regards,
Oleh

@ebfhub
Copy link

ebfhub commented Feb 7, 2020

Sorry, only in an internal project. Following through offer(), the passed buffer is copied into another before returning, i.e. on the return from offer() you can reuse that buffer.
e.g.
termBuffer.putBytes(termOffset + HEADER_LENGTH, srcBuffer, srcOffset, length);

On the receive side, the ByteBufWrappingDirectBuffer would need to be used synchronously (which we do.)

@OlegDokuka
Copy link
Member

Gotcha. Actually, I can imagine just minor limitation on such a transport which requires to deserialize before sending on another thread which should be fine

@OlegDokuka OlegDokuka self-assigned this Feb 9, 2020
@yschimke yschimke added this to the Backlog milestone Apr 22, 2021
@OlegDokuka OlegDokuka linked a pull request Apr 27, 2021 that will close this issue
@burakakca
Copy link

What is the current status of this topic?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants