Restinio and concurrency

[AurelienLP]

Hi!

First of all, thank you for your contributions in the community!

I'd like to know how restinio deals with concurrency. If my server runs with multiples threads, the requests will be dispatched into the different threads by themself.

If my handler contains some data that can be modified, should I deal with synchronisation myself or does all threads have their own context (like we could have with a threadpool?)

Thank you!

[eao197]

I'm not sure I fully understand your question.

You can run RESTinio on thread pool. In that case, RESTinio will protect own internals by using Asio's strand objects. But user-provided request-handlers will be called on different threads and if several handlers share some data that data should be protected by a user (by using mutexes, spin-locks, or something else).

I hope this answer will be useful, but if not feel free to ask additional questions, I'll try to provide more explanations.

[AurelienLP]

Hi @eao197!

I'll talk about my basic use case, maybe it'll make things easier to understand.

I have an application which contains some data objects. Some of them are const, some of them are not.

What I'd like to do it to create multiple threads, and pass my const objects by reference, and create an instance of the non const object by threads. This way, I won't have to handle concurrency between my threads myself (via mutexes for example).

Would I then be able to "send" this threads to RESTinio, so that every time a request is received, RESTinio will use one of the threads I have provided with my data?

Thanks again for your answer!

[eao197]

Hi @AurelienLP!

Let me describe my point of view and correct me please if I'm wrong somewhere.

You have several worker threads that you want to use to handle incoming requests. It is possible and there are at least two ways to achieve that.

The first way. You have a separate thread (or even separate thread pool) where RESTinio will handle all I/O, parsing of incoming data and controlling the writing of outgoing data. In that case, you use RESTinio's run function (or something like that) to run RESTinio's server on that separate thread (or thread pool). You also have several your worker threads with your objects (that you mentioned above). And your task, in that case, is to delegate request processing from RESTinio's thread to some of your worker threads. You can easily do that by using some kind of message-passing. There's an example in RESTinio that shows such approach. That example uses a message-queue from SObjectizer project but that can be any other thread-safe queue implementation. An another example of such an approach you find here (the source code of message-queue is here).

The second way. You don't want to have separate threads for RESTinio itself. And you want to perform all the work (including I/O, parsing incoming data, controlling the output, and so on) on your worker threads.

In that case, you have to have to create your own Asio's io_context object, create an instance of RESTinio's http_server_t class, pass that object to RESTinio's server via external_io_context, manually call io_context::run on every of your worker thread. In such scenario RESTinio will call your request-handlers from some of your worker threads.

[AurelienLP]

Hi @eao197,

Thank you for your answer!

To make it even clearer, I'd like to do something like https://github.com/Stiffstream/restinio/blob/v.0.6.8.1/dev/sample/express_router/main.cpp but in a multi threaded context.

Here, m_books in books_handler_t is not const so in a multi threaded context will have to use locks to handle modifications which I want to avoid.

The idea would be to have multiple threads, each one containing it's own instance of book_collection_t.

I'll try your different options and I'll keep you posted.

[eao197]

The main problem is the necessity of some execution context where RESTinio has to process I/O operations and the handling of incoming data.

Suppose that context is a single separate thread. On this thread, RESTinio accepts new connections, reads some data from accepted connections, parses the incoming data, and finds an appropriate request-handler. If the request-handler is found it will be called on the context of that separate thread.

If all application data belongs to that separate thread the things are very simple. All request-handlers will have access to all the data.

But if there is a thread pool for RESTinio the things become complex. RESTinio reads data from an input connection on one of the threads and parses data read right here, without switching to some other thread. A request-handler for the incoming request will be searched right here too, and if the handler is found, it will be called on the same thread.

The problem is that any thread from the pool can accept any request. And you can't know what a request is in the accepted connection and which data will be necessary for processing such request.

Let suppose that you want to handle GET requests on one thread, POST request on the second thread, and PUT request on the third thread. If all those threads are in the pool on that RESTinio is working then you can't make such separation. Because any thread from the poll can accept any of request types.

So the best way, I think, is to have a separate thread for RESTinio where all requests are accepted and all request-handlers are invoked. And a separate worker thread pool where all requests will be actually processed. A request-handler called by RESTinio on the separate thread detects which worker thread should be used and delegates actual request processing to that thread (by some kind of message-passing interface). Then request-handlers returns request_accepted to RESTinio and RESTinio handles next incoming request(s) while some worker thread does actual request processing.

[AurelienLP]

I've made a diagram

If I understand clearly, this is what you are suggesting?

[eao197]

Yes, something like that.

But note that I/O is efficient and RESTinio can handle dozen thousands of requests per sec on a single thread. So if you don't have actually big load you can just have a single I/O thread.

[AurelienLP]

Alright, thanks @eao197! If I manage to have a small program showcasing that use case, would it be ok for you if I create a MR?

[eao197]

What is "MR"?

[AurelienLP]

I meant PR (Pull Request)! Sorry, MR is for Merge Request! 😆

[eao197]

I don't think there is a need for yet another example in the RESTinio source tree. But if you make some public prototype for yourself I can review it.