About request-response uses

[codingcai]

Hi ,
Thank you very much for providing such a great project. I encountered two issues when I started using it and would like to seek some advice.

1） When I make a request in a multi-threaded context, how can I differentiate on the server side which thread from the client made the request, so that the response can be returned to the correct thread? In the iceoryx_examples, during client initialization, an application is initialized with iox::runtime::PoshRuntime::initRuntime(APP_NAME), which further initializes a client : iox::popo::Client<AddRequest, AddResponse> client({"Example", "Request-Response", "Add"}); . If I use a multi-threaded client, do I need to create a client for each thread? Would this allow the server to differentiate between them?

2）I have a requirement where, on the client side, I allocate a buffer with char *buffer = (char *)malloc(N * sizeof(char)) write content into it, and then I want to send this content to the server as a string. How can I send this to minimize data copying? Should I use the built-in string from iceoryx?
Thank you very much :-)

[elBoberido]

Hi @codingcai,

thanks for the kind words :)

To your questions. The endpoinds, or better instances of endpoints, are not thread-safe. You either need to guard the instance by a mutex or need to instantiate multiple instances. The reason for this is that each instance tracks the the memory it loaned and these structures are not thread-safe since it would affect performance when nit needed and can easily made thread-safe when needed, e.g. via a iox::smart_lock.

The server knows the ID of the client which sent the request and responds only to that client. It is also possible to set a sequence ID if you have multiple threads accessing the same client. The corresponding response will have the same ID. You just need to provide the response to the right thread.

To your second question. Using the iox::string is the easiest solution but it requires a compile time string size. If your data consists only of the string, you could also use the untyped API and allocate memory from the mempool. If you configure the mempool with several chunk sizes, it could be more dynamic and memory efficient but you lose type safety.

Hope this answers gelp you :)

[codingcai]

Hi @elBoberido ，
Thank you for your reply.
From your response, it seems that if we need to transfer data in a multi-threads environment, we need to use locks to ensure the correct distribution of data and I get it. I also have some questions about this question:

1. If I declare multiple clients in multiple threads, for example, with the following code, are the spaces allocated in shared memory isolated for these clients with different names, and do they not affect each other? Can I achieve a scenario where each thread has its own space that is unaffected by others? When initializing the server on the server side, to obtain data corresponding to a specific client, do the names need to be the same?

// client thread1 
iox::popo::Client<AddRequest, AddResponse> client({"Example", "Request-Response", "Test1"});
// client thread2  (use different space with Test1?)
iox::popo::Client<AddRequest, AddResponse> client({"Example", "Request-Response", "Test2"});

// in Server, could I get client data using different name ? 
// thread1 
iox::popo::Server<AddRequest, AddResponse> server({"Example", "Request-Response", "Test1"});
// thread2
iox::popo::Server<AddRequest, AddResponse> server({"Example", "Request-Response", "Test2"});

2. I want to use the request-response model. In the examples provided by GitHub, the request structure is all of int type. However, if my type is as shown below, and the size of the string is only known when it is copied, which built-in structure should I use?

struct operation {
   int type;
   int size;
   string data; // what type should I use  (copy data from user code)
};

[elBoberido]

Regarding 1: They have separate structures but (depending on your config) those structures are in the same shared memory since we never had the time to properly isolate them for iceoryx1. The names need to match in order to be connected. You can have only one server with the same name but multiple clients with the same name and each client gets only the response for its request.

Regarding 2: I fear you have to choose a max size for the iox::string and use that. Dynamic data types are not supported with iceoryx1 but it is on our todo list for the Rust based iceoryx2

[codingcai]

Thank you, elBoberido
Regarding 1: if I have a separate client in each thread, do I need to lock when allocating space with this client? Or is it the case that even if the clients are different, I still need to lock due to the use of the same shared memory?

For the second question, I think I can copy the structure into an untyped string on the client side, and then I will parse this string on the server side myself, interpreting the string into a custom structure. :-)

[elBoberido]

No, you do not need to lock if there are multiple clients, even if they talk to the same server. You only need to lock if you access the same client instance from multiple threads.

[codingcai]

I also have another problem.
When running the iox-cpp-request-response-basic-client and iox-cpp-request-response-basic-server examples, I encountered an issue where, if I remove the sleep code （ std::this_thread::sleep_for(SLEEP_TIME);）from both the client and server, the client might throw an error: Got Response with outdated sequence ID! Expected...
Does this mean that when using request-response, it is necessary to sleep for a certain period after sending a request? If so, how should I control this duration?

[elBoberido]

Which example exactly? I guess it is the basic one and you also removed the line with std::this_thread::sleep_for(DELAY_TIME). The problem in this case is that the example tries to get the response before the server sent it. Since there is no response, the client sends the next request and expects the response to that request but initially gets the response from the previous request. Therefore it is important to use a sequence ID and skip the outdated responses if you do not need them anymore.

You can either periodically poll for a response until there is one or you can use a waitset, which is also used in one of the examples. The waitset will block until the response arrived but also increases the latency a bit.

[codingcai]

You're correct; during my testing, I used the examples from client_cxx_basic.cpp and server_cxx_basic.cpp.
Based on your explanation, if I use

 //! [take response]
    while (client.take().and_then([&](const auto& responsePayload) {
        // do something
    }))

1. when there is no response returned, the program will not wait here but instead end this while loop and then send new data, right?

2. I looked at the client_cxx_wait.cpp example, and as I understand it, the program will wait on the waitset until there is data. If this is the case, why is there a certain amount of latency? If I manually wait periodically, which API should I use?

3. Additionally, in the request-response example, I understand that there is generally only one return value in the waitset ? (in my scenario, this means sending a request and then retrieving a response, for example, I need to implement a remote write operation).

[codingcai]

I have used the example ./iceperf-bench-leader -t iceoryx-cpp-waitset-api for performance testing with waitlist, and under the publish-subscriber pattern, it is 0.7 microseconds, but the waitlist pattern requires 12 microseconds, which is a significant difference.

In the request-response pattern, I need to wait for the response to return the result before I can proceed to the next step, but I want to have less latency (10+ microseconds is a bit much for me). What mode should I use? Would busy waiting be appropriate here?

In addition, when I tested with ./iceperf/iceperf-bench-leader -t iceoryx-cpp-api, I found that the latency was 0.7 microseconds on the x86 machine, but it was 5 microseconds on the ARM machine. Is this phenomenon observed in the official tests?

[elBoberido]

1. when there is no response returned, the program will not wait here but instead end this while loop and then send new data, right?

Yes, that's correct

2. I looked at the client_cxx_wait.cpp example, and as I understand it, the program will wait on the waitset until there is data. If this is the case, why is there a certain amount of latency? If I manually wait periodically, which API should I use?

Well, without the busy loop you have to tell the OS that you are waiting. This involves a context switch which is quite expensive. The lowest latency can be achieved by polling but this has the negative effect of keeping the CPU busy while it could do more important work. It is hard to tell which patter is best for you without knowing the architecture.

3. Additionally, in the request-response example, I understand that there is generally only one return value in the waitset ? (in my scenario, this means sending a request and then retrieving a response, for example, I need to implement a remote write operation).

It is not related to the WaitSet but each request has only one response. You can send multiple requests before taking the responses though. You also no not need to use the WaitSet and block till the response arrives but use the sequence ID to designate a response to a request when multiple requests are sent before a response is received.

I have used the example ./iceperf-bench-leader -t iceoryx-cpp-waitset-api for performance testing with waitlist, and under the publish-subscriber pattern, it is 0.7 microseconds, but the waitlist pattern requires 12 microseconds, which is a significant difference.

This is related to the context switches and can become even higher when the OS puts the process with the server into a deep sleep mode. On Linux this can happen when the server process doesn't have any activity for 10 to 100ms.

In the request-response pattern, I need to wait for the response to return the result before I can proceed to the next step, but I want to have less latency (10+ microseconds is a bit much for me). What mode should I use? Would busy waiting be appropriate here?

The fastest option (but also the one burning your CPU) is to poll for the response. You could use yield to give other threads a chance to perform some work but it might increase the latency.

In addition, when I tested with ./iceperf/iceperf-bench-leader -t iceoryx-cpp-api, I found that the latency was 0.7 microseconds on the x86 machine, but it was 5 microseconds on the ARM machine. Is this phenomenon observed in the official tests?

Well, it highly depends on which ARM core it runs. On a Raspberry PI 4 it should be faster than 5µs.

[codingcai]

I have another two questions. :-)

1. In our use case, we have multiple clients corresponding to a single server process. Each client sends a request to the server, and upon receiving the message, the server processes it and returns a response to the respective client. Within this framework:
   1） Is it necessary for us to assign a different client name during the initialization of each client so that the server can identify the different origins of the requests?
   2） If we are to assign unique names to each client, does this imply that the server must be informed of the name of each client at the time of its initialization (via some form of notification mechanism)?
   3） If my clients are added or removed at irregular times, would I require an additional registration mechanism to ensure that the server initializes a corresponding server entity for each client?
   The diagram provides a simple description of the overall framework.
   

2. When we start the client, server, and Doudi, is there a potential memory leak if requests are not processed correctly or the client/server exits abnormally (not a normal exit)? Will these abnormal operations cause the shared memory already allocated in Doudi to be released? (I haven't read this part of the source code yet..)
   In the examples, we usually manually call releaseResponse to free resources after we have used them. If we exit directly, what mechanism is to ensure that the shared memory resources are released?

[codingcai]

It seems if I use the discovery method in server , I could solve this problem ?:-)