Proposal: Pervasive Internal DDS
The core of this proposal is to replace various listener and callback mechanisms with the writer cache idea of DDS. Assume that we write two template classes WriterCache and ReaderCache. The caches support the instance and sample concepts. A ReaderCache can subscribe to at most one WriterCache of the same type. When a sample is written to the WriterCache, all of the ReaderCaches are updated synchronously.
Each ReaderCache has an optional listener which is analogous to a DDS Listener. The Listener can be invoked in a number of different modes, for example:
- Inline - The listener is invoked when the ReaderCache is updated by the WriterCache. This is the most efficient and dangerous from a concurrency perspective.
- Immediate - A job is queued for immediate execution of the writer thread. This assumes that the writer thread is processing an event handler. After the primary event handler, which may queue up a number of deferred jobs, it processes the deferred jobs before returning control to the reactor.
- Deferred - A job is queued for the invoking Reactor and the Reactor is notified. During handle_exception, the Reactor processes at least one queued job.
This pattern creates a standard approach to potentially deferred work which enables future improvements like a thread-pool reactor, eliminating the ReactorInterceptor, etc.
The desired outcome is to reduce the coupling between different sub-systems resulting in a more modular and testable design. A module may expose its Writer and Reader caches. It can then be tested by writing and reading samples. From a debugging perspective, the caches record the state of the system at points in time.
This can be used to decouple the DCPS-, Transport-, and Discovery-level entities. The ServiceParticipant et al. would maintain various writer caches for DCPS entities. Creating a writer for example, would create a new instance in the WriterWriterCache. Things interested in writers would be subscribed to the WriterWriterCache and receive events appropriately. (In this example, we are creating a Publisher and mirrors the BIT Subscriber.)
There are two efficiency issues that must be addressed. The first is how queuing and context-switching may affect the critical paths between transport and user-level readers and writers including the invocation of listeners. The Inline and Immediate listener invocation modes provide the leverage necessary for addressing this concern. Inline can be safely used if no locks are held when writing the sample. Transitively, this means that deadlocks will be avoided. There is a corner case of invoking a user-defined listener which then calls back into the core. Most likely, any listener reachable from user code should not use Inline.
The second efficiency concern is copying. This can be avoided through the use of suitable smart pointer types.
One disadvantage to using the Writer/Reader Cache Pattern is the creation and manipulation of the new data types.
Potential candidates for the Writer/Reader Cache Pattern are situations that require reverse locks and related work-arounds.
The Writer/Reader Cache Pattern can change should reduce the number of reverse locks and related work-arounds and create opportunities for other improvements related to reactive activities like a thread-pool reactor, eliminating the ReactorInterceptor, etc.
The Writer/Reader Cache pattern could be extended with other DDS ideas like multi-topic, content-filtering, etc.
None.
- Design, prototype, and evaluation (1w).
- Application to Network Config Monitor and unit tests (1w).
- Identify and prioritize other targets (1d).
- General application (TBD).
"Internal DDS" is the general term for this effort. This has yielded classes like InternalTopic, InternalDataWriter, InternalDataReader, and InternalDataReaderListener. InternalDDS has been applied to
- Network Config Monitor
- Multicast membership management
- Configuration
The Inline and Immediate listener modes have not been explored or implemented.