Skip to content

Proposal: Discovery Refactoring

Jump to bottom
Justin Wilson edited this page Nov 17, 2023 · 2 revisions

Introduction

The existing Discovery interface dds/DCPS/Discovery.h is a reflection of the original design of the InfoRepo. The interface consists of both actions and callbacks. An application acts on a discovery instance by adding participants, writers, readers, etc., and receives callbacks when the discovery instance determines that an association should be formed or dissolved.

The callbacks in the interface create problems with respect to multi-threading and long-term maintainability.

It can also be argued that Discovery takes on too much responsibility in that it controls associations. This is an artifact from the InfoRepo design and makes sense for efficiency. This functionality could be moved to the DCPS layer. With a new division of responsibility, Discovery is only responsible for publishing local entities to remotes and publishing remote entities locally. The DCPS layer, specifically participants, would subscribe and associate/disassociate as needed. The proposed design allows for either division of responsibility.

Proposed Design

The Service Participant will have a map from domain to Discovery instance.

Every discovery instance will have a Participant Discovery instance for each local participant in that domain.

A Participant Discovery instance consists of two main assemblies:

  1. A set of Internal DDS topics.
  2. A set of discovery drivers.

The set of Internal DDS topics are as follows:

  • Local Participant
  • Local Publications
  • Local Subscriptions
  • Remote Participants
  • Remote Publications
  • Remote Subscriptions

The Domain Participant writes to the Local topics and reads from the Remote topics. The Domain Participant has the responsibility of reading the Remote topics and maintaining associations with its contained entities. Note that the Local Participant topic should have a single record corresponding to the Domain Participant. The Domain Participant also has the responsibility of maintaining its BIT readers through data received from the Remote topics.

The set of discovery drivers read from the Local topics and write to the Remote topics. Initially, full set semantics are not necessary, i.e., the set will only contain one driver. Examples of Discovery Drivers includes RTPS Discovery, DCPSInfoRepo Discovery, Static Discovery, and In-process Discovery.

The use of Internal DDS Topics addresses issues of concurrency and provide a well-defined interface to discovery drivers.

Outline of Work

This work must be done for all existing discovery mechanisms: RTPS, Static, and InfoRepo. However, starting after the removal of the InfoRepo would be advantageous since there would be less code to update.

  1. Define data structures for Internal DDS Topics. The BIT types are a starting point but internally we would most likely include additional information.
  2. Define a Participant Discovery base class. The base class creates all of the Internal DDS Topics.
  3. Writing.
    1. DomainParticipantImpl writes samples on the Local Participant topic.
    2. DomainParticipantImpl or its delegate should write samples on the Local Publications topic.
    3. DomainParticipantImpl or its delegate should write samples on the Local Subscriptions topic.
    4. Discovery instances write samples on the Remote Participants topic.
    5. Discovery instances write samples on the Remote Publications topics.
    6. Discovery instances write samples on the Remote Subscription topics.
  4. BIT Maintenance.
    1. BIT creation moves from Discovery to DomainParticipantImpl.
    2. DomainParticipantImpl transduces Remote Participants topic to BIT.
    3. DomainParticipantImpl transduces Remote Publications topic to BIT.
    4. DomainParticipantImpl transduces Remote Subscriptions topic to BIT.
    5. Discovery implementations no longer responsible for maintaining BITs.
  5. Association logic refactor.
  6. Introduce Internal DDS Topic(s) for matching.
  7. Define a Matcher.
    1. Move existing match data structures, state-machine, and algorithms to DCPS layer.
    2. Type lookup will need to be exposed via Participant Discovery to make state-machine possible.
    3. The inputs to the Matcher are the Internal DDS Topics for local/remote publications/subscriptions.
    4. The output of the Matcher is a new topic that records local-local and local-remote matches.
    5. The DomainParticipantImpl can listen to the matches and add/remove associations as necessary.
    6. The Matcher could be operated locally by the DomainParticipantImpl or Discovery (RTPS Discovery) or remotely (InfoRepo).

Considerations for RtpsDiscovery

RtpsDiscovery is well-suited for these changes. SPDP and SEDP are transducers for the Internal DDS Topics. The association logic will be factored out and tested in the form of a local Matcher. New interfaces for initiating type lookup will need to be added. Internal DDS Topics should be considered for this to eliminate concurrency issues.

Considerations for InfoRepo

These changes can be taken up before the InfoRepo removal, if necessary. There are two possible designs.

First, the InfoRepo could just populate the BITs for each domain participant and a local (to the client) Matcher would drive association. This would remove the possibility of DCPSBit=0.

Second, the InfoRepo could perform the match and control a Matcher that just publishes the associations.

Considerations for Static Discovery

Currently, there are a number of hooks and special behaviors in the RTPS transport to support Static Discovery. Conceptually, these revolve around 1) advertising a local entity through some periodic message and 2) detecting remote entities through periodic messages and normal messages. Suppose that methods and/or Internal DDS Topics are added to the RTPS transport so that these behaviors can be controlled and detected outside of the RtpsTransport.

The Static Discovery implementation will read the Internal DDS Topics for local entities and drive the RtpsTransport to advertise the appropriate entities.

The Static Discovery implementation will read the Internal DDS Topic(s) of the RtpsTransport to detect remote entities that are active. It will then marry this information with its static database of domain entities and publish the topics for remote entities.

In-Process Discovery

An In-Process Discovery implementation would allow domain participants running in the same process to discover each other. The implementation is straightforward: the Discovery instance reads the Internal DDS Topics for local entities and re-publishes them to all of the Internal DDS Topics for remote entities. The DCPS layer would then take over and complete the associations. Loop-back Discovery might be a better name for this module.

[Thought exercise: would a localhost-only discovery be an extension of in-process (putting Internal DDS Topics in shmem or something like that), or would it be an extension of full RTPS Discovery (using shmem transport instead of udp), or would it just be RTPS sending to localhost?]

The primary use of In-Process Discovery will be writing tests, however, it may have uses beyond this and will serve as a reference example.

Considerations for Testing

The proposed design makes testing discovery straightforward.

A Participant Discovery instance has two responsibilities: convey local entities to remote participants and convey remote entities to a local participant. A (unit) test can write a sample to the Internal DDS Topics for local entities and check that the appropriate message was sent. A (unit) test can deliver a message and check that the appropriate sample was seen on the Internal DDS Topics.

Testing the DCPS layer is similar. Creating a participant, writer, or reader should result in a sample on an Internal DDS Topic for local entities. Writing a sample on an Internal DDS Topic for remote entities should result in a sample in the BITs. Writing a sample on an Internal DDS Topic could also trigger an association which can be detected via the DataReader and DataWriter interfaces.

The association logic can be tested independently. The Matcher consists of a transformational core wrapped in a reactive layer. The interface to the transformational core has methods for adding/removing local/remote writers/readers. It should have methods to check what associations exist. The reactive layer consists of a transducer that takes samples on the Internal DDS Topics and calls the appropriate method on the transformational core. The transformational core can write to an Internal DDS Topic directly (or a service that would do the same if that makes testing easier).

The reason the Matcher is “clever” is that it solves a number of problems around concurrency:

  1. The existing matching logic in RtpsDiscovery is complicated by locking. That is, it goes through phases where it locks something, acquires some data, releases the lock, and then has to check that the thing still exists. Operations on the Matcher will be atomic so the double-checking and convoluted error handling and correction can be eliminated.
  2. Adding an association, in RtpsDiscovery for example, results in an upcall to the local entity or entities being associated. This call proceeds eventually wends its way down to the transport. Publishing matches on an Internal DDS Topic moves the action to the DCPS layer which means discovery is no longer concerned with making an upcall.
Clone this wiki locally