Proposal: Observability
When debugging an OpenDDS application, it can be challenging to determine what's going on. This document aims to collect ideas for observing OpenDDS so that users and operators can debug more efficiently.
The preferred approach is to expose information using Internal DDS topics. Currently, there are two internal DDS topics:
-
NetworkInterfaceAddress
- Each sample consists of a network interface name, address, and a flag indicating the ability to multicast. On Linux, OpenDDS reads the netlink socket to determine the network interfaces and addresses that are available. Other platforms load this information at start time. OpenDDS uses this information to join and leave multicast groups on specific network interfaces. TheNetworkInterfaceAddress
topic can be accessed viaTheServiceParticipant->network_interface_address_topic()
. SeeNetworkConfigMonitor.h
. -
Config
- Each sample is a key-value pair where the key and value are both strings. OpenDDS uses this topic to store configuration information and distribute it to various subsystems. TheConfig
topic can be accessed viaTheServiceParticipant->config_topic()
. SeeConfigStoreImpl.h
.
The ParticipantLocation
topic and the ConnectionRecord
topic are implemented as DDS built-in topics (BITs).
These topics allow a user to monitor how a participant is connected to a remote participant and the connection to an RtpsRelay.
The lesson learned from this work was 1) the approach had the right intentions, 2) this information is useful, but 3) the exact semantics of DDS BITs made using and implementing this functionality difficult.
In OpenDDS 4, these will be converting to internal DDS topics.
If the transport and discovery refactoring proposed in OpenDDS 4 uses internal DDS topics, then these topics will be available for users.
Ideas for additional internal topics include:
- Local Entities - Samples describe participants, publishers/subscribers, writers/readers at the DCPS level and the RTPS/transport level. This can include local and advertised IP addresses.
- Remote Entities - Like Local Entities but covers discovered participants, writers, and readers. This can include advertised IP addresses.
- Matches - Records associations between local entities and remote entities.
- Detected Addresses - Record the actual addresses be used by remote entities.
- Performance Monitoring - Record throughput, latency, dropped messages, etc.