Skip to content

Proposal: Design use of 3rd‐party dependencies in v4

Jump to bottom
Justin Wilson edited this page Nov 17, 2023 · 1 revision

Introduction

OpenDDS has a number of dependencies that have grown in an organic way. For v4, we would like to have a reasoned approach to dependency management. The goal of this document is to evaluate the dependencies and alternatives and approaches to satisfying the dependencies with pros and cons.

Dependency Management

OpenDDS currently uses all of the following techniques for dependency management:

  • Raw download - e.g., ACE/TAO can be downloaded as an archive and built.
  • Clone - e.g., ACE/TAO can be cloned from a git repository. This can be used to retrieve a specific version or the “latest” version.
  • Submodule - GTest and RapidJSON are included via submodule. The configure script can automatically download, configure, and build these.
    Submodules allow a dependency to be pinned at a specific version.
  • External dependencies - Users can specify the path to various dependencies.

For non-header-only dependencies, the raw download, clone, and submodule approach requires configuration and building. This is typically the responsibility of the configure script. This may be trivial but it is yet another thing that must be maintained.

OpenDDS does not currently require users to use a package manager. A user may elect to use a package manager. The two leading candidates are conan and vcpkg. One advantage of using a package manager is the possibility of using binary packages which would reduce the setup time for developers and users. One benefit of using a package manager is that it provides a consistent way for handling dependencies. That is, package management could replace the other mechanisms. This could reduce the cost of maintenance and make OpenDDS more approachable.

The package manager should facilitate configurations with optional dependencies. For example, if security is not enabled, then it should be possible to omit openssl as a dependency.

The package manager should provide a distinction between a development dependency and a usage dependency. For example, the specific version of GTest used by OpenDDS should not impact a user of OpenDDS, i.e., there should not be a transitive dependency. For the other dependencies, users must be aware that the dependencies exist and may limit what they can do. These dependencies should take a version range so as to not restrict users unduly. In general, the success of package managers outside of the C++ community and now in the C++ community means that modern developers expect a package management system.

One possibility that needs to be considered is packaging OpenDDS itself. If OpenDDS is packaged for vcpkg and/or conan, then the OpenDDS package must handle its dependencies in a way that is most convenient for the user. (Adam started on a vcpkg but it is not complete.) This may mean having OpenDDS dependencies packaged in the same system as OpenDDS. Along the same lines, we need to consider possible impacts on the OpenDDS release process and artifacts. For example, will the process of a user downloading an archive and building be changed in a significantly inconvenient way?

Xerces

Xerces is used for parsing XML documents in DDS Security and QoS XML. It seems that DOM parsing is required. QoS XML is validating. If we validate, use namespaces, etc., then more complete implementations like xerces and libxmls are required. Relaxing these requirements creates more options.

These two links present alternatives for XML parsing:

The alternatives are:

Features Ubuntu package/Vcpkg/Conan/embeddable
xerces Full and exact compliance System, vcpkg, conan
rapidxml [preferred] Fast, DOM, header only Vcpkg, conan
pugixml Less fast than rapidxml but easier to use, XPath, DOM Vcpkg, conan
libxml2 Full compliance, parses huge documents, XPath. Vcpkg, conan
tinyxml. XPath, deprecated
tinyxml2 [preferred] DOM, two files (embeddable) Vcpkg, conan
TiCPP [not preferred] C++ frontend for tinyxml

If the XML dependency cannot be made private to OpenDDS, then the popularity and size of xerces and libxml2 mean that OpenDDS will have to accept a range of versions as the application itself might be using these libraries. This implies that we should accept an installed library (user or system) and/or use a package manager if possible.

Migrating the QoS XML support into a different project creates the possibility of using a different library with a potentially smaller footprint.

Decision: Keep Xerces as the XML parser.

OpenSSL

Openssl is used to provide cryptographic functions for DDS Security. It is (by far) the most commonly used library for this. https://www.ssltrust.com/blog/alternatives-to-openssl

Alternatives include:

Features Ubuntu package/Vcpkg/Conan/embeddable
openssl Complete, openssl 3 addresses some of the reasons for LibreSSL System, vcpkg, conan
Mozilla’s NSS Probably has the necessary features Vcpkg, conan
LibreSSL Forked from openssl to removed cruft, used by BSD Vcpkg, conan
BoringSSL Forked from openssl by Google. Google does not recommend using it.
Bouncy Castle Java and C#

If the “security” dependency cannot be made private to OpenDDS, then the popularity and size of openssl means that OpenDDS will have to accept a range of versions as the application itself might be using this library. This implies that we should accept an installed library (user or system) and/or use a package manager if possible.

Decision: Keep OpenSSL - could we require v3 or newer?

RapidJSON

RapidJSON provides conversion between C++ defined types and JSON using the ValueDispatcher interface. RtpsRelay uses this capability for logging. If we desire to provide this functionality to users, then RapidJSON will continue to be a dependency. There are many alternatives at https://www.json.org/json-en.html.

RapidJSON is also used by the wireshark dissector plugin.

Alternatives for using RapidJSON include:

  1. Submodule.
  2. Include the source.
  3. Use vcpkg or conan.
  4. System package.
  5. User-provided.

The size of rapidjson (784K) makes any of these methods viable. Accepting an external package via the system or user does not make sense since RapidJSON is header-only and hidden from the user, i.e., the headers are not installed and symbols are not available. Furthemore, it allows us to use a specific version which is simpler. Converting from submodule to including the source simplifies configuration. The recommended approach in order of maintenance effort is 1) include the source, 2) use vcpkg, 3) use a submodule. Making JsonValueReader/Writer non-optional would simplify things further while increasing the size of the library by ~2.5MB (size of JsonValueReader.o and JasonValueWriter.o in debug build).

Moving JsonValueReader/Writer to an external repository is currently not viable without moving the RtpsRelay and/or changing the way it logs.

Decision: Keep RapidJSON.

GTest

Google Test is used for unit testing. GTest is currently included via submodule or external dependency and must be built.

Alternatives for using GTest include:

  1. Submodule.
  2. Include the source.
  3. Use vcpkg or conan.
  4. System package.
  5. User-provided.

Accepting an external package via the system or user could result in an incompatible version. GTest compiles quickly so there is little downside to the submodule approach or including the source. Including the source, using a submodule, or using a package manager allows us to peg the version which is simpler. Any of the options is good.

Decision: GTest should continue to be optional as it is primarily used by OpenDDS developers.

Qt5/Qt6, glib, and Wireshark

Qt5/Qt6 are used by the monitoring application and iShapes demo. The Wireshark dissector depends on glib and Wireshark.

The new monitoring application based on XTypes will be hosted in an external repository. The existing monitoring application will be removed in v4.

Moving the wireshark dissector and shapes demo into an external repository is recommended. This removes Qt, glib, and Wireshark as a dependency.

ACE/TAO

In OpenDDS v3.x, the user can build ACE/TAO on their own and provide a path or the configure script will download ACE/TAO and arrange for it to be built. We have currently not explored the possibility of using packages for ACE/TAO (vcpkg).

One of the goals for v4 is to remove TAO as a dependency. The only parts of TAO that will be required are those needed for opendds_idl. Achieving this without any other improvement will provide a better experience for the user as the overall build time and artifact sizes will decrease.

Along these same lines is the idea of building OpenDDS with a pared-down ACE. Users that don’t use any other part of ACE are burdened with its legacy in the form of compiler warnings, bloated binaries, etc. Users that are using other parts of ACE should be allowed to build their applications with their preferred version of ACE. This either means that v4 should 1) allow users to manually provide a configured and built ACE for OpenDDS or 2) isolate OpenDDS’s dependency on ACE so that it uses a pinned version. (2) is the preferred option as it reduces the maintenance burden.

The current proposal is to create a fork of ACE called opendds-acetao. This fork would be customized to only include the features necessary for OpenDDS. This approach retains the relationship with upstream ACE so that updates can be easily merged.

Configure and Build for ACE (and TAO_IDL_FE):

  • Users can provide their own (needs to be within a supported branch/version like 7.x)
    • This could be via pkg manager
    • Must be already built
  • Normal use case is to use a submodule
    • Pinned version
    • "Reduced" build: ACE (subset - see Tim's script), TAO_IDL_FE, XML_Utils?
      • Versioned namespace can prevent name collisions
        • Only really works that way if none of its headers are exposed to the application
      • libACE_for_OpenDDS.so (include code from ACE_XML_Utils directly if Xerces is configured)
      • Script for making changes to repo based on upstream (delete directories we don't need)
    • Add CMakeLists.txt (and whatever else is needed) to our fork of ACE
      • OpenDDS's "top-level" CMakeLists.txt knows the relative path to ACE's CMakeLists.txt since it's a submodule
      • Based on user configuration (-D…) would include this via add_subdirectory OR using the provided path to an "external" ACE which is already built (find_package gives us ACE_ROOT)

For simplicity, the configure script or CMakeLists.txt would use a pegged version of opendds-acetao by default (using the git submodule). The only other option would be to use what is in the master branch.

Assuming that opendds-acetao exists, how is it pulled into OpenDDS? Options include:

  • Raw download
  • Submodule
  • A package manager

Raw download requires a tool like wget, curl, perl, or CMake that can perform the download. The submodule approach assumes that the user is compiling from the git repository. Using a package manager implies that opendds-acetao was suitably packaged.

Summary of Recommendations

  1. Produce a package for opendds-acetao.
  2. Use a package manager for all dependencies to achieve a consistent and modern way of dealing with dependencies. All of the current dependencies can be satisfied with vcpkg or conan.
    • Use a range of versions as users might need the flexibility.
    • The GTest dependency should be pegged as users are not subject to it.
    • ACE/TAO is the only dependency where a user can provide a path.
Dependency Required Range Notes
Xerces optional range User must opt-in due to size.
OpenSSL optional range User must opt-in due to size.
RapidJSON required pegged, submodule 2.5MB worst case. (header-only), Encapsulated in OpenDDS
GTest optional pegged, submodule Only needed for OpenDDS development.
Perl optional range Only needed to run tests
Qt5/Qt6 removed N/A Dependency transferred.
ACE/TAO required range Core dependency.

  • RapidJSON, GoogleTest:

    • Submodule or subdirectory (both peg the version):
      • Submodule lets us bring in upstream changes easier [preferred]
      • Subdirectory is simpler for users (don't need to git clone –recurse-submodules)

  • Xerces, OpenSSL

    • OpenDDS: find_package(OpenSSL) -> let it do its thing
    • User: system package, custom build from source, package manager (vcpkg style)
Clone this wiki locally