Fork of Drake's install_prereqs.sh package list is overly brittle

[jwnimmer-tri]

https://github.com/RobotLocomotion/drake-external-examples/blob/master/scripts/setup/linux/ubuntu/xenial/install_prereqs duplicates Drake's list of dependencies. This is a maintenance burden. We need to find a way to allow for everyday dependency changes to Drake without PR'ing to two separate places.

Conceivably, more intricate changes such as bumping Bazel, or changing the compiler, could have duplicated logic. But "Add system libfoo-dev" should not require a separate PR here.

[stonier]

Yes, already noted, but thanks for making it an issue.

[nuclearsandwich]

One option would be to have drake-external-examples's setup try to curl Drake's from GitHub. Could even get fancy with branch logic if you wanted.

#!/usr/bin/env bash

set -o errexit
set -o pipefail

DRAKE_BRANCH="${1:-master}"
PLATFORM="ubuntu/16.04"

DRAKE_INSTALL_PREREQS_URL="https://raw.githubusercontent.com/RobotLocomotion/drake/${DRAKE_BRANCH}/setup/${PLATFORM}/install_prereqs.sh"

if [[ ! $(which curl > /dev/null 2>&1) ]]; then
	apt-get update && apt-get install -y curl
fi

echo "Fetching Drake prereqs script."
curl "$DRAKE_INSTALL_PREREQS_URL" > drake-install-prereqs.sh
bash drake-install-prereqs.sh

It's just a mock-up, not comprehensive or robust, but one possible option.

[stonier]

I was thinking of having some install script installed by Drake so it can be used within and without. It probably would be a subset of what a Drake bazel workspace needs as well.

[stonier]

I believe the present solution was just a temporary copy/paste from jamie to get CI on board.

[jamiesnape]

Yes, we are going to install a version of the install_prereqs.sh script with the binary packages, which contains the subset of packages needed by users of those packages.

[EricCousineau-TRI]

Given the re-breakage to be fixed by #142, perhaps we slate this to be worked on in early Dec?

[jwnimmer-tri]

I broke drake-external-examples master again because of this via https://reviewable.io/reviews/RobotLocomotion/drake/12680. Drake's package list of source dependencies changed, but I failed to copy that change into this repository.

[jwnimmer-tri]

Another hiccup in #177.

[BetsyMcPhail]

This came up again with changes in RobotLocomotion/drake#15618

[RussTedrake]

I've been calling drake's installed install_prereqs transitively from the underactuated / manipulation install_prereqs for a long time. Am I to understand that everyone agrees this is simple to do here, but just nobody has taken the time to do it?

[jwnimmer-tri]

That is probably true. I've added this issue to the Build & Release project board now.

The only reason I say "probably" here is that we not only want to make the CI jobs here run more smoothly, but also we want to make sure this example repository is what we actually want to recommend to end-users. As we develop any patch towards this issue, we'll want to make sure to keep that second use case in mind.

For example, in Anzu, we do not transitively call Drake's install_prereqs. Instead, we pin Drake and pin its prereq packages, and update the two in lockstep on purpose. This can help because the packages-bionic-test-only.txt are not necessarily required for external users, though perhaps transitively calling using --without-test-only would suffice. Also, things like clang-9 are in the non-test source prereqs, but are only required when building using one of our Clang variants (like a sanitizer). It's not clear that we want Downstream projects to use that. It's possible that the answer to all of these is that "those are Drake bugs; the example should always just transitive call Drake's setup script", but even in that case we should be sure to file the Drake bugs as we prepare and review these changes.

For the "use Drake via a binary release" variants in this repository, transitively calling the binary's install_prereqs is probably best. For "rebuild Drake from source", it might be more subtle (or not).

[RussTedrake]

As I was most of the way through reading your third paragraph, I was indeed thinking "those sound like Drake bugs". :-)

[BetsyMcPhail]

@alesgenova this issue might be good for you to look at

[jwnimmer-tri]

We should finish #219 before starting this one.

[jwnimmer-tri]

To re-iterate the above, the following examples should shell out to drake/share/drake/setup/install_prereqs from whatever version of Drake's binaries that they have downloaded:

• drake_bazel_installed
• drake_cmake_installed
• drake_catkin_installed
• drake_ament_cmake_installed

For the apt-based examples, we should only need to apt install Drake -- the dependencies should come along as part of that install already, there should be no to additionally call install_prereqs:

• drake_cmake_installed_apt

Let's fix the above ones before we dive into the from-source examples.

[EricCousineau-TRI]

Dunno if it helps / hurts, but as an interim we recently started doing a kind-of hacky "boostrappy Bazel" approach for drake-ros Bazel workflows:
https://github.com/RobotLocomotion/drake-ros/blob/a4d976d6ef093031b4a08e31446527c814882501/.github/workflows/bazelized_drake_ros.yml#L25-L30
This involves installing minimum deps to effectively enable something akin to bazel fetch @drake, then possibly overriding those deps (e.g. different Bazel) when Drake itself installs. Plus, it caches the repository download of Drake.