-
Notifications
You must be signed in to change notification settings - Fork 82
Nimbus 2.4 Installer
The Nimbus 2.4 release will include a brand new installer. This work will be the first of many improvements intended to simplify the process of going from zero to Nimbus cloud on a cluster of machines.
It’s hard to get started with Nimbus. We have complicated dependencies and configuration which can be daunting for unfamiliar users. Installing requires many steps across multiple machines, with plenty of room for error.
Additionally, the installed Nimbus environment leaves much to be desired. The Nimbus services are deployed into a Globus 4.0.x container (which must first be installed and configured manually). This is perhaps a bit confusing as in most cases, users will only have Nimbus deployed to the container. Also, the container environment is missing facilities for managing services (start/stop/restart) and generating logfiles. Ultimately these problems do not make for a very polished installation.
The installer work for 2.4 is intended to be the first of several improvements to the Nimbus install and configure process. It focuses entirely on installing and configuring the service node. It does not attempt to simplify VMM node installation or establishing secure communication between nodes. These features will be added in future releases.
Many advanced users are used to installing Nimbus in the existing ways. We don’t want to force them to change, so ideally we will not break existing functionality. Users may for example choose to install their own GT container and use the ANT build and deploy script as usual. This is fine, but it will be their responsibility to also make the configuration changes that would have been done by the setup scripts in 2.4.
If possible, we should enable distribution of binary tarballs (and packages) instead of just source.
Although some user input is needed to configure the Nimbus services, configuration scripts should be written in such a way that these inputs can be specified non interactively—via configuration file or command line argument. This will facilitate automatic installations as well as packaging Nimbus for a Linux distribution.
Instead of installing Nimbus into a functional Globus 4.0.x container, the 2.4 installer introduces the concept of $NIMBUS_HOME. This is the Nimbus installation directory which contains an embedded GT container, the Nimbus web application, a toy CA, and a bundle of scripts used to manage and configure the service. Here is a listing of a Nimbus installation:
|-- bin
| |-- nimbus-configure
| `-- nimbusctl
|-- sbin
| |-- nimbusctl.py
| |-- run-services.sh
| |-- run-web.sh
| `-- setup.py
|-- services
| |-- bin
| |-- endorsed
| |-- etc
| |-- lib
| |-- share
| |-- var
| |-- client-config.wsdd
| |-- container-log4j.properties
| `-- log4j.properties
|-- var
| |-- ca
| |-- run
| |-- hostcert.pem
| |-- hostkey.pem
| |-- keystore.jks
| |-- services.log
| `-- web.log
|-- web
| |-- bin
| |-- lib
| |-- sbin
| |-- src
| |-- var
| `-- nimbusweb.conf
`-- nimbus-setup.conf
- bin/ contains scripts for configuring and managing the Nimbus services. A user may add this directory to their $PATH if so desired. More on these scripts later..
- sbin/ contains some scripts which are more for expert users. Many of them are called by the bin/ scripts
- services/ is a full Globus ws-core-4.0.8 installation. Nimbus service configuration lives under services/etc/ in all the same places as it used to.
- var/ contains the toy CA, host certificates and keys, logfiles, and process runfiles.
- web/ has the Nimbus web application which itself is pretty self-contained.
The installer requires Java 1.5+ and Python 2.4+ to be available on the machine. If you use the Nimbus web application, pyOpenSSL and python-sqlite2 are required.
To install Nimbus to /opt/nimbus2.4 run
$ bin/install /opt/nimbus2.4
from the Nimbus source repository root. The script creates the $NIMBUS_HOME directory, installs Nimbus, and runs an interactive configuration script. When completed, the services and web application are configured and in a runnable state. Nimbus will be set up in fake mode, but all services such as the metadata server, Nimbus admin API, and Context broker will be configured with sensible defaults (or based on user input).
To start the services up after installation, run:
$ cd /opt/nimbus2.4
$ bin/nimbusctl start
Launching services... OK
Launching web... OK
Several scripts are involved behind the scenes in the installation and configuration process. In this section $NIMBUS_SRC refers to the source repository and $NIMBUS_HOME refers to the installation directory.
$NIMBUS_SRC/bin/create-nimbus-home accepts a path argument of the destination install directory. It performs the following actions:
- Creates a Nimbus installation directory at $NIMBUS_HOME
- Installs some scripts into $NIMBUS_HOME/bin/ and $NIMBUS_HOME/sbin/
- Installs a clean web directory into $NIMBUS_HOME/web/
- Downloads and installs ws-core into $NIMBUS_HOME/services/
- Ant deploys Nimbus with GLOBUS_LOCATION=$NIMBUS_HOME/services/
$NIMBUS_HOME/bin/nimbus-configure interactively configures the Nimbus service. It is intended to be run repeatedly. If you want to change a configuration value for example, just rerun this script. Most of the functionality actually lives in $NIMBUS_HOME/sbin/nimbusconfigure.py. It performs the following actions:
- Generates CA and host certs in $NIMBUS_HOME/var/ if they do not exist. This will likely require input from the user.
- Performs the tasks currently done by autocontainer setup-container.sh script to set up the container, using the generated certs. This may require input from the user, specifically to confirm the logical hostname to use for services.
- Configures the web application to use the generated CA and certs.
- Configures the Context broker to use the generated CA.
- Generates a java keystore from the SSL cert+key and sets up appropriate paths in the Nimbus install, wherever Jetty is used (Query API)
It is also expected that additional functionality will be added to the configure script. For example, to import databases and configuration from older Nimbus installations.
$NIMBUS_HOME/bin/nimbusctl is a init-style script (start\|stop\|restart) that is used to manage the installed Nimbus services. It lives in $NIMBUS_HOME/bin/. It starts the GT container and the Django server as daemon processes. STDOUT and STDERR will be redirected to log files in $NIMBUS_HOME/var/. It also supports starting/stopping a single service with an optional first command, see “nimbusctl -h”.
$NIMBUS_SRC/bin/install is what the user invokes (in the standard install use case). It calls create-nimbus-home followed by nimbus-configure.
Aside from the functionality marked as “NOT DONE” above, there are some issues I am concerned about.
Upgrading is hard to do. You cannot bin/install to an existing Nimbus installation. Well, you can, but some things will be overwritten. I think we need a coherent vision for how upgrades should work before we release. The GT container has some facilities for upgrading which we can use and we may be able to leverage pip/virtualenv for upgrading the python components.
Absolute paths are everywhere. The Nimbus deploy system and much of the spring configuration use absolute system paths. This makes $NIMBUS_HOME much less portable and also makes binary packages/tarballs harder. We have a plan for removing most of the absolute paths, but it isn’t done yet and there could be others that are harder to fix.
Only one install path is well supported. The installer as it is doesn’t well support some important things like custom host certificates or installing only the services without the webapp (or vice versa). These things are all possible but require manual configuration (for which there is no documentation yet).
Not all user input is scriptable. Specifically, the creation of the Django administrative user requires interactive terminal input. For this and style reasons, I wonder about moving this function into the webapp itself. Like, the first time you hit the web server, it asks you to create this account before you can do anything else.