Elektra

Elektra is an opinionated Openstack Dashboard for Operators and Consumers of Openstack Services. Its goal is to make Openstack more accessible to end-users.

To that end Elektra provides web UIs that turn operator actions into user self-services.

User self-services:

• User onboarding
• Project creation and configuration
• Quota requests
• Authorizations and access control for projects and services
• Cost control

We have UIs for the following Openstack core services:

• Compute: servers, server images, server snapshots (Nova)
• Block storage: volumes, volume snapshots (Cinder)
• User and group roles assignments (Keystone)
• Secure key store (Barbican)
• Software-defined networks, routers, floating IPs, security groups (Neutron)
• Loadbalancing (Octavia LBaaS)
• DNS (Designate)
• Object storage (Swift)
• Shared file storage (Manila)

Extended services:

• SAP Automation as a Service
• SAP Hana as a Service
• SAP Kubernetes as a Service

Project Landing Page:

Where does the name come from?

In Greek mythology Elektra, the bright or brilliant one, is the Goddess of Clouds with a silver lining.

Installing and Running Elektra

Steps to setup a local development environemnt

MacOS

1. Install postgres database (current version is 12.x).

   brew install postgresql@12
   brew link postgresql@12
   createuser -s postgres

2. Install ruby version 2.7.6

   brew install ruby-install
   ruby-install ruby 2.7.6

3. Install chruby to change the current ruby version (optional). This is helpful when having projects with different ruby versions.

   brew install chruby

   Ensure to implement the necessary changes in your .bashrc or .zshrc according to brew:

   https://formulae.brew.sh/formula/chruby

4. Install nodejs if not installed. (current working version 12.22.6 but higher versions works also fine)

   brew install nodejs@12
   brew link nodejs@12

5. Install yarn (actual version is 1.19.2 but higher works also fine)

   brew install yarn@1.19.2

6. Clone this repository to your machine.

   git clone https://github.com/sapcc/elektra.git

7. Install bundler Cd into elektra/ directory and run:

   gem install bundler -v 2.3.13 (check for the actual version)

8. Compile and install elektra gems Cd into elektra/ directory and run:

   bundle install

9. Compile and install node modules Cd into elektra/ directory and run:

   bundle exec yarn

10. Create, migrate and seed the database Cd into elektra/ directory and run:

    bundle exec rake db:create db:migrate db:seed

11. Copy the env.sample file to a .env file and adjust the values

    • Set the MONSOONOPENSTACK_AUTH_API* values to your devstack/openstack configuration settings
    • Enter the database configuration parameters

12. Start the Elektra dashboard application a. Run rails puma server

    bin/rails server -p 3000

    b. Run react live compiling

    bin/yarn build --watch

    Browser access for Elektra: http://localhost:3000

Linux

1. Clone the repository with git clone https://github.com/sapcc/elektra.git
2. Install Yarn and PostgreSQL via package manager
3. Check if the ruby version in your package manager matches the version number in .ruby-version. If yes then install ruby via your package manager. If no then follow the extra steps:
   1. Set up rbenv and ruby-build according to their documentation.
   2. Install Ruby with rbenv install 2.7.6 (substitute the Ruby version with the one from the aforementioned file).
4. Install Ruby gems with bundle install.
5. Install JavaScript packages with yarn.
6. Create database if not already done ./testing/with-postgres-db.sh bin/rails db:prepare
7. In one terminal or tmux, run yarn build --watch to compile the JavaScript assets. Leave this running until you're done with development.
8. In a second terminal or tmux, run ./testing/with-postgres-db.sh bin/rails server -p 3000 to run the Ruby application. Leave this running, too.
9. Now you can access the GUI at http://localhost:3000. When coming in from a different machine, you need to set up forwarding for ports 3000, e.g. ssh -L 3000:127.0.0.1:8180 -L 8081:127.0.0.1:8081.

After each pull, you may have to repeat steps 4-5 if the Ruby version or any package versions were changed.

NixOS notes

Step 2 and 3 should be replaced with the following commands and the nix-shell must be kept open for the steps afterwards.

nix-shell
bundle config build.sqlite3 --with-sqlite3-include="$(nix-store -r "$(nix-instantiate '<nixpkgs>' -A sqlite.dev)")/include" --with-sqlite3-lib="$(nix-store -r "$(nix-instantiate '<nixpkgs>' -A sqlite.out)"'!out')/lib"

or with nix-command enabled:

nix develop -f shell.nix
bundle config build.sqlite3 --with-sqlite3-include="$(nix eval nixpkgs#sqlite.dev)/include" --with-sqlite3-lib="$(nix eval nixpkgs#sqlite.out)/lib"

Use Elektra Request Management

1. Create another administrator user for the Default domain with email address in Horizon or with CLI
2. Configure MONSOON_DASHBOARD_MAIL_SERVER accordingly
3. Restart Elektra Application

Available Services in Elektra

If elektra is configured against a standard DevStack installation, only the core services like identity, nova, neutron and cinder are (sometimes partly) available and shown in Elektra. Additional services like swift, LBaaS, manila, ... will only be shown when available from the DevStack backend side.

Create a new Plugin

For more information about plugins, see the chapter "What are Plugins?" below.

The complexity of a plugin may vary greatly depending on its purpose. For example a Lib Plugin includes no app tree and is not mountable. However, it contains a lib folder and therefore implements libraries which may be used in other plugins. The next complexity level is the ServiceLayer Plugin, which already contains a partial app tree but isn't mountable and doesn't define views, it offers a service or library which may be used in other plugins (the Core::ServiceLayer is an example of such a plugin). The last plugin type is the mountable plugin which includes a full app tree and its own routes and views and is able to be mounted and act as an isolated rails app (The network plugin is an example of a mountable plugin).

• Lib Plugin
  • includes a "lib" directory and no app tree
• ServiceLayer Plugin
  • includes an implementation of ServiceLayer and DomainModel
  • app tree partially available
  • is a Rails Engine
• Mountable Plugin
  • includes a full app tree
  • can be mounted and define own routes
  • is a Rails Engine

For ease-of-use we have provided a generator which generates a skeleton plugin folder structure with the necessary elements and some basic classes with the proper inheritance to get started. First decide which type of plugin you want to start developing (for more infos about plugins see "What are Plugins?" below):

Create Lib Plugin

cd [Elektra root]

bin/generate dashboard_plugin NAME

Create ServiceLayer Plugin

cd [Elektra root]

bin/generate dashboard_plugin NAME --service_layer

Create Mountable Plugin

cd [Elektra root]

bin/generate dashboard_plugin NAME --mountable

Create Mountable ServiceLayer-Plugin

cd [Elektra root]

bin/generate dashboard_plugin NAME --mountable --service_layer

Create React Plugin

cd [Elektra root]

bin/generate dashboard_plugin NAME --react

For more information use: bin/generate dashboard_plugin --help

Creating Migrations

If your plugin needs to save things in the Elektra database, you'll need to create a migration. Migrations and the models they belong to live within the plugin. One additional step is necessary to register your migration with the host app so that it is applied when rake db:migrate is called in the host app. To create a new migration in your plugin do the following:

Background: you have a plugin named my_plugin.

1. cd [Elektra root]/plugins/my_plugin

   Inside this (mountable) plugin you will find a bin folder and rails script within this folder.

2. bin/generate migration entries

   A new migration was generated under plugins/my_plugin/db/migrations/

3. Register this engine's migration for the global rake task

   initializer 'my_plugin.append_migrations' do |app|
     unless app.root.to_s == root.to_s
       config.paths["db/migrate"].expanded.each do |path|
         app.config.paths["db/migrate"].push(path)
       end
     end
   end

4. cd [Elektra root]

   rake db:migrate

Plugin Assets

The Elektra UI design is a theme for Twitter Bootstrap (v3.~). All components described in the Twitter Bootstrap documentation also work in Elektra. Additionally we have added some components of our own. We have included Font Awesome for icons.

Important: When building views for your plugin please check existing plugins for established best practices and patterns and also check with the core team so that the user experience stays the same or similar across plugins.

In many cases the provided styles will be enough to build your views. If you need extra styles or scripts please coordinate with the core team to see whether we should include them in the core styles so they become accessible for everybody or whether they should remain specific to your plugin. Assets that are specific to your plugin must be located in the assets folder in your plugin.

What are Plugins?

The concept of plugins aims to outsource parts of Elektra, thus enabling developers to work decoupled from the main app and from each other. Rather than putting everything in the "app" directory of the main app, the controllers, views and models are split into plugins. An Elektra plugin encapsulates functionality which belongs together conceptually and/or technically and which is to be integrated into Elektra for consumption by the end user. The network plugin for example contains all the necessary controllers and views as well as helper classes to create, edit and delete network objects.

The core app provides layout, user and token handling, manages the plugins and offers classes that can be extended by plugins to make use of the functionality they provide. For example, checking whether the user is registered or logged in and the logic for the rescoping is implemented in the DashboardController in the core app. Plugin controllers can inherit from this class and won't have to worry about user management themselves.

Furthermore, the core app provides a service layer through which plugins are able to access other plugins' service methods on the controller level.

Service Layer

In principle an Elektra plugin is able to store data in the Elektra database and to access it via the ActiveRecord layer. However, many plugins communicate via an API with services which persist the necessary data themselves. Elektra plugins use the Service Layer to communicate with these backend services. Services in this case are primarily OpenStack services like compute or identity. Though other custom services can also be accessed the same way by the plugin.

Important: The communication with such services requires a valid user token (OpenStack Keystone).

As described above, the DashboardController in the core app takes care of user authentication. Each plugin controller that inherits from this controller automatically includes a reference to current_user which represents the token. The plugin can now use the information in current_user (mainly the token) to interact with the backend services.

But how can a plugin, for example the compute plugin, access the network methods which are implemented in the network plugin? This is where the service layer comes into play. The DashboardController offers a method called services which contains the reference to all available plugin backend services. For example: services.networking.networks invokes the method networks from the network backend service. Thus, the Service Layer represents a communication channel to the main backend service a plugin consumes and also to the other plugin backend services.

Before you consume other backend services: Check how expensive a backend call is. If it is expensive take steps to reduce how often the call is made (e.g. by caching, displaying the information on a view that isn't accessed very often) or at least make the call asynchronously so as to not block the rest of the page from loading.

Driver Layer and Domain Model

To avoid services having to communicate directly with the API and each plugin having to implement its own client, we introduced a driver layer. This layer is located exactly between the service and the API client. Thereby it is possible to abstract the services from the specific client implementation. The driver implements methods that send or receive data to or from the API and are invoked directly by a service. The data format between service and driver is limited to the Ruby Hash. Hashes are in principle sufficient for further processing, but in the UI data is usually collected via HTML forms and must be validated before it is sent on to the API. Furthermore you often require helper methods that are not implemented in the hashes.

The mentioned drawbacks of pure hash use are eliminated by the concept of the Domain Model. The Domain Model wraps the data hash and implements methods that work on this hash. By inheriting from the core Domain Model (Core::ServiceLayer::Model) your model gets CRUD operations out of the box. You can then add additional methods for formatting, processing, etc.

Services call driver methods and map the responses to Domain Model objects and, conversely, the Domain Model objects are converted to hashes when they reach the driver layer. As a result, it is possible to work with real ruby objects in plugins rather than using hashes. In such Domain Model objects we can use validations and define helper methods.

Plugin Folder Structure

The following diagram illustrates how plugins are structured and which core classes to inherit to make it all work as described above.

Click here for a detailed class diagram

Adding gem dependencies with native extensions

The Elektra Docker image does not contain the build chain for compiling ruby extensions. Gems which contain native extensions need to be pre-built and packaged as alpine packages (apk).

Audit Log

Each controller which inherits from DashboardController provides access to audit log via audit_logger. Internally this logger uses the Rails logger and thus the existing log infrastructure.

How to use Audit Logger

audit_logger.info("user johndoe has deleted project 54353454353455435345")
# => [AUDIT LOG] user johndoe has deleted project 54353454353455435345

audit_logger.info(current_user, "has deleted project", @project_id)
# => [AUDIT LOG] CurrentUserWrapper johndoe (7ebe1bbd17b36c685389c29bd861d8c337d70a2f56022f80b71a5a13852e6f96) has deleted project JohnProject (adac5c36277b4346bbd631811af533f3)

audit_logger.info(user: johndoe, has: "deleted", project: "54353454353455435345")
# => [AUDIT LOG] user johndoe has deleted project 54353454353455435345

audit_logger.info("user johndoe", "has deleted", "project 54353454353455435345")
# => [AUDIT LOG] user johndoe has deleted project 54353454353455435345

Available Methods

• audit_logger.info
• audit_logger.warn
• audit_logger.error
• audit_logger.debug
• audit_logger.fatal

Catch Errors in Controller

The Elektra ApplicationController provides a class method which allows the catching of errors and will render a well designed error page.

rescue_and_render_exception_page [
  { "Excon::Error" => { title: 'Backend Service Error', description: 'Api Error', details: -> e {e.backtrace.join("\n")}}},
  { "Fog::OpenStack::Errors::ServiceError" => { title: 'Backend Service Error' }},
  "Core::ServiceLayer::Errors::ApiError"
]

Errors that are caught in this way, are rendered within the application layout so that the navigation remains visible. For example if a service is unavailable the user gets to see an error but she can still navigate to other services.

rescue_and_render_exception_page accepts an array of hashes and/or strings. In case you want to overwrite the rendered attributes you should provide a hash with a mapping.

Available attributes:

• title (error title)
• description (error message)
• details (some details like backtrace)
• exception_id (default is request uuid)
• warning (default false. If true a warning page is rendered instead of error page)

Display Quota Data

If the variable @quota_data is set the view will display all data inside this variable.

How to set @quota_data

This will load quota data from the database and update the usage attribute.

@quota_data = services.resource_management.quota_data(
  current_user.domain_id || current_user.project_domain_id,
  current_user.project_id,[
  {service_type: 'compute', resource_name: 'instances', usage: @instances.length},
  {service_type: 'compute', resource_name: 'cores', usage: cores},
  {service_type: 'compute', resource_name: 'ram', usage: ram}
])

Same example but without updating the usage attribute. It just loads the values from the database. Note that the database is not always up to date.

@quota_data = services.resource_management.quota_data(
  current_user.domain_id || current_user.project_domain_id,
  current_user.project_id,[
  {service_type: 'compute', resource_name: 'instances'},
  {service_type: 'compute', resource_name: 'cores'},
  {service_type: 'compute', resource_name: 'ram'}
])

Pagination

Controller

@images = paginatable(per_page: 15) do |pagination_options|
  services.image.images({sort_key: 'name', visibility: @visibility}.merge(pagination_options))
end

View

= render_paginatable(@images)

Mailer

From 19.08.2022 Elektra is using our own email service (Cronus) to send emails when users must be notified. This is the case for example when managing quota requests or creating new projects. Refer to config/mailer.md for details on how the mailer is setup.

Contributing

Please read CONTRIBUTING.md for details and the process for submitting pull requests to us.

Tailwind CSS Support

We use esbuild to build javascript bundles. In /app/javascript/essentials.js (which is referenced in the layout files) we import /app/javascript/tailwind.css file. This file also imports styles from juno-ui-components to use the juno color variables.

@import "https://assets.juno.qa-de-1.cloud.sap/libs/juno-ui-components@latest/build/styles.css";