Ansible In Unfetter
Ansible is an IT Automation framework, written in Python, that is designed to make it easier to build and configure infrastructure to match a desired configuration.
For Unfetter, we use Ansible Playbooks to make it easier to customize YOUR deployment of Unfetter without breaking the core script functions.
The different types of Unfetter Deployments are discussed on the Types and Deployments Page. Ansible knows which deployment type to use based on the host Group Variables, as discussed below.
Variables: Ansible Docs
Ansible variables are used to identify host specific attributes, control how the ansible playbooks build the docker images and containers, and allow for deployment specific configurations. You will change variables to customize the deployment of Unfetter in your environment
When you download Unfetter, the default variables will build a Production version of Unfetter with UAC using GitHub for OAuth. These variables are all assigned in the Group Variable files
Group Variables: Ansible Docs
Group Vars are variables that apply to a whole "group" of hosts. Group Variables are loaded from YML files in unfetter/ansible/group_vars.
We split hosts into the groups of development, production, uac, deployed and demo. A host can live in multiple groups, such as production AND uac. "development" and "production" groups control the ports that are exposed, whether to use pre-compiled Javascript or run the UI in a debug mode.
"uac" and "demo", on the other hand, controls if and how User Access Control is implemented.
"deployed" is a different group. Any hosts listed in this group will be the focused hosts of the playbooks. deploy-dev.yml and build-prod.yml are the only playbooks that ignore the "deployed" group, and are really intended for the developer's use.
Host Variables Ansible Docs
Unfetter recommends using Host Variables to OVERRIDE any of the Group Variables, rather than changing the Group Variables. This is because future versions of Group Vars might change, and you will have merge conflicts.
To customize your deployment, we recommend creating a prod-uac.yml file under a unfetter/ansible/host_vars directory under unfetter/ansible. This file will let you over ride any of the group variables you want.
Create a "host_vars" directory under ansible, and add a file for every host you wish to have host specific configurations. These variables will override the Group Vars. This is the recommended location to put your production specific variables in, so they are not overwritten by new versions of Unfetter.
In release 0.3.11 of Unfetter, there will be a demo host_vars file that will articulate the likely variables that need to be overwritten to deploy to your own environment
The override.yml file under group_vars will override the variables whether they are host vars, group vars, or role vars.
- backup_directory - This is the location where you want to store backups of configuration files, database, etc. This is defaulted to the ansible directory. however, you may wish to store backups somewhere else.
- api_domain - This is the domain in which you are hosting the API for Unfetter. This is used by the GitHub or GitLab authentication to validate your OAuth2 configuration. You can read more about GitHub authentication on our Authentication Wiki Page
- ui_domain - This is similar to the api_domain above
- https_proxy_url - This is the full hostname and port to the proxy server in use by the system where Unfetter will run. Syntax is "http://host:port"
Unfetter version 0.3.7 and later uses Docker named volumes to store all volatile and customer defined data. That includes the Mongo data, SSL certs, and any configuration file. When Unfetter is first started, Ansible will look to see if a named volume exists. If it does, then it uses the data in the volume. If the volume does not exist, it will look in the backup_directory for the files. The backups are made with the task-backup-vol-container.yml. That file is explained below.
If the volume does not exist, and the backup does not exist, then Unfetter will create new data from volumes and through user input.
The Ansible directory under the unfetter GitHub repository hosts the Ansible playbooks and roles. We assume at this point, you are familiar enough with Ansible to know how playbooks, variables and ansible roles work.
There are currently 3 types of playbooks that can be called directly from the command line with ansible-playbook
These playbooks are designed to deploy Unfetter through already created Docker Images. In the case of Development hosts, it will automatically build, from local source code, the Unfetter Discover stack from scratch.
We are relying on the hosts listed under the [deployed] group in the hosts.ini file to tell ansible WHICH hosts to deploy. You could deploy all versions of Unfetter on separate virtual machines all at once. deploy.yml takes whatever hosts are in this group and will deploy them.
ansible-playbook deploy.yml will deploy the production version of Unfetter, with UAC turned on, based on the local variables
ansible-playbook deploy-dev.yml will always build docker images and deploy the docker containers in the develop mode. This playbook is for developers specifically and should never be used in production.
These playbooks will build the docker images, but will not create and deploy the docker containers. If you wish to build Unfetter from source code, these are the playbooks to use.
ansible-playbook build-prod.yml will build all the docker images that are part of Docker Hub.
Tasks are configuration focused playbooks that perform a specific function or task. Although the build and deploy playbooks will get Unfetter running, these task playbooks are needed in the management, reconfiguration, or update of Unfetter
ansible-playbook will backup the database and configuration files from the Unfetter containers directly. The "backup_directory" variable tells ansible where to backup your files. All backups happen on the host Unfetter is running. Therefore, if you are remotely managing unfetter by running Ansible from an orchestration system, targeting a remote system, then the backup directory will be on that remote system. ❗️Unfetter must be running for this to work.
ansible-playbook task-prep-remote-host.yml will install all the baseline software needed to run Unfetter on a remote using SSH.
ansible-playbook task-reconfigure-private-vol.yml will forcibly recreate the private-vol volume that is used to setup GitHub or GitLab UAC. It will overwrite the backup files and the docker volumes
ansible-playbook task-upgrade* will run the scripts necessary to upgrade the backend database or configuration files for running next versions of Unfetter.
ansible-playbook task-push-registry.yml will properly tag and PUSH the docker images, created using build-prod.yml, to the Docker registry of choice.
Ansible is the way to manage the Unfetter application. You can read about Updating Unfetter using Ansible.
Made by NSA Information Assurance