Skip to content

Commit

Permalink
Removes Xenial-specific references from the docs
Browse files Browse the repository at this point in the history
Follow-up to freedomofpress/securedrop#5911,
in which we removed Xenial-specific codepaths from the code repo.
Here, we do the same throughout the documentation. There are notable
exceptions:

  * https://docs.securedrop.org/en/stable/v3_services.html
  * https://docs.securedrop.org/en/stable/upgrade/focal_migration.html

Those URLs remain untouched by these changes. We can follow up soon with
removal for those, too, but it seems helpful to maintain a bit longer.
  • Loading branch information
Conor Schaefer committed May 4, 2021
1 parent 6573c22 commit d66daa3
Show file tree
Hide file tree
Showing 9 changed files with 91 additions and 134 deletions.
9 changes: 0 additions & 9 deletions docs/admin.rst
Expand Up @@ -437,15 +437,6 @@ VERSION_CODENAME is "focal"
sudo unattended-upgrades
VERSION_CODENAME is "xenial"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code:: sh
sudo cron-apt -i -s
.. note::

Depending on the nature of the update (e.g., if the ``tor`` package is
Expand Down
4 changes: 2 additions & 2 deletions docs/development/contributing.rst
Expand Up @@ -154,8 +154,8 @@ are covered in :doc:`detailed documentation <release_management>`.
If you are a `Debian developer <https://www.debian.org/devel/>`__ you can help
improve packaging and the release process:

* `Building SecureDrop application and OSSEC packages <https://github.com/freedomofpress/securedrop/tree/develop/molecule/builder-xenial>`__ and `pending bugs and tasks <https://github.com/freedomofpress/securedrop/issues?q=is%3Aissue+is%3Aopen+package+label%3A%22goals%3A+packaging%22>`__
* Building `grsecurity kernels <https://github.com/freedomofpress/ansible-role-grsecurity>`__ and `pending bugs and tasks <https://github.com/freedomofpress/ansible-role-grsecurity/issues>`__
* `Building SecureDrop application and OSSEC packages <https://github.com/freedomofpress/securedrop/tree/develop/molecule/builder-focal>`__ and `pending bugs and tasks <https://github.com/freedomofpress/securedrop/issues?q=is%3Aissue+is%3Aopen+package+label%3A%22goals%3A+packaging%22>`__
* Building `grsecurity kernels <https://github.com/freedomofpress/kernel-builder>`__ and `pending bugs and tasks <https://github.com/freedomofpress/kernel-builder/issues>`__



Expand Down
3 changes: 1 addition & 2 deletions docs/development/dockerbuildmaint.rst
Expand Up @@ -5,8 +5,7 @@ Build container
We use a Docker build container to build our debian packages for SecureDrop (via ``make build-debs``
in the ``securedrop`` Github repository root directory). We keep images of this our container in a
Docker repository at **quay.io/freedomofpress**. The images are organized by Ubuntu release
version. For instance, you can find the images for Xenial at
**quay.io/freedomofpress/sd-docker-builder-xenial** and, for Focal, at
version. For instance, you can find the images for Focal at
**quay.io/freedomofpress/sd-docker-builder-focal**.

Maintaining images of our build container for each release is our way of recording the exact version
Expand Down
109 changes: 43 additions & 66 deletions docs/development/qubes_staging.rst
@@ -1,9 +1,9 @@
Virtual Environments: Using Qubes
=================================

SecureDrop currently uses Ubuntu Xenial as its server OS, and Focal support is under
development. The instructions below cover setting up a SecureDrop staging environment
using either Xenial or Focal under Qubes.
SecureDrop currently uses Ubuntu Focal as its server OS.
The instructions below cover setting up a SecureDrop staging environment
using Focal under Qubes.

It is assumed that you have an up-to-date Qubes installation on a compatible
laptop, with at least 16GB RAM and 60GB free disk space. The SecureDrop server VMs
Expand All @@ -12,21 +12,18 @@ accurately for Tor to start and hidden services to be available.

Overview
--------
.. note:: Throughout the following instructions, ``$SERVER_OS`` will refer to your choice
of either ``xenial`` or ``focal``.

Follow the the Qubes platform instructions in :doc:`setup_development`
to create a Debian 10 ``sd-dev`` Standalone VM. Once done, we'll create three new
Standalone (HVM) Qubes VMs for use with staging:

- ``sd-staging-base-$SERVER_OS``, a base VM for cloning reusable staging VMs
- ``sd-staging-app-base-$SERVER_OS``, a base VM for the *SecureDrop Application Server*
- ``sd-staging-mon-base-$SERVER_OS``, a base VM for the *SecureDrop Monitor Server*
- ``sd-staging-base-focal``, a base VM for cloning reusable staging VMs
- ``sd-staging-app-base-focal``, a base VM for the *SecureDrop Application Server*
- ``sd-staging-mon-base-focal``, a base VM for the *SecureDrop Monitor Server*

Download Ubuntu server ISO
----------------------------

On ``sd-dev``, download the latest Ubuntu server ISO for either Xenial or Focal,
On ``sd-dev``, download the latest Ubuntu server ISO for Focal,
along with corresponding checksum and signature files. See the
:ref:`hardware installation docs <download_ubuntu>`
for detailed instructions. If you opt for the command line instructions, omit
Expand All @@ -42,11 +39,11 @@ In ``dom0``, do the following:

.. code:: sh
qvm-create sd-staging-base-$SERVER_OS --class StandaloneVM --property virt_mode=hvm --label green
qvm-volume extend sd-staging-base-$SERVER_OS:root 20g
qvm-prefs sd-staging-base-$SERVER_OS memory 2000
qvm-prefs sd-staging-base-$SERVER_OS maxmem 2000
qvm-prefs sd-staging-base-$SERVER_OS kernel ''
qvm-create sd-staging-base-focal --class StandaloneVM --property virt_mode=hvm --label green
qvm-volume extend sd-staging-base-focal:root 20g
qvm-prefs sd-staging-base-focal memory 2000
qvm-prefs sd-staging-base-focal maxmem 2000
qvm-prefs sd-staging-base-focal kernel ''
The commands above will create a new StandaloneVM, expand the storage space
and memory available to it, as well as disable the integrated kernel support.
Expand All @@ -59,7 +56,7 @@ In ``dom0``:

.. code:: sh
qvm-start sd-staging-base-$SERVER_OS --cdrom=sd-dev:$ISO_PATH
qvm-start sd-staging-base-focal --cdrom=sd-dev:$ISO_PATH
where ``ISO_PATH`` is the full path to the Ubuntu ISO previously downloaded on ``sd-dev``.

Expand All @@ -68,11 +65,11 @@ Next, choose **Install Ubuntu**.
For the most part, the install process matches the
:ref:`hardware install flow <install_ubuntu>`, with a few exceptions:

- Server IP address: use value returned by ``qvm-prefs sd-staging-base-$SERVER_OS ip``,
- Server IP address: use value returned by ``qvm-prefs sd-staging-base-focal ip``,
with ``/24`` netmask suffix
- Gateway: use value returned by ``qvm-prefs sd-staging-base-$SERVER_OS visible_gateway``
- Gateway: use value returned by ``qvm-prefs sd-staging-base-focal visible_gateway``
- For DNS, use Qubes's DNS servers: ``10.139.1.1`` and ``10.139.1.2``.
- Hostname: ``sd-staging-base-$SERVER_OS``
- Hostname: ``sd-staging-base-focal``
- Domain name should be left blank

Make sure to configure LVM and use **Virtual disk 1 (xvda 20.0GB Xen Virtual Block device)**
Expand All @@ -86,15 +83,15 @@ Once installation is done, let the machine shut down and then restart it with

.. code:: sh
qvm-start sd-staging-base-$SERVER_OS
qvm-start sd-staging-base-focal
in ``dom0``. You should get a login prompt.

Initial VM configuration
------------------------

Before cloning this machine, we'll update software to reduce provisioning time
on the staging VMs. In the new ``sd-staging-base-$SERVER_OS`` VM's console, do:
on the staging VMs. In the new ``sd-staging-base-focal`` VM's console, do:

.. code:: sh
Expand Down Expand Up @@ -122,7 +119,7 @@ to
GRUB_CMDLINE_LINUX="net.ifnames=0 biosdevname=0"
When initial configuration is done, run ``qvm-shutdown sd-staging-base-$SERVER_OS`` to shut it down.
When initial configuration is done, run ``qvm-shutdown sd-staging-base-focal`` to shut it down.

Clone VMs
---------
Expand All @@ -134,31 +131,28 @@ documented below. Run the following in ``dom0``:

.. code:: sh
qvm-clone sd-staging-base-$SERVER_OS sd-staging-app-base-$SERVER_OS
qvm-clone sd-staging-base-$SERVER_OS sd-staging-mon-base-$SERVER_OS
qvm-prefs sd-staging-app-base-$SERVER_OS ip 10.137.0.50
qvm-prefs sd-staging-mon-base-$SERVER_OS ip 10.137.0.51
qvm-tags sd-staging-app-base-$SERVER_OS add created-by-sd-dev
qvm-tags sd-staging-mon-base-$SERVER_OS add created-by-sd-dev
qvm-clone sd-staging-base-focal sd-staging-app-base-focal
qvm-clone sd-staging-base-focal sd-staging-mon-base-focal
qvm-prefs sd-staging-app-base-focal ip 10.137.0.50
qvm-prefs sd-staging-mon-base-focal ip 10.137.0.51
qvm-tags sd-staging-app-base-focal add created-by-sd-dev
qvm-tags sd-staging-mon-base-focal add created-by-sd-dev
Now start both new VMs:

.. code:: sh
qvm-start sd-staging-app-base-$SERVER_OS
qvm-start sd-staging-mon-base-$SERVER_OS
qvm-start sd-staging-app-base-focal
qvm-start sd-staging-mon-base-focal
On the consoles which eventually appear, you should be able to log in with
``sdadmin/securedrop``.

Configure cloned VMs
~~~~~~~~~~~~~~~~~~~~

We'll need to fix each machine's idea of its own IP. The config location differs
on your OS choice:

- **Xenial:** In the console for each machine, edit ``/etc/network/interfaces`` to update the ``address`` line with the machine's IP.
- **Focal:** In the console for each machine, edit ``/etc/netplan/00-installer-config.yaml`` to update the ``addresses`` entry with the machine's IP.
We'll need to fix each machine's idea of its own IP. In the console for each machine,
edit ``/etc/netplan/00-installer-config.yaml`` to update the ``addresses`` entry with the machine's IP.

Edit ``/etc/hosts`` on each host to include the hostname and IP for itself.
Use ``app-staging`` and ``mon-staging`` as appropriate.
Expand Down Expand Up @@ -245,8 +239,9 @@ to set up the development environment.

Once finished, build the Debian packages for installation on the staging VMs:

- **Xenial:** use the command ``make build-debs``
- **Focal:** use the command ``make build-debs-focal``
.. code:: sh
make build-debs
Managing Qubes RPC for Admin API capability
-------------------------------------------
Expand Down Expand Up @@ -293,25 +288,26 @@ Creating staging instance
After creating the StandaloneVMs as described above:

* ``sd-dev``
* ``sd-staging-base-$SERVER_OS``
* ``sd-staging-app-base-$SERVER_OS``
* ``sd-staging-mon-base-$SERVER_OS``
* ``sd-staging-base-focal``
* ``sd-staging-app-base-focal``
* ``sd-staging-mon-base-focal``

And after building the SecureDrop .debs, we can finally provision the staging
environment:

- **Xenial:** run the command ``make staging``
- **Focal:** run the command ``make staging-focal``
.. code:: sh
make staging
The commands invoke the appropriate Molecule scenario for your choice of ``$SERVER_OS``.
The commands invoke the appropriate Molecule scenario for your choice of ``focal``.
You can also run constituent Molecule actions directly, rather than using
the Makefile target:

.. code:: sh
molecule create -s qubes-staging-$SERVER_OS
molecule converge -s qubes-staging-$SERVER_OS
molecule test -s qubes-staging-$SERVER_OS
molecule create -s qubes-staging-focal
molecule converge -s qubes-staging-focal
molecule test -s qubes-staging-focal
That's it. You should now have a running, configured SecureDrop staging instance
running on your Qubes machine. For day-to-day operation, you should run
Expand All @@ -320,7 +316,7 @@ to provision staging VMs on-demand. To remove the staging instance, use the Mole

.. code:: sh
molecule destroy -s qubes-staging-$SERVER_OS
molecule destroy -s qubes-staging-focal
Accessing the Journalist Interface (Staging) in Whonix-based VMs
----------------------------------------------------------------
Expand Down Expand Up @@ -383,22 +379,3 @@ At this point, you should be able to access the *Journalist Interface*

Note that you will have to replace the ``app-journalist.auth_private`` file
and reload Tor on the Whonix gateway every time you rebuild the staging environment.

Switching between Xenial and Focal
----------------------------------

Both environments may be set up on your Qubes workstation, but they cannot be run
simultaneously. To switch between them:

- Use the appropriate ``molecule destroy`` command to bring down the active environment.
- Remove SSH known host entries for the servers with the commands:

.. code:: sh
ssh-keygen -f "/home/user/.ssh/known_hosts" -R "10.137.0.50"
ssh-keygen -f "/home/user/.ssh/known_hosts" -R "10.137.0.51"
- Build environment-specific packages first if necessary with ``make build-debs``
or ``make build-debs-focal``.
- Run ``make staging`` or ``make staging-focal`` as appropriate.
2 changes: 1 addition & 1 deletion docs/development/release_management.rst
Expand Up @@ -255,7 +255,7 @@ Release Process
#. In a clone of the private
`securedrop-debian-packages-lfs <https://github.com/freedomofpress/securedrop-debian-packages-lfs>`_
repository, create a branch from ``main`` called ``release``.
#. In your local branch, commit the built packages to the ``core/xenial``
#. In your local branch, commit the built packages to the ``core/focal``
directory.

* If the release includes a Tor update, make sure to include the
Expand Down
8 changes: 4 additions & 4 deletions docs/development/setup_development.rst
Expand Up @@ -195,17 +195,17 @@ Ansible on your development workstation.
Ubuntu or Debian GNU/Linux
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. note:: Tested on: Ubuntu 16.04 and Debian GNU/Linux stretch
.. note:: Tested on: Debian GNU/Linux 10 Buster

.. code:: sh
sudo apt-get update
sudo apt-get install -y build-essential libssl-dev libffi-dev python3-dev \
dpkg-dev git linux-headers-$(uname -r) virtualbox
We recommend using the latest stable version of Vagrant, ``1.8.5`` at the time
of this writing, which might be newer than what is in your distro's package
repositories. Older versions of Vagrant has been known to cause problems
We recommend using the most recent version of Vagrant available in your distro's
package repositories. For Debian Stable, that's ``2.2.3`` at the time
of this writing. Older versions of Vagrant has been known to cause problems
(`GitHub #932`_, `GitHub #1381`_). If ``apt-cache policy vagrant`` says your
candidate version is not at least 1.8.5, you should download the current version
from the `Vagrant Downloads page`_ and then install it.
Expand Down
67 changes: 33 additions & 34 deletions docs/development/testing_continuous_integration.rst
Expand Up @@ -89,7 +89,6 @@ Run ``make help`` to see the full list of CI commands in the Makefile:
Makefile for developing and testing SecureDrop.
Subcommands:
ci-go Creates, provisions, tests, and destroys GCE host for testing staging environment.
ci-go-xenial Creates, provisions, tests, and destroys GCE host for testing staging environment under xenial.
ci-lint Runs linting in linting container.
ci-teardown Destroys GCE host for testing staging environment.
Expand Down Expand Up @@ -117,48 +116,48 @@ of a commit to apply to a branch in order disable the deletion for the Focal sta

.. code:: Diff
diff --git a/.circleci/config.yml b/.circleci/config.yml
index 4a9b0bd4c..d9aea01b8 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -354,13 +354,6 @@ jobs:
BASE_OS=focal make ci-go
no_output_timeout: 35m
- - run:
- name: Ensure environment torn down
- # Always report true, since env should will destroyed already
- # if all tests passed.
- command: make ci-teardown || true
- when: always
-
- store_test_results:
path: ~/sd/junit
diff --git a/devops/gce-nested/ci-go.sh b/devops/gce-nested/ci-go.sh
index 850324ecc..776120df4 100755
--- a/devops/gce-nested/ci-go.sh
+++ b/devops/gce-nested/ci-go.sh
@@ -16,4 +16,3 @@ export BASE_OS="${BASE_OS:-xenial}"
diff --git a/.circleci/config.yml b/.circleci/config.yml
index 4d61769f1..af74672bc 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -251,13 +251,6 @@ jobs:
make ci-go
no_output_timeout: 35m
- - run:
- name: Ensure environment torn down
- # Always report true, since env should will destroyed already
- # if all tests passed.
- command: make ci-teardown || true
- when: always
-
- store_test_results:
path: ~/sd/junit
diff --git a/devops/gce-nested/ci-go.sh b/devops/gce-nested/ci-go.sh
index ff80aa107..65bbcd7b9 100755
--- a/devops/gce-nested/ci-go.sh
+++ b/devops/gce-nested/ci-go.sh
@@ -16,4 +16,3 @@ export BASE_OS="${BASE_OS:-focal}"
./devops/gce-nested/gce-start.sh
./devops/gce-nested/gce-runner.sh
-./devops/gce-nested/gce-stop.sh
diff --git a/devops/scripts/create-staging-env b/devops/scripts/create-staging-env
index 3b9a2c7f8..df2ccfe3d 100755
--- a/devops/scripts/create-staging-env
+++ b/devops/scripts/create-staging-env
@@ -33,7 +33,7 @@ printf "Creating staging environment via '%s'...\\n" "${securedrop_staging_scena
-./devops/gce-nested/gce-stop.sh
diff --git a/devops/scripts/create-staging-env b/devops/scripts/create-staging-env
index 8b296be94..df8a4d674 100755
--- a/devops/scripts/create-staging-env
+++ b/devops/scripts/create-staging-env
@@ -32,7 +32,7 @@ printf "Creating staging environment via '%s'...\\n" "${securedrop_staging_scena
virtualenv_bootstrap
# Are we in CI? Then lets do full testing post install!
if [ "$USER" = "sdci" ]; then
- molecule test -s "${securedrop_staging_scenario}"
+ molecule test --destroy=never -s "${securedrop_staging_scenario}"
- molecule test -s "${securedrop_staging_scenario}"
+ molecule test --destroy=never -s "${securedrop_staging_scenario}"
else
molecule "${MOLECULE_ACTION:-converge}" -s "${securedrop_staging_scenario}" "${EXTRA_ANSIBLE_ARGS[@]}"
molecule "${MOLECULE_ACTION:-converge}" -s "${securedrop_staging_scenario}" "${EXTRA_ANSIBLE_ARGS[@]}"
fi
Once that commit is pushed, run the appropriate ``staging-test-with-rebase`` job
Once that commit is pushed, run the ``staging-test-with-rebase`` job
with ssh using with CircleCI. Once logged into that container, you can ssh into the
Google Compute host:

Expand Down
5 changes: 0 additions & 5 deletions docs/development/upgrade_testing.rst
Expand Up @@ -29,11 +29,6 @@ https://apt-test.freedom.press/. Both options are described below.
Upgrade testing using locally-built packages
--------------------------------------------

.. note::
As of ``0.12.1``, the default platform for upgrade testing
boxes is Ubuntu Xenial 16.04. We no longer support upgrade boxes
based on Ubuntu Trusty 14.04.

First, build the app code packages and create the environment:

.. code:: sh
Expand Down

0 comments on commit d66daa3

Please sign in to comment.