1.4.1 Test Plan
Kevin O'Gorman edited this page Jun 25, 2020
·
2 revisions
We're only testing on VMs and NUCs. The only expected change in the release is the behaviour of the securedrop-admin utility during configuration and installation, so the 1.4.1-specific changes section should be completed first.
For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report. Feel free to edit this message to update the plan as appropriate.
If you have submitted a QA report already for a 1.4.1 release candidate with successful Basic Server Testing and [[Application Acceptance Testing]], then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the 1.4.1-specific changes as well as changes since the previous release candidate.
- Install target:
- Tails version:
- Test Scenario:
- SSH over Tor:
- Onion service version:
- Release candidate:
- General notes:
- I can access both the source and journalist interfaces
- I can SSH into both machines over Tor
- AppArmor is loaded on app
- 0 processes are running unconfined
- AppArmor is loaded on mon
- 0 processes are running unconfined
- Both servers are running grsec kernels
- iptables rules loaded
- OSSEC emails begin to flow after install
- OSSEC emails are decrypted to correct key and I am able to decrypt them
- QA Matrix checks pass
- Can successfully add admin user and login
- I have backed up and successfully restored the app server following the backup documentation
- If doing upgrade testing, make a backup on 1.4.0 and restore this backup on 1.4.1
- "Send Test OSSEC Alert" button in the journalist triggers an OSSEC alert and an email is sent
- Can successfully add journalist account with HOTP authentication
- JS warning bar does not appear when using Security Slider high
- JS warning bar does appear when using Security Slider Low
- On generate page, refreshing codename produces a new 7-word codename
- On submit page, empty submissions produce flashed message
- On submit page, short message submitted successfully
- On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
- On submit page, file less than 500 MB submitted successfully
- Nonexistent codename cannot log in
- Empty codename cannot log in
- Legitimate codename can log in
- Returning user can view journalist replies - need to log into journalist interface to test
- Can log in with 2FA tokens
- incorrect password cannot log in
- invalid 2fa token cannot log in
- 2fa immediate reuse cannot log in
- Journalist account with HOTP can log in
- Filter by codename works
- Starring and unstarring works
- Click select all selects all submissions
- Selecting all and clicking "Download" works
- You can submit a reply and a flashed message and new row appears
- You cannot submit an empty reply
- Clicking "Delete Source And Submissions" and the source and docs are deleted
- You can click on a document and successfully decrypt using application private key
After updating to this release candidate and running securedrop-admin tailsconfig
- The Updater GUI appears on boot
- Updating occurs without issue
- On an Admin Workstation, check out the latest RC tag, run
./securedrop-admin setup. - Run
./securedrop-admin install:- Command exits immediately with message:
Please run "securedrop-admin sdconfig"
- Command exits immediately with message:
- Run
./securedrop-admin sdconfig, using instance-appropriate settings except for the v2 and v3 boolean options - attempt to choosenofor both of those.- Message is displayed informing user that since they chose not to enable v2 they must enable v3
- Choose
yesfor v3 and exit. Run./securedrop-admin install- Command starts successfully, user is prompted for server admin password
- Hit Ctrl-C to exit without installing. Run
./securedrop-admin sdconfig, again this time choosingyesfor both v2 and v3. Then run./securedrop-admin install- Command starts successfully, user is prompted for server admin password
- Hit Ctrl-C to exit without installing. Run
./securedrop-admin sdconfig, again this time choosingyesv2 andnofor v3. Then run./securedrop-admin install- Command starts successfully, user is prompted for server admin password
- Hit Ctrl-C to exit without installing. Edit the file
~/Persistent/securedrop/install_files/ansible-base/group-vars/all/site-specific, changing the value forv2_onion_servicesfromtruetokitten. Runsecuredrop-admin install- Command fails with an error message
must be either yes or no
- Command fails with an error message
- Run
./securedrop-admin sdconfig, changingkittentoyesfor v2 and leaving v3 asno. Then run./securedrop-admin install- Command starts successfully, user is prompted for server admin password
- Enter server admin password and complete installation. Then run
./securedrop-admin tailsconfig- Installation completes successfully
- v2 services are enabled, v3 services are disabled, and SSH and desktop shortcuts work as expected.
- On an Admin Workstation, check out the latest RC tag.
- Edit the file
~/Persistent/securedrop/install_files/ansible-base/group-vars/all/site-specific, changing the value forv2_onion_servicesfromtruetokitten. Runsecuredrop-admin install- Command fails with an error message
must be either yes or no
- Command fails with an error message
- Run
./securedrop-admin sdconfig, changingkittentoyesfor v2, leaving v3 asno, and adding an extra language option. Then run./securedrop-admin install- Command starts successfully, user is prompted for server admin password
- Enter server admin password and complete installation:
- Installation completes successfully
- v2 services are enabled, v3 services are disabled, and SSH and desktop shortcuts work as expected.
- extra language option is available in dropdown on Source and Journalist Interfaces.
- Ensure the builder image is up-to-date on release day
These tests should be performed the day of release prior to live debian packages on apt.freedom.press.
- Install or upgrade occurs without error
- Source interface is available and version string indicates it is 1.4.1
- A message can be successfully submitted
- The updater GUI appears on boot
- The update successfully occurs to 1.4.1
- After reboot, updater GUI no longer appears