Skip to content

Testing Requirements before Release

Jump to bottom
Geoff Nelson edited this page Jul 17, 2019 · 169 revisions

Manual Release Testing Requirements:

Manual Release Testing Steps:

Navbar

Links

  • Login/Logout
  • Register
  • My Dashboard
    • Make sure info display correctly
  • About
    • Make sure info display correctly
  • Global analysis popover
    • Hover over icon, tooltip should appear
    • Should open when icon is clicked
    • Click outside of popover to close
  • User Name redirect to profile (should edit profile)

Landing Page

  • Intro Section
    • Instance name is correct in header
    • Markdown looks correct
  • Video Carousel
    • Should be able to nav through videos
  • About Section
    • Markdown looks correct
  • Files Chart
    • Able to switch attributes
    • Links to the filtered user/files
  • Tools List
    • Displays correctly
    • Workflow's link work
  • Twitter Feed
    • Displays correctly

Cross-dataset file browser

On the dashboard, click on the "Browse Files" button.

  • Facets
    • Search: Enter a string (like "dna") and with each keypress the matching terms should be displayed, bolded in the facet names.
    • Sort: When a facet is folded down, you should be able to sort the values either numerically or alphabetically.
    • Apply: Click on a checkbox and the facet should be applied, and the url should update
    • Combining: Multiple values within a facet should be ORed. Distinct facets should be ANDed.
  • Download
    • Without adding any filters, click on "Download as CSV": You should get a large CSV: Make sure a URL is the first column, and that the number of rows is not a round number. (We want to be sure results aren't truncated.)
    • Add facet filter, download again, and make sure you have a smaller number of rows.
  • Grid
    • Click on a header to sort, and click again to reverse sort.
    • Click on the download icon and a download should begin.
    • Click on the document icon and you should be taken to the dataset.

Data set import

Tabular Metadata tab

  • Ensure this tab is selected by default
  • Tabular file + custom data upload
  • Confirm that example tabular metadata file can be downloaded
  • Confirm table preview displays correctly after selecting a file
  • Confirm Metadata Import displays properly
    • The Required fields (Title, Sample Identifier, Data File Column, Species Column) has default selection
    • Check form validation by removing a required field
  • Under COMPLETE SUBMISSION, change the selection for Does the Data File Column refer to local files? to Yes. Verify that the Check Data Files button is performing properly.
  • Verify that navigating away from file upload tab will trigger alert (once meta data file has been selected)
  • Verify that if one uploads a subset of the files referenced in a Tabular file that:
    • a: The subset of referenced files are automatically imported into Refinery
    • b: The rest of the files whose data files weren't manually uploaded have lightning bolts next to them (which indicates their non-imported status)
  • After submission, verify the automatic redirect to file browser
  • After redirect to file browser, verify that the original tabular metadata file is available for download on the "Details" tab
  • After redirect to file browser, verify that uploaded data sets are shown properly on the Dashboard
  • Hover over the download icons for the uploaded files: Verify that references to remote files from the tabular file still point to the original url unless the Import Now box under ADVANCED was checked during import (If Import Now was checked before the DataSet upload, the download icons should point to files local to the test instance).

ISA-Tab Metadata tab

Upload an ISA archive file referencing data files by URL (examples)

  • After upload, verify the automatic redirect to file browser
  • After redirect to file browser, verify that uploaded data sets are shown properly on the Dashboard

Upload an ISA archive file referencing data files directly (example)

  • Upload corresponding data files (rfc94.txt, rfc111.txt, rfc134.txt)
  • After upload, verify the automatic redirect to file browser
  • After redirect to file browser, verify that uploaded data sets are shown properly on the Dashboard
  • After redirect to file browser, verify that the original ISATab file is available for download on the "Details" tab

Upload an ISA archive file by url: rfc-test.zip

  • After upload, verify the automatic redirect to file browser
  • After redirect to file browser, verify that uploaded data sets are shown properly on the Dashboard
  • After redirect to file browser, verify that the original ISATab file is available for download on the "Details" tab

Upload a non-ISA archive file through this interface by url or file selection, and make sure a reasonable error message is displayed.

File Browser

Metadata Revisions

  • UI-based revison: click on the Modify button in the top right of the FileBrowser and ensure that you are able to double click on the available cells for editing and make some changes.
  • Re-Upload Metadata Edited Outside of Refinery: Upload a DataSet, and then click on the Upload button in the top right of the Filebrowser to upload an edited version of the same DataSet.
    • Be sure to check that any edits made outside of the system show in the fileBrowser after the revised submission happens.
    • Be sure to check that any uploaded datafiles persist when that revised metadata file referenceces them by the same name.
  • ** Download/Add/Remove Data Files **
    • Download a data file (down arrow in left column).
    • Open the drop down menu and remove data file.
    • Open the drop down menu and add the data file.

Files Tab

  1. Select/Deselect attribute filters
  2. Table Config (wrench button): Select/Deselect different attributes to show
  3. Reload page to see if the columns and facets are updated

Analyses Tab

  1. Ensure new analyses are showing up for datasets and refreshing (30secs)
  2. Cancel an analysis (Ensure it responds and page refreshes)
  3. Ensure global analyses status shows any new analyses
    • Running analyses should show the three different stages
    • Completed analyses will show colored completion (success/failure) state.
    • Click on analysis link will redirect you to file browser - analysis filtered & close popover
  4. Test collapse/expand for running analyses

Visualizations Tab

  1. Ensure new visualizations are showing up
  2. Relaunch a previously launched visualization
  3. View a Running visualization
  4. Ensure that a VisualizationTool's url relaunches said Tool after its container has been purged when navigated to

Details Tab

  1. Fields are populated correctly
  2. Able to edit and save fields
  3. META DATA download link
  4. Import shared ISA-Tab-based data set into user space
  5. Sharing: Correct owner is showing with profile link & groups with permission icons and link

Provenance Tab

  1. Zoom in/out on graph
  2. Drag view around
  3. Select/Deselect nodes
  4. Are the layer/analysis/analysis group/workflow buttons working?
  5. Attributes Filter

Tool launching:

Sample Datasets:

Galaxy Workflows:

NOTE: we currently have duplicates of some of our workflows to accommodate for different genome builds. Please follow the steps below for the duplicated workflows, but use one of the mentioned Sample Datasets for the respective genome build.

  • FASTQC

    • Ensure hg19-metadata-s3.txt is uploaded and navigate to its FileBrowser page.
    • Open tool panel and select FASTQC
    • Select multiple FastQ files to add
    • Hover over input groups column and check popover content
    • Check tool input group displays correct file names
    • Open/Collapse Description panel, Tool Input Control panel
    • Launch Analysis
    • After a successful Analysis, ensure that you get 2 Derived Data files (.zip & .txt) for each input FASTQ that was analyzed

  • RNA-seq (SE) Quantification

    • Ensure that one of the Sample Datasets corresponding to the species of the Workflow to execute has been uploaded and navigate to it in the FileBrowser
    • Open tool panel and select RNA-seq (SE) Quantification - <species>
    • Select multiple FastQ files to add
    • Hover over input groups column and check popover content
    • Check tool input group displays correct file names
    • Open/Collapse Description panel, Tool Input Control panel
    • Launch Analysis
    • After a successful Analysis, ensure that you get 3 Derived Data files (2x .tabular, 1x .bam) for each input FASTQ that was analyzed

  • RNA-seq (PE) Quantification

    • Ensure that one of the Sample Datasets corresponding to the species of the Workflow to execute has been uploaded and navigate to it in the FileBrowser
    • Open tool panel and select RNA-seq (PE) Quantification - <species>
    • Select pairs of FastQ files to add
    • Nav to a new group both through selection popover and tool input control
    • Hover over input groups column and check popover content
    • Toggle tool input group "Input Details" and check pair content
    • Test out remove and remove all functionality
    • Open/Collapse Description panel, Tool Input Control panel
    • Launch Analysis
    • After a successful Analysis, ensure that you get 3 Derived Data files (2x .tabular, 1x .bam) for each input FASTQ pair that was analyzed

  • ChIP-seq Peak Calling

    • Ensure that one of the Sample Datasets corresponding to the species of the Workflow to execute has been uploaded and navigate to it in the FileBrowser
    • Open tool panel and select ChIP-seq Peak Calling - <species>
    • Select pairs of FastQ files to add
    • Nav to a new group both through selection popover and tool input control
    • Hover over input groups column and check popover content
    • Toggle tool input group "Input Details" and check pair content
    • Test out remove and remove all functionality
    • Open/Collapse Description panel, Tool Input Control panel
    • Launch Analysis
    • After a successful Analysis, ensure that you get 3 Derived Data files (.pdf, .bedgraph, .bigwig) for each input FASTQ pair that was analyzed

Visualizations:

  • IGV

    • Navigate to a DataSet with successful RNA-Seq Analyses run upon it
    • Open tool panel and select IGV
    • Select some .bam files that have been imported into Refinery and launch IGV
    • Ensure that the bams can be visualized in IGV at the most zoomed-in level.
    • Try some examples of other filetypes supported by IGV.js. These examples are available at http://data.cloud.refinery-platform.org/?prefix=data/igv-sample/. They are: seg, gff, wig, bed, bedgraph, and vcf.

Dashboard

Data Set Card

  • Main Links: List & Upload

  • Filters Note: user can intermix owned, one group, & public filters, but only one group can be chosen at a time.

    • Primary Group - set/change, starred in groups filter, ?help popover works
    • Owned Filter
    • Groups Filter (it should sync with primary group filter or override it with a new group selection)
    • Public

  • Search: Min letters, different combos

    • Good examples: RNA-seq, ChIP-seq, Human, Mus musculus, ESC, Oct4, H3K27ac

  • Ensure data sets are displaying correctly (correct perms)

    • Correct icon perms owner & public
    • Correct modification date
    • Data set functions for owners
      • Delete
      • Share
      • Transfer

    History Card

    • Check events are listed for you (currently only showing user's events).
      • Data sets created
      • Tools launched
    • Owner, file browser, and tool hyperlinks work
    • Dates are accurate and in newest first order

    Collaboration Card

    • Groups are showing correctly
      • Manager listed
      • Users are listed with hyperlinks
    • Add groups
    • For groups you manager
      • Add users
      • Resend invitation
      • Revoke invitation
      • Remove User
      • Empty group and delete group

Administrator Testing Requirements

  • Make sure email notifications go out when running on AWS (For user registration and Analysis runs)
  • Ensure that for successful Analysis runs the following Galaxy artifacts are removed: Histories, Libraries.
  • Ensure that for cancelled/deleted Analyses the following Galaxy artifacts are removed: Histories, Libraries.
  • One tester with admin privs should confirm that new user registration works.
  • Provision an instance from scratch both locally using Vagrant and on AWS (makes sure no errors are reported)
  • Check that the following is present:
    • AnonymousUser object in the database
    • Guest account (disabled on AWS and enabled locally by default)
  • Create a Galaxy connector and a workflow engine, and import workflows from a Galaxy instance (Refinery test workflow, FastQC, ChIP-seq and RNA-seq)

Home

Administration

Development

Testing

Troubleshooting

Miscellaneous

Clone this wiki locally