README.Rmd

---
output: github_document
---

<!-- README.md is generated from README.Rmd. Please edit that file -->

```{r, echo = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  message = FALSE,
  warning = FALSE,
  comment = "#",
  fig.path = "tools/README-",
  fig.cap=""
)
```
   
   
[![Build Status](https://api.travis-ci.org/kassambara/fastqcr.png)](https://travis-ci.org/kassambara/fastqcr)
[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/fastqcr)](https://cran.r-project.org/package=fastqcr)
[![Downloads](https://cranlogs.r-pkg.org/badges/fastqcr)](https://cran.r-project.org/package=fastqcr)
[![Total Downloads](https://cranlogs.r-pkg.org/badges/grand-total/fastqcr?color=orange)](https://cranlogs.r-pkg.org/badges/grand-total/fastqcr)
      
      
<br/>  

<style>
.error,.notice,.warning,.success,.question{height:auto;padding:10px 10px 10px 40px;margin:5px auto 15px;line-height:20px;border:1px solid #FFF;border-radius:4px;position:relative;display:block;text-align:left}.question{background-color:#DAEEF8;border-color:#BDE9F2}.notice{background-color:#F0F0F0;border-color:#E2E2E2}.warning{background-color:#FDF9E4;border-color:#FBECCD}.error{background-color:#F3DFDF;border-color:#ECCDD2}.success{background-color:#E0F1D9;border-color:#D7EAC7}.block,.medium-block,.small-block{border:1px solid #CCC;border-top:2px solid #366393;border-bottom:1px solid #99B1CB;background:#F2F8FF;padding:10px}.block{width:auto;margin-top:10px;margin-bottom:10px}img{background-color:#fff;background-color:#FFF;border-radius:3px;border:1px solid #CCC;box-shadow:2px 2px 12px -5px #999;margin:0 5px;margin-bottom:5px;padding:5px;text-align:center}
</style>
    
    
    
# fastqcr: Quality Control of Sequencing Data
     
     
The FastQC, written by Simon Andrews at the Babraham Institute, is the most widely used sequence quality assessment tool for evaluating the raw reads from high throughput sequencing data.    
    
It produces, for each sample, an html report and a 'zip' file, which contains a file called fastqc_data.txt and summary.txt.   
    
If you have hundreds of samples, you’re not going to open up each HTML page. You need some way of looking at these data in aggregate.   
   
The **fastqcr** R package provides helper functions to easily and automatically parse, aggregate and analyze FastQC reports for large numbers of samples.
   
Additionally, the **fastqcr** package provides a convenient solution for building a multi-QC report and a one-sample FastQC report with the result interpretations. The online documentation is available at: https://rpkgs.datanovia.com/fastqcr/.
    
Examples of QC reports, generated automatically by the **fastqcr** R package, include:   
    
- [Multi-QC report for multiple samples](https://rpkgs.datanovia.com/fastqcr/qc-reports/fastqcr-multi-qc-report.html)
- [One sample QC report (+ interpretation)](https://rpkgs.datanovia.com/fastqcr/qc-reports/sample-qc-report-interpretation.html)
- [One sample QC report (no interpretation)](https://rpkgs.datanovia.com/fastqcr/qc-reports/sample-qc-report-without-interpretation.html)
    
    
![fastqcr logo](tools/fastqcr.png)

    
## Installation and loading
    
       
- fastqcr can be installed from [CRAN](https://cran.r-project.org/package=fastqcr) as follow:
   
```{r, eval = FALSE}
install.packages("fastqcr")
```
     
 
    
- Or, install the latest version from [GitHub](https://github.com/kassambara/fastqcr): 
    
   
```{r, eval = FALSE}
if(!require(devtools)) install.packages("devtools")
devtools::install_github("kassambara/fastqcr")
```

    
    
- Load fastqcr:  
     
     
```{r}
library("fastqcr")
```
    
    
## Quick Start
    
    
```{r, eval = FALSE}
library(fastqcr)

# Aggregating Multiple FastQC Reports into a Data Frame 
#%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

# Demo QC directory containing zipped FASTQC reports
qc.dir <- system.file("fastqc_results", package = "fastqcr")
qc <- qc_aggregate(qc.dir)
qc

# Inspecting QC Problems
#%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

# See which modules failed in the most samples
qc_fails(qc, "module")
# Or, see which samples failed the most
qc_fails(qc, "sample")

# Building Multi QC Reports
#%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
qc_report(qc.dir, result.file = "multi-qc-report" )

# Building One-Sample QC Reports (+ Interpretation)
#%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
qc.file <- system.file("fastqc_results", "S1_fastqc.zip", package = "fastqcr")
qc_report(qc.file, result.file = "one-sample-report",
          interpret = TRUE)
```

    

## Main Functions
   
   
**1) Installing and Running FastQC**
      
      
- **fastqc_install**(): Install the latest version of FastQC tool on Unix systems (MAC OSX and Linux)
   
- **fastqc**(): Run the FastQC tool from R.
     
         
     
**2) Aggregating and Summarizing Multiple FastQC Reports**
       
       
- **qc <- qc_aggregate**(): Aggregate multiple FastQC reports into a data frame.
   
- **summary**(qc): Generates a summary of qc_aggregate. 
  
- **qc_stats**(qc): General statistics of FastQC reports.
       
     
    
**3) Inspecting Problems**
      
      
- **qc_fails**(qc): Displays samples or modules that failed.
   
- **qc_warns**(qc): Displays samples or modules that warned.
    
- **qc_problems**(qc): Union of **qc_fails**() and **qc_warns**(). Display which samples or modules that failed or warned.
    
        
    
**4) Importing and Plotting FastQC Reports**
        
- **qc\_read**(): Read FastQC data into R.
   
- **qc\_plot**(qc): Plot FastQC data
      
          
      
**5) Building One-Sample and Multi-QC Reports**
     
- **qc\_report**(): Create an HTML file containing FastQC reports of one or multiple files. Inputs can be either a directory containing multiple FastQC reports or a single sample FastQC report.
     
          
     
**6) Others**

- **qc\_unzip**(): Unzip all zipped files in the qc.dir directory.
 <br/>
 
    
## Installing FastQC from R
    
You can install automatically the FastQC tool from R as follow:  
   
   
```{r, eval = FALSE}
fastqc_install()
```
    
    
## Running FastQC from R
   
The supported [file formats](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/2%20Basic%20Operations/2.1%20Opening%20a%20sequence%20file.html) by FastQC include:   
   
- FASTQ
- gzip compressed FASTQ
    
    
Suppose that your working directory is organized as follow:
    
- home
    - Documents
        - FASTQ
        
where, FASTQ is the directory containing your FASTQ files, for which you want to perform the quality control check.
   
   
To run FastQC from R, type this:   
    
  
```{r, eval = FALSE}
fastqc(fq.dir = "~/Documents/FASTQ", # FASTQ files directory
       qc.dir = "~/Documents/FASTQC", # Results direcory
       threads = 4                    # Number of threads
       )
```
    
    
## FastQC Reports
    
    
For each sample, FastQC performs a series of tests called *analysis modules*. 
    
    
These modules include:
    
- Basic Statistics,
- Per base sequence quality,
- Per tile sequence quality
- Per sequence quality scores,
- Per base sequence content,
- Per sequence GC content,
- Per base N content,
- Sequence Length Distribution,
- Sequence Duplication Levels,
- Overrepresented sequences,
- Adapter Content
- Kmer content
    
The interpretation of these modules are provided in the official documentation of the [FastQC tool](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/3%20Analysis%20Modules/).
    
   
## Aggregating Reports
   
Here, we provide an R function **qc_aggregate()** to walk the FastQC result directory, find all the FASTQC zipped output folders, read the **fastqc_data.txt** and the **summary.txt** files, and aggregate the information into a data frame.  
    

In the example below, we'll use a demo FastQC output directory available in the fastqcr package. 
   

   
```{r}
library(fastqcr)
# Demo QC dir
qc.dir <- system.file("fastqc_results", package = "fastqcr")
qc.dir
   
# List of files in the directory
list.files(qc.dir)
```
    
    
The demo QC directory contains five zipped folders corresponding to the FastQC output for 5 samples.
   
   
Aggregating FastQC reports: 

   
```{r, eval = FALSE}
qc <- qc_aggregate(qc.dir)
qc
```
    
   
```{r, echo = FALSE}
qc.dir <- "/Users/kassambara/Documents/R/MyPackages/fastqcr/inst/fastqc_results"
qc <- qc_aggregate(qc.dir, progressbar = FALSE)
```

    
The aggregated report looks like this:
   

   
```{r, echo = FALSE}
knitr::kable(dplyr::sample_n(qc, 10))
```
   
   
Column names:
   
- **sample**: sample names
- **module**: fastqc modules
- **status**: fastqc module status for each sample
- **tot.seq**: total sequences (i.e.: the number of reads)
- **seq.length**: sequence length
- **pct.gc**: percentage of GC content
- **pct.dup**: percentage of duplicate reads
    
    

```{block, type = "block"}  
The table shows, for each sample, the names of tested FastQC modules, the status of the test, as well as, some general statistics including the number of reads, the length of reads, the percentage of GC content and the percentage of duplicate reads.
```  
    

Once you have the aggregated data you can use the **dplyr** package to easily inspect modules that failed or warned in samples. For example, the following R code shows samples with warnings and/or failures: 
    
    
```{r}
library(dplyr)
qc %>%
  select(sample, module, status) %>%    
  filter(status %in% c("WARN", "FAIL")) %>%
  arrange(sample)
```
    
    
```{block, type = "success"}
In the next section, we'll describe some easy-to-use functions, available in the **fastqcr** package, for analyzing the aggregated data. 
```   
     
   
   

## Summarizing Reports
   
We start by presenting a summary and general statistics of the aggregated data.
    
    
### QC Summary   
    
- R function: **summary**()     
- Input data: aggregated data from **qc_aggregate**()
      
      
```{r}
# Summary of qc
summary(qc)
```
    
    
Column names:   
    
- *module*: fastqc modules
- *nb_samples*: the number of samples tested
- *nb_pass, nb_fail, nb_warn*: the number of samples that passed, failed and warned, respectively.
- *failed, warned*: the name of samples that failed and warned, respectively.
      
      
```{block, type = "block"} 
The table shows, for each FastQC module, the number and the name of samples that failed or warned.
```
    
    
### General statistics
    
- R function: **qc_stats**()
- Input data: aggregated data from **qc_aggregate**()

   
```{r}
qc_stats(qc)
```
    
    
Column names:   
    
- *pct.dup*: the percentage of duplicate reads,
- *pct.gc*: the percentage of GC content,
- *tot.seq*: total sequences or the number of reads and 
- *seq.length*: sequence length or the length of reads.
    
    
      
```{block, type = "block"} 
The table shows, for each sample, some general statistics such as the total number of reads, the length of reads, the percentage of GC content and the percentage of duplicate reads
```
   
   
    
## Inspecting Problems
   
Once you’ve got this aggregated data, it’s easy to figure out what (if anything) is wrong with your data. 
    
    
**1) R functions**. You can inspect problems per either modules or samples using the following R functions:  
   
- **qc_fails**(qc): Displays samples or modules that failed.
- **qc_warns**(qc): Displays samples or modules that warned.
- **qc_problems**(qc): Union of **qc_fails**() and **qc_warns**(). Display which samples or modules that failed or warned.
    
    
**2) Input data**: aggregated data from **qc_aggregate**()
     
     
**3) Output data**: Returns samples or FastQC modules with failures or warnings. By default, these functions return a compact output format. If you want a stretched format, specify the argument *compact = FALSE*.
   
The format and the interpretation of the outputs depend on the additional argument *element*, which value is one of c("sample", "module").
   
- If **element = "sample"** (default), results are samples with failed and/or warned modules. The results contain the following columns: 
    - sample (sample names), 
    - nb_problems (the number of modules with problems), 
    - module (the name of modules with problems).
- If **element = "module"**, results are modules that failed and/or warned in the most samples. The results contain the following columns: 
    - module (the name of module with problems), 
    - nb_problems (the number of samples with problems),
    - sample (the name of samples with problems)


    
    
### Per Module Problems
    
    
- **Modules that failed in the most samples**: 
    
```{r}
# See which module failed in the most samples
qc_fails(qc, "module")
```
    
   
```{block, type = "success"}
For each module, the number of problems (failures) and the name of samples, that failed, are shown.
```

    
- **Modules that warned in the most samples**:  
       
```{r}
# See which module warned in the most samples
qc_warns(qc, "module")
```
     
     
- **Modules that failed or warned**: Union of qc_fails() and qc_warns()
    
    
```{r}
# See which modules failed or warned.
qc_problems(qc, "module")
```
     
     
The output above is in a compact format. For a stretched format, type this:   
    
    
```{r}
qc_problems(qc, "module", compact = FALSE)
```
    
```{block, type = "success"}
In the the stretched format each row correspond to a unique sample. Additionally, the status of each module is specified.
```

     
It's also possible to display problems for one or more specified modules. For example,
      
      
```{r}
qc_problems(qc, "module",  name = "Per sequence GC content")
```
    
   
```{block, type = "warning"}
Note that, partial matching of name is allowed. For example, name = "Per sequence GC content" equates to name = "GC content".
```
  
     
```{r, eval = FALSE}
qc_problems(qc, "module",  name = "GC content")
```


     
### Per Sample Problems
     
  
- **Samples with one or more failed modules**
       
```{r}
# See which samples had one or more failed modules
qc_fails(qc, "sample")
```
   

```{block, type = "success"}
For each sample, the number of problems (failures) and the name of modules, that failed, are shown.
```
   
   
- **Samples with failed or warned modules**:
      
```{r}
# See which samples had one or more module with failure or warning
qc_problems(qc, "sample", compact = FALSE)
```
     
     
To specify the name of a sample of interest, type this:   
    
    
```{r}
qc_problems(qc, "sample", name = "S1")
```
    
    
## Building an HTML Report
   
The function **qc_report**() can be used to build a report of FastQC outputs. It creates an HTML file containing FastQC reports of one or multiple samples.   
    
Inputs can be either a directory containing multiple FastQC reports or a single sample FastQC report.
    
    
   
### Create a Multi-QC Report
    
    
We'll build a multi-qc report for the following demo QC directory:   
    
```{r}
# Demo QC Directory
qc.dir <- system.file("fastqc_results", package = "fastqcr")
qc.dir
```
    

    
```{r, eval = FALSE}
# Build a report
qc_report(qc.dir, result.file = "~/Desktop/multi-qc-result",
          experiment = "Exome sequencing of colon cancer cell lines")
```
    

```{block, type = "success"}
An example of report is available at: <a href= "https://rpkgs.datanovia.com/fastqcr/qc-reports/fastqcr-multi-qc-report.html", target = "_blank"> fastqcr multi-qc report</a>
```
    
    
### Create a One-Sample Report
   
   
We'll build a report for the following demo QC file:   
   
    
```{r}
 qc.file <- system.file("fastqc_results", "S1_fastqc.zip", package = "fastqcr")
qc.file
```

   
- **One-Sample QC report with plot interpretations**:   
   
```{r, eval = FALSE}
 qc_report(qc.file, result.file = "one-sample-report-with-interpretation",
   interpret = TRUE)
```
     

```{block, type = "success"}
An example of report is available at: <a href= "https://rpkgs.datanovia.com/fastqcr/qc-reports/sample-qc-report-interpretation.html", target = "_blank"> One sample QC report with interpretation</a>
```
   
    
- **One-Sample QC report without plot interpretations**:
     
 
```{r, eval = FALSE}
 qc_report(qc.file, result.file = "one-sample-report",
   interpret = FALSE)
```
     

```{block, type = "success"}
An example of report is available at: <a href= "https://rpkgs.datanovia.com/fastqcr/qc-reports/sample-qc-report-without-interpretation.html", target = "_blank"> One sample QC report without interpretation</a>
```
   
   
## Importing and Plotting a FastQC QC Report
   
   
We'll visualize the output for sample 1:
   
   
```{r}
# Demo file
qc.file <- system.file("fastqc_results", "S1_fastqc.zip",  package = "fastqcr")
qc.file
```
    
We start by reading the output using the function **qc_read**(), which returns a list of tibbles containing the data for specified modules:  
    
```{r, echo= FALSE}
qc.file = "/Users/kassambara/Documents/R/MyPackages/fastqcr/inst/fastqc_results/S1_fastqc.zip"
```

    
```{r}
# Read all modules
qc <- qc_read(qc.file)
# Elements contained in the qc object
names(qc)
```

   
The function **qc_plot**() is used to visualized the data of a specified module. Allowed values for the argument modules include one or the combination of:
    
- "Summary",
- "Basic Statistics",
- "Per base sequence quality",
- "Per sequence quality scores",
- "Per base sequence content",
- "Per sequence GC content",
- "Per base N content",
- "Sequence Length Distribution",
- "Sequence Duplication Levels",
- "Overrepresented sequences",
- "Adapter Content"
    
    
```{r qc-plot, fig.width=3.5, fig.height=3.5, fig.show='hold'}
qc_plot(qc, "Per sequence GC content")

qc_plot(qc, "Per base sequence quality")

qc_plot(qc, "Per sequence quality scores")

qc_plot(qc, "Per base sequence content")

qc_plot(qc, "Sequence duplication levels")
```
     
     
## Useful Links
   
- FastQC report for a [good Illumina dataset](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/good_sequence_short_fastqc.html)
- FastQC report for a [bad Illumina dataset](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/bad_sequence_fastqc.html)
- [Online documentation for each FastQC report](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/3%20Analysis%20Modules/)