Skip to content

Commit

Permalink
rm trailing whitespace
Browse files Browse the repository at this point in the history
  • Loading branch information
slager committed Apr 27, 2024
1 parent 08c8b22 commit 891030c
Show file tree
Hide file tree
Showing 9 changed files with 148 additions and 148 deletions.
12 changes: 6 additions & 6 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Contributor Code of Conduct

As contributors and maintainers of this project, we pledge to respect all people who
As contributors and maintainers of this project, we pledge to respect all people who
contribute through reporting issues, posting feature requests, updating documentation,
submitting pull requests or patches, and other activities.

Expand All @@ -13,13 +13,13 @@ imagery, derogatory comments or personal attacks, trolling, public or private ha
insults, or other unprofessional conduct.

Project maintainers have the right and responsibility to remove, edit, or reject comments,
commits, code, wiki edits, issues, and other contributions that are not aligned to this
Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed
commits, code, wiki edits, issues, and other contributions that are not aligned to this
Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed
from the project team.

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by
opening an issue or contacting one or more of the project maintainers.

This Code of Conduct is adapted from the Contributor Covenant
(http://contributor-covenant.org), version 1.0.0, available at
This Code of Conduct is adapted from the Contributor Covenant
(http://contributor-covenant.org), version 1.0.0, available at
http://contributor-covenant.org/version/1/0/0/
14 changes: 7 additions & 7 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Authors@R:
email = "ellishughes@live.com",
comment = c(ORCID = "0000-0003-0637-4436")),
person(given = "Jimmy", family = "Fulp",
role = c("ctb"),
role = c("ctb"),
email = "williamjfulp@gmail.com"),
person(given = "Marie", family = "Vendettuoli",
role = c("ctb"),
Expand All @@ -23,7 +23,7 @@ Authors@R:
email = "dslager@scharp.org",
comment = c(ORCID = "0000-0003-2525-2039")),
person(given = "Jason", family = "Taylor",
role = c("ctb"),
role = c("ctb"),
email = "jmtaylor@fredhutch.org"),
person(given = "Kara", family = "Woo",
role = "rev",
Expand All @@ -32,15 +32,15 @@ Authors@R:
role = "rev",
comment = "William reviewed the package for rOpenSci, see <https://github.com/ropensci/onboarding/issues/230>"))
Version: 0.15.9.9000
Description: A framework to help construct R data packages in a
reproducible manner. Potentially time consuming processing of
raw data sets into analysis ready data sets is done in a reproducible
Description: A framework to help construct R data packages in a
reproducible manner. Potentially time consuming processing of
raw data sets into analysis ready data sets is done in a reproducible
manner and decoupled from the usual 'R CMD build' process so that
data sets can be processed into R objects in the data package and
data sets can be processed into R objects in the data package and
the data package can then be shared, built, and installed by others
without the need to repeat computationally costly data processing.
The package maintains data provenance by turning the data processing
scripts into package vignettes, as well as enforcing documentation
scripts into package vignettes, as well as enforcing documentation
and version checking of included data objects. Data packages can be
version controlled on 'GitHub', and used to share data for manuscripts,
collaboration and reproducible research.
Expand Down
4 changes: 2 additions & 2 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
* New global option `DataPackageR_verbose` to suppress console output, e.g. during unit testing

# DataPackageR 0.15.8
* Fix to datapackager_object_read that was causing a test to break. `get` needs to have `inherits=FALSE`.
* Fix to datapackager_object_read that was causing a test to break. `get` needs to have `inherits=FALSE`.
* Other fixes for `usethis` 1.6.0
* Fixes to tests that were failing on CRAN
* In `package_build`, remove `devtools::reload` and put `devtools::unload` and in front of install.packages
Expand Down Expand Up @@ -88,7 +88,7 @@
- Improved package test.
- R scripts processed properly into vignettes.
- Packages installed and loaded after build to make vignettes and data sets accessible in same R session.
-
-

# DataPackageR 0.13.6

Expand Down
84 changes: 42 additions & 42 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: README
output: github_document
bibliography: bibliography.bib
editor_options:
editor_options:
chunk_output_type: inline
---

Expand Down Expand Up @@ -42,7 +42,7 @@ You have diverse raw data sets that you need to preprocess and tidy in order to:

- Perform data analysis
- Write a report
- Publish a paper
- Publish a paper
- Share data with colleagues and collaborators
- Save time in the future when you return to this project but have forgotten all about what you did.

Expand All @@ -55,67 +55,67 @@ You have diverse raw data sets that you need to preprocess and tidy in order to:
As described [elsewhere](https://github.com/ropensci/rrrpkg), packaging your data promotes reproducibility.
R's packaging infrastructure promotes unit testing, documentation, a reproducible build system, and has many other benefits.
Coopting it for packaging data sets is a natural fit.

- **Collaboration.**

A data set packaged in R is easy to distribute and share among collaborators, and is easy to install and use.
All the hard work you've put into documenting and standardizing the tidy data set comes right along with the data package.

- **Documentation.**

R's package system allows us to document data objects. What's more, the `roxygen2` package makes this very easy to do with [markup tags](https://r-pkgs.org/data.html).
R's package system allows us to document data objects. What's more, the `roxygen2` package makes this very easy to do with [markup tags](https://r-pkgs.org/data.html).
That documentation is the equivalent of a data dictionary and can be extremely valuable when returning to a project after a period of time.

- **Convenience.**

Data pre-processing can be time consuming, depending on the data type and raw data sets may be too large to share conveniently in a packaged format.
Data pre-processing can be time consuming, depending on the data type and raw data sets may be too large to share conveniently in a packaged format.
Packaging and sharing the small, tidied data saves the users computing time and time spent waiting for downloads.

## Challenges.

- **Package size limits.**

R packages have a 10MB size limit, at least on [CRAN](https://cran.r-project.org/web/packages/policies.html). Bioconductor [ExperimentHub](http://contributions.bioconductor.org/data.html#data) may be able to support larger data packages.

Sharing large volumes of raw data in an R package format is still not ideal, and there are public biological data repositories better suited for raw data: e.g., [GEO](https://www.ncbi.nlm.nih.gov/geo/), [SRA](https://www.ncbi.nlm.nih.gov/sra), [ImmPort](https://www.immport.org/), [ImmuneSpace](https://immunespace.org/), [FlowRepository](http://flowrepository.org/).

Tools like [datastorr](https://github.com/traitecoevo/datastorr) can help with this and we hope to integrate this into DataPackageR in the future.

- **Manual effort**

There is still a substantial manual effort to set up the correct directory structures for an R data package. This can dissuade many individuals, particularly new users who have never built an R package, from going this route.
- **Scale**

- **Scale**

Setting up and building R data packages by hand is a workable solution for a small project or a small number of projects, but when dealing with many projects each involving many data sets, tools are needed to help automate the process.

## DataPackageR

DataPackageR provides a number of benefits when packaging your data.

- It aims to automate away much of the tedium of packaging data sets without getting too much in the way, and keeps your processing workflow reproducible.

- It sets up the necessary package structure and files for a data package.
- It sets up the necessary package structure and files for a data package.

- It allows you to keep the large, raw data and only ship the packaged tidy data, saving space and time consumers of your data set need to spend downloading and re-processing it.

- It maintains a reproducible record (vignettes) of the data processing along with the package. Consumers of the data package can verify how the processing was done, increasing confidence in your data.
- It maintains a reproducible record (vignettes) of the data processing along with the package. Consumers of the data package can verify how the processing was done, increasing confidence in your data.

- It automates construction of the documentation and maintains a data set version and an md5 fingerprint of each data object in the package. If the data changes and the package is rebuilt, the data version is automatically updated.

## Similar work

There are a number of tools out there that address similar and complementary problems:
There are a number of tools out there that address similar and complementary problems:

- **datastorr**
[github repo](https://github.com/traitecoevo/datastorr)

Simple data retrieval and versioning using GitHub to store data.

- Caches downloads and uses github releases to version data.
- Deal consistently with translating the file stored online into a loaded data object
- Access multiple versions of the data at once
- Access multiple versions of the data at once

`datastorrr` could be used with DataPackageR to store / access remote raw data sets, remotely store / access tidied data that are too large to fit in the package itself.

- **fst**
Expand All @@ -127,19 +127,19 @@ There are a number of tools out there that address similar and complementary pro
[pdf](https://github.com/noamross/2018-04-18-rstats-nyc/blob/master/Noam_Ross_ModernDataPkg_rstatsnyc_2018-04-20.pdf)

A presentation from \@noamross touching on modern tools for open science and reproducibility. Discusses `datastorr` and `fst` as well as standardized metadata and documentation.

- **rrrpkg**
[github repo](https://github.com/ropensci/rrrpkg)

A document from rOpenSci describing using an R package as a research compendium. Based on ideas originally introduced by Robert Gentleman and Duncan Temple Lang (Gentleman and Lang (2004)<!--@Gentleman2004-oj-->)

- **template**
[github repo](https://github.com/ropensci/rrrpkg)

An R package template for data packages.

See the [publication](#publication) for further discussion.

## Installation

You can install the latest [CRAN](https://cran.r-project.org/package=DataPackageR) release of DataPackageR with:
Expand All @@ -157,9 +157,9 @@ remotes::install_github("ropensci/DataPackageR")

## Blog Post - building packages interactively.

See this [rOpenSci blog post](https://ropensci.org/blog/2018/09/18/datapackager/) on how to build data packages interactively using DataPackageR.
See this [rOpenSci blog post](https://ropensci.org/blog/2018/09/18/datapackager/) on how to build data packages interactively using DataPackageR.
This uses several new interfaces: `use_data_object()`, `use_processing_script()` and `use_raw_dataset()` to build up a data package, rather than assuming
the user has all the code and data ready to go for `datapackage_skeleton()`.
the user has all the code and data ready to go for `datapackage_skeleton()`.

## Example (assuming all code and data are available)

Expand All @@ -184,12 +184,12 @@ processing_code <- system.file(
# We pass in the Rmd file in the `processing_code` variable and the names of the data objects it creates (called "cars_over_20")
# The new package is called "mtcars20"
datapackage_skeleton(
"mtcars20", force = TRUE,
code_files = processing_code,
r_object_names = "cars_over_20",
path = tempdir())
"mtcars20", force = TRUE,
code_files = processing_code,
r_object_names = "cars_over_20",
path = tempdir())
# 3. Run the preprocessing code to build the cars_over_20 data set
# 3. Run the preprocessing code to build the cars_over_20 data set
# and reproducibly enclose it in the mtcars20 package.
# packageName is the full path to the package source directory created at step 2.
# You'll be prompted for a text description (one line) of the changes you're making.
Expand All @@ -198,7 +198,7 @@ datapackage_skeleton(
# "Package built in non-interactive mode". You may update it later.
package_build(packageName = file.path(tempdir(),"mtcars20"))
# Update the autogenerated roxygen documentation in data-raw/documentation.R.
# Update the autogenerated roxygen documentation in data-raw/documentation.R.
# edit(file.path(tempdir(),"mtcars20","R","mtcars20.R"))
# 4. Rebuild the documentation.
Expand Down Expand Up @@ -238,21 +238,21 @@ In order to have portable and reproducible code, you should not use absolute pat
Instead, `DataPackageR` provides several APIs to access the data package project root directory, the `inst/extdata` subdirectory, and the `data` subdirectory.

```{r, eval = FALSE}
# This returns the datapackage source
# root directory.
# In an R or Rmd processing script this can be used to build a path to a directory that is exteral to the package, for
# This returns the datapackage source
# root directory.
# In an R or Rmd processing script this can be used to build a path to a directory that is exteral to the package, for
# example if we are dealing with very large data sets where data cannot be packaged.
DataPackageR::project_path()
# This returns the
# inst/extdata directory.
# This returns the
# inst/extdata directory.
# Raw data sets that are included in the package should be placed there.
# They can be read from that location, which is returned by:
# They can be read from that location, which is returned by:
DataPackageR::project_extdata_path()
# This returns the path to the datapackage
# data directory. This can be used to access
# stored data objects already created and saved in `data` from
# This returns the path to the datapackage
# data directory. This can be used to access
# stored data objects already created and saved in `data` from
# other processing scripts.
DataPackageR::project_data_path()
```
Expand All @@ -266,7 +266,7 @@ The publication describing the package, (Finak *et al.*, 2018)<!--@10.12688/gate
The preprint is on [bioRxiv](https://doi.org/10.1101/342907).


## Code of conduct
## Code of conduct

Please note that this project is released with a [Contributor Code of Conduct](https://github.com/ropensci/DataPackageR/blob/main/CODE_OF_CONDUCT.md).
By participating in this project you agree to abide by its terms.
Expand Down
2 changes: 1 addition & 1 deletion inst/extdata/tests/extra.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ cars_over_20 = datapackager_object_read("cars_over_20")
print(cars_over_20)
```

This API reads from an environment named `ENVS`, containing `subsetCars` and any other previously built data set objects. It is passed into the render environment of `extra.Rmd` by DataPackageR at the `render()` call.
This API reads from an environment named `ENVS`, containing `subsetCars` and any other previously built data set objects. It is passed into the render environment of `extra.Rmd` by DataPackageR at the `render()` call.

## Additional data objects

Expand Down
22 changes: 11 additions & 11 deletions inst/extdata/tests/subsetCars.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ knitr::opts_chunk$set(echo = TRUE)
```

This is a simple Rmd file that demonstrates how DataPackageR processes Rmarkdown files and creates data sets
that are then stored in an R data package.
that are then stored in an R data package.

In the `config.yml` for this example, this file is listed first, and therefore processed first.

Expand All @@ -28,31 +28,31 @@ Let's say, for some reason, we are only interested in the stopping distances of
cars_over_20 = subset(cars, speed > 20)
```

The data frame `cars_over_20` now holds this information.
The data frame `cars_over_20` now holds this information.

# Storing data set objects and making making accessible to other processing scripts.
# Storing data set objects and making making accessible to other processing scripts.

When DataPackageR processes this file, it creates this `cars_over_20` object. After processing the file it does several things:

1. It compares the objects in the rmarkdown render environment of `subsetCars.Rmd` against the objects listed in the `config.yml` file `objects` property.
1. It compares the objects in the rmarkdown render environment of `subsetCars.Rmd` against the objects listed in the `config.yml` file `objects` property.
2. It finds `cars_over_20` is listed there, so it stores it in a new environment.
3. That environment is passed to subsequent R and Rmd files. Specifically when the `extra.Rmd` file is processed, it has access to an environment object that holds all the `objects` (defined in the yaml config) that have already been created and processed. This environment is passed into subsequent scripts at the `render()` call.
3. That environment is passed to subsequent R and Rmd files. Specifically when the `extra.Rmd` file is processed, it has access to an environment object that holds all the `objects` (defined in the yaml config) that have already been created and processed. This environment is passed into subsequent scripts at the `render()` call.

All of the above is done automatically. The user only needs to list the objects to be stored and passed to other scripts in the `config.yml` file.
All of the above is done automatically. The user only needs to list the objects to be stored and passed to other scripts in the `config.yml` file.

The `datapackager_object_read()` API can be used to retrieve these objects from the environment.
The `datapackager_object_read()` API can be used to retrieve these objects from the environment.

### Storing objects in the data package

In addition to passing around an environment to subsequent scripts, the `cars_over_20` object is stored in the data package `/data` directory as an `rda` file.
In addition to passing around an environment to subsequent scripts, the `cars_over_20` object is stored in the data package `/data` directory as an `rda` file.

Note that this is all done automatically. The user does not need to explicitly save anything, they only need to list the objects to be store in the `config.yml`.
Note that this is all done automatically. The user does not need to explicitly save anything, they only need to list the objects to be store in the `config.yml`.

This object is then accessible in the resulting package via the `data()` API, and its documentation is accessible via `?cars_over_20`.
This object is then accessible in the resulting package via the `data()` API, and its documentation is accessible via `?cars_over_20`.

### Data object documentation

The documentation for the `cars_over_20` object is created in a `subsetCars.R` file in the `/R` directory of the data package.
The documentation for the `cars_over_20` object is created in a `subsetCars.R` file in the `/R` directory of the data package.

While the data object document stub is created automatically, it must be edited by the user to provide additional details about the data object.

Expand Down

0 comments on commit 891030c

Please sign in to comment.