EDS is an accronym for Efficient single cell binary Data Storage format for the cell-feature count matrices.
Recent advancements in single-cell technologies have seen rapid increase in the amount of data. Most single-cell studies generate a cell by feature (can be gene) count matrices, where the number of cells are now reaching towards millions. Traditional Single-cell quantification pipelines use matrix market exchange (mtx) format (sometimes gzipped) for sharing the count matrices. However, the textual representation of mtx format makes it bigger in size compared to a compressed binary format. Our quantification tool alevin dumps the output in EDS format which saves storage space.
There are other formats (such as loom) which are designed for optimizing the query of the matrix. EDS is primarily designed to improve the storage efficiency rather than query and currently don't support random access to a cell (row).
We have a simple rust code inside the src-rs, it can be installed using cargo build --release and can be used as ./target/release/eds convert -i <input gzipped file currently [eds.gz | mtx.gz]> --[mtx | eds | h5 | csv] -c <num_cells> -f <num_features>.
-
Size on disk.
-
Matrix loading into memory time.
-
Memory required to load the matrix.
- Support delayedArray R object
- Random access through
EDS index
- Avi Srivastava
- Mike Love
- Rob Patro