Skip to content

Generates reproducible outputs and improves performance greatly

Compare
Choose a tag to compare
@sfchen sfchen released this 08 Oct 07:01
· 53 commits to master since this release

The threading and I/O modules have been completely rewritten to generate reproducible outputs and improve performance greatly. New libraries libisal and libdeflate were introduced to replace the slow zlib. Although this may bring some difficulties to compilation, it is all worthwhile for performance improvement.

In many cases, fastp v0.23.0 can be 2x as faster as previous versions. Especially, when the compression level is set to 6 or higher, the performance gain is very obvious.

Threading randomness issue has been addressed, so the output files are MD5 consistent when you run it twice, which means the results are completely reproducible.