Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new ZST compression plugin #205

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

asvinours
Copy link

I added a new compression plugin based on facebook zstandard compression system.

zstandard offers faster and better compression then gzip

$ time zstd -f -5 random.log -o random.log.zst
Compressed 179309772 bytes into 21463467 bytes ==> 11.97%                      

real    0m1.584s
user    0m0.408s
sys     0m1.072s

$ time lzop -qf1 -o random.log.lzo random.log

real    0m0.387s
user    0m0.304s
sys     0m0.068s

$ time gzip -c random.log > random.log.gz

real    0m3.224s
user    0m2.916s
sys     0m0.180s

$ ls -lh
172M Sep  1 13:43 random.log
49M Sep  1 13:43 random.log.lzo
31M Sep  1 18:58 random.log.gz
21M Sep  1 18:57 random.log.zst

Add new ZST compression plugin
Add test for the new ZST compression plugin
@repeatedly
Copy link
Member

Do other users still want this compression?
Zst is now widely used so bundle it is reasonable.

@github-actions
Copy link

github-actions bot commented Jul 6, 2021

This PR has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this PR will be closed in 30 days

@github-actions github-actions bot added the stale label Jul 6, 2021
@kenhys kenhys added enhancement and removed stale labels Jul 16, 2021
@kenhys
Copy link
Contributor

kenhys commented Jul 19, 2021

@asvinours

Oh, I've unexpectedly reflected merge commit.
Could you reset 06c8bd2 and rebase with the master, please?
And follow( DCO git commit --sign )

@siddharthg
Copy link

Any plans of merging this one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants