Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document meaning of "non-portable" #449

Closed
shuffle2 opened this issue Sep 6, 2020 · 6 comments
Closed

Document meaning of "non-portable" #449

shuffle2 opened this issue Sep 6, 2020 · 6 comments

Comments

@shuffle2
Copy link

shuffle2 commented Sep 6, 2020

I think it would be good to specify (in xxhash.h) what exactly is meant by portable.
For example there is:

The algorithm is portable: Like XXH32 and XXH64, it generates the same hash on all platforms.

Under the heading:

XXH3 is a new hash algorithm featuring:

Notably this doesn't mention the various vectorized implementations of XXH3, which other references (readme, github, etc) refer to as "non-portable".

My understanding is that "non-portable" only means that the code requires specific instruction set support in order to execute, not that the API will output different digests, given the same input. Is that correct? If so, perhaps different terms should be used, as "portable" and "non-portable" are not actually being used as inverses of each other.

p.s. That part of the header still references pre-0.8.0 release, and sounds as if the XXH3 implementation is still unstable...

@easyaspi314
Copy link
Contributor

easyaspi314 commented Sep 6, 2020

That is a good idea.

Basically, XXH3 is portable in that should compile on any compiler that can compile XXH64 — any sane C90 or C++98 compiler with a long long extension.

However, unlike XXH32 and XXH64, XXH3 can be further optimized with non-portable extensions like SIMD or 64-bit to 128-bit multiplication.

@shuffle2
Copy link
Author

shuffle2 commented Sep 6, 2020

It's important to note that what I care about is if the XXH3 algorithm, for all runtime implementations, is deterministic. For example, is there a guarantee that given identical inputs, outputs produced on a system using the NEON implementation will match those from an AVX2 system? (the comparison should hold for the full matrix of platforms supported by xxhash).

Otherwise, outputs from xxhash cannot be shared between systems using differently accelerated versions of the library.

I think/hope it is deterministic in this way, but it doesn't seem to be explicitly stated anywhere...

@easyaspi314
Copy link
Contributor

Yes. All implementations return the same value.

Cyan4973 added a commit that referenced this issue Sep 15, 2020
@Cyan4973
Copy link
Owner

The XXH3 presentation paragraph has been reworded,
hopefully reducing risks of confusion on this topic.

@alexttx
Copy link

alexttx commented Sep 22, 2020

Follow up question: have XXH32 and XXH64 been deterministic with respect to inputs and output values since release r42?

@Cyan4973
Copy link
Owner

yes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants