Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update zero-allocation-hashing and xx hash method name #959

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Update zero-allocation-hashing and xx hash method name #959

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 5 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ and what APIs have changed, if applicable.

## [Unreleased]

## [29.48.10] - 2023-12-20
- Update `zero-allocation-hashing` and use method `xx` rather than `xx_39`.

## [29.48.9] - 2023-12-20
- No-op proxy detector for xDS gRPC channel builder.
- Warning for unsupported `zero-allocation-hashing` library versions.
Expand Down Expand Up @@ -5591,7 +5594,8 @@ patch operations can re-use these classes for generating patch messages.

## [0.14.1]

[Unreleased]: https://github.com/linkedin/rest.li/compare/v29.48.9...master
[Unreleased]: https://github.com/linkedin/rest.li/compare/v29.48.10...master
[29.48.10]: https://github.com/linkedin/rest.li/compare/v29.48.9...v29.48.10
[29.48.9]: https://github.com/linkedin/rest.li/compare/v29.48.8...v29.48.9
[29.48.8]: https://github.com/linkedin/rest.li/compare/v29.48.7...v29.48.8
[29.48.7]: https://github.com/linkedin/rest.li/compare/v29.48.6...v29.48.7
Expand Down
2 changes: 1 addition & 1 deletion build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ project.ext.externalDependency = [
'spock': 'org.spockframework:spock-core:1.3-groovy-2.5',
'testng': 'org.testng:testng:6.13.1',
'velocity': 'org.apache.velocity:velocity-engine-core:2.2',
'zero_allocation_hashing': 'net.openhft:zero-allocation-hashing:0.7',
'zero_allocation_hashing': 'net.openhft:zero-allocation-hashing:0.16',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can use the custom fork now, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet; the custom fork is only released internally to LinkedIn. We need to update internal users to use our fork, and then change their code to use xx() instead of xx_r39(). After that, we are free to use net.openhft:zero-allocation-hashing:0.16 in Pegasus. We won't need to fork anymore at that point.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it. Thanks!

'zookeeper': 'org.apache.zookeeper:zookeeper:3.6.3',
'hdrhistogram': 'org.hdrhistogram:HdrHistogram:2.1.9',
'xchart': 'org.knowm.xchart:xchart:3.2.2',
Expand Down
13 changes: 2 additions & 11 deletions d2/src/main/java/com/linkedin/d2/balancer/util/hashing/MPConsistentHashRing.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,16 +53,7 @@ public class MPConsistentHashRing<T> implements Ring<T>
public static final int DEFAULT_POINTS_PER_HOST = 1;

private static final Logger LOG = LoggerFactory.getLogger(ConsistentHashRing.class);
static {
try {
LongHashFunction.class.getMethod("xx_r39", long.class);
} catch (NoSuchMethodException ex) {
LOG.error("Required method xx_r39 not found, this means an unsupported version of the "
+ "zero-allocation-hashing library is being used. Do not use later than 0.7 if you want to use pegasus", ex);
throw new RuntimeException(ex);
}
}
private static final LongHashFunction HASH_FUNCTION_0 = LongHashFunction.xx_r39(0xDEADBEEF);
private static final LongHashFunction HASH_FUNCTION_0 = LongHashFunction.xx(0xDEADBEEF);
private static final Charset UTF8 = Charset.forName("UTF-8");
/* we will only use the lower 32 bit of the hash code to avoid overflow */
private static final long MASK = 0x00000000FFFFFFFFL;
Expand Down Expand Up @@ -114,7 +105,7 @@ public MPConsistentHashRing(Map<T, Integer> pointsMap, int numProbes, int points
_hashFunctions = new LongHashFunction[_numProbes];
for (int i = 0; i < _numProbes; i++)
{
_hashFunctions[i] = LongHashFunction.xx_r39(i);
_hashFunctions[i] = LongHashFunction.xx(i);
}
}

Expand Down
2 changes: 1 addition & 1 deletion d2/src/main/java/com/linkedin/d2/balancer/util/hashing/XXHash.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
public class XXHash implements HashFunction<String[]> {
//In order to ensure the consistency of the hashing function,
// we seed the hash function in the beginning with fixed seed.
private static final LongHashFunction hashFunction = LongHashFunction.xx_r39(0xDEADBEEF);
private static final LongHashFunction hashFunction = LongHashFunction.xx(0xDEADBEEF);

@Override
public int hash(String[] keyTokens) {
Expand Down
2 changes: 1 addition & 1 deletion gradle.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
version=29.48.9
version=29.48.10
group=com.linkedin.pegasus
org.gradle.configureondemand=true
org.gradle.parallel=true
Expand Down