Comparing Hashing strategies

Overview

Chronicle has a number of implementations for hashing, including City and Murmur.  It also has it's own Vanilla Hash, but how was this tested?

What is Vanilla Hash?

Vanilla Hash is designed to be as simple as possible and be optimised for the Orthogonal Bits test (See below)  This was compared with City 1.1 and Murmur 3 hashing strategies.

This is the 99%tile latencies for filling the 64 byte/256 byte buffer with new data and generating a 64-bit hash. JMH was used to perform the measurements. See Main64bytes and Main256bytes


Hashing Strategy
64 byte 99%tile
256 byte 99%tile
Vanilla
67 ns
112 ns
City 1.1
90 ns
182 ns
Murmur 3
104 ns
211 ns

The full test of results are here.

What tests can you do to check a hashing strategy is good?

There is a number of simple tests you can do. The tests cannot identify a good hash, but they can show a hash to be a poor one. Passing one test could mean it will fail another.

In each case multiple tests are run with different random starting points.  A score is taken for the 99%th percentile, i.e. the worst 1%.  This is because you don't need a hash which works some of the time, or on average.  You need one which works most of the time. (In all cases you can invent a pathological case where any specific hash will break down)

For consistency, lower scores are better. The test should be constructed in such as that a score of 0 indicates the test is broken.

In each test, using an input of 8,192 bits, or 1024 KB, one bit at a time is toggled.  From these inputs, 8,192 x 64-bit hashes are generated.

For the random tests however, a sequence of random 64-bit values were taken. These are useful to get an idea of what a good number is for the hashing strategies tested.

Mask of Hash Score

In this test, each hash is modulus by 16,384 (double the number of hashes) and the number of collisions is reported.  Most hashing strategies did well for this test.

Avalanche Score

In this test, each hash is compared to the previous hash (with the previous bit toggled) to see how likely any given bit will be flipped.  The ideal is 50% and the sum of difference to 50% is taken with the worst 1% reported.

Speed in latency

In this test, the time it takes to perform the hash is recorded and the worst 1% latency reported.

Orthogonal Bits

The purpose of this tests is to ensure all the hashes have bits which are different as different as possible to every other hash produced.  Think of the 8 Queens Problem, except for 64-bit numbers.  The ideal is that every number has the same number of bits different to every other number and this is as high as possible.

In this test, every hash is compared to every other hash. A count of the number of bits which are different is taken.  If the number of different bits is less than 18, this is given a penalty score of 2^(17-n).  The less bits which are different the greater the penalty on an exponential scale. If any of the 8K hashes comapred to the other 8K hashes are different in less than 5 bits, this is a failure even if all the other pairs are fine.

I have called it an Orthogonal Bits test as you can model a 64-bit number as a 64 dimensional vector of bits.  Ideally you want the angle between all the hashes produced as high as possible.

Of all the tests, this one shows the highest difference between the String.hashCode() with HashMap.hash(int) and the other hashing strategies.

Testing String.hashCode()

String.hashCode() is a very poor hash, especially for the lower bits. It is standard and cannot be changed or break backward compatibility.  However, this doesn't have to be a problem as HashMap uses an agitate function which brings down some of the higher bits to randomise the lower ones.

int hash(int h) {
    // This function ensures that hashCodes that differ only by
    // constant multiples at each bit position have a bounded
    // number of collisions (approximately 8 at default load factor).
    h ^= (h >>> 20) ^ (h >>> 12);
    return h ^ (h >>> 7) ^ (h >>> 4);
}

Results

The CheckMain class run a suit of tests on each hashing strategy.

VANILLA
Orthogonal bits: 99%tile score: 6066
Speed: The 99%tile for latency was 0.223 us
Avalanche: The 99%tile of the drift from 50% was 0.55%
Mask of Hash: 99%tile collisions: 1815

CITY_1_1
Orthogonal bits: 99%tile score: 7395
Speed: The 99%tile for latency was 0.267 us
Avalanche: The 99%tile of the drift from 50% was 0.55%
Mask of Hash: 99%tile collisions: 1817

MURMUR_3
Orthogonal bits: 99%tile score: 7524
Speed: The 99%tile for latency was 0.378 us
Avalanche: The 99%tile of the drift from 50% was 0.54%
Mask of Hash: 99%tile collisions: 1815

STRING32
Orthogonal bits: 99%tile score: 295906433
Speed: The 99%tile for latency was 1.580 us
Avalanche: The 99%tile of the drift from 50% was 1.02%
Mask of Hash: 99%tile collisions: 1814

STRING64
Orthogonal bits: 99%tile score: 1939167
Speed: The 99%tile for latency was 1.520 us
Avalanche: The 99%tile of the drift from 50% was 0.61%
Mask of Hash: 99%tile collisions: 1816

STRING32_WITHOUT_AGITATE
Orthogonal bits: 99%tile score: 879390386
Speed: The 99%tile for latency was 1.573 us
Avalanche: The 99%tile of the drift from 50% was 3.53%
Mask of Hash: 99%tile collisions: 6593

RANDOM
Orthogonal bits: 99%tile score: 7444
Speed: The 99%tile for latency was 0.058 us
Avalanche: The 99%tile of the drift from 50% was 0.53%
Mask of Hash: 99%tile collisions: 1817

SECURE_RANDOM
Orthogonal bits: 99%tile score: 7449
Speed: The 99%tile for latency was 0.861 us
Avalanche: The 99%tile of the drift from 50% was 0.54%
Mask of Hash: 99%tile collisions: 1816

SEEDED_VANILLA
Orthogonal bits: 99%tile score: 6000
Speed: The 99%tile for latency was 0.219 us
Avalanche: The 99%tile of the drift from 50% was 0.55%
Mask of Hash: 99%tile collisions: 1814

SEEDED_CITY_1_1
Orthogonal bits: 99%tile score: 7313
Speed: The 99%tile for latency was 0.270 us
Avalanche: The 99%tile of the drift from 50% was 0.54%
Mask of Hash: 99%tile collisions: 1813

SEEDED_MURMUR_3
Orthogonal bits: 99%tile score: 7404
Speed: The 99%tile for latency was 0.359 us
Avalanche: The 99%tile of the drift from 50% was 0.53%
Mask of Hash: 99%tile collisions: 1810

Note: Seeded Vanilla Hash is part of Chronicle Enterprise

Conclusions

The Vanilla, City and Murmur hashers were the fastest.  

While String.hashCode() is simple, the multiplication operation on a per character basis is expensive.  By comparison all the others process 8 bytes at a time using longs.  See STRINGS32_WITHOUT_AGITATE compared with STRING32.  HashMap uses the later.

The 32-bit String hashCode() even with the agitate performed poorly on the Avalanche test. In SMHasher where this test comes from a score over 1% was considered a failure.

The Mask of Hash tests, while simple appears to be performed well in all cases. The exception being the String.hashCode() which as mentioned doesn't have very random low bits.

What I found interesting is how different the orthogonal test score were.  The first three hash stratgies were again consistently low. Even the 64-bit version of String.hashCode() has a high change of producing hashes with less than 18 bits different, in fact a lot of the bits are the same.

Disclaimer

Vanilla Hash was optimised for the Orthogonal Bits test.  As such it is no surprise that it gets a slightly better result.  This doesn't mean that Vanilla Hash is better than City or Murmur. It may just mean it is best for the Orthogonal Bits test.





Comments

Popular posts from this blog

Java is Very Fast, If You Don’t Create Many Objects

System wide unique nanosecond timestamps

Comparing Approaches to Durability in Low Latency Messaging Queues