Stefan Hermann

Ragnar Groot Koerkamp, Stefan Hermann, Peter Sanders et al.

A minimal perfect hash function (minimal PHF) is a data structure mapping a static set of $n$ keys to $n$ bins without collisions. Two natural generalizations are minimal $k$-PHFs where $n$ keys are mapped to $n/k$ bins of capacity $k$ each, and (non-minimal) PHFs with load factor $α < 1$ where the number of bins is increased by a factor of $1/α$, resulting in spare capacity. While there has been a recent surge of interest in perfect hashing generally, non-minimal $k$-PHFs have not been systematically studied despite a natural use case of speeding up static hash tables: The idea is that a small cache-resident $k$-PHF maps each key $x$ to a cache-line-sized bin of capacity $k$ where $x$ resides. Ideally, this yields a branchless lookup operation with a single cache miss working at high load factors for positive and negative queries alike. Our main theoretical contribution is to determine tight space lower bounds for $k$-PHFs for all pairs of $α \in (0,1]$ and $k \geq 1$. It turns out that combining $α < 1$ and $k \geq 2$ drastically reduces the space of $k$-PHFs, e.g. for $(k,α) = (16,0.8)$ the space lower bound is $0.027$ bits per key while for $(k,α) = (16,1.0)$ and $(k,α) = (1,0.8)$ the lower bounds are higher by factors of $\approx 8$ and $\approx 32$, respectively. On the practical side, we develop a $k$-PHF based on PtrHash and tune it for use in static hash tables. Empirically, our implementation produces $k$-PHFs of size roughly $50\%$ above the lower bound. A static hash set based on this $k$-PHF is consistently at least as fast as other hash sets for negative and mixed queries. On two of the three tested architectures it achieves up to $1.5\times$ speedup for large $n\geq 30M$ where a $1$-PHF does not fit in cache.

Stefan Hermann, Hans-Peter Lehmann, Giorgio Vinciguerra et al.

We consider the task of constructing a data structure for associating a static set of keys with values, while allowing arbitrary output values for queries involving keys outside the set. Compared to hash tables, these so-called static function data structures do not need to store the key set and thus use significantly less memory. Several techniques are known, with compressed static functions approaching the zero-order empirical entropy of the value sequence. In this paper, we introduce learned static functions, which use machine learning to capture correlations between keys and values. For each key, a model predicts a probability distribution over the values, from which we derive a key-specific prefix code to compactly encode the true value. The resulting codeword is stored in a classic static function data structure. This design allows learned static functions to break the zero-order entropy barrier while still supporting point queries. Our experiments show substantial space savings: up to one order of magnitude on real data, and up to three orders of magnitude on synthetic data.