Using Set Shaping Theory to Trade RAM Accesses for CPU Computation
For database indexing, SST offers a new structural preprocessing approach that improves retrieval efficiency by preventing cluster formation, though the gains are incremental over standard hashing methods.
The paper applies Set Shaping Theory (SST) as a preprocessing layer before existing hash-based indexing methods, showing that SST-augmented variants reduce RAM accesses, probe counts, and tail latency across load factors 0.75–0.95 and database sizes up to 500,000 records.
This paper studies Set Shaping Theory (SST) in a database-index setting under a revised interpretation: SST is not treated as a competing hashing method, but as a structural pre processing layer that can be applied before an existing indexing algorithm. The experimental question is therefore whether a method improves when it is used with SST rather than with out it. The study compares linear probing, double hashing, quadratic probing, and Robin Hood hashing against their corresponding SST-augmented variants for shaping orders K = 2,4,8. Beyond mean time, the benchmark reports mean successful probes, 95th and 99th percentile probes, collisions per stored record, and maxi mum cluster length. Experiments cover load factors from 0.75 to 0.95, database sizes from M =5000 to M =500000, query multipliers up to 200 lookups per stored record, and both uniform and hotspot query distributions. The results highlight two fundamental advantages. First, SST reduces the number of RAM accesses required during retrieval. By prevent ing clusters and long probe chains from forming at insertion time, the lookup phase requires fewer memory jumps, lower probe counts, and reduced tail latency. Second, the method introduces a new way of thinking about data storage: the data are not treated as fixed objects that must be placed passively into a table, but as reversible representations that can be struc turally adapted before being written. A small metadata tag records which transformation was selected, allowing the original key to remain recoverable and the lookup process to remain deterministic.This article is connected to the Set Shaping Theory simulator project, available online at https://sst-simulator.github.io/Set-Shaping-Theory-Simulator/ where it is possible to simulate part of the results presented in the article.