A closer look at TDFA
For developers needing efficient regex parsing with submatch extraction, this algorithm offers a practical, fast solution, though it is an incremental improvement over existing TDFA methods.
The paper presents a fast algorithm for regular expression parsing and submatch extraction using tagged deterministic finite automata, supporting multiple disambiguation policies. Benchmarks show the algorithm is very fast in practice, with implementations in RE2C and a Java library.
We present an algorithm for regular expression parsing and submatch extraction based on tagged deterministic finite automata. The algorithm works with different disambiguation policies. We give detailed pseudocode for the algorithm, covering important practical optimizations. All transformations from a regular expression to an optimized automaton are explained on a step-by-step example. We consider both ahead-of-time and just-in-time determinization and describe variants of the algorithm suited to each setting. We provide benchmarks showing that the algorithm is very fast in practice. Our research is based on two independent implementations: an open-source lexer generator RE2C and an experimental Java library.