Lost in Translation? Converting RegExes for Log Parsing into Dynatrace Pattern Language
This addresses a costly and error-prone migration issue for companies moving to modern log analytics platforms like Dynatrace, though it is incremental as it builds on existing conversion and optimization techniques.
The paper tackles the problem of manually converting regular expressions (RegExes) to Dynatrace Pattern Language (DPL) for log parsing migration, presenting Reptile, a tool that safely converted 73.7% of 946 RegExes and achieved F1-scores above 0.91 in optimization.
Log files provide valuable information for detecting and diagnosing problems in enterprise software applications and data centers. Several log analytics tools and platforms were developed to help filter and extract information from logs, typically using regular expressions (RegExes). Recent commercial log analytics platforms provide domain-specific languages specifically designed for log parsing, such as Grok or the Dynatrace Pattern Language (DPL). However, users who want to migrate to these platforms must manually convert their RegExes into the new pattern language, which is costly and error-prone. In this work, we present Reptile, which combines a rule-based approach for converting RegExes into DPL patterns with a best-effort approach for cases where a full conversion is impossible. Furthermore, it integrates GPT-4 to optimize the obtained DPL patterns. The evaluation with 946 RegExes collected from a large company shows that Reptile safely converted 73.7% of them. The evaluation of Reptile's pattern optimization with 23 real-world RegExes showed an F1-score and MCC above 0.91. These results are promising and have ample practical implications for companies that migrate to a modern log analytics platform, such as Dynatrace.