Cleaning Logs for Downstream Tasks (Registered Report)
For practitioners dealing with noisy logs, this work addresses the problem of irrelevant log messages degrading downstream task performance, but it is an incremental approach as it only presents a plan without empirical validation.
This study proposes LogPurifier, a task-agnostic log-cleaning method that removes free-standing messages based on dependency relationships between log templates, aiming to improve the effectiveness and efficiency of downstream tasks like model inference and anomaly detection. The paper presents a plan for empirical evaluation but does not report results.
Background: Software systems generate logs during execution to record critical events and runtime information for troubleshooting and monitoring. However, in practice, logs often contain significant amounts of redundant and irrelevant information, which can negatively impact the performance of downstream analysis tasks, such as model inference and anomaly detection. Objective: The objective of this study is to clean log data by identifying and removing free-standing messages -- messages that are not relevant to the execution behaviors of interest and are interleaved with messages capturing the system's functional behavior. Method: To address this objective, we propose LogPurifier, a task-agnostic log-cleaning approach based on dependency relationships between log message templates. The paper presents a plan for an empirical evaluation using a controlled experimental design to assess the impact of LogPurifier on the effectiveness and efficiency of two downstream tasks: model inference and anomaly detection.