Paying Attention to Attention: Highlighting Influential Samples in Sequential Analysis
This is an incremental improvement for researchers and practitioners using attention-based models in sequential analysis tasks like conversation monitoring.
The paper tackled the problem of uniform attention weights in hierarchical attention networks for real-time conversation monitoring, which prevented meaningful visualization, and developed a method using changes in turn importance over time to create more informative real-time visuals, as confirmed by human reviewers.
In (Yang et al. 2016), a hierarchical attention network (HAN) is created for document classification. The attention layer can be used to visualize text influential in classifying the document, thereby explaining the model's prediction. We successfully applied HAN to a sequential analysis task in the form of real-time monitoring of turn taking in conversations. However, we discovered instances where the attention weights were uniform at the stopping point (indicating all turns were equivalently influential to the classifier), preventing meaningful visualization for real-time human review or classifier improvement. We observed that attention weights for turns fluctuated as the conversations progressed, indicating turns had varying influence based on conversation state. Leveraging this observation, we develop a method to create more informative real-time visuals (as confirmed by human reviewers) in cases of uniform attention weights using the changes in turn importance as a conversation progresses over time.