Challenges and Thrills of Legal Arguments
This addresses a specific bottleneck in natural language processing for legal or conversational AI applications, but appears incremental as it extends existing transformer methods.
The paper tackled the problem of inter-sequence attention in conversation-like scenarios, which existing attention-based models do not address, by proposing HumBERT for continuous contextual argument generation, but no concrete results or numbers are provided.
State-of-the-art attention based models, mostly centered around the transformer architecture, solve the problem of sequence-to-sequence translation using the so-called scaled dot-product attention. While this technique is highly effective for estimating inter-token attention, it does not answer the question of inter-sequence attention when we deal with conversation-like scenarios. We propose an extension, HumBERT, that attempts to perform continuous contextual argument generation using locally trained transformers.