San-BERT: Extractive Summarization for Sanskrit Documents using BERT and it's variants
This work addresses the challenge of automated text summarization for Sanskrit, a low-resource language, by creating and releasing a public corpus and adapting existing NLP methods, representing an incremental advancement in domain-specific NLP.
The authors tackled the problem of extractive summarization for Sanskrit documents by developing BERT-based language models (BERT, ALBERT, RoBERTa) from a Devanagari Sanskrit corpus and applying dimensionality reduction and clustering to features, achieving results that demonstrate the feasibility of this approach for Sanskrit text summarization.
In this work, we develop language models for the Sanskrit language, namely Bidirectional Encoder Representations from Transformers (BERT) and its variants: A Lite BERT (ALBERT), and Robustly Optimized BERT (RoBERTa) using Devanagari Sanskrit text corpus. Then we extracted the features for the given text from these models. We applied the dimensional reduction and clustering techniques on the features to generate an extractive summary for a given Sanskrit document. Along with the extractive text summarization techniques, we have also created and released a Sanskrit Devanagari text corpus publicly.