On the Importance of Karaka Framework in Multi-modal Grounding
This work addresses a gap in applying CPG to multi-modal vision-language applications, which could improve semantic understanding in tasks like navigation, but it is incremental as it builds on existing grammar models.
The paper investigates the unexplored role of the Computational Paninian Grammar (CPG) dependency scheme in multi-modal grounding, specifically testing its advantages and disadvantages in a vision-language navigation task.
Computational Paninian Grammar model helps in decoding a natural language expression as a series of modifier-modified relations and therefore facilitates in identifying dependency relations closer to language (context) semantics compared to the usual Stanford dependency relations. However, the importance of this CPG dependency scheme has not been studied in the context of multi-modal vision and language applications. At IIIT Hyderabad, we plan to perform a novel study to explore the potential advantages and disadvantages of CPG framework in a vision-language navigation task setting, a popular and challenging multi-modal grounding task.