LLMs for Multi-Modal Knowledge Extraction and Analysis in Intelligence/Safety-Critical Applications
It addresses the problem of enabling safe LLM deployment in high-stakes domains for researchers and practitioners, but it is incremental as it synthesizes existing work without new methods or data.
This paper reviews literature on LLM vulnerabilities and assessment to synthesize the research landscape, aiming to identify critical advances needed for their safe use in intelligence and safety-critical applications, categorizing vulnerabilities into ten types and mapping them to an LLM life cycle.
Large Language Models have seen rapid progress in capability in recent years; this progress has been accelerating and their capabilities, measured by various benchmarks, are beginning to approach those of humans. There is a strong demand to use such models in a wide variety of applications but, due to unresolved vulnerabilities and limitations, great care needs to be used before applying them to intelligence and safety-critical applications. This paper reviews recent literature related to LLM assessment and vulnerabilities to synthesize the current research landscape and to help understand what advances are most critical to enable use of of these technologies in intelligence and safety-critical applications. The vulnerabilities are broken down into ten high-level categories and overlaid onto a high-level life cycle of an LLM. Some general categories of mitigations are reviewed.