Towards Secure and Private Language Models for Nuclear Power Plants
This work addresses the need for in-house AI solutions that meet cybersecurity and data confidentiality standards in the nuclear industry, though it is incremental in its approach.
This paper tackled the problem of developing a secure and private language model for nuclear power plants by building a domain-specific Large Language Model from the Essential CANDU textbook, achieving encouraging capture of specialized nuclear vocabulary despite limitations in syntactic coherence.
This paper introduces a domain-specific Large Language Model for nuclear applications, built from the publicly accessible Essential CANDU textbook. Drawing on a compact Transformer-based architecture, the model is trained on a single GPU to protect the sensitive data inherent in nuclear operations. Despite relying on a relatively small dataset, it shows encouraging signs of capturing specialized nuclear vocabulary, though the generated text sometimes lacks syntactic coherence. By focusing exclusively on nuclear content, this approach demonstrates the feasibility of in-house LLM solutions that align with rigorous cybersecurity and data confidentiality standards. Early successes in text generation underscore the model's utility for specialized tasks, while also revealing the need for richer corpora, more sophisticated preprocessing, and instruction fine-tuning to enhance domain accuracy. Future directions include extending the dataset to cover diverse nuclear subtopics, refining tokenization to reduce noise, and systematically evaluating the model's readiness for real-world applications in nuclear domain.