OMPILOT: Harnessing Transformer Models for Auto Parallelization to Shared Memory Computing Paradigms
This work addresses the challenge of efficient code parallelization for developers in high-performance computing, though it is incremental as it builds on existing LLM-based code translation techniques.
The paper tackles the problem of automatically parallelizing C++ code for shared-memory computing by introducing OMPILOT, a transformer model that translates C++ to OpenMP, achieving improved accuracy and robustness over traditional methods, as measured by a new metric called OMPBLEU.
Recent advances in large language models (LLMs) have significantly accelerated progress in code translation, enabling more accurate and efficient transformation across programming languages. While originally developed for natural language processing, LLMs have shown strong capabilities in modeling programming language syntax and semantics, outperforming traditional rule-based systems in both accuracy and flexibility. These models have streamlined cross-language conversion, reduced development overhead, and accelerated legacy code migration. In this paper, we introduce OMPILOT, a novel domain-specific encoder-decoder transformer tailored for translating C++ code into OpenMP, enabling effective shared-memory parallelization. OMPILOT leverages custom pre-training objectives that incorporate the semantics of parallel constructs and combines both unsupervised and supervised learning strategies to improve code translation robustness. Unlike previous work that focused primarily on loop-level transformations, OMPILOT operates at the function level to capture a wider semantic context. To evaluate our approach, we propose OMPBLEU, a novel composite metric specifically crafted to assess the correctness and quality of OpenMP parallel constructs, addressing limitations in conventional translation metrics.