Can Large Language Models Write Parallel Code?Daniel Nichols, Joshua H. Davis, Zhaojun Xie et al.
Large language models are increasingly becoming a popular tool for software development. Their ability to model and generate source code has been demonstrated in a variety of contexts, including code completion, summarization, translation, and lookup. However, they often struggle to generate code for complex programs. In this paper, we study the capabilities of state-of-the-art language models to generate parallel code. In order to evaluate language models, we create a benchmark, ParEval, consisting of prompts that represent 420 different coding tasks related to scientific and parallel computing. We use ParEval to evaluate the effectiveness of several state-of-the-art open- and closed-source language models on these tasks. We introduce novel metrics for evaluating the performance of generated code, and use them to explore how well each large language model performs for 12 different computational problem types and six different parallel programming models.
6.5CROct 7, 2014
A Covert Channel Based on Web Read-time ModulationJoshua Davis
A network covert channel is created that operates by modulating the time between web resource accesses, with an 'average web user' read-time used as a reference. While the covert channel may be classified as timing based, it does not operate by changing deterministic protocol attributes such as inter-packet delay, as do most timing based network covert channels. Instead, our channel communicates by modulating transaction level read-time, which in the web browsing case has significant non-deterministic components. The channel is thus immune to methods typically used to detect timing based network covert channels.
3.7CRAug 20, 2014
A Proposed System for Covert Communication to Distant and Broad Geographical AreasJoshua Davis
A covert communication system is developed that modulates Morse code characteristics and that delivers its mes- sage economically and to geographically remote areas using radio and EchoLink. Our system allows a covert message to be sent to a receiving individual by hiding it in an existing carrier Morse code message. The carrier need not be sent directly to the receiving person, though the receiver must have access to the signal. Illustratively, we propose that our system may be used as an alternative means of implementing numbers stations.
6.5CRAug 20, 2014
A Covert Channel Using Named ResourcesJoshua Davis, Victor S. Frost
A network covert channel is created that uses resource names such as addresses to convey information, and that approximates typical user behavior in order to blend in with its environment. The channel correlates available resource names with a user defined code-space, and transmits its covert message by selectively accessing resources associated with the message codes. In this paper we focus on an implementation of the channel using the Hypertext Transfer Protocol (HTTP) with Uniform Resource Locators (URLs) as the message names, though the system can be used in conjunction with a variety of protocols. The covert channel does not modify expected protocol structure as might be detected by simple inspection, and our HTTP implementation emulates transaction level web user behavior in order to avoid detection by statistical or behavioral analysis.