Michael J. Freedman

Andrew Or, Haoyu Zhang, Michael J. Freedman

State-of-the-art deep learning systems such as TensorFlow and PyTorch tightly couple the model with the underlying hardware. This coupling requires the user to modify application logic in order to run the same job across a different set of resources, thereby limiting the choice of hardware for a given workload and potentially forcing the user to forgo more efficient hardware configurations. We propose VirtualFlow, a system leveraging a novel abstraction called virtual node processing to decouple the model from the hardware. In each step of training or inference, the batch of input data is split across virtual nodes instead of hardware accelerators (e.g. GPUs and TPUs). Mapping multiple virtual nodes to each accelerator and processing them sequentially effectively time slices the batch, thereby allowing users to reduce the memory requirement of their workloads and mimic large batch sizes on small clusters. Using this technique, VirtualFlow enables many new use cases, such as reproducing training results across different hardware, resource elasticity, and heterogeneous training. In our evaluation, our implementation of VirtualFlow for TensorFlow achieved strong convergence guarantees across different hardware with out-of-the-box hyperparameters, up to 48% lower job completion times with resource elasticity, and up to 42% higher throughput with heterogeneous training.

Marcela S. Melara, Michael J. Freedman, Mic Bowman

Trusted executions environments (TEEs) such as Intel(R) SGX provide hardware-isolated execution areas in memory, called enclaves. By running only the most trusted application components in the enclave, TEEs enable developers to minimize the TCB of their applications thereby helping to protect sensitive application data. However, porting existing applications to TEEs often requires considerable refactoring efforts, as TEEs provide a restricted interface to standard OS features. To ease development efforts, TEE application developers often choose to run their unmodified application in a library OS container that provides a full in-enclave OS interface. Yet, this large-TCB development approach now leaves sensitive in-enclave data exposed to potential bugs or vulnerabilities in third-party code imported into the application. Importantly, because the TEE libOS and the application run in the same enclave address space, even the libOS management data structures (e.g. file descriptor table) may be vulnerable to attack, where in traditional OSes these data structures may be protected via privilege isolation. We present EnclaveDom, a privilege separation system for large-TCB TEE applications that partitions an enclave into tagged memory regions, and enforces per-region access rules at the granularity of individual in-enclave functions. EnclaveDom is implemented on Intel SGX using Memory Protection Keys (MPK) for memory tagging. To evaluate the security and performance impact of EnclaveDom, we integrated EnclaveDom with the Graphene-SGX library OS. While no product or component can be absolutely secure, our prototype helps protect internal libOS management data structures against tampering by application-level code. At every libOS system call, EnclaveDom then only grants access to those internal data structures which the syscall needs to perform its task.