Wenhui Zhang

Bertie Vidgen, Adarsh Agrawal, Ahmed M. Ahmed et al. · deepmind, oxford

This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models. We introduce a principled approach to specifying and constructing the benchmark, which for v0.5 covers only a single use case (an adult chatting to a general-purpose assistant in English), and a limited set of personas (i.e., typical users, malicious users, and vulnerable users). We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0.5 benchmark. We plan to release version 1.0 of the AI Safety Benchmark by the end of 2024. The v1.0 benchmark will provide meaningful insights into the safety of AI systems. However, the v0.5 benchmark should not be used to assess the safety of AI systems. We have sought to fully document the limitations, flaws, and challenges of v0.5. This release of v0.5 of the AI Safety Benchmark includes (1) a principled approach to specifying and constructing the benchmark, which comprises use cases, types of systems under test (SUTs), language and context, personas, tests, and test items; (2) a taxonomy of 13 hazard categories with definitions and subcategories; (3) tests for seven of the hazard categories, each comprising a unique set of test items, i.e., prompts. There are 43,090 test items in total, which we created with templates; (4) a grading system for AI systems against the benchmark; (5) an openly available platform, and downloadable tool, called ModelBench that can be used to evaluate the safety of AI systems on the benchmark; (6) an example evaluation report which benchmarks the performance of over a dozen openly available chat-tuned language models; (7) a test specification for the benchmark.

Wenhui Zhang, Trent Jaeger, Peng Liu

Over the years, the complexity of the Linux Security Module (LSM) is keeping increasing, and the count of the authorization hooks is nearly doubled. It is important to provide up-to-date measurement results of LSM for system practitioners so that they can make prudent trade-offs between security and performance. This work evaluates the overhead of LSM for file accesses on Linux v5.3.0. We build a performance evaluation framework for LSM. It has two parts, an extension of LMBench2.5 to evaluate the overhead of file operations for different security modules, and a security module with tunable latency for policy enforcement to study the impact of the latency of policy enforcement on the end-to-end latency of file operations. In our evaluation, we find opening a file would see about 87% (Linux v5.3) performance drop when the kernel is integrated with SELinux hooks (policy enforcement disabled) than without, while the figure was 27% (Linux v2.4.2). We found that performance of the above downgrade is affected by two parts, policy enforcement and hook placement. To further investigate the impact of policy enforcement and hook placement respectively, we build a Policy Testing Module, which reuses hook placements of LSM, while alternating latency of policy enforcement. With this module, we are able to quantitatively estimate the impact of the latency of policy enforcement on the end-to-end latency of file operations by using a multiple linear regression model and count policy authorization frequencies for each syscall. We then discuss and justify the evaluation results with static analysis on our enhanced syscalls' call graphs.

Juan Wang, Chengyang Fan, Jie Wang et al.

Virtual Trusted Platform Modules (vTPMs) have been widely used in commercial cloud platforms (e.g. Google Cloud, VMware Cloud, and Microsoft Azure) to provide virtual root-of-trust for virtual machines. Unfortunately, current state-of-the-art vTPM implementations are suffering from confidential data leakage and high performance overhead. In this paper, we present SvTPM, a secure and efficient software-based vTPM implementation based on hardware-rooted Trusted Execution Environment (TEE), providing a whole life cycle protection of vTPMs in the cloud. SvTPM offers strong isolation protection, so that cloud tenants or even cloud administrators cannot get vTPM's private keys or any other sensitive data. In SvTPM, we identify and solve a couple of critical security challenges for vTPM protection with SGX, such as NVRAM replacement attack, rollback attacks, trust establishment, and a fine-grained trusted clock. We implement a prototype of SvTPM on both QEMU and KVM. Performance evaluation results show that SvTPM achieves orders of magnitude of performance gains comparing to the vTPMs protected with physical TPM. The launch time of SvTPM is 2600$\times$ faster than vTPMs built upon hardware TPM. In the micro-benchmarks evaluation, we find that the command execution latency of SvTPM is smaller than or equal to the existing schemes.

Jing Ma, Wenhui Zhang

Security of complex systems is an important issue in software engineering. For complex computer systems involving many actors, security protocols are often used for the communication of sensitive data. Actor key compromise (AKC) denotes a situation where the long-term secret key of an actor may be known to an adversary for some reasons. Many protocols are not secure enough for ensuring security in such a situation. In this paper, we further study this problem by looking at potential types of attacks, defining their formal properties and providing solutions to enhance the level of security. As case studies, we analyze the vulnerabilities (with respect to potential AKC attacks) of practical protocols, including PKMv2RSA and Kerberos, and provide solutions to enhance the level of security of such protocols.