TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Models
This addresses the problem of ensuring trustworthy and responsible AI for users and developers, but it is incremental as it builds on existing ethical concerns with a new benchmark.
The study tackled the lack of comprehensive evaluation of ethical issues in large language models by introducing TrustGPT, a benchmark that assesses toxicity, bias, and value-alignment, aiming to enhance understanding and promote more ethical model development.
Large Language Models (LLMs) such as ChatGPT, have gained significant attention due to their impressive natural language processing capabilities. It is crucial to prioritize human-centered principles when utilizing these models. Safeguarding the ethical and moral compliance of LLMs is of utmost importance. However, individual ethical issues have not been well studied on the latest LLMs. Therefore, this study aims to address these gaps by introducing a new benchmark -- TrustGPT. TrustGPT provides a comprehensive evaluation of LLMs in three crucial areas: toxicity, bias, and value-alignment. Initially, TrustGPT examines toxicity in language models by employing toxic prompt templates derived from social norms. It then quantifies the extent of bias in models by measuring quantifiable toxicity values across different groups. Lastly, TrustGPT assesses the value of conversation generation models from both active value-alignment and passive value-alignment tasks. Through the implementation of TrustGPT, this research aims to enhance our understanding of the performance of conversation generation models and promote the development of language models that are more ethical and socially responsible.