Yanhong Xu

DeepSeek-AI, Xiao Bi, Deli Chen et al. · microsoft-research, pku

The rapid development of open-source large language models (LLMs) has been truly remarkable. However, the scaling law described in previous literature presents varying conclusions, which casts a dark cloud over scaling LLMs. We delve into the study of scaling laws and present our distinctive findings that facilitate scaling of large scale models in two commonly used open-source configurations, 7B and 67B. Guided by the scaling laws, we introduce DeepSeek LLM, a project dedicated to advancing open-source language models with a long-term perspective. To support the pre-training phase, we have developed a dataset that currently consists of 2 trillion tokens and is continuously expanding. We further conduct supervised fine-tuning (SFT) and Direct Preference Optimization (DPO) on DeepSeek LLM Base models, resulting in the creation of DeepSeek Chat models. Our evaluation results demonstrate that DeepSeek LLM 67B surpasses LLaMA-2 70B on various benchmarks, particularly in the domains of code, mathematics, and reasoning. Furthermore, open-ended evaluations reveal that DeepSeek LLM 67B Chat exhibits superior performance compared to GPT-3.5.

Yanhong Xu, Reihaneh Safavi-Naini, Khoa Nguyen et al.

Policy-based signatures (PBS) were proposed by Bellare and Fuchsbauer (PKC 2014) to allow an {\em authorized} member of an organization to sign a message on behalf of the organization. The user's authorization is determined by a policy managed by the organization's trusted authority, while the signature preserves the privacy of the organization's policy. Signing keys in PBS do not include user identity information and thus can be passed to others, violating the intention of employing PBS to restrict users' signing capability. In this paper, we introduce the notion of {\em traceability} for PBS by including user identity in the signing key such that the trusted authority will be able to open a suspicious signature and recover the signer's identity should the needs arise. We provide rigorous definitions and stringent security notions of traceable PBS (TPBS), capturing the properties of PBS suggested by Bellare-Fuchsbauer and resembling the "full traceability" requirement for group signatures put forward by Bellare-Micciancio-Warinschi (Eurocrypt 2003). As a proof of concept, we provide a modular construction of TPBS, based on a signature scheme, an encryption scheme and a zero-knowledge proof system. Furthermore, to demonstrate the feasibility of achieving TPBS from concrete, quantum-resistant assumptions, we give an instantiation based on lattices.