Yi Wu

Hongyu Wang, Shuming Ma, Li Dong et al. · microsoft-research

The increasing size of large language models has posed challenges for deployment and raised concerns about environmental impact due to high energy consumption. In this work, we introduce BitNet, a scalable and stable 1-bit Transformer architecture designed for large language models. Specifically, we introduce BitLinear as a drop-in replacement of the nn.Linear layer in order to train 1-bit weights from scratch. Experimental results on language modeling show that BitNet achieves competitive performance while substantially reducing memory footprint and energy consumption, compared to state-of-the-art 8-bit quantization methods and FP16 Transformer baselines. Furthermore, BitNet exhibits a scaling law akin to full-precision Transformers, suggesting its potential for effective scaling to even larger language models while maintaining efficiency and performance benefits.

Wei Zhang, Yi Wu, Zhishuog Zhang et al.

Attribute-based encryption (ABE) is a promising tool for implementing fine-grained access control.To solve the matters of security in single authority, access policy public, not traceable of malicious user,we proposed a scheme of multi-authority. Moreover, multi-authority may bring about the collusion of different authorities.In order to solve these problem,we proposed a scheme of access tree structure with policy hidden and access complex.Once the private key is leaked, our scheme can extract the user ID and find it.If the authorities share their information with each other,the scheme avoid them to combine together to compute the key information and decrypt the ciphertext.Finally,the scheme proved to be secure under selective-set of IND-CPA.