BitNet: Scaling 1-bit Transformers for Large Language Models
Research paper by Hongyu Wang and collaborators. Published 2023-10-17. Topics: BitNet, 1-bit Transformer, BitLinear, nn.Linear, 1-bit weights.
Resource information
- Authors
- Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, Furu Wei
- Research topics
- BitNet, 1-bit Transformer, BitLinear, nn.Linear, 1-bit weights, memory footprint, energy consumption, 8-bit quantization, FP16, scaling law, full-precision Transformers
- Publication date
- 2023-10-17
- Publication
- arXiv
- Last record update
- Nov 3, 2026