BitNet a4.8: 4-bit Activations for 1-bit LLMs
Research paper by Hongyu Wang and collaborators. Published 2024-11-07. Topics: 1-bit Large Language Models (LLMs), BitNet b1.58, BitNet a4.8, hybrid quantization, sparsification.
Resource information
- Authors
- Hongyu Wang, Shuming Ma, Furu Wei
- Research topics
- 1-bit Large Language Models (LLMs), BitNet b1.58, BitNet a4.8, hybrid quantization, sparsification, quantization errors, 4-bit activations, attention layers, feed-forward network layers, 8-bit quantization, INT4/FP4 kernels, 3-bit KV cache
- Publication date
- 2024-11-07
- Publication
- arXiv
- Last record update
- Nov 3, 2026