MLC LLM
Deploy compatible language models locally using a runtime and compiled model artifacts for the target device.
Published by MLC AI
How to use it
Choose an existing supported model build, or prepare weights and compile a model library for the platform you will deploy to. Run it with the local application or runtime interface.
Compiled libraries and quantized weights must match the model configuration and device target. Check a short generation test before packaging a mobile build or comparing runtime speed.
- Hardware
- NVIDIA CUDA, Vulkan, Apple Metal
- Model formats
- MLC weights, MLC compiled model library
- Operating systems
- Linux, Windows, macOS, Android, iOS
- Access
- Runs locally with target-specific compiled model artifacts.