Llama 3 8B preference tuning with LoRA and DPO
An Axolotl recipe that trains Llama 3 adapters from preferred and rejected chat responses.
Published by Axolotl
How to use it
This configuration uses Meta-Llama-3-8B-Instruct and a small Alpaca-derived preference dataset. Direct preference optimization compares paired responses for the same prompt.
Confirm that the chosen and rejected examples use the same prompt and a compatible chat format before replacing the sample data. Review the base model's access terms before downloading weights.
- Framework
- Axolotl
- Dataset
- fozziethebeat/alpaca_messages_2k_dpo_test
- Hardware
- Minimum VRAM unspecified.
- Access
- Public publisher configuration; model and dataset terms apply.