Gemma 3 1B instruction tuning with QLoRA
A compact Axolotl configuration for instruction tuning a 4-bit Gemma 3 checkpoint with trainable adapters.
Published by Axolotl
How to use it
The example uses google/gemma-3-1b-it and SlimOrcaDedupCleaned. It sets the Gemma 3 chat format and enables gradient checkpointing, providing a small starting point for adapting an instruction model.
Use the YAML with Axolotl's train command after obtaining access to the checkpoint. For your own data, check a few formatted conversations and reserve a validation split before starting a long run.
- Framework
- Axolotl
- Dataset
- cgato/SlimOrcaDedupCleaned
- Hardware
- Quantized training configuration. Minimum VRAM unspecified.
- Access
- Public publisher configuration; model and dataset terms apply.