Google's Gemma 2 2B, QLoRA fine-tuned on 10,000 retrieval-augmented examples — golden passage plus distractors — to answer from the right source and refuse when the answer is absent.
The Gemma base given the RAFT habit through a QLoRA adapter: answer from the passage among distractors, refuse when it is not there. Uses the same 10,000-example RAFT dataset as the 500M model, assembled to fit context.
2 epochs (1,250 steps, ~2.3 h on an L4). Judged correctness 0.975 [0.963–0.986] — the best in the whole family. The four-condition test shows a real, significant grounding gap of +0.083 F1 (having the right document measurably helps) and 84% correct abstention when the answer is absent.
| Class | Gemma2ForCausalLM |
| Layers | 26 |
| Hidden size | 2,304 |
| Attention | 8 heads / 4 KV · dim 256 · GQA |
| Feed-forward | GeGLU · inner 9,216 |
| Attention window | sliding 4,096 (alternating) · logit soft-cap |
| Norm | RMSNorm |
| Context | 8,192 tokens |
| Vocabulary | 256,128 |
| Embeddings | tied input/output |
| Init from | gemma-2-2b-it · 4-bit NF4 |
| Method | QLoRA · rank-16 |
| Trainable params | 20.8M LoRA (0.8%) |
| Training data | 10,000 RAFT examples · 2 epochs |
| Training tokens | 16.4M (8.2M/epoch × 2) |
| Optimizer steps | 1,250 |
our cost begins at fine-tuning — the base is Google's, imported free.
| Stage | Detail | Cost |
|---|---|---|
| QLoRA RAFT fine-tune | 1,250 steps on A100-40GB | $6.82 |
| Evaluation (shared) | 13 versions on 500 held-out questions, this model's share | $1.48 |
| Total | $8.30 |
Figures are Modal GPU usage (time × rate) across this model's lineage; shared datasets are charged at this model's share. Whether base pretraining is included is stated above — it is for the models pretrained here, and excluded for imported bases. Evaluation-derived metrics come from an independent blind-judge harness on a frozen, decontaminated held-out set. Serving is billed separately and scales to zero.