Fine-tuning, LoRA and QLoRA

Some problems don't go away with better instructions. The model returns valid JSON nine times out of ten, sorts tickets well except into your internal categories, or answers correctly but never in your brand's voice. Fine-tuningFine-tuning Further training a pretrained model on your own examples to change how it behaves. means continuing to train an existing model on your own examples, so it changes how it responds.

The obstacle used to be cost: adjusting every number in a model with billions of them takes several large graphics cards. LoRALoRA Low-rank adaptation: a cheaper way to fine-tune that trains small add-on matrices instead of all of the model's weights. and QLoRAQLoRA LoRA training on a compressed (quantized) copy of the model, so even large models can be fine-tuned on a single GPU. made it possible on one. This lesson covers when fine-tuning is the right tool, what a good dataset looks like, how LoRA and QLoRA work, and the mistakes that most often ruin the result.

Behavior or knowledge?

Before trainingTraining Adjusting model weights from data so its predictions improve. anything, ask what is missing. If the model lacks knowledge (facts, prices, policies), hand it the right documents at question time: that is RAGRAG Retrieval-augmented generation: finding relevant passages in your documents and adding them to the prompt before the model answers., covered in Embeddings and RAG. If it lacks a behavior (a fixed format, your own labels, a voice), that can be trained in.

The dataset

Fine-tuning learns from examples: an input and the ideal output for it. For a ticket classifier, one example pairs "I was charged twice" with "billing". This is called supervised fine-tuning (SFT).

Quality beats quantity: contradictory examples teach contradictions. Each example should teach a repeatable shape (a structure, a label set, a template), not an isolated fact. That is the risk of having a model generate examples from your documents: it tends to produce question-and-answer pairs about facts.

LoRA and QLoRA

Train a small correction, not the whole model

A model's knowledge lives in large grids of numbers called weights. Classic fine-tuning adjusts all of them, which needs enormous memory and produces a full copy of the model.

LoRA (Low-Rank Adaptation) freezes the original weights and learns a small correction that sits beside them. The insight is that one specific behavior does not require changing everything; a compact correction captures most of it. In a typical layer it has under one percent as many numbers as the layer itself.

The result is an adapterAdapter A small set of extra weights trained and attached to a model that otherwise stays unchanged, so its behavior changes without retraining the whole model.: a file of a few megabytes applied on top of an unchanged base modelBase model A model after its first, general training, which only continues text and has not yet been tuned to follow instructions.. One base can serve many adapters, one per task.

QLoRA: fitting on one card

Even if only the adapter trains, the whole base model must sit in the graphics card's memory. QLoRA loads that frozen base quantized, compressed to 4 bits per number, and trains the adapter at higher precision.

What usually goes wrong

Memorizing instead of learning. With few examples the model memorizes them: training numbers keep improving while results on new examples stall or worsen. This is overfittingOverfitting A model memorizing its training data instead of learning patterns that generalize.. The defenses are testing on examples it never saw (Evaluating models), training for only a few passes, and a negative control: a dataset with no pattern, which must fail.

Training on one format and serving another. Chat models read conversations wrapped in special markers, their chat template. If training shows plain "User: … Assistant: …" text but the live system sends the template, the adapter meets input unlike anything it learned.

From training to serving, and images

Training saves the adapter in one file format; the llama.cpp engine that serves it expects another, GGUFGGUF A single-file model format used by llama.cpp and similar tools to run models locally, holding the weights and their metadata, usually quantized. (Serving models). The conversion can fail, so treat it as a pipeline step. In Adapta, a failed conversion fails the whole job: an adapter that cannot be served is never registered.

Models that read images are tuned the same way. The usual approach freezes the part that reads the image and trains only the language side: the model can already see; it learns to answer in your format.

Putting it into practice

• Ask "behavior or knowledge?" first. If it is knowledge, a fine-tune is the wrong investment.
• Data before hardware. The real bottleneck is a clean, consistent set of hundreds of examples and someone responsible for keeping it current.
• Check the base model's license before investing. Several of Adapta's 3B models allow only non-commercial use (README; see Models and Hugging Face).
• Plan the adapter's lifecycle. It is tied to its exact base model; changing the base means training and testing again.

Go deeper

Adapta's own documentation walks through the same pipeline with real runs:

• Learning the system explains LoRA, QLoRA and the training dataset from first principles, and why training runs apart from serving.
• Fine-tuning, proven follows one real run, a ticket classifier trained on 36 examples, from dataset to a live endpoint.
• Fine-tuning use cases records GPU runs for extraction, classification, a fixed format and a negative control, with notes on how to read the scores.
• Fine-tune models that read your images covers the image path: how the dataset is packaged, what stays frozen and how the result is served.

Key takeaways

• Fine-tuning changes behavior; RAG supplies knowledge.
• LoRA trains a small correction beside a frozen model, so adapters weigh megabytes and one base serves many tasks.
• QLoRA compresses the frozen base so training fits on a modest graphics card.
• Training and serving must use the same input format, and every adapter is tested on unseen examples before it goes live.

Check yourself