AI Model Fine-Tuning Tutorial: From Dataset Preparation to LoRA Training

AI Model Fine-Tuning Tutorial: From Dataset Preparation to LoRA Training

Let's start with a real scenario: you are building a customer-service bot for your company. With GPT-4o or Claude directly, generic answers are fine, but the bot fumbles your own product terms and its tone always sounds vaguely corporate. You can keep stacking prompts, or you can fine-tune a small model on a few hundred real support conversations, run it in-house with faster responses and no per-token billing. This article is about the second path: from cleaning up a dataset to getting a model trained with LoRA.

General-purpose large models often underperform in specific domains. Fine-tuning allows developers to customize general models with their own data, significantly improving performance in specific scenarios.

First, Get the Choice Right: Fine-Tuning / Prompt Engineering / RAG

To decide, ask three questions: is there a stable "style or format requirement" on the output? Does the knowledge change frequently? Is there enough data (at least a few hundred items)? Only when the answer is "fixed style, stable knowledge, enough data" does fine-tuning pay off. If knowledge changes often, RAG fits better; if you only need occasional rephrasing, prompt engineering suffices.

Dataset Preparation: The Step That Decides Everything

The ceiling of fine-tuning is set by your data. The most common format is the conversation/instruction format:

{
  "messages": [
    {"role": "system", "content": "You are after-sales support for XX Appliances. Answer concisely: conclusion first, then steps."},
    {"role": "user", "content": "My air conditioner remote is not responding. What should I do?"},
    {"role": "assistant", "content": "First check whether the batteries are reversed. Replace them, press the reset button, and if it still does not respond, call the 400 hotline."}
  ]
}

On quantity: more is not always better. A few hundred high-quality conversations covering the main branches often beat tens of thousands of low-quality records. Check four things carefully:

• Accuracy: review every record by hand; wrong answers get "learned in" and amplified;
• Consistency: keep system prompts, tone, and format uniform across the whole set;
• Diversity: cover common cases, edge cases, and the "fallback wording" for refusing to answer;
• Balance: don't let one category take up 80%, or the model will be skewed.

If data is scarce, don't give up immediately: rewrite existing conversations by hand (paraphrase, add steps), or clean question-answer pairs out of old support tickets. Round-trip back-translation (e.g., EN to ZH and back) is also a common augmentation trick.

Full Fine-Tuning / LoRA / QLoRA: Which Route to Take

LoRA's core idea is to decompose the weight update into low-rank matrices:

W' = W + BA

W is frozen and only BA is trained, usually 0.1%–1% of the full parameter count. That is why a consumer 24GB GPU can train a 7B model, and why training time shrinks from "days" to "hours".

Hands-On: Fine-Tuning Llama 3 with LoRA

Using meta-llama/Llama-3.1-8B as an example. First install the dependencies:

pip install torch transformers datasets peft accelerate bitsandbytes

Load the model and configure LoRA:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import get_peft_model, LoraConfig, TaskType

model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B")

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.1,
    task_type=TaskType.CAUSAL_LM,
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()  # usually 0.x% of parameters are trainable

Useful starting values for hyperparameters: learning rate around 2e-4, r between 8 and 16, lora_alpha usually twice r, and 1–3 epochs. If the batch is too small, raise gradient_accumulation_steps; if you run out of memory, enable fp16 or bf16 first:

training_args = TrainingArguments(
    output_dir="./fine-tuned-model",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-4,
    fp16=True,
    save_steps=500,
    logging_steps=50,
)
trainer = Trainer(model=model, args=training_args, train_dataset=dataset)
trainer.train()

The trained LoRA weights take only a few hundred MB, so you can save them separately and merge at deployment time:

from peft import PeftModel
model = PeftModel.from_pretrained(base_model, "./lora-checkpoint")
merged_model = model.merge_and_unload()
merged_model.save_pretrained("./final-model")

A Complete Case: The Support-Bot Fine-Tuning Path

Back to the customer-service bot from the introduction. Suppose you fine-tune Qwen2.5-7B-Instruct with QLoRA on a single 12GB GPU:

1. Clean 800 question-answer pairs out of historical tickets, balanced across "product inquiry / troubleshooting / after-sales policy / cannot answer";
2. Train for 2 epochs with the QLoRA config above, roughly 2–3 hours;
3. Score a held-out 50-example test set by hand, watching whether answers follow the system prompt's format and whether the model fabricates policy;
4. Once quality is good enough, merge the weights and deploy in-house with vLLM or Ollama.

In practice, for this kind of "fixed phrasing + stable domain" use case, a fine-tuned 7B model often approaches a general flagship model while costing maybe a tenth as much per inference. That is why many companies would rather spend a few days fine-tuning than pay per call forever.

Evaluation and Iteration

Don't rush to production after training. Go through at least this checklist:

For a more systematic methodology, see the AI model evaluation guide; to run it cheaper, check the local deployment with Ollama guide.

Common Pitfalls

• Overfitting: too little data or too many epochs; the symptom is "parroting answers" — it fails as soon as the question is rephrased;
• Catastrophic forgetting: the model loses general capabilities learned in pretraining; fewer epochs and a small mix of general data help;
• Format inconsistency: if system prompts differ across training data, the model behaves inconsistently at inference;
• Treating fine-tuning as a silver bullet: when knowledge changes often or data is thin, use RAG and prompt engineering first.

Reference: Hugging Face PEFT docs https://huggingface.co/docs/peft/; LoRA paper https://arxiv.org/abs/2106.09685; QLoRA paper https://arxiv.org/abs/2305.14314