How to fine-tune an LLM: a developer's guide

Fine-tuning takes a model that already works and keeps training it on your examples. The mechanics are the ones pretraining uses: run a forward pass, measure next-token loss against the target text, backpropagate, update the weights. What changes is the data and the dose. You train on thousands of curated examples instead of a web crawl, with a learning rate small enough to nudge the weights without overwriting what the model already knows.

Here's how to fine-tune an LLM, in seven steps:

• Decide whether to fine-tune at all
• Pick a base model
• Prepare training data and hold out an eval set
• Choose full fine-tuning or LoRA
• Choose supervised fine-tuning or preference tuning
• Train, then evaluate against a baseline
• Deploy the result and keep measuring

For how the training stages fit together, see how large language models are trained.

When to fine-tune an LLM, and when not to

Most of the time, the answer is not yet. The usual argument against fine-tuning is that base models keep improving and a prompt costs nothing to change, so a tune is effort spent on a moving target. That's right for most projects and wrong for a few.

Try prompting and RAG first

Prompting is instant and reversible. A bad prompt costs a few tokens, while a bad fine-tune costs a training run and the days spent on its data. If the gap is knowledge (the model doesn't know your product catalog, or the facts change weekly), use retrieval-augmented generation (RAG), which puts current documents into the context at request time. Fine-tuning is not a way to load facts into a model. It teaches patterns. The RAG vs. fine-tuning comparison covers that decision in full.

Where fine-tuning wins

Four situations justify the work:

• Format or style a prompt can't lock in: A support model that drifts out of your brand voice by turn four, or a JSON extractor that breaks schema on one call in fifty, learns consistency from examples more reliably than from instructions.
• A long prompt on a large model: If a 70B-class model needs a 10,000-token prompt, a tuned 8B–9B model can often do the task with a one-line prompt, which means lower cost per call and a faster time to first token (TTFT).
• Behavior the base model resists: Over-refusal, verbosity and reflexive disclaimers were trained in, so examples of the behavior you want push back more directly than a system prompt does.
• Tool-calling and domain patterns: Choosing the right internal tool, or sorting tickets into your own taxonomy, is a pattern the model can learn from examples.

The hidden costs

The dataset takes longer than everything else, and a bad one produces a model that is confidently wrong. Without a held-out eval set you can't tell whether the tune helped. Each iteration takes hours to days instead of seconds. The tune stays pinned to its base, so when a better base ships you retrain or fall behind. And the result needs hosting.

Together AI's docs page on when to fine-tune vs. prompt gives a rule of thumb: if a good prompt gets you 90% of the way in a week, fine-tuning buys the last 10% at roughly 10× the cost in time and maintenance. It adds a volume test. At 50 calls a day the savings won't pay back the work. At 50,000 they will.

Pretraining vs. fine-tuning

Pretraining builds the base model through self-supervised next-token prediction on raw text, at a scale few teams will ever run. Meta pretrained its 405B Llama 3 model on 15.6 trillion tokens using up to 16,000 H100 GPUs (Grattafiori et al., 2024). Fine-tuning reuses the same objective at a tiny fraction of that scale. A dataset of 5,000 examples at 1,000 tokens each is 5M tokens per epoch, roughly one three-millionth of that pretraining corpus.

That ratio explains what fine-tuning can and can't do. Five million tokens can redirect skills the model already has. They can't install a new field of knowledge. Continued pretraining sits between the two: you train on raw, unlabeled domain text (contracts, clinical notes, a codebase) to shift the model's vocabulary and priors, usually before an instruction-tuning pass. How large language models are trained walks the stages in order.

Choose a base model

The base comes down to checkpoint type, size, and whether you can serve it afterward.

Default to the instruct checkpoint. Instruction-tuned models ship with a chat template and already follow instructions, which means your data only has to teach the task. Base checkpoints usually have no template, so they suit continued pretraining or formats that look nothing like chat. Reasoning models are a special case. Tuning one on data without reasoning traces can degrade its reasoning, so if your dataset has no chain of thought, start from an instruct model.

On size, pick the smallest model that clears your quality bar after tuning. A tuned 8B–9B model can match a prompted 70B-class model on one narrow task, and it's cheaper and faster on every call afterward. If it plateaus below target, move up a size.

Then check the license, the maximum training context length (it varies by model and is usually shorter for DPO than for SFT) and deployability. A model you can fine-tune is not always one you can host. The docs' supported models list warns that some models can be fine-tuned but not deployed on dedicated endpoints, so confirm the serving side before you pay for training.

Prepare your training data

Data decides the result more than any hyperparameter does. Field names, parameters, and defaults from here on come from Together's fine-tuning docs as of September 2026. Other training stacks use different names for the same ideas.

Data formats

Training files are JSONL, one example per line (wrapped here for readability). Together also accepts pre-tokenized Parquet, which skips tokenization on repeated runs, though not for preference data. Most jobs fit one of the next three formats. Conversational data uses a messages array with system, user and assistant roles, for single-turn or multi-turn chat:

{"messages": [
  {"role": "system", "content": "You are a support agent for Acme. Answer in two sentences or fewer."},
  {"role": "user", "content": "How do I rotate my API key?"},
  {"role": "assistant", "content": "Open Settings > API keys and click Rotate. The old key keeps working for 24 hours."}
]}

Instruction data pairs a prompt with a completion:

{"prompt": "Extract the invoice total as JSON.\n\nInvoice: ... Total due: $1,240.50", "completion": "{\"total\": 1240.50, \"currency\": \"USD\"}"}

Preference data, used for DPO, holds the context under input, plus exactly one preferred and one non-preferred assistant message:

{"input": {"messages": [
   {"role": "user", "content": "Can I get a refund after 30 days?"}
 ]},
 "preferred_output": [
   {"role": "assistant", "content": "No. Refunds close at 30 days, but I can offer account credit. Want me to apply it?"}
 ],
 "non_preferred_output": [
   {"role": "assistant", "content": "I'm sorry, but as an AI I can't help with refund decisions."}
 ]}

Narrower jobs get their own formats. Generic text uses a single text field, for continued pretraining or plain completion. Tool-calling data adds a tools array, with tool_calls on assistant messages. Reasoning data adds a reasoning field to assistant messages. A file has to commit to one format. If its lines could parse as two (both text and messages, say), the server rejects it.

How much data, and how clean

Plan for hundreds of examples at minimum and usually thousands. Consistency matters more than count, because the model copies whatever is consistent in your data, including the mistakes. Deduplicate, because near-identical rows act as extra epochs on those rows. Match the production input distribution. If real users send typos and half-finished questions, train on those. Before training, split off a validation set for loss curves and early stopping, and a test set of 50–200 labeled examples that you never train or tune hyperparameters against.

Loss masking and packing

Two preprocessing defaults change what the model learns. The first is which tokens produce loss. You're teaching the model to write outputs, so by default only assistant messages or the completion field produce loss. This is loss masking. The train_on_inputs parameter controls it, and a per-message weight of 0 or 1 masks individual turns.

The second concatenates short examples up to the maximum sequence length so that no compute goes to padding. This is sample packing. It's on by default for JSONL and it cuts the step count sharply, sometimes too sharply on a small dataset. As an illustration, 2,000 examples of 500 tokens make 1M tokens. Packed into 8,192-token sequences, that's ~122 sequences, or ~15 optimizer steps per epoch at a batch size of 8. With that few steps, add epochs, lower the batch size or turn packing off.

The most common silent bug here is a chat template mismatch. Training applies the base model's template to your messages. If you later serve with a different template, or hand-format special tokens into a text field, the model sees inputs it never saw in training. Nothing errors. Quality just drops.

Validate before you pay

tg files check verifies encoding, JSON parsing, line count and size before you upload. Full schema validation runs on the server after upload and reports through the file's processing_status. Then tg fine-tuning preview shows how rows tokenize under the base model's template, including which tokens contribute to the loss. The data preparation guide has the full schemas.

Full fine-tuning vs. LoRA vs. QLoRA

Full fine-tuning updates every weight, and memory is the constraint. Mixed-precision training with Adam holds about 16 bytes per parameter: fp16 weights and gradients, plus fp32 master weights, momentum and variance (Rajbhandari et al., 2019). For Llama 3.1 8B that is 8.03B × 16 bytes ≈ 128 GB before activations, more than one H100 80GB holds. In exchange every weight can move, which matters when the target is far from the base.

LoRA (Hu et al., 2021) freezes each pretrained weight matrix and learns a low-rank update: two small matrices of rank r whose product is added to the original. Only those receive weight gradients and optimizer state. On a matrix of shape d_out × d_in, the adapter adds r × (d_in + d_out) parameters. At rank 8 on all seven linear projections of Llama 3.1 8B, that's ~21M trainable parameters, or 0.26% of the model, and a ~42 MB adapter in bf16. The knobs are rank, alpha (a scaling factor on the update) and target modules (attention projections only, or all linear layers).

QLoRA (Dettmers et al., 2023) runs LoRA on a base model quantized to 4 bits, which let its authors tune a 65B model on one 48 GB GPU. The cost is extra compute, because the weights are dequantized to bf16 for every forward and backward pass. It's mainly a way to fit a large model onto your own limited hardware.

LoRA does have a ceiling. Biderman et al. (2024) found that it trailed full fine-tuning on code and math at standard ranks, but forgot less of the base model's other abilities. That's the right trade for a first run.

Supervised fine-tuning vs. preference tuning (DPO)

The training method is a separate choice from LoRA vs. full, and either method runs as either type.

Supervised fine-tuning (SFT) learns from one target completion per example. The model reads the input, and the loss pushes up the probability of each token in your reference output. It's the default for a new task, style or format, because a demonstration is the most direct way to show what you want.

Preference tuning learns from the gap between two responses to the same prompt. Direct Preference Optimization, or DPO (Rafailov et al., 2023), raises the probability of the preferred response and lowers the non-preferred one, both measured against a frozen reference copy of the model.

It needs no reward model and no sampling loop, which is what separates it from RLHF.

The dpo_beta parameter sets how far the model may drift from the reference. The documented default is 0.1; lower values update more aggressively and higher values stay closer to the reference.

DPO works best as a polishing step. Pair a specific failure (the unwanted disclaimer, the rambling answer) as the non-preferred output. It's a poor fit for tasks with one correct answer, such as extraction or classification, where SFT on the right answer is simpler. The preference fine-tuning docs list the loss variants exposed as parameters (length normalization, an added NLL term from RPO, SimPO), and the Direct Preference Optimization technical deep dive explains the loss.

When your data is far from what the base model saw in pretraining, run the two in sequence. Run SFT on the prompts plus preferred outputs, then start DPO from that checkpoint by passing it as from_checkpoint, which is continued fine-tuning from a checkpoint. Beyond both sits reinforcement learning against verifiable rewards, used for math, code and agentic tasks, which is a larger project than either.

Hyperparameters that matter

The values below start from the documented defaults, with the LoRA ranges taken from published recipes. Treat them as starting points.

Learning rate is the most sensitive of these. Full fine-tunes sit around 1e-5, which is also the documented default for both training types. Published LoRA recipes often run higher (the QLoRA paper used 2e-4 for 7B and 13B models and 1e-4 for 33B and 65B), because the pretrained weights are frozen and only the small adapter moves. Start at the documented default and raise it only if validation loss barely moves.

The documented default is 1 epoch, with a range of 1–20. Repeated passes over a few hundred examples overfit quickly, so watch validation loss, not training loss. Rows longer than the max sequence length are truncated, and a truncated row can lose the answer tokens, so filter or split long rows.

You don't have to guess the epoch count. Set it high, supply a validation file, set n_evals high enough to detect a plateau, and switch on early stopping, which is off by default. It halts the job once validation loss stops improving for a set number of evaluations. The best checkpoint becomes the final model, and unused steps are refunded.

Evaluate before and after

Baseline first

Before training, run the untuned base model on your held-out test set, and run the large prompted model you hope to replace on it too. Without both numbers you can't say what training bought you. They can also tell you to skip training. If the prompted small model is within a point of target, fix the prompt instead.

What to measure

Use a task metric wherever one exists: exact match or F1 for extraction and question answering, pass@k for code, schema validity for structured output. For open-ended output, use an LLM judge with a written rubric and spot-check its verdicts by hand.

Add a regression suite of general prompts that the tune was never meant to change. A narrow tune can erode broad skills, a failure called catastrophic forgetting, and the task metric alone won't show it. General methodology lives in the guide to LLM evaluation methods.

Reading training curves

Training loss falling while validation loss rises means overfitting, so stop earlier or add data. Training loss near zero means memorization. A validation loss that never moves points to a learning rate that's too low, or to a loss mask hiding the tokens you meant to train on. For DPO, watch two more metrics. Preference accuracy is the share of pairs where the preferred response earns the higher reward. KL divergence measures distance from the reference model, so a rising value is your cue to run the regression suite.

A 2024 walkthrough on fine-tuning Llama-3 8B to beat Llama-3 70B on math shows the pattern. Llama-3-8B scored 47% on 1,000 held-out math problems before tuning on the MathInstruct dataset and 65% after. Llama-3-70B scored 64% and GPT-4o 71% on the same problems, all graded by an LLM judge. The model names are dated, but the method still holds. Use one held-out set and measure the baseline and the comparison models the same way.

Serve your fine-tuned model

A LoRA job gives you two ways to ship. Merged weights fold the adapter into the base and produce a self-contained model. It runs anywhere the base runs with no added latency, but every tune is a full model copy. An adapter is a small file loaded on top of a shared base at serving time. Many adapters can share one base deployment, which saves GPU memory. Five merged variants of a 9B model in bf16 hold ~18 GB of weights each, or ~90 GB in total. Five adapters on one base hold ~18 GB plus a few hundred MB. The freed memory goes to KV cache memory, which is what sets concurrency. An unmerged adapter adds a little work to every request, because the low-rank update is applied at serving time. A full fine-tune produces full weights only.

On Together, fine-tuned models are served on dedicated endpoints, not through the per-token serverless API. Endpoints bill per minute per running replica, even when idle. Deploy a fine-tuned model covers the flow. For supported base models, the docs also describe how to serve multiple adapters on one endpoint, a feature marked as beta. Or download the merged weights or the adapter and self-host with vLLM or Hugging Face PEFT.

Either way, serve at the precision you evaluated. Quantizing after tuning changes the model, so re-run the eval set on the quantized version. For what happens at request time, see how LLM inference works.

What fine-tuning costs

Training is billed per token processed. The docs on how fine-tuning jobs are billed give the count as:

total_tokens = (n_epochs × training tokens) + (n_evals × validation tokens)

The rate per token depends on the model's size bracket, the method (SFT or DPO) and the type (LoRA or full). As an example with placeholder inputs, a 10M-token dataset trained for 3 epochs is 30M tokens. Ten evaluations on a 500K-token validation set add 5M, for 35M billed tokens. Multiply that by the rate R for your model, taken from the current per-token rates. The CLI and the estimate-price endpoint do this arithmetic before you submit. Cancelled and early-stopped jobs pay for completed steps only, and failed jobs are refunded.

Hosting is the larger line item over time. An endpoint left up all month runs ~720 hours, used or not, so weigh 720 × the hourly rate against what the tune saves. Replacing a 10,000-token prompt with a 100-token one at 50,000 calls a day removes 9,900 × 50,000 = 495M input tokens a day.

Four levers cut the training bill: LoRA instead of full fine-tuning, fewer epochs with early stopping, a smaller base model, and packing left on. With packing disabled, billed training tokens become rows × max sequence length, which means you pay for padding.

Fine-tune models on Together AI

If you'd rather not run the training stack yourself, Together's managed service for fine-tuning LLMs covers this workflow. You upload a JSONL file, train with LoRA or full fine-tuning using SFT or DPO, track validation metrics, then deploy to a dedicated endpoint or download the weights. The fine-tuning quickstart runs the whole loop on the CoQA conversational dataset in roughly 20 to 40 minutes.

FAQ

What is LLM fine-tuning? It's continued training of a pretrained model on task-specific examples so that its weights change. Prompting steers the same weights with text, and pretraining creates them from raw text.

When is fine-tuning worth it? It pays off for format and style consistency, for moving a high-volume task from a large prompted model to a small tuned one, and for behavior prompts can't change. It is rarely the first thing to try, because prompting and retrieval are cheaper to test.

How much data do you need to fine-tune an LLM? Hundreds of examples at minimum and typically thousands, plus 50–200 held-out eval examples. Clean examples that match production inputs beat a larger noisy set.

Should I use LoRA or full fine-tuning? Start with LoRA. Move to full fine-tuning only when a LoRA run with higher rank and alpha still plateaus below your target.

Does fine-tuning teach a model new knowledge? It shifts patterns and can absorb some domain vocabulary, but it's unreliable for facts. Facts that change belong in retrieval; see RAG vs. fine-tuning.

What is catastrophic forgetting and how do you prevent it? It's the loss of general capability when a model is overtrained on narrow data. Use a lower learning rate, fewer epochs and LoRA, and run a general-capability regression eval before you ship.