Fine-tuning open-weight models like Llama and Mistral

What does fine-tuning an open model do?

It continues training a pre-trained model on your own examples so it behaves the way your task needs: the same output format every time, your labels, your terminology and your tone.

For B2B teams the draw is control. A tuned open-weight model can run inside your own cloud account or data center, so prompts, documents and training data do not have to leave it. It also suits high-volume, narrow tasks such as classification, extraction or drafting in a fixed structure.

Which Llama and Mistral models qualify?

Those whose license fits your use, which you check model by model. Both publishers release several families, and the terms differ.

Meta publishes Llama models under version-specific community licenses. The Llama 4 Community License Agreement, effective 5 April 2025, requires anyone distributing the models or products built on them to "prominently display 'Built with Llama'", and a model created or improved with them and then distributed must have a name that starts with "Llama".^[1] A licensee whose products had more than 700 million monthly active users in the calendar month before the model's release date must request a separate license from Meta.^[1] The license incorporates an acceptable use policy, which also states that rights to the multimodal models in Llama 4 are not granted to individuals or companies based in the European Union, though this does not apply to end users of products that include them.^[2]

Mistral's models overview gives each model a license label. Mistral Small 4 and the Ministral 3 models (14B, 8B and 3B) are listed under Apache 2.0, Mistral Medium 3.5 under a modified MIT license, and other entries are labeled "Premier" or carry other terms.^[3] Labels change as models are added and retired, so read the overview and the license file for the exact version you download.

When does retrieval beat fine-tuning?

When the problem is facts, not behavior. The original retrieval-augmented generation (RAG) paper notes that for models storing knowledge only in their parameters, "updating their world knowledge" and showing provenance "remain open research problems."^[4] Retrieval keeps the facts in a store you can update and cite.

Try prompting and retrieval first. If the model has the facts but still breaks your format or labels, fine-tuning is the next step. Our guide to enterprise RAG pipelines covers the retrieval side.

Full fine-tuning or LoRA and QLoRA?

For most B2B tasks, a parameter-efficient method. Full fine-tuning updates every weight, which takes the most memory and produces a full copy of the model for each task.

LoRA freezes the pre-trained weights and trains small low-rank matrices added to each layer. Its authors report that, compared with full fine-tuning of GPT-3 175B, it cut trainable parameters by 10,000 times and GPU memory needs by 3 times, with no added inference latency.^[5]

QLoRA goes further by freezing a 4-bit quantized copy of the base model and training LoRA adapters on top. The paper reports enough memory savings to fine-tune a 65B-parameter model on a single 48GB GPU while keeping the task performance of full 16-bit fine-tuning.^[6]

Hugging Face's PEFT library implements these methods and says they fine-tune only a small number of extra parameters, cutting compute and storage costs while performing comparably to a fully fine-tuned model.^[7] Because adapters are small and separate, you can keep one base model and swap adapters per task.

How should you prepare training data?

Quality and format matter more than volume:

• Match production exactly. Each example should look like a real request and the answer you want, including any system instructions you will use live.

• Use the model's own chat template. Each model family expects its own prompt format; a mismatch quietly degrades results.

• Clean before you train. Remove duplicates, personal data you do not need, and wrong or inconsistent answers.

• Hold out a test set that never enters training.

Our guide to data preprocessing for LLMs covers cleaning in more depth.

How do you evaluate a tuned model?

Against the untuned model, on your held-out set, with the same scoring for both. Score what the business needs, such as correct labels, valid JSON or required fields present, and have a person review a sample of open-ended outputs.

Also test what you did not train for. A model tuned on a narrow dataset can get worse at general tasks it previously handled, so keep a small set of everyday prompts and check them after each run. Version every adapter with its data and settings so you can roll back.

Where should the model run?

Where your data rules allow. Self-hosting in your own cloud account or data center keeps prompts and training data under your access controls, logs and retention rules. A managed hosting service can be simpler, but read where it stores data and whether it trains on it.

Smaller models are easier to host close to the data; see small language models at the edge. Treat training data as a governed asset with access control, as covered in AI training governance and security.

Where should a B2B team start?

1. Write down the task, the output you expect and how you will score it.

2. Try prompting and retrieval on a base model first.

3. Pick a model whose license fits your use and markets.

4. Build a clean dataset and a held-out test set.

5. Train a LoRA or QLoRA adapter, compare it with the base model, then decide.

Our custom AI agents service covers these steps as part of our AI consulting work.