The single biggest determinant of a fine-tuned model’s quality is its training data. Training methods, hyperparameters and even the choice of base model matter less than whether your examples are correct, consistent and representative. Yet dataset preparation is often rushed, because it’s the least exciting part of the project.
This guide walks through how we prepare datasets for fine-tuning language models on business tasks — classification, extraction, drafting in a particular style, structured transformations and similar.
Step 1: Define the task precisely
Before collecting a single example, write down:
- The input the model will receive in production, exactly as it will arrive. If the production input includes a system prompt, metadata or retrieved context, so should the training examples.
- The desired output, including format. If it’s JSON, specify the schema. If it’s prose, specify length, tone and structure.
- What good looks like, in a way two different people would agree on.
- The edge cases: empty inputs, irrelevant inputs, ambiguous inputs, inputs in other languages. What should the model do with each?
This becomes the specification for your dataset and the basis for your labelling guidelines.
Step 2: Source real examples
The best training data comes from your actual operations:
- Historical records where humans already performed the task — categorised tickets, extracted invoice fields, approved replies, edited drafts.
- Expert annotations, where specialists label or write outputs for a sample of real inputs.
- Production logs from an existing AI feature, with corrections from users or reviewers.
Historical data is valuable but must be inspected. Past decisions may be inconsistent, outdated or wrong. A support team’s historical categories may have been applied loosely; old replies may reference policies that changed.
Step 3: Write labelling guidelines
If examples need to be created or corrected by people, write guidelines with:
- A clear definition of each label or output requirement.
- Positive and negative examples.
- Rules for edge cases and ties.
- What to do when unsure (flag rather than guess).
Pilot the guidelines: have two or three people label the same 50–100 items independently. Where they disagree, the guidelines are usually unclear. Revise and repeat until agreement is high. This step feels slow but saves enormous rework.
Step 4: Clean and normalise
Common cleaning steps:
- Remove broken records: truncated inputs, empty outputs, encoding errors.
- Normalise formatting: consistent whitespace, quote styles, date formats where they don’t carry meaning.
- Strip or mask sensitive data that the model shouldn’t learn — personal data, secrets, internal identifiers — unless your use case and legal basis require it.
- Fix label errors found in audits.
- Remove outdated examples that reflect old policies or products.
Step 5: Deduplicate
Duplicates and near-duplicates skew training toward over-represented patterns and, worse, can leak into your test set, making results look better than they are. Remove exact duplicates, then use similarity measures (for example, embeddings or MinHash) to find near-duplicates and decide whether to keep one representative.
Step 6: Balance and cover
Look at the distribution of your data:
- Are some categories or output types heavily over-represented?
- Are important rare cases — the ones that matter most when they occur — present at all?
- Does the data cover the variety of inputs seen in production: lengths, languages, channels, customer types?
You don’t always need perfect balance, but the model must see enough examples of each important behaviour. Under-represented cases are where synthetic data can help.
Step 7: Use synthetic data carefully
Language models can generate additional examples: variations of real inputs, rare edge cases, alternative phrasings or entire conversations. Done well, this improves coverage. Done carelessly, it teaches the model the generator’s quirks and errors.
Guidelines we follow:
- Seed from real examples so synthetic data stays close to the real distribution.
- Vary deliberately — personas, tones, lengths, error types — rather than asking for “more examples”.
- Validate every synthetic example, with automated checks (format, schema, rules) and human review of samples.
- Filter aggressively: discard low-quality and near-duplicate generations.
- Keep the test set real. Never evaluate on synthetic data produced by a similar model; it will flatter results.
- Check the generator’s licence terms, as some model providers restrict using outputs to train other models.
Step 8: Format for training
Most fine-tuning frameworks expect conversational records: a list of messages with roles (system, user, assistant). Use the chat template of your chosen base model; mismatched templates are a common cause of poor results. Keep the system prompt consistent with what you’ll use in production. For structured outputs, make sure every example’s output is valid against the schema — a few malformed examples can teach the model to produce malformed output.
Step 9: Split properly
Create at least:
- Training set — the bulk of the data.
- Validation set — for monitoring training and choosing checkpoints.
- Test set — held out completely until final evaluation.
Split in a way that reflects real use. If the model will see new customers, make sure test examples come from customers not in training. If data changes over time, consider a time-based split. Ensure no near-duplicates span splits.
Step 10: Document the dataset
Write a short dataset card: sources, time range, labelling process, cleaning steps, synthetic data share and generation method, known gaps and biases, and intended use. Version the dataset alongside the training code. When a model behaves unexpectedly months later, this documentation is how you’ll understand why.
How much data do you need?
There’s no universal number, but for narrow tasks on modern base models, a few hundred to a few thousand high-quality examples often produce strong results with LoRA or QLoRA. Broader behaviours need more. A useful approach is to train on increasing subsets — 25%, 50%, 100% — and watch the evaluation curve. If quality is still climbing steeply at 100%, more data will help; if it has flattened, focus on quality and coverage instead.
Keeping the dataset alive
A dataset is not finished when the first model ships. Production brings new inputs, new edge cases and new failure modes. Build a simple loop: log production inputs and outputs (with appropriate privacy controls), sample them regularly for review, and route corrected examples back into a staging area for the next dataset version. Keep the test set stable for comparability, but add a separate “recent” test set so you can see whether the model is keeping up with how usage is changing.
Version every dataset release, record which model was trained on which version, and keep a changelog of what was added or removed and why. When you retrain on a newer base model later, this history lets you reproduce and improve on past results instead of starting over.
Privacy and consent
Training data often comes from real customer interactions, which raises privacy questions that are easy to overlook. Check that you have a lawful basis to use the data for training, that contracts with customers or partners permit it, and that personal data is minimised or removed where the task doesn’t require it. Remember that models can sometimes reproduce fragments of their training data, so anything included should be acceptable to appear in an output. When in doubt, mask names, contact details and identifiers with consistent placeholders — the model learns the pattern without learning the person.
A final quality check
Before training, read a random sample of a hundred examples yourself. It’s the cheapest quality assurance there is, and it reliably catches problems no script found: a field that’s always empty, outputs that contradict the guidelines, or a whole batch with the wrong format.
Need help building a dataset? Our synthetic data and labelling and fine-tuning services cover the full pipeline.