LLM & generative AI
Instruction–response pairs, ranked comparisons, multi-turn dialogue and safety data, written by Japan-based annotators against a style guide your team has signed off.
The hard part of Japanese instruction data is not writing an answer. It is writing the same answer, in the same register, ten thousand times — and being able to say in writing why that register was chosen.
Instruction data written by many people drifts. One annotator ends every response with ですます, another switches to である halfway through a technical explanation, a third writes in the clipped style of a chat assistant. A model trained on the mixture learns that all three are acceptable, and then produces whichever one it feels like — which is the single most common complaint we hear about Japanese fine-tunes.
Register is not the only axis that drifts. Refusals are worse. Japanese declines indirectly, and an annotator who has not been told what a refusal should look like will write anything from a flat お断りします to a soft それは難しいかもしれません. If half the safety data refuses softly enough to be mistaken for compliance, the model learns to hedge instead of to refuse.
So the deliverable is never just the data. It is the data plus the written decisions that produced it: the style guide, the refusal template, the formatting rules, and the version of each that a given batch was written against.
At a glance
Dataset types
Five dataset families, usually combined. Each has a different definition of "correct", so each gets its own guideline section and its own review pass.
Prompt–response pairs that demonstrate the behaviour you want, written from scratch or rewritten from your existing content.
Two or more candidate responses compared against written criteria, with the reason for the choice recorded rather than inferred.
Conversations where context accumulates — the register set in turn one has to survive to turn eight, which is where most Japanese dialogue data falls apart.
Prompts that should be declined, prompts that look like they should be declined but should not, and the refusal wording that separates them.
Question, retrieved passage and grounded answer triples, plus the negative cases where the passage does not actually contain the answer.
Japanese specifics
These are the four decisions that, left unmade, produce a dataset that looks fine in review and behaves badly in the model.
Japanese forces a politeness choice in every sentence ending. ですます, である and だ are not interchangeable, and a model trained on a mixture will mix them inside a single response.
How we handle it The style guide fixes one default register per product surface, lists the exceptions, and gives worked examples of each. Register consistency is a scored item in review, not a matter of taste.
それはちょっと難しいです is a refusal, not a statement of difficulty. Annotators who label it literally teach the model to read a decline as a neutral remark.
How we handle it Refusal categories and their wording are defined up front, with the intended strength of each. Safety data is reviewed specifically for whether a refusal reads as a refusal to a Japanese speaker.
Japanese drops subjects that context makes obvious. In multi-turn data the referent can be four turns back, and an annotator writing a response in isolation will guess wrong.
How we handle it Multi-turn items are written and reviewed as whole conversations, never turn by turn. Where the referent is genuinely ambiguous, the conversation is either fixed or discarded — not labelled with a guess.
The same word appears in kanji, hiragana or katakana; numbers and punctuation appear full-width or half-width. Uncontrolled, this teaches the model that formatting is random.
How we handle it A normalization convention covers script choice for common words, numerals, punctuation, spacing around Latin text and list formatting. It is applied at write time and checked automatically before delivery.
Process
The pilot exists to break the first draft of the guideline. Production only starts once it stops breaking.
We agree what a good response looks like for your product: register, length, formatting, what to refuse and how, and what the model should do when it does not know.
The decisions become a written document with worked examples and counter-examples. Ambiguities you have not yet decided are surfaced here rather than absorbed silently.
A small batch is written by several annotators independently. Where they diverge, the guideline was unclear — we fix the guideline, not the annotators.
Every item is written by one annotator and reviewed by another, with a senior reviewer sampling and adjudicating disagreements in writing.
Delivery includes the data, the guideline version, per-record provenance, the QA report and a change log against the previous batch.
Delivery
Fields are agreed at kickoff; this is the default shape when you have no existing schema to match.
| Dataset | Default format | Records include |
|---|---|---|
| Instruction / SFT | JSONL, messages array | system / user / assistant turns, task tag, guideline version |
| Preference pairs | JSONL, chosen + rejected | both candidates, per-criterion scores, written rationale |
| Multi-turn dialogue | JSONL, one conversation per line | full turn list, persona notes, register tag |
| Safety and refusal | JSONL | prompt, category, expected behaviour, refusal wording used |
| RAG grounding | JSONL | question, passages, answer, citation spans, answerable flag |
Quality
Generic annotation QA does not catch register drift or a soft refusal. These checks do.
FAQ
The questions teams ask when they are scoping a first Japanese fine-tune.
Both. Writing from scratch is common for Japanese, because usable open instruction data in Japanese is scarce and machine-translated data carries English sentence structure into the model. When you already have content — support logs, manuals, internal documents — we more often rewrite from it, which grounds the data in your actual domain.
A written style guide with worked examples, a calibration round before production where several annotators write the same items independently, and register consistency as an explicitly scored item in review. Consistency is a measurable property here, not an aspiration — divergence between annotators is what we watch, and every resolved divergence is written back into the guide.
Not as a source of final data. Translated instruction data inherits English discourse structure, English politeness assumptions and English formatting habits, and models trained on it produce Japanese that native readers describe as translated. We do use translation as a scaffold for prompt ideas, but responses are written in Japanese by Japanese speakers.
Only where you have agreed to it, and always with human rewriting and review on top — never as an unreviewed pass-through. Whether AI assistance was used is recorded per record, so you can filter or audit it later. If you require fully human-authored data, we run the project that way and the provenance field proves it.
Smaller than most teams expect. A few thousand carefully written, register-consistent items usually moves a Japanese fine-tune further than ten times as much noisy data, because the model is learning a style as much as a task. We scope a pilot of a few hundred items first to prove the guideline, then size production from what that pilot measures.
LLM training data
A description of the task and a handful of example responses is enough to start. We come back with the taxonomy questions we would need answered and a scoped pilot.
NDA before you share any data. Pilot scoping is free.