Frequently asked questions

Japanese is what we are built for. We handle Japanese–English bilingual work — translation review, cross-lingual alignment, parallel evaluation sets — because that draws on the same Japanese judgment. For volume work in other languages we would rather tell you plainly that we are not the right vendor than take the project and subcontract it.

Yes. We work in client-provided platforms and virtual desktops, including where policy requires that data never leaves your infrastructure, and in our own secure environment when you would rather not provision accounts. The tooling changes; the guideline, the review passes and the QA reporting do not.

We collect and create data where it is Japanese-language work — writing instruction data, constructing evaluation prompts, recording scripted speech, sourcing document samples. We are not a general-purpose data collection operation, and for large-scale field collection you would be better served by a specialist.

Yes, and the first thing we do is read their guideline and re-annotate a sample of their delivered data blind. That tells you what you actually have — including whether the existing labels are internally consistent — before we add to it. Continuing a dataset without measuring what it currently contains is how inconsistency gets compounded.

Yes. A dedicated team arrangement is common for long-running AI programs, where the same annotators carry the guideline forward and their accumulated context is a real asset. It is also the arrangement where guideline drift matters most, so it comes with periodic re-calibration against a gold set.

Inter-annotator agreement with a chance-corrected coefficient, gold-set accuracy, adjudication rate, and error counts broken down by a named taxonomy rather than as a single defect total. Everything is reported per label or criterion rather than blended, because a blended figure hides exactly the category that is failing. The quality page sets out how each number is calculated and what it does not capture.

JSONL, CSV, CoNLL-U, COCO, SRT, CTM, TextGrid, hOCR and custom schemas defined at kickoff. Span data always includes character offsets underneath whatever tag format you take, so a later change of tokenizer is a re-export rather than a re-annotation. Every batch ships with its guideline version and a change log against the previous one.

Revisions go through the same review and adjudication path as first-pass work — a fast correction lane is how datasets drift between batches. Where the cause is a guideline gap we revise the guideline and re-work the affected items, not just the ones you flagged, because the same gap will have produced the same error elsewhere.

Throughput depends on task complexity, review depth and how much genuine ambiguity the data contains, and it varies more between tasks than most estimates admit. We measure real throughput during the pilot on your data and commit to a plan from that. A delivery date quoted before anyone has annotated your data is a guess presented as a commitment.

We look at the specific items with you. Usually one of three things is true: the guideline was ambiguous, in which case we revise it and re-work; the guideline was clear and misapplied, in which case we correct and feed it back into review; or the guideline says something you no longer want it to say, in which case the change is a scope decision. Sorting a disagreement into one of those three is more productive than arguing about the individual label.

For projects that need it, we build teams around the required domain expertise, which can include people with relevant qualifications or practical experience in fields such as law, tax, accounting, labor and social security, or corporate registration. We do not maintain a standing roster of licensed professionals. Availability is assessed per project against its scope, volume, specialisation and timeline, and if a qualification cannot be sourced for your project we will tell you before it starts.

By what kind of judgment the task requires. Terminology and document conventions can be transferred to a trained annotator with a good lexicon and a calibration round, and for most work that is the better answer — faster, more consistent and less expensive. A specialist is warranted when the judgment itself is professional: whether an interpretation is defensible, whether an answer would mislead a reader about their obligations.

Yes. Domain specialists can score generated responses for factual accuracy, reasoning quality, professional appropriateness and compliance with your criteria, and can produce preference rankings for alignment work, benchmark sets, retrieval evaluation and red teaming. The reasoning behind each judgment is recorded, which is usually what makes an expert evaluation worth its cost.

No. They are annotating and evaluating data. The output is training or evaluation data for your AI system, not legal, tax or accounting advice to you or to your users, and nothing produced in an annotation project should be presented to an end user as professional advice. We are explicit about this because the distinction matters to both of us.