What we can take on
Prompt and response
Open and closed question answering, classification, summarisation, extraction, rewriting.
Preference ranking
Pairwise and list-wise comparisons with a required written rationale.
Chain of thought
Step-by-step reasoning traces reviewed for validity, not just final answers.
Rubric rating
Helpfulness, harmlessness and factuality scored against a versioned rubric.
Multi-turn dialogue
Written conversations that hold context and persona across many turns.
Red teaming
Adversarial probing by vetted, restricted expert pools with a categorised output.
The types of generative ai services we run
The full breakdown, not the headline categories. Anything not on this list is a scoping call away.
Prompt and response. Open question answering, Closed and constrained answering, Summarisation and rewriting, Classification and extraction prompts, Instruction-following tasks.
Preference and ranking. Pairwise comparisons, List-wise ranking, Best-of-N selection, Side-by-side model comparison.
Reasoning. Chain-of-thought traces, Step-validity review, Maths and logic problem sets, Code reasoning traces.
Rubric rating. Helpfulness scoring, Harmlessness scoring, Factuality checks, Tone and persona adherence.
Safety and red teaming. Adversarial prompt probing, Jailbreak categorisation, Policy-violation flagging, Sensitive-topic response review.
The forms the data takes
What you actually receive: file types, structure and the metadata that travels with it.
- Prompt and response
- JSONL pairs with instruction, response, task type, language and difficulty.
- Preference pairs and rankings
- Pairwise or list-wise comparisons with the chosen response, rubric scores and the rater rationale.
- Reasoning traces
- Step-by-step solutions with per-step validity labels.
- Multi-turn dialogue
- Conversations with persona, context and turn-level labels.
- Rubric ratings
- Numeric and categorical scores per criterion, with rubric version and rater id.
- Red-team logs
- Probe, model reply, harm category and severity, in a restricted export.
How the data is collected
From the first brief to the batch you receive, step by step.
- 01
Written by qualified people
Prompts, answers and dialogues are produced by writers and domain experts who passed a rubric-based qualification, not scraped or machine-generated.
- 02
Briefed with a taxonomy
Every batch starts from a topic, task and difficulty taxonomy you approve, so coverage is designed rather than accidental.
- 03
Rated against a versioned rubric
Raters score responses for helpfulness, harm and factuality against a written rubric. The rubric version is stored with every judgement.
- 04
Reasons recorded, not just scores
Preference and rating tasks require a written rationale, so a surprising reward-model behaviour can be traced to what a rater actually thought.
- 05
Restricted pools for sensitive work
Red teaming and expert review run in vetted, access-controlled pools under NDA, with category-tagged output.
How a project runs
- 01
Scoping call
We agree volume, languages, demographic quotas, formats and what counts as a rejected item.
- 02
Guidelines and gold set
Written guidelines with worked examples, plus a gold set every contributor is tested against.
- 03
Pilot batch
A small paid batch you accept before anything scales. Most guideline problems surface here.
- 04
Production
Work flows through capture checks, automated QC, peer review and expert review.
- 05
Delivery
Incremental batches in your format with a data card, QA report and agreement scores.
- 06
Iteration
Rejections feed back into the guidelines so the same mistake is not made twice.
How quality is checked
Checks that run during the work, not a sample at the end.
- Rater qualification against the rubric before live work
- Hidden gold at a fixed rate, with monthly re-certification
- Agreement statistics reported per slice, not only overall
- Rationale review: weak or copied reasons are rejected
- Rubric version pinned to each delivered batch
Industries that use it
Where teams bring us generative ai services, and what they use it for.
- Technology and softwareInstruction tuning, evaluation sets and assistant behaviour.
- Banking and financeDomain assistants that must stay inside policy.
- HealthcareClinician-reviewed medical question answering.
- Retail and e-commerceShopping assistants and product-support bots.
- TelecomSupport copilots and agent-assist.
- EducationTutoring assistants in regional languages.
Why teams pick us for generative ai services
Rationales required
Every preference carries a reason in English and in the source language.
Agreement reported
Kappa per language and per prompt category, shipped with the batch.
Rubric versioning
The rubric is an artefact you receive, not a private internal document.
Expert pools
Legal, clinical and code tasks go to people with the credential to do them.
