A ≻ B human feedback for model training
Fifteen vendors that sell preference data, rubric scoring and expert feedback, compared on what they publish. Describe the project and we introduce the ones that fit, with your approval first.
As each company or paper publishes them. Not quotes.
Most RLHF budgets are rater minutes spent on records like this one: a prompt, two responses, a choice, a strength and a reason, scored against a written rubric.
Summarize our refund policy for a customer in two sentences.
Refunds are available. Contact support for more information about eligibility and timelines.
You can return unused items within 30 days for a full refund. Refunds reach your card within 5 business days.
B ≻ A strength 3 of 4
Invented example record. Names, policy and timings are made up.
RLHF
Raters judge model outputs, a reward model learns their judgments, and the model is tuned to score well against it.
DPO
Direct preference optimization trains straight from chosen and rejected responses, so it needs the same pairwise data.
RLAIF
A model applies written principles instead of people. Humans still write the principles, build a check set and audit the AI judge. RLHF, DPO or RLAIF
Rubric
Criteria in priority order, a defined scale and worked examples. Every delivered record should carry the rubric version it was scored against. How to write rubrics for a human feedback project
Fifteen vendors, grouped by how they deliver. Most quote on request. The full vendor table adds what each says it supplies, with a source for every line.
A vendor in this table can pay RLHF Companies when an introduction turns into a contract. Who appears, the grouping by delivery model and every description are not for sale.
Route 1
A managed service recruits, qualifies, pays and reviews the raters, then delivers finished records. Scale AI, Surge AI, Toloka, Mercor, micro1, Turing and others work this way.
Route 2
A platform sells the labeling tool and can add a workforce; a marketplace supplies participants for a study you design and pay. Labelbox, Kili Technology, SuperAnnotate and Prolific.
Human feedback is sold on time: per task, per rater hour, through platform fees with labor on top, or as a fixed statement of work. Few vendors publish a rate, so the number you can pin down yourself is how long one task takes.
The pricing models page has a calculator for your own numbers.
Worked example
240 rater hours, before anyone reviews a sample
Vendors quote on the same handful of facts. Briefs that leave them out get quotes nobody can compare. The brief builder turns these choices into text you can send.
Pairwise choice, ranking, rubric score, written demonstration or red teaming, and the criteria in priority order.
Generalists for tone and instruction following; people who can verify the answer for code, math, law or science.
Agreement on shared items, accuracy on your own gold tasks, and what happens to work that falls short.
Several hundred to around two thousand items on your real task, timed, before any production volume.
Give some items to two raters and measure how often they agree, corrected for chance. Cohen's kappa is the standard figure for two raters:
κ = (po − pe) / (1 − pe)
Alongside agreement, ask for accuracy on gold tasks you write, a reviewer's overturn rate on a random sample, and a rubric version on every record. What to request each week
Managed vendors quote after scoping, and none in our table publishes a per-task rate. The public floor is marketplace participant pay: Prolific's minimum is $8 an hour and it suggests $12, with its platform fee on top. Expert raters, overlap between raters and review layers raise the price from there. See pricing models.
We do not rank them. Vendors whose own pages describe this work include Scale AI, Surge AI, Toloka, Mercor, micro1, Turing, TELUS Digital, Handshake AI, Centific, Appen and Innodata, plus platforms such as Labelbox, Kili Technology and SuperAnnotate and the Prolific marketplace. The vendor table sets out what each publishes, with a source for every line.
Yes. The usual first step is a paid pilot of several hundred to around two thousand items on your real task. Self-serve routes are open to small teams: on Prolific you build the study and set the pay yourself, and Kili Technology offers a $0 trial for 2 to 5 seats capped at 100 assets. Managed vendors quote pilots after a scoping call. Our brief builder writes the brief.
A data service runs the raters for you: it recruits, qualifies and pays them, and delivers finished preference data. A labeling platform such as Labelbox, Kili Technology or SuperAnnotate sells the software your own people label in, with a workforce available as an extra. A marketplace such as Prolific sits between the two: you design the study and pay participants it supplies.
Send some items to two raters and measure agreement corrected for chance, usually Cohen's kappa. Hide gold tasks you wrote in the normal stream, have a reviewer re-score a random sample, and require a rater ID and rubric version on every record. Expect preference judgments to agree less than object labels: in OpenAI's InstructGPT work trained labelers agreed about 73% of the time. The quality controls page lists what to request each week.
Whoever your contract names, so settle it before the pilot. None of the vendor pages we list sets out default ownership terms. Ask in writing who owns the delivered records, whether the vendor may reuse your prompts or responses for other clients, and what happens to rater notes and rubric versions when the project ends.
Sketch the feedback work in about a minute. What comes back is one vendor name, not a round of cold calls.
RLHF Companies introduces teams to vendors, and the vendor can pay us for that introduction. Sending a brief costs your team nothing.
Keep prompts, model outputs, datasets and samples out of this form; describe the work in words instead.