The short answer
Rater pay tells you who is likely doing your task and how they are pushed to work. The few published figures run from Prolific's $8 hourly minimum for participants to the up-to-$150 hourly figure Alignerr states for experts. Ask any vendor what raters earn, whether pay is per task or per hour, and how quality affects it.
On this page
When you buy human feedback you are buying attention: someone reading two answers closely enough to tell which is better and why. Pay does not guarantee attention, but it shapes who shows up and how they spend their minutes, which makes it one of the few signals a buyer can check before the first delivery arrives.
What do raters actually get paid?
Most vendors do not publish what they pay raters, and fewer publish what they charge buyers. The public figures come from marketplaces and from the contributor-side brands that recruit raters.
| Company | Who is paid | Published figure |
|---|---|---|
| Prolific | Study participants, paid by the researcher | At least $8 an hour; $12 an hour recommended (platform fee charged on top) |
| Alignerr (Labelbox) | Subject-matter experts on AI lab projects | Up to $150 an hour, as stated on its about page |
| Handshake AI | Students and experts in the Handshake AI Fellowship | Not published: pay "varies by project" and depends on project type, subject expertise and experience |
A company in this table could pay RLHF Companies if one of our introductions became a contract with it. That possibility plays no part in what is reported here, and none of these companies is connected to the site.
The spread is the point. An $8 floor and a $150 ceiling describe different products. At the bottom, a general pool judging everyday chat; at the top, people who can verify a derivation or review a contract clause. Neither number is a buyer’s price: vendors add recruiting, management, review and margin.
What does an hourly rate tell you?
Who the rater pool is
A vendor that pays near marketplace rates is drawing on generalists, whatever the sales deck says about experts. If your rubric asks raters to verify that code compiles or that a chemistry answer is right, ask what the people doing it earn. A low figure with an “expert” label is a mismatch worth questioning.
Whether the work is worth doing carefully
Raters choose between projects. If yours pays less than the next one for the same effort, the strongest raters drift away and the pool you are left with changes under you. Agreement figures that slide over the weeks of a project can be a sign of this.
What the vendor keeps
Ask what share of your price reaches the raters. You do not need the exact margin, but a vendor that will not describe its pay range at all is asking you to trust its quality claims without one of the main inputs.
Is per-task pay worse than hourly pay?
Neither is wrong, but they push raters in different directions.
- Per-task pay rewards finishing quickly. That suits short, uniform comparisons, and it invites skimming on long or tricky items. Watch time per task: a cluster of raters far below the median is a warning.
- Hourly pay rewards care and suits expert review, red teaming and long agent traces. It needs closer tracking of throughput, because a slow rater costs you directly.
Many projects mix the two: hourly during the pilot, while everyone learns how long tasks really take, then a per-task rate for production set from those timings. The pricing models page has a calculator that turns items, minutes and pay into rater hours.
Does higher pay buy better data?
Not by itself. Pay recruits people and keeps them on your project; the rubric, training, gold tasks and audits are what turn their time into consistent labels. A well-paid pool working from a vague rubric produces confident disagreement. What pay does is raise the ceiling: it is hard to get expert judgments from people who are not paid like experts.
Two practical signs that pay and quality are lined up:
- Pay moves with quality. Raters with high gold-task accuracy get more work or a higher rate; raters who fall below a threshold get retrained or removed.
- Rater identity is stable. Delivered data carries anonymous rater IDs, and the same IDs recur across weeks. High churn means you are retraining a new pool every delivery.
Worker brands and the vendor you contract with
Several large vendors recruit through separate contributor brands: Scale AI runs Outlier, and Labelbox runs Alignerr. If you see these names in rater forums or job posts, the buyer’s contract is with Scale AI or Labelbox. It is still fair to ask the vendor which pool, and which pay band, your project will draw on.
Questions to put to any vendor about pay
- What hourly range do raters on a project like ours earn?
- Are they paid per task, per hour or a mix, and does pay change with accuracy?
- How are raters qualified for this subject, and what share of applicants pass?
- How many of the raters in week one will still be on the project in week six?
- Are raters allowed to use AI tools, and how is that checked?
A vendor that answers these plainly is easier to compare. When you are ready, the vendor table on the RLHF companies home page shows what each one publishes.
Pay is not the quality; it is the clearest early clue to the quality you are about to buy.