Most annotation proposals disagree because the request was vague, not because the vendors are wildly different. If one vendor assumes simple bounding boxes and another assumes pixel masks with a second review pass, their prices will sit far apart and neither will be wrong. The RFP is where you remove that guesswork.
Start with the task, not the vendor
Write the task as if the reader has never seen your data. Name the classes, say which shape each needs (box, polygon, keypoint, mask, span, transcript), and give two or three cases you already know are hard: a partly hidden object, a sarcastic review, two people talking over each other. Vendors price risk, and hard cases they can see are cheaper than hard cases they have to guess at.
Say what the labels are for. “Training a detector for a warehouse robot” tells a vendor how much a missed small object matters. “Evaluating a support chatbot” tells them the raters need to read policy, not just grammar.
Keep a label guide in progress, even a two-page one. You do not need to send it with the RFP, but you should be able to answer questions from it.
State volume and pace separately
Total volume and weekly throughput are different questions. A vendor can usually label 200,000 images over six months with a small, stable team; labeling the same 200,000 in three weeks needs a bigger crew, more management and often a higher price.
Give three numbers:
- pilot size (often 1,000 to 5,000 items),
- total expected volume, with a range if you are unsure,
- the weekly pace you need once the project is running.
If the work is ongoing, say so, and ask about a dedicated team or monthly retainer as well as per-item pricing.
Name the quality metric and who measures it
“High quality” means nothing in a proposal. Pick a measure you can check:
| If your task is | A fair quality measure |
|---|---|
| Classification or tagging | Accuracy against a gold set, plus agreement between annotators |
| Boxes, polygons, masks | Overlap with reference shapes (IoU) above a stated threshold |
| Transcription | Word error rate against a reference transcript |
| LLM ratings or preferences | Agreement between raters and with an expert panel |
Then say who scores it. The strongest setup is a gold set you labeled in-house, seeded into the work without the vendor knowing which items it covers. Ask each vendor what share of items they review internally, who does the review, and what happens to a delivery that misses the target. Our post on measuring annotation quality covers agreement scores and audits in more depth.
List the security evidence, not just the words
Many vendors say they are “enterprise grade”. Ask for evidence instead:
- SOC 2 Type II report. An independent auditor’s report on controls over a period of time. Ask which period it covers and whether you can read it under NDA.
- ISO/IEC 27001 certificate. A certified information security management system. Check which legal entity and which sites the certificate covers.
- Workforce location. Where annotators sit, whether they are employees or contractors, and whether a US-only team is possible.
- Data handling. Whether annotators can download files, whether work happens inside your cloud account, and how and when data is deleted at the end.
If you work under specific rules, such as export controls or a customer contract that limits where data can go, put that clause in the RFP word for word.
Ask for prices in one format
Vendors charge per item, per hour, per project, or through a platform subscription with labor on top. Comparing a per-hour quote with a per-box quote takes guesswork, so ask each vendor to price the same sample in two formats: per item and per hour, with their expected items per hour. That lets you convert one into the other.
Ask for a line-by-line list of what is included: project management, tooling, quality review, rework of rejected items, guideline changes mid-project. A low headline price with paid rework can cost more than a higher price that includes it. Our annotation pricing guide lists the prices vendors publish.
Settle ownership in the brief
State that you own the raw data, the labels, the guidelines and anything derived from them, and that the vendor may not use your data to train its own models unless you agree in writing. Most vendors accept this, and it is easier to set the expectation in the RFP than to argue it at contract stage.
Use a paid pilot as the real test
A proposal is a promise; a pilot is evidence. Pick your two or three best replies and run a paid pilot of the same items with each. Agree the pass mark before it starts, score every vendor against the same gold set, and time how long they take to absorb a guideline change. The vendor that asks good questions in week one is often the one that holds quality in month six.
A checklist you can copy
- Task, classes, shapes and three hard examples.
- Purpose of the labels.
- Data format, file sizes and where the work happens.
- Pilot size, total volume, weekly pace.
- Quality metric, target and who measures it.
- Security evidence needed and workforce location.
- Pricing in two formats, with inclusions listed.
- Ownership and data-use terms.
- Timeline: pilot start, review date, full start.
- Contact for questions and the reply deadline.
If you would like a shortlist before you send it, the best data annotation companies ranking compares 18 vendors on what they publish, and the RFP builder turns these choices into a printable one-page brief.
A clear brief is the cheapest quality control you will ever buy.