The Smart Pricing feature helps customers choose an appropriate annotator pay rate given the complexity of their jobs. The feature uses a machine-learning algorithm trained on thousands of jobs run on the platform to predict the hourly wage contributors will earn in a job prior to launch based on the job design. It will also provide the effective hourly rate contributors earn in a job after it’s been launched and some annotations have been collected.
Note: Smart Pricing is in beta release. Please reach out to your Customer Success Manager (CSM) if you would like access.
The machine-learning algorithm predicts the time it will take contributors to make a single judgment in a job based on the job design and instructions (this is the “estimated time per judgment”). If you select the option to “Predict Hourly pay”, enter a price per judgment to calculate the corresponding hourly pay rate (given the estimated time per judgment). If you select the option to “Predict Price per Judgment”, enter a target hourly pay rate in order to calculate the corresponding price per judgment.
Contributors are paid per judgment and are not guaranteed to earn the predicted hourly pay rate.
Figure 1: Smart Pricing showed via the Launch Page
After the job has accrued many contributor judgments, the module will display the time per judgment of the accrued trusted judgments and the hourly pay rate trusted contributors have earned. You can use this information to adjust your pay rate as you see fit.