This page displays information about the performance of individual contributors. Use this page to analyze a contributor's behavior on your job and award a bonus or remove the contributor from your job, if necessary.
Fig. 1: Contributors page sorted by Judgments count
Sort by judgments count
By default, judgments are sorted by Trust Score. Click the Sort by judgments count button to see contributors ordered by the number of judgments they submitted. This will also reveal the best contributors from your job because the contributors at the top of this list are the ones that have submitted the most judgments while maintaining a high Trust Score.
Fig. 2: Sorting by Judgement Count via the Contributors Page
Flag - Only this Job
Stops contributors from completing pages. Their judgments remain in their current state of tainted or non-tainted and will not be rejected.
Flag - All Jobs
Stops contributors from completing pages and all subsequent jobs deployed by the user. Judgments remain in their current state of tainted or non-tainted and will not be rejected.
Sends a message that appears in the 'inbox' of the Contributor's account.
Stops contributors from completing pages and removes the contributor's judgments from a job.
Note: This is best used when a job is running so that new judgments are ordered if a contributor’s judgments are thrown away. Rejecting upon job completion will result in some rows having less than the required judgments per row because you will have removed some contributor’s judgments.
This statistic displays the total number of judgments submitted by a contributor and the rate of judgment submission (judgments per hour).
Test Questions Seen / Missed / Forgiven
This displays the total amount of Test Questions seen, followed by the number of Test Questions missed and the number of Test Questions forgiven for the particular job.
This is the contributor's accuracy level in the current job, determined by their accuracy on the Test Questions they’ve seen in the job.
This is the name of the channel through which the contributor accessed the job.
View this segment of the page to see information pertaining to the status and origin of the contributor's account. Information includes:
This segment provides an overview of the contributor's metrics in the job.
Fig. 4: An individual contributor's performance details
View this segment of the page to see contributor payment history and amounts. An example conversions table is as follows:.
Fig. 4: Conversion information
This is the page ID that was completed by the contributor.
The date and time when the page was completed.
The amount the contributor earned in Dollars for completing the page.
Amount (in channel currency_
The amount in Dollars that was given to the channel that the contributor used to complete the page.
The amount of bonus provided by the job creator to the contributor. The total amount of bonus is shown in the total at the bottom of the table.
Displays contributor's answer distribution to a specified field, excluding their judgments on Test Questions. Select the field name to view the corresponding answer distribution across all judgments the contributor has submitted in the job.
Fig. 5: Individual Contributors Answer Distributions
Missed Test Questions
View this segment to see all of the Test Questions that were missed by the contributor. Each row displays information pertaining to one Test Question.
Fig. 6: A Contributor's missed Test Questions, which has been forgiven
Test Question ID
Shows a link for the Test Question based on its ID.
Shows the question (based on CML name attribute) that the contributor answered incorrectly. This is useful when there are multiple questions in your job.
The answer given by the contributor on that Test Question.
The correct answer(s) from the Test Question that contributors will be tested against.
Click the Forgive button to forgive the contributor’s incorrect answer for the corresponding Test Question. This will raise the Trust Score of the contributor.
Use this feature to award contributor bonuses for excellent performance or going above and beyond with their feedback on Test Questions. Once clicked, a window will appear asking confirmation for the amount (in cents) to be bonused, followed by the chance to submit a message along with your bonus.
*Please note that bonuses incur the appropriate markup for your team.