Test Question Creation for Bounding Box Tool

To create test questions for bounding box jobs:

  1. On the quality page, click “Create Test Questions”
  2. Add boxes around the objects in the way you specified via your job's instructions.
  3. If no boxes are needed, click 'Nothing to Box' Screenshot_2017-05-16_15.34.54.png
  4. Save Test Question

To review contributor responses to test questions:

  1. Select a test question from the quality page
  2. From the image annotation tool toolbar, select a "Judgment ID" via pulldown.Screenshot_2017-05-16_15.34.54_copy.png
  3. Edit, create or remove your own boxes based on feedback. Judgments are color coded based on if they match the gold responses. A score from zero to one is displayed based on the intersection over union formula and the average of all scores on the image are averaged and compared to the test question threshold set in the job design.Cityscape.png
  4. Save any edits that are made to update the evaluation of the existing contributors' work and ensure any future attempts to answer the test question will be properly evaluated.

Was this article helpful?
4 out of 4 found this helpful

Have more questions? Submit a request
Powered by Zendesk