Monitor & Test Question Review


Lesson Overview:

  • This lesson reviews job monitoring, which allows you to check on the health of your job and identify areas for improvement.
  • The first step in monitoring your job is reviewing your dashboard. This provides a high-level overview of the job.
  • By far the most important aspect of monitoring the job is reviewing your test questions.
  • Reviewing test questions:
    1. There is the option to sort by missed test questions. This brings the most-missed test questions to the top.
    2. Also helpful is to sort by contested, which brings the most-contested test questions to the top. This list will typically include the same questions as the most-missed, but not always: sometimes the most-missed test question is perfectly reasonable to the contributors so they do not contest it.
    3. When a test question is highly missed and highly contested, a ‘review’ tag appears next to it. Once you’ve reviewed the question and either made changes or deemed it correct as-is, you can mark it as “passed review.” This lets you know you don’t have to come back to it later.
    4. When you change the answer to a test question, this will update the accuracy of all contributors who answered. Anyone who was originally marked incorrect will be “forgiven”, and their accuracy will increase accordingly.

Steps To Complete:

  1. Navigate to ‘Monitor’ page
  2. Review job overview metrics
  3. Navigate to ‘Quality’
  4. Review all Test Questions that are highly missed/contested
  5. Update erroneous test questions
  6. Navigate back to ‘Monitor’ page
  7. Observe how updating inaccurate test questions changed the job overview metrics

Further Reading

Next Lesson: Results

Was this article helpful?
0 out of 0 found this helpful

Have more questions? Submit a request
Powered by Zendesk