All Collections
Assessments: Advanced Assessments
Advanced Assessment: Question Validation & Scoring Information
Advanced Assessment: Question Validation & Scoring Information
This article provides an overview of the common validation and scoring information (including auto scoring) for advanced assessments.
Brooke Fodor avatar
Written by Brooke Fodor
Updated over a week ago

Advanced assessments are one of the available assessment types in Otus. They are the most powerful and versatile, with over 60 available question types ranging from the basic multiple choice to graphing in the first quadrant. The Advanced Assessment item components are created by a partnering company called Learnosity.

Most questions available in advanced assessments have a means of auto-scoring, with the exception of the following:

  • Essay - Rich Text

  • Essay - Plain Text

  • Audio Recorder

  • Video Recorder

  • Highlight Image

  • Drawing

  • Math Essay

  • Chemistry Essay

  • File Upload

In general, any answer that involves extended written or recorded answers, as well as anything that would be free-hand drawn, cannot be auto-scored.

In this article, you will find the following sections:

Learnosity also provides an author guide to help with building questions.

Individual Question Scoring

Most questions that have auto-scoring will have the options shown below in the More Options section:

Auto Scoring Options:

  • Unscored/Practice usage: This option removes all scoring from the item. This item will still be on the assessment but will not count toward the final score.

  • Penalty points: Penalty scoring deducts a value from the student's score for each incorrect response on a question. For more information, click here.

  • Check answer button and Check answer attempts: This setting will allow students to check their answer a set number of times. For more information, click here.

  • Scoring type: Choose between Exact Match, Partial Match per Response, or Partial Match. For a more detailed explanation of each, please see below!

  • Enable auto-scoring: If you uncheck this option, this will require that the question be manually scored.

  • Minimum score if attempted: the student will receive points as long as the question has been attempted. If the question is left blank or is unanswered, the student will not earn those points.

Scoring Type

Most auto-scoring questions have three options available: Exact Match, Partial Match per Response, and Partial Match. Some questions only have Exact Match available. Below you will see how each scoring type impacts this example question.

Example scenario:

  • Multiple Choice Matrix question with four responses needed (one each for whale, alligator, turtle, dog).

  • Set for 1 point.

  • Student gave three out of four correct answers.

Example Correct Answer

Example Student Answer

➡️ Exact Match

  • The student must give a correct response for all components of the question to earn the number of points indicated in the point(s) field (1 for this example).

  • The student selected the wrong answer for "Turtle." Therefore, the student does not earn any points.

➡️ Partial Match Per Response

  • Each response is scored independently, and each is worth the number of points indicated in the point(s) field (1 for this example).

  • Since now each response is worth 1 point, there are a total of four points possible. The student answered three of the four correctly, earning 3 out of 4 points.

➡️ Partial Match

  • Similar to partial match per response, each response is scored independently, but the point value indicated in the point(s) field (1 for this example) is evenly split among the responses.

  • Under Partial Match there is also a Rounding option. There are two options to select from the drop-down menu: Round Down or None.

  • Now the entire question is worth 1 point again, but that 1 point is split evenly between the four responses, so they will each be worth 0.25 points. The student answered three out of the four correctly, earning 0.75 points.

Auto-Scoring Otus Live

Below is a video from our Otus Live series on auto-scoring options in Advanced Assessments.

Math-Specific Scoring

If using a math (or chemistry) question type, you will need to choose a scoring "method." This will determine how Otus determines whether certain numerical answers should be considered correct or incorrect.

➡️ equivLiteral:

  • Only accepts a response that is mathematically equivalent to the correct answer AND is given in the same form.

➡️ equivSymbolic:

  • Accepts a response that is mathematically equivalent to the correct answer, even if it is in a different form. This is best when working with equations or variables, where order or form is not important. For example, if the task is to multiply 3 and 2, any response that is mathematically equal to 6 will be considered as correct, such as 6/1, 12,1, 3+3, or 6.0.

➡️ equivValue:

  • Compares numerical values that may be represented in different ways, such as units of measurement. It is very similar to equivSymbolic, and most often it is not necessary to use equivValue. When in doubt, use equivSymbolic.

The other options, such as isSimplified, isUnit, or isFactorized, add another layer of scoring that check student answers for a specific form. For more information, please check out this resource.

💡 Tip from the Otus Team: Most of the articles for the individual math question types will have a more detailed explanation of how each scoring method impacts the answers. If you are looking for those examples, click here.

Math Scoring Otus Live

Below is a video from our Otus Live series on how to score math questions in Advanced Assessment.

Item Scoring

In addition to each individual question having scoring options, there are also three different ways to score the item as a whole; per question (default, most common), dichotomous, and dependent (EBSR).

Example scenario:

  • One item has three questions.

  • Each question is worth 1 point.

  • The item is worth a total of 3 points.

➡️ Per Question

  • This is the default scoring type. The score for the item is calculated by adding the score from each question on the item.

  • The student answered two out of the three correct, earning 2 out of 3 points.

➡️ Dichotomous

  • Dichotomously scored Items have only two possible outcomes: fully correct or fully incorrect. If the student answers any question on the item incorrectly, they will not earn any points for the entire item.

  • The student answered one of the questions incorrectly; therefore, they did not earn any points.

➡️ Dependent (EBSR)

  • In dependent scoring, the score taken from each question depends on the scores of previous questions on the Item. The rule is that if a question on the item is not fully correct, the user is not awarded a score for any subsequent questions on the item. Such scoring can be applied when there are two or more questions dependent on each other.

  • The student answered the second question incorrectly, which means the student will not earn points for any question(s) after #2, so the correct answer for #3 is not counted. This student earned 1 out of 3 points, for getting the first question correct.

By default, the item is set to score per question. If you want to change this to either dichotomous or dependent, follow the steps below:

  • Step 1: While you are editing the item, select Settings.

  • Step 2: Select the Details tab.

  • Step 3: Make your selection in the Scoring type box.

  • Step 4: Select Apply.

This section only applies to advanced assessments that utilize Standards Based Grading Scales (Non-Points). With this feature, you have the ability to set the default score a student receives for auto-graded questions. Adjusting auto-scoring settings will take place on the Assigning page.

  • This feature is assessment wide, not per individual question. This means that the level selected for Correct Answer and Incorrect Answer will be applied to all questions within the assessment.

  • A question is considered "correct" if ALL portions of the individual question are correct. Mid-level scale labels will not apply to partial correct answers. Questions are considered fully Correct or Incorrect.

➡️ Example: If partial credit is being applied and a student gets 3 out of 5 parts correct, they will be given the status of "Incorrect Answer".

❗ If you edit this setting after the assessment has been assigned, you will receive a message explaining that this only applies to future scores, not scores already collected.

Editing the Scoring

❗ Important:

You can edit the number of points, scoring type, or scoring method on any question in an advanced assessment. If the assessment has already been assigned, changes made will not update the assigned version of that assessment. You will need to unassign the assessment, make the changes, and then reassign the assessment. If students have already begun working or turned in the assessment, they will be starting fresh with a new copy of the assessment. Their previous work and scores will be removed permanently.

To edit a question, follow these steps:

  • Step 1: Navigate to your Assessments module.

  • Step 2: Stay in your Drafts.

  • Step 3: Locate the assessment and select it to open.

  • Step 4: Locate the item you wish to edit the point value for and select it to open.

  • Step 5: Select Edit at the top right corner.

  • Step 6: Locate the question within the item and select the pencil icon to edit.

  • Step 7: Scroll down and adjust point value. Select Save at the top right corner when done.

Next Steps

Did this answer your question?