Before You Begin
Is this your first time creating an Advanced Assessment? If so, you'll want to check out our Beginner's Guide to Advanced Assessments if you haven't already. Advanced Assessments are our most powerful and intricate assessment tool, and it will be helpful to know the basics before you begin to create one. Once you've taken a look at that guide, come on back here and get started with autoscore settings!
Most questions available in Advanced Assessments have a means of auto-scoring. The following question types do not have autoscoring, therefore, will not be addressed in this article.
Non Autoscored Questions:
|
In general, any answer that involves extended written or recorded answers, as well as anything that would be free-hand drawn, cannot be auto-scored. To see the scoring works for non auto-scored questions, please refer to the links in the table above for that specific type of question.
Individual Question Scoring
This article references the more general aspects of autoscoring. To see the rules and details for a specific question type, please reference the article written for that specific type of question. You can find a list of the question types here.
Most questions that autoscore have the options shown in the More Options section. To see the scoring options, expand the More Options menu by clicking on it.
General Autoscoring Options:
Unscored/Practice usage: This option removes all scoring from the item. This item will still be on the assessment but will not count toward the final score.
Penalty points: Penalty points deduct a value from the student's score for each incorrect response on a question.
Check answer button and Check answer attempts: This setting will allow students to check their answer a set number of times.
Scoring type: Choose between Exact Match, Partial Match per Response, or Partial Match. For a more detailed explanation of each, please see below.
Enable auto-scoring: If you uncheck this option, this will require that the question be manually scored.
Minimum score if attempted: the student will receive points as long as the question has been attempted. If the question is left blank or is unanswered, the student will not earn those points.
Scoring Type
Most autoscoring questions have three options available
We will take a closer look at each scoring type using the example scenario below:
Example scenario:
 |
Exact Match
The student must give a correct response for all components of the question to earn the number of points indicated in the point(s) field (1 for this example).
The student selected the wrong answer for "Turtle." Therefore, the student does not earn any points.
Partial Match Per Response
Each response is scored independently, and each is worth the number of points indicated in the point(s) field (1 for this example).
Since now each response is worth 1 point, there are a total of four points possible. The student answered three of the four correctly, earning 3 out of 4 points.
Partial Match
Similar to partial match per response, each response is scored independently, but the point value indicated in the point(s) field (1 for this example) is evenly split among the responses.
Under Partial Match there is also a Rounding option. There are two options to select from the drop-down menu: Round Down or None.
Now the entire question is worth 1 point again, but that 1 point is split evenly between the four responses, so they will each be worth 0.25 points. The student answered three out of the four correctly, earning 0.75 points.
Math-Specific Scoring
If using a math (or chemistry) question type, you will need to choose a scoring "method." This will determine how Otus determines whether certain numerical answers should be considered correct or incorrect.
equivLiteral
Only accepts a response that is mathematically equivalent to the correct answer AND is given in the same form.
equivSymbolic
Accepts a response that is mathematically equivalent to the correct answer, even if it is in a different form.
This is best when working with equations or variables, where order or form is not important.
For example, if the task is to multiply 3 and 2, any response that is mathematically equal to 6 will be considered as correct, such as 6/1, 12,1, 3+3, or 6.0.
equivValue
Compares numerical values that may be represented in different ways, such as units of measurement.
It is very similar to equivSymbolic, and most often it is not necessary to use equivValue. When in doubt, use equivSymbolic.
The other options, such as isSimplified, isUnit, or isFactorized, add another layer of scoring that check student answers for a specific form. For more information, please check out this resource.
💡 Tip: Most of the articles for the individual math question types will have a more detailed explanation of how each scoring method impacts the answers. If you are looking for those examples, click here.
Item Scoring
In addition to each individual question having scoring options, there are also three different ways to score the item as a whole:
Per question (default, most common)
Dependent (EBSR)
We will take a closer look at each scoring type using the example scenario below:
Example scenario:
|
Per Question
This is the default scoring type. The score for the item is calculated by adding the score from each question on the item.
The student answered two out of the three correct, earning 2 out of 3 points.
Dichotomous
Dichotomously scored Items have only two possible outcomes: fully correct or fully incorrect. If the student answers any question on the item incorrectly, they will not earn any points for the entire item.
The student answered one of the questions incorrectly; therefore, they did not earn any points.
Dependent (EBSR)
In dependent scoring, the score taken from each question depends on the scores of previous questions on the Item. The rule is that if a question on the item is not fully correct, the user is not awarded a score for any subsequent questions on the item. Such scoring can be applied when there are two or more questions dependent on each other.
The student answered the second question incorrectly, which means the student will not earn points for any question(s) after #2, so the correct answer for #3 is not counted. This student earned 1 out of 3 points, for getting the first question correct.
By default, the item is set to score per question. If you want to change this to either dichotomous or dependent, follow the steps below:
Step 1: While you are editing the item, select Settings.
Step 2: Select the Details tab.
Step 3: Make your selection in the Scoring type box.
Step 4: Select Apply.
Autoscore Settings for Standards Based Assessments
This section only applies to advanced assessments that utilize Standards Based Grading Scales (Non-Points). With this feature, you have the ability to set the default score a student receives for auto-graded questions. Adjusting auto-scoring settings will take place on the Assigning page.
This feature is assessment wide, not per individual question. This means that the level selected for Correct Answer and Incorrect Answer will be applied to all questions within the assessment.
A question is considered "correct" if ALL portions of the individual question are correct. Mid-level scale labels will not apply to partial correct answers. Questions are considered fully Correct or Incorrect.
Example: If partial credit is being applied and a student gets 3 out of 5 parts correct, they will be given the status of "Incorrect Answer".
💡Tip: If you edit this setting after the assessment has been assigned, you will receive a message explaining that this only applies to future scores, not scores already collected.
Edit the Autoscoring
❗ Important:
You can edit the number of points, scoring type, or scoring method on any question in an advanced assessment. If the assessment has already been assigned, changes made will not update the assigned version of that assessment. You will need to unassign the assessment, make the changes, and then reassign the assessment. If students have already begun working or turned in the assessment, they will be starting fresh with a new copy of the assessment. Their previous work and scores will be removed permanently.
To edit a question, follow these steps:
Step 1: Navigate to your Assessments module.
Step 2: Stay in your Drafts.
Step 3: Locate the assessment and select it to open.
 |
|
|
 |
Learnosity Resources
Learnosity provides resources to help with question building and autoscoring as well. Check them out below.
Otus Live Videos
Click to see Otus Live videos related to the above topics.
Click to see Otus Live videos related to the above topics.
This is a video on autoscoring options in Advanced Assessments.
This a video on how to score math questions in Advanced Assessment.
FAQ and Troubleshooting
Click to see FAQ and troubleshooting details.
Click to see FAQ and troubleshooting details.
Can autoscore settings be edited?
You can edit the number of points, scoring type, or scoring method on any question in an advanced assessment. If the assessment has already been assigned, changes made will not update the assigned version of that assessment. You will need to unassign the assessment, make the changes, and then reassign the assessment. If students have already begun working or turned in the assessment, they will be starting fresh with a new copy of the assessment. Their previous work and scores will be removed permanently.
How do I add alternate answers?
To add alternate answers, click the blue + icon in the scoring section. For more information, click here. If you are looking for information on a specific question type, click here.
How do I make something worth partial credit?
You will want to use the Partial Match or Partial Match per Response scoring type option.
How do I know how to determine the autoscoring for a specific type of question?
We have an article for every question type available in Advanced Assessments. Find the question you're looking for here.