Answer rules are build to help teachers add additional answers and create specific feedback, based on custom made checks including student answer dependent variables.
In this article we explain how answer rules can be used for different purposes, what the different options are and giving a detailed example.
What can you achieve with answer rules
- automatic specific feedback based on the (characteristics of the) student answer (e.g. give specific feedback to help correct a misconception)
- custom made answer checks by combining operations and check-methods (e.g. check if the derivative of the student answer equals a parameterised function)
- adding answer dependent variables in feedback (e.g. calculate the integral of the answer of the student and show it in the feedback at specific moments)
- adding additional correct answers
What are the options within answer rules
To explain the different options an example is used to go over each of the options. The example exercise is shown below:
The main answer rule used in this exercise is shown below:
This answer rule states: "if student.answer1 is Algebraically Equivalent to 1/3 x^3 + C then mark answer as correct and give specific feedback below".
Each answer rule has 3 main parts to configure
- statement: in this example "if student.answer1 is Algebraically Equivalent to 1/3 x^3 + C". A statement consists out of 1 or multiple clauses.
- correctness: in this example "correct"
- specific feedback (optional): in this example the text shown in light blue
Note: extensions to answer rules are coming
Extensions on answer rules are being actively developed. Currently, the following extensions are developed:
- additional operators to apply on the student answer
- additional check-methods to use in answer rule statements
- decision tree actions based on an answer rule (read more about decision trees here)
Please reach out if you are missing options or have ideas how extensions on answer rules can help you create better exercises.
Statement
A statement consists of 1 or more clauses in which each clause has four parts
- answer dependent variable (left hand side)
- inverse rule: "is" or "is not"
- relation (e.g. Algebraically Equivalent to, same Basis as)
- right hand side to compare left-hand-side to using relation
For the answer dependent variable you can select the answer of the student or any variables based on the answer of the student (e.g. the derivative of the answer).
In the example above the answer itself is selected by selecting "student.answer1".
The inverse ("is" or "is not") can be used to check if the statement is not true (e.g. "student.answer1 is not algebraically equivalent to x").
In the example above the inverse is set to the default value "is".
The relation defines how the answer dependent variable is compared to the right-hand-side. For the relation different check-methods can be used. In addition to "Algebraically Equivalent to", there are also check-methods such as "same Exact Form as", "same Basis as" and "Parallel to". For a full list of check-methods, check out this article.
In the example above the relation is set to "Algebraically Equivalent to".
At the right-hand-side you specify to which the left-hand-side (answer dependent variable) should be compared using the relation. Depending on the check-method the expected input is indicated above the right-hand-side input field.
In the example above the right-hand-side shows an algebraic expression "1/3 x^3 + C".
Correctness - Correct or Incorrect
When the statement is evaluated to true you can indicate whether the answer of the student is marked as correct or incorrect.
In the example above this option is set to "correct".
Specific feedback (optional)
Here you can specify the specific feedback which will be shown to the student when the statement is evaluated to true. In this specific feedback you can include answer dependent variables and parameters.
In the example above the answer dependent variable "derStudentAnswer" is used.
Multiple clauses and multiple answer rules
An answer rule consists out of a single statement and this single statement consists out of one or multiple clauses.
When to use multiple answer rules and when to use multiple clauses is explained below.
Multiple clauses
You will use multiple clauses if you want to check multiple characteristics of the student's answer to determine whether to provide specific feedback.
Clauses are evaluated one by one and only if ALL of them evaluate to true, the statement evaluates to true and thus the specific feedback is given to the student.
Below you can see an simple illustrative example of a statement with multiple clauses.
In this example two clauses are used in a single statement. These two clauses state the following:
- "student.answer1 is Parallel to [1,2,-1]"
- "student.answer1 is not Algebraically Equivalent to [1,2,-1]"
The answer of the student is marked as correct if they provide a vector which is parallel to [1,2,-1] AND the student's answer is not the vector [1,2,-1] itself.
Multiple answer rules
You can add multiple answer rules for each subquestion in an exercise. The answer rules are evaluated in order and only the first matching rule is used to generate the feedback for the student.
For each case you want to provide specific feedback, you create a new answer rule. The order in which you place them is the order in which they are evaluated.
Detailed example
For the example above we will add three additional answer rules. Two for common mistakes and one for a general mistake. For each of the rules we will provide specific feedback and create an answer dependent variable for one of the rules.
Common mistake 1: student forgets integration constant
Students often forget the integration constant. The answer rule to give specific feedback for that case is given below:
This rule is similar to the main rule. It checks whether the answer of the student is algebraically equivalent to the integrated function without C (1/3 x^3). In this case it also marks this answer as correct.
Although the answer is marked correct you can still provide feedback to indicate to the user they forgot to add the integration constant.
Below you see the feedback the student receives when answering with 1/3 x^3.
Common mistake 2: student adds another constant/variable to answer
Another common mistake is that the student adds another constant/variable to the integrated equation. For example: "1/3 x^3 + 1", or "1/3 x^3 + 3".
To catch these mistakes and provide specific feedback, we will create an answer dependent variable using the derivative operator. Below you can see how you define such a variable:
You can use all operators on student answers as you can use them on parameters. The available operators are actively expanded. The current list of operators can be found here.
With this new answer dependent variable we can create the answer rule:
In this rule we check whether the derivative of the student answer is algebraically equivalent to the original function. If the first two rules are not satisfied and this third rule is, it will mark the answer as incorrect and show the specific feedback shown at the bottom of the rule.
Below you see the feedback the student receives when answering with 1/3 x^3 + 1.
General mistake: feedback by taking the derivative of the answer
Finally, the student can give an algebraic expression which is not correct. Maybe you want to show the student what the result is of taking the derivative of their answer and show it is not equal to the original function.
To achieve this we use an inverted rule and an answer dependent variables in the feedback. Below you can see the configuration of the answer rule:
The rule states: "if the derivative of the student answer is not algebraically equivalent to the original function then mark as incorrect and show feedback below".
For example, student answers with x^3, the feedback will be as shown below:
We've created an exercise with four answer rules providing specific feedback to students when they make common mistakes and providing feedback with answer dependent variables using operations.
To explore further how you can use the answer rules for you specific exercise, read the following articles:
Extending answer rules options
We are actively extending the answer rule options. Currently the following extensions are developed:
- additional operators to apply on the student answer
- additional check-methods to use in answer rule statements
- decision tree actions based on an answer rule (read more about decision trees here)
If you have questions or you want to suggest extensions on answer rule options, operators or check methods, please reach out to use using the chat via the icon in the bottom right.