Function Repository Resource:

AssessmentTree

Source Notebook

Create a tree of AssessmentFunction expressions to traverse during assessment

Contributed by: Pulkit Gangwar, Danny Finn, Bob Sandheinrich and Christophe Pakleza

ResourceFunction["AssessmentTree"][nodes,path]

returns a tree of nodes which will be traversed according to path.

ResourceFunction["AssessmentTree"][assess]

automatically generates an AssessmentTree for AssessmentFunction assess for certain comparison methods.

ResourceFunction["AssessmentTree"][…][answer]

returns an AssessmentResultObject representing the correctness of answer as determined by traversing the ResourceFunction["AssessmentTree"].

Details and Options

The nodes can take the following forms:

{node₁,…}

list of AssessmentFunction objects, comparator functions and/or other ResourceFunction["AssessmentTree"] expressions

<|"lbl₁"→node₁,…|>

Association with keys being node labels and values being AssessmentFunction, comparator function or other ResourceFunction["AssessmentTree"] expressions

node

a single AssessmentFunction, comparator function or another ResourceFunction["AssessmentTree"]

path should take the form of an Association <|"origin"→<|True→"destination₁",False→"destination₂"|>,…|>. Assessment of the answer occurs at each node along the path. The correctness of an answer at "origin" determines whether the next node will be "destination₁" or "destination₂". "origin", "destination₁" and "destination₂" must match the nodelabels in nodes.

If the nodes are provided as a list, default nodelabels of "Node1", "Node2"… will be applied. These labels should be used for path in this case.

ResourceFunction["AssessmentTree"][assess] is a quick way to create an automatic ResourceFunction["AssessmentTree"] for the case where assess is an AssessmentFunction with a mathematical comparison method. It appends an end node to path which returns an explanation if the answer is mathematically equivalent to the correct answer, but has not been provided in the required form.

If any identifier is not present in the path, that node will be treated as a terminating node.

The following option values can be provided for the ResourceFunction["AssessmentTree"]:

"ScoreCombiner"

Total

function to combine node "Score" values

"AnswerCorrectCombiner"

Apply[Or]

function to combine node "AnswerCorrect" values

ResourceFunction["AssessmentTree"] enables follow-through marking in multi-part assessments. This means that if a student makes an error in an earlier part of a question, they can still earn marks for subsequent parts where they correctly use the incorrect answer from the earlier part, as long as their approach remains correct.

Examples

Basic Examples (2)

Create a simple AssessmentTree which first checks if the answer is prime, and second checks if it is odd:

In[1]:=

at = ResourceFunction["AssessmentTree"][
<|
"Node1" -> PrimeQ,
"Node2" -> OddQ
|>,
<|
"Node1" -> <|False -> "Node2", True -> "Node2"|>
|>
]

Out[1]=

It awards 1 point for each correct assessment node:

In[2]:=

Out[2]=

AssessmentTree can be nested:

In[3]:=

at = ResourceFunction["AssessmentTree"][
<|
"Outer1" -> ResourceFunction["AssessmentTree"][
<|
"Inner1" -> GreaterThan[5],
"Inner2" -> (Divisible[#, 3] &)
|>,
<|"Inner1" -> <|False -> "Inner2", True -> "Inner2"|>|>
],
"Outer2" -> AssessmentFunction[12]
|>,
<|"Outer1" -> <|True -> "Outer2"|>|>
]

Out[3]=

In[4]:=

Out[4]=

Scope (2)

Create an AssessmentTree that will evaluate the first node, and if the result is false, the second node will be evaluated. Note that the two AssessmentFunctions are applying different comparison methods:

In[5]:=

at = ResourceFunction["AssessmentTree"][
<|
"Node1" -> AssessmentFunction[
(x - 1) (x + 1) -> <|
"Explanation" -> "Well done, you successfully factorized the polynomial."|>,
"PolynomialResult"
],
"Node2" -> AssessmentFunction[
(x - 1) (x + 1) -> <|"Score" -> 0.5, "Explanation" -> StringJoin[
"Your answer is mathematically equivalent to the correct",
" answer, but has not been provided not in the required form."
]|>,
"CalculusResult"
]
|>,
<|
"Node1" -> <|False -> "Node2"|>
|>
]

Out[5]=

View the scores and explanations for different answers:

In[6]:=

$Dataset[<|# -> at[#][{"Score", "Explanation"}] & /@ {(x + 1) (x - 1), x^2 - 1}|>]$

Out[6]=

AssessmentTree also works with QuestionObject:

In[7]:=

$QuestionObject["Factor the polynomial \!$\*TagBox[\(\*SuperscriptBox[\(x$, $2$] - 1\), HoldForm]\).", at]$

Out[7]=

Often we will want to assess if the provided answer is mathematically equivalent to the correct answer, but not in the correct form. For this reason, it is possible to automatically create an AssessmentTree of this sort by wrapping AssessmentTree around an AssessmentFunction where the ComparisonMethod is one of {"ArithmeticResult", "PolynomialResult", "CalculusResult" or "AlgebraicValue"}:

In[8]:=

at = ResourceFunction["AssessmentTree"][
AssessmentFunction[
(x - 1) (x + 1) -> <|
"Explanation" -> "Well done, you successfully factorized the polynomial."|>,
"PolynomialResult"
]
]

Out[8]=

Create a summary of results for two different inputs:

In[9]:=

$Dataset[<|# -> at[#][{"Score", "Explanation"}] & /@ {(x + 1) (x - 1), x^2 - 1}|>]$

Out[9]=

Wrapping the AssessmentFunction in AssessmentTree invoked the inclusion of the second explanation into the assessment path.

Options (4)

ScoreCombiner (2)

By default, the combined score for an AssessmentTree is the Total of the scores of the nodes traversed:

In[10]:=

ResourceFunction["AssessmentTree"][
<|
"Node1" -> (True &), (* Always award 1 point *)
"Node2" -> (True &), (* Always award 1 point *)
"Node3" -> (False &) (* Always award 0 points *)
|>,
<|"Node1" -> <|True -> "Node2"|>, "Node2" -> <|True -> "Node3"|>|>
][x]["Score"]

Out[10]=

A custom score combiner can also be specified:

In[11]:=

Out[11]=

AnswerCorrectCombiner (2)

By default, Apply[Or] is applied to determine the combined correctness of an AssessmentTree:

In[12]:=

ResourceFunction["AssessmentTree"][
<|
"Node1" -> (True &), (* Always mark correct *)
"Node2" -> (True &), (* Always mark correct *)
"Node3" -> (False &) (* Always mark incorrect *)
|>,
<|"Node1" -> <|True -> "Node2"|>, "Node2" -> <|True -> "Node3"|>|>
][x]["AnswerCorrect"]

Out[12]=

A custom answer correct combiner can also be specified. The following is a table of useful combiner functions:

In[13]:=

(* Define a combiner testing function *)
at = ResourceFunction["AssessmentTree"][
<|
"Node1" -> (True &), (* Always mark correct *)
"Node2" -> (True &), (* Always mark correct *)
"Node3" -> (False &) (* Always mark incorrect *)
|>,
<|"Node1" -> <|True -> "Node2"|>, "Node2" -> <|True -> "Node3"|>|>,
"AnswerCorrectCombiner" -> #
][x]["AnswerCorrect"] &;
Dataset[<|
Apply[Or] -> <|"Description" -> "Partially Correct", "CombinedResult" -> at[Apply[Or]]|>,
Apply[And] -> <|"Description" -> "Fully Correct", "CombinedResult" -> at[Apply[And]]|>,
(First[Commonest[#]] &) -> <|"Description" -> "Mostly Correct", "CombinedResult" -> at[(First[Commonest[#]] &)]|>,
Last -> <|"Description" -> "Correct in the End", "CombinedResult" -> at[Last]|>
|>]

Out[14]=

Applications (2)

We can define an AssessmentTree for the following problem which will award the student 1 point if they correctly answer part (a), and award 1 point if they apply the correct methodology to part (b). Example: a) Integrate sin(x) with respect to x. b) Evaluate the definite integral from x = 0 to x = π.

In[15]:=

$at = ResourceFunction["AssessmentTree"][ <| "Indefinite" -> (AssessmentFunction[ (-Cos[x] + C[1]) -> <| "Score" -> 1, "Explanation" -> "You successfully integrated sin(x)." |>, "CalculusResult" ][ #[[1]] (* 1st answer *) ] &), "Definite" -> (AssessmentFunction[ (* Answer key defined as a function of the 1st answer *) ((#[[1]] /. x -> \[Pi]) - (#[[1]] /. x -> 0)) -> <| "Score" -> 1, "Explanation" -> "You successfully applied the limits to the definite integral." |> ][ #[[2]] (* 2nd answer *) ] &) |>, <| (* Both branches of the tree are the same *) "Indefinite" -> <|True -> "Definite", False -> "Definite"|> |>, "AnswerCorrectCombiner" -> Apply[And] ]$

Out[15]=

In[16]:=

$QuestionObject[ QuestionInterface["TextCompletion", <| "Prompt" -> "Definite integral", "Template" -> "Integrate sin(x) with respect to x. `` Evaluate the definite integral from x = 0 to x = \[Pi]. ``" |>], at[Interpreter["MathExpression"][#]] & ]$

Out[16]=

Dataset[<|# → at[#][{"Score", "Explanation"}]& /@ { {-Cos[x], 2}, (* Student gets both parts correct *) {Cos[x], -2} (* Student forgets the minus sign in part (a) but successfully applies limits in part (b) *) }|>]

Out[17]=

Example: Given the matrix a) Compute the determinant. b) Compute the adjugate. c) Hence, compute the inverse.

We can define an AssessmentTree which will award the student 1 point if they correctly answer part (a), 1 point if they correctly answer part (b), and award 1 point if they apply the correct methodology to part (c):

In[18]:=

at = With[{mat = {{1, 2}, {3, 4}}},
ResourceFunction["AssessmentTree"][
<|
"Determinant" -> (AssessmentFunction[
Det[mat] -> <|
"Score" -> 1, "Explanation" -> "You successfully computed the determinant."
|>
][#[[1]] (* 1st answer *)] &),
"Adjugate" -> (AssessmentFunction[
Adjugate[mat] -> <|
"Score" -> 1, "Explanation" -> "You successfully computed the adjugate."
|>
][#[[2]] (* 2nd answer *)] &),
"Inverse" -> (AssessmentFunction[
(* Answer key defined as a function of the 1st and 2nd answers *)
(#[[2]] / #[[1]]) -> <|
"Score" -> 1, "Explanation" -> "You applied the correct method to combine the determinant and adjugate to produce the inverse."
|>
][#[[3]] (* 3rd answer *)] &)
|>,
<|
(* Both branches of the tree are the same *)
"Determinant" -> <|True -> "Adjugate", False -> "Adjugate"|>,
"Adjugate" -> <|True -> "Inverse", False -> "Inverse"|>
|>
]
]

Out[18]=

In[19]:=

$QuestionObject[ QuestionInterface["TextCompletion", <| "Prompt" -> "Given the matrix \!$\*TagBox[\((\[NoBreak]\*GridBox[{ {1, 2}, {3, 4} },\nGridBoxAlignment->{\"Columns\" -> {{Center}}, \"Rows\" -> {{Baseline}}},\nGridBoxSpacings->{\"Columns\" -> {\nOffset[0.27999999999999997`], {\nOffset[0.7]}, \nOffset[0.27999999999999997`]}, \"Rows\" -> {\nOffset[0.2], {\nOffset[0.4]}, \nOffset[0.2]}}]\[NoBreak])$, Function[BoxForm`e$, \nMatrixForm[BoxForm`e$]]]\)", "Template" -> "a) Compute the determinant. `` b) Compute the adjugate. `` c) Hence, compute the inverse. ``" |>], at[Interpreter["MathExpression"][#]] & ]$