|
Sept 14
Ordered-Category
Attribute Coding Framework for Cognitive Assessments
Tzur Karelitz,
Ph.D.
BEAR Center, GSE,
UC Berkeley
Some
Cognitive Diagnostic Assessment models define skills as binary.
Examinees are described as either `skill masters' or `non-masters'
and items as either requiring the skill or not. This approach is
based on the Q matrix theory (Tatsuoka, 1995), that uses a binary
matrix to represent task requirements in terms of underlying skills
and knowledge.
I propose an Ordered Category Attribute Coding (OCAC) framework
designed to enhance the diagnostic information provided by such
models. This approach defines any skill, k, by the M_k steps taken
to master it. Consequently, the entries of the categorical Q matrix
represent skills' mastery level required by test items and examinees'
knowledge patterns represent their location on the learning path
of each skill. To illustrate the OCAC approach, consider non-native
English speakers who study English. They learn the various tenses,
and how to apply them in different settings. For instance, the attribute
``Mastery of past tense'' can be performed at many levels:
None-
no ability to use past tense.
Basic- ability to transform a regular verb to past tense.
Moderate- ability to transform a sentence from present simple to
past simple, using regular verbs.
Advanced- ability to transform a sentence from any tense to past
tense, using regular verbs.
Master- ability to transform a sentence from any tense to past tense,
using regular and irregular verbs.
An
exam in English tense proficiency can be represented by crossing
every tense with these 5 mastery levels. The flexibility of the
OCAC framework allows for a more informative, parsimonious and efficient
representation of task requirements and examinee knowledge. The
levels of required skills can be estimated simultaneously with the
examinees knowledge states as well as noise parameters, with high
recovery rate of simulated and real data.
|
