Diagnostic classification
models

A brief introduction

W. Jake Thompson, Ph.D.

What are diagnostic models?

  • Traditional assessments and psychometric models measure an overall skill or ability
  • Assume a continuous latent trait

A normal distribution with images of Taylor Swift from each era overlayed.

  • The output is a weak ordering due to error in estimates
    • Confident Taylor Swift (debut) is the worst
    • Not confident on ordering toward the middle of the distribution

A normal distribution with images of Taylor Swift from each era overlayed.

  • Limited in the types of questions that can be answered.
    • Why is Taylor Swift (debut) so low?
    • What aspects do each era demonstrate proficiency or competency of?
    • How much skill is “enough” to be competent?

A normal distribution with images of Taylor Swift from each era overlayed.

Diagnostic measurement

  • Designed to be multidimensional
  • No continuum of student achievement
  • Categorical constructs
    • Usually binary (e.g., master/nonmaster, proficient/not proficient)
  • Several different names in the literature
    • Diagnostic classification models (DCMs)
    • Cognitive diagnostic models (CDMs)
    • Skills assessment models
    • Latent response models
    • Restricted latent class models

Diagnostic music assessment

  • Rather than measuring overall musical knowledge, we can break music down into set of skills or attributes
    • Songwriting
    • Production
    • Vocals

Three circles representing the 3 attributes. The bottom half of each circle is shaded dark, and the top half is light, to indicate there are two categories for each attribute.

  • Attributes are categorical, often dichotomous (e.g., proficient vs. non-proficient)

Diagnostic classification models

  • DCMs place individuals into groups according to proficiency of multiple attributes
songwriting production vocals
Xmark Xmark Xmark
Xmark Check Check
Check Xmark Check
Check Check Check

Benefits of DCMs

  • Fine-grained, multidimensional results. Answer more questions:
    • Why is Taylor Swift (debut) so low?
      • Subpar songwriting, production, and vocals
    • What aspects are albums competent/proficient in?
      • DCMs provide classifications directly
  • High reliability with fewer items
    • Less information need to classify than to place precisely along a scale
songwriting production vocals
Xmark Xmark Xmark
Xmark Check Check
Check Xmark Check
Check Check Check

Results from DCM-based assessments

songwriting production vocals
Xmark Xmark Xmark
Check Xmark Xmark