Advanced diagnostic
classification modeling

With R and Stan

W. Jake Thompson, Ph.D.

Welcome!

Who am I?


W. Jake Thompson, Ph.D.

  • Assistant Director of Psychometrics
    • ATLAS | University of Kansas
  • Research: Applications of diagnostic psychometric models

  @wjakethompson
  @wjakethompson.com

Acknowledgements

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grants R305D210045 and R305D240032 to the University of Kansas Center for Research, Inc., ATLAS. The opinions expressed are those of the authors and do not represent the views of the the Institute or the U.S. Department of Education.

Logo for the Institute of Education Sciences.

Schedule

Part 1: Foundations

Measurement and structural models

Part 2: Applications

Model misspecifications

Materials

All materials are available on the workshop website:

https://learn.r-dcm.org

Installation

  • Required
    • R (≥ 4.5.0)
    • rstan (≥ 2.32.7)
    • measr (≥ 2.0.1)
  • Recommended
    • RStudio (≥ 2026.01.1-403)
    • cmdstanr (≥ 0.9.0)
    • CmdStan (≥ 2.38.0)

Copy and run

install.packages(c("measr", "tidyverse", "pysch", "here", "fs", "usethis"))


# Optional
install.packages(
  c("rstan", "StanHeaders", "cmdstanr"),
  repos = c("https://stan-dev.r-universe.dev", getOption("repos"))
)

## check toolchain
cmdstanr::check_cmdstan_toolchain()
cmdstanr::install_cmdstan(cores = 2)

For help installing RStan, CmdStanR, or configuring the toolchain, see the prework page.

DCMs: A quick review

What are diagnostic models?

  • DCMs are psychometric models designed to classify
    • There is no scale score
    • We can define our attributes in any way that we choose
    • Items depend on the attribute definitions
    • Classifications are probabilistic
  • DCMs provide valuable information with more feasible data demands than other psychometric models
    • Higher reliability than IRT/MIRT models
    • Complex item structures possible
    • Criterion-referenced interpretations
    • Alignment of assessment goals and psychometric model

Statistical foundation

  • Latent class models use responses to probabilistically place individuals into latent classes

  • DCMs are confirmatory latent class models

    • Latent classes specified a priori as attribute profiles
    • Q-matrix specifies item-attribute structure
    • Person parameters are attribute proficiency probabilities

Terminology

  • Respondents (r): The individuals from whom behavioral data are collected

    • For today, this is dichotomous assessment item responses
    • Not limited to only item responses in practice

  • Items (i): Assessment questions used to classify/diagnose respondents

  • Attributes (a): Unobserved latent categorical characteristics underlying the behaviors (i.e., diagnostic status)

    • Latent variables

Attribute profiles

  • With binary attributes, there are 2A possible profiles

  • Example 3-attribute assessment:

[0, 0, 0]
[1, 0, 0]
[0, 1, 0]
[0, 0, 1]
[1, 1, 0]
[1, 0, 1]
[0, 1, 1]
[1, 1, 1]

DCMs as latent class models

\[ \color{#D55E00}{P(X_r=x_r)} = \sum_{c=1}^C\color{#009E73}{\nu_c} \prod_{i=1}^I\color{#56B4E9}{\pi_{ic}^{x_{ir}}(1-\pi_{ic})^{1 - x_{ir}}} \]

Observed data: Probability of observing examinee r's item reponses
Structural component: Proportion of examinees in each class
Measurement component: Product of item response probabilities

Structural models

\[ \color{#D55E00}{P(X_r=x_r)} = \sum_{c=1}^C\color{#009E73}{\nu_c} \prod_{i=1}^I\color{#56B4E9}{\pi_{ic}^{x_{ir}}(1-\pi_{ic})^{1 - x_{ir}}} \]

Structural component: Proportion of examinees in each class

Measurement models

\[ \color{#D55E00}{P(X_r=x_r)} = \sum_{c=1}^C\color{#009E73}{\nu_c} \prod_{i=1}^I\color{#56B4E9}{\pi_{ic}^{x_{ir}}(1-\pi_{ic})^{1 - x_{ir}}} \]

Measurement component: Product of item response probabilities
  • Traditional psychometrics: Item response theory, classical test theory
    • A single, unidimensional construct
    • Student results estimated on a continuum
    • Performance on individual items determined by an “item characteristic curve”
  • DCMs: Many different options

Estimation options

Existing software


Software Programs

  • Mplus, flexMIRT, mdltm
  • Limitations
    • Tedious to implement, expensive, limited licenses, etc.

R Packages

  • CDM, GDINA, mirt, blatant
  • Limitations
    • Limited to constrained DCMs, under-documented
    • Different packages have different functionality, and don’t talk to each other

DCMs with Stan

Stan logo.

Limitations of Stan for DCMs

  • Very tedious—prone to typos

  • Complexity increases with the number of attributes and item structure

  • Stan code must be customized to each particular Q-matrix

  • Just need to automate the creation of the Stan scripts…

Hex logo for the measr R package.

What is measr?

  • R package that automates the creation of Stan scripts for DCMs
  • Wraps rstan or cmdstanr to estimate the models
  • Provides additional functions to automate the evaluation of DCMs
    • Model fit
    • Classification accuracy and consistency

The rest of today


Diagnostic classification models

A quick review

https://learn.r-dcm.org