Bayesian Psychometric Modeling

A Single Cohesive Framework of Tools and Procedures for Psychometrics and Assessment Bayesian Psychometric Modeling presents a unified Bayesian approach across traditionally separate families of psychometric models. It shows that Bayesian techniques, as alternatives to conventional approaches, offer distinct and profound advantages in achieving many goals of psychometrics. Adopting a Bayesian approach can aid in unifying seemingly disparate—and sometimes conflicting—ideas and activities in psychometrics. This book explains both how to perform psychometrics using Bayesian methods and why many of the activities in psychometrics align with Bayesian thinking. The first part of the book introduces foundational principles and statistical models, including conceptual issues, normal distribution models, Markov chain Monte Carlo estimation, and regression. Focusing more directly on psychometrics, the second part covers popular psychometric models, including classical test theory, factor analysis, item response theory, latent class analysis, and Bayesian networks. Throughout the book, procedures are illustrated using examples primarily from educational assessments. A supplementary website provides the datasets, WinBUGS code, R code, and Netica files used in the examples.

FoundationsOverview of Assessment and Psychometric Modeling Assessment as Evidentiary Reasoning The Role of Probability The Role of Context in the Assessment Argument Evidence-Centered Design Summary and Looking Ahead Introduction to Bayesian Inference Review of Frequentist Inference via Maximum Likelihood Bayesian Inference Bernoulli and Binomial Models Summarizing Posterior Distributions Graphical Model Representation Analyses Using WinBUGS Summary and Bibliographic Note Exercises Conceptual Issues in Bayesian Inference Relative Influence of the Prior Distribution and the Data Specifying Prior Distributions Comparing Bayesian and Frequentist Inferences and Interpretations Exchangeability, Conditional Independence, and Bayesian Inference Why Bayes? Conceptualizations of Bayesian Modeling Summary and Bibliographic Note Normal Distribution Models Model with Unknown Mean and Known Variance Model with Known Mean and Unknown Variance Model with Unknown Mean and Unknown Variance Summary Exercises Markov Chain Monte Carlo Estimation Overview of MCMC Gibbs Sampling Metropolis Sampling How MCMC Facilitates Bayesian Modeling Metropolis–Hastings Sampling Single-Component-Metropolis or Metropolis-within-Gibbs Sampling Practical Issues in MCMC Summary and Bibliographic Note Exercises Regression Background and Notation Conditional Probability of the Data Conditionally Conjugate Prior Complete Model and Posterior Distribution MCMC Estimation Example: Regressing Test Scores on Previous Test Scores Summary and Bibliographic Note Exercises PsychometricsCanonical Bayesian Psychometric ModelingThree Kinds of DAGsCanonical Psychometric Model Bayesian Analysis Bayesian Methods and Conventional Psychometric Modeling Summary and Looking Ahead Exercises Classical Test Theory CTT with Known Measurement Model Parameters and Hyperparameters, Single Observable (Test or Measure) CTT with Known Measurement Model Parameters and Hyperparameters, Multiple Observables (Tests or Measures) CTT with Unknown Measurement Model Parameters and Hyperparameters Summary and Bibliographic Note Exercises Confirmatory Factor Analysis Conventional Factor Analysis Bayesian Factor Analysis Example: Single Latent Variable (Factor) Model Example: Multiple Latent Variable (Factor) Model CFA Using Summary Level Statistics Comparing DAGs and Path Diagrams A Hierarchical Model Construction Perspective Flexible Bayesian Modeling Latent Variable Indeterminacies from a Bayesian Modeling Perspective Summary and Bibliographic Note Exercises Model Evaluation Interpretability of the Results Model Checking Model Comparison Exercises Item Response Theory Conventional Item Response Theory Models for Dichotomous Observables Bayesian Modeling of Item Response Theory Models for Dichotomous Observables Conventional Item Response Theory Models for Polytomous Observables Bayesian Modeling of Item Response Theory Models for Polytomous Observables Multidimensional Item Response Theory Models Illustrative Applications Alternative Prior Distributions for Measurement Model Parameters Latent Response Variable Formulation and Data-Augmented Gibbs Sampling Summary and Bibliographic Note Exercises Missing Data Modeling Core Concepts in Missing Data Theory Inference under Ignorability Inference under Nonignorability Multiple Imputation Latent Variables, Missing Data, Parameters, and Unknowns Summary and Bibliographic Note Exercises Latent Class AnalysisConventional Latent Class Analysis Bayesian Latent Class Analysis Bayesian Analysis for Dichotomous Latent and Observable Variables Example: Academic Cheating Latent Variable Indeterminacies from a Bayesian Modeling Perspective Summary and Bibliographic Note Exercises Bayesian Networks Overview of Bayesian Networks Bayesian Networks as Psychometric Models Fitting Bayesian Networks Diagnostic Classification Models Bayesian Networks in Complex AssessmentDynamic Bayesian NetworksSummary and Bibliographic Note Exercises ConclusionBayes as a Useful Framework Some Caution in Mechanically (or Unthinkingly) Using Bayesian Approaches Final Words Appendix A: Full Conditional Distributions Appendix B: Probability Distributions References Index

Roy Levy is an associate professor of measurement and statistical analysis in the T. Denny Sanford School of Social and Family Dynamics at Arizona State University. His primary research and teaching interests include methodological developments and applications of psychometrics and statistics to assessment, education, and the social sciences. He has received awards from the President of the United States, Division D of the American Educational Research Association, and the National Council on Measurement in Education. Robert J. Mislevy is the Frederic M. Lord Chair in Measurement and Statistics at Educational Testing Service. He was previously a professor of measurement and statistics at the University of Maryland and an affiliated professor of second language acquisition and survey methodology. His research applies developments in statistics, technology, and psychology to practical problems in assessment, including the development of multiple-imputation analysis in the National Assessment of Educational Progress. He is a member of the National Academy of Education and has been a president of the Psychometric Society. He has received awards from the National Council on Measurement in Education and Division D of the American Educational Research Association.