PLACE:  319 Snedecor Hall

SPEAKER:
Amy Goodwin Froelich
University of Illinois

TITLE:
Statistical Issues Related to Item Response Theory

ABSTRACT:
 

The field of Item Response Theory (IRT) focuses on the statistical modeling and analysis of examinee responses to test questions (items).  In IRT, we are concerned with modeling the joint distribution of the examine response vector to the test items and the latent examine ability vector.  The vector of latent
examine abilities can be thought of as the set of dimensions or abilities that determine an examinee's responses to test items.  Thus, examine responses to an item are dependent upon the examinee's vector of latent abilities.  The function that specifies the conditional probability of an examine response to
an item given latent ability is called an Item Response Function (IRF).  The item response functions for a test are assumed to be conditionally independent given ability.  IRFs are modeled either parametrically (commonly assumed to follow a logistic model with one, two or three parameters) or non-parametrically (assumed to be monotone increasing as ability increases).

One theoretically challenging area of research in parametric IRT is the joint estimation of the parameters of the IRFs and the latent examine abilities.  The method currently used by most researchers to jointly estimate both the item parameters and examine abilities is the marginal maximum likelihood estimation (MMLE) method.  In this context, the item parameters are structural parameters, and the examine abilities are incidental parameters. Neyman & Scott (1948) showed that maximum likelihood estimates of such structural parameters are not necessarily consistent when structural parameters are jointly estimated with incidental parameters, since increasing sample size increases the number of incidental parameters to be estimated.  A proof of the consistency and asymptotic normality of the item parameter estimates obtained from the MMLE
method as both the number of items and the number of examinees tends to infinity for the one and two parameter logistic models will be presented.  This result is the first step in showing the MMLE method produces consistent estimates of both item parameters and examine abilities.

One area of research in non-parametric IRT is the assessment of the number of dimensions of the latent examine ability vector.  Based on the theory of item pair conditional covariances, Stout (1987) developed the hypothesis testing procedure DIMTEST to determine if test data is unidimensional (the vector of
latent abilities need consist of only one dimension).  A major improvement of the DIMTEST procedure will be presented.  This improvement consists of a new bias correction method for the DIMTEST statistic based on statistical resampling of the test data.  The resampling method consists of two parts. First, kernel smoothing, a non-parametric estimation method, is used to estimate the test's item response functions.  Then, given these estimated item response functions and a given examine ability distribution, additional data
sets are generated under the null hypothesis of unidimensionality.  A Monte-Carlo simulation study shows this new version of the DIMTEST procedure has Type I error rates approximately equal to the nominal rate in most cases and extremely good power to detect multidimensionality.
 
 
 
 

COFFEE: 3:40p.m., 104 Snedecor Hall