We examined differential item functioning dif indicators for four variables that repeatedly have. A biased item deviates from the item response theory irt models used in naep, because the probability of doing well on the item depends not only on what the examinee knows and can do and on the item as reflected in the item parameters, but also on a characteristic of the item that is unrelated to the construct being measured. An r package for rajus differential functioning of items and tests framework. Logistic regression provides a flexible framework for detecting various types of differential item functioning dif. Dif occurs when examinees from different groups show differing probabilities of success on or endorsing the item after matching on the construct that the item is intended to measure notice that this is exactly the definition of mi applied to test items. S115s123 o ne approach to the detection of differential item functioning dif is to use logistic regression lrbased techniques. Judicious application of this methodology by the researchers, however, requires an understanding of the technical complexities involved. It occurs when test items function differently for students from two different comparison groups that are matched by the construct. From what i found it is clear that the model would look something like. Irt differential item functioning tool assess computerized. An item displays dif when test takers possessing the same amount of an ability or trait, but belonging to different subgroups, do not share the same likelihood of correctly answering the item. Measurement invariance and differential item functioning. A new method for estimating differential item functioning dif for. Software for the computation of the statistics involved in item response theory likelihoodratio tests for differential item functioning, 2001, unpublished manuscript to complete dif analyses.
This article introduces these 45 r packages with their descriptions and features. Paper 29002015 multiple ways to detect differential item. A general framework and an r package for the detection of. The definitions, methods, and interpretations of differential item functioning are extended to the transpose of the usual personitem matrices.
Differential item functioning dif differential item functioning dif refers to the differences in item functioning. This paper presents dfit, an r package that implements the differential functioning of items and tests framework as well as the monte carlo item parameter replication approach for producing cutoff points for differential item functioning indices. Software for analyzing differential item functioning. Pdf differential item functioning dif has been increasingly applied in. Table 30 supports the investigation of item bias, differential item functioning dif, i.
Try to search for differential item functioning bayesian to find some resources. The purpose of dif analyses is to detect response differences of items in questionnaires, rating scales, or tests across different. Differential item functioning differential item functioning dif analysis can be used to examine whether items function similarly across different groups and identify items that appear to be too easy or difficult after controlling for the ability levels of the compared groups. Tilburg university differential item functioning and educational risk. Why differential item functioning analyses are an important part of instrument development and. Differential item functioning dif is when a test item favors or hinders a characteristic exhibited by group members of a testtaking population. The rows in each group refer to the levels from lower to higher, with the fourth row indicating the sum of each ability level. An r package for rajus differential functioning of. This issue, known as test bias, has been the subject of a great deal of recent research, and a technique called differential item functioning dif analysis has become the new standard in psychometric bias analysis. A items exhibiting no dif, b items exhibiting a weak indication of dif, or c items exhibiting a strong indication of dif. Differential item functioning dif is the preferred psychometric term for what is otherwise known as item bias. Average item scores for subgroups having the same overall score on the test are compared to determine whether the item is measuring in essentially the same way for all. In this investigation, a large sample, representative of a major university on key demographic. Naep technical documentation number of items by severity of differential item functioning in the mathematics combined national and state assessment.
Starting from a framework for classifying dif detection methods and from a comparative. The analysis of differential item functioning dif examines whether item responses differ according to characteristics such as language and ethnicity, when people with matching ability levels respond differently to the items. Therefore, a broad range of items is needed, which difficultylevels scatters widely over the scale. Average item scores for subgroups having the same overall score on the test are compared to determine whether the item is measuring in essentially the same way for all subgroups.
The torr is designed for use in school and university settings, and therefore, its measurement invariance across diverse groups is critical. As such, software that estimates twoparameter irt models is required for. In brief, differential item functioning dif occurs when groups such as defined by gender, ethnicity, age, or education have different probabilities of endorsing a given item on a multiitem scale after controlling for overall scale scores. Ministry of science and innovation under the european regional development fund. Pdf an introduction to differential item functioning. Differential item functioning dif is a statistical characteristic of an item that shows the extent to which the item might be measuring different abilities for members of separate subgroups. If dif is found for many items on the test, the final test scores do. Burton, the effect of item screening on test scores and test characteristics. Dif analyses are statistical procedures used to determine to what extent the content of an item affects the item endorsement of subgroups of testtakers. An overview of differential item functioning in multistage computer. Pdf an introduction to differential item functioning researchgate. The primary purpose is to enhance diagnostic assessment in which individual differences in scores between content domains are clarified by conditioning the scores on item difficulty. The chull software program is a matlab graphical user interface that performs the chull procedure. Current issues 951 people who answered the item correctly at the ability level m, and the proportion of people who answered the item correctly at the ability level m, respectively.
The purpose of the present analysis is to use differential item functioning dif to identify differences in the performance of native and immigrant students in pisa 2009 that can be directly related to their responses to particular items. Thus, differentially functioning items elicit different. This has become an impediment in the way of specially nonmathematically oriented researches. A variety of statistical procedures have been developed to assess dif in tests of dichotomous hills, 1989. How to write syntax for differential item functioning dif. We analyzed 95 cognitive reading items, administered to students in 29 european countries. Differential item functioning dif is a statistical characteristic of an item that shows the extent to. The primary concern in test development and test use, as bachman 1990. Previous efforts extended the framework by using item response theory irt based trait scores, and by employing an iterative process using groupspecific item parameters to account for dif in the trait scores, analogous to purification. Differential item functioning dif has been increasingly applied in fairness studies in psychometric circles. Im calibrating an item pool which enables enables especially to measure different abilitylevels. Measuring differential item and test functioning across. Figure 1 displays a scatterplot for the males and females item difficulties.
Lewis, a note on the value of including the studied item in the test score when analyzing test items for dif. We provide a tutorial on differential item functioning dif analysis. Three examples are used to illustrate this approach. Doing so requires a careful balancing of the contributions of technology, psychometrics, test design, and the learning sciences. Neither the list of the software nor the studies cited are meant to be. Software for analyzing differential item functioning using the. Differential item functioning magnitude and impact. Thus it can be chosen as the member of anchor items.
Several methods have been proposed in recent decades for identifying items that function differently between two or more groups of examinees. This article presents an ordinal logistic model for. Dorans, evaluating hypotheses about differential item functioning. Perhaps the item is tapping a secondary factor or factors overandabove the one of interest. Frontiers assessing differential item functioning on the. Logistic regression modeling as a unitary framework for binary and likerttype ordinal item scores. This is the webpage for the handbook on differential item functioning. The test of relational reasoning torr is designed to assess the ability to identify complex patterns within visuospatial stimuli. It also describes possible advanced irt models using r packages, as well as dichotomous and polytomous irt models, and r packages that contain applications such as differential item functioning and equating are also introduced. Naep analysis and scaling differential item functioning. Analyze dif with specialized software like dfit or parscale.
If the factor bringing about such a difference is not part of the construct of focus in the test, then the test would be biased. A new method for estimating differential item functioning dif for multiple groups and polytomous items. Human development index with uneven distribution of wealth as. Comparison of three software programs for evaluating dif. Differential item functioning dif is an important issue of interest in psychometrics and educational measurement. Package difr may 14, 2018 type package title collection of methods to detect dichotomous differential item functioning dif version 5. Differential item functioning dif, as an assessment tool, has been widely used in quantitative psychology, educational measurement, business management, and insurance and healthcare industries. This analysis can be performed by calculating various statistics, one of the most important being the mantelhaenszel, which can be carried out with. Differential item functioning analysis with ordinal. Naep tdw number of items by severity of differential. Software for analyzing differential item functioning using the mantel haenszel and standardization procedures.
1496 145 1388 1157 912 1310 927 1024 72 1285 683 601 479 337 750 1431 666 1022 1345 1249 1037 290 642 488 1142 71 1415 744 347 1200 1044 710 302 973 1132 936 502 528 202