Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale
American Journal of Theoretical and Applied Statistics
Volume 3, Issue 2, March 2014, Pages: 31-38
Received: Dec. 31, 2013; Published: Mar. 20, 2014
Views 2651      Downloads 206
Authors
T. Oguz Basokcu, Department of Assessment and Evaluation in Education, Ege University, İzmir, Turkey
Tuncay Ogretmen, Department of Assessment and Evaluation in Education, Ege University, İzmir, Turkey
Article Tools
PDF
Follow on us
Abstract
This study aims to compare parametric and nonparametric methods based on Item Response Theory in determining differential item functioning in polytomous scales. DIF analysis based on parametric IRT was conducted by using parameters comparison method. For nonparametric IRT analysis, DIF is determined by comparison of area indices pertaining to ICC obtained for reference and focal groups. The Comparisons were conducted on data sets from TIMSS 2011 8th Class students survey where data set pertaining to responses of students to "Attitudes Toward Mathematics" composing of samplings from Turkey and South Korea and it was determined if it incorporated DIF according to country and sex differences. It is observed that parametric and nonparametric methods produce generally similar results for DIF analysis in terms of countries. Nevertheless, DIF analysis results for country based sex groups differed according to techniques based on parametric and nonparametric IRT. Results of the study showed that items incorporating DIF differed as to preferred technique. This indicated importance of choosing the best technique in studies to detect whether scale items incorporates DIF or not.
Keywords
Differential Item Functioning,Item Response Theory, Nonparametric Differential Item Functioning, Nonparametric IRT
To cite this article
T. Oguz Basokcu, Tuncay Ogretmen, Comparison of Parametric and Nonparametric Item Response Techniques in Determining Differential Item Functioning in Polytomous Scale, American Journal of Theoretical and Applied Statistics. Vol. 3, No. 2, 2014, pp. 31-38. doi: 10.11648/j.ajtas.20140302.11
References
[1]
Adams, R. J., & Rowe, K. J. (1988). Item Bias In J. P. Keeves (Ed.), Educational research, methodology, and measurement:An international handbook. (pp. 398-403). Oxford: Pergamon Press.
[2]
Adams, R. J., & Rowe, K. J. (1988). Item Bias In J. P. Keeves (Ed.), Educational research, methodology, and measurement:An international handbook. (pp. 398-403). Oxford: Pergamon Press.
[3]
Allen, M. J., & Yen, W. M. (1979). Introduction to Measurement Theory: Brooks/Cole Pub.
[4]
Carter, N. T., &Zickar, M. J. (2011). A Comparison of the LR and DFIT Frameworks of Differential Functioning Applied to the Generalized Graded Unfolding Model. Applied Psychological Measurement, 35(8), 623-642. doi: 10.1177/0146621611427898
[5]
Cohen, A. S., Kim, S.-H., & Baker, F. B. (1993). Detection of Differential Item Functioning in the Graded Response Model. Applied Psychological Measurement, 17(4), 335-350. doi: 10.1177/014662169301700402
[6]
Devine, P. J., &Raju, N. S. (1982). Extent of Overlap among Four Item Bias Methods. Educational and Psychological Measurement, 42(4), 1049-1066. doi: 10.1177/001316448204200412
[7]
Douglas, J. (1997). Joint consistency of nonparametric item characteristic curve and ability estimation. Psychometrika, 62(1), 7-28. doi: 10.1007/bf02294778
[8]
Finch, H. (2005). The MIMIC Model as a Method for Detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT Likelihood Ratio. Applied Psychological Measurement, 29(4), 278-295. doi: 10.1177/0146621605275728
[9]
Glickman, M., Seal, P., &Eisen, S. (2009). A non-parametric Bayesian diagnostic for detecting differential item functioning in IRT models. Health Services and Outcomes Research Methodology, 9(3), 145-161. doi: 10.1007/s10742-009-0052-4
[10]
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory: SAGE Publications.
[11]
Holland, P. W., & Thayer, D. T. (1986). Differential item performance and the Mantel-Haenszel procedure Technical Report (Vol. 86-69). Princeton, NJ: Educational Testing Service.
[12]
Holland, P. W., Wainer, H., & Service, E. T. (1993). Differential Item Functioning: Taylor & Francis.
[13]
Lord, F. M. (1977). A Broad-Range Tailored Test of Verbal Ability. Applied Psychological Measurement, 1(1), 95-100. doi: 10.1177/014662167700100115
[14]
Lord, F. M. (1980). Applications of item response theory to practical testing problems: Erlbaum Associates.
[15]
Mellenberg, G. J. (1983). Conditional Item Bias Methods. In S. H. Irvine & W. J. Barry (Eds.), Human Assesment and Cultural Factors (pp. 123-144). Newyork: Plenum Pres.
[16]
Murphy, K. R., &Davidshofer, C. O. (1998). Psychological testing: principles and applications: Prentice Hall.
[17]
Nozawa, Y. (2008). Comparison of Parametric and Nonparametric IRT Equating Methods Under The Common-Item Nonequıvalent Groups Desing. Doctor of Philosopy, The University of Iowa.
[18]
Raju, N. (1988). The area between two item characteristic curves. Psychometrika, 53(4), 495-502. doi: 10.1007/bf02294403
[19]
Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56(4), 611-630. doi: 10.1007/bf02294494
[20]
Ramsay, J. O. (2000). TestGraf A Program for the Graphical Analysis of Multiple Choice Test and Questionnaire Data Canada, Montreal, Quebec:McGill University
[21]
Rodney, G. L., &Drasgow, F. (1990). Evaluation of Two Methods for Estimating Item Response Theory Parameters When Assessing Differential Item Functioning. . Journal of Applied Psychology, 75(2), 164-174.
[22]
Roju, N. S., van der Linden, W. J., & Fleer, P. F. (1995). IRT-Based Internal Measures of Differential Functioning of Items and Tests. Applied Psychological Measurement, 19(4), 353-368. doi: 10.1177/014662169501900405
[23]
Roussos, L. A., & Stout, W. F. (1996). Simulation Studies of the Effects of Small Sample Size and Studied Item Parameters on SIBTEST and Mantel-Haenszel Type I Error Performance. Journal of Educational Measurement, 33(2), 215-230. doi: 10.1111/j.1745-3984.1996.tb00490.x
[24]
Rudner, L. M., Getson, P. R., & Knight, D. L. (1980). Biased Item Detection Techniques. Journal of Educational Statistics, 5(3), 213-233. doi: 10.2307/1164965
[25]
Shepard, L. A., Camilli, G., & Williams, D. M. (1985). Validity of Approximation Techniques for Detecting Item Bias. Journal of Educational Measurement, 22(2), 77-105. doi: 10.2307/1435009
[26]
Sijtsma, K., &Molenaar, I. W. (2002). Introduction to Nonparametric Item Response Theory: SAGE Publications.
[27]
Torgerson, W. S. (1958). Theory and methods of scaling: Wiley.
[28]
Wand, P., & Jones, C. (1994). Kernel Smoothing: Taylor & Francis.
[29]
Wang, W., Tay, L., &Drasgow, F. (2013). Detecting Differential Item Functioning of Polytomous Items for an Ideal Point Response Process. Applied Psychological Measurement, 37(4), 316-335. doi: 10.1177/0146621613476156
[30]
Zumbo, B. D., &Hubley, A. M. (2003). Item Bias. Encyclopedia of Psychological Assessment. SAGE Publications Ltd. London: SAGE Publications Ltd.
ADDRESS
Science Publishing Group
1 Rockefeller Plaza,
10th and 11th Floors,
New York, NY 10020
U.S.A.
Tel: (001)347-983-5186