Psychological Development and Education ›› 2012, Vol. 28 ›› Issue (3): 329-336.

Previous Articles    

An M-Rasch Approach to Domain Score Reporting: Solution to Bandwidth-fidelity Dilemma

ZENG Ping-fei1, YU Na1, XIN Tao1, WANG Ye-hui2   

  1. 1. Institute of Developmental Psychology, Beijing Normal University, Beijing 100875;
    2. State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875
  • Online:2012-05-15 Published:2012-05-15

PDF (PC)

Abstract: Due to broad content coverage and limited testing time,the large scale assessment is challenged by the bandwidth-fidelity dilemma.This study is to explore how the multi-dimensional Rasch model would improve reliability in the within-item multi-dimensionality data.The results demonstrate that both 4-dimensional and 15-dimensional Rasch models fit data,which supports the construct validity of the test.The uni-dimensional model underestimates the correlation between domains due to measurement error.The multi-dimensional Rasch analysis yields a higher level of measurement precision and a more appropriate estimate for the correlation between domains as compared to uni-dimensional approach.The comparison between 4-dimension and 15-dimension analysis shows that the increase on the number of dimensions can compensate the effect of scale length reduction to a certain extent,as long as the correlations between the specific domain and the others are relatively high enough.In conclusion,the multi-dimensional Rasch analysis yields more reliable domain score than uni-dimensional Rasch model in the within-item multidimenionality context.For the test with fixed length,reliable scores can be reported on the more specified content domains,as long as there are high correlations between domains.

Key words: large scale assessment, bandwidth-fidelity dilemma, multi-dimensional Rasch model, within item dimensionality

CLC Number: 

  • B841
[1] Adams,R.J.,Wilson,M.,& Wang,W.C.(1997).The multidimensional random coefficients multinomial logit model.AppliedPsychological Measurement,21,1-23.
[2] Bock,R.D.,& Mislevy,R.J.(1981).An item response curve model for matrix-sampling data:The California grade-three assessment.New Directions for Testing and Measurement,10,65-90.
[3] Bock,R.D.,Thissen,D.,& Zimowski,M.F.(1997).IRT estimation of domain scores.Journal of educational measurement,34(3),197-211.
[4] Cheng,Y.Y.,Wang,W.C.,& Ho,Y.-H.(2009).
[5] Multidimensional Rasch analysis of a psychological test with multiple subtests:A statistical solution for the bandwidth——fidelity dilemma.Educational and Psychological Measurement,69(3),369-388.
[6] Cronbach,L.J.,& Gleser,G.C.(1965).Psychological tests and personnel decisions.Urbana:University of Illinois Press Urbana.Kahraman,N.,& Kamata,A.(2004).Increasing the precision of subscale scores by using out-of-scale information.Applied Psychological Measurement,28(6),407-426.
[7] Murphy,K.R.(1993).Honesty in the workplace.Pacific Grove:Brooks/Cole Pub Co.
[8] Ones,D.S.,& Viswesvaran,C.(1996).Bandwidth-fidelity dilemma in personality measurement for personnel selection.Journal of Organizational Behavior,17(6),609-626.
[9] Organization for Economic Co-operation and Development.(2005).The PISA 2003 technical report.Paris:Author.
[10] Pommerich,M.,Nicewander,W.A.,& Hanson,B.A.(1999).Estimating average domain scores.Journal of Educational Measurement,36(3),199-216.
[11] Sheng,Y.,& Wikle,C.K.(2008).Bayesian multidimensional IRT models with a hierarchical structure.Educational and Psychological Measurement,68(3),413-430.
[12] Wainer,H.,Sheehan,K.M.,& Wang,X.(2000).Some paths toward making praxis scores more useful.Journal of Educational Measurement,37(2),113-140.
[13] Wang,W.C.,Chen,P.H.,& Cheng,Y.Y.(2004).Improving measurement precision of test batteries using multidimensional item response models.Psychological Methods,9(1),116-135.
[14] Wright,B.D.,Linacre,J.M.,Gustafson,J.E.,& Martin-Lof,P.(1994).Reasonable mean-square fit values.Rasch Measurement Transactions,8(3),370.
[15] Wright,B.D.,& Mok,M.(2000).Understanding Rasch measurement:Rasch models overview.Journal of Applied
[16] Measurement,1(1),83-106.
[17] Wu,M.L.,Adams,R.J.,& Wilson,M.R.(1998).ConQuest ［Computer software and manual］.Camberwell,Victoria,Australia:Australian Council for Educational Research.
[18] Yao,L.,& Boughton,K.A.(2007).A multidimensional item response modeling approach for improving subscale proficiency estimation and classification.Applied Psychological Measurement,31(2),83-105.
[19] Yen,W.M.(1987).A Bayesian/IRT index of objective performance.Paper presented at the annual meeting of the Psychometric Society,Montreal,Quebec,Canada.
[20] 李凌艳,辛涛,董奇.(2007).矩阵取样技术在大尺度教育测评中的运用.北京师范大学学报(社会科学版),(6),19-25.
[1] TANG Wen-qing, FANG Jie, JIANG Xiang-mei, ZHANG Min-qiang. Review of the Application of Longitudinal Study Method in Psychological Researches in China [J]. Psychological Development and Education, 2014, 30(2): 216-224.
[2] ZHANG Xue-xin. Promoting the Rapid Growth of Chinese Science Journalswith the Open-Review System [J]. Psychological Development and Education, 2013, 29(1): 109-111.
[3] LI Guang-ming, ZHANG Min-qiang. Analysis of Cross-distribution for Estimating Variance Components in Generalizability Theory [J]. Psychological Development and Education, 2012, 28(6): 665-672.
[4] KANG Ting-hu, BAI Xue-jun. The Influence of Task Types and Gender Attribute on Interpersonal Trust [J]. Psychological Development and Education, 2012, 28(5): 456-462.
[5] MA Wen-chao, BIAN Yu-fang, LUO Fang. Internet Addiction Disorder: Categories or Dimensions? [J]. Psychological Development and Education, 2012, 28(5): 554-560.
[6] Xin Tao, Xie Min. Group-level Domain Score and Its Estimation Methods [J]. Psychological Development and Education, 2010, 26(4): 416-422.
[7] JIA Ning, BAI Xue-jun, SHEN De-li. Research Methods of the Accuracy of Judgment in Learning [J]. Psychological Development and Education, 2006, 22(3): 103-109.
[8] WANG Deng-feng, CUI Hong, HU Jun-sheng, CHEN Xia. Development of Chinese Adolescent Personality Scale(QZPS-Q) [J]. Psychological Development and Education, 2006, 22(3): 110-115.
[9] . [J]. Psychological Development and Education, 2006, 22(3): 120-124,128.
[10] JIAO Li-ya, XIN Tao. Comparison of Four Equating Methods in Common-Item NonEquivalent Group Design Based on The Classical Test Theory [J]. Psychological Development and Education, 2006, 22(1): 97-102.
[11] Zhang Ling-ling, Zhang Wen-xin, Ji Lin-qin, Jari-Erik Nurmi. Psychometric Analysis of Adolescent Future Orientation Questionnaire in Chinese Cultural Context [J]. Psychological Development and Education, 2006, 22(1): 103-108.
[12] WANG Yong-li 1LIN Chong-de, YU Guo-liang. The Development of the Children’ Social-life Adjustment Scale and Its Application [J]. Psychological Development and Education, 2005, 21(1): 109-114.
[13] LI Hong, KAM Weng, Boey. Assessing Psychological Well-being of College Student:Psychometric Properties of GHQ-20 [J]. Psychological Development and Education, 2002, 18(1): 75-79.
[14] . Revision for CCTDI(Chinese Version) [J]. Psychological Development and Education, 2001, 17(3): 47-51.
[15] . [J]. Psychological Development and Education, 2001, 17(3): 52-55.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of Psychological Development and Education
Supported by Beijing Magtech Co.Ltd