Psychological Development and Education ›› 2012, Vol. 28 ›› Issue (6): 665-672.

Previous Articles    

Analysis of Cross-distribution for Estimating Variance Components in Generalizability Theory

LI Guang-ming1, ZHANG Min-qiang2   

  1. 1. Department of Psychology, School of Education, Guangzhou University, Guangzhou 510006, China;
    2. Research Center of Psychological Application, South China Normal University, Guangzhou 510631, China
  • Online:2012-11-15 Published:2012-11-15

PDF (PC)

Abstract: Estimating variability is an essential part of generalizability theory and is of central importance.The study adopted Monte Carlo data simulation technique to explore the effect of three data distribution on four method of estimating variance components for generalizability theory.Three data distribution were normal data distribution, dichotomous data distribution and polytomous data distribution.Four estimated methods were traditional method, bootstrap method,jackknife method and Markov Chain Monte Carlo method(MCMC).The results show that the performance of four methods is different for three data distribution.Traditional method is good for normal distribution data and polychromous distribution data.But it is not good and needs to be adjusted for dichotomous distribution data.Jackknife method accurately estimates variance components for three data distribution.As for estimating variance components,adjusted bootstrap method is better than unadjusted bootstrap methods.Compared with MCMC method with non-informative priors,MCMC method with informative priors is good for estimating variance components in generalizability theory.Data distribution has an effect on the method of estimating variance components for generalizability theory.Those methods,which can be applied for normal data distribution,could not be applied for other distribution data such as dichotomous data distribution and polytomous data distribution.Data distribution imposes restrictions on estimating variance components for these four methods.So different methods need be distinguished to use to do a good analysis of cross-distribution for estimating variance components in generalizability theory.

Key words: Generalizability Theory, Estimating variance components, Analysis of cross-distribution, Monte Carlo data simulation

CLC Number: 

  • B841
[1] American Educational Research Association,American Psychological Association,National Council on Measurement in Education.(1985).Standards for educational and psychological testing.Washington,DC:Author.
[2] Brennan,R.L.(1992).Elements of generalizability theory(Rev. ed.).Iowa City,IA:ACT.
[3] Brennan,R.L.(2000).(Mis)conceptions about generalizability theory.Educational Measurement:Issues and Practice,19(1),5-10.
[4] Brennan,R.L.(2001).Generalizability theory.New York:Springer-Verlag.
[5] Brennan,R.L.,Harris,D.J.,&Hanson,B.A.(1987).The bootstrap and other procedures for examining the variability of estimated variance components in testing contexts(ACT Research Report Series87-7).Iowa City,IA:American College Testing Program.
[6] Briggs,D.C.,&Wilson,M.(2007).Generalizability in item response modeling.Journal of Educational Measurement,44(2),131-155.
[7] Feng,W.C.(2002).Applicability of the jackknife procedure for estimating standard errors of variance component estimates in selected random effects G study designs.Unpublished doctoral dissertation,University of Iowa.
[8] Lane,S.,Liu,M.,Ankenmann,R.D.,&Stone,C.A.(1996).Generalizability and validity of mathematics performance assessment.Journal of Educational Measurement,33(1),71-92.
[9] Lunn,D.J.,Thomas,A.,Best,N.,&Spiegelhalter,D.(2000).WinBUGS——a Bayesian modelling framework:concepts,structure, and extensibility.Statistics and Computing,10,325-337.
[10] Martyn,P.,Nicky,B.,Kate,C.,&Karen,V.(2006).CODA:Convergence Diagnosis and Output Analysis for MCMC.R News,6,7-11.
[11] Othman,A.R.(1995).Examining task sampling variability in science performance assessments.Unpublished doctoral dissertation,University of California,Santa Barbara.
[12] Ross,I.,&Robert,G.(1996).R:A language for data analysis and graphics.Journal of Computational and Graphical Statistics,5(3),299-314.
[13] Shavelson,R.J.,&Webb,N.M.(1991).Generalizability theory:A primer(pp.1-15).Newbury Park,CA:Sage.
[14] Sturtz,S.,Ligges,U.,&Gelman,A.(2005).R2WinBUGS:A Package for Running WinBUGS from R.Journal of Statistical Software,12(3),1-16.
[15] Tong,Y.,&Brennan,R.L.(2006).Bootstrap techniques for estimating in generalizability theory(CASMA Research Report No 15).Iowa City,IA:Center for Advanced Studies in Measurement and Assessment,University of Iowa.Available from http://www.education.uiowa.edu/casma
[16] Tong,Y.,&Brennan,R.L.(2007).Bootstrap estimates of standard errors in generalizability theory.Educational and Psychological Measurement,67(5),804-817.
[17] Wiley,E.W.(2001).Bootstrap strategies for variance component estimation:Theoretical and empirical results.Unpublished doctoral dissertation,Stanford University,Stanford,CA.
[18] 焦璨,张敏强,黄庆均,张文怡,黎光明.(2008).非正态分布测量数据对克伦巴赫信度α系数的影响.应用心理学,14(3),276-281.
[19] 漆书青,戴海崎,丁树良.(2002).现代教育与心理测量学原理(pp.42-78).北京:高等教育出版社.
[20] 杨志明,张雷.(2003).测评的概化理论及其应用.北京:教育科学出版社.
[1] LI Guangming, CHEN Zihao, ZHANG Minqiang. Estimating the Best Sample Size of Teaching Level Evaluation for College Teachers under Budget Constraints in Generalizability Theory [J]. Psychological Development and Education, 2020, 36(3): 378-384.
[2] LI Guangming, ZHANG Minqiang. Weight Effect Analysis of Multivariate Generalizability Theory for Teaching Level Evaluation of College Teachers [J]. Psychological Development and Education, 2017, 33(1): 122-128.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of Psychological Development and Education
Supported by Beijing Magtech Co.Ltd