Psychological Development and Education ›› 2017, Vol. 33 ›› Issue (1): 122-128.doi: 10.16187/j.cnki.issn1001-4918.2017.01.14

Previous Articles    

Weight Effect Analysis of Multivariate Generalizability Theory for Teaching Level Evaluation of College Teachers

LI Guangming, ZHANG Minqiang   

  1. School of Psychology, Center for Studies of Psychological Application in South China Normal University, Guangzhou 510631
  • Online:2017-01-15 Published:2017-01-15

PDF (PC)

Abstract: Exemplified teaching level evaluation for college teachers, the paper explored if the weights had an effect on composite reliability of multivariate generalizability theory (MGT). By the designed questionnaire of teaching level evaluation for college teachers, 534 students were asked to evaluate 16 english teachers. We used MGT to analyze the multivariate nested collected data. The results showed that:(1) as for teaching level evaluation for college teachers, three weights, i.e. estimation weights, priori weights and effective weights, had a large difference for composite reliability of MGT; (2)exemplified the teaching level evaluation and based on a full analysis about three weights how to affect the composite reliability, this study proposed a pattern of three weights effect analysis for composite reliability of MGT, which could provide a scientific reference for the proper use of weight in MGT.

Key words: college teachers, teaching level evaluation, multivariate generalizability theory (MGT), weight effect, composite reliability

CLC Number: 

  • B844

Brennan, R. L. (2001a). Generalizability theory. New York:Springer-Verlag.

Brennan, R. L. (2001b). Manual of mGENOVA. Iowa City, IA:Iowa Testing Programs, University of Iowa.

Casabianca, J. M., Lockwood, J. R., & McCaffrey, D. F. (2015). Trends in classroom observation scores. Educational and Psychological Measurement. 75(2), 311-337.

Chang, L., & Hocevar, D. (2000). Models of generalizability theory in analyzing existing faculty evaluation data. Applied Measurement in Education,13(3), 255-275.

Crooks, T. J., & Kane, M. T. (1981). The generalizability of student rating of instructors:Item specificity and section effects. Research in Higher Education, 15(4), 305-313.

Dolmans, D. J., Wolfhagen, H. P., Scherpbier, A. J., & van der Vleuten, C. P. (2003). Development of an instrument to evaluate the effectiveness of teachers in guiding small groups. Higher Education, 46(4), 431-446.

Kolen, M. J., & Brennan, R. L. (2004). Test equating, linking, and scaling:Methods and practices (2nd ed.). New York:Springer-Verlag.

Marcoulides, G. A. (1994). Selecting weighting schemes in multivariate generalizability studies. Educational and Psychological Measurement, 54(1), 3-7.

Meyer, J. P., Liu, X., & Mashburn, A. J. (2014). A practical solution to optimizing the reliability of teaching observation measures under budget constraints. Educational and Psychological Measurement, 74(2), 280-291.

Shin, Y., & Raudenbush, S. W. (2012). Confidence bounds and power for the reliability of observational measures on the quality of a social setting. Psychometrika, 77(3), 543-560.

漆书青, 戴海崎, 丁树良. (2002). 现代教育与心理测量学原理. 北京:高等教育出版社.

王烨晖, 边玉芳, 辛涛. (2011). 垂直等值的应用及最新发展述评. 心理学探新, 31(5), 472-476.

杨志明, 张雷. (2003). 测评的概化理论及其应用. 北京:教育科学出版社.
[1] LI Guangming, CHEN Zihao, ZHANG Minqiang. Estimating the Best Sample Size of Teaching Level Evaluation for College Teachers under Budget Constraints in Generalizability Theory [J]. Psychological Development and Education, 2020, 36(3): 378-384.
[2] HAN Ming, DONG Xue-an, FAN Dan, HE Xian-you. Developing Questionnaire of Psychological Contract for College Teachers [J]. Psychological Development and Education, 2010, 26(3): 315-321.
[3] HAN Ming, DONG Xue-an, FAN Dan, HE Xian-you. Developing Questionnaire of Psychological Contract for College Teachers [J]. Psychological Development and Education, 2010, 26(3): 315-321.
[4] HAN Ming, DONG Xue-an, FAN Dan, HE Xian-you. Developing Questionnaire of Psychological Contract for College Teachers [J]. Psychological Development and Education, 2010, 26(3): 315-321.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of Psychological Development and Education
Supported by Beijing Magtech Co.Ltd