心理发展与教育 ›› 2020, Vol. 36 ›› Issue (3): 378-384.doi: 10.16187/j.cnki.issn1001-4918.2020.03.15

• 理论探讨与进展 • 上一篇    

高校教师教学水平评价概化理论预算限制下最佳样本量估计

黎光明, 陈子豪, 张敏强   

  1. 华南师范大学心理学院、心理应用研究中心, 广州 510631
  • 发布日期:2020-06-02
  • 通讯作者: 张敏强,E-mail:2640726401@qq.com E-mail:2640726401@qq.com
  • 基金资助:
    国家自然科学基金面上项目（31470050）；教育部人文社会科学研究规划基金项目 （18YJA190006、19YJA880046）；广东省哲学社会科学“十三五”规划一般项目（GD17CXL01）；广州市哲学社会科学“十三五”规划一般项目（2017GZYB111）。

Estimating the Best Sample Size of Teaching Level Evaluation for College Teachers under Budget Constraints in Generalizability Theory

LI Guangming, CHEN Zihao, ZHANG Minqiang   

  1. School of Psychology, Center for Studies of Psychological Application, South China Normal University, Guangzhou 510631
  • Published:2020-06-02

RichHTML

PDF (PC)

摘要： 使用“高校教师教学水平评价问卷”，要求566名学生对19名教师进行评价，对收集到的数据作不同的概化设计，包括t×i、（s：t）×i、（s：t）×（i：v）和（s：t）×（i：v）×o四种设计。基于概化理论，结合预算限制，统一LaGrange乘法公式，自行推导不同设计的最佳样本量公式，联合估计的方差分量，计算出不同设计的最佳样本量。结果表明：（1）LaGrange乘法统一公式表现出较强的通用性，能够适用于预算限制下各种概化设计；（2）评价场合是影响高校教师教学水平评价一个相当重要的因素；（3）（s：t）×（i：v）×o是高校教师教学水平评价概化理论预算限制下最优概化设计；（4）高校教师教学水平评价概化理论预算限制下，每位教师最佳评价学生人数为20人，每个维度最佳评价题目数为3题。

关键词: 概化理论, 预算限制, 最佳样本量估计, 高校教师, 教学水平评价

Abstract: Using the evaluation questionnaire of Teachers' Teaching Level Evaluation Scale for College Students(TTLES-CS), 566 students were asked to evaluate 19 teachers, and the data were generalized to four kinds of designs such as t×i,(s:t)×i,(s:t)×(i:v) and(s:t)×(i:v)×o. Based on generalizability theory, we combined the budget constraints and derived a unification algorithm adapting with the optimal sample size estimation for different designs of generalizability theory, i.e. LaGrange multiplication. On this basis, we applied LaGrange multiplication to estimate the optimal observation number in different generalizability designs. The results show that: (1) LaGrange multiplication that we derived has a stronger robustness, which can apply to different study designs effectively under budget constraints. (2) Occasion is an important factor that has a great effect on teachers' teaching level evaluation. So, we need attach importance to it. (3) According to the actual circumstances under the budget constraints, the design of(s:t)×(i:v)×o is the optimal generalized design. (4) Under budget constraints of teachers' teaching level evaluation for college teachers in generalizability theory, the best sample size of students is 20 for each teacher and the best sample size of items is 3 for each dimension.

Key words: generalizability theory, budget constraints, estimating the best sample size, college teachers, teaching level evaluation

中图分类号: 

  • B844
Brennan, R. L. (2001a). Manual for urGENOVA. Iowa City, IA:Iowa Testing Programs, University of Iowa.
Brennan, R. L. (2001b). Generalizability theory. New York:Springer-Verlag.
Casabianca, J. M., Lockwood, J. R., & McCaffrey, D. F. (2015). Trends in classroom observation scores. Educational and Psychological Measurement, 75(2), 311-337.
Chang, L., & Hocevar, D. (2000). Models of generalizability theory in analyzing existing faculty evaluation data. Applied Measurement in Education, 13(3), 255-275.
Gitomer, D. H., Bell, C. A., Qi, Y., McCaffrey, D. F., Hamre, B. K., & Pianta, R. C. (2014). The instructional challenge in improving teaching quality:Lessons from a classroom observation protocol. Teachers College Record, 116(6), 1-32.
Goldstein, Z., & Marcoulides, G. A. (1991). Maximizing the coefficient of generalizability in decision studies. Educational and Psychological Measurement, 51(1), 79-88.
Hill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough:Teaching observation systems and a case for the generalizability study. Educational Researcher, 41(2), 56-64.
Iqbal, I., Lee, J. D., Pearson, M. L., & Albon, S. P. (2016). Student and faculty perceptions of student evaluations of teaching in a Canadian pharmacy school. Currents in Pharmacy Teaching & Learning, 8(2), 191-199.
Lakes, K. D. (2013). Restricted sample variance reduces generalizability. Psychological Assessment, 25(2), 643-650.
Marcoulides. G. A. (1993). Maximizing power in generalizability studies under budget constraints. Journal of Educational Statistics, 18(2), 197-206.
Marcoulides, G. A. (1997). Optimizing measurement designs with budget constraints:The variable cost case. Educational and Psychological Measurement, 57(5), 808-812.
Meyer, P. J., Liu, X., & Mashburn, A. J. (2014). A practical solution to optimizing the reliability of teaching observation measures under budget constraints. Educational and Psychological Measurement, 74(2), 280-291.
Wolbing, T., & Riordan, P. (2016). How beauty works. Theoretical mechanisms and two empirical applications on students' evaluation of teaching. Social Science Research, 57(5), 253-272.
乐美玲, 辛涛. (2015). 高等教育评估的几个关键问题. 清华大学教育研究, 36(1), 48-52.
王政民, 吴阔华. (1999). 高等数学(第三版). 南昌:江西高校出版社.
杨志明, 张雷. (2003). 测评的概化理论及其应用.北京:教育科学出版社.
[1] 黎光明, 张敏强. 高校教师教学水平评价多元概化理论权重效应分析[J]. 心理发展与教育, 2017, 33(1): 122-128.
[2] 黎光明, 张敏强. 概化理论方差分量估计的跨分布分析[J]. 心理发展与教育, 2012, 28(6): 665-672.
[3] 韩明, 董学安, 范丹, 何先友. 高校教师心理契约问卷的编制[J]. 心理发展与教育, 2010, 26(3): 315-321.
[4] 韩明, 董学安, 范丹, 何先友. 高校教师心理契约问卷的编制[J]. 心理发展与教育, 2010, 26(3): 315-321.
[5] 韩明, 董学安, 范丹, 何先友. 高校教师心理契约问卷的编制[J]. 心理发展与教育, 2010, 26(3): 315-321.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2018 《心理发展与教育》编辑部
地址：北京新街口外大街19号北京师范大学院内　 邮政编码：100875
联系电话：010-58807700 E-mail:pdae@bnu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发