Psychological Development and Education ›› 2026, Vol. 42 ›› Issue (1): 18-27.doi: 10.16187/j.cnki.issn1001-4918.2026.01.03

Previous Articles     Next Articles

The Processing of Text-graphic Information Knowledge Integration of College Students: Evidence from ERP

ZHAO Xiaomei, ZHAO Xinrui, WANG Keke   

  1. College of Education, Hebei Normal University, Shijiazhuang 050024
  • Published:2026-01-19

RichHTML

PDF (PC)

Abstract: This study utilized an adapted graphic presentation paradigm to investigate the performance of two separate yet related knowledge integrations involving graphic and text information, and the theoretical basis was further clarified. The investigation involved conducting behavioral experiments and ERP technology. The findings reveal the following: (1) The form of information presentation influences knowledge integration. Whether it pertains to accuracy rate or response time, the performance of integrating graphic-text and text-graphic is inferior to that of text-text and graphic- graphic integration. Moreover, the integration performance of text-based tests surpasses that of graphic-based tests; (2) The knowledge integration of graphic and text information occurs within the time frame of 300 to 500ms. Notably, there exist significant differences in the N400 amplitudes induced by various presentation forms. The N400 induced by text-text integration is the largest, followed by graphic-text integration and text-graphic integration. The research results presented above demonstrate the impact of information presentation on knowledge integration. It is observed that the inclusion of graphics does not promote the integration of graphic and text information. This study offers valuable insights into the process of knowledge integration between graphic and text information, and provides experimental evidence that supports the cognitive theory of multimedia learning's perspective on the integration of graphic and text information.

Key words: knowledge integration, text-graphic information, multimedia learning, college students

Bauer, P. J., & Jackson, F. L.(2015). Semantic elaboration: ERPs reveal rapid transition from novel to known. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(1), 271-282.
Bauer, P. J., & Larkina, M. (2017). Realizing relevance: The influence of domain-specific information on generation of new knowledge through integration in 4 to 8 year old children. Child Development, 88(1), 247-262.
Bauer, P. J., Cronin-Golomb, L. M., Porter, B. M., Jaganjac, A., & Miller, H. E.(2021). Integration of memory content in adults and children: Developmental differences in task conditions and functional consequences. Journal of Experimental Psychology: General, 150(7), 1259-1278.
Bauer, P. J., Esposito, A. G., & Daly, J. J.(2020). Self-derivation through memory integration: A model for accumulation of semantic knowledge. Learning and Instruction, 66(1), 101271.https://doi.org/10.1016/j.learninstruc.2019.101271
Bauer, P. J., King, J. E., Larkina, M., Varga, N. L., & White, E. A.(2012). Characters and clues: Factors affecting children's extension of knowledge through integration of separate episodes. Journal of Experimental Child Psychology, 111(4), 681-694.
Brouwer, H., Fitz, H., & Hoeks, J.(2012). Getting real about semantic illusions: Rethinking the functional role of the p600 in language comprehension. Brain Research, 1446(1), 127-143.
Coltheart, M.(1980). Iconic memory and visible persistence. Perception & Psychophysics, 27(3), 183-228.
Dugan, J. A., & Bauer, P. J.(2022). Putting the pieces together: Cognitive correlates of self-derivation of new knowledge in elementary school classrooms. Journal of Experimental Child Psychology, 221, 105441.https://doi.org/10.1016/j.jecp.2022.105441
Dugan, J. A., Lee, K., Hanft, M. H., & Bauer, P. J.(2023). Selection of information necessary for successful self-derivation. Cognitive Development, 68(2023), 101364.https://doi.org/10.1016/j.cogdev.2023.101364
Esposito, A. G., & Bauer, P. J.(2017). Going beyond the lesson: Self-generating new factual knowledge in the classroom. Journal of Experimental Child Psychology, 153, 110-125.
Esposito, A. G., & Bauer, P. J.(2019). From bench to classroom: Collaborating within a dual-language education model. Journal of Cognition and Development, 20(2), 165-181.
Esposito, A. G., Lee, K., Dugan, J. A., Lauer, J. E., & Bauer, P. J.(2021). Relating a picture and 1000 words: Self-derivation through integration within and across presentation formats. Cognitive Development, 60, 101099.https://doi.org/10.1016/j.cogdev.2021.101099
Hagoort, P., & Brown, C. M.(1999). Gender Electrified: ERP Evidence on the Syntactic Nature of Gender Processing. Journal of Psycholinguistic Research, 28(6), 715-728.
Johnen, A.-K., & Harrison, N. R.(2020). Level of uncertainty about the affective nature of a pictorial stimulus influences anticipatory neural processes: An event-related potential (ERP) study. Neuropsychologia, 146(2020), 107525.https://doi.org/10.1016/j.neuropsychologia.2020.107525
Kim, A. E., Oines, L., & Miyake, A.(2018). Individual differences in verbal working memory underlie a tradeoff between semantic and structural processing difficulty during language comprehension: An ERP investigation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 44(3), 406 -420.
Kutas, M., & Federmeier, K. D.(2011). Thirty Years and Counting: Finding Meaning in the N400 Component of the Event-Related Brain Potential (ERP). Annual Review of Psychology, 62(1), 621-647.
Li, S., Chen, S., Zhang, H., Zhao, Q., Zhou, Z., Huang, F., … Hong, J. (2020). Dynamic cognitive processes of text-picture integration revealed by event-related potentials. Brain Research, 1726, 146513.https://doi.org/10.1016/j.brainres.2019.146513
Magnus, L., Schütte, K., & Schwanewedel, J. (2020). Challenges Solving Science Tasks with Text-Picture Combinations Persist beyond Secondary School. Journal of Research on Educational Effectiveness, 13(4), 759-783.
Mayer, R. E.(1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32(1), 1-19.
Mayer, R. E.(2001). Multimedia learning. Cambridge: Cambridge University Press.
Mayer, R. E. (2009). Multimedia learning (2nd ed.). Cambridge: Cambridge University Press.
Mayer, R. E., & Gallini, J. K.(1990). When Is an Illustration Worth Ten Thousand Words? Journal of Educational Psychology, 82(4), 715-726.
Menendez, D.(2023). Cues to generality: Integrating linguistic and visual information when generalizing biological information. Journal of Educational Psychology, 115(8), 1110-1124.
Miller-Goldwater, H. E., Cronin-Golomb, L. M., Porter, B. M., & Bauer, P. J.(2021). Developmental differences in reactivation underlying self-derivation of new knowledge through memory integration. Cognitive Psychology, 129, 101413.https://doi.org/10.1016/j.cogpsych.2021.101413
Scharinger, C., Schüler, A., & Gerjets, P.(2020). Using eye-tracking and EEG to study the mental processing demands during learning of text-picture combinations. International Journal of Psychophysiology, 158, 201-214.
Schnotz, W., & Bannert, M.(2003). Construction and interference in learning from multiple representation. Learning & Instruction, 13(2), 141-156.
Schnotz, W., Ludewig, U., Ullrich, M., Horz, H., McElvany, N., & Baumert, J.(2014). Strategy shifts during learning from texts and pictures. Journal of Educational Psychology, 106(4), 974-989.
Schüler, A.(2017). Investigating gaze behavior during processing of inconsistent text-picture information: Evidence for text-picture integration. Learning and Instruction, 49, 218-231.
Schüler, A., Arndt, J., & Scheiter, K.(2015). Processing multimedia material: Does integration of text and pictures result in a single or two interconnected mental representations? Learning and Instruction, 35, 62-72.
Stuellein, N., Radach, R. R., Jacobs, A. M., & Hofmann, M. J.(2016). No one way ticket from orthography to semantics in recognition memory: N400 and P200 effects of associations. Brain Research, 1639, 88-98.
Varga, N. L., Esposito, A. G., & Bauer, P. J.(2019). Cognitive correlates of memory integration across development: Explaining variability in an educationally relevant phenomenon. Journal of Experimental Psychology: General, 148(4), 739-762.
Varga, N. L., & Bauer P.J. (2013). Effects of delays on 6-year-old children’s self-generation and retention of knowledge through integration. Mind, Brain, and Education, 115(2), 326-341.
Yum, Y. N., Holcomb, P. J., & Grainger, J. (2011). Words and pictures: An electrophysiological investigation of domain specific processing in native Chinese and English speakers. Neuropsychologia,49(7),1910-1922.
Zhao, F., Gaschler, R., Wagner, I., & Schnotz, W.(2023). Lower grade students tend to give up early in multimedia learning. European Journal of Psychology of Education, 38(2), 545-565.
胡成霞 .(2023). 从图文关系嬗变看教科书插图的价值定位及实现路径·课程·教材·教法, 43(11), 35-42.
李松清, 赵庆柏, 周治金, 张依. (2015). 多媒体学习中图文加工的认知神经机制. 心理科学进展, 23(8), 1361-1370.
王彭, 李培, 张珊珊, 杨亦鸣. (2022). 汉语空缺动词语义加工的ERPs证据——兼谈汉语动词空缺句.外国语(上海外国语大学学报), 45(1), 14-24.
王穗苹,黄健. (2019). 语言理解中的语义加工:不同模态神经影像的研究.生理学报, 71(1), 127-139.
杨邵峰,张志超,冀婷. (2022). 抑制还是干扰？来自提取练习范式的证据.心理与行为研究, 20(6),753-759.
张积家,王斌. (2024). 义符认知功能的心理语言学探索——三十年研究工作之回顾. 西北师大学报(社会科学版),61(01),104-117.
张钦, 丁锦红, 郭春彦, 王争艳. (2003). 名词与动词加工的ERP差异.心理学报, 35(6), 753-760.
赵丽波,张悦,于祎雯,邓丽芳. (2017). 提取引起的遗忘的认知机制.心理科学进展, 25(10),1738-1748.
赵笑梅, 靳伟琼, 王莹平, 褚铮. (2021). 知识整合的时间进程:ERP研究.心理科学, 44(5), 1089-1096.
赵笑梅,程释,刘子涵,刘红. (2025). 不同语言条件和执行功能对大学生记忆整合的影响.心理发展与教育, 41(2),153-162.
周文洁, 邓丽群, 丁锦红. (2021). 物体颜色对情景记忆的影响.心理学报, 53(3), 229-243.
[1] DING Yue, LIU Lijia, KOU Yu. Interpersonal Relationship Satisfaction in the Digital Age: The Role of Online Positive Feedback and Perceived Social Support [J]. Psychological Development and Education, 2026, 42(1): 39-49.
[2] WANG Zhen, SHI Baoying, ZHAO Haoli, GUO Yunfei, WANG Enguo. Effects of Emotional Expression and Role of Affective Pedagogical Agent on Video Learning: The Moderating Role of Learners’ Experience [J]. Psychological Development and Education, 2026, 42(1): 60-69.
[3] QIAO Peihua, CHENG Meixia, WANG Fuxing. Pedagogical Agent with Testing Interaction Could Facilitate Multimedia Learning [J]. Psychological Development and Education, 2025, 41(6): 850-858.
[4] WEI Jun, YI Cong, ZHOU Xiting. College Students’ Academic Contingent Self-worth and Academic Involution Behaviors: Analyses Based on Latent Growth Modeling [J]. Psychological Development and Education, 2025, 41(5): 692-700.
[5] JIAO Jiao, MU Dan, SHI Lu. Intimate Relationship Satisfaction on College Students’ Subjective Well-being: A Diary Method Study [J]. Psychological Development and Education, 2025, 41(5): 720-729.
[6] HU Xinyi, CHEN Yinghe. The Relationship among College Students’ Goal Reengagement Capacity, Ego-depletion and Well-being: Moderation Effect of Meaning in Life [J]. Psychological Development and Education, 2025, 41(3): 348-356.
[7] XIE Li, JIN Hui, WANG Zhiying, HU Yufei, YANG Xilan, XI Juan. The Influence of Endogenous Power for Poverty Elimination on Impoverished College Students’ Learning Engagement: The Mediating Effect of Self-control and the Moderating Effect of Social Support [J]. Psychological Development and Education, 2025, 41(3): 386-397.
[8] ZHAO Xiaomei, CHENG Shi, LIU Zihan, LIU Hong. The Effects of Different Language Conditions and Executive Functions on Memory Integration in College Students [J]. Psychological Development and Education, 2025, 41(2): 153-162.
[9] LIAO Chenxi, WANG Dahua. The Influence of Attachment on Self-esteem and Interpersonal Adjustment in College Students [J]. Psychological Development and Education, 2025, 41(2): 196-205.
[10] GUO Ying, CHEN Xiaodan, LIU Wenzhi, LIU Yutong, ZHU Bi. Development of Semantic False Memories in Adolescents: Study and Test Factors [J]. Psychological Development and Education, 2025, 41(1): 12-21.
[11] SHI Zifu, ZHOU Zhihao, XU Lei, CHEN Huohong, GUAN Jinliang, LIU Chengzhen. The Relationship between Negative Parenting Style and College Students’ Malevolent Creativity Behaviors: A Moderated Mediation Model [J]. Psychological Development and Education, 2024, 40(6): 808-815.
[12] JIA Lina, YIN Xiaojuan. The Relationship between Inhibitory Differences and the Continued Influence Effect of Misinformation: The Role of Information Relevance [J]. Psychological Development and Education, 2024, 40(5): 609-615.
[13] LI Jiaojiao, XU Bibo, YUAN Hailong, YIN Xiyang. The Relationship between Social Exclusion and Malevolent Creativity of College Students: The Chain Mediating Effect of Coping Styles and Aggressiveness [J]. Psychological Development and Education, 2024, 40(5): 667-674.
[14] SU Binyuan, GUO Qianlan, XIE Binru, ZHANG Wei. The Measurement Index Identifying College Students at High Risk of Suicide: From Personal-centered Perspective [J]. Psychological Development and Education, 2024, 40(4): 572-588.
[15] HU Zhiqin, XIONG Meng. The Relationship between Relative Deprivation and Cyberbullying in College Students:The Mediating Effect of Moral Disengagement and Moderating Effect of Moral Identity [J]. Psychological Development and Education, 2024, 40(3): 346-356.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of Psychological Development and Education
Supported by Beijing Magtech Co.Ltd