SATKT：基于Transformer的稀疏注意力知识追踪模型

诸葛斌; 肖梦凡; 汪盈; 董黎刚; 洪金珠; 蒋献

doi:10.11959/j.issn.1000-0801.DXKX250469

您当前的位置：

首页 >

文章列表页 >

SATKT：基于Transformer的稀疏注意力知识追踪模型

研究与开发 | 更新时间：2026-07-21

- SATKT：基于Transformer的稀疏注意力知识追踪模型
- SATKT: a sparse attention knowledge tracing model based on Transformer
- 电信科学 2026年42卷第6期页码：174-187
- 作者机构：
  
  1.浙江工商大学信息与电子工程学院（萨塞克斯人工智能学院），浙江杭州 310018
  2.浙江工商大学统计与数据科学学院，浙江杭州 310018
- 作者简介：
  
  诸葛斌（1976− ），男，博士，浙江工商大学信息与电子工程学院教授，主要研究方向为网络和通信技术、互联网技术和网络安全。
  肖梦凡（2001− ），女，浙江工商大学信息与电子工程学院硕士生，主要研究方向为智慧教育和个性化推荐。
  汪盈（2000− ），女，浙江工商大学信息与电子工程学院硕士生，主要研究方向为智慧教育和个性化推荐。
  董黎刚（1972− ），男，博士，浙江工商大学信息与电子工程学院教授，主要研究方向为智能网络、在线教育。
  洪金珠（1978− ），女，浙江工商大学统计与数据科学学院讲师，主要研究方向为在线教育、大数据。
  蒋献（1988− ），男，浙江工商大学信息与电子工程学院讲师、实验员，主要研究方向为在线教育。
- 基金信息：
  
  浙江省领雁研发项目(2025C02024);浙江省教育厅一般项目(Y202558980);桐乡通用人工智能研究院项目(TAGI2-A-2024-0005);浙江省高等教育“十四五”第二批研究生教学改革项目(JGCG2024225);2024年度浙江工商大学高等教育研究课题(Xgy2419);浙江省大学生科技成果推广项目·新苗计划(2025R408B075)
- DOI：10.11959/j.issn.1000-0801.DXKX250469
  中图分类号： TP301
- 收稿：2025-07-22，
  
  修回：2025-11-08，
  
  录用：2025-12-16，
  
  网络首发：2026-07-21，
  
  纸质出版：2026-06-20
- 稿件说明：
移动端阅览
诸葛斌,肖梦凡,汪盈等.SATKT：基于Transformer的稀疏注意力知识追踪模型[J].电信科学,2026,42(06):174-187.

Zhuge Bin,Xiao Mengfan,Wang Ying,et al.SATKT: a sparse attention knowledge tracing model based on Transformer[J].Telecommunications Science,2026,42(06):174-187.
诸葛斌,肖梦凡,汪盈等.SATKT：基于Transformer的稀疏注意力知识追踪模型[J].电信科学,2026,42(06):174-187. DOI： 10.11959/j.issn.1000-0801.DXKX250469.

Zhuge Bin,Xiao Mengfan,Wang Ying,et al.SATKT: a sparse attention knowledge tracing model based on Transformer[J].Telecommunications Science,2026,42(06):174-187. DOI： 10.11959/j.issn.1000-0801.DXKX250469.

摘要

随着在线教育的迅速发展，准确建模学习者的知识状态并为其提供个性化支持已成为实现高效教学的关键。知识追踪旨在基于学生的历史答题数据，动态建模其潜在知识状态。近年来，尽管Transformer模型以其卓越的序列数据处理能力被引入知识追踪领域，但其在处理长序列时仍面临计算复杂度高、预测准确率有限等挑战。对此，提出一种融合稀疏注意力机制的Transformer知识追踪模型——SATKT（sparse attention Transformer knowledge tracing）模型。该模型通过问题知识嵌入模块，从题目和知识点两个层面全面刻画学生的知识状态演化过程；引入稀疏注意力机制，有选择性地关注关键交互信息；结合混合损失函数，实现多目标联合优化，进一步提升模型的收敛稳定性与预测精度。在4个知识追踪数据集上的实验结果表明，SATKT在AUC指标上相较现有主流模型平均提升约1.33%，在ACC与RMSE指标上也表现更优，展现出更高的预测准确率与更强的泛化能力。本研究为智慧教育场景下的个性化学习分析提供了一种有效且可扩展的新思路。

Abstract

With the rapid development of online education

how to accurately model learners' knowledge status and provide personalized support has become a key part of realizing efficient teaching systems. Knowledge tracing aims to model students' potential knowledge status by using their historical answer data. In recent years

the Transformer model has been introduced into the field of knowledge tracing for its excellent sequence data processing capabilities

but its computational complexity is high when processing long sequences

and there is still room for further improvement in prediction accuracy. To address this

a Transformer-based knowledge tracing model integrating sparse attention mechanism

named SATKT (sparse attention Transformer knowledge tracing)

was proposed. In this model

a question‑knowledge embedding module was employed to comprehensively capture the evolution of students’ knowledge states from both the question level and the knowledge concept level. A sparse attention mechanism was introduced to selectively focus on key interaction information. Furthermore

a hybrid loss function was incorporated to achieve multi‑objective joint optimization

thereby further improving the convergence stability and prediction accuracy of the model. Experimental results on four knowledge tracing datasets show that SATKT achieves an average improvement of approximately 1.33% in AUC compared with existing mainstream models

and also performes better in terms of ACC and RMSE

demonstrating higher prediction accuracy and stronger generalization ability. This study provides an effective and scalable new approach for personalized learning analysis in smart education scenarios.

关键词

Keywords

references

Pirolli P , Kairam S . A knowledge-tracing model of learning from a social tagging system [J ] . User Modeling and User-Adapted Interaction , 2013 , 23 ( 2 ): 139 - 168 .

Shen S H , Liu Q , Huang Z Y , et al . A survey of knowledge tracing: models, variants, and applications [J ] . IEEE Transactions on Learning Technologies , 2024 , 17 : 1858 - 1879 .

Park S , Lee D , Park H . Enhancing knowledge tracing with concept map and response disentanglement [J ] . Knowledge-Based Systems , 2024 , 302 : 112346 .

刘淇 , 陈恩红 , 朱天宇 , 等 . 面向在线智慧学习的教育数据挖掘技术研究 [J ] . 模式识别与人工智能 , 2018 , 31 ( 1 ): 77 - 90 .

Liu Q , Chen E H , Zhu T Y , et al . Research on educational data mining for online intelligent learning [J ] . Pattern Recognition and Artificial Intelligence , 2018 , 31 ( 1 ): 77 - 90 .

Ghosh A , Heffernan N , Lan A S . Context-aware attentive knowledge tracing [C ] // Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . New York : ACM Press , 2020 : 2330 - 2339 .

Yeung C K , Yeung D Y . Addressing two problems in deep knowledge tracing via prediction-consistent regularization [C ] // Proceedings of the Fifth Annual ACM Conference on Learning at Scale . New York : ACM Press , 2018 : 1 - 10 .

Corbett A T , Anderson J R . Knowledge tracing: Modeling the acquisition of procedural knowledge [J ] . User Modelling and User-Adapted Interaction , 1995 , 4 ( 4 ): 253 - 278 .

Embretson S E , Reise S P . Item response theory [M ] . Hove : Psychology Press , 2000

Piech C , Bassen J , Huang J , et al . Deep knowledge tracing [C ] // Proceedings of the 29th International Conference on Neural Information Processing Systems . Cambridge : MIT Press , 2015 : 505 - 513 .

Abdelrahman G , Wang Q , Nunes B . Knowledge tracing: a survey [J ] . ACM Computing Surveys , 2023 , 55 ( 11 ): 1 - 37 .

Zhang J N , Shi X J , King I , et al . Dynamic key-value memory networks for knowledge tracing [C ] // Proceedings of the 26th International Conference on World Wide Web . International World Wide Web Conferences Steering Committee , 2017 : 765 - 774 .

Abdelrahman G , Wang Q . Knowledge tracing with sequential key-value memory networks [C ] // Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York : ACM Press , 2019 : 175 - 184 .

Chen J H , Liu Z T , Huang S Y , et al . Improving interpretability of deep sequential knowledge tracing models with question-centric cognitive representations [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2023 , 37 ( 12 ): 14196 - 14204 .

Nagatani K , Zhang Q , Sato M , et al . Augmenting knowledge tracing by considering forgetting behavior [C ] // Proceedings of the World Wide Web Conference . New York : ACM Press , 2019 : 3101 - 3107 .

Guan Q , Duan X , Bian K , et al . KVFKT: a new horizon in knowledge tracing with attention-based embedding and forgetting curve integration [C ] // Proceedings of the 31st International Conference on Computational Linguistics . Abu Dhabi ： Association for Computational Linguistics , 2025 : 4399 - 4409 .

Li S T , Shen S H , Su Y , et al . STHKT: spatiotemporal knowledge tracing with topological hawkes process [J ] . Expert Systems with Applications , 2025 , 259 : 125248 .

Vaswani A , Shazeer N , Parmar N , et al . Attention is all you need [C ] // Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017) . New York : ACM Press , 2017 : 6000 - 6010 .

Pandey S , Karypis G . A self-attentive model for knowledge tracing [C ] // Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019) . Montreal, Canada : International Educational Data Mining Society , 2019 : 384 - 389 .

Choi Y , Lee Y , Cho J , et al . Towards an appropriate query, key, and value computation for knowledge tracing [C ] // Proceedings of the Seventh ACM Conference on Learning @ Scale . New York : ACM Press , 2020 : 341 - 344 .

Shin D , Shim Y , Yu H , et al . SAINT+: integrating temporal features for EdNet correctness prediction [C ] // Proceedings of the LAK21: 11th International Learning Analytics and Knowledge Conference . New York : ACM Press , 2021 : 490 - 496 .

Yin Y , Dai L , Huang Z Y , et al . Tracing knowledge instead of patterns: stable knowledge tracing with diagnostic transformer [C ] // Proceedings of the ACM Web Conference 2023 . New York : ACM Press , 2023 : 855 - 864 .

Zhan H J , Kim J J , Liu G M . Contrastive learning with bidirectional transformers for knowledge tracing [C ] // Proceedings of the ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . Piscataway : IEEE Press , 2024 : 5040 - 5044 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于知识追踪机的多特征融合习题推荐模型

融合Transformer和Inception的网络入侵检测研究

一种复数域轻量化知识蒸馏驱动的调制识别模型

基于改进Transformer的电信重投报告自动生成方法研究

智能体互联网服务语义标识和路由技术研究