SATKT: a sparse attention knowledge tracing model based on Transformer

Zhuge Bin; Xiao Mengfan; Wang Ying; Dong Ligang; Hong Jinzhu; Jiang Xian

doi:10.11959/j.issn.1000-0801.DXKX250469

您当前的位置：

首页 >

文章列表页 >

SATKT: a sparse attention knowledge tracing model based on Transformer

Research and Development | 更新时间：2026-07-21

- SATKT: a sparse attention knowledge tracing model based on Transformer
- Telecommunications Science Vol. 42, Issue 6, Pages: 174-187(2026)
- 作者机构：
  
  1.浙江工商大学信息与电子工程学院（萨塞克斯人工智能学院），浙江杭州 310018
  2.浙江工商大学统计与数据科学学院，浙江杭州 310018
- 作者简介：
- 基金信息：
  
  Zhejiang Lingyan Research and Development Project(2025C02024);General Research Project of Zhejiang Provincial Department of Education(Y202558980);Tongxiang General Artificial Intelligence Research Institute(TAGI2-A-2024-0005);Zhejiang Province Higher Education “14th Five-Year Plan” Second Batch of Postgraduate Teaching Reform Projects(JGCG2024225);2024 Zhejiang Gongshang University Higher Education Research Project(Xgy2419);Zhejiang Provincial College Students’ Scientific and Technological Achievements Promotion Project–Xinmiao Talents Program(2025R408B075)
- DOI：10.11959/j.issn.1000-0801.DXKX250469
  CLC： TP301
- Received：22 July 2025，
  
  Revised：2025-11-08，
  
  Accepted：16 December 2025，
  
  Online First：21 July 2026，
  
  Published：20 June 2026
- 稿件说明：
移动端阅览
诸葛斌,肖梦凡,汪盈等.SATKT：基于Transformer的稀疏注意力知识追踪模型[J].电信科学,2026,42(06):174-187.

Zhuge Bin,Xiao Mengfan,Wang Ying,et al.SATKT: a sparse attention knowledge tracing model based on Transformer[J].Telecommunications Science,2026,42(06):174-187.
诸葛斌,肖梦凡,汪盈等.SATKT：基于Transformer的稀疏注意力知识追踪模型[J].电信科学,2026,42(06):174-187. DOI： 10.11959/j.issn.1000-0801.DXKX250469.

Zhuge Bin,Xiao Mengfan,Wang Ying,et al.SATKT: a sparse attention knowledge tracing model based on Transformer[J].Telecommunications Science,2026,42(06):174-187. DOI： 10.11959/j.issn.1000-0801.DXKX250469.

摘要

随着在线教育的迅速发展，准确建模学习者的知识状态并为其提供个性化支持已成为实现高效教学的关键。知识追踪旨在基于学生的历史答题数据，动态建模其潜在知识状态。近年来，尽管Transformer模型以其卓越的序列数据处理能力被引入知识追踪领域，但其在处理长序列时仍面临计算复杂度高、预测准确率有限等挑战。对此，提出一种融合稀疏注意力机制的Transformer知识追踪模型——SATKT（sparse attention Transformer knowledge tracing）模型。该模型通过问题知识嵌入模块，从题目和知识点两个层面全面刻画学生的知识状态演化过程；引入稀疏注意力机制，有选择性地关注关键交互信息；结合混合损失函数，实现多目标联合优化，进一步提升模型的收敛稳定性与预测精度。在4个知识追踪数据集上的实验结果表明，SATKT在AUC指标上相较现有主流模型平均提升约1.33%，在ACC与RMSE指标上也表现更优，展现出更高的预测准确率与更强的泛化能力。本研究为智慧教育场景下的个性化学习分析提供了一种有效且可扩展的新思路。

Abstract

With the rapid development of online education

how to accurately model learners' knowledge status and provide personalized support has become a key part of realizing efficient teaching systems. Knowledge tracing aims to model students' potential knowledge status by using their historical answer data. In recent years

the Transformer model has been introduced into the field of knowledge tracing for its excellent sequence data processing capabilities

but its computational complexity is high when processing long sequences

and there is still room for further improvement in prediction accuracy. To address this

a Transformer-based knowledge tracing model integrating sparse attention mechanism

named SATKT (sparse attention Transformer knowledge tracing)

was proposed. In this model

a question‑knowledge embedding module was employed to comprehensively capture the evolution of students’ knowledge states from both the question level and the knowledge concept level. A sparse attention mechanism was introduced to selectively focus on key interaction information. Furthermore

a hybrid loss function was incorporated to achieve multi‑objective joint optimization

thereby further improving the convergence stability and prediction accuracy of the model. Experimental results on four knowledge tracing datasets show that SATKT achieves an average improvement of approximately 1.33% in AUC compared with existing mainstream models

and also performes better in terms of ACC and RMSE

demonstrating higher prediction accuracy and stronger generalization ability. This study provides an effective and scalable new approach for personalized learning analysis in smart education scenarios.

关键词

Keywords

references

Pirolli P , Kairam S . A knowledge-tracing model of learning from a social tagging system [J ] . User Modeling and User-Adapted Interaction , 2013 , 23 ( 2 ): 139 - 168 .

Shen S H , Liu Q , Huang Z Y , et al . A survey of knowledge tracing: models, variants, and applications [J ] . IEEE Transactions on Learning Technologies , 2024 , 17 : 1858 - 1879 .

Park S , Lee D , Park H . Enhancing knowledge tracing with concept map and response disentanglement [J ] . Knowledge-Based Systems , 2024 , 302 : 112346 .

刘淇 , 陈恩红 , 朱天宇 , 等 . 面向在线智慧学习的教育数据挖掘技术研究 [J ] . 模式识别与人工智能 , 2018 , 31 ( 1 ): 77 - 90 .

Liu Q , Chen E H , Zhu T Y , et al . Research on educational data mining for online intelligent learning [J ] . Pattern Recognition and Artificial Intelligence , 2018 , 31 ( 1 ): 77 - 90 .

Ghosh A , Heffernan N , Lan A S . Context-aware attentive knowledge tracing [C ] // Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . New York : ACM Press , 2020 : 2330 - 2339 .

Yeung C K , Yeung D Y . Addressing two problems in deep knowledge tracing via prediction-consistent regularization [C ] // Proceedings of the Fifth Annual ACM Conference on Learning at Scale . New York : ACM Press , 2018 : 1 - 10 .

Corbett A T , Anderson J R . Knowledge tracing: Modeling the acquisition of procedural knowledge [J ] . User Modelling and User-Adapted Interaction , 1995 , 4 ( 4 ): 253 - 278 .

Embretson S E , Reise S P . Item response theory [M ] . Hove : Psychology Press , 2000

Piech C , Bassen J , Huang J , et al . Deep knowledge tracing [C ] // Proceedings of the 29th International Conference on Neural Information Processing Systems . Cambridge : MIT Press , 2015 : 505 - 513 .

Abdelrahman G , Wang Q , Nunes B . Knowledge tracing: a survey [J ] . ACM Computing Surveys , 2023 , 55 ( 11 ): 1 - 37 .

Zhang J N , Shi X J , King I , et al . Dynamic key-value memory networks for knowledge tracing [C ] // Proceedings of the 26th International Conference on World Wide Web . International World Wide Web Conferences Steering Committee , 2017 : 765 - 774 .

Abdelrahman G , Wang Q . Knowledge tracing with sequential key-value memory networks [C ] // Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York : ACM Press , 2019 : 175 - 184 .

Chen J H , Liu Z T , Huang S Y , et al . Improving interpretability of deep sequential knowledge tracing models with question-centric cognitive representations [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2023 , 37 ( 12 ): 14196 - 14204 .

Nagatani K , Zhang Q , Sato M , et al . Augmenting knowledge tracing by considering forgetting behavior [C ] // Proceedings of the World Wide Web Conference . New York : ACM Press , 2019 : 3101 - 3107 .

Guan Q , Duan X , Bian K , et al . KVFKT: a new horizon in knowledge tracing with attention-based embedding and forgetting curve integration [C ] // Proceedings of the 31st International Conference on Computational Linguistics . Abu Dhabi ： Association for Computational Linguistics , 2025 : 4399 - 4409 .

Li S T , Shen S H , Su Y , et al . STHKT: spatiotemporal knowledge tracing with topological hawkes process [J ] . Expert Systems with Applications , 2025 , 259 : 125248 .

Vaswani A , Shazeer N , Parmar N , et al . Attention is all you need [C ] // Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017) . New York : ACM Press , 2017 : 6000 - 6010 .

Pandey S , Karypis G . A self-attentive model for knowledge tracing [C ] // Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019) . Montreal, Canada : International Educational Data Mining Society , 2019 : 384 - 389 .

Choi Y , Lee Y , Cho J , et al . Towards an appropriate query, key, and value computation for knowledge tracing [C ] // Proceedings of the Seventh ACM Conference on Learning @ Scale . New York : ACM Press , 2020 : 341 - 344 .

Shin D , Shim Y , Yu H , et al . SAINT+: integrating temporal features for EdNet correctness prediction [C ] // Proceedings of the LAK21: 11th International Learning Analytics and Knowledge Conference . New York : ACM Press , 2021 : 490 - 496 .

Yin Y , Dai L , Huang Z Y , et al . Tracing knowledge instead of patterns: stable knowledge tracing with diagnostic transformer [C ] // Proceedings of the ACM Web Conference 2023 . New York : ACM Press , 2023 : 855 - 864 .

Zhan H J , Kim J J , Liu G M . Contrastive learning with bidirectional transformers for knowledge tracing [C ] // Proceedings of the ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . Piscataway : IEEE Press , 2024 : 5040 - 5044 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A multi-feature fusion exercise recommendation model based on knowledge tracing machines

Research on network intrusion detection based on fusion Transformer and Inception

A modulation recognition model driven by lightweight knowledge distillation in the complex number domain

Research on the automatic generation method of telecom re-complaints report based on improved Transformer model

Research on evaluation of optical transport network health model based on data augmentation technology

Related Author

JIANG Xian

DONG Ligang

WANG Bingyan

YAN Lei

XIAO Mengfan

WANG Ying

ZHUGE Bin

Song Zhenfeng

Related Institution

College of Information and Electronic Engineering, Zhejiang Gongshang University

College of Information Network Security, People’s Public Security University of China

College of Information Network Security, People's Public Security University of China

School of Information Science and Technology, Tianjin University of Finance and Economics

China Telecom Corporation Guangzhou Branch

⁰