FedGMH: research on label interference elimination via global multi-head

Sun Zhenfeng; Ni Zhengwei

doi:10.11959/j.issn.1000-0801.2026005

您当前的位置：

首页 >

文章列表页 >

FedGMH: research on label interference elimination via global multi-head

Research and Development | 更新时间：2026-03-02

- FedGMH: research on label interference elimination via global multi-head
- Telecommunications Science Vol. 42, Issue 1, Pages: 159-173(2026)
- 作者机构：
  
  浙江工商大学信息与电子工程学院，浙江杭州 310018
- 作者简介：
- 基金信息：
  
  The Tongxiang General Artificial Intelligence Research Institute Project(TAGI2-B-2024-0017)
- DOI：10.11959/j.issn.1000-0801.2026005
  CLC： TN918;TP309
- Received：09 March 2025，
  
  Revised：2025-07-10，
  
  Accepted：28 July 2025，
  
  Published：20 January 2026
- 稿件说明：
移动端阅览
孙圳峰,倪郑威.FedGMH：基于全局多头部的标签干扰消除方法研究[J].电信科学,2026,42(01):159-173.

Sun Zhenfeng,Ni Zhengwei.FedGMH: research on label interference elimination via global multi-head[J].Telecommunications Science,2026,42(01):159-173.
孙圳峰,倪郑威.FedGMH：基于全局多头部的标签干扰消除方法研究[J].电信科学,2026,42(01):159-173. DOI： 10.11959/j.issn.1000-0801.2026005.

Sun Zhenfeng,Ni Zhengwei.FedGMH: research on label interference elimination via global multi-head[J].Telecommunications Science,2026,42(01):159-173. DOI： 10.11959/j.issn.1000-0801.2026005.

摘要

个性化联邦学习因其在应对数据异质性和隐私保护方面的优势而备受关注。现有算法专注于平衡全局信息和个性化信息之间的矛盾，忽视了全局信息中的不同标签信息带来的干扰，尤其在维护单一全局头部的算法中，容易出现标签间特征冲突导致的收敛困难。为此，提出一种新的算法——全局多头部联邦学习（federated learning with global multi-head，FedGMH）算法，该算法在服务器创建多个全局头部，每个头部专门处理一种标签信息，而客户端下载与本地标签相关的全局头部，从而避免无关标签信息的干扰。此外，FedGMH引入参数级聚合机制：评估头部参数重要性，并将关键参数更新为全局多头部的加权参数，以加快收敛速度并且提高准确率。在3个视觉数据集上的大量实验表明，FedGMH优于现有的先进算法。

Abstract

Personalized federated learning has garnered significant attention due to its advantages in addressing data heterogeneity and privacy protection. However

existing algorithms predominantly focus on balancing the contradiction between global and personalized information

often overlooking the interference caused by distinct label information within global features. Particularly in algorithms maintaining a single global head

conflicts between label-specific features can lead to convergence challenges. To address this

a novel algorithm—federated learning with global multi-head (FedGMH) was proposed. The proposed algorithm creates multiple global heads on the server

each dedicated to processing one category of label information. Clients selectively download global heads relevant to their local labels

thereby avoiding interference from unrelated label information. Furthermore

FedGMH incorporates a parameter-level aggregation mechanism: it assesses the importance of head parameters and updates critical parameters to a weighted ensemble of the global multi-head

accelerating convergence and improving accuracy. Extensive experiments on three visual datasets demonstrate that FedGMH outperforms state-of-the-art baseline algorithms.

关键词

Keywords

references

Islam T U , Ghasemi R , Mohammed N . Privacy-preserving federated learning model for healthcare data [C ] // Proceedings of the 2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC) . Piscataway : IEEE Press , 2022 : 281 - 287 .

Sheller M J , Edwards B , Reina G A , et al . Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data [J ] . Scientific Reports , 2020 , 10 : 12598 .

Sadilek A , Liu L Y , Nguyen D , et al . Privacy-first health research with federated learning [J ] . NPJ Digital Medicine , 2021 , 4 : 132 .

Awosika T , Shukla R M , Pranggono B . Transparency and privacy: the role of explainable AI and federated learning in financial fraud detection [J ] . IEEE Access , 2024 , 12 : 64551 - 64560 .

Yang W S , Zhang Y H , Ye K J , et al . FFD: a federated learning based method for credit card fraud detection [C ] // Big Data-BigData 2019 . Cham : Springer , 2019 : 18 - 32 .

Ferdous Aurna N , Delwar Hossain M , Khan L , et al . FedFusion: adaptive model fusion for addressing feature discrepancies in federated credit card fraud detection [J ] . IEEE Access , 2024 , 12 : 136962 - 136978 .

Liu L , Tian Y X , Chakraborty C , et al . Multilevel federated learning-based intelligent traffic flow forecasting for transportation network management [J ] . IEEE Transactions on Network and Service Management , 2023 , 20 ( 2 ): 1446 - 1458 .

Liu T B , Wang Y , Zhou H Y , et al . Distributed short-term traffic flow prediction based on integrating federated learning and TCN [J ] . IEEE Access , 2024 , 12 : 148026 - 148036 .

Tan A Z , Yu H , Cui L Z , et al . Towards personalized federated learning [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2023 , 34 ( 12 ): 9587 - 9603 .

Arivazhagan M G , Aggarwal V , Singh A K , et al . Federated learning with personalization layers [PP ] . arXiv preprint , 2019 , arXiv: 1912.00818 .

Collins L , Hassani H , Mokhtari A , et al . Exploiting shared representations for personalized federated learning [PP ] . arXiv preprint , 2021 , arXiv: 2102.07078 .

Mcmahan H B , Moore E , Ramage D , et al . Communication-efficient learning of deep networks from decentralized data [C ] // Proceedings of the International Conference on Artificial Intelligence and Statistics , 2016

Li Q B , Diao Y Q , Chen Q , et al . Federated learning on non-IID data silos: an experimental study [C ] // Proceedings of the 2022 IEEE 38th International Conference on Data Engineering (ICDE) . Piscataway : IEEE Press , 2022 : 965 - 978 .

Yurochkin M , Agarwal M , Ghosh S , et al . Bayesian nonparametric federated learning of neural networks [PP ] . arXiv preprint , 2019 , arXiv: 1905.12022 .

Li Y P , Lyu X C . Convergence analysis of sequential split learning on heterogeneous data [PP ] . arXiv preprint , 2023 , arXiv: 2302.01633 .

Yi L P , Wang G , Liu X G , et al . FedGH: heterogeneous federated learning with generalized global header [C ] // Proceedings of the 31st ACM International Conference on Multimedia . New York : ACM Press , 2023 : 8686 - 8696 .

Chen H Y , Chao W L . On bridging generic and personalized federated learning for image classification [PP ] . arXiv preprint , 2021 , arXiv: 2107.00778 .

Zhang J Q , Hua Y , Wang H , et al . FedCP: separating feature information for personalized federated learning via conditional policy [C ] // Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . New York : ACM Press , 2023 : 3249 - 3261 .

Wu X H , Liu X F , Niu J W , et al . Bold but cautious: unlocking the potential of personalized federated learning through cautiously aggressive collaboration [C ] // Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2023 : 19318 - 19327 .

Li T , Sahu A K , Zaheer M , et al . Federated optimization in heterogeneous networks [PP ] arXiv preprint , 2020 , arXiv: 1812.06127 .

Li T , Hu S Y , Beirami A , et al . Ditto: fair and robust federated learning through personalization [PP ] . arXiv preprint , 2020 , arXiv: 2012.04221 .

Dinh C T , Tran N H , Nguyen J D . Personalized federated learning with Moreau envelope [PP ] . arXiv preprint , 2020 ,arXiv: 2006.08848 .

Chen F , Luo M , Dong Z H , et al . Federated meta-learning with fast convergence and efficient communication [PP ] . arXiv preprint , 2019 , arXiv:1802. 07876 ,

Fallah A , Mokhtari A , Ozdaglar A E . Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach [C ] // Advances in Neural Information Processing Systems . New York : Curran Associates, Inc , 2020 : 3557 - 3568 .

Liang P P , Liu T , Liu Z Y , et al . Think locally, act globally: federated learning with local and global representations [PP ] . arXiv preprint , 2020 , arXiv: 2001.01523 .

Deng Y Y , Kamani M M , Mahdavi M . Adaptive personalized federated learning [PP ] . arXiv preprint , 2020 ,arXiv: 2003.13461 .

Zhang J Q , Hua Y , Wang H , et al . FedALA: adaptive local aggregation for personalized federated learning [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2023 , 37 ( 9 ): 11237 - 11244 .

Zhang M , Sapra K , Fidler S , et al . Personalized federated learning with first order model optimization [PP ] . arXiv preprint , 2020 , arXiv: 2012.08565 .

Huang Y T , Chu L Y , Zhou Z R , et al . Personalized cross-silo federated learning on non-IID data [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 9 ): 7865 - 7873 .

Zhang J Q , Hua Y , Wang H , et al . GPFL: simultaneously learning global and personalized feature information for personalized federated learning [C ] // Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2023 : 5018 - 5028 .

Li X X , Jiang M R , Zhang X F , et al . FedBN: federated learning on non-IID features via local batch normalization [PP ] . arXiv preprint , 2021 , arXiv: 2102.07623 .

Shen Y Q , Zhou Y Y , Yu L Q . CD2-pFed: cyclic distillation-guided channel decoupling for model personalization in federated learning [C ] // Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2022 : 10031 - 10040 .

Paszke A , Gross S , Massa F , et al . PyTorch: an imperative style, high-performance deep learning library [C ] // Advances in Neural Information Processing Systems . New York : Curran Associates, Inc , 2019 : 8024 - 8035 .

Geiping J , Bauermeister H , Dröge H , et al . Inverting gradients-how easy is it to break privacy in federated learning [C ] // Advances in Neural Information Processing Systems . New York : Curran Associates, Inc , 2020 : 16937 - 16947 .

Luo M , Chen F , Hu D P , et al . No fear of heterogeneity: classifier calibration for federated learning with non-IID data [C ] // Advances in Neural Information Processing Systems . New York : Curran Associates, Inc , 2021 : 5972 - 5984 .

Harris C R , Millman K J , Van Der Walt S J , et al . Array programming with NumPy [J ] . Nature , 2020 , 585 ( 7825 ): 357 - 362 .

Krizhevsky A , Hinton G . Learning multiple layers of features from tiny images [J ] . Handbook of Systemic Autoimmune Diseases , 2009 , 1 ( 4 ).

Gao L , Li Z X , Wu C , et al . FediOS: decoupling orthogonal subspaces for personalization in feature-skew federated learning [PP ] . arXiv preprint , 2023 , arXiv: 2311.18559 .

Lecun Y , Bottou L , Bengio Y , et al . Gradient-based learning applied to document recognition [J ] . Proceedings of the IEEE , 1998 , 86 ( 11 ): 2278 - 2324 .

Views

179

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

FedDACS: personalized knowledge transfer based on distribution-aware controlled sampling optimization

A personalized mixture-of-experts federated fine-tuning framework for vision-language models

Related Author

SUN Zhenfeng

NI Zhengwei

Qi Xiaochao

Ni Zhengwei

DING Meilin

LI Rongsheng

ZHAO Dongming

GAO Fei

Related Institution

School of Information and Electronic Engineering, Zhejiang Gongshang University

College of Intelligence and Computing,Tianjin University

Artificial Intelligence Industry Research Institute,China Mobile Communications Group Tianjin Co., Ltd.

⁰