FedDACS: 基于分布感知可控采样优化的个性化群体知识迁移

齐霄超; 倪郑威

doi:10.11959/j.issn.1000-0801.DXKX250206

您当前的位置：

首页 >

文章列表页 >

FedDACS: 基于分布感知可控采样优化的个性化群体知识迁移

研究与开发 | 更新时间：2026-05-13

- FedDACS: 基于分布感知可控采样优化的个性化群体知识迁移
- FedDACS: personalized knowledge transfer based on distribution-aware controlled sampling optimization
- 电信科学 2026年42卷第4期页码：88-100
- 作者机构：
  
  浙江工商大学信息与电子工程学院，浙江杭州 310018
- 作者简介：
  
  [ "齐霄超（2001- ），女，浙江工商大学信息与电子工程学院硕士生，主要研究方向为个性化联邦学习。" ]
  [ "倪郑威（1989- ），男，博士，浙江工商大学信息与电子工程学院副研究员、硕士生导师，主要研究方向为人工智能、无线网络。" ]
- 基金信息：
  
  桐乡市通用人工智能研究院项目(TAGI2-B-2024-0017)
- DOI：10.11959/j.issn.1000-0801.DXKX250206
  中图分类号： TP181
- 收稿：2025-03-29，
  
  修回：2025-06-23，
  
  录用：2025-07-07，
  
  纸质出版：2026-04-20
- 稿件说明：
移动端阅览
齐霄超,倪郑威.FedDACS: 基于分布感知可控采样优化的个性化群体知识迁移[J].电信科学,2026,42(04):88-100.

Qi Xiaochao,Ni Zhengwei.FedDACS: personalized knowledge transfer based on distribution-aware controlled sampling optimization[J].Telecommunications Science,2026,42(04):88-100.
齐霄超,倪郑威.FedDACS: 基于分布感知可控采样优化的个性化群体知识迁移[J].电信科学,2026,42(04):88-100. DOI： 10.11959/j.issn.1000-0801.DXKX250206.

Qi Xiaochao,Ni Zhengwei.FedDACS: personalized knowledge transfer based on distribution-aware controlled sampling optimization[J].Telecommunications Science,2026,42(04):88-100. DOI： 10.11959/j.issn.1000-0801.DXKX250206.

摘要

个性化联邦学习作为分布式机器学习领域的前沿范式，通过为各客户端训练专属模型来应对数据异构性挑战。现有研究采用群体知识迁移技术，在服务器端部署共享预测器提升模型在基类上的泛化能力，在本地更新特征提取器提升模型个性化能力。但仍存在两方面显著缺陷：其一，全局共享的预测器难以适配高度异构的客户端分布，导致准确率下降；其二，知识迁移过程中易受分布差异干扰，出现负迁移效应。为此，提出了分布感知可控采样优化的个性化群体知识迁移框架（FedDACS）。该框架将服务器端传统单一共享预测器改为个性化预测器阵列，每个客户端对应专属预测器模块，与本地特征提取器构成个性化模型。此外，引入了分布感知可控采样（DACS）技术，通过实时分析各客户端数据分布，动态调整特征空间采样策略，控制个性化预测器的输入。该框架可有效抑制分布差异导致的负迁移效应。为了验证FedDACS的有效性，在CIFAR-10与CIFAR-100数据集上构建了7种非独立同分布（Non-IID）场景进行系统性验证。实验结果表明，相较于Fedgkt、FedProto等8种基准方法，FedDACS在CIFAR-10数据集上实现平均用户准确率提升幅度达6.85%，在CIFAR-100数据集上提升幅度达5.28%。验证了所提FedDACS方法能够有效提升个性化模型在不同数据异质性场景下的性能。

Abstract

Personalized federated learning

as a cutting-edge paradigm in the field of distributed machine learning

is developed to address the challenges of data heterogeneity by training exclusive models for each client. Existing research employes population knowledge transfer techniques

where a shared predictor is deployed on the server to enhance the generalization ability of the model over the base class

while local feature extractors are updated to improve the personalization ability of the model. However

two significant drawbacks remain. Firstly

the globally shared predictor struggles to adapt to highly heterogeneous client distributions

resulting in decreased accuracy. Secondly

the knowledge transfer process is easily disturbed by distribution differences

leading to negative transfer effects. To address these issues

a distribution-aware controlled sampling optimization framework for personalized population knowledge transfer (FedDACS) was proposed. The traditional single shared predictor on the server was transformed into an array of personalized predictors

with each client corresponding to an exclusive predictor module

forming a personalized model alongside local feature extractors. Furthermore

a distribution-aware controlled sampling technique (DACS) was introduced

which dynamically adjusted the feature space sampling strategy by analyzing the data distributions of each client in real-time

thereby controlling the input of personalized predictors. The negative transfer effects caused by distribution differences were effectively mitigated. To validate the effectiveness of FedDACS

seven Non-IID scenarios were constructed on CIFAR-10 and CIFAR-100 datasets for systematic validation. Experimental results demonstrate that

compared to eight baseline methods such as Fedgkt and FedProto

FedDACS achieves an average user accuracy improvement of 6.85% on CIFAR-10 and an enhancement of 5.28% on CIFAR-100. These results indicate that the proposed FedDACS method can effectively improve the performance of personalized models in various data heterogeneity scenarios.

关键词

Keywords

references

Tan A Z , Yu H , Cui L Z , et al . Towards personalized federated learning [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2023 , 34 ( 12 ): 9587 - 9603 .

Fallah A , Mokhtari A , Ozdaglar A E . Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach [C ] // Proceedings of the 2020 Neural Information Processing Systems . Montreal : Neural Information Processing Systems , 2020 .

Zhao Y , Li M , Lai L Z , et al . Federated learning with non-IID data [EB ] . 2018 .

Yang M , Wang X M , Zhu H B , et al . Federated learning with class imbalance reduction [C ] // Proceedings of the 2021 29th European Signal Processing Conference (EUSIPCO) . Piscataway : IEEE Press , 2021 : 2174 - 2178 .

Li T , Sahu A K , Zaheer M , et al . Federated optimization in heterogeneous networks [C ] // Proceedings of the 2020 Machine Learning and Systems , 2020 , 2 : 429 - 450 .

Li D L , Wang J P . FedMD: heterogenous federated learning via model distillation [EB ] . 2019 .

Liang P P , Liu T , Liu Z Y , et al . Think locally, act globally: federated learning with local and global representations [EB ] . 2020 .

SMITH V , CHIANG C K , SANJABI M , et al . Federated multi-task learning [C ] // Proceedings of the 2025 Neural Information Processing Systems . Montreal : Neural Information Processing Systems , 2025 .

Deng Y Y , Kamani M M , Mahdavi M . Adaptive personalized federated learning [EB ] . 2020 .

Lin T , Kong L J , Stich S U , et al . Ensemble distillation for robust model fusion in federated learning [EB ] . 2020 .

Chen H C , Wang J , Haris V , et al . The best of both worlds: accurate global and personalized models through federated learning with data-free hyper-knowledge distillation [EB ] . 2023 .

He C Y , Annavaram M , Avestimehr S . Group knowledge transfer: federated learning of large CNNs at the edge [EB ] . 2020 .

Wu Z Y , Sun S , Wang Y W , et al . Exploring the distributed knowledge congruence in proxy-data-free federated distillation [J ] . ACM Transactions on Intelligent Systems and Technology , 2024 , 15 ( 2 ): 1 - 34 .

Wu Z Y , Sun S , Wang Y W , et al . FedICT: federated multi-task distillation for multi-access edge computing [J ] . IEEE Transactions on Parallel and Distributed Systems , 2024 , 35 ( 6 ): 1107 - 1121 .

Itahara S , Nishio T , Koda Y , et al . Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-IID private data [J ] . IEEE Transactions on Mobile Computing , 2023 , 22 ( 1 ): 191 - 205 .

Chang H Y , Shejwalkar V , Shokri R , et al . Cronus: robust and heterogeneous collaborative learning with black-box knowledge transfer [EB ] . 2019 .

Cho Y J , Wang J Y , Chiruvolu T , et al . Personalized federated learning for heterogeneous clients with clustered knowledge transfer [EB ] . 2021 .

Zhang J , Guo S , Ma X , et al . Parameterized knowledge transfer for personalized federated learning [C ] // Proceedings of the 2021 Neural Information Processing Systems . Montreal : Neural Information Processing Systems , 2021 , 34 : 10092 - 10104 .

Cheng S J , Wu J W , Xiao Y H , et al . FedGEMS: federated learning of larger server models via selective knowledge fusion [EB ] . 2021 .

Xie C L , Huang D A , Chu W D , et al . Perada: parameter-efficient federated learning personalization with generalization guarantees [C ] // Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2024 : 23838 - 23848 .

Wu Z Y , Sun S , Wang Y W , et al . FedCache: a knowledge cache-driven federated learning architecture for personalized edge intelligence [J ] . IEEE Transactions on Mobile Computing , 2024 , 23 ( 10 ): 9368 - 9382 .

Tan Y , Long G D , Liu L , et al . FedProto: federated prototype learning across heterogeneous clients [J ] . Proceedings of the AAAI Conference on Artificial Intelligence. Menlo Park: AAAI Press , 2022 , 36 ( 8 ): 8432 - 8440 .

Chen Y Q , Lu W , Qin X , et al . MetaFed: federated learning among federations with cyclic knowledge distillation for personalized healthcare [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2024 , 35 ( 11 ): 16671 - 16682 .

庄福振 , 罗平 , 何清 , 等 . 迁移学习研究进展 [J ] . 软件学报 , 2015 , 26 ( 1 ): 26 - 39 .

Zhuang F Z , Luo P , He Q , et al . Survey on transfer learning research [J ] . Journal of Software , 2015 , 26 ( 1 ): 26 - 39 .

Lee G , Jeong M , Shin Y , et al . Preservation of the global knowledge by not-true distillation in federated learning [EB ] . 2021 .

He Y T , Chen Y Q , Yang X D , et al . Class-wise adaptive self distillation for federated learning on Non-IID data (student abstract) [C ] // Proceedings of the 2022 AAAI Conference on Artificial Intelligence . Menlo Park : AAAI Press , 2022 , 36 ( 11 ): 12967 - 12968 .

Karimireddy S P , Kale S , Mohri M , et al . Scaffold: stochastic controlled averaging for federated learning [C ] // Proceedings of the 2020 Conference and Workshop on Neural Information Processing Systems . Montreal : Neural Information Processing Systems , 2020 : 5132 - 5143 .

Yao D Z , Pan W N , Dai Y T , et al . Local-global knowledge distillation in heterogeneous federated learning with Non-IID data [EB ] . 2021 .

Li Q B , Diao Y Q , Chen Q , et al . Federated learning on Non-IID data silos: an experimental study [C ] // Proceedings of the 2022 IEEE 38th International Conference on Data Engineering (ICDE) . Piscataway : IEEE Press , 2022 : 965 - 978 .

Nguyen Q , Pham H H , Wong K S , et al . FedDCT: federated learning of large convolutional neural networks on resource-constrained devices using divide and collaborative training [J ] . IEEE Transactions on Network and Service Management , 2024 , 21 ( 1 ): 418 - 436 .

Zhang J Q , Hua Y , Wang H , et al . GPFL: simultaneously learning global and personalized feature information for personalized federated learning [C ] // Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2023 : 5018 - 5028 .

Mills J , Hu J , Min G Y . Multi-task federated learning for personalised deep neural networks in edge computing [J ] . IEEE Transactions on Parallel and Distributed Systems , 2022 , 33 ( 3 ): 630 - 641 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

FedGMH：基于全局多头部的标签干扰消除方法研究

针对视觉-语言模型的个性化混合专家联邦微调框架