DS-PPO: Cloud-Edge-End Collaborative Scheduling of Heterogeneous Hardware Resources for Highly Concurrent CNN Model Inference

ZHANG Kuo; YAN Yaqi; AN Ying; DONG Yuchi; ZHANG Mingui; YUAN Gang

doi:null

您当前的位置：

首页 >

文章列表页 >

DS-PPO: Cloud-Edge-End Collaborative Scheduling of Heterogeneous Hardware Resources for Highly Concurrent CNN Model Inference

更新时间：2026-05-11

- DS-PPO: Cloud-Edge-End Collaborative Scheduling of Heterogeneous Hardware Resources for Highly Concurrent CNN Model Inference
- Telecommunications Science (2026)
- 作者机构：
  
  1.中国铁塔股份有限公司北京 100089
  2.中移（苏州）软件技术有限公司苏州 215000
- 作者简介：
- 基金信息：
- DOI：
  CLC： TP393
- Received：12 January 2026，
  
  Revised：2026-02-12，
  
  Accepted：20 March 2026，
- 稿件说明：
移动端阅览
ZHANG Kuo, YAN Yaqi, AN Ying, et al. DS-PPO: Cloud-Edge-End Collaborative Scheduling of Heterogeneous Hardware Resources for Highly Concurrent CNN Model Inference[J/OL]. Telecommunications Science, 2026.
DOI：

ZHANG Kuo, YAN Yaqi, AN Ying, et al. DS-PPO: Cloud-Edge-End Collaborative Scheduling of Heterogeneous Hardware Resources for Highly Concurrent CNN Model Inference[J/OL]. Telecommunications Science, 2026. DOI：

摘要

随着人工智能物联网（AIoT）技术的发展和硬件性能的提升，云边端协同场景下引入卷积神经网络（CNN）模型进行推理已成为趋势，以满足智能AI应用对实时性的需求。针对异构计算资源（如CPU、GPU、NPU、FPGA）的高效利用问题，提出了一种创新的任务调度系统，以支持高并发CNN模型推理。该系统采用DeepSets模型进行任务和环境特征提取，并提升近端策略优化算法的泛化能力实现任务调度决策，确保在复杂多变的环境中实现精准高效的任务分配。系统已在Kubernetes和Kosmos平台上部署，并通过高性能服务器、NVIDIA Jetson Xavier NX和NVIDIA Jetson Nano等异构处理单元进行了验证。实验结果显示，相较于现有调度算法，该系统的性能提升了275.93%，资源利用率提高38.86%，显著提高了云边端计算的效率和可靠性，为智能AI应用提供了坚实的支持。

Abstract

With the advancement of artificial intelligence of things (AIoT) technology and the enhancement of hardware performance

convolutional neural network (CNN) models were increasingly introduced for inference in collaborative cloud-edge-end scenarios to satisfy real-time requirements of intelligent AI applications. To address the efficient utilization of heterogeneous computing resources including CPUs

GPUs

NPUs

and FPGAs

an innovative task scheduling system was proposed to support high-concurrency CNN model inference. The DeepSets model was employed to extract features from tasks and environmental constraints

while the generalization capability of the Proximal Policy Optimization (PPO) algorithm was enhanced to achieve task scheduling decisions. Precise and efficient task allocation was ensured in complex dynamic environments through this approach. The proposed system was deployed on Kubernetes and Kosmos platforms

with comprehensive validation conducted across various heterogeneous processing units

including high-performance servers

NVIDIA Jetson Xavier NX

and NVIDIA Jetson Nano devices. Experimental results demonstrate that compared to existing scheduling algorithms

the proposed system improves performance by 275.93% and increases resource utilization by 38.86%

significantly enhancing the efficiency and reliability of cloud-edge computing frameworks and providing robust support for intelligent AI applications.

关键词

Keywords

references

Hosseinzadeh M , Azhir E , Lansky J , et al . Task scheduling mechanisms for fog computing: a systematic survey [J ] . IEEE Access , 2023 , 11 : 50994 - 51017 .

Jiang M , Wu T , Wang Z , et al . A multi-intersection vehicular cooperative control based on end-edge-cloud computing [J ] . IEEE Transactions on Vehicular Technology , 2022 , 71 ( 3 ): 2459 - 2471 .

Ren J , Jiang H , Shen X , et al . Editorial of ccf transactions on networking: special issue on intelligence-enabled end-edge-cloud orchestrated computing [J ] . CCF Transactions on Networking , 2020 , 3 ( 3 ): 155 - 157 .

Ren J , Zhang D , He S , et al . A survey on end-edge-cloud orchestrated network computing paradigms: Transparent computing, mobile edge computing, fog computing, and cloudlet [J ] . ACM Computing Surveys (CSUR) , 2019 , 52 ( 6 ): 1 - 36 .

Attiya I , Abd Elaziz M , Abualigah L , et al . An improved hybrid swarm intelligence for scheduling IoT application tasks in the cloud [J ] . IEEE Transactions on Industrial Informatics , 2022 , 18 ( 9 ): 6264 - 6272 .

Alsaidy S A , Abbood A D , Sahib M A . Heuristic initialization of PSO task scheduling algorithm in cloud computing [J ] . Journal of King Saud University-Computer and Information Sciences , 2022 , 34 ( 6 ): 2370 - 2382 .

Wang L , Pan Z , Wang J . A review of reinforcement learning based intelligent optimization for manufacturing scheduling [J ] . Complex System Modeling and Simulation , 2021 , 1 ( 4 ): 257 - 270 .

Houssein E H , Gad A G , Wazery Y M , et al . Task scheduling in cloud computing based on meta-heuristics: review, taxonomy, open challenges, and future trends [J ] . Swarm and Evolutionary Computation , 2021 , 62 : 100841 .

Yuan H , Bi J , Zhou M C . Multiqueue scheduling of heterogeneous tasks with bounded response time in hybrid green IaaS clouds [J ] . IEEE Transactions on Industrial Informatics , 2019 , 15 ( 10 ): 5404 - 5412 .

Zhou J , Sun J , Cong P , et al . Security-critical energy-aware task scheduling for heterogeneous real-time MPSoCs in IoT [J ] . IEEE Transactions on Services Computing , 2019 , 13 ( 4 ): 745 - 758 .

Carrión C . Kubernetes scheduling: Taxonomy, ongoing issues and challenges [J ] . ACM Computing Surveys , 2022 , 55 ( 7 ): 1 - 37 .

Narayanan D , Santhanam K , Kazhamiaka F , et al . {Heterogeneity-Aware} cluster scheduling policies for deep learning workloads [C ] // 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). 2020 : 481 - 498 .

Feng J , Zhang W , Pei Q , et al . Heterogeneous computation and resource allocation for wireless powered federated edge learning systems [J ] . IEEE Transactions on Communications , 2022 , 70 ( 5 ): 3220 - 3233 .

Zhong Z , Buyya R . A cost-efficient container orchestration strategy in kubernetes-based cloud computing infrastructures with heterogeneous resources [J ] . ACM Transactions on Internet Technology (TOIT) , 2020 , 20 ( 2 ): 1 - 24 .

Abdulazeez D H , Askar S K . Offloading mechanisms based on reinforcement learning and deep learning algorithms in the fog computing environment [J ] . Ieee Access , 2023 , 11 : 12555 - 12586 .

Jung Y , Park J . Bayesian Convolutional Deep Sets with Task-Dependent Stationary Prior [C ] // International Conference on Artificial Intelligence and Statistics . PMLR , 2023 : 3795 - 3824 .

He H , Meng X , Wang Y , et al . Deep reinforcement learning based energy management strategies for electrified vehicles: Recent advances and perspectives [J ] . Renewable and Sustainable Energy Reviews , 2024 , 192 : 114248 .

Xie B , Bian Y , Chen Y , et al . Enhancing neural subset selection: Integrating background information into set representations [J ] . arXiv preprint arXiv: 2402.03139 , 2024 .

Kim C H , Kim S J , Choi H L . Heterogeneous satellite task scheduling with revisit time minimization using milp formulation [C ] //AIAA SCITECH 2024 F orum . 2024 : 0747 .

Cui H , Tang Z , Lou J , et al . Latency-aware container scheduling in edge cluster upgrades: A deep reinforcement learning approach [J ] . IEEE Transactions on Services Computing , 2024 , 17 ( 5 ): 2530 - 2543 .

Cheng Y , Guo Q , Wang X . Proximal policy optimization with advantage reuse competition [J ] . IEEE Transactions on Artificial Intelligence , 2024 , 5 ( 8 ): 3915 - 3925 .

Du Y , Li J . A deep reinforcement learning based algorithm for a distributed precast concrete production scheduling [J ] . International Journal of Production Economics , 2024 , 268 : 109102 .

Yassami M , Ashtari P . A novel hybrid optimization algorithm: Dynamic hybrid optimization algorithm [J ] . Multimedia Tools and Applications , 2023 , 82 ( 21 ): 31947 - 31979 .

Premkumar M , Sowmya R , Ramakrishnan C , et al . An efficient and reliable scheduling algorithm for unit commitment scheme in microgrid systems using enhanced mixed integer particle swarm optimizer considering uncertainties [J ] . Energy Reports , 2023 , 9 : 1029 - 1053 .

Rouskas G N . First-Fit: A universal algorithm for spectrum assignment [C ] // GLOBECOM 2023-2023 IEEE Global Communications Conference . IEEE , 2023 : 2123 - 2128 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

An unbalanced sample prediction algorithm based on LSTM and CNN in the new power systems

Multi-user fine-grained task offloading scheduling strategy under cloud-edge-end collaboration

Computational measurement and task scheduling: a study on IoT edge device strategies

Improved ant colony algorithm based cloud computing user task scheduling algorithm

Multi-constrained trusted cooperative task migration strategy for edge computing

Related Author

MA Jianbo

FAN Yeping

LI Zhihao

ZHANG Qi

XIE Mande

HUANG Zhufang

SUN Hao

XU Qingqing

Related Institution

Anhui Jiyuan Software Co., Ltd.

School of Information and Electronic Engineering, Zhejiang Gongshang University

China Mobile Research Institute

School of Mathematics and Information Engineering,Jiaxing University

School of Science, Jiangxi University of Science and Technology

⁰