Research progress of KPI anomaly detection in intelligent operation and maintenance

Su WANG; Hua LU; Shuo WANG; Lei CAI; Tao HUANG

doi:10.11959/j.issn.1000-0801.2021105

您当前的位置：

首页 >

文章列表页 >

Research progress of KPI anomaly detection in intelligent operation and maintenance

Topic: Integration of Communication and AI | 更新时间：2024-06-05

- Research progress of KPI anomaly detection in intelligent operation and maintenance
- Telecommunications Science Vol. 37, Issue 5, Pages: 42-51(2021)
- 作者机构：
  
  1. 北京邮电大学网络与交换国家重点实验室, 北京 100876
  2. 广东省新一代通信与网络创新研究院，广东广州 510663
  3. 网络通信与安全紫金山实验室，江苏南京 211111
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2021105
  CLC： TP393
- Published Online：2021-05，
  
  Published：20 May 2021
- 稿件说明：
移动端阅览
Su WANG, Hua LU, Shuo WANG, et al. Research progress of KPI anomaly detection in intelligent operation and maintenance[J]. Telecommunications science, 2021, 37(5): 42-51.
DOI：

Su WANG, Hua LU, Shuo WANG, et al. Research progress of KPI anomaly detection in intelligent operation and maintenance[J]. Telecommunications science, 2021, 37(5): 42-51. DOI： 10.11959/j.issn.1000-0801.2021105.

摘要

现有的网络监控和故障修复大多依赖规则系统或者人工处理，然而随着网络规模的不断增大和业务的多样化，这种方式难以满足要求。随着机器学习和深度学习等技术的快速发展，智能运维理论也取得了长足进步，利用人工智能技术提升网络运维智能化能力。KPI（key performance indicator）异常检测是智能运维的一项底层核心技术。针对KPI异常检测技术研究展开综述，对KPI数据和KPI异常进行了描述，并从单指标和多指标两个方面详细介绍了KPI异常检测技术的研究现状；分析了KPI检测的部署应用问题，讨论了未来的研究方向。

Abstract

Existing network monitoring and fault repair mostly rely on rule systems or manual processing.However

the increase in network scale and the diversification of services make this approach difficult to deal with.With the rapid development of technology such as machine learning and deep learning

intelligent operation and maintenance theory has also made great progress

using artificial intelligence technology to enhance the intelligent ability of network operation and maintenance.KPI (key performance indicator) anomaly detection is an underlying core technology of intelligent operation and maintenance.A survey on the KPI anomaly detection technology was given.Firstly

the KPI data and KPI anomalies were described.Then the research progress of single-dimensional KPI and multi-dimensional KPI anomaly detection were introduced.Then

the deployment and application problems of KPI anomaly detection were analyzed.Finally

future research directions were discussed.

关键词

Keywords

references

李青 . 基于大数据分析的云资源池告警信息关联方案 [J ] . 电信科学 , 2020 , 36 ( 10 ): 159 - 171 .

LI Q . Alarm information association scheme of cloud resource pool based on big data analysis [J ] . Telecommunications Science , 2020 , 36 ( 10 ): 159 - 171 .

裴丹 , 张圣林 , 裴昶华 . 基于机器学习的智能运维 [J ] . 中国计算机学会通讯 , 2017 , 13 ( 12 ): 67 - 73 .

PEI D , ZHANG S L , PEI C H . Intelligent operation and main-tenance based on machine learning [J ] . Communications of CCF , 2017 , 13 ( 12 ): 67 - 73 .

CHEN W X , XU H W , LI Z Y , et al . Unsupervised anomaly detection for intricate KPIs via adversarial training of VAE [C ] // Proceedings of IEEE INFOCOM 2019 - IEEE Conference on Computer Communications . Piscataway: IEEE Press , 2019 : 1891 - 1899 .

CHANDOLA V , BANERJEE A , KUMAR V . Anomaly detection [J ] . ACM Computing Surveys , 2009 , 41 ( 3 ): 1 - 58 .

Amazon Web Services . Amazon cloudwatch alarm [EB ] . 2018 .

SIFFER A , FOUQUE P A , TERMIER A , et al . Anomaly detection in streams with extreme value theory [C ] // Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . New York: ACM Press , 2017 : 1067 - 1075 .

ZHANG Y , GE Z H , GREENBERG A , et al . Network anomography [C ] // Proceedings of the 5th ACM SIGCOMM conference on Internet measurement - IMC '05 . New York: ACM Press , 2005 : 30 - 30 .

YAN H , FLAVEL A , GE Z H , et al . Argus: End-to-end service anomaly detection and localization from an ISP's point of view [C ] // Proceedings of 2012 IEEE INFOCOM . Piscataway:IEEE Press , 2012 : 2756 - 2760 .

TAYLOR S J , LETHAM B . Forecasting at scale [J ] . The American Statistician , 2018 , 72 ( 1 ): 37 - 45 .

BREUNIG M M , KRIEGEL H P , NG R T , et al . LOF: identifying density-based local outliers [C ] // Proceedings of the 2000 ACM SIGMOD International Conference on Management of data - SIGMOD '00 . New York: ACM Press , 2000 : 93 - 104 .

KRIEGEL H P , S HUBERT M , ZIMEK A . Angle-based outlier detection in high-dimensional data [C ] // Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD 08 . New York:ACM Press , 2008 : 444 - 452 .

LAPTEV N , AMIZADEH S , FLINT I . Generic and scalable framework for automated time-series anomaly detection [C ] // Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . New York: ACM Press , 2015 : 1939 - 1947 .

LIU D P , ZHAO Y J , XU H W , et al . Opprentice: towards practical and automatic anomaly detection through machine learning [C ] // Proceedings of the 2015 Internet Measurement Conference . New York: ACM Press , 2015 : 211 - 224 .

AMER M , GOLDSTEIN M , ABDENNADHER S . Enhancing one-class support vector machines for unsupervised anomaly detection [C ] // Proceedings of the ACM SIGKDD Workshop on Outlier Detection and Description - ODD '13 . New York: ACM Press , 2013 : 8 - 15 .

YANG X W , LATECKI L J , POKRAJAC D . Outlier detection with globally optimal exemplar-based GMM [C ] // Proceedings of the 2009 SIAM International Conference on Data Mining . Philadelphia: Society for Industrial and Applied Mathematics , 2009 : 145 - 154 .

MUNZ G , LI S , CARLE G . Traffic anomaly detection using k-means clustering [C ] // Proceedings of GI/ITG Workshop MMBnet .[S.l.:s.n. ] , 2007 : 13 - 14 .

AN J , CHO S . Variational autoencoder based anomaly detection using reconstruction probability [J ] . Special Lecture on IE , 2015 , 2 ( 1 ): 1 - 18 .

GOODFELLOW I , POUGET-ABADIE J MIRZA M , et al . Generative adversarial nets [C ] // Advances in neural information processing systems .[S.l.:s.n. ] , 2014 : 2672 - 2680 .

OORD A , KALCHBRENNER N , KAVUKCUOGLU K . Pixel recurrent neural networks [J ] . arXiv preprint arXiv:1601.06759 , 2016 .

XU H W , FENG Y , CHEN J , et al . Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications [C ] // Proceedings of the 2018 World Wide Web Conference on World Wide Web-WWW '18 . New York:ACM Press , 2018 : 187 - 196 .

LI Z Y , CHEN W X , PEI D . Robust and unsupervised KPI anomaly detection based on conditional variational autoencoder [C ] // Proceedings of 2018 IEEE 37th International Performance Computing and Communications Conference (IPCCC) . Piscataway:IEEE Press , 2018 : 1 - 9 .

ARJOVSKY M , CHINTALA S , BOTTOU L . Wasserstein gan [J ] . arXiv preprint arXiv:1701.07875 , 2017 .

胡珉 , 白雪 , 徐伟 , 等 . 多维时间序列异常检测算法综述 [J ] . 计算机应用 , 2020 , 40 ( 6 ): 1553 - 1564 .

HU M , BAI X , XU W , et al . Review of anomaly detection algo-rithms for multidimensional time series [J ] . Journal of Computer Applications , 2020 , 40 ( 6 ): 1553 - 1564 .

Arundo Analytics . Anomaly detection toolkit [EB ] . 2019 .

GOLDSTEIN M , DENGEL A . Histogram-based outlier score (hbos): a fast unsupervised anomaly detection algorithm [J ] . KI-2012: Poster and Demo Track , 2012 : 59 - 63 .

陈兴蜀 , 江天宇 , 曾雪梅 , 等 . 基于多维时间序列分析的网络异常检测 [J ] . 工程科学与技术 , 2017 , 49 ( 1 ): 144 - 150 .

CHEN X S , JIANG T Y , ZENG X M , et al . Network anomaly detector based on multiple time series analysis [J ] . Advanced Engineering Sciences , 2017 , 49 ( 1 ): 144 - 150 .

HU M , JI Z W , YAN K , et al . Detecting anomalies in time series data via a meta-feature based approach [J ] . IEEE Access , 2018 ( 6 ): 27760 - 27776 .

HUNDMAN K , CONSTANTINOU V , LAPORTE C , et al . Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding [C ] // Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ＆Data Mining . New York: ACM Press , 2018 : 387 - 395 .

MALHOTRA P , RAMAKRISHNAN A , ANAND G , et al . LSTM-based encoder-decoder for multi-sensor anomaly detection [J ] . arXiv preprint arXiv:1607.00148 , 2016 .

PARK D , HOSHI Y , KEMP C C . A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder [J ] . IEEE Robotics and Automation Letters , 2018 , 3 ( 3 ): 1544 - 1551 .

SU Y , ZHAO Y J , NIU C H , et al . Robust anomaly detection for multivariate time series through stochastic recurrent neural network [C ] // Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining . New York: ACM Press , 2019 : 2828 - 2837 .

ZHANG C X , SONG D J , CHEN Y C , et al . A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2019 ( 33 ): 1409 - 1416 .

ZHAO N W , ZHU J , WANG Y , et al . Automatic and generic periodicity adaptation for KPI anomaly detection [J ] . IEEE Transactions on Network and Service Management , 2019 , 16 ( 3 ): 1170 - 1183 .

MA M H , ZHANG S L , PEI D , et al . Robust and rapid adaption for concept drift in software system anomaly detection [C ] // Proceedings of 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE) . Piscataway: IEEE Press , 2018 : 13 - 24 .

LI Z H , ZHAO Y J , LIU R , et al . Robust and rapid clustering of KPIs for large-scale anomaly detection [C ] // Proceedings of 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS) . Piscataway: IEEE Press , 2018 : 1 - 10 .

BU J H , LIU Y , ZHANG S L , et al . Rapid deployment of anomaly detection models for large number of emerging KPI streams [C ] // Proceedings of 2018 IEEE 37th International Performance Computing and Communications Conference (IPCCC) . Piscataway: IEEE Press , 2018 : 1 - 8 .

CHENG Z Q , YANG Y , WANG W , et al . Time2Graph: revisiting time series modeling with dynamic shapelets [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 4 ): 3617 - 3624 .

SHEN Z Y , CUI P , LIU J S , et al . Stable learning via differentiated variable decorrelation [C ] // Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery ＆Data Mining . New York: ACM Press , 2020 : 2185 - 2193 .

HUANG Q B , WEI J L , CAI Y , et al . Aligned dual channel graph convolutional network for visual question answering [C ] // Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Online . Stroudsburg: Association for Computational Linguistics , 2020 : 7166 - 7176 .

CHANG H , RONG Y , XU T Y , et al . A restricted black-box adversarial framework towards attacking graph embedding models [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 4 ): 3389 - 3396 .

Views

932

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on the application of knowledge graph in intelligent operation and maintenance for optical transport network

Research on anomaly detection algorithm based on sparse variational autoencoder using spike and slab prior

Research and application practice of knowledge graph technology system for telecom-operators

Anomaly detection algorithm based on Gaussian mixture variational auto encoder network

Quality analysis method of Web browsing based on big data

Related Author

SHAO Zhugui

SHANG Fengliang

XU Jianrong

LIU Cui

YANG Ganggang

Huahua CHEN

Zhe CHEN

Dongming ZHAO

Related Institution

Research Institute of China Telecom Co., Ltd.

China Telecom Co.,Ltd.

School of Information and Electrical Engineering,Zhejiang Gongshang University

LiLAB,University of Florida,Gainesville

⁰