基于用户移动网络接入位置的高效分布式相似矩阵计算方法

王源; 江昊; 吴明; 姚冬桂; 张毅; 羿舒文; 汪海; 吴静

doi:10.11959/j.issn.1000-0801.2018136

您当前的位置：

首页 >

文章列表页 >

基于用户移动网络接入位置的高效分布式相似矩阵计算方法

研究与开发 | 更新时间：2024-06-05

- 基于用户移动网络接入位置的高效分布式相似矩阵计算方法
- A distributed high efficiency similarity matrix computation method based on users’ mobile network access location
- 电信科学 2018年34卷第5期页码：26-38
- 作者机构：
  
  1. 武汉大学电子信息学院，湖北武汉 430072
  2. 武汉船舶通信研究所，湖北武汉 430070
  3. 武汉技师学院，湖北武汉 430051
- 作者简介：
  
  [ "王源（1993–），女，武汉大学电子信息学院硕士生，主要研究方向为大数据分析、大数据平台开发和应用等。" ]
  [ "江昊（1976–），男，武汉大学电子信息学院教授、博士生导师，主要研究方向为无线网络、移动互联网、大数据分析与挖掘、空间综合信息网络等。" ]
  [ "吴明（1972–），男，现就职于武汉船舶通信研究所，主要研究方向为通信与信息系统。" ]
  [ "姚冬桂（1967–），女，武汉技师学院高级讲师，主要研究方向为大数据开发与应用。" ]
  [ "张毅（1994–），男，武汉大学电子信息学院硕士生，主要研究方向为大数据分析、机器学习算法研究和统计方法等。" ]
  [ "羿舒文（1992–），男，武汉大学电子信息学院博士生，主要研究方向为数据挖掘、移动无线网络等。" ]
  [ "汪海（1992–），男，武汉大学电子信息学院博士生，主要研究方向为无线网络、软件定义网络、数据分析与挖掘。" ]
  [ "吴静（1981–），女，武汉大学电子信息学院副教授，主要研究方向为网络管理和性能优化、网络路由技术、分布式计算技术等。" ]
- 基金信息：
  
  国家重点研发计划基金资助项目;National Key Research and Development Project(2017YFC0503801);中国电子科技集团公司第54研究所高校合作基金资助项目;Higher Education Cooperation Foundation of the 54th Research Institute of China Electronics Technology Group Corporation;中央高校自主科研项目;Autonomous Science Research Project in Central Affiliated University(2042014gf020);中央高校自主科研项目;Autonomous Science Research Project in Central Affiliated University(2042014kf0256);湖北省自然科学基金面上项目;Natural Science Foundation of Hubei Province of China(2014CFB716);湖北省自然科学基金面上项目;Natural Science Foundation of Hubei Province of China(2011CDA042);国家自然科学基金资助项目;The National Natural Science Foundation of China(61371126)
- DOI：10.11959/j.issn.1000-0801.2018136
  中图分类号： TP393
- 网络出版日期：2018-05，
  
  纸质出版日期：2018-05-20
- 稿件说明：
移动端阅览
王源, 江昊, 吴明, 等. 基于用户移动网络接入位置的高效分布式相似矩阵计算方法[J]. 电信科学, 2018,34(5):26-38.

Yuan WANG, Hao JIANG, Ming WU, et al. A distributed high efficiency similarity matrix computation method based on users’ mobile network access location[J]. Telecommunications science, 2018, 34(5): 26-38.
王源, 江昊, 吴明, 等. 基于用户移动网络接入位置的高效分布式相似矩阵计算方法[J]. 电信科学, 2018,34(5):26-38. DOI： 10.11959/j.issn.1000-0801.2018136.

Yuan WANG, Hao JIANG, Ming WU, et al. A distributed high efficiency similarity matrix computation method based on users’ mobile network access location[J]. Telecommunications science, 2018, 34(5): 26-38. DOI： 10.11959/j.issn.1000-0801.2018136.

摘要

研究了一种基于用户移动网络接入位置的高效分布式相似矩阵计算方法，利用Hadoop生态系统中的MapReduce 计算框架，依据地理位置信息对用户进行划分并进行相似度计算。实验结果表明，该方法在计算效率上相较于现有相似矩阵计算方法最大提升了近25倍。将相似矩阵计算结果应用于用户社区发现，该方法与现有相似矩阵计算方法得到的相似矩阵有几乎一致的社区发现结果，一致率高达99.9%。

Abstract

A distributed similarity matrix computation method based on users’ mobile network access location was proposed.MapReduce computing framework in the Hadoop ecosystem was used

and users’ mobile network access location information was considered as prior knowledge

then users were divided based on geographic location information and the similarity calculation was performed.The experimental results show that the computational efficiency of the proposed method is increased by nearly 25 times compared with the existing similarity matrix computation method.As for the outcome of the community detection

the proposed method get nearly the same results as the existing similar matrix computation method

the agreement rate is nearly 99.9%.

关键词

Keywords

references

SCRIPPS J , TREFFTZ C . Parallelizing an algorithm to find communities using the Jaccard metric [C ] // IEEE International Conference on Electro/Information Technology,June 8-12,2015,London,UK . Piscataway:IEEE Press , 2015 .

HURLEY N , DURIAKOVA E . Reformulations of the map equation for community finding and blockmodelling [C ] // IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining,August 25-28,2015,Paris,France . Piscataway:IEEE Press , 2015 .

RESTREPO A , SOLANO A , SCRIPPS J , et al . High-performance implementations of a clustering algorithm for finding network communities [C ] // IEEE International Conference on Electro/Information Technology,May 6-8,2012,Indianapolis,USA . Piscataway:IEEE Press , 2012 .

GUNTHER J H , HOFFMAN K H . Numerische mathematik [M ] . Berlin : SpringerPress , 1991 .

STEWART G W . Jampack:a Java package for matrix computations [J ] . 2017 .

JOE H , CLEVE M , PETER W . JAMA:a Java matrix package [EB ] . 2017 .

NGUYEN D K , LAVALLEE I , BUI M , et al . A general scalable parallelizing of strassen’s algorithm for matrix multiplication on distributed memory computers [C ] // ACIS International Conference on Computer and Information Science,July 14-16,2005,Washington,DC,USA . Piscataway:IEEE Press , 2005 .

LIN C , HUANG Z H , YANG F , et al . Identify content quality in online social networks [J ] . IET Communications , 2012 , 6 ( 12 ): 1618 - 1624 .

孙远帅 , 陈垚 , 官新均 , 等 . 基于Hadoop的大矩阵乘法处理方法 [J ] . 计算机应用 , 2013 , 33 ( 12 ): 3339 - 3344 .

SUN Y S , CHEN Y , GUAN X J , et al . Approach of large matrix multiplication based on Hadoop [J ] . Journal of Computer Applications , 2013 , 33 ( 12 ): 3339 - 3344 .

REZA M , SINHA A , NAG R , et al . CUDA-enabled Hadoop cluster for sparse matrix vector multiplication [C ] // IEEE International Conference on Recent Trends in Information Systems,July 9-11,2015,Kolkata,India . Piscataway:IEEE Press , 2015 .

MALYSIAK D , KOPINSKI T . A generic and adaptive approach for workload distribution in multi-tier cluster systems with an application to distributed matrix multiplication [C ] // IEEE International Symposium on Computational Intelligence and Informatics,November 19-21,2015,Budapest,Hungary . Piscataway:IEEE Press , 2015 .

GIZA-BELCIUG F , PENTIUC S G . Parallelization of similarity matrix calculus in ontology mapping systems [C ] // Roedunet International Conference-Networking in Education and Research,September 24-26,2015,Craiova,Romania . Piscataway:IEEE Press , 2015 .

ZHANG R , WANG Y . An enhanced agglomerative fuzzy k-means clustering method with MapReduce implementation on Hadoop platform [C ] // International Conference on Progress in Informatics and Computing,May 16-18,2014,Shanghai,China . Piscataway:IEEE Press , 2014 .

MANN K S , KAUR N . Cloud-deployable health data mining using secured framework for clinical decision support system [C ] // International Conference and Workshop on Computing and Communication,October 15-17,2015,Vancouver,BC,Canada . Piscataway:IEEE Press , 2015 .

SHAHRIVARI S , JALILI S . Single-pass and linear-time k-means clustering based on MapReduce [J ] . Information Systems , 2016 ( 60 ): 1 - 12 .

LU S , TONG W , CHEN Z . Implementation of the KNN algorithm based on Hadoop [C ] // International Conference on Smart and Sustainable City and Big Data,July 26-27,2015,Shanghai,China . Birmingham:IET Press , 2015 .

SONG G , ROCHAS J , BEZE L , et al . K nearest neighbour joins for big data on MapReduce:a theoretical and experimental analysis [J ] . IEEE Transactions on Knowledge ＆ Data Engineering , 2016 , 28 ( 9 ): 2376 - 2392 .

MIEGHEM P V , BLENN N , DOERR C . Lognormal distribution in the digg online social network [J ] . European Physical Journal B , 2011 , 83 ( 2 ): 251 - 261 .

MAHANTI A , CARLSSON N , MAHANTI A , et al . A tale of the tails:power-laws in internet measurements [J ] . IEEE Network , 2013 , 27 ( 1 ): 59 - 64 .

ZHOU C , JIANG H , CHEN Y , et al . TCB:a feature transformation method based central behavior for user interest prediction on mobile big data [J ] . International Journal of Distributed Sensor Networks , 2016 , 12 ( 9 ).

WU L , JIANG H , ZHENG H , et al . Long tail and small world characteristic of mobile internet traffic dynamics [C ] // IEEE International Conference on Systems,Man and Cybernetics,October 5-8,2014,San Diego,CA,USA . Piscataway:IEEE Press , 2014 .

WU L , LI Y , ZHOU C , et al . Statistic analysis of data access behavior in the mobile internet [C ] // IEEE/CIC International Conference on Communications in China,August 12-14,2013,Xi’an,China . Piscataway:IEEE Press , 2013 .

ZHANG N , ZHENG G , CHEN H , et al . HBaseSpatial:a scalable spatial data storage based on HBase [C ] // IEEE International Conference on Trust,Security and Privacy in Computing and Communications,September 24-26,2014,Beijing,China . Piscataway:IEEE Press , 2014 .

王荣 . 基于三元组表表示的稀疏矩阵的快速转置算法及其改进 [J ] . 现代电子技术 , 2008 , 31 ( 22 ): 78 - 79 .

WANG R . Improvement on fast transposition algorithm to sparse matrix expressed by triple list [J ] . Modern Electronics Technique , 2008 , 31 ( 22 ): 78 - 79 .

AFSHANG M , DHILLON H S , CHONG P H J . Fundamentals of cluster-centric content placement in cache-enabled device-to-device networks [J ] . IEEE Transactions on Communications , 2015 , 64 ( 6 ): 2511 - 2526 .

ZHOU B , CUI Y , TAO M . Stochastic content-centric multicast scheduling for cache-enabled heterogeneous cellular networks [J ] . IEEE Transactions on Wireless Communications , 2016 , 15 ( 9 ): 6284 - 6297 .

浏览量

490

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于MapReduce的连续SkyIine查询

电网数据存储技术选型的研究

运营商DNS海量数据分析技术优化和应用

基于PaaS技术的大数据云化平台实践

基于Hadoop的电信大数据采集方案研究与实现