Applications of a big data platform based on PaaS

Haiyong XU; Yan HUANG

doi:10.11959/j.issn.1000-0801.2018043

您当前的位置：

首页 >

文章列表页 >

Applications of a big data platform based on PaaS

Operator big data column | 更新时间：2024-06-05

- Applications of a big data platform based on PaaS
- Telecommunications Science Vol. 34, Issue 1, Pages: 148-157(2018)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2018043
  CLC： TP311
- Published Online：2018-01，
  
  Published：20 January 2018
- 稿件说明：
移动端阅览
Haiyong XU, Yan HUANG. Applications of a big data platform based on PaaS[J]. Telecommunications science, 2018, 34(1): 148-157.
DOI：

Haiyong XU, Yan HUANG. Applications of a big data platform based on PaaS[J]. Telecommunications science, 2018, 34(1): 148-157. DOI： 10.11959/j.issn.1000-0801.2018043.

摘要

大数据平台作为数据存储、处理和服务的基础平台，需支撑内部和外部多种大数据应用的开发及运行。提出一种把PaaS（platform as a service，平台即服务）技术应用到大数据平台的方案，详细描述了大数据PaaS云化平台的架构体系以及分层调度、多租户管理、Hadoop任务全局调度等关键技术方案。实践证明，该平台可以显著提高大数据应用的开发效率，实现应用快速部署上线；同时，其良好的弹性伸缩能力可以有效地提高大数据应用系统资源的使用效率。

Abstract

As a platform of data storage

processing

and service

big data platform needs to support internal and external various big data applications.A solution applied PaaS technology to the big data platform was proposed.Details of the key technical solutions such as the architecture of PaaS cloud computing platform

hierarchical scheduling scheme

multi-tenant management and global scheduling of Hadoop tasks were described.It was proved that the platform could remarkably improve big data application development efficiency and help application deployment fast.In the meantime

good elastic scaling capability can effectively improve big data application system resource utilization.

关键词

Keywords

references

Gartner . Platform as a service(PaaS) [EB/OL ] .（2014-04-28）[2017-11-28 ] . https://www.gartner.com/it-glossary/platform-asa-service-paas/ https://www.gartner.com/it-glossary/platform-asa-service-paas/ .

齐磊 , 张海峰 , 张天骁 , 等 . 基于容器技术的 PaaS 云平台方案 [J ] . 电信科学 , 2017 , 33 ( 4 ): 177 - 182 .

QI L , ZHANG H F , ZHANGT X , et al . PaaS cloud platform based on container technology [J ] . Telecommunications Science , 2017 , 33 ( 4 ): 177 - 182 .

VERMA A , PEDROSA L , KORUPOLU M , et al . Large-scale cluster management at Google with Borg [C ] // 10th European Conference on Computer Systems,April 21-24,2015,Bordeaux,France . New York:ACM Press , 2015 :18.

张基 , 李大中 , 张呈宇 , 等 . DC/OS 关键技术与应用场景 [J ] . 电信科学 , 2016 , 32 ( 12 ): 13 - 19 .

ZHANG J , LI D Z , ZHANG C Y , et al . Key technologies and application scenario of DC/OS [J ] . Telecommunications Science , 2016 , 32 ( 12 ): 13 - 19 .

BERNSTEIN D . Containers and cloud:from LXC to Docker to Kubernetes [J ] . IEEE Cloud Computing , 2015 , 1 ( 3 ): 81 - 84 .

BALALAIE A , HEYDARNOORI A , JAMSHIDI P . Microservices Architecture enables DevOps:migration to a cloud-native architecture [J ] . IEEE Software , 2016 , 33 ( 3 ): 42 - 52 .

王峰 , 雷葆华 . Hadoop分布式文件系统的模型分析 [J ] . 电信科学 , 2010 , 26 ( 12 ): 95 - 99 .

WANG F , LEI B H . Modeling and analysis of Hadoop distributed file system [J ] . Telecommunications Science , 2010 , 26 ( 12 ): 95 - 99 .

KONAR M , . Apache Hadoop YARN:yet another resource negotiator [C ] // ACM Symposium on Cloud Computing,Oct 1-3,2013,Santa Clara,CA,USA . New York:ACM Press , 2013 :5.

LU X , LIANG F , WANG B , et al . DataMPI:Extending MPI to Hadoop-like big data computing [C ] // International Parallel and Distributed Processing Symposium,May 19-23,2014,Phoenix,Arizona,USA . Piscataway:IEEE Press , 2014 : 829 - 838 .

郑文武 , 李先绪 , 黄执勤 . 云计算中的并行计算技术分析 [J ] . 电信科学 , 2012 , 27 ( 12 ): 31 - 38 .

ZHENG W W , LI X X , HUANG Z Q . Technolical analysis of parallel computing in cloud computing [J ] . Telecommunications Science , 2012 , 27 ( 12 ): 31 - 38 .

李小宁 , 李磊 , 金连文 , 等 . 基于OpenStack构建私有云计算平台 [J ] . 电信科学 , 2012 , 27 ( 9 ): 1 - 8 .

LI X N , LI L , JIN L W , et al . Constructing a private cloud computing platform based on OpenStack [J ] . Telecommunications Science , 2012 , 27 ( 9 ): 1 - 8 .

Views

1150

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Construction of big data open platform for telecom operators

Research and implementation on acquisition scheme of telecom big data based on Hadoop

Practice and thinking on the transition of telecom operator analysis support system in big data era

Hadoop bottleneck detection algorithm based on information gain

An associated perception import method for big data

Related Author

Haifeng ZHANG

Zhao DONG

Juan LI

Lei QI

Huiqiong GU

Baoyou WANG

Jing QIAN

Shijin YUAN

Related Institution

School of Software Engineering,Tongji University

Shanghai Branch of China United Network Communication Co.,Ltd.

School of Computers,Guangdong University of Technology

Guangzhou Useease Information Technology Co.,Ltd.

Jiyuan Branch of China Telecom Co.，Ltd.

⁰