Research and practice on working flow engine of big data platform for gaming service

Jie CHEN; Juan WU; Yang SU; Yong TANG

doi:10.3969/j.issn.1000-0801.2016.02.019

您当前的位置：

首页 >

文章列表页 >

Research and practice on working flow engine of big data platform for gaming service

Overview | 更新时间：2024-06-05

- Research and practice on working flow engine of big data platform for gaming service
- Telecommunication Science Vol. 32, Issue 2, Pages: 134-139(2016)
- 作者机构：
  
  1. 炫彩互动网络科技有限公司，江苏南京210029
  2. 中国电信股份有限公司广州研究院，广东广州 510630
- 作者简介：
- 基金信息：
- DOI：10.3969/j.issn.1000-0801.2016.02.019
  CLC： TP311
- Published Online：2016-02，
  
  Published：15 February 2016
- 稿件说明：
移动端阅览
Jie CHEN, Juan WU, Yang SU, et al. Research and practice on working flow engine of big data platform for gaming service[J]. Telecommunication science, 2016, 32(2): 134-139.
DOI：

Jie CHEN, Juan WU, Yang SU, et al. Research and practice on working flow engine of big data platform for gaming service[J]. Telecommunication science, 2016, 32(2): 134-139. DOI： 10.3969/j.issn.1000-0801.2016.02.019.

摘要

随着业务人员对数据分析的需求不断增加，基于开源Hadoop的游戏大数据平台运行着越来越多相互依赖的作业任务。为了更加高效地完成作业管理和调度，升级了Zeus构建工作流引擎，优化Zeus任务启动过程，实现了多线程启动任务，缩短了启动时间。通过扩展Zeus系统，使其支持Oracle存储过程。同时，利用管道技术，实现了异构数据源的无缝高效共享和传输。

Abstract

The big data platform for gaming service is based on Hadoop open source technology. With the demand growth of the service operation staff，there are more and more jobs which depend on each other running on the big data platform. A new working flow engine was developed on Zeus to manage and schedule jobs efficiently. In order to shorten the start time，Zeus task starting process was optimized and multi-threaded start task was achieved. Zeus was also improved to support the Oracle stored procedure. Pipeline technology was used to seamless and efficient sharing and transmission of heterogeneous data sources.

关键词

Keywords

references

苏洋，刘晓军，唐勇，等 . 游戏大数据平台研究与实践 [J ] . 电信科学， 2014 ， 30 （ 10 ）： 21 - 26 .

SU Y ， LIU X J ， TANG Y ， et al . Research and practice of big data platform of games [J ] . Telecommunications Science ， 2014 ， 30 （ 10 ）： 21 - 26 .

Apache Oozie workflow scheduler for Hadoop [EB/OL ] . [ 2015 - 05 - 30 ] . http://oozie.apache.org/ http://oozie.apache.org/ .

Open-source workflow manager [E B/OL ] . [ 2014 - 02 - 19 ] . http://azkaban.github.io/ http://azkaban.github.io/ .

Workflow engines for Hadoop，Joe Crobak [EB/OL ] . [ 2012 - 07 - 05 ] . http://www.crobak.org/2012/07/workflow-engines-for-hadoop/ http://www.crobak.org/2012/07/workflow-engines-for-hadoop/ .

淘宝Hadoop作业平台宙斯Zeus [EB/OL ] . [ 2015 - 10 - 16 ] . https://github.com/alibaba/zeus https://github.com/alibaba/zeus .

Taobao Hadoop worker platform Zeus [EB/OL ] . [ 2015 - 10 - 16 ] . https://github.com/alibaba/zeus https://github.com/alibaba/zeus .

MAPREDUCE-2364 [EB/OL ] . [ 2011 - 03 - 07 ] . https://issues.apache.org/jira/browse/MAPREDUCE-2364 https://issues.apache.org/jira/browse/MAPREDUCE-2364 .

WHITE T . Hadoop权威指南（第3版）[M ] . 曾大聘，周傲英，译. 北京：清华大学出版社， 2015 ： 125 - 130 .

WHITE T . Hadoop：The Definite Guide [M ] . Translated by ZENG D R ， ZHOU A Y . Beijing ： Tsinghua University Press ， 2015 ： 125 - 130 .

Views

595

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Open source technology of NFV

Application and development of CORD

Key technologies and application scenario of DC/OS

Standards and open source technology of NFV

Research on Technical Architecture on the SPICE of Open Source VDI

Related Author

Feng WANG

Huiling ZHAO

Wei ZHOU

Xiongyan TANG

Jiheng ZHANG

Dazhong LI

Chengyu ZHANG

Jinwu WEI

Related Institution

China Unicom Research Institute

China Unicom Group Information Division

⁰