Implementation of a big data anonymization system based on Spark

Chaoyi BIAN; Shaomin ZHU; Tao ZHOU

doi:10.11959/j.issn.1000-0801.2018133

您当前的位置：

首页 >

文章列表页 >

Implementation of a big data anonymization system based on Spark

Power informatization column | 更新时间：2024-06-05

- Implementation of a big data anonymization system based on Spark
- Telecommunications Science Vol. 34, Issue 4, Pages: 156-161(2018)
- 作者机构：
  
  1. 北京启明星辰信息安全技术有限公司，北京 100193
  2. 北京邮电大学，北京 100876
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2018133
  CLC： TP391
- Published Online：2018-04，
  
  Published：20 April 2018
- 稿件说明：
移动端阅览
Chaoyi BIAN, Shaomin ZHU, Tao ZHOU. Implementation of a big data anonymization system based on Spark[J]. Telecommunications science, 2018, 34(4): 156-161.
DOI：

Chaoyi BIAN, Shaomin ZHU, Tao ZHOU. Implementation of a big data anonymization system based on Spark[J]. Telecommunications science, 2018, 34(4): 156-161. DOI： 10.11959/j.issn.1000-0801.2018133.

摘要

分组匿名化框架是一类经典的数据匿名化技术，它通过构造匿名记录的组，使得同一组内的不同数据无法被识别区分，从而达到隐私防护的效果。电力行业大数据分析涉及电力企业核心数据、用户隐私数据，其数据敏感度更强，传统的数据匿名化系统已经无法满足电力行业大数据业务应用和安全防护的需要。基于此，设计并实现了一种基于Spark的新型大数据匿名化系统，提供对Hadoop平台上多种数据格式的支持，并有效提高对大数据的匿名化处理效率。

Abstract

Group based anonymization is a classical data anonymization framework

which achieves the effect of privacy protection by constructing groups of anonymized data records ensuring that records in the same group cannot be distinguished with each other.The electric power industry big data analysis involves the core data of the power enterprises and the user privacy data

the data sensitivity is stronger

traditional data anonymization systems are unable to meet the needs of big data business applications and safety protection of electric power industry.A new big data anonymization system based on Spark was designed and implemented

which could provide the support for multiple data formats stored on Hadoop and substantially improve the processing efficiency for big data.

关键词

Keywords

references

Apache Software Foundation . Apache Hadoop EB ] . 2011 .

Apache Software Foundation . Apache Spark EB ] . 2014 .

SWEENEY L . K-anonymity:a model for protecting privacy [J ] . International Journal of Uncertainty,Fuzziness and Knowledge-Based Systems , 2002 , 10 ( 5 ): 557 - 570 .

MACHANAVAJJHALA A , KIFER D , GEHRKE J , et al . L-diversity:privacy beyondk-anonymity [J ] . ACM Transactions on Knowledge Discovery from Data (TKDD) , 2007 , 1 ( 1 ): 3 .

FUNG B , WANG K , CHEN R , et al . Privacy-preserving data publishing:a survey of recent developments [J ] . ACM Computing Surveys (Csur) , 2010 , 42 ( 4 ): 14 .

PRASSER F , KOHLMAYER F . Putting statistical disclosure control into practice:the ARX data anonymization tool [M ] . Berlin : Springer International Publishing , 2015 .

JADHAV R H , INGLE R B . A survey on data anonymization approaches using MapReduce framework on cloud [J ] . Digital Image Processing , 2015 , 7 ( 2 ): 48 - 49 .

LEFEVRE K , DEWITT D J , RAMAKRISHNAN R . Mondrian multidimensional k-anonymity [C ] // International Conference on Data Engineering,April 3-7,2006,Atlanta,GA,USA . Piscataway:IEEE Press , 2006 :25.

QIYUAN G . A survey on data anonymization approaches using MapReduce framework on cloud EB ] . 2015 .

Views

895

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A network traffic classification method based on random forest and improved convolutional neural network

Network architecture with intrinsic security

Survey on network security based on blockchain

An improved CLTree algorithm

Wireless VPN network structure and safety protection in the north and south call center of SGCC

Related Author

Xiaoya GAN

Yaguan QIAN

Bensheng YUN

Weiyu JIANG

Bingyang LIU

Chuang WANG

Liang XIAO

Dongjin XU

Related Institution

Shanghai Research Institute of China Telecom Co.，Ltd.

Anhui Branch of China Telecom Co.， Ltd.

⁰