A survey of neural architecture search

Mingjie HE; Jie ZHANG; Shiguang SHAN

doi:10.11959/j.issn.1000-0801.2019097

您当前的位置：

首页 >

文章列表页 >

A survey of neural architecture search

Topics:intelligent communication technologies and applications | 更新时间：2024-06-05

- A survey of neural architecture search
- Telecommunications Science Vol. 35, Issue 5, Pages: 43-50(2019)
- 作者机构：
  
  1. 中国科学院计算技术研究所中国科学院智能信息处理重点实验室，北京 100190
  2. 中国科学院大学，北京 100049
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-0801.2019097
  CLC： TP183
- Published Online：2019-05，
  
  Published：20 May 2019
- 稿件说明：
移动端阅览
Mingjie HE, Jie ZHANG, Shiguang SHAN. A survey of neural architecture search[J]. Telecommunications science, 2019, 35(5): 43-50.
DOI：

Mingjie HE, Jie ZHANG, Shiguang SHAN. A survey of neural architecture search[J]. Telecommunications science, 2019, 35(5): 43-50. DOI： 10.11959/j.issn.1000-0801.2019097.

摘要

近年来，深度学习技术在大量的计算视觉任务上取得了巨大的成功，深度神经结构是一个决定性能的关键要素，全自动的神经结构搜索方法的研究近年来受到了越来越多的关注。全自动的神经结构搜索方法是指针对特定的任务，通过算法自动地学习出适用的深度神经结构。各类神经结构搜索方法在探索高性能、高效率的神经结构方面已经展示出了巨大的潜力。从性能评估方法、搜索空间、结构搜索策略 3 个维度对神经结构搜索方法进行了分类概述：重点介绍了4种降低计算开销的性能评估方法，2类典型的神经结构搜索空间以及基于离散空间和基于连续空间的2种搜索策略。基于连续空间的NAS算法正逐渐成为NAS算法的重要发展方向。

Abstract

Recently

deep learning has achieved impressive success on various computer vision tasks.The neural architecture is usually a key factor which directly determines the performance of the deep learning algorithm.The automated neural architecture search methods have attracted more and more attentions in recent years.The neural architecture search is the automated process of seeking the optimal neural architecture for specific tasks.Currently

the neural architecture search methods have shown great potential in exploring high-performance and high-efficiency neural architectures.In this paper

a survey in this research field and categorize existing methods based on their performance estimation methods

search spaces and architecture search strategies were presented.Specifically

there were four performance estimation methods for computation cost reduction

two typical neural architecture search spaces and two types of search strategies based on discrete and continuous spaces respectively.Neural architecture search methods based on continuous space are becoming the trend of researches on neural architecture search.

关键词

Keywords

references

DENG J , DONG W , SOCHER R , et al . ImageNet:a large-scale hierarchical image database [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 20-25,2009,Miami,Florida,USA . Piscataway:IEEE Press , 2009 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . Imagenet classification with deep convolutional neural networks [C ] // Advances in Neural Information Processing Systems (NIPS),Dec 3-8,2012,Lake Tahoe,Nevada,USA . New York:Curran Associates Inc , 2012 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [J ] . Computer Science , 2004 .

SZEGEDY C , LIU W , JIA Y , et al . Going deeper with convolutions [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 7-12,2015,Boston,MA,USA . Piscataway:IEEE Press , 2015 .

HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 27-30,2016,Las Vegas,NV,USA . Piscataway:IEEE Press , 2016 .

HE K , ZHANG X , REN S , et al . Identity mappings in deep residual networks [C ] // European Conference on Computer Vision (ECCV),Oct 11-14,2016,Amsterdam,Netherlands . Cham:Springer , 2016 .

HUANG G , LIU Z , VAN DER MAATEN L , et al . Densely connected convolutional networks [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),July 21-26,2017,Honolulu,HI,USA . Piscataway:IEEE Press , 2017 .

XIE S , GIRSHICK R , DOLL R P , et al . Aggregated residual transformations for deep neural networks [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),July 21-26,2017,Honolulu,HI,USA . Piscataway:IEEE Press , 2017 .

CHEN Y , LI J , XIAO H , et al . Dual path networks [C ] // Advances in Neural Information Processing Systems (NIPS),Dec 4-9,2017,Long Beach,CA,USA . New York:Curran Associates Inc , 2017 .

HU J , SHEN L , SUN G . Squeeze-and-excitation networks [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 18-22,2018,Salt Lake,UT,USA . Piscataway:IEEE Press , 2018 .

ELSKEN T , METZEN J H , HUTTER F . Neural architecture search:A survey [J ] . arXiv:1808.05377 , 2018 .

LIU C , ZOPH B , NEUMANN M , et al . Progressive neural architecture search [C ] // European Conference on Computer Vision (ECCV),Sep 8-14,2018,Munich,Germany . Cham:Springer , 2018 .

LUO R , TIAN F , QIN T , et al . Neural architecture optimization [C ] // Advances in Neural Information Processing Systems (NIPS),Dec 3-8,2018,Montreal,Canada . New York:Curran Associates,Inc , 2018 .

PHAM H , GUAN M Y , ZOPH B , et al . Efficient neural architecture search via parameter sharing [C ] // International Conference on Machine Learning (ICML),July 10-15,2018,Stockholmsmässan,Stockholm,Sweden.[S.l.:sn] . 2018 .

ZHONG Z , YAN J , WU W , et al . Practical block-wise neural network architecture generation [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 18-22,2018,Salt Lake City,UT,USA . Piscataway:IEEE Press , 2018 .

ZOPH B , VASUDEVAN V , SHLENS J , et al . Learning transferable architectures for scalable image recognition [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 18-22,2018,Salt Lake City,UT,USA . Piscataway:IEEE Press , 2018 .

TAN M , CHEN B , PANG R , et al . Mnasnet:platform-aware neural architecture search for mobile [J ] . arXiv:1807.11626 , 2018 .

REAL E , AGGARWAL A , HUANG Y , et al . Regularized evolution for image classifier architecture search [C ] // Thirty-Third AAAI Conference on Artificial Intelligence,Jan 27-Feb 1,2019,Honolulu,Hawaii,USA.[S.l.:sn] . 2019 .

CAI H , YANG J , ZHANG W , et al . Path-level network transformation for efficient architecture search [C ] // International Conference on Machine Learning (ICML),July 10-15,2018,Stockholmsmässan,Stockholm,Sweden.[S.l.:sn] . 2018 .

CHRABASZCZ P , LOSHCHILOV I , HUTTER F . A downsampled variant of imagenet as an alternative to the cifar datasets [J ] . arXiv:1707.08819 , 2017 .

RAWAL A , MIIKKULAINEN R . From nodes to networks:evolving recurrent neural networks [J ] . arXiv:1803.04439 , 2018 .

KRIZHEVSKY A , HINTON G . Learning multiple layers of features from tiny images [R ] . 2009 .

SANDLER M , HOWARD A , ZHU M , et al . Mobilenetv2:inverted residuals and linear bottlenecks [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 18-22,2018,Salt Lake City,UT,USA . Piscataway:IEEE Press , 2018 .

ZOPH B , LE Q V . Neural architecture search with reinforcement learning [C ] // International Conference on Learning Representations (ICLR),April 24-26,2017,Toulon,France.[S.l.:sn] . 2017 .

BAKER B , GUPTA O , NAIK N , et al . Designing neural network architectures using reinforcement learning [C ] // International Conference on Learning Representations (ICLR),April 24-26,2017,Toulon,France.[S.l.:sn] . 2017 .

CAI H , CHEN T , ZHANG W , et al . Reinforcement learning for architecture search by network transformation [J ] . arXiv:1707.04873 , 2017 .

LIU H , SIMONYAN K , VINYALS O , et al . Hierarchical representations for efficient architecture search [C ] // International Conference on Learning Representations (ICLR),April 30-May 3,2018,Vancouver,BC,Canada.[S.l.:s . n] , 2018 .

REAL E , MOORE S , SELLE A , et al . Large-scale evolution of image classifiers [C ] // International Conference on Machine Learning (ICML),Aug 6-11,2017,Sydney,NSW,Australia.[S.l.:sn] . 2017 .

XIE L , YUILLE A . Genetic CNN [C ] // IEEE International Conference on Computer Vision (ICCV),Oct 22-29,2017,Venice,Italy . Piscataway:IEEE Press , 2017 .

Views

253

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

GMTBLC: a deep learning-based bi-modal network traffic classification method

Research on dynamic allocation of network slicing resources based on OS-MBRL

Research on antenna dip angle recognition method based on improved YOLOv5

Mixed signal modulation recognition method based on temporal depth residual shrinkage network

Research on fault prediction of computer network nodes driven by log information

Related Author

WEI Debin

JIANG Qinlong

WEN Jinglong

WANG Xinrui

YAN Jiahui

ZHONG Weixuan

DONG Ligang

JIANG Xian

Related Institution

School of Information Engineering, Dalian University

UT Starcom Telecom Co., Ltd.

Zhejiang Gongshang University

China Mobile Group Design Institute Co., Ltd.

China Mobile Group Co., Ltd.

⁰