神经结构搜索进展概述

何明捷; 张杰; 山世光

doi:10.11959/j.issn.1000-0801.2019097

您当前的位置：

首页 >

文章列表页 >

神经结构搜索进展概述

专题：智能通信技术及应用 | 更新时间：2024-06-05

- 神经结构搜索进展概述
- A survey of neural architecture search
- 电信科学 2019年35卷第5期页码：43-50
- 作者机构：
  
  1. 中国科学院计算技术研究所中国科学院智能信息处理重点实验室，北京 100190
  2. 中国科学院大学，北京 100049
- 作者简介：
  
  [ "何明捷（1990- ），男，中国科学院计算技术研究所工程师，中国科学院大学博士生，主要研究方向为计算机视觉与机器学习。" ]
  [ "张杰（1988- ），男，中国科学院计算技术研究所助理研究员，主要研究方向为计算机视觉与机器学习。" ]
  [ "山世光（1975- ），男，中国科学院计算技术研究所研究员、博士生导师，中国科学院智能信息处理重点实验室常务副主任。国家自然科学基金委员会“优秀青年科学基金项目”入选者，第三批国家“万人计划”入选者，科技部“创新人才推进计划”中青年科技创新领军人才，CCF 青年科学家奖获得者。主要研究方向为计算机视觉和机器学习。" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61806188)
- DOI：10.11959/j.issn.1000-0801.2019097
  中图分类号： TP183
- 网络出版日期：2019-05，
  
  纸质出版日期：2019-05-20
- 稿件说明：
移动端阅览
何明捷, 张杰, 山世光. 神经结构搜索进展概述[J]. 电信科学, 2019,35(5):43-50.

Mingjie HE, Jie ZHANG, Shiguang SHAN. A survey of neural architecture search[J]. Telecommunications science, 2019, 35(5): 43-50.
何明捷, 张杰, 山世光. 神经结构搜索进展概述[J]. 电信科学, 2019,35(5):43-50. DOI： 10.11959/j.issn.1000-0801.2019097.

Mingjie HE, Jie ZHANG, Shiguang SHAN. A survey of neural architecture search[J]. Telecommunications science, 2019, 35(5): 43-50. DOI： 10.11959/j.issn.1000-0801.2019097.

摘要

近年来，深度学习技术在大量的计算视觉任务上取得了巨大的成功，深度神经结构是一个决定性能的关键要素，全自动的神经结构搜索方法的研究近年来受到了越来越多的关注。全自动的神经结构搜索方法是指针对特定的任务，通过算法自动地学习出适用的深度神经结构。各类神经结构搜索方法在探索高性能、高效率的神经结构方面已经展示出了巨大的潜力。从性能评估方法、搜索空间、结构搜索策略 3 个维度对神经结构搜索方法进行了分类概述：重点介绍了4种降低计算开销的性能评估方法，2类典型的神经结构搜索空间以及基于离散空间和基于连续空间的2种搜索策略。基于连续空间的NAS算法正逐渐成为NAS算法的重要发展方向。

Abstract

Recently

deep learning has achieved impressive success on various computer vision tasks.The neural architecture is usually a key factor which directly determines the performance of the deep learning algorithm.The automated neural architecture search methods have attracted more and more attentions in recent years.The neural architecture search is the automated process of seeking the optimal neural architecture for specific tasks.Currently

the neural architecture search methods have shown great potential in exploring high-performance and high-efficiency neural architectures.In this paper

a survey in this research field and categorize existing methods based on their performance estimation methods

search spaces and architecture search strategies were presented.Specifically

there were four performance estimation methods for computation cost reduction

two typical neural architecture search spaces and two types of search strategies based on discrete and continuous spaces respectively.Neural architecture search methods based on continuous space are becoming the trend of researches on neural architecture search.

关键词

Keywords

references

DENG J , DONG W , SOCHER R , et al . ImageNet:a large-scale hierarchical image database [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 20-25,2009,Miami,Florida,USA . Piscataway:IEEE Press , 2009 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . Imagenet classification with deep convolutional neural networks [C ] // Advances in Neural Information Processing Systems (NIPS),Dec 3-8,2012,Lake Tahoe,Nevada,USA . New York:Curran Associates Inc , 2012 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [J ] . Computer Science , 2004 .

SZEGEDY C , LIU W , JIA Y , et al . Going deeper with convolutions [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 7-12,2015,Boston,MA,USA . Piscataway:IEEE Press , 2015 .

HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 27-30,2016,Las Vegas,NV,USA . Piscataway:IEEE Press , 2016 .

HE K , ZHANG X , REN S , et al . Identity mappings in deep residual networks [C ] // European Conference on Computer Vision (ECCV),Oct 11-14,2016,Amsterdam,Netherlands . Cham:Springer , 2016 .

HUANG G , LIU Z , VAN DER MAATEN L , et al . Densely connected convolutional networks [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),July 21-26,2017,Honolulu,HI,USA . Piscataway:IEEE Press , 2017 .

XIE S , GIRSHICK R , DOLL R P , et al . Aggregated residual transformations for deep neural networks [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),July 21-26,2017,Honolulu,HI,USA . Piscataway:IEEE Press , 2017 .

CHEN Y , LI J , XIAO H , et al . Dual path networks [C ] // Advances in Neural Information Processing Systems (NIPS),Dec 4-9,2017,Long Beach,CA,USA . New York:Curran Associates Inc , 2017 .

HU J , SHEN L , SUN G . Squeeze-and-excitation networks [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 18-22,2018,Salt Lake,UT,USA . Piscataway:IEEE Press , 2018 .

ELSKEN T , METZEN J H , HUTTER F . Neural architecture search:A survey [J ] . arXiv:1808.05377 , 2018 .

LIU C , ZOPH B , NEUMANN M , et al . Progressive neural architecture search [C ] // European Conference on Computer Vision (ECCV),Sep 8-14,2018,Munich,Germany . Cham:Springer , 2018 .

LUO R , TIAN F , QIN T , et al . Neural architecture optimization [C ] // Advances in Neural Information Processing Systems (NIPS),Dec 3-8,2018,Montreal,Canada . New York:Curran Associates,Inc , 2018 .

PHAM H , GUAN M Y , ZOPH B , et al . Efficient neural architecture search via parameter sharing [C ] // International Conference on Machine Learning (ICML),July 10-15,2018,Stockholmsmässan,Stockholm,Sweden.[S.l.:sn] . 2018 .

ZHONG Z , YAN J , WU W , et al . Practical block-wise neural network architecture generation [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 18-22,2018,Salt Lake City,UT,USA . Piscataway:IEEE Press , 2018 .

ZOPH B , VASUDEVAN V , SHLENS J , et al . Learning transferable architectures for scalable image recognition [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 18-22,2018,Salt Lake City,UT,USA . Piscataway:IEEE Press , 2018 .

TAN M , CHEN B , PANG R , et al . Mnasnet:platform-aware neural architecture search for mobile [J ] . arXiv:1807.11626 , 2018 .

REAL E , AGGARWAL A , HUANG Y , et al . Regularized evolution for image classifier architecture search [C ] // Thirty-Third AAAI Conference on Artificial Intelligence,Jan 27-Feb 1,2019,Honolulu,Hawaii,USA.[S.l.:sn] . 2019 .

CAI H , YANG J , ZHANG W , et al . Path-level network transformation for efficient architecture search [C ] // International Conference on Machine Learning (ICML),July 10-15,2018,Stockholmsmässan,Stockholm,Sweden.[S.l.:sn] . 2018 .

CHRABASZCZ P , LOSHCHILOV I , HUTTER F . A downsampled variant of imagenet as an alternative to the cifar datasets [J ] . arXiv:1707.08819 , 2017 .

RAWAL A , MIIKKULAINEN R . From nodes to networks:evolving recurrent neural networks [J ] . arXiv:1803.04439 , 2018 .

KRIZHEVSKY A , HINTON G . Learning multiple layers of features from tiny images [R ] . 2009 .

SANDLER M , HOWARD A , ZHU M , et al . Mobilenetv2:inverted residuals and linear bottlenecks [C ] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 18-22,2018,Salt Lake City,UT,USA . Piscataway:IEEE Press , 2018 .

ZOPH B , LE Q V . Neural architecture search with reinforcement learning [C ] // International Conference on Learning Representations (ICLR),April 24-26,2017,Toulon,France.[S.l.:sn] . 2017 .

BAKER B , GUPTA O , NAIK N , et al . Designing neural network architectures using reinforcement learning [C ] // International Conference on Learning Representations (ICLR),April 24-26,2017,Toulon,France.[S.l.:sn] . 2017 .

CAI H , CHEN T , ZHANG W , et al . Reinforcement learning for architecture search by network transformation [J ] . arXiv:1707.04873 , 2017 .

LIU H , SIMONYAN K , VINYALS O , et al . Hierarchical representations for efficient architecture search [C ] // International Conference on Learning Representations (ICLR),April 30-May 3,2018,Vancouver,BC,Canada.[S.l.:s . n] , 2018 .

REAL E , MOORE S , SELLE A , et al . Large-scale evolution of image classifiers [C ] // International Conference on Machine Learning (ICML),Aug 6-11,2017,Sydney,NSW,Australia.[S.l.:sn] . 2017 .

XIE L , YUILLE A . Genetic CNN [C ] // IEEE International Conference on Computer Vision (ICCV),Oct 22-29,2017,Venice,Italy . Piscataway:IEEE Press , 2017 .

浏览量

253

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

GMTBLC：基于深度学习的双模态网络流量分类

基于OS-MBRL的网络切片资源动态分配方法研究

基于改进YOLOv5的天线下倾角识别方法研究

基于时序深度残差收缩网络的混叠信号调制识别方法

日志信息驱动的计算机网络节点故障预测研究