前瞻科技 ›› 2023, Vol. 2 ›› Issue (1): 78-89.DOI: 10.3981/j.issn.2097-0781.2023.01.006

• 综述与述评 • 上一篇    下一篇

鲁棒神经网络的训练方法研究进展与前景

梁震1(), 刘万伟2,(), 吴陶然3,4, 任德金3,4, 薛白3   

  1. 1.国防科技大学量子信息研究所兼高性能计算国家重点实验室,长沙 410073
    2.国防科技大学计算机学院,长沙 410073
    3.中国科学院软件研究所计算机科学国家重点实验室,北京 100190
    4.中国科学院大学计算机科学与技术学院,北京 100190
  • 收稿日期:2022-12-24 修回日期:2023-02-01 出版日期:2023-03-20 发布日期:2023-03-27
  • 通讯作者:
  • 作者简介:梁震,博士研究生。主要研究方向为AI可解释性与AI形式化验证等。电子信箱:liangzhen@nudt.edu.cn
    刘万伟,教授,中国计算机学会高级会员。主要研究方向为自动机理论、形式化方法、AI形式化验证等。在IEEE Transactions on Sustainable Energy、ACM Transactions on Quantum Computing、ICSE、ASE、CAV、TACAS、IJCAI等期刊/会议发表多篇论文。参与开发的验证工具在TACAS SV-comp比赛中多次获得第一名。电子信箱:wwliu@nudt.edu.cn
  • 基金资助:
    国家自然科学基金(61872371);国家自然科学基金(61836005);国家自然科学基金(62032024)

Advances and Prospects of Training Methods for Robust Neural Networks

LIANG Zhen1(), LIU Wanwei2,(), WU Taoran3,4, REN Dejin3,4, XUE Bai3   

  1. 1. Institute of Quantum Information & State Key Laboratory of High Performance Computing, National University of Defense Technology, Changsha 410073, China
    2. College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China
    3. State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China
    4. School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100190, China
  • Received:2022-12-24 Revised:2023-02-01 Online:2023-03-20 Published:2023-03-27
  • Contact:

摘要:

近年来,深度神经网络已经发展成为深度学习的重要计算模型,神经网络的鲁棒性对于其在安全攸关领域的部署至关重要。因此,如何训练鲁棒的神经网络是备受学术界和工业界关注的热点问题。文章介绍了目前主流的3类鲁棒神经网络的训练方法,即基于数据增强训练、基于对抗训练和利普希茨鲁棒性训练;并介绍了其各自方法的核心思想、代表性研究工作和适用范围。同时,将近年来的鲁棒神经网络训练方法的优缺点进行比较,对应到神经网络训练的要素上进行深入分析和对照,并对各类训练方法得到的神经网络的鲁棒性的评价指标进行了介绍和比较。最后,分析了目前鲁棒神经网络训练的难点和热点,展望了该领域可能的研究方向,并提出建议。

关键词: 深度神经网络, 鲁棒性, 神经网络训练

Abstract:

In recent years, deep neural networks have developed into important computing models for deep learning, whose robustness is essential for their deployment in safety-critical areas. Therefore, the way to train robust neural networks is a popular issue that has attracted the attention of academia and industry. In this paper, three mainstream classes of training methods for robust neural networks are introduced, i.e., the method based on data enhancement, that based on adversarial training, and the Lipschitz robust training method. Meanwhile, their core ideas, representative work, and the application scope are introduced. Then, the advantages and disadvantages of the training methods in recent years are compared, and in-depth analysis and comparison are carried out when they correspond to key elements in neural network training. The robustness evaluation metrics of neural networks obtained by these training methods are introduced and compared. Finally, hotspots and challenges of robust neural network training are analyzed, and the possible future directions and some suggestions are briefly summarized.

Key words: deep neural network, robustness, neural network training