Fellowship - Research on neural architecture search for automated designing the architectures of deep neural network

Job Reference: 000081

Location: Japan

Closing Date: 06/12/2019

Job Posted Date: 30/08/2019

Salary: C10M JPY

Employment Type: Toshiba Fellowship

Business Type: Research & Development

  which complement the incomplete MBD, and achieve both the shortest design turn-around-time (TAT) and the best system performance. Also in the system operations, the aging performance deteriorations and their variations are difficult to predict in the design process are detected by IoT and AI. The results are not only reported to the system operator but also act as feedback to the next system design, and the system reliability is further improved. This research is in the cross research domain, and the chances that the applied researcher is familiar with all the related layers are slim. The Fellowship researcher will pursue research based-on his/her wide, deep knowledge, rich ideas and systematic thinking regardless of one’s major. This research area is likely to be one of the most important ones for the next generation of Toshiba with its wide variety of power electronics businesses. Moreover, Toshiba is the best environment for such research activity considering the existence of all the related layers of power electronics technologies and applications.(TM).Research on neural architecture search for automated designing the architectures of deep neural networkResearch on neural architecture search for automated designing the architectures of deep neural network


In recent years, the research of deep learning has shown remarkable results in a wide range of computer vision tasks and novel network architectures with high recognition performance have been proposed. However, due to the demand for increased computational power and memory, it has become difficult to embed them into edge devices and introduce them to manufacturing fields that handle big data. TOSHIBA has been developing research on AI technologies toward the growth of our enterprise business, such as productivity improvement in our factories [1] and automated driving [2]. In order to further accelerate the practical applications of basic research into innovative next-generation AI technology, TOSHIBA and RIKEN established RIKEN AIP – TOSHIBA collaboration center in April 2017 [1]. Through the research of network architecture optimization, TOSHIBA and RIKEN discovered that the group sparsity of weighted parameters occur under usual conditions of deep learning, and theoretically elucidated the principle [4][5]. Moreover, we developed the method of reducing a generalization error using gradient noise convolution for distributed learning of large-scale data [6]. Through these technologies, TOSHIBA has achieved the top-level performance in the world on both an open dataset and practical dataset.

Description on Research

In this fellowship, we are looking for a researcher on neural architecture search technology to design DNN models automatically in edge-devices with limited computation resources. For example, we aim to build an automated design logic of DNN models which can achieve high performance based on a given target hardware constraint. Additionally, we will give candidates  the opportunity to work on their proposed research theme such as a design of loss functions, optimization methods (Bayesian optimization or reinforcement learning), and representation learning based not only on a graph theory but also a method using ordinary differentiable equation which has attracted attention in recent years. This fellowship will be conducted as one theme of the RIKEN AIP-TOSHIBA collaboration center.
The goal of this research is to build the basics of an innovative technology for TOSHIBA in the field of logistics, infrastructure and automated driving.

Required Knowledge and Skills

Candidates are required to have a profound knowledge of at least one of the following fields: pattern recognition, machine learning, deep learning, reinforcement learning and robotics. They should have software implementation skills such as C, C++, C#, MATLAB(R) or python(TM).

Related Papers

[1] Y. Tao, et al. “RDEC: Integrating Regularization into Deep Embedded Clustering for Imbalanced Datasets,” ACML 2018, https://arxiv.org/abs/1812.02293.
[2] Rudra P K Poudel, et al., “ContextNet: Exploring Context and Detail for Semantic Segmentation in Real-time,” 29th British Machine Vision Conference (BMVC2018), 2018.
[3] https://www.toshiba.co.jp/about/press/2017_03/pr1001.htm
[4] A. Yaguchi, et al. "Adam Induces Implicit Weight Sparsity in Rectifier Neural Networks," ICMLA 2018, https://arxiv.org/abs/1812.08119.
[5] https://www.toshiba.co.jp/about/press/2018_12/pr1702.htm
[6] K. Haruki, et al. “Gradient Noise Convolution (GNC): Smoothing Loss Function with Distributed Large-Batch SGD,” under review, https://arxiv.org/abs/1906.10822.