Foundations and Trends® in Electronic Design Automation > Vol 13 > Issue 4

From CNN to DNN Hardware Accelerators: A Survey on Design, Exploration, Simulation, and Frameworks

By Leonardo Rezende Juracy, School of Technology, Pontifical Catholic University of Rio Grande do Sul – PUCRS, Brazil, leonardo.juracy@edu.pucrs.br | Rafael Garibotti, School of Technology, Pontifical Catholic University of Rio Grande do Sul – PUCRS, Brazil, rafael.garibotti@pucrs.br | Fernando Gehm Moraes, School of Technology, Pontifical Catholic University of Rio Grande do Sul – PUCRS, Brazil, fernando.moraes@pucrs.br

 
Suggested Citation
Leonardo Rezende Juracy, Rafael Garibotti and Fernando Gehm Moraes (2023), "From CNN to DNN Hardware Accelerators: A Survey on Design, Exploration, Simulation, and Frameworks", Foundations and Trends® in Electronic Design Automation: Vol. 13: No. 4, pp 270-344. http://dx.doi.org/10.1561/1000000060

Publication Date: 06 Mar 2023
© 2023 L. R. Juracy et al.
 
Subjects
System level design,  Circuit level design,  Logic design,  Physical design,  Reconfigurable systems
 

Free Preview:

Download extract

Share

Download article
In this article:
1. Introduction
2. Basic Concepts
3. CNN and DNN Hardware Accelerators
4. Hardware Design Space Exploration Frameworks and Simulators
5. Conclusion
Acknowledgements
References

Abstract

Over the past decade, a massive proliferation of machine learning algorithms has emerged, from applications for surveillance to self-driving cars. The turning point occurred with the arrival of Convolutional Neural Network (CNN) models and the incredible accuracy brought by Deep Neural Networks (DNNs) at the cost of high computational complexity. In this growing environment, graphic processing units (GPUs) have become the de facto reference platform for the training and inference phases of CNNs and DNNs due to their high processing parallelism and memory bandwidth. However, GPUs are power-hungry architectures. To enable the deployment of CNN and DNN applications on energy-constrained devices (e.g., IoT devices), industry and academic research have moved towards hardware accelerators. Following the evolution of neural networks (from CNNs to DNNs), this survey sheds light on the impact of this architectural shift and discusses hardware accelerator trends in terms of design, exploration, simulation, and frameworks developed in both academia and industry.

DOI:10.1561/1000000060
ISBN: 978-1-63828-162-7
88 pp. $65.00
Buy book (pb)
 
ISBN: 978-1-63828-163-4
88 pp. $150.00
Buy E-book (.pdf)
Table of contents:
1. Introduction
2. Basic Concepts
3. CNN and DNN Hardware Accelerators
4. Hardware Design Space Exploration Frameworks and Simulators
5. Conclusion
Acknowledgements
References

From CNN to DNN Hardware Accelerators: A Survey on Design, Exploration, Simulation, and Frameworks

The past decade has witnessed the consolidation of Artificial Intelligence technology, thanks to the popularization of Machine Learning (ML) models. The technological boom of ML models started in 2012 when the world was stunned by the record-breaking classification performance achieved by combining an ML model with a high computational performance graphic processing unit (GPU). Since then, ML models received ever-increasing attention, being applied in different areas such as computational vision, virtual reality, voice assistants, chatbots, and self-driving vehicles.

The most popular ML models are brain-inspired models such as Neural Networks (NNs), including Convolutional Neural Networks (CNNs) and, more recently, Deep Neural Networks (DNNs). They are characterized by resembling the human brain, performing data processing by mimicking synapses using thousands of interconnected neurons in a network.

In this growing environment, GPUs have become the de facto reference platform for the training and inference phases of CNNs and DNNs, due to their high processing parallelism and memory bandwidth. However, GPUs are power-hungry architectures. To enable the deployment of CNN and DNN applications on energy-constrained devices (e.g., IoT devices), industry and academic research have moved towards hardware accelerators. Following the evolution of neural networks from CNNs to DNNs, this monograph sheds light on the impact of this architectural shift and discusses hardware accelerator trends in terms of design, exploration, simulation, and frameworks developed in both academia and industry.

 
EDA-060