Efficient Execution of Irregular Dataflow Graphs Hardware/Software Co-optimization for Probabilistic AI and Sparse Linear Algebra

This book focuses on the acceleration of emerging irregular sparse workloads, posed by novel artificial intelligent (AI) models and sparse linear algebra. Specifically, the book outlines several co-optimized hardware-software solutions for a highly promising class of emerging sparse AI models called Probabilistic Circuit (PC) and a similar sparse matrix workload for triangular linear systems (SpTRSV). The authors describe optimizations for the entire stack, targeting applications, compilation, hardware architecture and silicon implementation, resulting in orders of magnitude higher performance and energy-efficiency compared to the existing state-of-the-art solutions. Thus, this book provides important building blocks for the upcoming generation of edge AI platforms.

Chapter 1. Irregular workloads at risk of losing the hardware lottery.- Chapter 2. Suitable data representation: A study of fixed point, floating point,and positTM formats for probabilistic AI.- Chapter 3. GraphOpt: constrained-optimization-based parallelization of irregular workloads for multicore processors.- Chapter 4. DAG Processing Unit version 1 (DPU): Efficient execution of irregular workloads on a multicore processor.- Chapter 5. DAG Processing Unit version 2 (DPU-v2): Efficient execution of irregular workloads on a spatial datapath.- Chapter 6. Conclusions and future work.

Nimish Shah is an Electrical Engineering PhD student at KU Leuven, Belgium, advised by Prof. Marian Verhelst. His research focuses on hardware-software co-optimization for the energy-efficient acceleration of machine learning, sparse algebra, and graph workloads. Prior to debuting his doctoral studies, Mr. Shah was an ASIC engineer in a GPU design team at Nvidia, Bengaluru, India, working on data (de)compression hardware. He received an M.Tech in Electronic Systems Design from the Indian Institute of Science, and a B.Tech in Electronics Engineering from S.V. National Institute of Technology, Surat, India. Mr. Shah is a recipient of the HiSilicon Sponsorship Grant for MPW Prototyping, the ISSCC Code-a-Chip Travel Grant Award 2023, and the student travel grant for the IEEE/ACM MICRO 2022 conference. His research contributed to a joint project with Intel that won the "Intel 2021 Outstanding Researcher Award". He is also the recipient of the DESE Design Medal for excellence in his master’s studies and thesis work at IISc, and was selected as a fellow for the undergraduate engineering stream under the prestigious Kishore Vaigyanik Protsahan Yojana (KVPY) 2010 of the Government of India.

Wannes Meert received his degrees of Master of Electrotechnical Engineering, Micro-electronics (2005), Master of Artificial Intelligence (2006) and Ph.D. in Computer Science (2011) from KU Leuven. He is an IOF research manager in the DTAI section at KU Leuven. His work is focused on applying machine learning, artificial intelligence and anomaly detection technology to industrial application domains with various industrial and academic partners..

Marian Verhelst is a full professor at the MICAS laboratories of KU Leuven and a research director at imec. Her research focuses on embedded machine learning, hardware accelerators, HW-algorithm co-design and low-power edge processing. She received a PhD from KU Leuven in 2008, and worked as a research scientist at Intel Labs, Hillsboro OR from 2008 till 2010. Marian is a member of the board of directors of tinyML and active in the TPC’s of DATE, ISSCC, VLSI and ESSCIRC, was the chair of tinyML2021 and TPC co-chair of AICAS2020. Marian is an IEEE SSCS Distinguished Lecturer, was a member of the Young Academy of Belgium, an associate editor for TVLSI, TCAS-II and JSSC and a member of the STEM advisory committee to the Flemish Government. Marian received the laureate prize of the Royal Academy of Belgium in 2016, the 2021 Intel Outstanding Researcher Award, and the André Mischke YAE Prize for Science and Policy in 2021.