Machine Learning on Commodity Tiny Devices Theory and Practice

This book aims at the tiny machine learning (TinyML) software and hardware synergy for edge intelligence applications. This book presents on-device learning techniques covering model-level neural network design, algorithm-level training optimization and hardware-level instruction acceleration. Analyzing the limitations of conventional in-cloud computing would reveal that on-device learning is a promising research direction to meet the requirements of edge intelligence applications. As to the cutting-edge research of TinyML, implementing a high-efficiency learning framework and enabling system-level acceleration is one of the most fundamental issues. This book presents a comprehensive discussion of the latest research progress and provides system-level insights on designing TinyML frameworks, including neural network design, training algorithm optimization and domain-specific hardware acceleration. It identifies the main challenges when deploying TinyML tasks in the real world and guides the researchers to deploy a reliable learning system. This book will be of interest to students and scholars in the field of edge intelligence, especially to those with sufficient professional Edge AI skills. It will also be an excellent guide for researchers to implement high-performance TinyML systems.

1. Introduction 2. Fundamentals: On-device Learning Paradigm 3. Preliminary: Theories and Algorithms 4. Model-level Design: Computation Acceleration and Communication Saving 5. Hardware-level Design: Neural Engines and Tensor Accelerators 6. Infrastructure-level Design: Serverless and Decentralized Machine Learning 7. System-level Design: from Standalone to Clusters 8. Application: Image-based Visual Perception 9. Application: Video-based Real-time Processing 10. Application: Privacy, Security, Robustness and Trustworthiness in Edge AI

Song Guo is a Full Professor leading the Edge Intelligence Lab and Research Group of Networking and Mobile Computing at the Hong Kong Polytechnic University. Professor Guo is a Fellow of the Canadian Academy of Engineering, Fellow of the IEEE, Fellow of the AAIA and Clarivate Highly Cited Researcher. Qihua Zhou is a PhD student with the Department of Computing at the Hong Kong Polytechnic University. His research interests include distributed AI systems, large-scale parallel processing, TinyML systems and domain-specific accelerators.