Chapter 1 Introduction. 1
1.1 AI's Historical Changes 1
1.2 What Is Deep Learning?. 3
1.3 Practical Applications of Deep Learning. 4
1.4 Structure of the Book. 7
1.5 Introduction to MindSpore. 7
Chapter 2 Deep Learning Basics. 18
2.1 Regression Algorithms. 18
2.2 Gradient Descent 21
2.3 Classification Algorithms. 25
2.4 Overfitting and Underfitting. 28
Chapter 3 DNN.. 32
3.1 Feedforward Network. 32
3.2 Backpropagation. 34
3.3 Generalization Ability. 38
3.4 Implementing Simple Neural Networks Using MindSpore. 39
Chapter 4 Training of DNNs. 45
4.1 Main Challenges to Deep Learning Systems 45
4.2 Regularization. 48
4.3 Dropout 51
4.4 Adaptive Learning Rate. 55
4.5 Batch Normalization. 59
4.6 Implementing DNNs Using MindSpore. 61
Chapter 5 Convolutional Neural Network. 66
5.1 Convolution. 66
5.2 Pooling. 69
5.3 Residual Network. 71
5.4 Application: Image Classification. 74
5.5 Implementing Image Classification Based on the
DNN Using MindSpore. 79
Chapter 6 RNN.. 89
6.1 Overview.. 89
6.2 Deep RNN.. 90
6.3 Challenges of Long-Term Dependency. 91
6.4 LSTM Network and GRU.. 93
6.5 Application: Text Prediction. 96
6.6 Implementing Text Prediction Based on LSTM Using MindSpore. 97
Chapter 7 Unsupervised Learning: Word Vector. 101
7.1 Word2Vec. 102
7.2 GloVe. 114
7.3 Transformer 121
7.4 BERT.. 130
7.5 Comparison Between Typical Word Vector Generation Algorithms. 137
7.6 Application: Automatic Question Answering. 139
7.7 Implementing BERT-based Automatic Answering Using MindSpore. 154
Chapter 8 Unsupervised Learning: Graph Vector. 159
8.1 Graph Vector Overview.. 159
8.2 DeepWalk Algorithm... 161
8.3 LINE Algorithm... 166
8.4 Node2Vec Algorithm... 170
8.5 GCN Algorithm... 174
8.6 GAT Algorithm... 179
8.7 Application: Recommendation System.. 183
Chapter 9 Unsupervised Learning: Deep Generative Model 191
9.1 Variational Autoencoder 191
9.2 Generative Adversarial Network. 200
9.3 Application: Data Augmentation. 208
9.4 Implementing GAN-based Data Augmentation Using MindSpore. 221
Chapter 10 Deep Reinforcement Learning. 225
10.1 Basic Concepts of Reinforcement Learning. 225
10.2 Basic Solution Method. 230
10.3 Deep Reinforcement Learning Algorithm... 235
10.4 Latest Applications. 247
10.5 Implementing DQN-based Game Using MindSpore. 253
Chapter 11 Automated Machine Learning. 255
11.1 AutoML Framework. 255
11.2 Existing AutoML Systems. 278
11.3 Meta Learning. 288
11.4 Implementing AutoML Using MindSpore. 294
Chapter 12 Device-Cloud Collaboration. 302
12.1 On-device Inference. 302
12.2 Device-Cloud Transfer Learning. 304
12.3 Device-Cloud Federated Learning. 308
12.4 Device-Cloud Collaboration Framework. 313
Chapter 13 Deep Learning Visualization. 322
13.1 Overview.. 322
13.2 MindSpore Visualization. 337
Chapter 14 Data Preparation for Deep Learning. 354
14.1 Overview of Data Format 354
14.2 Data Format in Deep Learning. 355
14.3 Common Data Formats for Deep Learning. 362
