Representation Learning for Natural Language Processing

This open access book provides an overview of the recent advances in representation learning theory, algorithms and applications for natural language processing (NLP). It is divided into three parts. Part I presents the representation learning techniques for multiple language entries, including words, phrases, sentences and documents. Part II then introduces the representation techniques for those objects that are closely related to NLP, including entity-based world knowledge, sememe-based linguistic knowledge, networks, and cross-modal entries. Lastly, Part III provides open resource tools for representation learning techniques, and discusses the remaining challenges and future research directions.

The theories and algorithms of representation learning presented can also benefit other related domains such as machine learning, social network analysis, semantic Web, information retrieval, data mining and computational biology. This book is intended for advanced undergraduate and graduate students, post-doctoral fellows, researchers, lecturers, and industrial engineers, as well as anyone interested in representation learning and natural language processing.

1. Representation Learning and NLP.- 2. Word Representation.- 3. Compositional Semantics.- 4. Sentence Representation.- 5. Document Representation.- 6. Sememe Knowledge Representation.- 7. World Knowledge Representation.- 8. Network Representation.- 9. Cross-Modal Representation.- 10. Resources.- 11. Outlook.

Zhiyuan Liu is an Associate Professor at the Department of Computer Science and Technology at Tsinghua University, China. His research interests include representation learning, knowledge graphs and social computation, and he has published more than 80 papers in at leading conferences and in respected journals. He has received several awards/honors, including Excellent Doctoral Dissertation awards from Tsinghua University and the Chinese Association for Artificial Intelligence, and was named as one of  MIT Technology Review Innovators Under 35 China (MIT TR-35 China). He has served as area chair for various conferences, including ACL, EMNLP, COLING.

Yankai Lin is a researcher at the Pattern Recognition Center, Tencent Wechat. He received his Ph.D. degree in Computer Science from Tsinghua in 2019. His research interests include representation learning, information extraction and question answering. He has published more than 10 papers at international conferences, including ACL, EMNLP, IJCAI and AAAI. He was named an Academic Rising Star of Tsinghua University and a Baidu Scholar.

Maosong Sun is a Professor at the Department of Computer Science and Technology and the Executive Vice Dean of the Institute for Artificial Intelligence, Tsinghua University. His research interests include natural language processing, machine learning, computational humanities and social sciences. He is the chief scientist of the National Key Basic Research and Development Program (973 Program) and the chief expert of various major National Social Science Fund of China projects. He has published over 100 papers at leading conferences and in respected journals. He is the Director of Tsinghua University-National University of Singapore Joint Research Center on Next Generation Search Technologies, and the editor-in-chief of the Journal of Chinese Information Processing. He received the Nationwide Distinguished Practitioner award from the State Commission for Language Affairs, People’s Republic of China, in 2007, and the National Excellent Scientific and Technological Practitioner award from the China Association for Science and Technology in 2016.