Self-Organising Neural Networks Independent Component Analysis and Blind Source Separation

The conception of fresh ideas and the development of new techniques for Blind Source Separation and Independent Component Analysis have been rapid in recent years. It is also encouraging, from the perspective of the many scientists involved in this fascinating area of research, to witness the growing list of successful applications of these methods to a diverse range of practical everyday problems. This growth has been due, in part, to the number of promising young and enthusiastic researchers who have committed their efforts to expanding the current body of knowledge within this field of research. The author of this book is among one of their number. I trust that the present book by Dr. Mark Girolami will provide a rapid and effective means of communicating some of these new ideas to a wide international audience and that in turn this will expand further the growth of knowledge. In my opinion this book makes an important contribution to the theory of Independent Component Analysis and Blind Source Separation. This opens a range of exciting methods, techniques and algorithms for applied researchers and practitioner engineers, especially from the perspective of artificial neural networks and information theory. It has been interesting to see how rapidly the scientific literature in this area has grown.

1. Introduction.- 1.1 Self-Organisation and Blind Signal Processing.- 1.2 Outline of Book Chapters.- 2. Background to Blind Source Separation.- 2.1 Problem Formulation.- 2.2 Entropy and Information.- 2.3 A Contrast Function for ICA.- 2.4 Cumulant Expansions of Probability Densities and Higher Order Statistics.- 2.5 Gradient Based Function Optimisation.- 3. Fourth Order Cumulant Based Blind Source Separation.- 3.1 Early Algorithms and Techniques.- 3.2 The Method of Contrast Minimisation.- 3.3 Adaptive Source Separation Methods.- 3.4 Conclusions.- 4. Self-Organising Neural Networks.- 4.1 Linear Self-Organising Neural Networks.- 4.2 Non-Linear Self-Organising Neural Networks.- 4.3 Conclusions.- 5. The Non-Linear PCA Algorithm and Blind Source Separation.- 5.1 Introduction.- 5.2 Non-Linear PCA Algorithm and Source Separation.- 5.3 Non-Linear PCA Algorithm Cost Function.- 5.4 Non-Linear PCA Algorithm Activation Function.- 5.5 Conclusions.- 6. Non-Linear Feature Extraction and Blind Source Separation.- 6.1 Introduction.- 6.2 Structure Identification in Multivariate Data.- 6.3 Neural Network Implementation of Exploratory Projection Pursuit.- 6.4 Neural Exploratory Projection Pursuit and Blind Source Separation.- 6.5 Kurtosis Extrema.- 6.6 Finding Interesting and Independent Directions.- 6.7 Finding Multiple Interesting and Independent Directions Using Symmetric Feedback and Adaptive Whitening.- 6.8 Finding Multiple Interesting and Independent Directions Using Hierarchic Feedback and Adaptive Whitening.- 6.9 Simulations.- 6.10 Adaptive BSS Using a Deflationary EPP Network.- 6.11 Conclusions.- 7. Information Theoretic Non-Linear Feature Extraction And Blind Source Separation.- 7.1 Introduction.- 7.2 Information Theoretic Indices for EPP.- 7.3 Maximum Negentropy Learning.- 7.4 General Maximum Negentropy Learning.- 7.5 Stability Analysis of Generalised Algorithm.- 7.6 Simulation Results.- 7.7 Conclusions.- 8. Temporal Anti-Hebbian Learning.- 8.1 Introduction.- 8.2 Blind Source Separation of Convolutive Mixtures.- 8.3 Temporal Linear Anti-Hebbian Model.- 8.4 Comparative Simulation.- 8.5 Review of Existing Work on Adaptive Separation of Convolutive Mixtures.- 8.6 Maximum Likelihood Estimation and Source Separation.- 8.7 Temporal Anti-Hebbian Learning Based on Maximum Likelihood Estimation.- 8.8 Comparative Simulations Using Varying PDF Models.- 8.9 Conclusions.- 9. Applications.- 9.1 Introduction.- 9.2 Industrial Applications.- 9.3 Biomedical Applications.- 9.4 ICA: A Data Mining Tool.- 9.5 Experimental Results.- 9.6 Conclusions.- References.