Theory of Reconstruction from Image Motion

The image taken by a moving camera changes with time. These image motions contain information about the motion of the camera and about the shapes of the objects in the field of view. There are two main types of image motion, finite displacements and image velocities. Finite displacements are described by the point correspondences between two images of the same scene taken from different positions. Image velocities are the velocities of the points in the image as they move over the projection surface. Reconstruction is the task of obtaining from the image-motions information about the camera motion or about the shapes of objects in the field of view. In this book the theory underlying reconstruction is described. Reconstruction from image motion is the subject matter of two different sci­ entific disciplines, photogrammetry and computer vision. In photogrammetry the accuracy of reconstruction is emphasised; in computer vision the emphasis is on methods for obtaining information from images in real time in order to guide a mechanical device such as a robot arm or an automatic vehicle. This book arises from recent work carried out in computer vision. Computer vision is a young field but it is developing rapidly. The earliest papers on reconstruction in the computer vision literature date back only to the mid 1970s. As computer vision develops, the mathematical techniques applied to the analysis of recon­ struction become more appropriate and more powerful.

1 Introduction.- 1.1 Background.- 1.2 Reconstruction.- 1.3 Conventions About the Image.- 1.4 Mathematical Background.- 1.4.1 Terminology.- 1.4.2 Euclidean Space and Projective Space.- 1.4.3 Algebraic Curves.- References.- 2 Reconstruction from Image Correspondences.- 2.1 Euclidean Framework for Reconstruction.- 2.1.1 Euclidean Treatment of Ambiguity.- 2.1.2 Ambiguity and Instability.- 2.1.3 The Maximum Number of Reconstructions.- 2.2 Essential Matrices.- 2.2.1 A Characterisation of Essential Matrices.- 2.2.2 The Singular Value Decomposition.- 2.2.3 Symmetric and Antisymmetric Parts.- 2.2.4 Ambiguity.- 2.3 Projective Framework for Reconstruction.- 2.3.1 The Epipolar Transformation.- 2.3.2 Ambiguity.- 2.3.3 The Intersection of a Critical Surface Pair.- 2.4 Reconstruction up to a Collineation.- 2.4.1 Reconstruction Based on the Epipolar Transformation.- 2.4.2 Ambiguity.- 2.4.3 Critical Surfaces.- References.- 3 Critical Surfaces and Horopter Curves.- 3.1 The Absolute Conic.- 3.1.1 The Absolute Conic and Camera Calibration.- 3.1.2 Involutions and the Absolute Conic.- 3.2 Rectangular Quadrics.- 3.2.1 Algebraic Characterisations of Rectangular Quadrics.- 3.2.2 Rigid Involutions of Rectangular Quadrics.- 3.3 Horopter Curves.- 3.3.1 Characterisations of Horopter Curves.- 3.3.2 Rigid Involutions of Horopter Curves.- 3.3.3 The Centre of a Horopter Curve.- 3.3.4 Examples.- 3.3.5 Horopter Curves on Rectangular Quadrics.- 3.4 Horopter Curves and Reconstruction.- 3.4.1 A Formula for ??.- 3.4.2 Two Cubic Constraints on Critical Surfaces.- 3.4.3 An Example.- 3.5 Reconstruction up to a Collineation.- References.- 4 Reconstruction from Image Velocities.- 4.1 Framework.- 4.2 Ambiguity.- 4.2.1 Preliminary Results.- 4.2.2 Critical Surfaces.- 4.2.3 Singular Critical Surfaces.- 4.2.4 Critical Surface Pairs.- 4.2.5 Cubic Polynomial Constraints on Critical Surfaces.- 4.2.6 The Maximum Number of Reconstructions.- 4.3 Algebraic Properties of Four Image Velocity Vectors.- 4.3.1 The Quartic Polynomial Constraint.- 4.3.2 Irregular Image Velocity Fields.- 4.3.3 The Effects of Small Perturbations.- 4.3.4 Symmetric Arrangements of Base Points: The Square.- 4.3.5 Symmetric Arrangements of Base Points: The Tripod.- 4.4 The Linear System of Quartics.- 4.4.1 The Dimension of the Linear System.- 4.4.2 Singular Points of Quartics.- 4.4.3 Base Points of the Linear System.- 4.5 The Derivatives of the Image Velocity Field.- 4.5.1 The Derivatives to Second Order.- 4.5.2 Polynomial Constraints on the Translational Velocity.- 4.5.3 Time to Contact.- References.- 5 Reconstruction from Minimal Data.- 5.1 Kruppa’s Method.- 5.1.1 The Homography.- 5.1.2 Constraints Arising from the Camera Calibration.- 5.1.3 The Two Sextics.- 5.2 Demazure’s Method.- 5.2.1 The Variety of Essential Matrices.- 5.2.2 Properties of the Variety of Essential Matrices.- 5.2.3 Linear Subspaces of ?8.- 5.2.4 Ten Distinct Intersections.- 5.3 Reconstruction up to a Collineation.- 5.3.1 Sturm’s Method.- 5.3.2 An Algebraic Method.- 5.4 Reconstruction From Five Image Velocity Vectors.- 5.4.1 Preliminary Results.- 5.4.2 The Quartic Constraints.- 5.4.3 Counting the Solutions.- 5.4.4 Critical Surfaces.- 5.4.5 Counting the Critical Surfaces.- References.- 6 Algorithms.- 6.1 Reconstruction from Image Correspondences.- 6.1.1 An SVD Based Algorithm.- 6.1.2 Descent Algorithms.- 6.2 Reconstruction from Image Velocities.- 6.2.1 A Least Squares Algorithm.- 6.2.2 Properties of the Least Squares Error Function.- 6.2.3 Irregular Image Velocity Fields.- 6.2.4 The First Order Algorithm.- 6.2.5 Constraints on the Translational Velocity.- References.