An Invitation to 3-D Vision: From Images to Models

Abstract

This book is intended to give students at the advanced undergraduate or introductory graduate level and researchers in computer vision, robotics, and computer graphics a self-contained introduction to the geometry of 3-D vision: that is, the reconstruction of 3-D models of objects from a collection of 2-D images. The only prerequisite for this book is a course in linear algebra at the undergraduate level. As timely research summary, two bursts of manuscripts were published in the past on a geometric approach to computer vision: the ones that were published in mid 1990’s on the geometry of two views (e.g., Faugeras 1993, Weng, Ahuja and Huang 1993, Maybank 1993), and the ones that were recently published on the geometry of multiple views (e.g., Hartley and Zisserman 2000, Faugeras, Luong and Papadopoulo 2001).1 While a majority of those manuscripts were to summarize up to date research results by the time they were published, we sense that now the time is ripe for putting a coherent part of that material in a unified and yet simplified framework which can be used for pedegogical purposes. Although the approach we are to take here deviates from those old ones and the techniques we use are mainly linear algebra, this book nonetheless gives a comprehensive coverage of what is known to date on the geometry of 3-D vision. It also builds on a homogeneous terminology a solid analytical foundation for future research in this young field. This book is organized as follows. Following a brief introduction, Part I provides background materials for the rest of the book. Two fundamental transformations in multiple view geometry, namely the rigid-body motion and perspective projection, are introduced in Chapters 2 and 3 respectively. Image formation and feature extraction are discussed in Chapter 4. The two chapters in Part II cover the classic theory of two view geometry based on the so-called epipolar constraint. Theory and algorithms are developed for both discrete and continuous motions, and for both calibrated and uncalibrated camera models. Although the epipolar constraint has been very successful in the two view case, Part III shows that a more proper tool for studying the geometry of multiple views is the so-called rank condition on the multiple view matrix, which trivially implies all the constraints among multiple images that are known to date, in particular the epipolar constraint. The theory culminates in Chapter 10 with a unified theorem on a rank condition for arbitrarily mixed point, line and plane features. It captures all possible constraints among multiple images of these geometric primitives, and serves as a key to both geometric analysis and algorithmic development. Based on the theory and conceptual algorithms developed in early part of the book, Part IV develops practical reconstruction algorithms step by step, as well as discusses possible extensions of the theory covered in this book. Different parts and chapters of this book have been taught as a one- semester course at the University of California at Berkeley, the University of Illinois at Urbana-Champaign, and the George Mason University, and as a two quater course at the University of California at Los Angles. There is apparantly adequate material for lectures of one and a half semester or two quaters. Advanced topics suggested in Part IV or chosen by the instructor can be added to the second half of the second semester if a two-semester course is offered. Given below are some suggestions for course development based on this book: 1. A one-semester course: Appendix A and Chapters 1 - 7 and part of Chapters 8 and 13. 2. A two-quater course: Chapters 1 - 6 for the first quater and Chapters 7 - 10 and 13 for the second quater. 3. A two-semester course: Appendix A and Chapters 1 - 7 for the first semester; Chapters 8 - 10 and the instructor’s choice of some advanced topics from chapters in Part IV for the second semester. Exercises are provided at the end of each chapter. They consist of three types: 1. drill exercises that help student understand the theory covered in each chapter; 2. programming exercises that help student grasp the algorithms developed in each chapter; 3. exercises that guide student to creatively develop a solution to a specialized case that is related to but not necessarily covered by the general theorems in the book. Solutions to most of the exercises will be made available upon the publication of this book. Software for examples and algorithms in this book will also be made available at a designated website.