This book presents a broad review of state-of-the-art 3D video production technologies and applications. The text opens with a concise introduction to the field, before examining the design and calibration methods for multi-view camera systems, including practical implementation technologies. A range of algorithms are then described for producing 3D video from video data. A selection of 3D video applications are also demonstrated. Features: describes real-time synchronized multi-view video capture, and object tracking with a group of active cameras; discusses geometric and photometric camera calibration, and 3D video studio design with active cameras; examines 3D shape and motion reconstruction, texture mapping and image rendering, and lighting environment estimation; demonstrates attractive 3D visualization, visual contents analysis and editing, 3D body action analysis, and data compression; highlights the remaining challenges and the exciting avenues for future research in 3D video technology.

It has long been a dream to realize machines with flexible visual perception capability. Research on digital image processing by computers was initiated about 30 years ago, and since then a wide variety of image processing algorithms have been devised. Using such image processing algorithms and advanced hardware technologies, many practical ma chines with visual recognition capability have been implemented and are used in various fields: optical character readers and design chart readers in offices, position-sensing and inspection systems in factories, computer tomography and medical X-ray and microscope examination systems in hospitals, and so on. Although these machines are useful for specific tasks, their capabilities are limited. That is, they can analyze only simple images which are recorded under very carefully adjusted photographic conditions: objects to be recognized are isolated against a uniform background and under well-controlled artificial lighting. In the late 1970s, many image understanding systems were de veloped to study the automatic interpretation of complex natural scenes. They introduced artificial intelligence techniques to represent the knowl edge about scenes and to realize flexible control structures. The first author developed an automatic aerial photograph interpretation system based on the blackboard model (Naga1980). Although these systems could analyze fairly complex scenes, their capabilities were still limited; the types of recognizable objects were limited and various recognition vii viii Preface errors occurred due to noise and the imperfection of segmentation algorithms."

It has long been a dream to realize machines with flexible visual perception capability. Research on digital image processing by computers was initiated about 30 years ago, and since then a wide variety of image processing algorithms have been devised. Using such image processing algorithms and advanced hardware technologies, many practical ma chines with visual recognition capability have been implemented and are used in various fields: optical character readers and design chart readers in offices, position-sensing and inspection systems in factories, computer tomography and medical X-ray and microscope examination systems in hospitals, and so on. Although these machines are useful for specific tasks, their capabilities are limited. That is, they can analyze only simple images which are recorded under very carefully adjusted photographic conditions: objects to be recognized are isolated against a uniform background and under well-controlled artificial lighting. In the late 1970s, many image understanding systems were de veloped to study the automatic interpretation of complex natural scenes. They introduced artificial intelligence techniques to represent the knowl edge about scenes and to realize flexible control structures. The first author developed an automatic aerial photograph interpretation system based on the blackboard model (Naga1980). Although these systems could analyze fairly complex scenes, their capabilities were still limited; the types of recognizable objects were limited and various recognition vii viii Preface errors occurred due to noise and the imperfection of segmentation algorithms."

It is most appropriate that the first volume to appear in the series "Advanced Applications in Pattern Recognition" should be this monograph by Nagao and Matsuyama. The work described here is a deep unification and synthesis of the two fundamental approaches to pat tern recognition: numerical (also known as "statistical") and struc tural ("linguistic," "syntactic"). The power and unity of the meth odology flow from the apparently effortless and natural use of the knowledge-base framework illuminated by the best results of artificial intelligence research. An integral part of the work is the algorithmic solution of many hitherto incompletely or clumsily treated problems. It was on the occasion of a laboratory visit in connection with the 4th IJCPR (of hich Professor Nagao was the very able Program Chairman) that I saw in operation the system described here. On the spot I expressed the desire to see the work described for the inter national technical audience in this series and the authors were kind enough to agree to contribute to a new and unknown series. With the publication of this monograph on the eve of the 5th ICPR my wish is fu1fi11 d. I want to thank here the authors and Plenum Publishing Corporation for making this volume and the series a reality."