For many years, color has been used in image understanding. Early landsat sensors, for example, provided image measurements for a set of visible and infrared bands. These images were used to classify areas of the earth according to known spectral signatures for region classes such as water or vegetation. In this context, the biggest motivation for using color was that it provided additional features at each pixel in an image to be used by a pattern classification system. In most cases, these additional features improved classification accuracy. For such applications, however, color was treated as a generic feature along with others such as shape of texture measures, and the unique physical attributes of color were not utilized.

In recent years, machine vision researchers have started to decipher and exploit the vast amount of information about a scene that is encoded in color images. The key to progress has been the realization of the importance of physical models that relate a three dimensional scene to its color image. Physical models characterize the relevant components of a scene including illumination, material surfaces, and sensors. To be useful, these models must also describe the color image formation process so that from the arrangement of a scene we can predict the resulting color image. From an analysis of these models, several important uses for color in machine vision have been identified. The articles in this volume describe these physical models and their implications along with algorithms that exploit these models.

The introductory article provides a general introduction to color vision by surveying many of the aspects and uses of color and by establishing terminology. A large set of references is given to guide further reading. Physical models underlying color image formation are presented in the next two sections. The articles in the first section discuss image formation in general and examine issues related to color sensing and the representation of surfaces and light sources. The second section contains a set of articles that focus specifically on color reflection models. These models are the foundation of the algorithms derived in later sections.

The use of color reflection models has significantly improved the capability of image segmentation algorithms. Such algorithms have the critical requirements of locating the different surfaces or objects present in a scene. In the third section, a collection of papers shows how color can be used to segment various optical phenomena that have traditionally caused difficulty for segmentation programs. Such phenomena include shadows, highlights, shading, and interreflection. Several of the articles present segmentation results that demonstrate vividly the power of the physics-based approach.

Color is also a valuable cue for recognizing objects. Unfortunately, the color measurements made by a vision system depend on the ambient illumination as well as on the intrinsic color of an object. In order to recognize an object in different lighting environments, a system must have the ability to discount the effects of the illumination. This ability is called color constancy. Most of the recent work on color constancy has been based on explicit physical models describing the lights and surfaces in a scene. This approach has led to several useful color constancy algorithms based on a range of models. A representative collection of these algorithms and their underlying physical assumptions are described in the fourth section.

Until recently, highlights were usually considered a nuisance to machine vision systems. The analysis of reflection models, however, has revealed that highlights contain a wealth of useful spectral and geometric information about a scene. In many situations, for example, the spectral composition of a highlight is approximately the same as the spectral composition of the illuminant. The papers in the fifth section present algorithms for finding highlights and for using them to infer properties of lights and surfaces in a scene. Algorithms are derived, for example, to estimate illuminant invariant color descriptions for a surface.

Another optical process, interreflection, has confounded segmentation and shape estimation algorithms for many years. Interreflection occurs when a visible surface patch is illuminated by light reflected from another surface. In general, this illumination from another surface changes the color and intensity of the light reflected by the surface patch. Thus, interreflection can be regarded as an important special case of spatially varying illumination. Researchers have recently developed models for interreflection and techniques to remove its effects from images. Some authors have observed that, like highlights, interreflection regions can be used to compute additional information about a scene. The articles in the sixth section present models for interreflection and algorithms for interpreting interreflection effects in images.

Back to the ICVL homepage