Image Search and Retrieval Strategies

AbstractThe proliferation of computer technology and digital image‐acquisition hardware has led to the widespread use of image data across a variety of applications including astronomy, art, natural resources, engineering design, military, business operation, medicine, education, etc. Major research activities in the digital image databases surged after the U.S. government's Digital Library Initiative (DLI) from 1994 to 1998. The DLI research team at the University of California at Berkeley developed a work‐centered digital information system that contains 450 digital text documents, 200 air photos, and over 11,000 ground photographs. The system allowed a user or a working group to access its own collections of varying data types, and to generate new materials to be added to the collection. The DLI project at the University of California (UC) at Santa Barbara focused on the development of digital databases that contain geographically referenced materials such as maps and aerial photos. The research team developed a number of tools for browsing and retrieving map images at multiple resolution. Two other major efforts in developing digital image databases were at The National Library of Medicine (NLM), a component of the National Institutes of Health (NIH), and at Time Warner. More recently, research scientists at the University of California at Berkeley and the Fine Arts Museums of San Francisco successfully launched on the Internet the largest art image database in the world, the Thinker ImageBase.Automatic image indexing and retrieving technology is fundamental in digital image databases. There are two general strategies in image retrieval, browsing and searching. Browsing is an information retrieval strategy in which a user navigates through an ordered arrangement of images by making selections from the progressive levels of a hierarchy into which the available images have been logically grouped. Image browsing relies on the user's cognitive abilities to recognize images of interest without having to formulate a specific query. Searching is an information retrieval strategy in which the user communicates with an information retrieval system by an interface that requires the user to input a search query.Images can be indexed and retrieved by text‐tag information and/or by image content. Conventionally, images are indexed using text‐tag information, such as title, key words, date of the work, artist, author, photographer, legend, captions, etc. Images are often subject to a wide range of interpretations, and textual descriptions can only begin to capture the richness and complexity of the semantic content of a visual image. In many applications, the complexity of the information embedded in image content such as types of objects, object attributes, and spatial relationship of objects, etc, cannot be synthesized in a few key words. It has been reported that users querying an image collection tend to be much more specific in their requests and information needs than when querying a text database. Furthermore, text indexing requires large human effort in creating the meta‐data that enables visual queries and is language dependent. Large databases containing thousands and millions of digital images that can occupy gigabytes of space are almost impossible for manual indexing and searching. The demand for systems using pictorial information combined with textual description in image retrieval is growing and automatic indexing and retrieval based on image content has become the most promising techniques for large image databases and digital image libraries. This article describes the techniques developed for content‐based retrieval.