An Approach to Knowledge-Directed Image Analysis

A vision system is described which uses a semantic network model and a distributed control structure to accomplish the image analysis process. The process of "understanding an image" leads to the instantiation of a subset of the model, and the identification of nodes in the instance of the model with image features. The instantiated nodes and the relations between them form another data structure called the sketchmap. The sketchmap explicates the relation of the model to the image; this model-image mapping is accomplished by mapping procedures which are part of the procedural knowledge in the model. The procedures are accompanied by descriptions which contain at least pre-and post-conditions for the procedure and performance measures for it. Nodes which have attached procedures may also have an executive procedure attached. This executive is responsible for deciding which of several possibly effective procedures to run. Thus through the executive the system does a very general kind of procedure invocation based not only on what the executive knows about global state, but on a rich description of the procedure's capabilities. The user's program is generally responsible for allocating effort at a level above that of the individual executive procedure. Thus no single domain-independent formulation or methodology is imposed on all vision tasks. One facility provided by the system is the use of geometric constraints between model objects to guide search for the objects in the image. The system is an attempt to bring together many current ideas in artificial intelligence and vision programming and thereby to cast some light on fundamental problems of computer perception. The semantic network facilitates the interplay between geometric and other relational constraints which are used to direct and limit search. The use of attached procedures in the network gives a mix of declarative and procedural knowledge, and the executive provides an unusually powerful procedure invocation scheme. The multiplicity of procedures allows modelling objects under radically different conditions and levels of detail. This tends to make the system robust in that an object which could not be located initially may be found later when knowledge about the image has increased. The system is illustrated throughout the chapter with illustrations from two particular applications: the finding of ships in a dock scene and the finding of ribs in a chest X-ray film