Knowledge based image understanding by iterative optimization

Abstract In this paper knowledge based image interpretation is formulated and solved as an optimization problem which takes into account the observed image data, the available task specic knowledge, and the requirements of an application. Knowl-edge is represented by a semantic network consistint of concepts (nodes) and links (edges). Concepts are further dened by attributes, relations, and a judgment func-tion. The interface between the symbolic knowledge base and the results of image (or signal) processing and initial segmentation is specied via primitive concepts. We present a recently developed approach to optimal interpretation that is based on the automatic conversion of the concept oriented semantic network to an attribute centered representation and the use of iterative optimization procedures, like e.g. sim-ulated annealing or genetic algorithms. We show that this is a feasible approach which provides `any{time ' capability and allows parallel processing. It provides a well{dened combination of signal and symbol oriented processing by optimizing a heuristic judg-ment function. The general ideas have been applied to various problems of image and speech understanding. As an example we describe the recognition of streets from TV image sequences to demonstrate the eciency of iterative optimization. Key Words: semantic network, iterative optimization, knowledge based image analysis