Incorporating visual imagery into a cognitive architecture: An initial theory, design and implementation

Abstract: Humans use visual imagery for a variety of purposes including reasoning about visual properties and spatial relationships, anticipating future events, recalling past experiences, and memorizing concepts and facts. Cognitive architectures have traditionally ignored visual imagery, but we have started to explore this phenomenon in the context of Soar. Soar provides the underlying control, memory, and learning mechanisms while visual imagery provides efficient representations and processes for finding an object’s visual features and spatial relationships not explicitly encoded with symbols. This multi-modal approach enables the architecture to use the most efficient representation for the appropriate computation and requires less domain knowledge for visual-spatial type environments and tasks. This paper outlines our theory and the corresponding architecture, design, and implementation. We present initial results in two small problem domains