Learning with Incomplete Selective Perception

Thesis Proposal. Dana H. Ballard, thesis advisor.An agent with selective perception focuses its sensors on those parts of the environment that are relevant to the task at hand. Selective perception is an efficient method of gathering information from the world, but it presents problems for a learning agent when different actions are required in situations for which the selective perception system cannot produce distinguishing outputs. If this happens the agent is said to have incomplete perception, and the agent may be able to use internal state determined by past perceptions and actions in order to choose the correct action. I propose research on learning algorithms that use short-term memory to disambiguate the incomplete perception that arises with selective perception. I present the Utile Distinction Memory (UDM) algorithm that solves the incomplete perception problem using a partially observable Markov decision process to represent the agent's internal state space. A significant feature of the algorithm is that it will build an internal state space proportionate to the task at hand, not as large as would be required to represent all of the perceivable world. A second algorithm, part of work in progress, will keep the advantages of UDM while improving learning speed and the ability to recognize the significance of memories that span multiple time steps. Learning to use memory is difficult and will require a strong bias to learn efficiently. I will investigate "learning by watching" as a method of providing bias. Two applications I propose to study are: driving a simulated car using vision from the driver's point of view; and setting a table with human cooperation or interference. Using the results of psychophysical experiments, I will compare my algorithm's perceptual actions with the perceptual actions made by human subjects