Visuomotor transformations for eye–hand coordination

Abstract: In recent years the scientific community has come to appreciate that the early cortical representations for visually guided arm movements are probably coded in a visual frame, i.e. relative to retinal landmarks. While this scheme accounts for many behavioral and neurophysiological observations, it also poses certain problems for manual control. For example, how are these oculocentric representations updated across eye movements, and how are they then transformed into useful commands for accurate movements of the arm relative to the body? Also, since we have two eyes, which is used as the reference point in eye–hand alignment tasks like pointing? We show that patterns of errors in human pointing suggest that early oculocentric representations for arm movement are remapped relative to the gaze direction during each saccade. To then transform these oculocentric representations into useful commands for accurate movements of the arm relative to the body, the brain correctly incorporates the three-dimensional, rotary geometry of the eyes when interpreting retinal images. We also explore the possibility that the eye–hand coordination systems uses a strategy like ocular dominance, but switches alignment between the left and right eye in order to maximize eye–hand coordination in the best field of view. Finally, we describe the influence of eye position on eye–hand alignment, and then consider how head orientation influences the linkage between oculocentric visual frames and bodycentric motor frames. These findings are framed in terms of our ‘conversion-on-demand ’ model, which suggests a virtual representation of egocentric space, i.e. one in which only those representations selected for action are put through the complex visuomotor transformations required for interaction with actual objects in personal space