An artificial sensory neuron with visual-haptic fusion

Human behaviors are extremely sophisticated, relying on the adaptive, plastic and event-driven network of sensory neurons. Such neuronal system analyzes multiple sensory cues efficiently to establish accurate depiction of the environment. Here, we develop a bimodal artificial sensory neuron to implement the sensory fusion processes. Such a bimodal artificial sensory neuron collects optic and pressure information from the photodetector and pressure sensors respectively, transmits the bimodal information through an ionic cable, and integrates them into post-synaptic currents by a synaptic transistor. The sensory neuron can be excited in multiple levels by synchronizing the two sensory cues, which enables the manipulating of skeletal myotubes and a robotic hand. Furthermore, enhanced recognition capability achieved on fused visual/haptic cues is confirmed by simulation of a multi-transparency pattern recognition task. Our biomimetic design has the potential to advance technologies in cyborg and neuromorphic systems by endowing them with supramodal perceptual capabilities.Ministry of Education (MOE)National Research Foundation (NRF)Published versionThe authors thank the financial support from the Agency for Science, Technology and Research (A*STAR) under its AME Programmatic Funding Scheme (project no. A18A1b0045) Cyber-Physiochemical Interfaces (CPI) Programme, the National Research Foundation, Prime Minister’s office, Singapore, under its NRF Investigatorship (NRFNRFI2017-07), and Singapore Ministry of Education (MOE2017-T2-2-107)