A Situationally Aware Voice-Commandable Robotic Forklift Working Alongside People in Unstructured Outdoor Environments

One long-standing challenge in robotics is the realization of mobile autonomous robots able to operate safely in human workplaces, and be accepted by the human occupants. We describe the development of a multi-ton robotic forklift intended to operate alongside people and vehicles, handling palletized materials within ex-isting, active outdoor storage facilities.The system has four novel characteristics. The first is a multimodal interface that allows users to efficiently convey task-level commands to the robot using a combination of pen-based gestures and natural lan-guage speech. These tasks include the manipulation, transport, and placement of palletized cargo within dynamic, human-occupied warehouses. The second is the robot’s ability to learn the visual identity of an object from a single user-provided example and use the learned model to reliably and persistently detect objects de-spite significant spatial and temporal excursions. The third is a reliance on lo-cal sensing that allows the robot to handle variable palletized cargo and navigate within dynamic, minimally-prepared environments without GPS. The fourth con