Human-In-The-Loop Planning and Control for Non-Prehensile Manipulation in Clutter

This thesis presents motion planning and control algorithms to tackle the problem of Reaching Through Clutter (RTC). I consider problems where a robot needs to reach in cluttered environments, like a fridge or a shelf, to retrieve a goal object. The robot considers non-prehensile manipulation and is, therefore, required to interact with objects and push them out of the way to create space. Reaching Through Clutter is in the NP-Hard class and is, therefore, a computationally challenging problem. This is due to several problems; the state space is of high dimensionality, the system is under-actuated, and such manipulation requires physics-reasoning through a physics simulator, which is computationally expensive to run. All of these are motion planning challenges, but beyond motion planning, physics-uncertainty poses challenges when executing a valid solution in the real-world. That is, a valid trajectory in simulation could be invalid during execution due to physics prediction errors. I focus on all of these challenges and propose algorithms with a human-in-the-loop. The systems I propose in this thesis focus on minimal human input that results in significantly higher success rates and faster planning times. Initially, I employed kinodynamic sampling-based planners, like kinodynamic RRT and KPIECE, and propose a framework with a human-in-the-loop. The results suggest that human input is effective, and the framework outperforms the baselines both in success rate and planning times. However, the solutions suffer from lengthy and noisy trajectories, which causes trajectories to fail in the real-world due to the physics uncertainty problem. To address these shortcomings, I proposed a stochastic trajectory optimization-based planner with online-replanning that significantly improves the quality of the trajectories, the success rate in the real-world, and the planning times. Finally, throughout this thesis, I argue that human time is valuable, and propose approaches that allow a single human operator to guide up to twenty robots simultaneously using a predictive approach. This approach predicts future optimization costs and employs human help earlier for robots that deal with hard instances of the problem