Cooperative path planning and cooperative perception for UAVs swarm

In this research Pythagorean Hodograph based path planning and camera based cooperative perception are investigated separately and then these two entirely separate areas (Path Planning and Perception) are integrated for the application in online pop-up obstacle locating & avoidance and moving target tracking & surveillance in dynamic environments. The path planning is integrated with the cooperative perception to deal with the challenges posed by the dynamic environment. The aim of this integration is to achieve maximum autonomy required to execute a mission autonomously by multiple fixed wings UAVs in a dynamic environment. During the mission execution, the cooperating UAVs start from some initial location in the operating environment and finish at some final location while trying to achieve the mission’s objectives in a cooperative way. Naturally planning a feasible (safe and flyable) path for each participating UAV from initial position to a final location becomes a compulsory task of mission planning. For fixed wing UAVs flyable paths mean, paths which have tangential and curvature continuity and which obey the kinematic and dynamic constraint of the UAVs. In this research an algorithm based on Pythagorean hodograph curves is developed and used for planning feasible (safe and flyable) paths. The Pythagorean hodograph (PH) yields paths of exact length having tangential and curvature continuity. These continuous paths are made flyable for the UAVs by imposing the kinematic constraints of the UAVs. These constraints are imposed by the curvature and torsion manipulation of the planned paths. The safety of these paths is ensured by making it free of inter collisions between the vehicles and collisions with the known obstacles. These feasible paths are known as the initial paths or reference trajectories. In this research the operating environment is assumed to be dynamic in which changes are taking place at all times. Each UAV taking part in the mission is equipped with a vision sensor to perceive these changes continuously in a cooperative way. As the mission is assumed to be executed in day light, therefore light intensity video camera is used as a vision sensor. A perception algorithm for locating an object cooperatively in 3D is developed in this research. This algorithm is based on the optimization of errors in target position acquired by the on board camera. The algorithm is used by the cooperative perception system for optimal position estimation of the object in the scene. The target position information between the participating UAVs is exchanged through wireless communication for data fusion purposes. After developing efficient algorithms for path planning and cooperative perception, the two algorithms are integrated to be used in reactive obstacle avoidance and target tracking. During the mission, when the UAVs start their flight on the reference trajectories generated by the path planning algorithm, the perception algorithm comes into action. During the travel on these paths if the perception system of any of the UAVs detects an interrupting obstacle which was not known a priori in the map, then the exact location of this obstacle is determined with the help of the perception algorithm in a cooperative way. Using the location of the interrupting obstacle determined by the perception algorithm the path planning algorithm plans an evasive manoeuvre for the corresponding UAV to avoid it. After avoiding the obstacle the UAV comes back to its reference trajectory as soon as possible. In the operation of surveillance and tracking during the mission, the onboard perception algorithm locates an object of interest dynamically and the Pythagorean hodograph (PH) path planner uses this location to generate the paths for the cooperating UAVs to keep in close proximity of the target. In this case the close proximity of the target means to follow the moving target in such a way that it remains in the fields of views of the UAVs cameras at any time. By this integration of path planning and cooperative perception the continuous surveillance and tracking of the target was made possible even when the individual UAV experiences failure. During this research the mid flight obstacle locating & avoidance, and target surveillance & tracking have been successfully achieved by the integration of the path planning and cooperative perception. The purpose of this integration is to achieve an enhanced autonomy for the cooperating group of UAVs to increase the probability of their survival in mission being executed in dynamic environments