Fast models and reinforcement learning control of unsteady aerodynamics

Large amplitude flow disturbances and gusts can drastically alter the aerodynamic forces on airfoils or structures. The modeling and control of the aerodynamic response of the flow around the body is complicated by the inherent nonlinearities and high dimensionality of this system, which increase the cost and complexity of the numerical tools that enable this. This work introduces two numerical tools for the efficient modeling of unsteady aerodynamic flows and explores the use of deep reinforcement learning to perform airfoil pitch control during flow disturbances. The first numerical tool we introduce is a grid-based potential flow solver. We focus on the discrete streamfunction and model the flow around bodies and point vortices immersed in a cartesian grid. This tool reformulates the inviscid vortex-in-cell method as a saddle-point problem where constraints such as the no-flow-through and Kutta condition can be added as Lagrange multipliers. The second numerical tool uses a potential flow to model wind tunnel walls and irrotational wind tunnel gust generators in a viscous flow simulation. This technique allows us to still accurately model the flow around the test subject while accounting for the effects of the wind tunnel in a way that doesn't drastically increase the cost of the simulation. Lastly, in our explorative study of reinforcement learning of airfoil pitch control, we try out the training of a control policy in a classical unsteady aerodynamics environment and a viscous, low-Reynolds number flow environment to minimize the lift variations caused by flow disturbances and compare the performance of controllers, or agents, that observe different types of information about the states of the system