A Massively Parallel Unstructured Overset Method for DNS/LES of Moving Bodies in Turbulent Flows

University of Minnesota Ph.D. dissertation. December 2018. Major: Aerospace Engineering and Mechanics. Advisor: Krishnan Mahesh. 1 computer file (PDF); ix, 86 pages.An unstructured overset method capable of performing direct numerical simulation (DNS) and large eddy simulation (LES) of many (O(10^5)) moving bodies, utilizing many computational cores (O(10^5)), in turbulent, incompressible fluid flow is presented. Unstructured meshes are attached to bodies and placed within a fixed background domain. Body meshes are allowed to arbitrarily overlap and move throughout the domain. In order to connect overset meshes, elements exterior to the solution domain must be removed from the simulation. In regions with many overlapping meshes, elements must be selectively removed to reduce redundancy while maintaining a solution over the entire domain. Around masked regions interpolation partner pairing is required between meshes to provide boundary conditions. For general unstructured meshes, these steps involve challenging computational geometry calculations which must be efficient and automatic. For many moving meshes, each step must be massively parallel and scale to large numbers of computational cores. To establish communication patterns a parallelized master/slave algorithm is used which minimizes global communication and storage. To remove elements a parallel ‘Forest Fire’ flood-fill algorithm is used to set a masking variable. For interpolation partner pairing, and other necessary searches, k-dimensional tree data structures (k-d trees) are extensively used. Often in a calculation, the connectivity between overset meshes remains largely the same between time steps. The temporal coherence of the various objects in the connectivity calculation is directly used to only update necessary information with time, resulting in substantial cost savings. Details of the different algorithms are presented. Resulting connectivity and timings are shown for complex geometries. Parallel scaling is demonstrated for 100,000 spherical particles within a channel up to 492,000 processors. Within each mesh a high resolution, unstructured, non-dissipative finite volume method is used to solve for the flow field. Boundary conditions for each mesh are provided by interpolation from flow solutions on overlapping meshes. When many unstructured meshes of different resolution overlap, care is required in the connection between the different flow solutions. An interpolant is created which seeks to preserve volume conservation of flow quantities between meshes regardless of overlapping mesh differences. An implicit fractional step method is used for time advancement, requiring the calculation of a predicted fluid velocity and corrector pressure field. For the predictor step, the resulting interpolation is directly introduced into the implicit equations for the predicted flow field. For the corrected pressure field, the continuity between meshes is weakly enforced using a penalty formulation. The pressure formulation is symmetric, positive-definite and non-singular resulting in a formulation which is readily solvable using traditional iterative matrix inversion techniques.An Arbitrary Euler-Lagrangian (ALE) method coupled to a 6 degrees of freedom rigid body equation system (6-DOF) is used for body motion. For rotation, a quaternion representation is used to solve Euler’s equations of rigid body motion. A linear spring damper model, which uses geometry information readily available from the overset assembly process, is used for collisions. Validation of the method for canonical flow fields is presented including assessment of order of accuracy and kinetic energy conservation properties. Empty over- set patches in canonical flows and particle-resolved direct numerical simulation (PR-DNS) validation of single particles in various flow fields are presented. LES results of a P4381 propeller under crashback conditions are shown to show the methods capability to simulate highly unsteady turbulent flows over complex, moving geometry. The physics of particles in different fluid flows are analyzed using the overset method. The motion of solitary particles settling in quiescent fluid is first assessed. There is disagreement in previous work on the type of trajectories and the physical parameters at which they occur. A specific region of disagreement for heavy particles at moderate Reynolds number is investigated. It is found that the overset simulations reproduce experimental findings of Horowitz and Williamson [1] rather than other work. To the best of our knowledge, this is the first time a numerical study has reproduced their results. Many particles in a vertical turbulent channel flow are then studied. The particles are heavy and in the two-way coupled regime. Using the overset simulations it is found that the particles move with significantly different velocities than the fluid over large regions of the channel. The turbulent kinetic energy of the channel, and Reynolds stresses are noticeably impacted by the presence of the particles. A higher concentration of particles is observed near the channel walls, with a nearly uniform distribution occurring near the channel centerline. Collisions are found to largely occur between particles and the walls of the channel. The results for the flow and particle statistics are consistent previous numerical and experimental findings for sim- ilar turbulent channel cases. The channel simulations illustrate the method’s capability to perform large-scale simulations of turbulent fluid flow when many moving bodies are present