Studies of turbulence structure and turbulent mixing using petascale computing

A large direct numerical simulation database spanning a wide range of Reynolds and Schmidt number is used to examine fundamental laws governing passive scalar mixing and turbulence structure. Efficient parallel algorithms have been developed to calculate quantities useful in examining the Kolmogorov small-scale phenomenology. These new algorithms are used to analyze data sets with Taylor scale Reynolds numbers as high as 650 with grid-spacing as small as the Kolmogrov length scale. Direct numerical simulation codes using pseudo-spectral methods typically use transpose based three-dimensional (3D) Fast Fourier Transforms (FFT). The ALLTOALL type routines to perform global transposes have a quadratic dependence on message size and typically show limited scaling at very large problem sizes. A hybrid MPI/OpenMP 3D FFT kernel has been developed that divides the work among the threads and schedules them in a pipelined fashion. All threads perform the communication, although not concurrently, with the aim of minimizing thread-idling time and increasing the overlap between communication and computation. The new algorithm is seen to reduce the communication time by as much as 30% at large core-counts, as compared to pure-MPI communication. Turbulent mixing is important in a wide range of fields ranging from combustion to cosmology. Schmidt numbers range from O(1) to O(0.01) in these applications. The Schmidt number dependence of the second-order scalar structure function and the applicability of the so-called Yaglomﾒs relation is examined in isotropic turbulence with a uniform mean scalar gradient. At the moderate Reynolds numbers currently achievable, the dynamics of strongly diffusive scalars is inherently different from moderately diffusive Schmidt numbers. Results at Schmidt number as low as 1/2048 show that the range of scales in the scalar field become quite narrow with the distribution of the small-scales approaching a Gaussian shape. A much weaker alignment between velocity gradients and principal strain rates and a strong departure from Yaglomﾒs relation have also been observed. Evaluation of different terms in the scalar structure function budget equation assuming statistical stationarity in time shows that with decreasing Schmidt number, the production and diffusion terms dominate at the intermediate scales possibly leading to non-universal behavior for the low-to-moderate Peclet number regime considered in this study. One of the few exact, non-trivial results in hydrodynamic theory is the so-called Kolmogorov 4/5th law. Agreement for the third order longitudinal structure function with the 4/5 plateau is used to measure the extent of the inertial range, both in experiments and simulations. Direct numerical simulation techniques to obtain the third order structure structure functions typically use component averaging, combined with time averaging over multiple eddy-turnover times. However, anisotropic large scale effects tend to limit the inertial range with significant variance in the components of the structure functions in the intermediate scale ranges along the Cartesian directions. The net result is that the asymptotic 4/5 plateau is not attained. Motivated by recent theoretical developments we present an efficient parallel algorithm to compute spherical averages in a periodic domain. The spherically averaged third-order structure function is shown to attain the K41 plateau in time-local fashion, which decreases the need for running direct numerical simulations for multiple eddy-turnover times. It is well known that the intermittent character of the energy dissipation rate leads to discrepancies between experiments and theory in calculating higher order moments of velocity increments. As a correction, the use of three-dimensional local averages has been proposed in the literature. Kolmogorov used the local 3D averaged dissipation rate to propose a refined similarity theory. An algorithm to calculate 3D local averages has been developed which is shown to scale well up to 32k cores. The algorithm, computes local averages over overlapping regions in space for a range of separation distances, resulting in N^3 samples of the locally averaged dissipation for each averaging length. In light of this new calculation, the refined similarity theory of Kolmogorov is examined using the 3D local averages at high Reynolds number and/or high resolution.Ph.D