Multivariable Orthogonal Polynomials as Coupling Coefficients for Lie and Quantum Algebra Representations

The main topic of the thesis is the connection between representation theory and special functions. We study matrix elements, coupling coefficient, and recoupling coefficients for the simplest Lie and quantum groups. We show that a large number of multivariable orthogonal and biorthogonal polynomials occurring in the literature may be obtained as coupling coefficients (generalized Clebsch-Gordan coefficients) for multiple tensor products of highest weight representations of the group SU(1,1). In many cases such polynomials have appeared in applications (physics and statistics), and they are also connected with sperical harmonics. The algebraic interpretation yields a simple and unified approach to the study of these polynomials. The corresponding theory can be developed for the group SU(2) and the oscillator group. Our original motivation came from ``Hankel theory'', more precisely from the higher order Hankel operators introduced by Svante Janson and Jaak Peetre. The Fourier kernels of such operators are Clebsch-Gordan coefficients, and similarly multivariable coupling coefficients are Fourier kernels of certain multilinear forms. We obtain a Schatten class criterion for these higher order Hankel forms. We give two new proofs of the triple sum formula for Wigner 9j-symbols. These are recoupling coefficients for four-fold tensor products, and appear in the theory of angular momentum in quantum mechanics. We show that general Askey-Wilson and q-Racah polynomials arise as matrix elements for the SU(1,1) and SU(2) quantum group, respectively. To obtain this interpretation we introduce some new generalized group elements which include the quantum Weyl element as a degenerate case. We also consider coupling coefficients in the quantum group case. These are multivariable generalizations of the q-Racah and Askey-Wilson polynomials; however, we focus on the more elementary case of multivariable q-Hahn polynomials. We prove a binomial formula for two variables satisfying a quadratic relation