On the initial conditions of scalar and tensor fluctuations in

We have considered the perturbation equations governing the growth of fluctuations during inflation in generalized scalar tensor theory $$f(R,\phi )$$. We have found that the scalar metric perturbations at very early times are negligible compared to the scalar field perturbation, just like general relativity. At sufficiently early times, when the physical momentum of perturbation mode, q / a is much larger than the Hubble parameter H, i.e. $$q/a\gg H$$, we have obtained the metric and scalar field perturbation in the form of WKB solutions up to an undetermined coefficient. Then we have quantized the scalar fluctuations and expanded the metric and the scalar field perturbations with the help of annihilation and creation operators of the scalar field perturbation. The standard commutation relations of annihilation and creation operators fix the unknown coefficient. Going over to the gauge invariant quantities which are conserved beyond the horizon, we have obtained the initial condition of the generalized Mukhanov–Sasaki equation. Then a similar procedure is performed for the case of tensor metric perturbation. As an example of the generalized Mukhanov–Sasaki equation and its initial condition, we have proposed a power-law functional form as $$f(R,\phi )=f_0 R^m \phi ^n$$ and obtained an exact inflationary solution. In this background, then we have discussed how the scalar and tensor fluctuations grow