A systematic study of the factorization of long-range azimuthal two-particle correlations into a product of single-particle anisotropies is presented as a function of $p_\mathrm{T}$ and $\eta$ of both particles, and as a function of the particle multiplicity in PbPb and pPb collisions. The data were taken with the CMS detector for PbPb collisions at $\sqrt{s_{\mathrm{NN}}}$ = 2.76 TeV and pPb collisions at $\sqrt{s_{\mathrm{NN}}}$ = 5.02 TeV, covering a very wide range of multiplicity. Factorization is observed to be broken as a function of both particle $p_\mathrm{T}$ and $\eta$. When measured with particles of different $p_\mathrm{T}$, the magnitude of the factorization breakdown for the second Fourier harmonic reaches 20% for very centra...