There are two difficulties with the implementation of the characteristic function-based estimators. First, the optimal instrument yielding the ML efficiency depends on the unknown probability density function. Second, the need to use a large set of moment conditions leads to the singularity of the covariance matrix. We resolve the two problems in the framework of GMM with a continuum of moment conditions. A new optimal instrument relies on the double indexing and, as a result, has a simple exponential form. The singularity problem is addressed via a penalization term. We introduce HAC-type estimators for non-Markov models. A simulated method of moments is proposed for non-analytical cases