Several real-world problems (e.g., in bioinformatics/proteomics, or in recognition of video sequences) can be described as classification tasks over sequences of structured data, i.e. sequences of graphs, in a natural way. This paper presents a novel machine that can learn and carry out decision-making over sequences of graphical data. The machine involves a hidden Markov model whose state-emission probabilities are defined over graphs. This is realized by combining recursive encoding networks and constrained radial basis function networks. A global optimization algorithm which regards to the machine as a unity (instead of a bare superposition of separate modules) is introduced, via gradient-ascent over the maximum-likelihood criterion with...