This paper presents a framework for parallel tracking of human hands and faces in real time, and is a partial solution to a larger project on human-robot interaction which aims at training autistic children using a humanoid robot in a realistic non-restricted environment. In addition to the framework, the results of tracking different hand waving patterns are shown. These patterns provide an easy to understand profile of hand waving, and can serve as the input for a classification algorithm