Natural Language Descriptions of Human Behavior from Video Sequences

Abstract. This contribution addresses the generation of textual de-scriptions in several natural languages for evaluation of human behavior in video sequences. The problem is tackled by converting geometrical in-formation extracted from videos of the scenario into predicates in fuzzy logic formalism, which facilitates the internal representations of the con-ceptual data and allows the temporal analysis of situations in a deter-ministic fashion, by means of Situation Graph Trees (SGTs). The results of the analysis are stored in structures proposed by the Discourse Rep-resentation Theory (DRT), which facilitate a subsequent generation of natural language text. This set of tools has been proved to be perfectly suitable for the specified purpose.