Cognitive SystemsH.I. Christensen, A. Sloman, G-J. Kruijff & J. Wyatt (Eds.)January, 2009Quotes from the CoSy Science AdvisorsWhile there is still a long way to come close to the objective of the Eu-ropean Commission Cognitive Systems initiative “to construct physi-cally instantiated or embodied systems that can perceive, understand,. . . and interact with their environments and evolve in order to achievehuman-like performance” this book is about one of the funded projectsin this initiative. It gives and excellent insight into the challenges andbenefits of working in an large interdisciplinary team to better under-stand the human mind and in order to build intelligent machines.Heinrich Bülthoff, MPIKOne of the great challenges of the 21st century is to build a robotthat can perceive and act within its environment and communicatewith people, while also exhibiting the cognitive capabilities that leadto performance like that of people. This book reports on the EuropeanUnion project on Cognitive Systems. It offers detailed explanations ofthe exciting progress made on this challenge and serves as a foundationfor the science of Cognitive Systems in the next part of this century.Candance Sidner, BAE SystemsPrefaceThe present volume is a report on the research results generated by the project“Cognitive Systems for Cognitive Assistants” (CoSy), which was sponsoredby the European Commission during the period 2004-2008.The CoSy team was assembled to study the problem of embodied cog-nitive systems for domestic service tasks such as guidance, fetch and carry,etc. The main aim of the project has been to study the core technologiesneeded to build such systems rather than mere application development. Thekey competencies needed are: systems architectures, scalable knowledge rep-resentation, adaptive embodiment,categorical perception, planning and errorrecovery, learning, and situated dialog systems. All of these aspects were stud-ied in the context of CoSy and exemplified using two “demonstrator scenarios”that were conceived to allow studies / evaluation in an integrated context.The volume is organized into 4 parts. The introduction outlines the over-all problem domain and the CoSy approach to the problem. The second partcontains a number of chapters that detail progress on topical problems acrossarchitectures, perception, learning, planning and dialog systems. These com-petencies were integrated into systems as described in the third part of thebook. The final section provides a perspective on the results obtained andconsiders some possible issues for future research.The project has published extensively throughout its life and links to publi-cations can be found at the project web facility www.cognitivesystems.org,where copies of associated project deliverables also can be retrieved. The CoSyweb facility contains also a page with material that supplements the book. Thepage has pointers to published material, associated datasets, videos and opensoftware. The electronic version of the book also has embedded links to theweb facility and published papers. I.e., referenced material published by theconsortium can be accessed through embedded links.The consortium would like to express our gratitude for the support the Eu-ropean Commission has provided for this research. In addition we are gratefulfor the guidance and feedback we have received from our scientific advisors:Prof. Heinrich Bülthoff - MPIK, Prof. Benjamin Kuipers - UT Austin, Dr.VIII PrefaceCandy Sidner - BAE Systems. We are also grateful for the support from theproject reviewers: Prof. Igor Alexander - Univ. College London, Prof. MarkSteedman - Univ. of Edinburgh, Prof. John Tsotsos - York Univ. and Prof.Oliver Brock, UMASS. Finally, we appreciate the guidance from the associ-ated EU project officer Cecile Huet.Atlanta, Saarbrücken & Birmingham Henrik I. ChristensenJanuary 2009 Geert-Jan KruijffAaron SlomanJeremy WyattContentsPart I Introduction1 Cognitive Systems IntroductionHenrik I. Christensen, Aaron Sloman, Geert-Jan Kruijff, Jeremy L.Wyatt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Part II Component Science2 Architecture and RepresentationsNick Hawes, Jeremy L Wyatt, Aaron Sloman, MohanSridharan, Richard Dearden, Henrik Jacobsson, Geert-Jan Kruijff . . . . 533 The Sensorimotor Approach in CoSy: The Example ofDimensionality ReductionDavid Philipona, J. Kevin O’Regan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 974 Categorical PerceptionMario Fritz, Mykhaylo Andriluka, Sanja Fidler, Michael Stark, AlesLeonardis, Bernt Schiele . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1355 Semantic Modelling of SpaceAndrzej Pronobis, Patric Jensfelt, Kristoffer Sjöö, HendrikZender, Geert-Jan M. Kruijff, Oscar Martinez Mozos, WolframBurgard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1696 Planning and Failure DetectionMichael Brenner, Christian Plagemann, Bernhard Nebel, WolframBurgard, Nick Hawes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2277 Multi-modal LearningX ContentsDanijel Skočaj, Matej Kristan, Alen Vrečko, Aleš Leonardis, MarioFritz, Michael Stark, Bernt Schiele, Somboon Hongeng, Jeremy L.Wyatt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2698 Situated Dialogue Processing for Human-Robot InteractionGeert-Jan M. Kruijff, Pierre Lison, Trevor Benjamin, HenrikJacobsson, Hendrik Zender, Ivana Kruijff-Korbayová . . . . . . . . . . . . . . . . . 315Part III Integration and Systems9 The PlayMate SystemNick Hawes, Jeremy Wyatt, Aaron Sloman, Mohan Sridharan, MarekKopicki, Somboon Hongeng, Ian Calvert, Geert-Jan Kruijff, HenrikJacobsson, Michael Brenner, Danijel Skočaj, Alen Vrečko, NikodemMajer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37310 The Explorer SystemKristoffer Sjöö, Hendrik Zender, Patric Jensfelt, Geert-Jan M.Kruijff, Andrzej Pronobis, Nick Hawes, Michael Brenner . . . . . . . . . . . 40111 Lessons Learnt from Scenario-Based IntegrationNick Hawes, Michael Zillich, Patric Jensfelt, . . . . . . . . . . . . . . . . . . . . . 429Part IV Summary & Outlook12 Cross-Disciplinary Reflections: Philosophical RoboticsAaron Sloman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44713 Lessons and OutlookHenrik I. Christensen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493List of ContributorsMykhaylo AndrilukaTU DarmstadtMultimodal Interactive SystemsHochschulstrasse 10D-64289 Darmstadt, Germanyandriluka@cs.tu-darmstadt.deTrevor BenjaminDFKI GmbHSaarbrücken, GermanyMichael BrennerAlbert-Ludwigs Universität FreiburgDepartment of Computer ScienceFreiburg, Germanybrenner@informatik.uni-freiburg.deWolfram BurgardAlbert-Ludwigs Universität FreiburgDepartment of Computer ScienceFreiburg, Germanyburgard@informatik.uni-freiburg.deIan CalvertUniversity of BirminghamSchool of Computer ScienceEdgbastonBirmingham, B15 2TT UKHenrik I. ChristensenGeorgia Institute of TechnologyRobotics and Intelligent MachinesAtlanta, GA 30332-0280hichristensen@gmail.comRichard DeardenUniversity of BirminghamSchool of Computer ScienceEdgbastonBirmingham, B15 2TT UKrwd@cs.bham.ac.ukSanja FidlerUniversity of LjubljanaFaculty of Computer and InformationScienceVisual Cognitive Systems LaboratoryTrzasaka 25SE-1001 Ljubljana, Sloveniasanja.fidler@fri.uni-lj.siMario FritzTU DarmstadtMultimodal Interactive SystemsHochschulstrasse 10D-64289 Darmstadt, Germanyfritz@cs.tu-darmstadt.deXII List of ContributorsNick HawesUniversity of BirminghamSchool of Computer ScienceEdgbastonBirmingham, B15 2TT UKnah@cs.bham.ac.ukSomboon HongengUniversity of BirminghamSchool of Computer ScienceEdgbastonBirmingham, B15 2TT UKS.Hongeng@cs.bham.ac.ukHenrik JacobssonDFKI GmbHSaarbrücken, Germanyhenrikj@dfki.dePatric JensfeltRoyal Institute of Technology (KTH)Center for Autonomous SystemsSE-100 44 Stockholm, Swedenpatric@csc.kth.seMarek KopickiUniversity of BirminghamSchool of Computer ScienceEdgbastonBirmingham, B15 2TT UKmzs@cs.bham.ac.ukMatej KristanUniversity of LjubljanaFaculty of Computer and InformationScienceVisual Cognitive Systems LaboratoryTrzasaka 25SE-1001 Ljubljana, Sloveniamatej.kristan@fri.uni-lj.siGeert-Jan KruijffDFKI GmbHSaarbrücken, Germanygj@dfki.deIvana Kruijff-KorbayovaDFKI GmbHSaarbrücken, Germanyivana.kruijff@dfki.deAles LeonardisUniversity of LjubljanaFaculty of Computer and InformationScienceVisual Cognitive Systems LaboratoryTrzasaka 25SE-1001 Ljubljana, Sloveniaales.leonardis@fri.uni-lj.siPierre LisonDFKI GmbHSaarbrücken, GermanyPierre.Lison@dfki.deNikodem MajerTU DarmstadtMultimodal Interactive SystemsHochschulstrasse 10D-64289 Darmstadt, Germanymajer@cs.tu-darmstadt.deOscar Martinez MozosAlbert-Ludwigs Universität FreiburgDepartment of Computer ScienceFreiburg, Germanyomartine@informatik.uni-freiburg.deBernhard NebelAlbert-Ludwigs Universität FreiburgDepartment of Computer ScienceFreiburg, Germanynebel@informatik.uni-freiburg.deJ. Kevin O’ReganLaboratoire Psychologie de laPerception,Université Paris Descartes andCNRSParis, Francejkrevin.oregan@gmail.comList of Contributors XIIIDavid PhiliponaLaboratoire Psychologie de laPerception,Université Paris Descartes andCNRSParis, FranceChristian PlagemannAlbert-Ludwigs Universität FreiburgDepartment of Computer ScienceFreiburg, Germanyplagem@informatik.uni-freiburg.deAndrzej PronobisRoyal Institute of Technology (KTH)Center for Autonomous SystemsSE-100 44 Stockholm, Swedenpronobis@csc.kth.seBernt SchieleTU DarmstadtMultimodal Interactive SystemsHochschulstrasse 10D-64289 Darmstadt, Germanyschiele@cs.tu-darmstadt.deKristoffer SjööRoyal Institute of Technology (KTH)Center for Autonomous SystemsSE-100 44 Stockholm, Swedenkrsj@csc.kth.seDanijel SkocajUniversity of LjubljanaFaculty of Computer and InformationScienceVisual Cognitive Systems LaboratoryTrzasaka 25SE-1001 Ljubljana, Sloveniadanijel.skocaj@fri.uni-lj.siAaron SlomanUniversity of BirminghamSchool of Computer ScienceEdgbastonBirmingham, B15 2TT UKaxs@cs.bham.ac.ukMohan SridharanTexas Tech at Abilene302 Pine StreetAbilene, TX 79601, USAmohan.sridharan@ttu.eduMichael StarkTU DarmstadtMultimodal Interactive SystemsHochschulstrasse 10D-64289 Darmstadt, Germanystark@cs.tu-darmstadt.deJeremy WyattUniversity of BirminghamSchool of Computer ScienceEdgbastonBirmingham, B15 2TT UKjlw@cs.bham.ac.ukHenrik ZenderDFKI GmbHSaarbrücken, Germanyzender@dfki.dePart IIntroduction1Cognitive Systems IntroductionHenrik I. Christensen1, Aaron Sloman2, Geert-Jan Kruijff3, Jeremy L.Wyatt21 Robotics and Intelligent Machines, Georgia Institute of Technology, Atlanta, Ga.USA hic@cc.gatech.edu2 Intelligent Robotics Lab, School of Computer Science, University ofBirmingham, Birmingham, UK, {axs,jlw}@cs.bham.ac.uk3 DFKI GmbH, Saarbrücken, Germany, gj@dfki.de1.1 IntroductionThe CoSy project was setup under the assumption that the visionary FP6objective“To construct physically instantiated ... systems that can perceive, un-derstand ... and interact with their environment, and evolve in or-der to achieve human-like performance in activities requiring context-(situation and task) specific knowledge”is far beyond the state of the art and will remain so for many years. Fromthis vision several intermediate targets were defined. Achieving these targetswould provide a launch pad for further work on the long term vision.In particular it has been an objective to advance the science of cogni-tive systems through a multi-disciplinary investigation of requirements, de-sign options and trade-offs for human-like, autonomous, integrated, physical(e.g. robot) systems, including requirements for architectures, for forms ofrepresentation, for perceptual mechanisms, for learning, planning, reasoning,motivation, action, and communication.To validate science progress a succession of increasingly ambitious workingsystems are constructed to test and demonstrate the ideas. Devising demand-ing but achievable test scenarios, including scenarios in which a machine notonly performs some task but shows that it understands what it has done, andwhy, is an integral part of the empirical study of cognitive systems.In this chapter the basic objectives, expected results and organization ofthe project will be presented, whereas the remainder of the book presentresults that have been obtained during the CoSy project. The final chaptersof the book will provide reflections on progress in terms of new insight andmajor lessons.4 Christensen et al.1.2 Objective of project1.2.1 The problemDespite impressive progress in many specific sub-topics in AI and CognitiveScience, the field as a whole moves slowly. Most systems able to performcomplex tasks that humans and other animals can perform easily, for instancerobot manipulators, or intelligent advisers, have to be carefully crafted. What-ever intelligence they have could be described as ‘insect-like’ insofar as theyhave capabilities that they do not understand, they do not know why they dothings one way rather than another, they cannot explain what they are doing,they cannot improve their performance by taking advice from a human, andthey cannot give advice or help to someone else doing similar tasks. Part ofthe reason for this is that over the last few decades research has become frag-mented: with many individuals and research teams focusing their efforts onnarrowly defined problems in vision, or learning, or language understanding,or problem solving, or mobile robotics, for instance.1.2.2 The way forwardA key part of the CoSy effort has been to try to overcome some of theselimitations by using ideas from relevant disciplines to investigate an ambitiousvision of a highly competent robot, combining many different capabilities in acoherent manner, for instance a subset of the capabilities of a typical human4-5 year old child. The scientific importance of this objective is that sucha robot requires generic capabilities providing a platform for many differentsorts of subsequent development, since a child of that age can develop in anyhuman culture and benefit from many forms of education. However, we donot underestimate the profound difficulties of this challenge.The research makes use of and feeds results into the various componentdisciplines of AI and cognitive science, for instance, new results on percep-tion, learning, reasoning, language processing, memory, plan execution, andstudies of motivation and emotion. Perhaps more importantly: the projectnot only benefits from other disciplines but has also tried to provide newsubstantive contributions to those disciplines in the form of new theories andworking models. The detailed tasks of developing working systems generatenew research questions for the contributing disciplines.1.2.3 Steps to successThe goal of producing a robot with many of the capabilities of a human childis unrealistic for a five year research project: it is an significant long termchallenge. However, by analysing the many requirements for moving in thatdirection, one can derive sets of successively less challenging sub-goals that1 Cognitive Systems Introduction 5provide steps towards the distant goal. Some of these sub-goals are achievablein the time-frame of the project and form the main deliverables of the project.An important part of the effort has been to consider two main kinds ofdeliverables: theory and implementation.1. Theory deliverables:A body of theory, at different levels of abstraction, regarding require-ments, architectures, forms of representation, kinds of ontologies, typesof reasoning, kinds of knowledge, and varieties of mechanisms relevantto embodied, integrated, multi-functional intelligent systems. The resultsare expected to be useful both for enhancing scientific understanding ofnaturally occurring intelligent systems (e.g. humans and other animals)and for the design of artificial intelligent systems.The theory results are built around the core idea of a self-modifying archi-tecture comprising different sorts of capabilities which develop over time.The results cover both analysis of requirements for such an architectureand also design options with their trade-offs.Key ideas for the architecture are informed by biological considerations,e.g. the notion of an architecture combining components which developedat different evolutionary epochs, and which in humans operate concur-rently, performing different categories of tasks, for instance:• reactive components controlling the details of execution of skilled be-haviours (these are evolutionarily old and use mechanisms shared withmany species)• deliberative components supporting thought and reasoning about whatmight happen next or what should be done at some later stage (theseare found in fewer species and require highly specialised mechanismsand forms of representation – including human language in some cases)• self-reflective, meta-management components that monitor, evaluate,and (partially) control and redirect the reactive and deliberative pro-cesses (these are probably rare in animals because they require the abil-ity to represent, compare and evaluate information-processing tasksand strategies, as opposed to merely implementing them).The requirements for perceptual and motor systems that operate concur-rently with, and in close coordination with, processes in all the differentarchitectural layers are analysed and architectures are proposed for bothperception (e.g. [134]) and action mechanisms.Learning processes are different within and between different parts of thearchitecture and current theories of learning have to be substantially ex-tended to explain how, for instance (a) kinds of learning that extend theindividual’s ontology for perceiving and thinking about the environment,and (b) kinds of learning that develop fluency and speed in motor per-formance, e.g. because the reactive layer is trained by processes in thedeliberative layer.6 Christensen et al.Different varieties of communication and social interaction are related tothe different architectural layers: for instance, (a) dancing, fighting andmoving heavy objects require coupled reactive systems; (b) linked col-laborative actions spanning spatial and temporal gaps, e.g. in buildinghouses and bridges, require deliberative capabilities; (c) the ability to em-pathise, exhort, persuade, may require extensions of self-understandingin the meta-management system to support other-understanding. (All ofthese influences can go both ways: e.g. meeting requirements for socialdevelopments may enhance individual capabilities.)As different sorts of architectures with these general features are possibleit is important to consider an analysis of architectural options and trade-offs.2. Implementation deliverables:Several implementations of working systems demonstrating successful ap-plication of the theory, e.g. in a robot capable of performing a collectionof diverse tasks in a variety of scenarios described in the project, includingvisual and other forms of perception, learning, reasoning, and communi-cation (with another robot or a human) both in order to collaborate onsome task and in order to explain what it is doing and why. Our aim hasbeen to create robotic cognitive systems that have a kind of autonomythat will justify describing it as having its own goals, which may changeover time.The distinctive features of such a robot include integration of sub-functions (e.g. vision and other senses can be combined in making senseof a scene, vision can be used to disambiguate an utterance by looking atwhat the utterance refers to, and learning processes can enhance differentkinds of capabilities, including linguistic, visual, reasoning, planning, andmotor skills).Integration does not imply homogeneity, such as use of the same type ofrepresentation throughout the system. For instance, low level visual mech-anisms finding edge-features, optical flow patterns, etc. use different formsof representation from higher level mechanisms e.g. those recognizing anddescribing whole objects. Likewise, planning mechanisms will need differ-ent forms of representation from fine-grained motor control mechanisms.Nature vs. Nurture: A major research issue concerns how much shouldbe programmed into such a robot and how much will have to be learntby interacting with the environment, including teachers and other agents.There are several AI projects aiming to develop intelligent systems on thebasis of powerful and general learning mechanisms starting from some-thing close to a “Tabula rasa” (e.g. the COG project at MIT, the CyberlifeResearch “Lucy” Project, and the Dav project at Michigan State Univer-sity). Their hope is that the structure of the environment will cause thelearning mechanisms to induce all required information about the natureof the environment.1 Cognitive Systems Introduction 7Such projects are likely to be defeated by explosive search spaces requiringevolutionary time-scales for success.Biological evolution enables individuals to avoid this problem by providinglarge amounts of “innate” information in the genomes of all species. In thecase of humans this seems to include meta-level information about whatkinds of things are good to learn, helping to drive the learning processesas well as specific mechanisms, forms of representation, and architecturesto enable them to work.Although such debates are most commonly associated with requirementsfor language learning (e.g. [130] the issues are deep and general. For in-stance, Kant [70] argued two centuries ago that notions of space, time andcausation are presupposed by and cannot be learnt from perceptual expe-riences. Similar points could be made about notions of meaning, purposeand action.Instead of taking a dogmatic stance on what needs to be innate CoSy hasexplored various alternatives for amounts and types of innate knowledgeand included a first analysis of the trade-offs.1.3 A motivating exampleAt the core of the CoSy project have been the methods for construction ofembodied artifacts, such as robots, with advanced cognitive functions. Suchsystems should be endowed with facilities for automatic interpretation of theenvironment (in terms of mapping of the environment, recognition of a largenumber of objects, etc.), adaptive acquisition of new skills and tasks in co-operation with a human user, methods for advanced manipulation to allowthe system to perform extended missions on behalf of the users and reasoningmethods to ensure advanced autonomy. The interpretation facilities can beused both to ensure autonomy and to verbalise knowledge and mission pa-rameters to a user. Advanced service robots can be used for a large range oftasks in our everyday environment ranging from vacuuming to clearing the ta-ble after dinner. The tasks can also be in terms of mobility support for elderlyand handicapped. To be of true utility to an average citizen it is essential thatthe system has facilities for automatic adaptation to the environment of theuser, it must also be able to adapt to the habits of the owner, it must be ableto understand the importance and consequences of instructions, and it mustbe sufficiently flexible to learn new skills that it was not endowed with fromthe factory. An example scenario is outlined below.1. The Hartmut family decides to acquire a CoSy system to assist them intheir everyday life. A system is acquired from the local Robs-R-Us chainof stores2. Upon arrival at home the system is unpackaged and booted up. Uponwakeup the CoSy system greets the owners and asks them to give their8 Christensen et al.names (Helga and Oskar). The owners are asked to give it a name (Fido).The system then asks for a tour of the home. As they traverse the home itbuilds up a map of the house, and objects of particular interest are namedto the system (this is the refrigerator), ...3. When asked to clean the table after dinner, Fido already has a built-in strategy for cleaning of tables, but it needs to know the location ofthe dishwasher (Oskar shows him the dishwasher – a Bosch 4590). Uponrecognition of the dishwasher Fido downloads an operations manual fromthe Cosy corporate site. It then goes about its task.Oskar notices that Fido is removing cutlery and dishes but moves glassesto one side of the table. He asks: ‘Why are you leaving the glasses?’ Fidoanswers: ‘My movements can be clumsy, so I’ll put them in the dishwasherafter all the objects that might break them.’4. After dishes, cutlery, glasses and bottles have been cleared, a single objectis left on the table. Fido asks Helga: What am I to do about this objectleft on the table? This is an Orrefors vase please leave it on the table!Fido inquires if it is Ok to turn on the dishwasher.5. A few months later the Hartmut family acquires a new GE refrigerator.When Fido detects the new item, he inquires about the new objects. Heis told that it is a GE-Economy Fridge. Upon learning this he downloadsthe latest skills from the CoSy skill database to allow him to operate theunit in terms of opening of the door and placing objects on the shelves,etc.6. Later they replace the door to the bathroom with a sliding door, a newobject that Fido has never seen before. He asks Oskar to demonstrate tohim how to operate the door. Initially Oskar shows him how to operatethe door (learning by demonstration). Fido tries to replicate the actionand is corrected when he pushes to hard.7. One day Julie, the 8 year old grandchild of Helga and Oskar, comes tovisit. At first Julie is a little afraid of Fido and keeps her distance. Laterwhen she is alone with Fido she says to Fido: Can you play Ludo? Fidosays: What is Ludo? Julie: A game - I’ll teach you. Will you play? Fidounderstands the general category of game, does not know Ludo, but rea-sons that since Julie wants him to play, and he is not required for anythingelse at that time, he should oblige. He considers downloading informationabout Ludo from the Cosy corporate site, but reasons that if Julie offersto teach him she would probably like to do that, and he wants to pleaseher. So he does not download the information and says: Yes please. Showme what to do. They both enjoy the ensuing interaction.The above scenario is deliberately futuristic, but it does at the same timeillustrate “automatic mapping”, “recognition of objects”, “recognition of lim-itations in knowledge or skills”, “natural interaction using speech and ges-tures”, “never ending dialogues” to resolve ambiguities, and affective statesand processes.1 Cognitive Systems Introduction 9These are key issues that have been studied in the CoSy projects. Throughintegration of perception, action generation, autonomy, flexible user interfacesit has, however, been possible to approach the problem of building truly flex-ible cognitive systems that can be used to implement scenarios as outlinedabove.1.4 Organization of the research/Research QuestionsIn the study of cognitive systems for cognitive assistants a number of keyresearch challenges were identified: architectures, representations, learning,perception-action modelling, communication, and planning & failure handling.In addition to addressing these issues at a fundamental level there is alsoa need to consider the integration of the techniques into complete systems.Two scenarios were identified for the study of integrated systems: i) explo-ration/mapping of space, and ii) models of objects and concepts. Initiallywe will discuss the research challenges (Sections 1.4.1 – 1.4.7) followed by adiscussion of the integrated scenarios (Sections 1.4.9 – 1.4.10).The overall work has been organised according to two major milestones1. “Using intermodality and affordances for the acquisition of concepts, cat-egories and language”The goal of the first milestone was to develop an agent that was capableof exploring unknown spaces and objects in that space, learning represen-tations of spaces and objects based on a known ontology. The agent candetermine what it does not know (or about which it is not clear), andcan carry out a dialogue with another agent to learn more. Fundamen-tal to the latter effort was the acquisition of strategies for representing,coordinating and understanding multi-modal action & dialog acts.2. “Introspection of models & representations; planning for autonomy – goalseeking”Based on the idea of an embodied cognitive agent that is able to explore itsenvironment, and communicate with other agents about descriptions forspaces and objects in that environment, the goal of the second milestoneiwas to develop an agent that can perceive of its environment in termsof affordances, acquire knowledge about such affordances, and use suchknowledge to understand the intentions of other embodied agents in thatenvironment. Based on such understanding, the agent is able to (re-)planits own actions and dialogues to achieve its own goals, possibly requiringcommunication with other agents to request or negotiate cooperation.1.4.1 ArchitectureFor many years, research in AI and computational cognitive science focusedon forms of representation, algorithms to operate on them, and knowledge to10 Christensen et al.be encoded and deployed or derived. In the last decade or two it has becomeclear that there is also a need to investigate alternative ways of putting piecestogether into a complex functioning system, possibly including parts thatoperate concurrently and asynchronously on different sub-tasks, for instance,perception, action, reasoning and communicating.Fig. 1.1. The CogAff schema: superimposing 2 functional divisions, giving 9 cate-gories of possible sub-mechanisms and many possible connections – not all shown.Unfortunately this has led to a plethora of architectures being pro-posed including, for example SOAR, ACT (and its successors), PRODIGY,ICARUS, 3T, APEX, CLARION, CIRCA, EPIC, Subsumption architectures,H-COGAFF, and Minsky’s emotion machine. Many of these were discussedat a DARPA/NSF-funded workshop in Stanford in March 2003.4 One reasonwhy this is a problem is that there is no agreement on what the space of pos-sible architectures is like, nor on the terminology for describing architecturesor on criteria for evaluating and comparing them.One of the tasks for this project, therefore, has been to produce a frame-work for describing and comparing architectures. A first draft and relativelysimple example of such a framework is the CogAff schema developed at theUniversity of Birmingham, and described in [133], partly inspired by [95]. Thisprovides a way of classifying components of an architecture in terms of theirfunctional role, starting with a crude three-way division between perceptual,central and action components, and another three-way division between com-4 Most of the presentations are online here:http://www.isle.org/symposia/cogarch/, including a useful survey paper byLangley and Laird. Another useful overview can be found in [43]1 Cognitive Systems Introduction 11ponents concerned with reactive, deliberative or meta-management functions.Superimposing these modes of division gives a grid of nine types of compo-nents which may or may not be present in an architecture, and which may beconnected in various ways to other components as shown in Figure 1.1.In recent years many researchers influenced by the work of Brooks [24] haveattempted to design robots using only the bottom layer of the CogAff grid,the reactive layer, arguing that either features of the environment or emergentinteractions between many individuals will produce effects that were thoughtto require deliberative and other mechanisms. Others have challenged thisview arguing that it suffices only for simple organisms and insect-like robots.Our approach has not been to engage in such battles but to try to under-stand under which conditions the various types of architectural componentsare useful. For example if building something at location A requires materialsknown to be at location B, and the agent does not have innately determinedreactive behaviours that cause fetching of materials from B, then the delib-erative ability to consider and evaluate a possible journey to B in advance ofdoing it will be useful. Of course this requires suitable forms of representationfor learning and storing re-usable generalizations and perceptual mechanismsthat can ‘chunk’ the environment into categories that support learning of use-ful generalisations. Without discretization, planning several steps ahead ina purely continuous space would be very difficult, although discretization ofpercepts may not be needed for certain kinds of reactive behaviours involvingcontinuous control and no predictions.One of the particularly interesting issues to explore is whether the kind ofself-understanding that so many AI systems lack can be provided on the basisof a meta-management architectural layer permitting observation, classifica-tion, evaluation and possibly some control of internal states and processes,especially deliberative processes that are capable of getting stuck in loops,wasting resources be repeating sub-tasks or in other ways performing sub-optimally. An important form of learning might include detecting such casesand finding out how to prevent them or reduce their effects. An early exam-ple of this sort of thing was demonstrated 30 years ago by Sussman [140],but never developed further. One of the issues to be explored is how our no-tion of meta-management relates to notions of “executive function” used inpsychology and psychiatry. It is possible that empirical research on executivefunctions in humans can contribute design ideas for the construction of ar-tificial cognitive systems. Likewise our design work may shed new light onpuzzles about the nature of executive functions in humans.Another interesting and important issue to consider is how such an archi-tecture might develop. This relates to questions about the pros and cons ofvarious degrees of complexity of the innate mechanisms given to a robot byits designers, or to animals by evolution.12 Christensen et al.Architecture Deliverables:The study of architectures has continued throughout the project. It has threesorts of outputs:• A succession of theoretical papers analysing the variety of possible archi-tectures and their trade-offs in various contexts• experimental designs for working systems to help investigate the theoriesand demonstrate the ideasSoftware tools:A major requirement for the success of a project like this is the availability oftools that support construction of a complex architecture with many interact-ing, concurrently active components doing different things, e.g. concurrentlyactive mechanisms involved in perception (concurrently processing at differ-ent levels of abstraction – eg. feature detection and perception of affordances),reasoning, motive generation, planning, plan execution, communication withother agents, evaluation of whatever’s going on, learning of various kinds, etc.Cosy has developed a platform for rapid prototyping and rapid deploymentof intelligent systems combining multiple software and hardware resources.There were quite a lot of toolkits already available but mostly they tend eitherto be committed to a particular sort of architecture (e.g. SOAR, ACT-R,PRS, etc.) or else aimed at multi-agent systems composed of lots of relativelysimple agents perhaps distributed over many machines. More general andopen ended toolkits include the SimAgent toolkit developed at Birmingham[135], the Cogent toolkit developed at Birkbeck College and the Mozart toolkitdeveloped in Saarbrücken and elsewhere. The project has evaluated these andother tools and developed a new improved tool - CAST that is presented inChapter 2.A cognitive systems project aimed at producing an integrated physicallyembodied agent with diverse but cooperative concurrently active and possiblystrongly interacting capabilities needs tools that can be used to explore andtest out ideas about different sorts of architectures – preferably rapidly andeasily.One of the requirements that arises in connection with architectures thatinclude a meta-management layer is the need for mechanisms that allow self-observation during program execution. The SimAgent toolkit provides somesupport for that, used in work by Kennedy on self-monitoring intrusion de-tectors [71].For all these explorations, merely using a powerful programming language,e.g. Prolog, or Lisp or Java is clearly not enough. For instance, different sortsof languages have been required for the different components (e.g. low-level vi-sion, reasoning about where to go next, understanding or composing sentences,etc.) Facilities to support concurrency, self-observation and self-modification1 Cognitive Systems Introduction 13of running systems should if possible be general and re-usable. Rapid proto-typing, i.e. rapidly producing new experimental implementations that can berun, is essential for research and exploration, as opposed to formal specifica-tion and verification, which are often the main concerns of software engineers:that only makes sense if the problem has already been understood, so thatresearchers know what they are trying to verify.Often, discovering requirements is the hardest part of the research task inAI and exploratory design is a major tool for doing that (you soon find outthat things you have built are no good for surprising reasons, narrowing thesearch space for what a working system needs to be able to do.)Moreover, as the prototypes get more complex the human interface re-quirements for interactive debugging, testing and interrogating parts of thesystem, also get more complex and those requirements have to be met as wellas support for whatever is being built.Note on representations:There have been many debates in recent years about whether animals orrobots need representations (E.g. see [24]). Our view on this is that anythingthat senses the environment and on the basis of what is sensed uses its ownenergy stores in order to select among possible actions is using informationto determine what to do. Biological evolution discovered many variations onthat theme depending on the kind of information acquired, how it is processed,how it is used, when it is used (e.g. long term storage may be required) howit is transformed, how it is combined with other information, and how it iscommunicated. In all cases there is some medium used for the information,but there are great differences between different media, including whetherthey are discrete or continuous, one-dimensional or multidimensional, whatsorts of structures they can have, and so on. Some people may be inclined toargue about whether some of them are really representations or not, but weavoid such disputes by investigation what kinds of representations they are,and what their costs and benefits are to the organism.1.4.2 RepresentationsA very important issue in the project has been the design of representations forobjects, scenes, actions, dialogues and affordances. The representations musttake into account visual, haptic, proprioceptive, and auditory — particularlylinguistic — information, capturing temporal as well as static aspects. Manyof these representations also need to be suitable for higher-level cognitiveprocesses, such as deliberative and reflective processing. Others may be usedonly for reactive control. To accomplish the tasks, that formed for the basisfor the project, the representations should• enable integration of representations of objects, scenes, actions, events,causal relations and affordances; these representations should be linkedtogether to form a consistent framework.14 Christensen et al.• allow incremental updating or sometimes correction.• allow different types of learning (supervised, unsupervised, reinforcement).• allow integration of various modalities, of very different input signals (vi-sual, tactile, auditory, etc.), into a common framework of representations.• deal with multiple cues, enable multiple representations of the same object,scene, event or action.• be suitable for recognition and categorization in the presence of a clutteredbackground and variable illumination.• take into account the context of an object, an action or a scene• be scalable; they should enable efficient learning, storing of the represen-tations, and recognition even when the number of objects, scenes and ac-tions increases (in fact, with an increase in the number of entities, learningshould become more efficient).• accommodate semantics (in terms of language); they should be linked withsymbolic descriptions.• be suitable for acquiring, describing, and using spatial relationships, bothquantitatively and qualitatively.• be suitable for higher level cognitive tasks, such as reasoning or planning.• enable the formation of concepts and hierarchies of concepts.• allow introspection leading to refinement of the representations, and theconstruction of new representations at new levels of abstraction (e.g. cap-turing and exploiting temporal, or geometric regularities not explicitlyprogrammed in by the designer).Many of these issues have been addressed prior to CoSy. They were, how-ever, addressed separately, considering only one, or a few aspects at a time.They have never been tackled together, in a common framework. This has beenan important part of this project. The representations (and, consequently, thelearning and usage of the representations) of objects, scenes, actions, events,causes and affordances should be linked together in a unifying framework. Therepresentations may be obtained in different ways, but processes employingdifferent representations need to able to exchange information, and to updatesome representations on the basis of others.Task Specific vs. General RepresentationsFor a cognitive system there is a need for representations which can, on theone hand, be quite general, to serve different tasks and goals, and, on the otherhand, be tailored for particular tasks. The advantage with task specific rep-resentations is that they are efficient to use, and often compact. The problemwith specific representations is that they can only be used for the task thatthey were trained for. One example is task specific recognition or classifica-tion e.g., a system trained to distinguish faces with glasses from faces withoutglasses would be inappropriate for other tasks e.g., to distinguish male fromfemale faces. Another example would be a reactive controller for plugging a1 Cognitive Systems Introduction 15socket into a power point, which is not directly of use for another task suchas grasping a cup.A number of unsupervised learning methods produce generative represen-tations which allow partial approximate reconstruction, or hallucination, ofthe input data. According to Grossberg [54], reconstruction is central for atwo-way processing cognitive system. Memory traces in the short term mem-ory could activate the long term memory forming a reconstruction, which getscompared to the momentary input. This mechanism allows categorization andestablishment of a reference frame, forming a set of high-level expectations.These expectations influence further processing in a top-down manner, by lim-iting the contextual frame, checking for consistency, providing missing dataetc. Cognition is therefore performed as a two-way, bottom-up and top-downprocess. Comparing the input to the “hallucination” can reinforce the goodhypotheses in the interpretation and inhibit inconsistent parts such as noiseand occlusions. A two-way interaction also enables the processing and organi-zation of a huge amount of data, in contrast to the one-way processing withno feedback. While being significantly faster [114], the lack of interaction withthe higher levels of a one-way processing schema could lead to a combinatorialexplosion of complexity.Nevertheless, supervised non-generative methods can produce simpler rep-resentations, which can be more efficient for specific tasks they are tuned for.Our goal should be to combine both, unsupervised generative methods andsupervised non-generative methods, in order to obtain representations, whichare general enough to enable a two-way processing and robust recognition,and to complement them with features, which are tailored for specific tasks.1.4.3 LearningModes of learningLearning can be considered using a number of different modes:Tutor Driven A user (tutor) shows to the system an object or an action andexplains to the cognitive system what he/she is showing or doing. The tu-tor can also provide figure-ground segmentation (e.g., showing the objecton a black background), which facilitates the learning process and enablesthe creation of a reliable representation. In a similar manner, the tutorcan guide the agent around the space and give it additional information(e.g. coordinates) to facilitate the creation of a map. In such a learningscenario, the user has a high degree of involvement.Tutor Supervised A cognitive system detects a new object, an action, event,affordance or a scene by itself and builds its representation in an unsu-pervised manner. Then it asks the user to provide additional informationabout the object or action (e.g., What is this object, action or room?).The involvement of the user in the learning process is now significantly16 Christensen et al.reduced, yet it still assures that the produced representations are reli-able/consistent.Exploratory A cognitive system detects a new instance of an object, action,event or scene, tries to recognize (categorize) it (using previously acquiredknowledge, statistical properties, etc.), and then updates the representa-tions accordingly. Such a learning scenario does not require any involve-ment on the part of the user.To speed up the initial phase of the learning process and to enable devel-opment of consistent basic concepts, one could start with mainly tutor-drivenlearning with many user interactions. These concepts would consequently beused to detect new concepts with limited help from the user (an importantrole will be played by the dialogue between the cognitive system and a user).Later on in the process, when the ontology is sufficiently large, many newconcepts could be acquired without user interaction.Learning is embedded in the perception-action loop but must go beyondsimple percept-react cycles involving deliberative processes such as reason-ing and planning. This is particularly important in tutor supervised and ex-ploratory modes of learning, where the agent must plan how to interact withthe world so as to learn effectively. This will in turn require reflective pro-cesses to modify the learning strategy, as well as a model of motivation andattention.Continuous learningAs mentioned in Section 1.2.3 in order to avoid evolutionary time-scales inthe development of the system, it has to initially encompass a certain level ofpre-defined functionality and knowledge. It then has to be able to build newconcepts upon these. It needs to do this throughout its lifetime.It is therefore important that the representations employed allow the learn-ing to be a continuous, open-ended, life-long process. The representationsshould not be learned once and for all in a training stage and then used intheir fixed form for recognition, planning and acting. They should be con-tinuously updated over time, adapting to the changes in the environment,new tasks, user reactions, user preferences, etc. So there will be no strict dis-tinction between the activities of learning, planning, recognising, acting andintrospection: these activities will be interleaved.This is a non-trivial challenge. For example, most of the state-of-the-art al-gorithms for visual learning and recognition do not consider continuous learn-ing. They follow the standard paradigm, dividing the off-line learning stageand the recognition stage [155, 153, 123, 84, 85, 68, 2, 45, 78]. Most of theseapproaches are not designed in a way which would enable efficient incrementallearning, which is a basic prerequisite for continuous learning.When having a hierarchical or multi-layered organization of representa-tions, one has to ensure, when performing continuous learning, that the rep-resentations are updated on all levels and that also all mutual dependencies are1 Cognitive Systems Introduction 17updated accordingly, to assure the consistency of the representations. Someof these updates would need to be performed on-line (thus they have to besimple and fast) while some of them may be performed off-line from time totime (during the agent’s “sleeping mode”) by processing several updates atonce.There are two issues which are very important for reliable continuous learn-ing. First, the representations have to be carefully chosen, such that they mustallow efficient upgrading with the newly acquired data. Second, it is importanthow new data is extracted and prepared. When this is performed under theuser’s supervision, this operation is risk-free in the sense that the algorithmcan update the model with high confidence. On the other hand, when theinformation, which is to be added into the representation, is autonomouslyextracted by the agent in an unsupervised manner (e.g., using a recognitionprocedure) there exists a possibility of propagating an erroneous extractionfrom the data through the learning process. Consequently, the representationcould be corrupted with false data, resulting in poorer performance and a lessreliable representation. Robust mechanisms, which prevent such propagationof errors, play an important role in the process of continuous learning.Similarly, the already learned concepts should facilitate learning of newconcepts. For instance, it should be easier to learn to recognise a new objectafter a hundred objects have previously been learned than only after a fewobjects have been considered. In the former case, information is captured inthe representations of already learned objects, which should make learning ofthe new object easier. Such behavior (which is in accordance with some of theproperties of human perception) is not a characteristic of many state-of-the-art algorithms for visual learning.1.4.4 Perception-Action ModellingIn planning and acting approaches fall into two broad classes, although thereare now points of contact. On the one hand there are abstract relationalrepresentations of the effects of actions as used in classical AI planning. Onthe other there are probabilistic models of action effects in continuous spacesas used in robot localisation and mapping. The former are general, and non-task specific, but assume observability of the world, and a hand-constructedabstraction from sensor data. Reasoning with them is also inefficient. Thelatter capture uncertainty in both action and observation, and are tractablefor localisation and path planning in continuous spaces. They are, however,typically tied to geometric representations of space, and in some cases toquite strong assumptions about the sensors used. One of the tasks of thisprojects has been to try to connect these different types of representation insuch a way that updates to one representation can be propagated to otherrepresentations.In neither case are the representations of actions entirely suitable for con-tinuous learning. Some attempts have been made to achieve this. Early tech-18 Christensen et al.niques like chunking attempted to capture reusable patterns of behaviour,and probabilistic models of actions allow the natural integration of new ex-periences into existing action models. But our agents also need to learn toidentify and acquire new action models on different time-scales, recognise newevents, introspect about their causes and thus identify affordances for specifictasks.Analogously in the literature on planning and learning to act there hasbeen some work on automatic relevance detection when learning action mod-els. This can be seen as a simple form of affordance learning, and has beenstudied in, for example, reinforcement learning. These algorithms learn actionmodels that are more efficient because they are task specific. These are ac-quired by identifying features that are relevant to predicting the outcome onthe task. The state-of-the-art techniques, however, work only with unstruc-tured or propositional representations, and rely on statistical methods to de-tect the affordances. Another issue has been how knowledge be transferredfrom one task specific representation to another.1.4.5 Continuous Planning and Acting in Dynamic MultiagentEnvironmentsContinuous planningPlanning and acting in realistic environments poses a number of difficultiesto artificial agents many of which are due to the dynamic nature and partialobservability of such environments: Other agents’ actions as well as naturallyoccurring events (e.g. sunset) may change the agent’s surroundings in waysit cannot foresee, control or even perceive. It is thus crucial that during planexecution agents react according to the perceived changes (e.g. stop movingwhen there is a human standing in your way). Unfortunately, on the one handnot all plans can easily be repaired during plan execution, on the other handwith increasing dynamics of the world an agent’s knowledge will become lessaccurate and its plans more likely to be not executable or to achieve undesiredeffects. However, it is also not possible to give up planning at all or to usepurely reactive forms of planning [3] since its lack of flexibility does not allowto account for diverse and varying goals and its lack of knowledge may preventan agent from selecting a “good” action to perform without some lookaheadplanning.Thus contingencies have to be taken into account already during the plan-ning phase. Traditionally, research in AI Planning has addressed this problemin the subfields of conditional planning [115, 14, 60] and probabilistic plan-ning [21, 82]. The problem solved by conditional and probabilistic planningalgorithms is to find plans that will work not only for a given initial statebut under all circumstances imaginable, a fact that makes the problem com-putationally hard [115, 82]. Considering the large number of unobservablefeatures and of possible contingencies in dynamic multiagent environments it1 Cognitive Systems Introduction 19is clear that even small-sized problems will be hard to solve by conditionaland probabilistic planners.Luckily, in realistic domains (like those considered in the CoSy project)there is often no need to devise universally executable plans before acting asagents will continuously plan, act, monitor their actions, and learn in the sameenvironment. Therefore, in this project techniques have been developed thatallow agents to postpone the resolution of contingencies and handle themonly when they actually occur. Ideally, an agent learns from past experi-ences which contingencies it can easily solve by re-planning at execution time.During the planning phase the agent can then simply “ignore its ignorance”about those contingencies. Instead of a conditional plan fragment, agents useso-called assertions in their plans, non-conditional pseudo actions achievingfacts whose actual achievement will be planned for later. Assertions consist ofpreconditions and effects just like normal actions and special planning triggerconditions describing under which conditions in later phases of a planning-execution cycle detailed planning for the assertion’s effect will be carried out.Trigger conditions mainly consist in a set of state variables whose values areunknown to the agent in the current state. As soon as the agent can observethe actual values of these variables a detailed sub-plan achieving the assertionis searched for.The planning agent research goes beyond classical planning in integratingplanning and execution into a continuous planning agent. It contrast to relatedapproaches [156, 8, 158] the research goal of this project is not focused onplan repair or re-planning5 but on planning under incomplete informationand partial observability. Unlike the conditional and probabilistic plannerspresented earlier contingency resolution is based on postponing the solutionof parts of the planning problem to later points in time. In their use of abstractpseudo actions that are decomposed into executable actions the resulting plansresemble hierarchical task networks [159, 41, 42]. In our approach, however,the abstraction hierarchy is not explicitly given, but the decompositions canbe planned by the agent itself during a later phase of the planning-execution-monitoring cycle. Also the purpose of the abstraction is different from HTNplanning: while HTN decompositions embody knowledge about how to solvesubtasks, in our project assertions essentially represent a way to reason underimperfect knowledge.The approach to automatically interleaved planning and execution takenin this project relies on explicit modeling of an agent’s perception. Assertionachievement is planned for only when the observations specified by the trigger-ing conditions can be made. It is therefore an important part of the proposedproject to model different perceptors and sensing capabilities in a general way.There are two possibilities for modeling sensing: automatic sensing and activesensing by executing sensory actions [121] which both are allowed in our rep-resentation. Making perception explicit in this manner allows agents to plan5 Modern planners like FF [62] allow fast re-planning whenever it is necessary.20 Christensen et al.their own sensing, i.e. to actively try to reach states in which observationscan be made and thus to reduce their ignorance. Both kinds of sensing aremodeled as perceptive events, active sensing events being triggered by explicitsensing actions of the agent while automatic sensing events “fire” similarly todomain axioms [142] (also sometimes called derived predicates, causal rules,or indirect effects). In contrast to earlier approaches to planning with sens-ing [44, 72, 53, 9] the use of perceptive events allows to separate modeling ofthe physical effects of events from their perception by an agent. This separa-tion is especially important for active failure diagnosis (cf. section 1.4.5), butalso to describe when several agents with different perceptors are present ina common environment (cf. section 1.4.5).Active failure diagnosisIn the recent years, tremendous advances have been achieved in the field ofmobile robotics. The majority of the work, however, has focused on navigationaspects such as path planning, localization and mapping. In most approachesit is typically assumed that the sensors and actuators of the robot are reliablein the sense that their input always corresponds to the expected input andthat there is no malfunction of the sensors or actuators. Accordingly, thesesystems are prone to failure in situations in which a sensor breaks and provideserroneous data that does not correspond to the assumptions.Recently, several techniques have been developed to allow mobile robotsto detect or reason about potential failures. For example, Murphy and Her-shberger [89] present an architecture that allows a mobile robot to exploitcausal models for generating tests about potential failures or changes in theenvironment. Leuschen et al. [81] use a mixture of methods to analyze thereliability of a robot. However, they do not provide means for fault detec-tion. Scheding et al. [122] analyze the effects of faults to the innovation of aKalman filter. They introduce a metric for determining the detectability offaults. Washington [152] uses a combination of Kalman filters and Markovdecision processes to identify faults on a mobile platform. Whereas Kalmanfilters are efficient means for detecting faults, their drawback lies in the limitedcapabilities to represent ambiguous situations in which multiple failures couldcase the same effect. To also deal with ambiguous situations recently parti-cle filters have been used with great success for fault detection in technicalsystems [34, 33, 88, 150, 151, 149]One important problem in the context of applying particle filters for faultdiagnosis in complex systems is caused by the high dimensionality of the statespace. In practice, a huge number of particles is needed to track all poten-tial fault states of the system. Therefore, recent approaches have especiallyaddressed the problem of how to reduce the number of samples during thesampling process [88, 151, 149].All these approaches, however, are passive in the sense that they do notexploit the actuators of the robot to identify potential faults. By generating1 Cognitive Systems Introduction 21appropriate actions, such as moving for example, the robot can actively gener-ate measurements that make the robot more certain about potential failures,i.e. a malfunction of its laser range scanner. This way, the complexity of thestate tracking problem can significantly be reduced. In particular, we haveinvestigated techniques allowing mobile robots to identify and to deal withpossible sensor failures. A crucial aspect in this context is active exploration,i.e. the search for actions that allow the robot to identify whether the currentmeasurements are due to failures of the sensors or whether they correspond tothe current state of the environment. Promising approaches for exploration ofstate spaces for active state estimation have recently been applied by variousauthors [25, 69, 65, 73, 120]. Most of the systems rely on approximations ofpartially observable Markov decision processes.Throughout this project we have been especially interested in the integra-tion of symbolic information into the process of active exploration for actiongeneration. Especially we have been interested in triggering the generationof samples in the state estimation component based on assumptions repre-sented at the higher-level planning system (cf. section 1.4.5). In many cases afault will be detected by the planning system due to a time-out. The explicitrepresentation of assertions (such as camera is functional, light is switchedon) allows the system to generate samples in appropriate regions of the statespace. This way we exploit high-level knowledge about the robot to efficientlygenerate actions that reduce the uncertainty at lower levels. To generate sam-ples we have used techniques similar to those presented by Lenser and Velosoas well as Thrun et al. [79, 144]. Similar integrations of high-level and sym-bolic and probabilistic representations has been applied successfully in thepast [58, 13]. Independently of actions triggered by the planning system, therobot permanently monitors its state using a particle filter. To efficiently de-tect failures, however, we exploit knowledge about the effects of typical faultsand generate the samples according to likely states [79, 144].Based on the sample-based representation of the posterior about the po-tential states of the system the robot can then generate actions that reduce itsuncertainty about the potential failure conditions. In this context, techniquessuch as those mentioned above [25, 69, 65, 73, 120] are used. Once a fault(s)has been identified, the high-level system is notified so that appropriate ac-tions can be generated at the planning level.Collaborative planning and actingThe modeling and planning techniques suggested in section 1.4.5 help indi-vidual agents to act continuously in dynamic environments. However, thosetechniques do not account for the fact that the dynamic nature of most en-vironments is a consequence of the presence of multiple agents. If this is thecase, instead of being a handicap the dynamics can be exploited by means ofcooperation. The notion of cooperation is at the heart of the CoSy project:artificial and human agents cooperate to solve problems, both by supplying22 Christensen et al.each other with information and by executing parts of collaboratively devel-oped plans. But cooperation is not limited to human-robot interaction. Arobot must also interact with other “intelligent” household appliances. (Forexample, the Fido robot and the dishwasher may work out a plan together inwhich the dishwasher closes it door and starts cleaning the dishes when Fidohas filled it). Sometimes it will even be necessary to collaborate with remoteagents via the Internet (for example the CoSy repository might be such anagent, providing details of a sub-plan for Fido).Collaborative action planning is not possible without communication.However, different groups of agents may have different ways to communicate.Groups of artificial agents can communicate using special-purpose languageswith rigid protocols, while human-robot interactions should allow the humanto use more convenient methods (cf. 1.4.6). For the sake of scientific clarity,we have adopted an abstract view of communication, assuming that a groupof agents has decided on a common language and protocol that allows them tocommunicate beliefs and intentions by abstract “communicative acts” if theydecide to do so. In this context the only focus is to investigate when and whyagents should engage in communication, not how. For the case of human-robotinteraction the latter question is dealt with when formulating how a concretedialogue can be generated on the basis of the abstract communicative actsproduced during the planning process.Planning in multiagent systems or Multiagent Planning (MAP) is an am-biguous notion. It can describe both a planning process distributed amongmultiple planning agents as well as the concurrent execution of plans involv-ing multiple acting agents. In most real-world domains both dimensions aredirectly coupled as agents are both searching for and executing plans. Tra-ditionally however, subfields of AI have almost exclusively concentrated oneither one of the two aspects. The field of AI Planning, for example, has de-veloped formalizations of concurrent plans featuring durational, overlappingactions and nontrivial joint effects [105, 51, 17, 10, 48, 20], but to date onlycentralized planning algorithms have been developed for these expressive for-malisms. In contrast, distributed planning algorithms developed in the fieldsof Distributed Artificial Intelligence (DAI) and Multiagent Systems (MAS)have mainly focused on coordination and merging of individually found plans[49, 50, 40, 126, 145], ignoring the specific issues of both planning for concur-rent execution and planning with incomplete knowledge (cf. 1.4.5).The example of planning with incomplete knowledge is typical for realisticmultiagent applications like the ones studied in the CoSy project. It also showshow an artificial separation of single-agent planning, multiagent coordination,and individual execution has led previous research to neglect important prop-erties of MAP domains: centralized planning algorithms, for example, haveignored the need for coordinative actions (synchronization, communication)in a multiagent plan; distributed planners have assumed that plans (or sets ofpossible plans) can be found individually before coordination starts; nontrivialeffects or interferences of synchronously executed actions have been modeled1 Cognitive Systems Introduction 23and taken into account by neither approach. Just as the continuous planningagent presented in section 1.4.5 is intended to couple planning, monitoring,and execution its extension to the multiagent case in this project interleaves(and thus integrate) planning, coordination, and execution in a novel way.A key element for this integration is a formalism that allows agents toreason about their distributed planning problem as well as about the concur-rent execution of the partial plans found so far. Parts of such a formalismhave already been developed [22, 23] and has now been extended in the CoSyproject.Several other formalizations of multiagent planning problems and plansrely on hierarchical task decompositions [38, 37, 55] which allow to circum-vent some of the aforementioned problems. Most hierarchical MAP algorithmsassume implicitly that these decompositions are predefined, i.e. explicitly for-mulated by the domain designers, and that they are commonly known amongthe agents [37, 35]. However, in general this cannot be assumed. In fact, dia-logues often manifest an ongoing distributed search for a valid task decompo-sition [83]. Assertions as a new form of abstraction introduced in this project(cf. 1.4.5) allows agents to build and learn hierarchies automatically and in-teractively. In the multiagent setting individual planning and learning of as-sertions have been extended with the possibility to share assertions throughcommunication or devise them collaborative through dialogue.If planning and execution of partial plans are to be interleaved it is anecessary requirement that even the plan fragments produced are at leastexecutable. Many classical and modern planning algorithms do not have thisproperty. For example, regressive partial-order planning algorithms [86, 106,93] search for solutions starting from the goals and may produce an executableplan only when the problem is completely solved. The same is true for manyhierarchical planners [141, 157]. However, in recent years planners based onheuristic forward-search have been developed [11, 18, 62] that not only aremore efficient than previous approaches but also produce executable planfragments during the search process. Unfortunately, these algorithms producetotally-ordered sequences of actions which makes coordination (i.e. mergingor synchronization) of several agents’ plans difficult. In the CoSy project wehave investigated techniques for progressive partial-order planning that allowone to combine executability and efficiency of forward-search techniques withthe flexibility of partial-order planning. When used within the continuousframework of planning with assertions forward-search techniques can be alsoextended to hierarchical planning [91].1.4.6 Models of action & communication for embodied cognitiveagentsOften, an agent finds itself in an environment that it cannot fully know, con-trol, or introspect. It is here important to explore the possibilities that collab-orative action and communication offer to an agent to explore its environment,24 Christensen et al.either on its own or together with other agents. Particularly in the case of jointexploration and problem-solving, natural language is the most efficient andeffective way to communicate between artificial agents and humans: Insteadof learning an artifact-specific language, a human can use the communicationmeans she is accustomed to with little or no learning curve. This perspectivecomplements the other two settings, where we focus primarily on how an agentcan explore its environment through perception of the environment’s spatialorganization, and the objects found in the environment.An important aspect of natural communication is that it is inherentlymulti-modal. Humans communicate not only using natural language, but alsothrough gesture and other non-verbal acts such as body positioning, move-ment, or pointing. There are long-standing efforts to construct multi-modaldialogue systems that are able to capture various of these aspects: Not onlycommunication from system to human can use a variety of modalities (forexample spoken and graphical output), but also that from human to sys-tem (for example spoken input and gesture, e.g. [29], or pen-based input, e.g.[66, 101, 102, 67]).However, despite significant advances in the design and building of multi-modal dialogue systems over the past decades, these systems are still mostlyconfined to relatively simple tasks that require little or no flexibility in thekind of communication to be expected in such a setting. Most if not all ofthe system’s behavior is structured to follow rigid predefined patterns. Thisdoes not allow such systems to carry out more flexible and adaptive, naturalconversations between human and computer.The literature on multi-modal dialogue systems [29, 7, 138] has raised sev-eral issues that need to be addressed, if we are to develop systems that arecapable of flexible and adaptive, natural communication, thus going beyondthe typical master-slave relation common to dialogue systems for information-seeking, moving to mixed-iniative systems as for example required in a col-laborative problem-solving setting.Integration of communication and action. Under a Gricean view of lan-guage, dialogue rests on coordinated reasoning about and determination ofcommunicative intentions [5, 138]. To understand language, the issue is tobuild a representation of the communicated content (language-as-product)and relate that content to (communicative) intentions (language-as-action).This presents challenges both in utterance interpretation (for example whencombining multiple modalities like speech and gesture, cf. e.g. [101, 67]) andgeneration (e.g. [139, 137]). We need to understand intentions as commit-ments in linguistic action, cf. [110], preferably separating generic strategiesfor (collaborative) problem-solving from domain-specific knowledge [5, 6, 15].Recognition of intention, attention, and grounding/understanding.For an important part, communication is about communicating intentions[30, 110, 111], possibly in the context of planning and expressing complex(group) action [55, 83]. This is linguistically realized through different types of1 Cognitive Systems Introduction 25dialogue moves (e.g. Searle’s speech acts, DAMSL, or the VerbMobil ontology)or generic problem-solving acts [6, 15], with the possibility to shift attentionamong intentions [56, 15].Besides recognizing intentions, and shifts in attention, an important pointis the grounding of understanding. An agent must be able to signal whether itunderstood what was communicated, and it must be able to recognize whetheranother agent has or has not understood. The agent needs to be able to askfor and provide feedback or further clarification, as necessary. This also relatesto the problem of belief -modeling, possibly involving notions such as trust.Mixed-initiative An inherent aspect of natural dialogues is mixed-initiative.Agents must be able to lead or yield control in the conversation in ways thatbest accomplish their goals, i.e. they should be able to take or release theinitative to speed up the solution when opportunities arise [4]. To be able tomodel mixed-initiative, incremental strategies are needed for understanding,planning, scheduling and execution/generation, cf. [138, 7]. This issue becomeseven more apparent for embodied cognitive agents: Being set in a partiallyunknown and unpredictable environment, agents can hardly expect to workout the full details of a plan at the outset without ever having to changeit in the light of the perceived results – new opportunities will arise (theramification and qualification problems, cf. [112]).The embodiment and the cognitive character of the systems we have con-sidered in this project add several new issues and twists.Cognitively plausible architecture Multi-modal dialogue systems are en-gineered so as to optimally perform the task that they have been designed for.Resource-bounds, introduced to increase the naturalness of human-systemcommunication, may appear ad-hoc unless they are motivated on cognitivegrounds. For example, there is no reason for a system’s memory and its rea-soning capabilities to be bounded; it is humans who forget or are unaware ofsome logical consequences of their beliefs (cf. the logical omniscience problem).One step in a more cognitively oriented direction is the introduction of in-crementality and resource-boundedness into dialogue systems capable of flex-ible and adaptive, natural dialogues [7, 138]. The challenge is to let thesemechanisms be based on models of human sentence processing, offering thepossibility to handle complexity of communication in a way that is natural toa human language user.Furthermore, multi-modal dialogue systems are normally endowed withprecoded world knowledge and linguistic skills required for carrying out theirtasks. From a cognitive perspective, the challenge is to develop models thatcould explain how agents develop such knowledge and skills, i.e. acquire themover time through interaction with the environment and other agents.Finally, multi-modal dialogue systems usually have dedicated input chan-nels. For example, the pen-input channel is dedicated to pen-input, and notto any other form of interacting with the environment. In a cognitively plau-26 Christensen et al.sible setting, channels are more likely to be multi-functional: Human visualperception not only perceives gestures in communication, but for examplealso objects and events in the environment. This gives rise to various chal-lenges for both recognition and realization: How to separate different types ofinformation arriving over a multi-functional input channel? How could differ-ent multi-functional modalities be combined to aid, and correct, recognition(cross-modal recognition)? How can a particular type of information be sentthrough a multi-functional modality unambiguously?Embodiment in an unknown environment Most multi-modal dialoguesystems developed so far are not embodied – either not at all, like information-seeking dialogue systems or more complex systems like TRAINS or TRIPS[5, 7], or they are set in a virtual environment, e.g. [28, 29]. Embodiment in aphysical, unknown environment adds new challenges.Because the environment itself may be dynamic, the agent needs incre-mental strategies to handle not only flexible dialogue, but also environmentinterrupts.The qualification and ramification problems of the agent’s knowledge ofthe world [112], coupled with the vagaries of sensoric-motoric control andperception, require the agent to actively perceive the environment and theoutcome of its actions; it is not “said and thereby done”: Communicatedintentions, and the actions they represent, may work out differently in realitythan expected.1.4.7 Multi-Modal recognition and categorisationThe capacity to recognize and categorize objects plays a crucial role for cog-nitive systems in order to compartmentalize the huge numbers of objects ithas to handle into manageable categories. In order to achieve this, the abilityto learn from experience and adapt its object models must be at the coreof any cognitive system. There is consequently a need for methods for au-tomatic acquisition of models, and structuring those models into hierarchiesto allow the system to operate in an open-ended fashion, i.e. beyond initialspecifications. This is related to points made in other sections about the needfor perceptual discretization or chunking to support deliberative processes ofexploring branching possibilities, e.g. searching for plans or explanations, inorder to avoid intractable branching continua.Quite interestingly, for humans it was shown that entry-level categorization(i.e. “Is this a dog/cat?”) is much faster in human vision than recognition oridentification (“Is this my dog/cat?”). These findings suggest that humansdo a sort of coarse to fine categorization and recognition of objects. This isalso reflected in language by the way humans talk about objects and scenes.Evolution probably also developed the ability to categorise, e.g. food, prey,shelter, before the ability to identify individuals, which for many organismsis not a requirement.1 Cognitive Systems Introduction 27In this project we have taken the approach that dialogue models and ob-ject models should cross-fertilize through cross-modal integration. Learning,and linguistic semantic ontologies have been tightly coupled with perceptualand planning ontologies. On the one hand language can serve as a triggerand provide guidance to learn and acquire new models but also to structuremodels and representations. On the other hand the role of object models andrepresentations used in perception and action may serve as partial determi-nants of the semantics of the language. This is one example of what we refer toas cross-modal integration. As mentioned before, an important issue for anycognitive system is the ability to communicate about its internal knowledgeand state so that a particular challenge will be also to communicate aboutthe acquired models and model hierarchies of objects.Recognition of objects is obviously one of the most fundamental problemsof computer vision and consequently has been studied widely. However, oneproblem in recognition is that most approaches have been domain specific ortailored to narrow domains. In addition most approaches have been basedon single visual cues such as invariants (points, lines, structures). It is heresuggested that recognition must be based on use of multiple visual cues as nosingle cue is robust in a general setting [127, 87, 132]. In addition there is aneed to exploit contextual information to constrain the set of potential objects[146]. Goal-oriented information has been utilized in a few recent efforts suchas [32, 47]. In addition context information has been used in a number of otherefforts as for example reported by [74, 90, 36].In the context of categorization some have attempted to use the idea offunctional representations for modeling of object categories such as chairs[136]. These efforts have, however, assumed access to a full CAD model ofthe objects of interest, which clearly is unrealistic. Recently there have beenefforts to use simple geometric features for categorization of objects such ascars and tools [154, 46], but it is not immediately obvious that this effort willscale and generalize. In general conceptual models have been used as the basisfor categorization, yet it is not immediately obvious that such collections ofobjects form visual categories in a natural fashion and there is thus a need torevise the concept of categorization to match visual categories, which may becombined into super-sets that form conceptual categories.However by interacting with the objects and observing others using andinteracting with these objects another route for cross-modal learning is pos-sible and have been pursued in this project: Here, cross-modal integrationis between perception and action where the system can start to representand build models of object affordances and then can recognize and categorizeobjects based on those models.Therefore, through construction of multi-cue and cross-modal representa-tions that integrate these representations it is likely that more efficient androbust mechanisms for priming, verification and recognition can be derived.Another key ingredient is the use of task-oriented context driven methods forassociation in the context of multiple cues that each are distorted due to un-28 Christensen et al.certainty. Therefore as part of this project the visual recognition a multi-cueand cross-modal basis for recognition and categorization have been studied.Another important and notorious problem in computer vision in general,is figure-ground segmentation in which objects are separated from the back-ground. A very rich set of studies have addressed this problem. A fundamentalpremise is that figure ground segmentation only makes sense when studied ina task context, i.e. the segmentation is task dependent. In this context the roleof discriminant vs. representational model characteristics have also been stud-ied. The former are required to recognize a large variety of objects, whereasthe latter allow segmentation of those objects in novel scenes.Our work on the scenario “Models for Objects and Concepts” combines theabove ideas regarding the need for cross-modal and multi-directional process-ing in segmenting the environment in a context- and goal-dependent manner,including detection of regions of the environment about which the agent is un-clear or confused. Depending on the agent’s current goals, such regions mayeither be ignored, or else investigated more fully by re-directing attention topossible sources of disambiguation or possible cues to check doubtful partialinterpretations. This not only helps to facilitate figure ground segmentation(combining bottom-up and top-down processing) but also allows testing, ver-ification, and subsequent improvement of the models learned by the agent.1.4.8 Scenario driven researchSections 1.4.1 – 1.4.7 have discussed basic research challenges involved in thedesign of cognitive systems. Each of these key challenges are important bythemselves but more importantly they also have to be studied in the contextof integrated systems, where the system level issues can be studied as partof operational systems embedded in natural environments. To complementthe studied of individual challenges two scenarios were defined as a basis forsystems integration and empirical studies. The scenarios were:Exploration/Mapping of Space A fundamental competence of a cognitive agentis the ability to recognize its own interaction with the environment and tobe able to generate an internal model of its environment as a prerequisitefor operation and interaction with the environment. In these scenario theissue of self-recognition and insertion into the environment. This involvesmodelling of perception interaction and recognition of the embodiment.Once such a competence is available the agent can move about in the en-vironment and generate an internal model of the environment. Without acomplete/comprehensive method for categorisation of objects/structuresin the environment the agent must be assisted by a tutor to recognize /disambiguate structures in the environment. The emphasis of this scenariois thus on spatial models and self-awareness.Models of objects and concepts For interaction with the environment, rea-soning about scenes, and communication with other agents it is essential1 Cognitive Systems Introduction 29that the system has facilities for acquisition of models of objects, eventsand structures. This involves both static objects and dynamic phenom-ena. The number of objects present in a natural environment calls notonly for recognition, but for categorisation of objects to place them in aspatio-temporal hierarchy. For operation and interaction with objects pureRE-cognition will facilitate a limited complexity of interaction and thereis thus a need to recognize/classify affordances as a more generic levelof modelling of structures and objects. In this scenario the emphasis hasbeen on categorisation of objects, events and structures. Again the agentwill cooperate with other agents in its environment to accomplish its mis-sions/goals. In addition the agent will actively explore the environmentto “discover” new objects and structures.Each of these three scenarios discussed below:1.4.9 Exploration/Mapping of SpaceA fundamental aspect of any autonomous / intelligent system is the abilityto navigate through semi-structured environments. Navigation involves threequestions: i) Where am I? ii) How do I get to my destination? and iii) howdo I detect that I have arrived at the destination? All of these questions areclosely related to a representation of space and equally important the abilityto reason about space. For autonomous operation in everyday life the systemneeds• A self-image that models the relationship between perception and action,to understand how control actions influences perception of the world.• Methods for localisation in the world (detection of present position)• Methods for construction/acquisition of a map of the environment• Methods to plan a sequence of actions (subject to the constraints/competencies)for task achievementIn terms of basic cognitive competencies (representation, recognition, learningand reasoning) each of these four issues have differing implications on thesystem architecture, models of the environments, interaction with other agentsand operation.Self-recognition and spatial insertionAffordances and newer approaches:In AI and robotics, the classic approach to building an agent that is ableto spatially explore its environment has been to assume that it is necessaryto create, within the agent, an internal representation or cognitive map of theenvironment in which spatial locations (coded either in absolute or relativecoordinate systems) are associated with objects situated at those locations.In biological systems however such an approach is unrealistic because boththe notion of location and the notion of object are not well defined.30 Christensen et al.As concerns the notion of location, biological agents grow and change, andso are often faced with modifications in their physical make-up which makeit difficult to obtain accurate, repeatable measurements. An agent might forexample be carrying a load or wearing interfering apparel, or moving in asituation where terrain characteristics like mud or water modify the charac-teristics of its motion and change the relation between its motor commandsand the spatial displacement they produce.Similarly, a biological agent has sensors whose properties may change overtime, or which may be modified by environmental factors: an agent may bemoving at night, in fog, under water, in a dust storm, in conditions of unusuallighting, where the response characteristics of its sensors might be severelymodified.Both these examples show that the notion of “location”, simply conceivedas a coordinate system related to agent displacements, as measured by itssensors and modified by its effectors, cannot be easily maintained in robustbiological systems.As concerns the notion of object, here again similar problems arise. Anagent that establishes a map of its environment in terms of objects must nec-essarily rely on some way of describing and coding the objects. And yet inreal biological life, the sensors used to identify objects may be damaged oraffected by environmental conditions. Similarly, the objects themselves maysuffer modifications or changes that make them no longer identifiable by simplepattern matching techniques. For example, houses may be repainted, table-cloths changed, furniture replaced, vegetation may grow, etc.The notion of affordance [52] may be biology’s answer to these problems,and a development of this notion, the notion of sensorimotor contingency[100], can be of additional help.Take as an example the notions of “wall”, “door”, “table”. In a classicalapproach to wayfinding, such entities may be stored in the cognitive mapof a region by way of their particular visual characteristics: surface color,shapes, orientation, etc., and their associated locations. But exact identityor location of physical characteristics are generally unimportant as far as abiological agent is concerned. What is important is what the agent can dowith respect to these entities. A wall is something which you can move alongbut not through. A door is a thing through which you can see things outsidethe room, and by which you can go out of the room. A table is a thing thatyou can put objects on.To a large extent these properties are invariant with surface propertiesand visual aspect. The tablecloth can be changed, the size of the door can bemodified, the wall can have a new picture hanging on it.The trick is to find a way of coding such aspects of the agent’s environment,a way which does not depend on the particular visual aspects. One partialanswer to the problem is to use multimodal sensory cues. Making use of tactile,auditory, or other sense modalities may provide precious help.1 Cognitive Systems Introduction 31But an additional, perhaps vital factor may be that in order to succeedin this task, biological agents are helped by use of their ability to move andmanipulate their environments. For example, an open door can be detected bythe fact that because of parallax, when the agent moves there is a well-definedpatch of visual flow (coming from outside the door) which moves slower thanthe surrounding surfaces (corresponding to nearby walls). A table is a thingsuch that when the agent puts its hand on it, the table resists its pressure,and such that when the agent moves with respect to it, the flow field producedcorresponds to that of a horizontal surface.The world as an outside memory:A second strategy that appears to be used by biological agents in orderto maximize their efficiency appears to be the idea of relying on the outsideworld as its own representation.Instead of making a detailed internal representation of the world, with therisk that this representation will become out-of-date when the environmentchanges, biological agents often use crude global cues in their environments toallow them to home in to elements that they are interested in. For example,instead of coding the GPS coordinates of the coffee container, humans knowit is in the cupboard, and that the cupboard is in the kitchen. Once they havemoved into the kitchen they look for the cupboard and find the coffee.This way of coding the environment has several advantages. The first isthat it requires little memory. The second is that it is robust with respect tovariations in object properties and locations. Changes in the exact positionof the cupboard in the kitchen, the coffee in the cupboard make no differenceto performance. The exact shape of the cupboard, the marks on the coffeepackage are of little importance, since these have not been stored. This wayof coding information is also insensitive to changes in sensor and effectorproperties: Since it is not visual templates of objects which are coded butsensory and motor, code-independent aspects of the way the objects changeas the agent moves, degradation or modification of the agent’s sensory andmotor apparatus will have little effect on behavior.Clearly however this coding strategy is not always possible. There are manycases where surface markings, shapes etc. of objects, which cannot be codedin an intrinsic, code-independent sensorimotor fashion, must be retained. Fur-thermore there are cases where the exact identity of an object is importantand must be stored, for example when an agent is required to detect smallchanges in the environment. In that case a supplementary encoding strategymust be used.The problem of how to combine and coordinate both modes of representa-tion must be solved. It is possible that this might be done within a Bayesianframework. It is possible that such an approach might provide an instantia-tion of the biological notion of attention, in which agents select useful partsof their environment to attend to, be it at the time of encoding or duringeveryday exploration.32 Christensen et al.Mapping of the environmentMapping of the environment [116] has been studied both in terms of dataacquisition with subsequent map construction (off-line) [39],[75], [125] andmore recently as part of Simultaneous Localisation and Mapping (SLAM)[113], [143], [57], [80] and [160]. The by far most dominating approach tolocalisation and mapping is based on use of geometric features encoded in anoccupancy grid or as a metric map of features such as lines, corners, featurecompositions, etc. The advantage of such maps is that they can be constructedusing standard parametric estimation techniques such as Kalman filtering.Such maps do, however, have a number of problems:• Geometry may not in an accurate fashion encode the concept of “place”.• It is not immediately obvious that the approach scales to natural largescale environments.• The maps are unrelated to the embodiment of the agent.• The maps may fail to contain crucial information for planning, spatialreasoning, etc.While the geometry based methods are well suited for basic navigationthere is obviously a need for a scalable, embodied, and richer representationfor cognitive systems. The requirements for a model of the environment (forthe purpose of moving about) are manifold:• Encoding of position of objects/places• Encoding of environmental topology• Invariant to changes to perception system• Invariant to changes in action system• Facilitate spatial reasoning• Allow localisationIn principle two different approaches can be considered to the problem ofspatial exploration/mapping. i) direct mapping of space based on a sensori-motor map, ii) encoding of space as a hybrid topological, geometric/semanticmap that facilitates reasoning. Both types of representation carry differenttypes of advantages, which is the reason why both approaches have beenpursued as part of the research in Cosy.To enable operation in the vicinity of objects and allow interaction withthem there is a need to consider the “affordances” of the objects, i.e. the typeof interaction and operations that an object facilitates. Acquisition of suchmaps can be acquired through direct interaction with the objects throughwhich the perception-action coupling can be determined and invariances thatallow RE-cognition can be identified. The exploration can here use (at least)three modes of acquisition.1. Autonomous exploration: in which the agents explores the environmenton its own and gradually builds up a representation. The exploration canboth both random or pseudo structured in terms of coverage1 Cognitive Systems Introduction 332. Operator Assisted: as the agent moves through the environment the var-ious objects/structures are identified / labelled by the operator and theagent can query the operator about the objects detected.3. Supervised: the robot is given a tour of the environment and all objectsare directly identified to the agent.In practical terms it is of interest to perform research on the use of all threemodes of operation to consider the differences in generated representationsand in terms of efficiency of learning.In parallel to the generation of sensori-motor maps and their integrationof structures in the environment it is of interest to consider more traditionalhybrid representations of space in which a hybrid topological/graph basedmodel is used for encoding of the over-all structure of the environment. Toallow recognition of places there is a need to “categorize” objects into classesthat allow relocalisation. At the same there there is a need to construct spatialmaps in which the relations between places and objects are made explicit.Where traditional localisation and mapping has considered geometric mapsthe idea is here to utilize a semantic approach to localisation in which placerecognition is based on recognition of objects specific to a location ratherthan pure geometric location. The places are then integrated into a graphstructure that encode spatial relations. Such a representation will not onlyallow localisation and mapping but also reasoning about space. A crucial issueconsidered here is the handling of uncertainty to allow the representation tohave convergence in terms of geometric accuracy and handling of topologicalevents such as loop closing. In this approach there is also a need consider theefficiency of mapping in relation to each of the three interaction approachesoutlined above.1.4.10 Models for object and conceptsGenerally, an agent lives and operates in an external world which is nei-ther fully known to or predictable by the agent, nor fully introspectable – norepresentation is ever equal to the external world it represents. In order tounderstand, reason and act in such an unknown and ‘hostile’ environment theagent has to continuously learn and update its concepts and models aboutthis world.This scenario in particular concentrates on the formation of concepts aboutobjects and object manipulation, as well as action, activities and events. Aparticular emphasis is on the joint exploration of, communication about, anddealing with unknown reality (objects & actions external to the embodiedagents), whereby “unknown” means not only “not-seen-before” but also un-known in the sense of (ontologically) unknown affordances (“ontological nov-elty”; surprise & true abduction). In this context the different modes of learn-ing (tutor driven, tutor supervised, and exploratory) are particularly impor-tant and their interplay have been explored in this scenario.34 Christensen et al.More specifically we have started from the exploration of “static” objectsand simple action, yielding concepts based on a given ontology, objects aboutwhich the agent can obtain further information (introspection) using com-munication with other agents (requiring interpretation of deictic references).Also we have worked on the perception, representation, and interpretation ofobjects & actions in terms of affordances; introspection of an agent’s notion ofspace in terms of (affect-)act-cause-effect, using abductive reasoning (“WHYdon’t I understand?”) and communication with other agents.In this scenario we deal with the following cognitive abilities of an agent:• The agent should be able to learn which features of an object, or an ar-rangement of objects provide an affordance for achieving a certain goal.The agent should be able to classify or categorise objects by their affor-dances.• The agent should be able to verbalise the properties of objects, the rela-tionships between objects, and the presence (or lack) of affordances, forsemi-novel scenes.• The agent should be able to infer and verbalise its beliefs about the causesof events.• The agent should be able to learn to categorise and recognize objects incluttered visual fields under standard lighting conditions using time seriesof visual, haptic, verbal, and proprioceptive information.• Representations of objects should encompass more than just static visualinformation; they should take into account also temporal aspects, othermodalities, and higher-level cognitive processes in order to enable differentmodes of life-long learning.• Robustness is an important aspect of a cognitive agent since it is equippedwith imperfect sensors and effectors, and it is supposed to operate in apartially unpredictable real-world environment.• The algorithms for visual learning and recognition should not require auser-specified figure-ground segmentation beforehand. Instead, the systemshould arrive at a segmentation on its own based on previous knowledgeand expectations using a combination of multiple cues.• Robust learning (not just robust recognition) is another very importantissue and should be strongly intertwined with the process of continuouslearning.• Contextual information plays an important role during recognition andlearning and should be therefore integrated into the representation of theobjects and actions.Learning of spatial-temporal concepts and causesIn the scenario on exploration and mapping of space we have devised repre-sentations suitable for expressing, among other things, the qualitative spatialrelationships that exist between an agent and an environment. In the objectsand concepts scenario this has been extended to deal specifically with the1 Cognitive Systems Introduction 35spatial relationships that exist between objects, and their other properties. Inaddition to static spatial relationships an important element of this scenariowill be representations used to express, and learn the way that these relation-ships and properties may be altered. In effect we aimed to construct actionmodels automatically, selecting the correct levels of abstraction to use in ahierarchy of representations.Currently we have methods that are capable of expressing action models ateither the sub-object (geometric) level or the object level. The former are prob-abilistic and essentially pose the problem of recognition of action sequencesas being akin to the problem of localisation. They naturally have a time seriesformulation which is capable of representing the effects of actions, and thereare also algorithms for learning action models. These types of representationsare widely used in modelling visual behaviour [26]. They are flexible, but aredata-driven, and prone to over-fitting. They have also been applied in limitedcases to categorising time series information from robots. Open questions inthis area include the issue of how to learn more structured models directlyfrom data; how to identify events occurring over different time scales; and howto learn to segment robot time series into natural and meaningful chunks.The abstraction required to model actions at an object level is typicallydone by hand, in for example the formulation of AI planning operators incognitive robotics. These do not interface easily with sensory information.This work uses action models based on logics suitable for dealing with events.Some workers have now linked these directly to sensory processing to per-form recognition and inference [128, 129, 131]. These systems currently allowrecognition of events from visual or distance data and in some cases inferenceof their causes by abduction. There are also rigourous inference procedures,although these are typically intractable. Neither do they typically incorporatemultiple modes.Reasoning about causes at an object level is also important because itgives us a possible way of identifying affordances and expressing affordancesin a relational manner. For example we can say that a teacup and teapotprovide an affordance for pouring tea if the pot’s spout is above the teacup.The exact vertical distance between them and the exact position of the cuprelative to the spout are part of a geometric description which can be ignoredat more abstract levels of reasoning.In this scenario we will develop methods that connect the two levels ofdescription: object and geometric. As in the case of object recognition weuse context, so that recognition of behaviour and object will influence oneanother. In summary representations for reasoning about changes in objectsand their relations that we have designed will:• Efficiently infer causes of events.• Give insights into natural phenomena such as change blindness and erro-neous assignment of cause.36 Christensen et al.• Allow the recognition and categorisation of objects by the types of inter-action that they may be engaged in.• Allow reasoning about change at different levels of abstraction.• Allow learning about the invariant properties of objects under changes tothem.Before even beginning to be useful, an agent has to understand its ownbody, and the relation of its body to the environment. In the imitation scenariofor example, the agent has to be able to move its arm to a target position, doeye-hand coordination, and do these things irrespective of whether the arm isloaded with a load, impeded by some obstacle, or moving in unusual conditionssuch as injury or mechanical dysfunction, and irrespective of similar variationsin its sensor characteristics.Classical schemes for accomplishing these goals generally involve makingprecise models of the agent’s motor and sensory apparatus, and generally useanalytical means to calculate the required control sequences.We propose an alternative scheme based on the notion of intrinsic senso-rimotor laws. These are intrinsic invariants of the functions that link motorcommands to the resulting sensory changes. The invariants are intrinsic in thesense that they are independent of the exact physical structure and charac-teristics of the agent, because they correspond to environmental affordances,and not to aspects of the agent itself. For example, moving the fingertip in astraight line is something that can be described in a way that does not dependon the sensory or motor code that is used by the agent [99, 107, 108].Multi-modal learning of object categoriesRepresentations. Most of the state-of-the-art algorithms for visual learn-ing and recognition represent objects and object categories as collections (orconstellations) of descriptors or parts, where each part has its distinctive ap-pearance and spatial position relative to the other parts [155, 153, 123, 84,85, 68, 2, 45, 78].These representations are, however, far too narrow, inefficient, and in-sufficient for accomplishing tasks which are envisioned in the project. Therepresentations of objects should encompass much more than just static vi-sual information; they should take into account also affordances, temporalaspects, other modalities, and higher-level cognitive processes. They shouldalso enable different modes of life-long continuous learning. The representa-tions of objects and concepts should be merged together with representationsof places in a unifying framework.From the point of view suggested in [99], [107] and [108] the interventionof action and multimodal intervention can be conceptualized and modellednaturally by appealing to the notion of intrinsic sensorimotor invariants. Likeaffordances, these presuppose, in an essential manner, active exploration onthe part of the agent. For example, within this framework, an object is an1 Cognitive Systems Introduction 37entity that produces sensory changes which can be reproduced in other en-vironments (that is, said more anthropocentrically: an object can be movedaround). Object categories can also be conveniently described in this senso-rimotor framework: a table is a thing whose top surface has certain intrinsicproperties: the agent can glide its hand smoothly over it; the tabletop suf-fers certain visual transformations characteristic of flatness when the cameramoves with respect to it.Continuous learning. An essential characteristic of a cognitive systemis its ability to learn. The cognitive system has to be able to modify, adapt,and expand the ontology. Only by learning can a cognitive system developand adapt to new situations enabling autonomous higher level reasoning andbehaviors in a general real-world settings. It is very important that learningis a continuous, open-ended, life-long process.Most of the state-of-the-art algorithms for visual learning and recognitiondo not consider continuous learning and follow the standard paradigm of thedivision on the off-line learning stage (characterized by a controlled environ-ment, and slow, computationally demanding processing) and on the recogni-tion stage, where the processing should be fast and should enable recognitionand categorization (also) in unconstrained environments [155, 153, 123, 84,85, 68, 2, 45, 78].Most of these approaches are not designed in a way, which would enableefficient updating of the learned representations, as a basic prerequisite forcontinuous learning. In contrast, we have designed representations, which willnot be learned once and for all in the training stage and then used in their fixedform at the recognition stage. They will be continuously updated over time,adapting to the changes in the environment, user reactions, user preferences,etc. A strict distinction between the learning and the recognition stage shoulddiminish and learning and recognition should be performed intertwiningly.The idea of continuous learning is natural within a sensorimotor, action-based framework, where the system is continuously evaluating the effect of itsown actions on its sensory input, thereby effectively continuously recalibratingits own body schema and sensor properties.Robustness. Robustness should be another important property of a cog-nitive agent, since it is equipped with imperfect sensors and effectors, and itis supposed to operate in a partially unpredictable real-world environment.The algorithms for object recognition and categorization should be able torobustly operate in such non-ideal conditions as well. They should not requirean explicit and precise figure-ground segmentation beforehand. An implicitfigure-ground segmentation should be carried out using the combination ofbottom-up and top-down approaches, intertwined with the object recognitionprocedure [78, 19].Furthermore, within an active, sensorimotor approach, the problem offigure-ground separation is radically simplified, since the agent can, by con-trolling its own motions, generate sensory changes which allow it to isolateobjects from their surroundings.38 Christensen et al.Robust learning is even a greater challenge than robust recognition. Sinceduring the learning process a model of an object is being built, there is noprevious knowledge, which could be used to determine the relevance of localdescriptors for characterizing the particular object. Therefore, robust learningshould be strongly intertwined with the process of continuous learning, whichcould provide enough redundant information to determine statistically con-sistent data (this information can also be provided by a user acting as tutor).Only the consistent data would then be used to build the representations ofobjects, enabling robust learning (and updating of the representations) undernon-ideal real-world conditions.Context. Another issue, which has been addressed in the project, is con-textual information. Traditional object recognition algorithms can, in general,recognize only isolated objects. Some newer object recognition algorithms arerobust and enable recognition of objects in the presence of occlusions andcluttered background without explicit figure-ground segmentation. However,all these algorithms tend to discard background regions and use only the in-formation contained in the foreground. However, the background (contextual)information can be very useful: objects usually appear in a specific environ-ment (context). Thus, context can be very useful for reducing the numberof objects, which can appear in a particular environment, it can increase theconfidence about the identity of the objects (resolve possible ambiguities),and it can narrow the image area, which has to be explored (set the focus ofattention). These facts have been considered very rarely in the algorithms forobject recognition [146, 98]. Therefore, we aimed at incorporating contextualinformation into the representation of the objects and scenes. The contextualinformation should be continuously updated, reflecting the changes in the realenvironment.Categorisation. Objects and concepts can be recognized/classified ondifferent levels of abstraction, considering different properties and followingdifferent goals. Traditionally, computer vision research has frequently focusedon the recognition of specific object instances. While there has been consider-able interest in the more general problem of object categorisation from the be-ginning, research has only recently made progress on recognizing object classmembers in challenging real-world settings [124, 104, 155, 153, 2, 45, 77, 78].These approaches try to implement basic-level object classification [117]using visual input. Having other modalities as well, one could also considerthem for a more reliable object classification. For example, basic level cat-egories are also specified by common motor programs for interaction withcategory members [117, 76]. Therefore functional categories could arise bygrouping objects with similar affordances, ad-hoc categories [12] could resultfrom objects’ contextual information. Using higher level cognitive processes,the different types of higher level categorization (beyond basic level) could bepossible.Architecture. In this scenario the implications for the overall architecturehave been investigated. It may for example turn out useful to have different1 Cognitive Systems Introduction 39perceptual and learning processes in a reactive layer, in a deliberative layerand in a meta-management layer that includes information about intelligentsystems. All these mechanisms will of course share the same physical sensorsand may share a lot of the low level visual analysis mechanisms.However it is possible that outputs from those low level systems will goin parallel to different perceptual systems performing different sorts of tasks,including for instance feedback control of actions, rapid recognition of ob-stacles within reactive motion systems, classification of objects and eventsat a level of abstraction required for learning re-usable generalisations aboutthe environment, and interpretation of mental states and intentions of otheragents.Communication and Language One of the main necessities for an em-bodied cognitive agent is to be able to communicate about the environment itfinds itself in, to be able to learn more about the environment or to coordinateactions among multiple agents so as to collaborate in solving a task.Whether using communication about the environment for learning or forcoordinating actions, a fundamental issue is how language can be used torefer to objects and locations in the environment, either by describing themdirectly or by referring to them. The long-term goal we want to explore in thisproject how such descriptions and references can be interpreted and realized ina cross-modal setting, integrating different modalities such as speech, gesture,vision, and movement (“body positioning”).Not only does a cross-modal integration of modalities often lead to moreefficient and more natural communication. It is also an essential aspect oflearning through language, where an agent uses communication to learn moreabout the environment it finds itself in. For example, the agent could com-municate with an expert who guides the agent and provides all the necessaryinformation. Learning through language involves acquiring new words as wellas new ways of talking about and dealing with reality, and thus inherentlyrequires the integration of information from various modalities.In turn this leads to the interesting issue of how to ground the semanticslearned for the new word. On the one hand, the semantics should reflect thesensory data, and the categorization of that data in relation to an ontology –the word meaning needs to be fit into the agent’s existing knowledge of theenvironment. On the other hand, the semantics of the word puts it in semanticrelations with other words, e.g. via synonymy (different words, same meaning),hypernymy (super/sub-concept) and hyponymy (sub/super-concept). This re-flects a lexical organization which determines for example in what contextsit would be appropriate to use the word as a description. The interactionbetween linguistic knowledge and situational knowledge again stresses thebroader understanding of cross-modality we develop in this project.40 Christensen et al.1.5 Consortium(Embodied) Cognitive systems naturally involve research on robotics, per-ception, action modelling, reasoning, learning, language and representations.Consequently the CoSy consortium brings together core competence from eachof these disciplines in addition to systems integration. The involves institu-tions are briefly summarized below.The Centre for Autonomous Systems (CAS), KTH has been the coordi-nator of the consortium. CAS has a proven record of constructing (robot)systems for natural environments. The research involves computational vi-sion, control methods, behaviour based systems, and systems integration. InCoSy the research has been on perception-action modelling, categorisation,integration and spatial mapping.DFKI is the largest European R&D Center in Artificial Intelligence. Itsmain areas of research are language technology, intelligent user interfaces, au-tomated reasoning, multi-agent technology, knowledge management and visu-alization. In CoSy DFKI has mainly contributed expertise and technology inlanguage processing and dialogue modelling.University of Freiburg has a long term record on representations, formalmethods, planning and mobile agents. The group is well known for it basicresearch on artificial intelligence. In this project the group has in particularfocused on planning, failure detection and multi-agent interaction.The University of Birmingham has a strong record on general aspectsof artificial intelligence, cognitive systems, architectures and robotics. Thepartners brought a strong AI / Cognition experience to the consortium.The partner from TU Darmstadt has a solid background in computervision, multi-cue integration and learning. In relation to CoSy TUD has inparticular performed research on categorisation, object categorisation, crossmodal representations, and learning.University of Ljubjlana has a strong research effort on computational vi-sion and robotics. The work has considered both integration, recognition,learning and visual representations. In CoSy the research has emphasized cat-egorisation, learning and representations.The Laboratory of Experimental Psychology, CNRS has long studied therole of perception action modelling and in particular the sensory character-istics of vision and it implications for perception in general. LPE-CNRS hasin particular been involved in perception-action modelling, categorisation andmulti-cue integration.1.6 Organization of the bookThe CoSy project ended in August 2008 and the main results of the four yearresearch program are reported in the following chapters.1 Cognitive Systems Introduction 41Chapter 2 present the main results in terms of design of a common ar-chitecture for cognitive systems and corresponding considerations related torepresentations. A key aspect of design of system is the perception-actionembedded or spatial insertion into the world. Aspects related to perception-action integration are presented in Chapter 3. The generation of spatial mapsin terms of basic geometry, topology and semantics is presented in Chapter4. Visual perception for mobility and manipulation is presented in Chapter 5.Cognition heavily relies on planning and recovery from failures as presentedin Chapter 6. Another key aspect of a cognitive system is adaptation andlearning across perceptual modalities, skills and tasks as presented in Chap-ter 7. Cognitive systems does not exist in isolation but interact closely withhumans as part of task specification, skill learning and clarification. Human-robot interaction based on situated dialogue is thus a key aspect of a systemas presented in Chapter 8. Chapters 9 and 10 present demonstration systemsand the final two chapters present reflections on lessons learnt during CoSyand challenges for the future.References1. Proceedings of the 12th National Conference of the American Association forArtificial Intelligence (AAAI-94), Seattle, WA, July 1994. The MIT Press.2. S. Agarwal and D. Roth. Learning a sparse representation for object detection.In ECCV02, pages 113–130, 2002.3. P. Agre and D. Chapman. Pengi: An implementation of a theory of situatedaction, 1987.4. James Allen, Donna Byron, Myroslava Dzikovska, George Ferguson, LucianGalescu, and Amanda Stent. An architecture for a generic dialogue shell.Journal of Natural Language Engineering, 6(3):1–16, 2000.5. James Allen, Donna Byron, Myroslava Dzikovska, George Ferguson, LucianGalescu, and Amanda Stent. Towards conversational human-computer inter-action. AI Magazine, 22(4):27–37, 2001.6. James Allen and George Ferguson. Human-machine collaborative planning.In Proceedings of the Third International NASA Workshop on Planning andScheduling for Space, Houston TX, 2002.7. James Allen, George Ferguson, and Amanda Stent. An architecture for morerealistic conversational systems. In Proceedings of Intelligent User Interfaces2001 (IUI-01), pages 1–8, 2001.8. José A. Ambros-Ingerson and Sam Steel. Integrating planning, execution andmonitoring. In Proceedings of the 7th National Conference of the AmericanAssociation for Artificial Intelligence (AAAI-88), pages 83–88, Saint Paul, MI,August 1988.9. Naveen Ashish, Craig A. Knoblock, and Alon Y. Levy. Information gatheringplans with sensing actions. In Recent Advances in AI Planning. 4th EuropeanConference on Planning (ECP’97), pages 13–25, 1997.10. F. Bacchus and M. Ady. Planning with resources and concurrency: A forwardchaining approach. In IJCAI-01 [63].42 Christensen et al.11. Fahiem Bacchus and Froduald Kabanza. Using temporal logic to control searchin a forward chaining planner. In M. Ghallab and A. Milani, editors, NewDirections in AI Planning (EWSP’95), Amsterdam, The Netherlands, 1996.IOS Press.12. L.W. Barsalou. Ad-hoc categories. Memory and Cognition, 11:211–227, 1983.13. M. Beetz, W. Burgard, D. Fox, and A. B. Cremers. Integrating active localiza-tion into high-level robot control systems. Robotics and Autonomous Systems,23:205–220, 1998.14. Piergiorgio Bertoli, Alessandro Cimatti, Marco Roveri, and Paolo Traverso.Planning in nondeterministic domains under partial observability via symbolicmodel checking. In IJCAI-01 [63], pages 473–478.15. Nate Blaylock, James Allen, and George Ferguson. Managing communicativeintentions with collaborative problem solving. In Ronnie Smith and Jan vanKuppevelt, editors, Current and New Directions in Discourse and Dialogue.Kluwer Academic Publishers, Dordrecht, The Netherlands, 2003.16. A. Blum and T. Mitchell. Combining labeled and unlabeled data with co-training. In Proceedings of the 11th Annual Conference on ComputationalLearning Theory (COLT-98), 1998.17. Avrim Blum and Merrick L. Furst. Fast planning through planning graphanalysis. Artificial Intelligence, 90(1-2), 1997.18. Blai Bonet and Héctor Geffner. Planning as heuristic search. Artificial Intel-ligence, 129(1-2):5–33, 2001.19. E. Borenstein and S. Ullman. Class-specific, top-down segmentation. InECCV02, pages 109–124, 2002.20. Craig Boutilier and Ronen Brafman. Partial order planning with concurrentinteracting actions. Journal of Artificial Intelligence Research, 2001.21. Craig Boutilier and David Poole. Computing optimal policies for partiallyobservable decision processes using compact representations. In Proceedingsof the 13th National Conference of the American Association for ArtificialIntelligence (AAAI-96), pages 1168–1175. The MIT Press, July 1996.22. Michael Brenner. Multiagent planning with partially ordered temporal plans.In IJCAI-03 [64].23. Michael Brenner. Multiagent planning with partially ordered temporal plans.Technical report, Institut für Informatik, Univerisität Freiburg, Germany, 2003.24. Rodney A. Brooks. Intelligence without representation. Artificial Intelligence,47:139–159, 1991.25. W. Burgard, D. Fox, and S. Thrun. Active mobile robot localization. In Proc. ofthe International Joint Conference on Artificial Intelligence (IJCAI), 1997.26. Hilary Buxton. Learning and understanding dynamic scene activity. In Euro-pean Conference on Computer Vision: ECCV Generative Model Based VisionWorkshop, pages 71–81, 2002.27. Peter Carbonetto and Nando de Freitos. Why can’t José talk? the problem oflearning semantic associations in a robot environment. In Proceedings of theHLT-NAACL 2003 Workshop on Learning Word Meaning from Non-LinguisticData, pages 54–61, Edmonton, Canada, 2003.28. J. Cassell, J. Bickmore, I. Campbell, K. Chang, H. Vilhjalmsson, and H.Yan.Requirements for an architecture for embodied conversational characters. InProceedings Eurographics 1999, Berlin, Germany, 1999. Springer-Verlag.29. Justine Cassell. Embodied conversational agents: Representation and intelli-gence in user interfaces. AI Magazine, 22(3):67–83, 2001.1 Cognitive Systems Introduction 4330. Philip R. Cohen and Hector J. Levesque. Intention is choice with commitment.Artificial Intelligence, 42(2-3):213–361, 1990.31. D. Cohn, L. Atlas, and R. Ladner. Improving generalization with active learn-ing. Machine Learning, 15(2):201–221, 1994.32. J.L. Crowley and H. Christensen. Vision as Process. ESPRIT BR Series,Springer Verlag, 1995.33. N. de Freitas. Rao-blackwellised particle filtering for fault diagnosis. In InProc. of the IEEE Aerospace Conference, 2001.34. R. Dearden and D. Clancy. Particle filters for real-time fault detection in plan-etary rovers. In Proc. of the Thirteenth International Workshop on Principlesof Diagnosis, 2002.35. Keith Decker and Victor Lesser. Designing a family of coordination algorithms.In Proceedings of the 1st International Conference on Multiagent Systems (IC-MAS’95), pages 73–80, San Francisco, CA, 1995. MIT Press.36. S. Dickenson, H.I. Christensen, J. Tsotsos, and G. Olofsson. Active recognitionintegrating attention and view point control. CVIU, 67(3):239–260, 1997.37. E. Durfee and T. Montgomery. Coordination as distributed search in hierar-chical behavior space. IEEE Transactions on Systems, Man, and Cybernetics,1991.38. Edmund Durfee. Coordination of Distributed Problem Solvers. Kluwer, 1988.39. A. Elfes. Sonar-based real-world mapping and navigation. IEEE Journal ofRobotics and Automation, RA-3(3):249–265, June 1987.40. Eithan Ephrati and Jeffrey S. Rosenschein. Divide and conquer in multi-agentplanning. In AAAI-94 [1].41. Kutluhan Erol, James A. Hendler, and Dana S. Nau. HTN planning: Com-plexity and expressivity. In AAAI-94 [1], pages 1123–1129.42. Kutluhan Erol, James A. Hendler, and Dana S. Nau. Complexity results forhierarchical task-network planning. Annals of Mathematics and Artificial In-telligence, 18:69–93, 1996.43. F.E. Ritter et al., editor. Techniques for Modeling Human Performance in Syn-thetic Environments: A Supplementary Review. Defense Technical InformationCenter, Ft Belvoir, VA, 2002.44. Oren Etzioni, Steve Hanks, Daniel Weld, Denise Draper, Neal Lesh, and MikeWilliamson. An approach to planning with incomplete information. In Nebelet al. [92], pages 115–125.45. R. Fergus, P. Perona, and A. Zisserman. Object class recognition by unsuper-vised scale-invariant learning. In CVPR03, pages II: 264–271, 2003.46. R. Fergus, Z. Zisserman, and P. Perona. Object class recognition by unsuper-vised scale-invariant learning. In CVPR. IEEE Press, June 2003.47. G. Fink, N. Jungclaus, F. Kummert, H. Ritter, and G. Sagerer. A distributedsystem for integrated speech and image understanding. In International Sym-posium on Aritifical Intelligence, pages 117–126, Cancun, Mexico, 1996.48. Maria Fox and Derek Long. PDDL 2.1: an Extension to PDDL for ExpressingTemporal Planning Domains, 2002.49. Michael P. Georgeff. Communication and interaction in multi-agent planning.In Proceedings of the 3rd National Conference of the American Associationfor Artificial Intelligence (AAAI-83), pages 125–129, Washington, DC, August1983.44 Christensen et al.50. Michael P. Georgeff. A theory of action for multiagent planning. In Proceed-ings of the 4th National Conference of the American Association for ArtificialIntelligence (AAAI-84), pages 121–125, Austin, TX, 1984.51. M. Ghallab and H. Laruelle. Representation and control in IxTeT, a temporalplanner. In Proceedings of the 2nd International Conference on Artificial In-telligence Planning Systems (AIPS-94), Chicago, IL, 1994. AAAI Press, MenloPark.52. J. Gibson. The Perception of the Visual World. Houghton Mifflin, BostonUSA, 1950.53. Keith Golden and Daniel Weld. Representing sensing actions: The middleground revisited. In Luigia Carlucci Aiello, Jon Doyle, and Stuart Shapiro,editors, KR’96: Principles of Knowledge Representation and Reasoning, pages174–185. Morgan Kaufmann, San Francisco, California, 1996.54. S. Grossberg. The link between brains, learning, attention, and consciousness.Consciousness and Cognition, 8:1–44, 1998.55. Barbara J. Grosz and Sarit Kraus. Collaborative plans for complex groupaction. Artificial Intelligence, 86(2):269–357, 1996.56. Barbara J. Grosz and Candice L. Sidner. Attention, intentions, and the struc-ture of discourse. Computational Linguistics, 12(3):175–204, 1986.57. Jose Guivant, Favio Masson, and Eduardo Nebot. Simultaneous localizationand map building using natural features and absolute information. Roboticsand Autonomous Systems, 2002.58. D. Hähnel, W. Burgard, and G. Lakemeyer. GOLEX — bridging the gap be-tween logic (GOLOG) and a real robot. In A. Gnter O. Herzog, editor, Proc. ofthe 22nd German Conference on Artificial Intelligence (KI’98), Bremen, Ger-many, 1998. Springer-Verlag.59. M.A.K. Halliday. Three aspects of children’s language development:Learning language, learning through language, learning about language.http://www.ncte.org, unpublished.60. Eric A. Hansen and Shlomo Zilberstein. LAO * : A heuristic search algorithmthat finds solutions with loops. Artificial Intelligence, 129(1-2):35–62, 2001.61. Stevan Harnad. The symbol grounding problem. Physica D, 42:335–346, 1990.62. Jörg Hoffmann and Bernhard Nebel. The FF planning system: Fast plan gen-eration through heuristic search. Journal of Artificial Intelligence Research,14:253–302, 2001.63. Proceedings of the 17th International Joint Conference on Artificial Intelligence(IJCAI-01), Seattle, WA, August 2001. Morgan Kaufmann.64. Proceedings of the 18th International Joint Conference on Artificial Intelligence(IJCAI-03), Acapulco, Mexico, August 2003. Morgan Kaufmann.65. P. Jensfelt and S. Kristensen. Active global localisation for a mobile robot usingmultiple hypothesis tracking. IEEE Transactions on Robotics and Automation,17(5), 2001.66. Michael Johnston. Deixis and conjunction in multimodal systems. In Proceed-ings of COLING-2000, 2000.67. Michael Johnston, Srinivas Bangalore, Gunaranjan Vasireddy, Amanda Stent,Patrick Ehlen, Marilyn Walker, Steve Whittaker, and Preetam Maloor.MATCH: An architecture for multimodal dialogue systems. In Proceedingsof the 40th Annual Meeting of the Association for Computational Linguistics(ACL’02), pages 376–383, 2002.1 Cognitive Systems Introduction 4568. D. Jugessur and G. Dudek. Local appearance for robust object recognition. InCVPR00, pages I: 834–839, 2000.69. L.P. Kaelbling, A.R. Cassandra, and J.A. Kurien. Acting under uncertainty:Discrete Bayesian models for mobile-robot navigation. In Proc. of the Inter-national Conference on Robots and Intelligent Systems (IROS), 1996.70. I. Kant. Critique of Pure Reason. Macmillan, London, 1781. Translated (1929)by Norman Kemp Smith.71. C. Kennedy and A. Sloman. Autonomous Recovery from Hostile Code Insertionusing Distributed Reflection. Journal of Cognitive Systems Research,, 4(2):89–117, 2003.72. Craig A. Knoblock. Planning, executing, sensing, and replanning for infor-mation gathering. In Chris Mellish, editor, Proceedings of the Fourteenth In-ternational Joint Conference on Artificial Intelligence, pages 1686–1693, SanFrancisco, 1995. Morgan Kaufmann.73. S. Koenig and R. Simmons. A robot navigation architecture based on partiallyobservable Markov decision process models. In D. Kortenkamp, R. P. Bonasso,and R. Murphy, editors, Artificial Intelligence and Mobile Robots. MIT/AAAIPress, Cambridge, MA, 1998.74. D. Koller, K. Daniilidis, and H.-H. Nagel. Model-based object tracking inmonocular image sequences of road traffic scenes. IJCV, 10(3):257–281, 1993.75. Benjamin Kuipers and Yung-Tai Byun. A robot exploration and mappingstrategy based on a semantic hierachy of spatial representations. Journal ofRobotics and Autonomous Systems, 8:47–63, 1991.76. G. Lakoff. Women, Fire, and Dangerous Things – What Categories Revealabout the Mind. Univ. of Chicago Press, Chicago, 1987.77. B. Leibe and B. Schiele. Analyzing appearance and contour based methods forobject categorization. In CVPR’03, Madison, WI, June 2003.78. B. Leibe and B. Schiele. Interleaved object categorization and segmentation.In Proceedings of British Machine Vision Conference (BMVC’03), Sept. 2003.79. S. Lenser and M. Veloso. Sensor resetting localization for poorly modelledmobile robots. In Proc. of the IEEE Int. Conf. on Robotics and Automation(ICRA), 2000.80. John J. Leonard and Hans Jacob S. Feder. A computationally effecient mathodfor large-scale concurrent mapping and localization. In Proc. 9th InternationalSymposium on Robotics Research, December 1999.81. M. Leuschen, I. Walker, and J. Cavallaro. Robot reliability through fuzzymarkov models. In In Proc. IEEE Annual Reliability and Maintainability Sym-posium, 1998.82. Michael L. Littman, Judy Goldsmith, and Martin Mundhenk. The computa-tional complexity of probabilistic planning. Journal of Artificial IntelligenceResearch, 9:1–36, 1998.83. Karen E. Lochbaum. A collaborative planning model of intentional structure.Computational Linguistics, 24(4):525–572, 1998.84. D.G. Lowe. Towards a computational model for object recognition in IT cortex.In First IEEE International Workshop on Biologically Motivated ComputerVision, pages 20–31, May 2000.85. D.G. Lowe. Local feature view clustering for 3D object recognition. InCVPR01, pages I:682–688, 2001.46 Christensen et al.86. David A. McAllester and David Rosenblitt. Systematic nonlinear planning.In Proceedings of the 9th National Conference of the American Associationfor Artificial Intelligence (AAAI-91), pages 634–639, Anaheim, CA, July 1991.The MIT Press.87. M. L. Minsky. Steps towards artificial intelligence. In E.A. Feigenbaum andJ. Feldman, editors, Computers and Thought, pages 406–450. McGraw-Hill,New York, 1963.88. R. Morales-Menéndez, N. de Freitas, and D. Poole. Real-time monitoring ofcomplex industrial processes with particle filters. In In Proc. of the NeuralInformation Processing Systems Conference (NIPS), 2002.89. R. Murphy and D. Hershberger. Handling sensing failures in autonomous mo-bile robots. International Journal of Robotics Research, 2000.90. H.-H. Nagel. Image sequence evaluation: 30 years and still going strong. InICPR, pages 148–158, Washington DC, 2000. IEEE Press.91. D. Nau, Y. Cao, A. Lotem, and H. Munoz-Avila. SHOP: Simple hierarchicalordered planner. In T. Dean, editor, Proceedings of the 15th International JointConference on Artificial Intelligence (IJCAI-99), Stockholm, SE, August 1999.Morgan Kaufmann.92. B. Nebel, W. Swartout, and C. Rich, editors. Principles of Knowledge Rep-resentation and Reasoning: Proceedings of the 3rd International Conference(KR-92), Cambridge, MA, October 1992. Morgan Kaufmann.93. XuanLong Nguyen and Subbarao Kambhampati. Reviving partial order plan-ning. In IJCAI-01 [63].94. Monica N. Nicolescu and Maja J. Matarić. Learning and interacting in human-robot domains. IEEE Transactions on Systems, Man and Cybernetics, 31, 2001.95. N.J. Nilsson. Artificial Intelligence: A New Synthesis. Morgan Kaufmann, SanFrancisco, 1998.96. Tim Oates, Zachary Eyler-Walker, and Paul R. Cohen. Using syntax to learnsemantics: An experiment in language acquisition with a mobile robot. Com-puter science department technical report 99-35, University of Massachusettsat Amherst, 1999.97. Tim Oates, Zachary Eyler-Walker, and Paul R. Cohen. Toward natural lan-guage interfaces for robotic agents: Grounding linguistic meaning in sensors.In Proceedings of the Fourth International Conference on Autonomous Agents,pages 227–228, 2000.98. A. Oliva, A. Torralba, M. S. Castelhano, and J. M. Henderson. Top-downcontrol of visual attention in object detection. In Proceedings of the IEEEInternational Conference on Image Processing, 2003.99. J.K. O’Regan and A. Noë. A sensorimotor account of vision and visual con-sciousness. Behavirol and Brian Sciences, 24(4):883–917, 2001.100. J.K. O’Regan, R.A. Rensink, and J.J. Clark. Change-blindness as a result of”mudsplashes”. Nature, 398(34), 1999.101. Sharon L. Oviatt. Advances in the robust processing of multimodal speech andpen systems. In P. C. Yuen, Y.Y. Tang, and P.S. Wang, editors, Multimodal In-terfacesB for Human Machine Communication, Series on Machine Perceptionand Artificial Intelligence, pages 203–218. World Scientific Publisher, London,United Kingdom, 2001.102. Sharon L. Oviatt. Breaking the robustness barrier: Recent progress on thedesign of robust multimodal systems. In M. Zelkowitz, editor, Advances inComputers, pages 305–341. Academic Press, 2002.1 Cognitive Systems Introduction 47103. Brian E. Pangburn, S. Sitherama Iyengar, Robert C. Mathews, and Jonathan P.Ayo. Ebla: A perceptually grounded model of language acquisition. In Pro-ceedings of the HLT-NAACL 2003 Workshop on Learning Word Meaning fromNon-Linguistic Data, pages 46–53, Edmonton, Canada, 2003.104. C. Papageorgiou and T. Poggio. A trainable system for object detection. IJCV,38(1):15–33, 2000.105. Edwin P. D. Pednault. Formulating multiagent, dynamic-world problems inthe classical planning framework. In M. P. Georgeff and A. Lansky, editors,Reasoning about Actions and Plans: Proceedings of the 1986 Workshop, pages47–82, Timberline, OR, June 1986. Morgan Kaufmann.106. J. Scott Penberthy and Daniel S. Weld. UCPOP: A sound, complete, partialorder planner for ADL. In Nebel et al. [92], pages 103–114.107. D. Philipona, J.K. O’Regan, and J.-P. Nadal. Is there anything out there? infer-ring space from sensorimotor dependencies. . Neural Computation, 15(9):2029–2050, 2003.108. D. Philipona, J.K. O’Regan, J.-P. Nadal, and O.J.-M.D. Coenen. Perceptionof the structure of the physical world using unknown multimodal sensors andeffectors. In NIPS, 2003. in press.109. Martha E. Pollack. Inferring domain plans in question-answering. PhD thesis,University of Pennsylvania, 1986.110. Martha E. Pollack. The uses of plans. Artificial Intelligence, 57(1):43–68, 1992.111. Anand S. Rao and Michael P. Georgeff. Modeling rational agents within aBDI-architecture. In James Allen, Richard Fikes, and Erik Sandewall, editors,Proceedings of the 2nd International Conference on Principles of KnowledgeRepresentation and Reasoning (KR’91), pages 473–484, San Mateo CA, 1991.Morgan Kaufmann.112. Raymond Reiter. Knowledge in Action: Logical Foundations for Specifying andImplementing Dynamical Systems. The MIT Press, Cambridge MA, 2001.113. W. D. Rencken. Concurrent localisation and map building for mobile robotsusing ultrasonic sensors. In IROS ’93, volume 3, pages 2192–2197. IEEE, July1993.114. M. Riesenhuber and T. Poggio. Models of object recognition. Nature Neuro-science, 3:1199–1204, 2000.115. Jussi Rintanen. Constructing conditional plans by a theorem-prover. Journalof Artificial Intelligence Research, 10:323–352, 1999.116. Arthur Robinson and Barbara Petchenik. The Nature of Maps: Essays towardUnderstanding Maps and Mapping. The University of Chicago Press, Chicago,1976.117. E. Rosch, C. Mervis, W. Gray, D. Johnson, and P. Boyes-Braem. Basic objectsin natural categories. Cognitive Psychology, 8:382–439, 1976.118. Deb K. Roy. Learning from Sights and Sounds: A Computational Model. PhDthesis, MIT Media Laboratory, Massachusetts Institute of Technology, 1999.119. Deb K. Roy. Learning words and syntax for a scene description task. ComputerSpeech and Language, 16(3), 2002.120. N. Roy, W. Burgard, D. Fox, and S. Thrun. Coastal navigation: Mobile robotnavigation with uncertainty in dynamic environments. In Proc. of the IEEEInt. Conf. on Robotics and Automation (ICRA), 1999.121. Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach.Prentice-Hall, Englewood Cliffs, NJ, second edition, 2003.48 Christensen et al.122. S. Scheding, E. Nebot, and H. Durrant-Whyte. The detection of faults innavigation systems: A frequency domain approach. In Proc. of the IEEEInt. Conf. on Robotics and Automation (ICRA), 1998.123. B. Schiele and J. L. Crowley. Recognition without correspondence using mul-tidimensional receptive field histograms. IJCV, 36(1):31–50, January 2000.124. H. Schneiderman and T. Kanade. A statistical method of 3d object detectionapplied to faces and cars. In CVPR’00, pages 746–751, 2000.125. B. Schölkopf and H. A. Mallot. View-based cognitive mapping and path plan-ning. Technical Report 7, Max-Planck -Institut fur biologische Kybernetik,November 1994.126. A. El Fallah Seghrouchni and Serge Haddad. A recursive model for distributedplanning. In Proceedings of the 2nd International Conference on Multi-AgentSystems (ICMAS’96), pages 307–314, Kyoto, Japan, 1996. MIT Press.127. O. Selfridge. A paradigm for learning. In Symposium on Mechanization ofThought Processes: National Physical Laboratory Symposium, 1959.128. Murray Shanahan. A logical account of the common sense informatic situationfor a mobile robot. Electronic Transactions on Artificial Intelligence, 2:69–104,1998.129. Murray Shanahan. A logical account of perception incorporating feedback andexpectation. In Proceedings of Knowledge Representation 2002, pages 3–13,2002.130. S.G. Shanker and B.J. King. The emergence of a new paradigm in ape languageresearch. Behavioral and Brain Sciences, 25(5):605–606, 2002.131. Jeffrey Siskind. Grounding the lexical semantics of verbs in visual percep-tion using force dynamics and event logic. Journal of Artificial IntelligenceResearch, 15:31–90, 2001.132. A. Sloman. The Computer Revolution in Philosophy. HarvesterPress (and Humanities Press), Hassocks, Sussex, 1978. Online athttp://www.cs.bham.ac.uk/research/cogaff/crp.133. A. Sloman. Beyond shallow models of emotion. Cognitive Processing: Interna-tional Quarterly of Cognitive Science, 2(1):177–198, 2001.134. A. Sloman. Evolvable biologically plausible visual architectures. In T. Cootesand C. Taylor, editors, Proceedings of British Machine Vision Conference,pages 313–322, Manchester, 2001. BMVA.135. A. Sloman and B.S. Logan. Building cognitively rich agents using theSim agent toolkit. Communications of the Association for Computing Ma-chinery, 42(3):71–77, March 1999.136. L. Stark and K.W. Boyer. Generic object recognition using form and functions.Machine Perception and Artifical Intelligence, 10, 1996. World Scientific Press,Singapore.137. Matthew Stone. Modality in Dialogue: Planning, Pragmatics and Computation.PhD thesis, Department of Computer and Information Science, University ofPennsylvania, 1998.138. Matthew Stone. Communicative intentions and conversational processes inhuman-human and human-computer dialogue. In John Trueswell and MichaelTanenhaus, editors, World-Situated Language Use. The MIT Press, CambridgeMA, 2003.139. Matthew Stone and Christine Doran. Sentence planning as description usingtree-adjoining grammar. In Proceedings of ACL 1997, pages 198–205, 1997.1 Cognitive Systems Introduction 49140. G.J. Sussman. A computational model of skill acquisition. American Elsevier,1975.141. Austin Tate. Generating project networks. In Proceedings of the 5th Interna-tional Joint Conference on Artificial Intelligence (IJCAI-77), pages 888–893,Cambridge, MA, August 1977. William Kaufmann.142. Sylvie Thiebeaux, Jörg Hoffmann, and Bernhard Nebel. In defense of axiomsin pddl. In IJCAI-03 [64].143. S. Thrun, D. Fox, W. Burgard, and F. Dellaert. Robust monte carlo localisationfor mobile robots. Artificial Intelligence, 128(1–2):99–142, May 2001.144. S. Thrun, D. Fox, W. Burgard, and Dellaert. F. Robust Monte-Carlo localiza-tion for mobile robots. Artificial Intelligence, 128(1-2), 2001.145. Hans Tonino, André Bos, Mathijs de Weerdt, and Cees Witteveen. Plan coor-dination by revision in collective agent based systems. Artificial Intelligence,142(2), 2002.146. A. Torralba. Contextual priming for object detection. IJCV, 53(2):153–167,July 2003.147. Paulina Varchavskaia. Behavior-based early language development on a hu-manoid robot. In Second International Workshop on Epigenetic Robotics, Ed-inburgh, United Kingdom, 2002.148. Paulina Varchavskaia. Early pragmatic language development for an infantrobot. Master’s thesis, Department of Electrical Engineering and ComputerScience, Massachusetts Institute of Technology, 2002.149. V. Verma, G. Gordon, and R. Simmons. Efficient monitoring for planetaryrovers. In International Symposium on Artificial Intelligence and Robotics inSpace (iSAIRAS), 2003.150. V. Verma, J. Langford, and R. Simmons. Non-parametric fault identificationfor space rovers. In International Symposium on Artificial Intelligence andRobotics in Space (iSAIRAS), 2001.151. V. Verma, S. Thrun, and R. Simmons. Variable resolution particle filter. InProc. of the International Joint Conference on Artificial Intelligence (IJCAI),2003.152. R. Washington. On-board real-time state and fault identification for rovers. InProc. of the IEEE Int. Conf. on Robotics and Automation (ICRA), 2000.153. M. Weber, M. Welling, and P. Perona. Towards automatic discovery of objectcategories. In CVPR00, pages II: 101–108, 2000.154. M. Weber, M. Welling, and P. Perona. Unsupervised learning of models forrecognition. In ECCV, LNCS 1842, pages 18–32. Springer Verlag, 2000.155. M. Weber, M. Welling, and P. Perona. Unsupervised learning of models forvisual object class recognition. In ECCV00, pages 18–32, 2000.156. David E. Wilkins. Practical Planning. Morgan Kaufmann, San Francisco, CA,1988.157. David E. Wilkins. Can AI planners solve practical problems? ComputationalIntelligence, 6(4):232–246, 1990.158. David E. Wilkins, Karen L. Myers, John D. Lowrance, and Leonard P. Wesley.Planning and reacting in uncertain and dynamic environments. Journal ofExperimental and Theoretical AI, 7(1):197–227, 1995.159. Qiang Yang. Intelligent Planning: A decomposition and abstraction based ap-proach. Springer-Verlag, Berlin, Heidelberg, New York, 1997.50 Christensen et al.160. Guido Zunino and Henrik I Christensen. Simultaneous mapping and localisa-tion in domestic environments. In R. Dillmann, editor, Multi-Sensory Fusionand Integration for Intelligent Systems, pages 67–72, Baden-Baden, DE, Au-gust 2001.Part IIComponent Science2Architecture and RepresentationsNick Hawes1, Jeremy L Wyatt1, Aaron Sloman1, MohanSridharan1, Richard Dearden1, Henrik Jacobsson2, Geert-Jan Kruijff21 Intelligent Robotics Lab, School of Computer Science, University ofBirmingham, Birmingham, UK, {nah,jlw,axs,rwd}@cs.bham.ac.uk2 DFKI GmbH, Saarbrücken, Germany, {henrikj,gj}@dfki.de2.1 IntroductionThe study of architectures to support intelligent behaviour is certainly thebroadest, and arguably one of the most ill-defined enterprises in AI and Cog-nitive Science. The basic scientific question we seek to answer is: “What arethe trade-offs between the different ways that intelligent systems might struc-tured?” These trade-offs depend in large part on what kinds of tasks andenvironment a system operates under (niche space), and also what aspects ofthe design space we deem to be architectural. In CoSy we have tried to answerthat question in several ways. First by thinking about the requirements on ar-chitectures that arise from our particular scenarios (parts of niche space). Sec-ond by building systems that follow well-defined architectural rules, and usingthese systems to carry out experiments on variations of those rules. Third byusing the insights from system building to improve our understanding of thetrade-offs between different architectural choices, i.e. between different partialdesigns. Our objective in CoSy has not been to come up with just anotherrobot architecture, but instead to try to make some small steps forward in anew science of architectures.When the project started we had several specific scientific goals for ourwork. First, we wanted to identify a space of possible architectures that in-cludes most of the good ones for our scenarios. Our methodology is to definethis space of possible architectures using an architectural schema, which is asimply a set of constraints and rules on what architectures we allow. We referto specific points within it as architectural instantiations. Second we aimedto develop a toolkit and an experimental methodology to explore this space,identifying the trade-offs as we move through it. Finally, our third goal wasto use our toolkit to implement quickly and easily robotic systems that adhereto certain architectural principles. Those systems are described in Chapters 9and 10. In this chapter we will describe our schema; some specialisations ofit; experiments showing the trade-offs between some of those specialisations;and the toolkit we devised. In addition we will describe how our work sits in54 Hawes et al.between the two major camps of work on architectures for intelligence. Thespecific contributions are:• a set of requirements for embodied cognition as a way of generating anarchitectural schema.• a new architectural schema that draws on important work from both cog-nitive architecture and robotic architecture traditions.• a toolkit for implementing architectures within this schema.• a series of robotic implementations that utilise the schema and provide aproof of concept (covered in Chapters 9 and 10).• a set of well defined problems in architectures, posed if the schema isaccepted: binding; filtering; management; action fusion.• experimental profiling of the effects of steps through schema space, specif-ically with respect to the filtering problem.The contributions are sequenced in the chapter as follows. First we makesome introductory remarks about the science of architectures, our methodol-ogy, and define our terminology. Second we describe run-time and design-timerequirements that arise from the needs of the CoSy scenarios, and of whichour architectural theory must take account. After this we describe our archi-tectural schema (CAS) and show how it meets these requirements. Fourth, wedescribe how the schema poses at least four further problems that we referto as the binding problem, the filtering problem, the processing managementproblem, and the action fusion problem. Solutions to these problems providea second level of constraints and thus a more specific architectural schema.Fifth, we describe the relationship between CAS and other schemas. We followthese contributions with a brief discussion of future research directions.2.2 Architectures and the science of cognitive systemsWe start by clarifying the role of work on ”architectures”. AI is a sciencethat attempts to understand intelligent systems partly through the process ofsynthesising them, and partly by analysis of the systems that result. In mostareas of AI while the problems are technically difficult, the methodology isclear. When considering integrated systems, however, the methodology itselfis a stumbling block. How can we objectively compare the architectures fortwo systems that might perform different tasks, in different environments,using different processing content? The short answer is that we can’t. Yetmost examples of work on architectures in robotics are precisely systems thatcombine architectural choices with task specific processing content, and thatare evaluated in different environments from one another. From the point ofview of a science of architectures for embodied intelligence this is no good. Fora science of architectures we need to separate the processing content from thearchitectural bones on which it is hung. In other words we want to assess how2 Architecture and Representations 55the changes in the architecture alter the run time properties of that system,independent of changes in the information processing content.This is tricky, since in both natural and artificial systems the shape of thearchitecture is intimately related to the shape of the pieces it pulls together. Innature, moreover, the information processing architecture is intimately relatedto the hardware implementation and the information processing functionsimplemented in that hardware. On the engineering side, complete cognitivesystems are still so difficult to construct that the architecture is essentiallyalmost always bespoke. Separation of architecture and content in a theory ismade harder by the fact that the processing in an intelligent system can bedescribed at a variety of levels of abstraction. We may speak of the neural ormachine architecture, of information processing architecture, or of the cogni-tive architecture. In a complete theory of architectures for intelligence we needa clear account of how these different levels of description are related. At thecurrent time we do not even have a clearly defined terminology to distinguishthe different uses of the word.There is an important distinction in the literature between architecturesthat are entirely specified in advance and those that are partially specified. InCoSy, following [1, 2], we use the notion of an architectural schema to refer toarchitectures of the latter type. In simple terms we can think of an architecturalschema as a set of constraints that defines a space of possible architecturesand ways of moving through that space. We can add constraints to a schemato create sub-schemas, i.e. sub-spaces. If we add enough constraints we cancreate a fully specified architecture, and these we refer to as architecturalinstantiations. Hereon, we frequently use the terms schema and instantiationas shorthand for these terms.If we have a software toolkit that implements a particular schema, it canbe used to implement specific working systems that follow the schema. Theseimplemented systems require us to make both a complete set of architecturalchoices, and to have processing content. As described above the two are veryhard to separate. Our approach must therefore be based on a judicious divisionof our design choices into architectural ones (i.e. ones we can easily vary inthe schema), and those concerned with content (i.e. which vary outside theschema).In CoSy the specific schema we have designed is called CAS (Cognitive sys-tems Architecture Schema). Our toolkit implementing this schema is calledCAST. It allows researchers to implement systems that fall within the spacedefined by CAS. CAST is particularly powerful because it allows us to imple-ment a system, and then change some of the architectural choices independentof the processing content of that specific system. Thus CAST allows us to takesteps through the architectural space, and to isolate the effects on systembehaviour due to architectural changes. Thus CAST provides an importantelement necessary for a science of architectures as described above.Overall our architectural work in CoSy is neither concerned with trying tomodel humans or any other specific type of animal, nor with trying to compete56 Hawes et al.on performance on a specific set of tasks. Rather it is concerned with tryingto understand the possibilities and trade-offs involved in different designs inrelation to different sets of requirements. In our approach we have severalstages through which we must go:• We identify sets of scenarios (tasks and environments) that we want oursystems to solve. We then identify the requirements on architectures arisingfrom these scenarios.• These requirements lead us to hypothesise an initial architectural schema,together with a toolkit implementing the schema.• Using the scenarios and the schema toolkit we produce one or more systemsbased on one or more instantiations of the schema. In these designs thereis, as far as possible, a clear separation between the architectural choicesand the processing content.• We analyse our architectural choices, both empirically, and by introspect-ing on the flaws in our designs. We use the toolkit to make architecturalchanges to these designs, while holding the processing content fixed.• Using the insights gained we can refine our schema.In the rest of this document we will step through this process, stoppingalong the way to reflect on the choices we made, and to compare our schemawith previous work. We now turn to the two scenarios driving CoSy, and tothe requirements arising from them.2.3 Requirements for architectures for cognitive robotsAs stated previously our starting point is to analyse scenarios in order todefine requirements for architectures. This is an example of backward chainingresearch in which we consider goals beyond our immediate capabilities, andwork backwards to define sub-problems that are more easily tackled — whilealso understanding a little of the way their solutions might be composed. Inthe CoSy project we picked scenarios based on robot systems that are ableto interact with humans, either while mobile in an office or home setting (theExplorer); or while manipulating objects in a space shared with a human (the .PlayMate). In the Explorer scenario typical tasks might be human augmentedmapping of an office environment, using natural language utterances to nameand describe places, and with mixed initiative dialogue. For example in thisscenario the robot can ask the human what type of room it is in, if it is unsure,and must understand questions, instructions to learn, and instructions to act.In the PlayMate scenario the tasks include the robot answering questionsabout the identity and properties of objects on the table and also about thespatial relationships between them. The robot may be given instructions tomanipulate the objects to alter their spatial relationships, and to learn notjust word labels for objects, but also the meanings of the property words usedto describe an object (e.g. what visual features correspond to the word red).2 Architecture and Representations 57What sorts of requirements do these scenarios and other typical robotscenarios place upon the robot designs, and in particular upon the manner inwhich the pieces of the intelligent system are put together? There are a numberof well-known properties of the robot-environment interaction that any robotsystem operating in human environments typically has to deal with. Our run-time requirements therefore include the ability to deal with the followingproperties of this interaction:• Dynamism: The world changes frequently, rapidly and independently ofthe robot.• Uncertainty: Sensors are inaccurate, and the actions of the robot often failto have the planned for consequences.• Multiple modalities: Many robots — but especially the robots in our twoscenarios — must use information from multiple sensory modalities in or-der to make decisions. In our case these include simple haptics, vision,proprioception and speech. All but the most trivial robot also has multi-ple modes of action (manipulation, looking, facial expression, utterances,locomotion) enabled by multiple motor systems and the actions of thesemust be coordinated.• Re-taskability: In our scenarios our robots must be re-taskable, either byothers, or autonomously. Behaviour should be goal directed, but not tothe extent that it cannot be re-tailored to the context, perhaps on the fly,switch to a new task, or interleave new tasks with old ones.These are quite general run-time requirements on the interaction betweena robot and its environment, i.e. requirements on the system during per-formance. Out of these arise run-time and design-time requirements on thearchitectural schema itself. Architectures for artificial cognition do not juststructure the way components work together during a system run, but struc-ture the engineering efforts of the designers. A good software implementationof an architecture or architecture schema should therefore assist the designprocess. This means that the architecture should make it easier to design acognitive system, and easier to evaluate a system, and understand the causesof its particular behaviour. A proper engineering science of cognition will alsorequire the architecture to have a number of other properties:• Understandability: Cognitive systems of the order of complexity we aredescribing must be understandable. This means that at least some of thetokens within the system need to be semantically transparent to the de-signers at some level of abstraction. If we are to understand why our robotssucceed or fail in a task, and how they can be re-engineered, then we musthave the ability to look at the tokens within the system, and to allocatethem meaning as designers. An architecture schema for cognitive systemstherefore needs to provide a variety of clear ways by which tokens can berelated to one another.58 Hawes et al.• Incrementality in design: Large, bug-free, complex software systems needto be constructed and tested incrementally. This requires that a schemaallow new sub-systems to be added without completely redesigning theexisting sub-systems.• Multiple specialist representations: the field of AI has fragmented, andthe sub-disciplines have developed their own specialist representations forinference and decision- making. In designing robots with multiple forms ofsensing and acting we need to bridge the gaps between them.• Parallel processing: many of the algorithms employed in vision, languageprocessing, planning, and learning are computationally demanding. A se-rial model of processing is thus unworkable. There is a need for perceptualcomponents to run in parallel, so that the system may respond rapidly tochange. Action components must also run in parallel so that the robot cando more than thing at a time, e.g. looking and reaching.• Asynchronous updating: information arrives in different modalities at dif-ferent rates. In addition processing in some modalities is slower than inothers. This requires us to accept that updating of information will occurasynchronously across the system.Given these constraints the management of information flow in the robotsystem becomes key. How should information from one sub-system be commu-nicated to others? How should decisions to act be combined and sequenced?How should we determine whether separate pieces of information are related?These are questions to which we should provide architectural answers. Follow-ing from our requirements together with the specification of the Explorer andPlayMate scenarios are a number of useful properties of robot control systemsthat would satisfy those requirements (though they may not be the only wayof satisfying them), and of an architectural schema should take account. Inorder to satisfy our requirements in our scenarios we will assume the followingprinciples:1. Our robots will have representations of entities in the world at a varietyof levels of abstraction from the sensory information.2. Some of these representations will have roots in multiple sensory modali-ties.3. There will be many concurrently running sub-systems in our robots refin-ing and using these representations.4. Both the PlayMate and the Explorer must represent and reason abouthypothetical future states, in order to be able to plan, answer questions,etc. They will therefore require representations that support this kind ofreasoning.5. A large part of what our architectural solution will be concerned with isthe refinement, sharing and transmission of these representations by andto these different sub-systems.2 Architecture and Representations 596. The system will not be able to draw all possible inferences from the avail-able sensory information, and thus will have to make judicious choicesabout which processing to perform.7. There will multiple modes of action which have to be coordinated.In the next section we describe the architectural schema CAS. CAS en-compasses systems that satisfy our assumptions above, and also satisfies therequirements that precede them. In the following sections we also explain why.2.4 A new architectural schemaThe requirements and assumptions described in the previous section give riseto a space of possible architecture schema designs. In order to produce a singleschema to constrain our research work we must design a schema that satis-fies all of these requirements, whilst still being general enough to capture aselection of the space of possible designs. It is also important that any designreflects both our previous experiences as system designers (i.e. we have knowl-edge about what works and what doesn’t work), and the experience of thewider research community (i.e. what concrete designs have proven successfulin the past). Given all of these (interacting) constraints, it is not possible toclaim that the following design is the best possible schema design for our sce-narios. Instead we put it forward as an initial attempt at producing a schemato satisfy our requirements given our experience.2.4.1 Key Features of CASTo quickly convey the features of CAS we summarise them below. More detailis given in the following sections.• Distributed shared memory: The schema contains sub-architectures eachof which has a blackboard (working memory). These sub-architecturesare loosely coupled to reduce complex inter-dependencies. Systems couldcontain sub-architectures for motor control, vision, action, planning, lan-guage etc. The structure is recursive: sub-architectures can contain othersub-architectures.• Parallel refinement of shared representations: Each sub-architecture con-tains a number of processing components which run in parallel andthat asynchronously read and update shared information via the sub-architecture specific working memory.• Limited privileges: Each of these sub-architecture working memories is onlywritable by processes within its sub-architecture, and by a small numberof privileged global processes (e.g. a global goal manager).• Control of information and processing: Information flow is controlled bygoals generated within the architecture at run time, allowing it to deal with60 Hawes et al.new problems and opportunities. This allows the schema to support differ-ent approaches to processing (e.g. incremental processing, forward chain-ing, backward chaining etc.). The schema distinguishes between two classesof goal: global goals (that require coordination across sub-architectures),and local goals (that are dealt with inside a single sub-architecture).• Knowledge management by ontologies: The knowledge that can be usedwithin a sub-architecture is defined by an ontology for that sub-architecture.Relationships between the ontologies in different sub-architectures are de-fined by a set of general ontologies. These ontologies can also be used todefine knowledge at an architecture-general level.2.4.2 Sub-architecture designComponentsOur schema starts on the level of a collection of processing components. Ev-ery component is concurrently active, allowing them to process in parallel.This satisfies our requirement of supporting concurrent processing. We do notspecify any constraints on the contents of components: they could have be-have like a node in a connectionist network, an activity in a behaviour basedsystem [3], or a unit of processing in a decomposition by information pro-cessing function. Components can take input either directly from sensors, orfrom the working memory. They can also directly control actuators in themanner of closed loop controllers, or initiate fixed action patterns. Compo-nents can have processing triggered by the appearance of certain informationon the shared working memory, and can modify structures on that memory.Components may also have their own private memory. Finally componentsare of two types: managed and unmanaged. Unmanaged components are low-latency processes that run all the time. They are useful for several types ofprocessing. They can be used for low-latency early processing of informationcoming directly from sensors at a high rate. In a visual system, for exam-ple, they could correspond to the earliest stages of pre-attentive processing,pulling high bandwidth data from cameras at frame rate and pumping the pro-cessed frames onto the working memory. Alternatively they could implementreflexes, providing rapid reaction to sensory information; or they could imple-ment monitors on signals that raise alarms or actions elsewhere in the system,or modify global parameters. Managed components by contrast are typicallycomputationally expensive processes, and the schema assumes that there arenot the computational resources available to run them all. They therefore postrequests to run to the task manager associated with the sub-architecture.Shared WorkspacesRather than exchange information directly, processing components are con-nected to a shared working memory. The content of the working memory is2 Architecture and Representations 61solely composed of the outputs of processing components. Working memoriesalso connect to and exchange information with other working memories inother sub-architectures. In our implementation of CAS the communicationmethod between the working memory and the components determines the ef-ficiency of the model. But for now let us consider simply that the schema itselfallows reading and writing to working memories, and transfer of informationbetween them.This use of shared working memories is particularly well suited to thecollaborative refinement of shared structures. In this approach to informationprocessing, a number of components use the data available to them to in-crementally update an entry on shared working memory. In this manner theresults of processing done by one component can restrict the processing op-tions available to the others in an informed way. As all components are activein parallel, the collective total processing effort (i.e. the amount of work doneby all the components in solving a problem) may be reduced by sharing infor-mation in this way. This feature turns out to be very powerful aspect of theschema.Processing managementOur previously discussed requirements included the requirement that any de-sign should support the explicit control of processing. Although control strate-gies could be implemented using communication via working memory entries,failing to support control requirements in the schema would mean that theywould fall outside of the regions of design space we could explicitly manipu-late with it. It would also mean that system designers would have to reinventthe control strategies they required with each new instantiation. The schematherefore supports control strategies by including a dedicated control compo-nent, the task manager, in each sub-architecture. In addition to the usual com-ponent connections to working memory, each task manager has a dedicatedcontrol connection to each component in its sub-architecture. Task managersare also connected across sub-architectures, allowing control strategies to becoordinated across entire instantiations. The task manager can operate in ei-ther a demand driven mode or in request mode. In the demand driven modecomponents request permission to perform a particular task and then havethis request accepted or rejected by the task manager. In request mode, thetask manager sends task requests to components which can then be acceptedor rejected.2.4.3 System wide designWhile a system could be composed of a single sub-architecture we intend thatthere should typically be several sub-architectures in operation. Support formultiple sub-architectures is required in the schema for a number of reasons.It allows the designer of an instantiation to include separate modules in their62 Hawes et al.Fig. 2.1. Left panel: A single sub-architecture within CAS. There are components,which run in parallel, asynchronously updating shared structures on a commonworking memory. They can also take input from sensors or give output to actuators.The task manager determines whether managed components are able to run, whileunmanaged components always run. Right panel: sub-architectures are coupled tomake an overall system. Local changes are transmitted globally between workingmemories, not components directly.system, where each sub-architecture plays a specialised role in processing.This contributes towards satisfying the requirement of supporting the multiplerepresentations required to develop a robotic system from currently availabletechnology. Multiple sub-architectures also, if implemented correctly, improvesupport for concurrency.In the integrated systems we describe in Chapters 9 and 10 we have be-tween three and nine sub-architectures. The schema, note, makes no assump-tions about whether system decomposition should be predominantly accordingto behaviour or information processing function. What is important is thatthe decomposition groups components that commonly exchange informationvia shared structures. When events occur on a working memory computationlinear in the number of components must be performed to trigger process-ing by the right components. Early on in our system building experienceswe discovered that doing this for all components in the whole system meantthe system was paralysed by information change. One of the main benefits ofhaving distributed working memories is precisely that the working memorycan act as a gate-keeper for its local components, only letting them becomeaware of events elsewhere in the system when necessary. We show why thisarrangement is beneficial in our exploration of different sub-schemas later inthe chapter. Having loosely coupled sub-architectures also allows us to explorearchitecture sub-schemas where there are global controllers that utilize very2 Architecture and Representations 63abstract representations. We explore some specific architectural instantiationsin Chapter 10.In the next subsection we describe in more the way that CAS is imple-mented in CAST. This includes discussion of the memory model, and thecommunication model. Following that we define four problems that are raisedby CAS, and indeed by any architecture schema which satisfies our assump-tions from Section 2.3. After describing these we describe sub-schemas forCAS that address each of the four problems.2.4.4 CAST: A toolkit implementing CASIn Section 2.4 we described how CAS works at an abstract, conceptual level.However, the details of the implementation of our schema in CAST determinea great deal about its efficiency, and how easy (or hard) it is for it to bespecialised in one way or another — i.e. the kinds of moves the implementationsupports through the space of architectures. In particular the communication,filtering and memory access models employed by CAST are key in how efficientparticular architectural instantiations of the schema tend to be.In our schema, a working memory is an associative container that mapsbetween unique identifiers and working memory entries. Each entry is an in-stance of a type, which can be considered as analogous to a class description. Aworking memory entry can be any information that can be encapsulated intoa single object class. A system that includes visual components may, for exam-ple, include entries that describe regions of interest and visually determinedobjects. A system that must navigate through a building may have entriesrepresenting maps of various floors, and objects that have been detected.Components can add new entries to working memory, and overwrite ordelete existing entries. Components can retrieve entries from working mem-ory using three access modes: id access, type access and change access. Forid access the component provides a unique id and then retrieves the entryassociated with that id. For type access the component specifies a type andretrieves all the entries on working memory that are instances of this type.Whilst these two access modes provide the basic mechanisms for accessing thecontents of working memory, they are relatively limited in their use for mostprocessing tasks. Typically most component-level processing can be charac-terised by a model in which a component waits until a particular change hasoccurred to an entry on working memory, before processing the changed entry(or a related entry). To support this processing model, components can sub-scribe to change events. Change events are generated by the working memoryto describe the operations being performed on the entries it contains. Dif-ferent instantiations of the schema may support different content for changeevents, but a minimum set of useful information is the unique id and typeof the changed entry, the name of component the made the change, and theoperation performed to create the change (i.e. whether the entry was added,overwritten or deleted).64 Hawes et al.As stated previously an instantiation of the schema can be composed of oneor more sub-architectures. The addition of multiple sub-architectures requiresthat the single sub-architecture schemes for working memory access are ex-tended. We assume that a design would not include multiple sub-architecturesunless necessary for imposing modularity on the processing, and so use sub-architecture boundaries to impose restrictions on cross sub-architecture com-munication. By default a change event is only broadcast within the sub-architecture in which it was generated. This restricts knowledge about thechanges on working memory to within a sub-architecture. If a componentshould require information about changes on a working memory in a differ-ent sub-architecture, it can choose to subscribe to these changes. This actionopens up a connection between the two working memories in question (theworking memory where the change was generated, and the working memorylocal to the component requesting the information), down which all requestedchanges are sent as they are generated. The receiving working memory thenincludes any changes it receives from other sub-architectures in the list ofchanges it broadcasts within its sub-architecture. The schema allows restric-tions to be placed on which working memories a component can access. Thedefault is that a component can read entries from any working memory, butonly write to the working memory in its own sub-architecture. Any variationof this scheme can be specified by an instantiation of the schema, allowingcomponents to write to working memories as required.As described in Section 2.4 support for multiple sub-architectures is re-quired in the schema for a number of reasons. One of these is that in the givenan efficient implementation it increases the support for concurrency in thesystem. In CAST although components are concurrently active, their parallelinteractions via working memory have the potential to become a bottleneckin processing. By distributing processing across multiple working memories,CAST allows designers to avoid these potential delays caused by unrelatedprocessing.2.5 Four problemsFrom our first year of system building experience in the PlayMate, and ourattempts to build systems using other architectures (e.g. OAA) we realisedthat there were a number of problems that must be addressed by all em-bodied intelligent systems that exchange representations between differentsub-systems, and which must satisfy the requirements and assumptions welisted in Section 2.3. We refer to these as the binding problem, the filteringproblem, the processing management problem and the action fusion problem.We define them as follows.The binding problem arises as soon as we have a system with two pieces ofinformation in different places that refer to the same entity. In some instances,to produce coherent decision-making and action execution we need to relate2 Architecture and Representations 65those pieces of information to one another. For example, if I have several blueobjects in front of me, and someone refers to an entity as being “blue” how doI decide which entity that I can see is the object of the referent? In general,given many pieces of information residing in different sub-systems, how shouldthe overall system efficiently decide which pieces are related to which otherpieces and in what ways? In short how do we match information from onecomponent with information from another? The binding problem occurs inmany forms, and is a well studied phenomenon in the visual system [4], andin neural architectures [5].The filtering problem arises as soon as a piece of information is createdin one sub-system. How should the system decide where else that piece ofinformation is needed? The key issue here, as stated above, is that we donot want to share all information with all sub-systems. The problem is thatwhere information is needed depends both on the context and the problemthe system is trying to solve [6]. We call this the filtering problem becausewe think of it precisely as the problem of deciding what information to send,and what information to filter away, i.e. to hold back from other sub-systems.Filtering mechanisms need to be cheap, context sensitive and to generate fewfalse positives, and very few false negatives.The processing management problem is the problem that arises because wedo not have enough computational resources to draw all possible inferencesfrom the sensory data. It is acknowledged that many animals utilise someform of attention to limit processing. On the one hand we can limit whatinformation is processed, and on the other we can limit the processing thatis done to the information selected. In other words we want to manage theprocessing according to the task. We want to investigate the different possiblemechanisms and the trade-offs between them. In some solutions or modelsof human processing the solutions to the binding, filtering and processingmanagement problems are intimately related.The action fusion problem arises when different sub-systems recommendactions to motor systems that need to be fused or otherwise coordinated. Per-haps two behaviours are vying for control of a motor subsystem, or perhapsthe activities of two separate motor systems need to be coordinated. In eithercase the architectural schema must have mechanisms for enabling this coordi-nation. Behaviour based systems use arbitration or fusion mechanisms, whichare limited to a very small number of tasks. Planners use action models tocoordinate activity, and can produce controllers on the fly for many differenttasks, but each of which has only limited feedback. The architectural questionis again what the trade-offs are between different approaches. In CoSy we havenot yet investigated this question, but sketch some possible choices at the endof the chapter.We now go on to describe, for each of the first three problems, the solutionspaces that we have investigated. We have captured the best of these solutionsand incorporated them into CAS as sub-schemas, i.e. as more specific partsof our architectural schema.66 Hawes et al.2.5.1 BindingRequirements on bindingEarlier, we stated that systems suitable for the PlayMate and Explorer scenar-ios often need to connect related information across sub-systems. We referredto this as the binding problem3 [7, 8]. As a simple example take a system thatcan discuss and manipulate objects in a tabletop scene. Perhaps it receivesthe instruction “put the blue things to the left of the red thing”. To carry thisout it must be able to relate disparate pieces of information. The object iden-tities and their properties from vision must be connected with correspondinginformation from the utterance. The robot also needs to connect not just in-formation about physical entities, but also about relations between them —there is information in this example about current spatial relationships fromboth vision and language. Finally the goal state from language must be relatedto a possible spatial configuration, and the objects in this hypothetical staterelated to the objects in the current state. In fact the binding problem existsin a much larger range of designs than those necessary for the PlayMate orExplorer or indeed those covered by CAS. It exists in any system where infor-mation from multiple sub-systems must be fused in order to make decisions.The binding problem is related to theory tethering4 and symbol grounding [9],in that some kind of binding must underlie either approach. After four yearsof investigating different approaches we have included a method for solvingthe binding problem in CAS. In this section we describe our solution as itcurrently stands.There are two constraints from our scenarios that have influenced our ap-proach to binding. The first is that one of the main features of both our sce-narios is the need for an ability to perform deliberative reasoning, by whichwe mean processes that explicitly represent and reason about hypotheticalworld states. The second is that because the systems are also embodied, de-liberation relies on representations derived from perceptual subsystems thatare unreliable and update at unpredictable rates. Embodiment also requiresthat representations are interpretable by effector sub-systems.These constraints lead us to impose the following three requirements onour solution to the binding problem. First, binding must produce representa-tions that are stable for the duration of the deliberative processes they areinvolved in. For example, a representation of an object from vision shouldremain stable across multiple image frames if that object is to be involved ina planning process. Second, these representations produced by binding mustbe at a level of abstraction appropriate for the processing they will be involvedin. For example, the positions of objects on a tabletop could be representedmetrically or as predicate relations. The first is necessary for visual servoing,3 We realise that there are other binding problems in other fields, e.g. neuroscience,but they are somewhat different to the problem here.4 See http://www.eucognition.org/wiki/index.php?title=Symbol Tethering2 Architecture and Representations 67the second for understanding utterances about the scene. The two require-ments are linked: the level of detail influences temporal stability, in that moreabstract representations are typically more durable. Our third requirementarises because the symbols to be bound come from concurrently active, asyn-chronously updating sub-systems, binding cannot happen in a synchronousmanner. To keep a representation of the state as current as possible, it is im-portant that perceptual information is processed as soon as it is generated.Therefore it is important that any representation generated by binding canbe incrementally and asynchronously extended as soon as new information isavailable. To summarise, the requirements on our approach to binding are:• The representations it produces should be stable across the lifetime of theprocessing for which they are necessary.• The representations should have the appropriate level of abstraction forthe processing for which they are necessary.• The process of binding must be proceed in an asynchronous and incremen-tal manner.The binding solution: overviewImplicit BindingOur solution to the binding problem within CAS relies on the separation ofrepresentations into two levels of abstraction. At the low level we have sub-architecture specific representations (Figure 2.2). Within a sub-architecturethe representations that reside on the working memory can be structured,they could for example be slot and filler structures. These structures allowthe results of components to be bound together: two components can fillin different slots in the same structure. This could be because the differentpieces of information in the two slots were derived from the same originaldata (e.g. the same Region of Interest in an image), or from two different dataitems that the system designers deem to be related (e.g. a ROI tracked acrosstwo frames). This kind of binding therefore relies entirely on the structureof the representations, and the relationship of the processing components asdecided at design time. The binding is not explicitly decided by the system,and thus we call it implicit binding. Implicit binding has turned out to bea very powerful feature of CAS. In addition since structures on the workingmemories may contain links to other structures — possibly on other workingmemories — there is the ability to use implicit binding to bind informationalong a single processing trail. This kind of book-keeping turns out to beanother powerful feature of CAS. The reason for this is that it allows usto abstract information that changes slowly from information that changesrapidly, e.g. the identity of an object from the visual description of that object.We can then store the slowly changing information in a new structure that hasa reference back to the fast changing structure. So, in our example, the pose of68 Hawes et al.the object could be recovered without re-abstraction by using the referencesin the processing trail. This is another kind of implicit binding.Implicit binding, while powerful, is not suitable in all situations. If thedecision to bind two pieces of information from two different sub-architecturesmust be made at run-time then we need explicit binding. In explicit bindingthe two pieces of information are translated into a general representation, andin this form are written to the working memory of a specialised binding sub-architecture. A third comparison component decides whether they should bebound into a set. This set is written to the binding working memory and isglobally readable by the whole system.The motivation for this centralised approach rather than any other is oneof minimising the effort of translation. Suppose we have no general represen-tation. This means that if every one of N sub-architectures needs informationfrom all the others N · (N − 1) pair-wise translation processes are required.Whereas if we have a general representation only at most N ·2 translators areneeded5. From a systems engineering viewpoint, translation to a single generalrepresentation makes the translation process more modular and less redun-dant (although this depends on implementation details). It is more modularin the sense that all translations from a collection of sub-architecture specificrepresentations happen in one place in the system, rather than (potentially)in N − 1 places. It reduces redundancy because the N − 1 translations mayfeature many similar operations, whereas these can all be grouped into a singletranslation into the general representation.Fig. 2.2. The binding sub-architecture.To implement this scheme (see Figure 2.2) each sub-architecture has aspecial component called a binding monitor that translates any structureson its working memory into binding proxies that are written to the binder’s5 This assumes separate steps for translating into and out of the general represen-tation, which may be reduced into a single step in practice.2 Architecture and Representations 69working memory. A proxy is simply a set of binding features. Features are inturn simply attribute-value pairs. For example, a visual sub-architecture maycreate proxies for visible objects that have feature attributes for colour, shape,ontological category etc. The role of a binding monitor is to keep its proxieslinked to the source information in its sub-architecture, and to update themas that source changes. This link allows other components to operate on theproxies in place of the source data.All the binding monitors in the system write their proxies to the work-ing memory in the binding sub-architecture. There a collection of componentswhich we will refer to as the binder groups these proxies into binding unionsbased on whether their features match. Unions also have minimal internalstructure, but are instead composed of the union of the sets of features fromits bound proxies. The set of unions on binding working memory representsthe current best architecture-wide hypothesis of the current state of the thingsthe sub-architectures can represent about the world (including its own inter-nal processing). This is based on the assumption that the underlying prox-ies and features are also the best hypotheses from the corresponding sub-architectures. The comparison of features is performed by feature compara-tors. Each comparator compares features to determine whether they couldrefer to the same underlying item of information (e.g. whether the colour orspatial location of two objects is the same). The collected results of these com-parisons is used to determine whether a proxy could be bound into a unionwith other proxies. We now give a formal description of binding.The binding solution: detailsThe set of possible features is broad:Definition 1. A feature space Φx ∈ Φ is any data format in the space of allpossible data formats, Φ. φxi ∈ Φx denotes an instantiation of a particularrepresentation where x should be interpreted as any feature space name.For example, φColourLabelred ∈ ΦColour denotes the colour “red” in the repre-sentation space of colours. In our CAST instantiation, Φ corresponds to anyrepresentation that may inhabit a working memory.Information from the sub-architectures (SAs) is shared as a collection ofproxies:Definition 2. A binding proxy is a structure p = 〈Fp, up〉 where Fp is a setof instantiated features of different types (i.e. Fp = {φx11 , φx22 . . . φxnn }) and uprefers to a binding union with which the proxy is bound (see below).The unions should express information from proxies that, by all accounts (cf.Algorithm 2.3), refer to the same entity. Unions simply inherit the features ofthe bound proxies and are defined as:70 Hawes et al.Definition 3. A binding union is a structure u = 〈Fu,Pu〉 where Pu refersto the subset of proxies unified by the union u and Fu is defined as the unionof the features in all proxies in Pu.The problem for the binder is to assess whether two proxies are matchingor not. By matching we mean that they should refer to the same thing. Todo this, all new or updated proxies are compared to all unions on the basisof their respective features. The basis of this comparison is that each pair offeature types has an associated comparator function:Definition 4. A feature comparator is a function ∆ : Φx × Φy → {true,false, indeterminate} returning a value corresponding to whether two fea-ture instances are equivalent (or similar enough) or not. The comparator canalso choose to not return a definite answer if the answer is undefined, or theuncertainty is too big (i.e. indeterminate).Obviously, indeterminate is the only answer most such comparators canreturn, e.g. the comparison of a ΦColour and a ΦPosition is likely undefined6.However, for many pairs of features there exist informative comparators. Forexample, features such as linguistic concepts can be compared to other con-cepts (with ontological reasoning) or physical positions can be compared toareas.Definition 5. Two feature spaces (Φx, Φy) are comparable iff ∃(φxi , φyj ) ∈(Φx, Φy) such that ∆(φxi , φyj ) 6= indeterminate.The more pairs of features from different SAs that are comparable, the morelikely it is that proxies from these SAs will be accurately matched.To compare a proxy and a union, the corresponding feature sets are thebasis for scoring:Definition 6. The binding scorer is a function S+ : P ×U → N where P andU denote the set of all proxies and unions respectively andS+(p, u) =∑φxi ∈Fp∑φyj∈Fu{1 if ∆(φxi , φyj ) = true ∧ φxi 6= φyj0 otherwisewhere Fp and Fu are the feature sets of p and u respectively.Note that identical features are not counted. This to prevent a union gettinga higher score just because it is compared to one of its member proxies (thiswould sometimes prevent a proxy switching to a better union). The numberof feature mismatches is also counted (i.e. with true replaced with false inS+). That function is here denoted S− : P × U → N.S+ and S− are the basis for selecting the best among all unions for eachnew or updated proxy. This is conducted by the function bestUnionsforProxy6 Of course, in the implementation, such undefined comparators are never invoked.Mathematically, however, this is exactly what happens.2 Architecture and Representations 71Architec ure and Repr sentations 23bestUnionsforProxy(p,U)Input: A proxy, p, and the set of all unions, U .Output: Best union(s) with which a proxy should bind.beginbest := !;max := 0;for "u # U doif S!(p, u) = 0 $ S+(p, u) > max thenbest := {u};max := S+(p, u);else if S!(p, u) = 0 $ S+(p, u) = max thenbest := best % {u};endendreturn best;endAlgorithm 1: The algorithm which computes the set of best candidateunions for being bound with a new or updated proxy (see definitions 1-6 foran explanation of the notations).Note that identical features are not counted. This to prevent a union gettinga higher score just because it is compared to one of its member proxies (thiswould sometimes prevent a proxy switching to a better union). The numberof feature mismatches is also counted (i.e. with true replaced with false inS+). That function is here denoted S! : P ! U " N.It is important to state that S+ and S! are implemented asynchronouslywith respect to the comparators. Until a comparator has returned an answer,S+ and S! will simply assume that the answer is neither true or false, i.e.indeterminate.S+ and S! are the basis for selecting the best among all unions for eachnew or updated proxy. This is conducted by the function bestUnionsforProxydescribed in Algorithm 1. The result of best = bestUnionsforProxy is a setof zero, one or more unions. If |best| = 0 then a new union will be created forthe proxy p alone (i.e. with all the features of p). If |best| = 1, then the proxyis bound to that union.When |best| # 2 we are faced with a disambiguation problem. To avoiddeadlocks in such cases the binder can select a random union from best forbinding. However, bindings are sticky, meaning that if an already bound proxysubsequently matches a union in a larger “best”-list, then it will not switch toany of those unions. This to avoid excess processing in, and signalling from,the binder. This also helps to satisfy our requirement for symbols to be stableas far as possible. Disambiguation problems cannot be solved by the binderitself, but it can request help from others SAs. This may result, for example, inthe communication SA initiating a clarification dialogue with a human tutor.Fig. 2.3. The algorithm which computes the set of best candidate unions for beingbound with a new or updated proxy (see definitions 1-6 for an explanation of thenotations).described in Algorithm 2.3. The result of best = bestUnionsforProxy is aset of zero, one or more unions. If |best| = 0 then a new union will be createdfor the proxy p alone (i.e. with all the features of p). If |best| = 1, then theproxy is bound to that union.When |best| ≥ 2 we are faced with a ambiguity. To avoid deadlocks in suchcases the binder can select a random union from best for binding. However,bindings are sticky, meaning that if an already bound proxy subsequentlymatches a union in a larger “best”-list, then it will not switch to any of thoseunions. This to avoid excess processing in, and signaling from, the binder.This also helps to satisfy our requirement for sy bols to be stable as far aspossible. Ambiguities cannot be solved by the binder itself, but it can requesthelp from other SAs. This may result, in principle, in the communication SAinitiating a clarification dialogue with a human tutor.Relations and Gr upsThe proxies and unions described so far have been assumed to roughly corre-spond directly to physical objects. They may also correspond to more abstractentities as well. To support this, two special proxy types are implemented ina slightly different manner: proxies denoting groups of proxies, and proxiesdenoting relationships between proxies.Since proxies contain features that are of any representable type, prox-ies can also have features attributable to groups and relations, e.g. cardinalityand relative metric information respectively, and explicit references to relatingproxies. Currently we handle groups in a fairly simple yet direct way: a spe-cial kind of “group proxy” is created exactly like an ordinary binding proxy72 Hawes et al.with all the features that the members of the group have in common (e.g.“the blue balls to the left of the mug” creates a group with features φConceptballand φColourLabelblue and with a spatial relation φSpatialRelleft of -proxy to the φConceptmug -proxy. A separate process in the binding SA (the “group manager”) thenspawns off individual proxies which inherit the features of the group proxy.Every time an individual is bound to something, a new proxy is spawned7. Toall the other processes, the individuals appear as an endless supply of normalproxies.Relation proxies are implemented in a similar way as standard proxies, butwith additional features indicating the other proxies involved in the relation.Features of relation proxies are thus compared using the same mechanismthat compares the features of standard proxies. For example, spatial metricfeatures, e.g. φR3(x,y,z), could in principle be compared to a linguistic featuredescribing the same relation, e.g. φSpatialRelleft of . It has turned out that featuresthat link relations to normal proxies and vice versa make the scoring ineffi-cient. Therefore, a separate scoring scheme similar to that in definition 6 isused to assess how well proxies match to unions w.r.t. their relational context.Assume that union u1 has a relation (union) u1→2 to union u2 If a nowthree additional proxies are added, arranged internally as two proxies and arelation proxy between them, pa, pb and pa→b respectively, it is possible thatthey will be bound with the existing unions. First of all, if S+(pa, u1) = 0 andS−(pa, u1) = 0, then pa may be unified despite no convincing score if pa→b isunified with u1→2. In other words, the relational context of an ordinary proxy,as defined by the relational proxies, can tip over the balance and favour thatit binds with an existing union despite that there are no features that match.The relational proxy pa→b may also be unified under similar conditions (i.e.where pa is unified with u1). If S−(pa, u1) > 0 or S−(pb, u2) > 0, however, therelation will not bind even if S+(pa→b, u1→2) > 0 and S−(pa→b, u1→2) > 0.The reason is that two relations that are between entities that cannot be thesame, can also not be the same relation.Visual & Spatial Reference ResolutionTo illustrate how our binder supports a number of behaviours typically re-quired of robots that interact with humans, the following sections present anumber of examples taken from the Explorer and PlayMate systems. Perhapsthe most common use of information fusion systems is to interpret linguisticreferences in terms of visual information. Our binder handles this task as aninstance of a more general problem of information fusion. We will here con-sider the simple situation where we have a red object and two blue objects onthe table. The objects are arranged in a straight line of alternating colours.The human then asks the robot to “put the blue objects to the left of the redobjects”.7 With some obvious limitations to allow finite groups and to prevent excess proxiesbeing generated when members of different groups merge.2 Architecture and Representations 73We will start our example in the visual sub-architecture, where changedetection, tracking and segmentation components create representations ofthe objects in the scene. These objects have 3D poses and bounding boxesand a number of slots for visual properties such as colour, shape and size.These slots are filled by a recogniser that has been previously trained (seeChapter 7) using input from a human trainer [10]. For this example we assumethe recogniser correctly extracts the colours of the objects as red and blue.When the scene becomes stable (determined by the change detector) the visualsubarchitecture binding monitor creates a proxy for each of the currentlyvisible objects. As the visual property recogniser processes the objects, themonitor updates the proxies with features reflecting these properties. This isan incremental process, so the visual proxies are updated asynchronously asthe objects are processed. At this point only the visual proxies are present inthe binding working memory, each one is bound to its own union.The presence of objects in the visual working memory is also noticed bythe components in the spatial subarchitecture. These abstract the objects aspoints on the tabletop, and the spatial binding monitor creates a proxy foreach. These proxies are tagged with the ID of the visual proxy for the corre-sponding object so they are bound correctly8. Concurrently with the proxycreation, qualitative spatial relations between the spatial objects are addedto working memory. These are generated by components using potential-field-based models of spatial relations [11]. In our example the two blue objects areto the left and to the right of the red object respectively. They are both alsonear the red object (but not near each other). As these relations are added,the spatial binding monitor reflects them on the binding working memory asrelation proxies between the spatial proxies. The binder uses these as the ba-sis of relations between the unions featuring the spatial proxies. This providesour basic model of the current state.When the human speaks to the robot, a speech recognition module in thecommunication subarchitecture is triggered. The resulting speech string iswritten to the communication working memory. This triggers a cycle of deepsentence analysis and dialogue interpretation, yielding a structured logicaldescription of the utterance’s content. From this structure the communicationbinding monitor generates communication proxies for the discourse referentsfrom the utterance and the relations between them. These proxies includefeatures that can match against both those attached to visual proxies (colour,shape and size), and those attached to spatial proxies (relations based onspatial preposition). In the example two proxies are generated: one normalproxy for the red object, and one group proxy for the blue objects. The binderuses the features of these communication proxies to bind them into unionswith the visual and spatial proxies. In the example the φColourLabelred -proxy isbound together with the visual and spatial proxies relating the red object,8 A similar, but more general, functionality could be generated by matchinglocation-derived features.74 Hawes et al.and the φColourLabelblue -proxies spawned from the corresponding group proxy(see Section 2.5.1) are bound with the remaining proxies for the blue objects.This provides the system with an interpretation of the utterance in terms ofthe visual scene.In this example, the process of reference resolution involves simply ensuringthat the communication proxies referring to visual entities (i.e. those referringto objects in the tabletop scenario) are bound to unions that have a visualcomponent. If the utterance contains spatial language, then relation proxiesare generated by the communication binding monitor. This causes the bindingprocess to bind proxies via the relations between proxies as well as the featuresof single proxies. Failure to bind proxies can trigger a number of differentprocesses.Binding summaryIn this section we have described two main mechanisms to tackle the bindingproblem: implicit binding and explicit binding. Implicit binding is essentially adesign time choice, while explicit binding is a run time decision by the systemitself. We have described how implicit binding also allows us to implementstable abstract features that are linked to rapidly changing features via thecreation of processing trails. Finally we have described how explicit bindingoccurs. In addition to the basic mechanism we have also explored the problemof early binding. This is when possible bindings can be used to cut down thenumber of possible interpretations in some sub-architecture specific process. Inother words tentative bindings across sub-architectures can prune hypotheseswithin those sub-architectures. This kind of approach we term incrementalbinding and it is described in Chapter 8. The key issue with binding is whetheror not a centralised approach to the problem is the right one. We have shownthat it is possible, not for which kinds of niches it is the right choice.2.5.2 FilteringPreviously we described the filtering problem as being how to decide effi-ciently where a piece of information that arises in one sub-system needs to besent. In other words it is a problem of efficient information sharing. In CASour atomic information generating sub-systems are components. There is aspace of possible models for information sharing between components, rang-ing from point-to-point communication (i.e. that used by our OAA-based firstPlayMate system) to a broadcast model. Between these two extremes exista range of possible systems in which components share information with asubset of components. Which model is chosen can have a great impact on thefinal system behaviour. In this section we use the shared memory-based de-sign of CAS to explore the effects of varying the information sharing patternsbetween components empirically. Specifically we achieve this by altering theratio of components to sub-architectures in a subset of the PlayMate system.2 Architecture and Representations 75We start with an n-m design where n components are divided betweenm sub-architectures, where n > m > 1. The design is part of the Play-Mate system described in Chapter 10, in which components are assigned tosub-architectures based on functionality (vision, binding or qualitative spatialreasoning), although for this experimental work arbitrary n-m assignmentsare also possible (and would explore a wider area of design space). We thenreconfigure this system to generate architectures at two extremes of the de-sign space for information sharing models. At one extreme we have an n-1design in which all n components from the original system are in the samesub-architecture. At the other extreme of design space we have an n-n designin which every component is in a sub-architecture of its own. Each of thesedesigns can be considered a schema specialisation of the CAS schema fromwhich a full instantiation can be made.These various designs are intended to approximate, within the constraintsof CAS, various possible designs used by existing systems. The n-1 designrepresents systems with a single shared data store to which all componentshave the same access. The n-m design represents systems in which a designerhas imposed some modularity which limits how data is shared between com-ponents. The n-n design represents a system in which a no data is shared, butis instead transmitted directly between components. In the first two designsa component has do to extra work to determine what information it requiresfrom the available shared information. In the latter two designs a componentmust do extra work to obtain information that is not immediately availableto it (i.e. information that is not in it’s subarchitecture’s working memory).In order to isolate the effects of the architectural alterations from theother runtime behaviours of the resulting systems, it is important that thesearchitectural differences are the only differences that exist between the finalCAS instantiations. It is critical that the systems are compared on the sametask using the same components. CAST was designed to support this kindof experimentation: it allows the structure of instantiations to be changedconsiderably, with few, if any, changes to component code. This has allowedus to take the original implementation described above and create the n-1,n-m, and n-n instantiations without changing component code. This meansthat we can satisfy our original aim of comparing near-identical systems onthe same tasks, with the only variations between them being architecturalones.To measure the effects of the architecture variations, we require metricsthat can be used to highlight these effects. We previously presented a list ofpossible metrics that could be recorded in an implemented CAS system todemonstrate the trade-offs in design space [12, 13]. Ultimately we are inter-ested in measuring how changes to the way information is shared impacts onthe external behaviour of the systems, e.g. how often it successfully completesa task. However, given the limited functionality of our experimental system,these kind of behaviour metrics are relatively uninformative. Instead we have76 Hawes et al.Fig. 2.4. Left panel: Average number of relevant change events received per com-ponent. Right Panel: Average filtering effort per relevant change event received.chosen to focus on lower-level properties of the system. We have comparedthe systems on:1. variations in the number of filtering operations needed to obtain thechange events necessary to get information to components as required bythe task.2. variations in the number of communication events required to moveinformation around the system.As discussed previously communication and change events underlie the be-haviour of almost all of the processing performed by a system. Thereforechanges in these metrics demonstrate how moving through the space of infor-mation sharing models supported by CAS influences the information process-ing profile of implemented systems.We studied the three different designs in two configurations: one withvision and binding sub-architectures, and the second with these plus the ad-dition of the QSR subarchitecture. This resulted in six final instantiationswhich we tested on three different simulated scenes: scenes containing one ob-ject, two objects and three objects. Each instantiation was run twenty timeson each scene to account for variations unrelated to the system’s design andimplementation.The results for the filtering metric are based around the notion of a rele-vant event. A relevant event is a change event that a component is filteringfor (i.e. an event that it has subscribed to). Figure 2.4 demonstrates the per-centage of relevant events received per component in each instantiation. 100%means that a component only receives change events it is listening for. A lowerpercentage means that the connectivity of the system allows more than therelevant change events to get the component, which then has to filter outthe relevant ones. This is perfectly natural in a shared memory system. Theresults demonstrate that a component in an n-1 instantiation receives the low-2 Architecture and Representations 77est percentage of relevant events. This is because within a subarchitecture, allchanges are broadcast to all components, requiring each component to do alot of filtering work. A component in an n-n instantiation receives the greatestpercentage of relevant changes. This is because each component is shielded bya subarchitecture working memory that only allows change events that arerelevant to the attached components to pass. In the n-n case because only asingle component is in each subarchitecture this number is predictably high9.This figure demonstrates the benefits of a directly connected instantiation:components only receive the information they need.However, this increase in the percentage of relevant changes received comesat a cost. If we factor in the filtering operations being performed at a subar-chitecture level (which could be considered as “routing” operations), we canproduce a figure demonstrating the total number of filtering operations (i.e.both those at a subarchitecture and a component level) per relevant changereceived. This is presented in Figure 2.4. This shows a striking similarity be-tween the results for the n-1 and n-n instantiations, both of which require alarger number of filtering operations per relevant change than the n-m instan-tiations. In the n-m systems, the arrangement of components into functionallythemed sub-architectures results in both smaller numbers of change events be-ing broadcast within sub-architectures (because there are fewer components ineach one), and a smaller number of change events being broadcast outside ofsub-architectures (because the functional grouping means that some changesare only required within particular sub-architectures). These facts mean thatan individual component in an n-m instantiation receives fewer irrelevantchange events that must be rejected by its filter. Conversely a component inthe other instantiations must filter relevant changes from a stream of changescontaining all of the change events in the system. In the n-1 instantiationsthis is because all of these changes are broadcast within a subarchitecture.In the n-n instantiations this is because all of these changes are broadcastbetween sub-architectures. Figure 2.5 (left panel) shows that these results arerobust against changes in the number of objects in a scene. Also, the natureof the results did not change between the systems with vision and bindingcomponents, and those with the additional QSR components.Figure 2.5 (centre panel) demonstrates the average number of commu-nication events per system run across the various scenes and configurationsfor the three different connectivity instantiations. This shows that an n-ninstantiation requires approximately 4000 more communication events on av-erage to perform the same task as the n-1 instantiation, which itself requiresapproximately 2000 more communication events than the n-m instantiation.Figure 2.5 (right panel) demonstrates that this result is robust in the face ofchanges to the number of objects in a scene. The nature of the results also9 The events required by the manager component in each subarchitecture mean therelevant percentage for the n-n instantiations is not 100%.78 Hawes et al.Fig. 2.5. Left panel: Average filtering effort per relevant event compared to scenecomplexity. Centre panel: Average total communication events per instantiation run.Right panel: Average total communication events per instantiation run comparedto scene complexity.did not change between the systems with vision and binding components, andthose with the additional QSR components.This result is due to two properties of the systems. In the n-n system,every interaction between a component a working memory (whether it’s anoperation on information or the propagation of a change event) requires anadditional communication event. This is because all components are sepa-rated by sub-architectures as well as working memories. In addition to this,the number of change events propagated through the systems greatly effectthe amount of communication events that occur. In the n-n and n-1 instanti-ations, the fact that they effectively broadcast all change events throughoutthe system contributes significantly to the communication overhead of thesystem.2.5.3 Filtering summaryFrom these results we can conclude that a functionally-decomposed n-m CASinstantiation occupies a “sweet spot” in architectural design space with ref-erence to filtering and communication costs. This sweet spot occurs becausehaving too much information shared between components in a system (then-1 extreme) means that all components incur an overhead associated withfiltering out relevant information from the irrelevant information. At the otherextreme, when information is not shared by default (the n-n extreme) thereare extra communication costs due to duplicated transmissions between pairsof components, and (in CAS-derived systems at least) the “routing” over-head of transmitting information to the correct components (i.e. the filteringperformed by working memories rather than components).In addition we have demonstrated here an empirical approach to compar-ing different points within design space, where we have held the content of thesystem constant, while making architectural changes. In this way we have alsoshown how to use CAST to help carry out the empirical part of the science ofarchitectures we discussed at the beginning of the chapter. We now proceed2 Architecture and Representations 79to discuss the third of our problems, that of managing processing across thesystem.2.5.4 Processing ManagementIn this section we discuss the problem of how a complex artificial cognitivesystem such as the Explorer or PlayMate systems we describe in Chapters 9and 10 should manage their internal activity. In particular, when the process-ing possibilities exceed the processing resources, how should the robot choosewhat kind of processing to do? We refer to this as the processing managementproblem. We sketch several possible solutions to it, and then discuss the solu-tion we have been exploring, which relies on technologies for planning underuncertainty.To begin with consider a visual system that contains some of the manyalgorithms and representations described in Chapters 4 and 7. Each of theserequires considerable computation to run, even in their classification (or non-learning) mode. In a robot with multiple competences we will need all ofthose vision algorithms and many more besides. For any natural visual scenerunning all such algorithms on all parts of the image is not feasible. This isnot a problem in so far as we never need to perform all visual processing ona scene: the vision we need is determined by the task we are performing. Totackle this there has been much work on attention, and in particular the useof visual saliency models to identify which parts of the scene to process. Therehas been little or no work, however, on how to select which algorithms to run.It is this issue that we address here.In our PlayMate domain, both a robot and human can converse aboutand manipulate objects on a table top (see [14]). As described previously,typical visual processing tasks in this domain require the ability to find thecolour, shape identity or category of objects in the scene to support dialoguesabout their properties; to see where to grasp an object; to plan an obstaclefree path to do so and then move it to a new location; to identify groupsof objects and understand their spatial relations; and to recognize actionsthe human performs on the objects. Each of these vision problems is hardin itself, together they are extremely challenging. The challenge is to build avision system able to tackle all these tasks. One early architectural approach torobot vision was to attempt a general purpose, complete scene reconstruction,and then query this model for each task. This is still not possible and in theopinion of many vision researchers will remain so. An idea with a growing bodyof evidence from both animals and robots is that some visual processing canbe made more effective by tailoring it to the task/environment ([15, 16, 17].Consider the scene in Figure 2.6, and consider the types of visual opera-tions that the robot would need to perform to answer a variety of questionsthat a human might ask it about the scene: “is there a blue triangle in thescene?”, “what is the colour of the mug?”, “how many objects are there inthe scene?”. In order to answer these questions, the robot has at its disposal80 Hawes et al.Fig. 2.6. A picture of the typical table top scenario—ROIs detected for processingare bounded by rectangular boxes.a range of information processing functions and sensing actions. But, in anyreasonably complex scenario (such as the one described above), it is not feasi-ble (and definitely not efficient) for the robot to run all available informationprocessing functions and sensing actions, especially since the cognitive robotsystem needs to respond to human queries/commands in real-time.There are many approaches that could be taken. The key choices concernhow bottom up and top down processing are mixed. Within the constraintsprovided by CAS we have explored at least three approaches. In the first twoapproaches, processing opportunities can be identified as data arrives, andrequests for processing resources be posted by the components to a local taskmanager. The task manager may simply have a policy which is unvarying,e.g. it permits all requests, or grants them up to some load threshold. Thisis what we have done for most our visual systems to date, because they havebeen very small. Alternatively the allocation policy could change accordingto the mode that the task manager is in. This is the approach we took in thecommunication sub-system for the Explorer and PlayMate systems. In thissub-architecture the task manager has modes, which it can switch between,and which are associated with different resource allocation policies. The thirdapproach, and the one we detail here is to drive the processing in a largelytop down way. In reality a mix of both top down and bottom up processingwill be required. A top down approach essentially takes the current robottask, and uses this to determine which processing will be performed. Thereare several ways that we can conceive of the top determination of processing.In principle a (perhaps learned) task specific visual routine could be invokedfrom a library. The problem with this approach is that it is not at all obvioushow to generalise from one task to another from a set of learned instances.A different approach would be to use planning to compose a completely newvisual routine on the fly. The problem with this approach is that the planning2 Architecture and Representations 81process is itself expensive. However, we explore this approach, and show thatit can work.There already exists a body of impressive work on planning of image pro-cessing ([18, 19, 20, 21]). However, it is largely used for single images, re-quires specialist domain knowledge to perform re-planning or plan repair, hasonly been extended to robotic systems in the most limited ways, and posesthe problem in an essentially deterministic planning framework or as a MDP([21]). In our approach we push the field of planning visual processing in anew direction by posing the problem as an instance of probabilistic sequen-tial decision making. We pose it as a Partially Observable Markov DecisionProcess (POMDP), thereby taking explicit, quantitative account of the un-reliability of visual processing. Our main technical contribution is that weshow how to contain one aspect of the intractability inherent in POMDPs forthis domain by defining a new kind of hierarchical POMDP. We compare thisapproach with an earlier formulation based on the Continual Planning (CP)framework of [22]. Using a real robot domain, we show empirically that bothplanning methods are quicker than naive visual processing of the whole scene,even taking into account the planning time. The key benefit of the POMDPapproach is that the plans, while taking slightly longer to execute than thoseproduced by the CP approach, provide significantly more reliable visual pro-cessing than either naive processing or the CP approach. We give an overviewof the POMDP approach here, describe the results, and relate it back to theCAS framework.A POMDP formulation of visual processingIn robot applications, typically the true state of the world cannot be observeddirectly. The robot can only revise its belief about the possible current statesby executing actions, for instance one of the visual operators.We pose the problem as an instance of probabilistic sequential decisionmaking, and more specifically as a Partially Observable Markov Decision Pro-cess (POMDP) where we explicitly model the unreliability of the visual op-erators/actions. This probabilistic formulation enables the robot to maintaina probability distribution (the belief state) over the true underlying state.To do this we need an observation model that captures the likelihood of theoutcomes from each action. In this paper, we only consider actions that havepurely informational effects. In other words, we do not consider actions suchas poking the object to determine its properties, with the consequence thatthe underlying state does not change when the actions are applied. However,the POMDP formulation allows us to do this, which is necessary if we wishto model the effects of operators that split ROIs, move the camera, or movethe objects to gain visual information about them.Each action considers the true underlying state to be composed of thenormal class labels (e.g. red(R), green(G), blue(B) for color; circle(C), trian-gle(T), square(S) for shape; picture, mug, box for sift), a label to denote the82 Hawes et al.absence of any object/valid class—empty (E), and a label to denote the pres-ence of multiple classes (M). The observation model for each action provides aprobability distribution over the set composed of the normal class labels, theclass label empty (E) that implies that the match probability corresponding tothe normal class labels is very low, and unknown (U) that means that there isno single class label to be relied upon and that multiple classes may thereforebe present. Note that U is an observation, whereas M is part of the underlyingstate: they are not the same, since they are not perfectly correlated.Since visual operators only update belief states, we include “special ac-tions” that answer the query by “saying” (not to be confused with language-based communication) which underlying state is most likely to be the truestate. Such actions cause a transition to a terminal state where no furtheractions are applied. In the description below, for ease of explanation (andwithout loss of generality) we only consider two operators: color and shape,denoting them with the subscripts c, s respectively. States and observationsare distinguished by the superscripts a, o respectively.Consider a single ROI in the scene—it has a POMDP associated with itfor the goal of answering a specific query. This POMDP is defined by the tuple〈S,A, T ,Z,O,R〉:• S : Sc × Ss ∪ term, the set of states, is a cartesian product of the statespaces of the individual actions. It also includes a terminal state (term).Sc : {Eac , Rac , Gac , Bac ,Mc}, Ss : {Eas , Cas , T as , Sas ,Ms}• A : {color, shape, sRed, sGreen, sBlue} is the set of actions. The first twoentries are the visual processing actions. The rest are special actions thatrepresent responses to the query such as “say blue”, and lead to term. Herewe only specify “say” actions for color labels, but others may be addedtrivially.• T : S ×A× S → [0, 1] represents the state transition function. For visualprocessing actions it is an identity matrix, since the underlying state ofthe world does not change when they are applied. For special actions itrepresents a transition to term.• Z : {Eoc , Roc , Goc , Boc , Uc, Eos , Cos , T os , Sos , Us} is the set of observations, aconcatenation of the observations for each visual processing action.• O : S×A×Z → [0, 1] is the observation function, a matrix of size |S|×|Z|for each action under consideration. It is learned by the robot for the visualactions (described in the next section), and it is a uniform distribution forthe special actions.• R : S ×A → <, specifies the reward, mapping from the state-action spaceto real numbers. In our case:∀s ∈ S, R(s, shape) = −1.25 · f(ROI-size)R(s, color) = −2.5 · f(ROI-size)R(s, special actions) = ±100 · α2 Architecture and Representations 83For visual actions, the cost depends on the size of the ROI (polynomialfunction of ROI size) and the relative computational complexity (the coloroperator is twice as costly as shape). For special actions, a large +ve (-ve)reward is assigned for making a right (wrong) decision for a given query.For e.g. while determining the ROI’s color:R(RacT as , sRed) = 100 · α,R(Bac T as , sGreen) = −100 · αbut while computing the location of red objects:R(Bac T as , sGreen) = 100 · α. The variable α enables the trade-off betweenthe computational costs of visual processing and the reliability of the an-swer to the query.Our visual planning task for a single ROI now involves solving thisPOMDP to find a policy that maximizes reward from the initial belief state.Plan execution corresponds to traversing a policy tree, repeatedly choosingthe action with the highest value at the current belief state, and updating thebelief state after executing that action and getting a particular observation. Inorder to ensure that the observations are independent (required for POMDPbelief updating to hold), we take a new image of the scene if an action isrepeated on the same ROI.Actual scenes will have several objects and hence several ROIs. Attemptingto solve a POMDP in the joint space of all ROIs soon becomes intractable dueto an exponential state explosion, even for a small set of ROIs and actions.For a single ROI with m features (color, shape, etc.) each with n values, thePOMDP has an underlying space of nm; for k ROIs the overall space becomes:nmk. Instead, we propose a hierarchical decomposition: we model each ROIwith a lower-level (LL) POMDP as described above, and use a higher-level(HL) POMDP to choose, at each step, the ROI whose policy tree (generated bysolving the corresponding LL-POMDP) is to be executed. This decomposesthe overall problem into one POMDP with state space k, and k POMDPswith state space nm. Space does not permit us to give the full details of thehierarchical POMDP formulation here, but these can be found in [23]. Thekey technical point is that in the HL-POMDP the observation function andthe cost/reward specification for each action is based on the policy tree of aLL-POMDP that corresponds to that action. An example of the type of policytree found for a LL-POMDP is given in Figure 2.7 where the LL-POMDP’spolicy tree has the root node representing the initial belief when the visualroutine is called. At each node, the LL-POMDP’s policy is used to determinethe best action, and all possible observations are considered to determine theresultant beliefs and hence populate the next level of the tree.Once the observation functions and costs are computed, the HL-POMDPmodel can be built and solved to yield the HL policy. During execution, theHL-POMDP’s policy is queried for the best action choice, which causes theexecution of one of the LL-POMDP policies, resulting in a sequence of visualoperators being applied on one of the image ROIs. The answer provided bythe LL-POMDP’s policy execution causes a belief update in the HL-POMDP,84 Hawes et al.Fig. 2.7. Policy Tree of an ROI—each node represents a belief state and specifiesthe action to take.and the process continues until a terminal action is chosen at the HL, therebyanswering the query posed. Here it provides the locations of all blue objectsin the scene. For simpler occurrence queries (e.g. “Is there a blue object in thescene?”) the execution can be terminated at the first occurrence of the objectin a ROI. Both the LL and HL POMDPs are query dependent. Solving thePOMDPs efficiently is hence crucial to overall performance.A Continual Planning FormulationThe Continual Planning (CP) approach of [22] interleaves planning, plan exe-cution and plan monitoring. Unlike classical planning approaches that requireprior knowledge of state, action outcomes, and all contingencies, an agent inCP postpones reasoning about unknowable or uncertain states until more in-formation is available. It achieves this by allowing actions to assert that thepreconditions for the action will be met when the agent reaches that pointin the execution of the plan, and if those preconditions are not met duringexecution (or are met earlier), replanning is triggered. But there is no rep-resentation of the uncertainty/noise in the observation/actions. It uses thePDDL ([24]) syntax and is based on the FF planner of [25]. Consider theexample of a color operator:(:action colorDetector:agent (?a - robot):parameters (?vr - visRegion ?colorP - colorProp ):precondition (not (applied-colorDetector ?vr) ):replan (containsColor ?vr ?colorP)2 Architecture and Representations 85:effect (and(applied-colorDetector ?vr)(containsColor ?vr ?colorP) ) )The parameters are a color-property (e.g. blue) being searched for in aparticular ROI. It can be applied on any ROI that satisfies the precondition i.e.it has not already been analyzed. The expected result is that the desired coloris found in the ROI. The “replan:” condition ensures that if the robot observesthe ROI’s color by another process, replanning is triggered to generate a planthat excludes this action. This new plan will use the containsColor fact fromthe new state instead. In addition, if the results of executing a plan step arenot as expected, replanning (triggered by execution monitoring) ensures thatother ROIs are considered. Other operators are defined similarly, and based onthe goal state definition the planner chooses the sequence of operators whoseeffects provide parts of the goal state—the next section provides an example.The CP approach to the problem is more responsive to an unpredictableworld than a non-continual classical planning approach would be, and it cantherefore reduce planning time in the event of deviations from expectations.But, while actions still have non-deterministic effects, there is no means foraccumulating belief over successive applications of operators. We show thatthe HiPPo formulation provides significantly better performance than CP indomains with uncertainty.An example queryFigs 8(a)-8(d) show one execution example for an image with two ROIs.The example query is to determine the presence and location of one ormore blue circles in the scene (Fig 8(a)). Since both ROIs are equally likelytarget locations, the HL-POMDP first chooses to execute the policy tree ofthe first ROI (action u1 in Fig 8(b)). The corresponding LL-POMDP runs thecolor operator on the ROI. The outcome of applying any individual operatoris the observation with the maximum probability, which is used to updatethe subsequent belief state—in this case the answer is red. Even though it ismore costly, the color operator is applied before shape because it has a higherlikelihood of success, based on the learned observation functions. When theoutcome of red increases the likelihood (belief) of the states that represent the“Red” property as compared to the other states, the likelihood of finding ablue circle is reduced significantly. The dynamic reward specification (α = 0.2)ensures that without further investigation (for instance with a shape opera-tor), the best action chosen at the next level is a terminal action associatedwith the “Red” property—in this case it is sRedSquare. The HL-POMDP re-ceives the input that a red square is found in R1, leading to a belief updateand subsequent action selection (action u2 in Fig 8(c)). Then the policy treeof the LL-POMDP of R2 is invoked, causing the color and shape operatorsto be applied in turn on the ROI. The higher noise in the shape operator isthe reason why it has to be applied twice before the uncertainty is sufficiently86 Hawes et al.(a) Input image. (b) Execution Step 1.(c) Execution Step 2. (d) Execution Step 3.Fig. 2.8. Example query: “Where is the Blue Circle?” Dynamic reward specificationin the LL-POMDP allows for early termination when negative evidence is found.reduced to cause the choice of a terminal action (sBlueCircle)—the increasedreliability therefore comes at the cost of execution overhead. This results inthe belief update and terminal action selection in the HL-POMDP—the finalanswer is (s¬R1∧R2), i.e. that a blue circle exists in R2 and not R1 (Fig 8(d)).In our HiPPo representation, each HL-POMDP action chooses to executethe policy of one of the LL-POMDPs until termination, instead of performingjust one action. The challenge here is the difficulty of translating from theLL belief to the HL belief in a way that can be planned with. The executionexample above shows that our approach still does the right thing, i.e. it stopsearly if it finds negative evidence for the target object. Finding positive evi-dence can only increase the posterior of the ROI currently being explored, soif the HL-POMDP were to choose the next action, it would choose to explorethe same ROI again.If the same problem were to be solved with the CP approach, the goalstate would be defined as the PDDL string:(and (exists ?vr - visRegion) (and (containsColor ?vr Blue) (containsShape?vr Circle) ) )2 Architecture and Representations 87(a) HiPPo vs. joint POMDP. JointPOMDP soon becomes intractable.(b) Planning times of HiPPo vs. CP.Policy-caching makes results compara-ble.(c) Execution times of HiPPo, CP vs.No planning. Planning makes execu-tion faster.(d) Planning+execution times ofHiPPo, CP vs. No planning. Planningapproaches reduce processing time.Fig. 2.9. Experimental Results—Comparing planning and execution times of theplanners against no planning.i.e. the goal is to determine the existence of a ROI which has the colorblue and shape circle. The planner must then find a sequence of opera-tors to satisfy the goal state. In this case it leads to the creation of theplan:(colorDetector robot vr0 blue)(shapeDetector robot vr0 circle)i.e. the robot is to apply the color operator, followed by the shape operatoron the first ROI. There is a single execution of each operator on the ROI. Evenif (due to image noise) an operator determines a wrong class label as the closestmatch with a low probability, there is no direct mechanism to incorporatethe knowledge. Any thresholds will have to be carefully tuned to preventmis-classifications. Assuming that the color operator works correctly in thisexample, it would classify the ROI as being red, which would be determinedin the plan monitoring phase. Since the desired outcome (finding blue in thefirst ROI) was not achieved, replanning is triggered to create a new plan with88 Hawes et al.the same steps, but to be applied on the second ROI. This new plan leads tothe desired conclusion of finding the blue circle in R2 (assuming the operatorswork correctly).An experimental studyIn considering the trade-offs between the different types of solution to theprocessing management problem we consider several hypotheses that we cantest. Specifically we hypothesise that:• The hierarchical-POMDP planning (HiPPo) formulation is more efficientthan the standard POMDP formulation.• HiPPo and CP have comparable plan-time complexity.• Planning is significantly more efficient than blindly applying all operatorson the scene.• HiPPo has higher execution time than CP but provides more reliable re-sults.In order to test these hypotheses we ran several experiments on the robotin the tabletop scenario. Objects were placed on the table and the robot had toanalyze the scene to answer user-provided queries. Query categories include:• Occurrence queries: Is there a red mug in the scene?• Location queries: Where in the image is the blue circle?• Property queries: What is the color of the box?• Scene queries: How many green squares are there in the scene?For each query category, we ran experiments over ∼ 15 different querieswith multiple trials for each such query, thereby representing a range of vi-sual operator combinations in the planning approaches. We also repeated thequeries for different numbers of ROIs in the image. In addition, we also im-plemented the naive approach of applying all available operators (color, shapeand sift in our experiments) on each ROI, until a ROI with the desired prop-erties is found and/or all ROIs have been analyzed.Unlike the standard POMDP solution that considers the joint state spaceof several ROIs, the hierarchical representation does not provide the optimalsolution (policy). Executing the hierarchical policy may be arbitrarily worsethan the optimal policy. For instance, in the search for the blue region, thehierarchical representation is optimal iff every ROI is blue-colored. But asseen in Figure 9(a) that compares the planning complexity of HiPPo withthe standard POMDP solution, the non-hierarchical approach soon becomesintractable. The hierarchical approach provides a significant reduction in theplanning time and (as seen below) still increases reliability significantly.Next, we compare the planning times of HiPPo and CP approaches asa function of the number of ROIs in the scene—Figure 9(b). The standardhierarchical approach takes more time than CP. But, the computationallyintensive part of HiPPo is the computation of the policies for the ROIs. Since2 Architecture and Representations 89Approach % ReliabilityNaive 76.67CP 76.67HiPPo 91.67Table 2.1. Reliability of visual processingthe policies computed for a specific query are essentially the same for allscene ROIs, they can be cached and not repeated for every ROI. This simpleadjustment drastically reduces the planning time and makes it comparable tothe CP approach.Figure 9(c) compares the execution time of the planning approachesagainst applying all the operators on each ROI until the desired result isfound. The HiPPo approach has a larger execution time than CP because itmay apply the same operator multiple times to a single ROI (in different im-ages of the same scene) in order to reduce the uncertainty in its belief state.In all our experiments the algorithms are being tested on-board a cognitiverobot which has multiple modules to analyze input from different modalities(vision, tactile, speech) and has to bind the information from the differentsources. Hence, though the individual actions are optimized and represent thestate-of-the-art in visual processing, they take some time to execute on therobot.A key goal of our approach is not only to reduce overall planning and execu-tion time, but to improve the reliability of the visual processing. In these termsthe benefits are very clear, as can be seen in Table 2.1. The direct applicationof the actions on all the ROIs in the scene results in an average classifica-tion accuracy of 76.67%, i.e. the sensing actions misclassify around one-fourthof the objects. Using CP also results in the same accuracy of 76.67%, i.e.it only reduces the execution time since there is no direct mechanism in thenon-probabilistic planner to exploit the outputs of the individual operators (adistribution over the possible outcomes). The HiPPo approach is designed toutilize these outputs to reduce the uncertainty in belief, and though it causesan increase in the execution time, it results in much higher classification accu-racy: 91.67%. It is able to recover from a few noisy images where the operatorsare not able to provide the correct class label, and it fails only in cases wherethere is consistent noise. A similar performance is observed if additional noiseis added during execution. As the non-hierarchical POMDP approach takesdays to compute the plan for just two ROIs we did not compute the optimalplan for scenes with more than two ROIs, but for the cases where a plan wascomputed, there was no significant difference between the optimal approachand HiPPo in terms of the execution time and reliability, even though thepolicy generated by HiPPo can be arbitrarily worse than that generated bythe non-hierarchical approach.A significant benefit of the POMDP approach is that it provides a readymechanism to include initial belief in decision-making. For instance, in the90 Hawes et al.example considered above, if R2 has a higher initial belief that it contains ablue circle, then the cost of executing that ROI’s policy would be lower andit would automatically get chosen to be analyzed first leading to a quickerresponse to the query.Figure 9(d) shows a comparison of the combined planning and executiontimes for HiPPo, CP, and the naive approach of applying all actions in all ROIs(no planning). As the figure shows, planning is worthwhile even on scenes withonly two ROIs. In simple cases where there are only a couple of operatorsand/or only one operator for each feature (color, shape, object recognitionetc) one may argue that rules may be written to decide on the sequence ofoperations. But as soon as the number of operators increase and/or there ismore than one operator for each feature (e.g. two actions that can find colorin a ROI, each with a different reliability), planning becomes an appealingoption.Summary of processing management workIn this sub-section we have explored the implications of the fact that withina complex cognitive system with many goals it will not be possible to per-form all processing. We have studied this problem within the context of vi-sion, specifically the kind of visual sub-architecture we use for the PlayMatescenario. Architecturally there are many possible solutions. Bottom up, data-driven processing is implemented naturally in CAS. In this section we haveshown how to augment it with top down control, achieved using techniquesfor planning under uncertainty, and continual planning.2.6 The relationship of CAS to previous work onarchitectures2.6.1 Cognitive ArchitecturesThere have been several attempts at unified theories of intelligence from withincognitive science. At least two of these emphasise the role of production sys-tems. In SOAR Newell and Laird [26] proposed a production system model inwhich serial application of rules, written in a common form, modified represen-tations held in a workspace shared by those rules. An important componentof the theory was that there was a single unified representational languagewithin which all data held in the shared workspace was expressed. Anotherkey idea was that a set of meta-rules controlled the serial application of theseproductions. These three key ideas: a single shared workspace, serial appli-cation of processing elements, and a common representational language havebeen extremely influential. They are both simple to comprehend and allowthe construction of effective systems. In ACT-R [27] John Andersen and col-leagues have taken some of the elements of production systems and used them2 Architecture and Representations 91to produce models of aspects of human cognition that produce testable pre-dictions. In ACT-R productions now represent the serial actions of processingin the thalamus and connect to information in buffers. Together these simu-late the behaviour of multiple thalamic-cortical loops. ACT-R has been usedto construct models that give impressively accurate predictions for humanperformance on a range of tasks, including reading and mental arithmetic.ACT-R models rely heavily on the provision of timing information about de-lays in each stage of processing. Both SOAR and ACT-R have in common thefact that they have widely available languages that allow researchers to im-plement computational models. Finally in Global Workspace Theory (GWT),Baars and Shanahan [6] have proposed the idea that conscious thought is ex-plainable at an abstract level by the idea of a global workspace. The key ideain GWT is that local processes propose items to be posted onto a single globalworkspace, and that mechanisms exist that select one collection of items thatare in turn re-transmitted to all the local processes. It can be seen that allthree theories emphasise the idea of a single shared workspace for parts ofcognition. There is evidence however, at least from robotics, that such a sin-gle workspace is an incomplete architectural account of intelligence. I nowturn to describe ideas from robotics on architectures, in order to compare andcontrast them with the ideas from work on cognitive architectures.2.6.2 Robotic ArchitecturesThe first significant attempt to implement what might be loosely called acognitive robot was the Shakey project [28]. Detailed examination of theirapproach bears fruit. The architecture in Shakey was dominated by a cen-tral workspace within which all data about the contingent state of the worldwas written in a single representational language. In the case of Shakey thiswas a form of first order predicate logic minus quantification. Sensory pro-cessing was essentially a business of abstracting from the raw sensor data tothis predicate description. Typing of entities was captured using predicates,and the representation also captured some metric information for the highlysimplified world. Qualitative action effects were captured using STRIPS oper-ators, which addressed some of the difficulties previously encountered by thesituation calculus. This declarative knowledge about action effects was usedin a planning process that had available all knowledge in the world model.In addition to this the robot had fixed routines that would update the worldmodel when sufficient uncertainty had accumulated about its state. This wasthe way that gross error recovery occurred: through re-sensing and then re-calculating the world model. Simpler errors were handled using IntermediateLevel Actions (ILAs). These were essentially discrete closed loop controllersthat relied upon the world to settle between each step and thus couldn’t dealeasily with ongoing rapid change. Overall, Shakey shares, at an architecturallevel, some of the assumptions of SOAR and ACT-R. It relies on serial appli-cation of planning operators to simulate trajectories through the state space92 Hawes et al.and selects courses of action based on those. It uses a single representationallanguage, although it does reason with that representation using two differ-ent kinds of reasoning. It has completely serial control of execution: only oneILA is in control of the robot at a time. It collects all contingent knowledgeabout the world in a single shared workspace. It handles error recovery in twoways: re-sensing leading to model updating and re-planning, and closed looprecovery from errors without planning. Finally it does not provide an archi-tectural answer to the problem of sensory interpretation: perceptual routinesexisted in Shakey, but there are no architectural constraints or aids to howthey operate, communicate, or share information. They serve only to provideinformation to a central model in a unified language. The classic story aboutShakey given by behaviour based roboticists, is that it could never have workedoutside of its carefully controlled environment, and that even within it perfor-mance was unreliable. Shakey was able to perform different tasks, but it reliedupon an accurate world model. It was able to construct a sufficiently accuraterepresentation under benign sensory conditions, but robot vision researchersunsuccessfully spent the decade following the Shakey project trying to extendthis approach of scene reconstruction to more natural visual environments.Thirty-five years later our ability to perform scene or surface reconstructionis still poor, although it has improved. Of course to be fair to the designersof Shakey it is not clear that they took a principled stance on whether allaspects of a scene should be recovered, only that attempts to extend theirapproach often sought to do this, and have largely failed to date. A strongreaction to this paradigm that occurred in the mid 1980s was exemplified bythe behaviour based approach to robotics [29]. The approach is characterisedby a number of authors including proponents and sceptics. One key idea isthat the system is almost entirely representation free. The meaning of thisstatement depends on what is meant by representation. Kirsch [30] describesthree types of representation: data items the values of which co-vary withfeatures in the world; declarative statements which release their informationwhen queried; and predicate descriptions that allow generalisation by type,leading to the ability to reason about inheritance. Brooks’ early robots werecertainly representation free on the basis of either of the last two definitions.Furthermore, while modules may have representations of the first kind, theydo not typically transmit these representations to other modules for furtherprocessing or consumption. In other words there is no sharing of information,only competition for control of the robot. Other important aspects of theapproach are that many controllers run in parallel, that each is relatively sim-ple, and that their action recommendations are fused through a single globalmechanism. The obvious weakness of the behaviour based approach is the lackof evidence of its ability to scale to higher cognitive functions, despite nearly aquarter of a century of effort. Instead, roboticists typically use behaviours asthe lowest level of control in a hierarchical system. Three tiered architecturessuch as 3T [31] employ behaviours at the lowest level, and link these to asymbolic planning level via a sequencing level in which transitions are made2 Architecture and Representations 93from behaviour to behaviour using a finite state machine like representation.Representations have made a re-appearance via advances in filtering and sta-tistical approaches imported from machine learning. Behaviours, rather thantruly behaviour based approaches have thus been merged into the tool-boxof techniques employed by most roboticists within architectures, rather thanbeing an architectural choice in their own right.2.7 Summary of contributions and conclusionsIn this chapter we have explored a small part of the space of designs for a par-ticular part of niche space. Working from run and design time requirementsimposed by our combination of task and environment (our two scenarios) wehave suggested one architectural schema (CAS). We have argued that CASincludes a large number of architectural instantiations and sub-schemas thatmeet those requirements. From our experience of building real robot systemsusing the software implementation of the schema (CAST) we have identifiedfour problems which are common to a very large range of systems, and whichwe argue warrant architectural solutions. These are the problems of binding,filtering, processing management and action fusion. We have gone on to detailour work within CAST to create solutions to each of these problems. Finallywe have tried to show that an empirical science of architectures is possible.The work in this aspect of CoSy has been exceptionally useful in allowing usto integrate the work of many of the other chapters. In particular we believethat any true systems approach to AI must include an architectural theory. Webelieve that CAS represents a significant step forward in architectures for em-bodied cognitive systems because of the way that it combines the parallelismand incrementality of behaviour based systems with the use of representationsthat should lie at the heart of any cognitive system.References1. A. Sloman, M. Scheutz, A framework for comparing agent architectures, in: InUK Workshop on Computational Intelligence, 2002, pp. 169–176.2. M. Minsky, P. Singh, A. Sloman, The st. thomas common sense symposium:Designing architectures for human-level intelligence, AI Magazine Summer 2004.3. R. C. Arkin, Behavior-Based Robotics. Intelligent robots and autonomousagents., MIT Press, 1998.4. J. Wolfe, K. Cave, The psychophysical evidence for a binding problem in humanvision, Neuron 24 (1) (1999) 11–17.5. F. van der Velde, M. de Kamps, Neural blackboard architectures of combinato-rial structures in cognition., Behavioral and Brain Sciences 29 (2006) 37–70.6. M. P. Shanahan, B. J. Baars, Applying global workspace theory to the frameproblem, Cognition 98 (2) (2005) 157–176.94 Hawes et al.7. G.-J. M. Kruijff, J. D. Kelleher, N. Hawes, Information fusion for visualreference resolution in dynamic situated dialogue, in: E. Andre, L. Dybkjaer,W. Minker, H. Neumann, M. Weber (Eds.), Perception and Interactive Tech-nologies: International Tutorial and Research Workshop, PIT 2006, Vol. 4021of Lecture Notes in Computer Science, Springer Berlin / Heidelberg, KlosterIrsee, Germany, 2006, pp. 117 – 128.URL http://www.cognitivesystems.org/cosybook/chap2.asp#Kruijff/etal:20068. H. Jacobsson, N. Hawes, G.-J. Kruijff, J. Wyatt, Crossmodal content bindingin information-processing architectures, in: HRI ’08: Proceedings of the 3rdACM/IEEE International Conference on Human Robot Interaction, ACM,New York, NY, USA, 2008, pp. 81–88.URL http://www.cognitivesystems.org/cosybook/chap2.asp#Jacobsson/etal:2008a9. S. Harnad, The symbol grounding problem, Physica D: Nonlinear Phenomena42 (1990) 335–346.10. D. Skočaj, M. Kristan, A. Leonardis, Continuous learning of simple visual con-cepts using incremental kernel density estimation, in: International Conferenceon Computer Vision Theory and Applications, Funchal, Madeira, Portugal,2008, pp. 598–604.URL http://cognitivesystems.org/cosybook/chap2.asp#skocajVISAPP0811. M. Brenner, N. Hawes, J. Kelleher, J. Wyatt, Mediating between qualitativeand quantitative representations for task-orientated human-robot interaction,in: M. M. Veloso (Ed.), IJCAI 2007, Proceedings of the 20th International JointConference on Artificial Intelligence, 2007, pp. 2072–2077.URL http://www.cognitivesystems.org/cosybook/chap2.asp#Brenner/etal:200712. N. Hawes, A. Sloman, J. Wyatt, Towards an empirical exploration of designspace, in: G. A. Kaminka, C. R. Burghart (Eds.), Evaluating Architectures forIntelligence: Papers from the 2007 AAAI Workshop, AAAI Press, Vancouver,Canada, 2007, pp. 31 – 35.URL http://www.cognitivesystems.org/cosybook/chap2.asp#Hawes/etal:2007c13. N. Hawes, J. Wyatt, A. Sloman, Exploring design space for an integratedintelligent system, in: M. Bramer, F. Coenen, M. Petridis (Eds.), Researchand Development in Intelligent Systems XXV: Proceedings of AI-2008, TheTwenty-eighth SGAI International Conference on Innovative Techniques andApplications of Artificial Intelligence, Springer, Cambridge, England, 2008.URL http://www.cognitivesystems.org/publications/cosyBib2008.asp#Hawes/etal:200814. N. Hawes, A. Sloman, J. Wyatt, M. Zillich, H. Jacobsson, G.-J. Kruijff, M. Bren-ner, G. Berginc, D. Skočaj, Towards an integrated robot with multiple cognitivefunctions, in: R. C. Holte, A. Howe (Eds.), Proceedings of the Twenty-SecondAAAI Conference on Artificial Intelligence (AAAI 2008), AAAI Press, Vancou-ver, Canada, 2007, pp. 1548 – 1553.URL http://www.cognitivesystems.org/cosybook/chap2.asp#Hawes/etal:2007a15. D. N. Lee, The Optical Flow-field: The Foundation of Vision, PhilosophicalTransactions of the Royal Society London B 290 (1980) 169–179.2 Architecture and Representations 9516. I. Horswill, Polly: A Vision-Based Artificial Agent, in: AAAI, 1993, pp. 824–829.17. M. F. Land, M. Hayhoe, In What Ways do Eye Movements Contribute to Ev-eryday Activities, Vision Research 41 (2001) 3559–3565.18. R. Clouard, A. Elmoataz, C. Porquet, M. Revenu, Borg: A knowledge-basedsystem for automatic generation of image processing programs, PAMI 21.19. M. Thonnat, S. Moisan, What can program supervision do for program reuse?,IEE Proc. Software.20. S. Moisan, Program supervision: Yakl and pegase+ reference and user manual,Rapport de Recherche 5066, INRIA, Sophia Antipolis, France (December 2003).21. L. Li, V. Bulitko, R. Greiner, I. Levner, Improving an Adaptive Image Interpre-tation System by Leveraging, in: Australian and New Zealand Conference onIntelligent Information Systems, 2003.22. M. Brenner, B. Nebel, Continual planning and acting in dynamic multiagentenvironments, in: International Symposium on Practical Cognitive Agents andRobots, Perth, Australia, 2006.URL http://www.cognitivesystems.org/cosybook/chap6.asp#Brenner/etal:2006a23. M. Sridharan, J. Wyatt, R. Dearden, HiPPo: Hierarchical POMDPs for Plan-ning Information Processing and Sensing Actions on a Robot, in: InternationalConference on Automated Planning and Scheduling (ICAPS), 2008.URL http://www.cognitivesystems.org/cosybook/chap2.asp#Sridharan/etal:2008a24. D. McDermott, PDDL: The Planning Domain Definition Language, TechnicalReport TR-98-003/DCS TR-1165, Tech. rep., Yale Center for ComputationalVision and Control (1998).25. J. Hoffmann, B. Nebel, The FF Planning System: Fast Plan Generation ThroughHeuristic Search, Journal of Artificial Intelligence Research 14 (2001) 253–302.26. J. E. Laird, A. Newell, P. S. Rosenbloom, Soar: An architecture for generalintelligence, Artificial Intelligence 33 (3) (1987) 1–64.27. J. Anderson, D. Bothell, M. Byrne, S. Douglass, C. Lebiere, Y. Qin, An inte-grated theory of the mind, Psychological Review 111 (4) (2004) 1036–1060.28. N. Nilsson, Shakey the robot, Tech. Rep. Tech Note 323, AI Center, SRI Inter-national (April 1984).29. R. A. Brooks, Intelligence without representation, Artificial Intelligence (47)(1991) 139–159.30. D. Kirsch, Today the earwig, tomorrow man?, Artificial Intelligence 47 (1991)161–184.31. E. Gat, On three-layer architectures, in: Artificial Intelligence and MobileRobots, 1997.3The Sensorimotor Approach in CoSy: TheExample of Dimensionality ReductionDavid Philipona, J. Kevin O’ReganLaboratoire Psychologie de la Perception, Université Paris Descartes and CNRSjkevin.oregan@gmail.com3.1 IntroductionThis chapter presents an algorithm implementing a basic requirement for anyinterface between sensory data and cognitive functions: dimensionality reduc-tion. The algorithm extends the classical framework of dimensionality reduc-tion to the case where sensory data are acquired through an embodied agent,by grounding the metric that is at the basis of the dimensionality reductionin the sensorimotor abilities of the agent. The final objective (which was notrealized because of time constraints) within CoSy was to build on this to pro-vide a demonstration of some basic unsupervised learning of interactions withspace and objects, as would be required in an explorer-type scenario.The algorithm provides several functionalities for the local analysis of sen-sory data: it can estimate a code independent distance between sensory datapoints, suggest actions to be taken so as to link neighboring sensory configu-rations, and decompose possible sensory variations into sets of variations withstructurally different properties. Additionally, it can provide a local flatten-ing of the sensory manifold and therefore it can be used in the way classicaldimensionality reduction is used.The work described is situated within an approach called the “sensorimo-tor approach” ([1]). The approach derives its motivation from the idea thatbiological and artificial systems that have a variety of different types of sen-sory inputs, and that have diverse and complicated effector mechanisms, arebest described in a way that does not depend in an essential way on the par-ticular sensory or motor systems involved, but is instead code-independent.Note that the sensorimotor approach, as understood here, is quite a radicaldeparture from classic views of natural or artificial perception. It should bedistinguished from other approaches where the term “sensorimotor” is usedmore loosely (cf. e.g. [2]), and from approaches like those of‘ ‘Active Vision”(c.f. [3, 4, 5, 6, 7]) where action plays an important role only to the extent that98 Philipona and O’Reganit increases the amount of information available to the system. Here the sen-sorimotor approach is taken to imply the idea that the only type of knowledgeconcerning the environment that a system can access is knowledge about thelaws that link the system’s actions to the resulting changes in sensory input.These sensorimotor laws are the only basic entities that a system can everhave access to.The idea applies in an obvious way to geometrical notions: A system’sunderstanding of the dimensionality and structure of the (probably Euclidean)space in which it is embedded is constrained by the particular movementsthe system can effect, and by the system’s ability to observe the sensorychanges these movements produce. But the idea must also apply to all othersensory notions that a system possesses: for example the notion of its ownbody characteristics, or its tactile perception or its perception of sound orcolor.Indeed, a dditional work performed during the CoSy project using the sen-sorimotor approach concerned color. This work, though very interesting froma fundamental point of view, was of less direct concern to the CoSy platformsbecause using biologically inspired color perception was not considered a mainpriority in the project. The work can be referred to in [8].3.2 Artificial agents and human perceptionSensory data to be dealt with by embodied agents are complex, high dimen-sional and grounded in the continuous domain. On the other hand cognitivefunctions like navigation, planning and communication usually are consideredalgorithmic in nature. For this reason, most research in the field (although foran exception see e.g. [9]) considers that actions, sensory configurations andpossible states of the system are taken from finite sets (this is for instance thestandard framework in machine learning, topological map building, language,dynamic logic, qualitative reasoning: see for instance [10, 11, 12]). The com-plexity of these algorithms is often exponential in the number of such states,and consequently they crucially require tractable information as inputs. Thisrequires modules to be implement that interface low level data acquisition andhigher cognitive processing: for instance, modules that convert visual flow intoa small collection of coordinates and attributes that identify the position andproperties of objects in the field of view.Evolution has endowed biological organisms with neuronal processingschemes adapted to the organisms’ sensors. By analogy it makes sense forartificial systems to consider creating interfacing modules specifically engi-neered for the agent’s sensory devices and the data format used by cognitivefunctions. On the other hand, while device dependent solutions will ensurea tractable level of efficiency and robustness, they also constrain a system’s3 Dimensionality Reduction 99ability to adapt as task requirements and sensorimotor competences change,or as the agent’s body suffers evolution, modifications, changes in environ-mental conditions, or damage. Such issues might presumably be handled ata cognitive level, but if information is erroneously discarded or even merelyimpoverished by early perceptual processing then the task may be very hard.In the case of humans, this seems essentially unrealistic in the long term asit would mobilize too many of the resources required by cognitive functions.Finally, if modifications occur at the sensorimotor level then it seems thatmethods considering a continuous framework will be more adaptable thanmethods conceptually based on the manipulation of a limited set of entities.These arguments all suggest investigating the question of an unsupervisedand multimodal approach to sensory data analysis. Insofar as global interac-tions with space and objects are based on the coupling of cognitive functionswith the mastery of local interactions [13], such an approach would providethe required basic tools. It would ultimately lay the ground for unsupervisedalgorithms converging to the flexible exploitation of the sensorimotor abilitiesavailable to the agent.The issues faced by research in the field of embodied articial agents are alsorelated to research in the field of human perception. There is a long historyof theories about the sensorimotor grounding of perceptual experience, espe-cially as concerns space: Helmholtz at the end of the 19th century had alreadysuggested a general formulation of the notion of displacement for a sensorimo-tor system [14], and Poincaré expressed his thoughts on the relation betweencompensability and space a few decades later [15]. More recently Piaget devel-oped the idea that sensorimotor interactions were fundamental in structuringan individual’s cognitive and perceptual abilities [16], and Gibson foundeda theory of perception tightly linked with action [17]. As a natural outcomeof this history several issues addressed today in developmental psychologyrevolve around the interplay between sensorimotor skills, cognitive/symbolicabilities and perception.In the field of psychophysics, the importance of sensorimotor interactionshas been recently pushed to quite an extreme limit in the idea that perceptionshould be envisioned in terms of cognitive access to the mastery of sensorimo-tor abilities, rather than in terms of passive sensory processing decoupled fromboth cognitive and motor constraints [1]. Though this is more of philosophicalimportance than critical to a concrete project like CoSy, it is worth notingthat the basic motivation for such a radical shift, as compared to the stan-dard view of sensory processing, lies in a central difficulty faced by theoriesof perception: the difficulty in explaining the existence and character of thequalia of sensations [18]. Thus, an approach to perception emphasizing neu-ronal processing will be faced with the a priori impossible task of finding anexplanation of qualia within properties of this processing or properties of theneuronal substrate underlying this processing [19, 20, 21]. On the other hand100 Philipona and O’Reganan approach emphasizing sensorimotor interactions has instead the easier taskof matching qualia to properties of sensorimotor interactions.3.3 Dimensionality reductionTo address the mismatch between the complexity existing at the sensor leveland the requirement that cognitive processing should be tractable, a com-mon hypothesis is that perception is preceded by a process that extracts lowdimensional representations of the sensory information. The so-called SwissRoll gives a clear illustration of the situation. Given data points in the ”high”-dimensional space R3 that really lie on a rolled plane, metrical reduction al-gorithms will search for a mapping from R3 to R2 that does not alter geodesicdistances between points.Fig. 3.1. Unfolding the Swiss Roll. A, the manifold on which the data lies; B, thedata points; C, the resulting embedding. From [22].However, as will be shown in the following overview, current research onmethods of dimension reduction has mainly focused on technical aspects of di-mensionality reduction itself without questioning the metric used on the datato be reduced. It is usually assumed that the format of data representationshould itself suggest a metric. But according to the sensorimotor approach,we expect that actions, rather than the peculiarities of particular data rep-resentations, are what should essentially determine the metric to be used onsensory data.3.3.1 Review of classical, “passive” approachesVariance based methodsPrincipal Component Analysis (PCA) is a ubiquitous method when data lie ina vector space, and it exists in several flavors: variance analysis and reduced3 Dimensionality Reduction 101rank analysis [23], Probabilistic PCA [24], incremental PCA [25], recursivePCA [26], etc. The basic idea is to provide the d-dimensional linear approxi-mation of a dataset yielding minimal error with respect to the n-dimensionaloriginal data (n ≥ d). Approximation is obtained by retaining from the dataonly their orthogonal projection on a d-dimensional subspace accounting foras much as possible of data variance. Actually, PCA provides an ordered setof n approximations of dimensions d = 1 . . . n having this property. It can beperformed algebraically or using neural networks. Note that both the notionof minimal error and orthogonal projection follow from the specification of anorm on the data.Several generalizations of PCA have been developed. Projection Pursuit(PP) searches for axes in input space that optimize a user defined index de-scribing the ”interestingness” of the axis [27, 28]. If the index of an axis is thevariance of data along this axis then PP equates with PCA, but other indexessuch as data kurtosis or multimodality can also be considered. Generalizationof the Gaussian assumption underlying PCA to the more general assumptionof an exponential family of probabilities has also been investigated [29].Practical attempts towards a nonlinear version of PCA are found in LocalPCA [30] or mixture of principal component analysers [31], which indepen-dently analyses cells of a partition of the input space, and Locally WeightedPCA (LWPCA), which interpolates with radial functions between different lo-cal models [32]. Kernel PCA [33] provides a nonlinear version of PCA throughan implicit mapping of data in a higher dimensional space on which PCA isperformed, computations being optimized by the fact that only scalar productsof mapped data are actually required. Principal Curves (PC) and PrincipalSurfaces provide a more geometrical viewpoint on a nonlinear version of PCA:instead of searching for subspaces fitting the data, PC searches for surfacesfitting the data [34]. By analogy with the property satisfied by principal sub-spaces in the case of data drawn from a Gaussian distribution, each point of aprincipal surface is expected to be the mean of the data whose projection onthe surface is this point. This Riemannian approach is quite general in extent,and there are developments for the more constrained (hence richer) case ofLie groups [35].Distance matching methodsThe methods of MultiDimensional Scaling (MDS) originally developed forpsychophysics [36] can also be used for dimension reduction, by retainingfrom the data only their distances, and then searching from scratch for lowdimensional representations satisfying the same set of distances. The simplestapproach uses a least square citerion and is purely algebraic, but there arenumerous variants. Notably, nonmetric MDS aims at replicating the rankorders for distances instead of the actual distances [37, 38], and is based onminimization of a complex cost function for the mismatch of distances instead102 Philipona and O’Reganof using the least square criterion. Sammon’s NonLinear Mapping (NLM)follows the same philosophy as nonlinear MDS but uses yet another criterionfor the mismatch of distances [39].Laplacian eigenmaps can be seen as a form of distance matching [40]. Theidea is to find real functions of unit norm over the data manifold that pre-serve locality as much as possible (i.e. close points are mapped onto close realvalues). An embedding can then be found from the fact that an orthonormalbasis of such functions is the solution of a general eigenvalue problem, withthe size of the eigenvalues quantifiying the goodness of locality preservationfor the associated eigenfunctions.Vector quantization methodsSelf-Organizing Maps (SOM) represent a somewhat different trend of research,based on the idea of Vector Quantization (VQ) with topological constraints[41]: SOM provides a feature map from a specified set of representatives (called‘neurons” in the SOM nomenclature or “codebooks” in the VQ nomenclature)in a topological space to a set of prototypes in input space, so as to yieldminimal quantization error in input space. Curvilinear Component Analysis(CCA) circumvents the requirement of a predefined neighborhood relation be-tween representatives by adapting both the set of prototypes, using standardVQ methods, and their representatives, by a gradient descent minimizing themismatch between the set of local distances for input data and the set of lo-cal distances for representatives [42]. Neural Gas (NG) and Growing NeuralGas (NNG) provide efficient VQ methods able to adapt to data with differentdimensionalities at different locations [43, 44]. Approaches to dimension re-duction based on Shannon information theory [45] should also be mentionedas they are closely related with VQ.Geometric methodsIn recent years, two methods of dimension reduction for data lying on a non-linear manifold have been very popular: Isomap [22] and Locally Linear Em-bedding (LLE) [46]. Isomap first computes geodesic distances between pointswithin the input manifold using shortest path algorithms, and then appliesMDS to represent the data in a space such that Euclidean distances corre-spond to the computed geodesic distances. Curvilinear Dimension Analysis(CDA) is very similar in spirit, its main difference being to use CCA insteadof MDS [47]. As for LLE, it is based on the preservation of local linear rela-tionships instead of the preservation of local distance relationships: LLE firstexpresses each point as a linear combination of its neighbors in input spaceand then computes a low dimensional representation satisfying identical linearrelationships. A fundamental difference between Isomap and LLE is that thefirst tries to preserve global geodesic distances over the whole set of data while3 Dimensionality Reduction 103the second attempts to preserve local linear relationships between neighboringpoints: the first is global, the second local.3.3.2 Standard issuesAlgebraic versus gradient based/neural network methodsApproaches providing algebraic solutions such as PCA, classical MDS, Isomapand LLE profit from a clear analytical framework for studying the robustnessof the algorithms and from highly optimized routines developed on computersfor linear algebra. On the other hand approaches with gradient based solutionsare more biologically appealing and suggest the existence of very efficientimplementations in physical dynamical systems, that are presumably betterbehaved than algorithmic processing. It is therefore difficult to judge theseapproaches on the basis of this algebraic versus gradient based distinction,even though this point is often discussed in the arguments supporting eachapproach.On the other hand, what may have an impact is the possible existence ofstrong discontinuities in the behavior of an algorithm with respect to its dataand parameters. The behavior of Isomap, for instance, has been argued to bestrongly dependent on the size of the neighborhood that is considered to es-timate geodesic distances [48]: a neighborhood which is too large might causecollapse of nearby parts of the manifold on which the data lie and a neigh-borhood which is too small might lead to conjecture a myriad of unconnectedcomponents. This is a very unfortunate behavior, but it seems unavoidable ina numeric context which spoils the notion of continuity on which the geometricintuition of such algorithms is grounded.Linear versus nonlinear approachesPCA definitely possesses numerous advantages: it is easy to implement, ro-bust (in the sense that it will not often yield absurd results), its memory andcomputational requirements scale reasonably with respect to the dimension-ality of the data, and for about the same cost as a unique approximation itactually provides a series of approximations with increasing degree of accu-racy. It must however be recalled that principal components are defined onlyon the basis of the variance of the data along a family of orthonormal axis.This means that the extracted components will not necessarily have an intu-itive meaning. Indeed PP demonstrates that other criteria can equally well beconsidered, and may be more relevant from a statistical point of view. Thelinear assumption underlying PCA will also lead to an overestimation of thenumber of coordinates needed to approximate the data if the input dataset liesin a nonlinear manifold. Further, distances between approximations may besubstantially different from distances in input space if the latter are geodesicdistances. These two points are the main reasons that drove research fromlinear approaches to nonlinear ones.104 Philipona and O’ReganTopological and geometrical constraints in the nonlinear caseNonlinear approaches suffer from more subtle difficulties than might be imag-ined from the local notion of a smooth manifold: in particular, there may bea global topological incompatibility between the d-dimensional submanifoldconsidered for reduction and the Euclidean space Rd in which a low dimen-sional representation is sought. Consider a circle in R2: any global attempt torepresent it in the Euclidean space R having its intrinsic dimension will fail topreserve such an elementary feature as neighbor relations between its points.There is research aiming to specifically address this issue either by cutting thedata manifold so as to achieve an embedding in a Euclidean space having theintrinsic dimension d of the manifold [49], or by using Whitney’s theorem topreserve the global topological structure of the data manifold but achievingan embedding in a Euclidean space of dimension 2d+ 1 [50].As for metrical aspects, it may sometimes be the case that there is anincompatibility between the intrinsic curvature of the data manifold and theflatness of the representational space: consider the case of data lying on afishbowl in R3. Such data cannot be represented in R2 without introducinga systematic distortion of distances as compared to geodesic distances on thedata. A possible way to overcome this problem in some cases is to relax theassumption of a Riemannian metric on the data manifold, and use a conformalmetric (i.e. undetermined by a scale factor at each point of the manifold) [51].This can however only solve very specific cases.3.3.3 The central issue of the metricAll the approaches reviewed above have a fundamental problem. They assumea particular metric specified in input space, but in general they give no reasonwhy this metric should be prefered over another one. Yet clearly the metricused for dimension reduction is crucial in determining 1) aspects of the datathat can be neglected in the reduction process, 2) the kind of distance rela-tionships that have to be preserved in representational space, and 3) the kindof coordinates yielded by dimension reduction.Metrics based on noiseDimensionality reduction presumably takes from its oldest solution, PCA, theconceptual framework that lets the Euclidean norm be considered as a naturaldefault choice for a metric on data. Consider the very simple example of two-dimensional data obtained from one-dimensional data by a noisy process (thestandard situation underlying PCA), so thatY = u ·X + εwhere u ∈ R2 is some vector, X ∈ R are the ”true” data and ε is fromwhite normal noise. The choice of norm ||y||2 = yT y = y21 + y22 will equate3 Dimensionality Reduction 105statistical concerns with a geometrical picture: the estimation for X thatresults from maximum likelihood can be read as the orthogonal projectionof data on the line yielding minimum distance expectation between data andtheir projections. This expectation equates in turn the geometrical picturewith PCA since it can be shown that minimization of distance expectationis achieved by using the line yielding maximum projected variance. Here itmust be emphasized that “orthogonal projection” and “distance” both dependon the norm just defined. Hence if one assumes that data are corrupted byindependent and identical noise on each coordinate, the Euclidean norm is anatural criterion to minimize as regards approximation concerns.Note of course that this is already no longer the case if the noise sourcesare not identical and independant. In order for the minimization of distanceexpectation to yield the maximum likelihood estimate for X, a coordinatesystem whitening the noise has to be considered: put Ỹ = Cov(ε)−1/2Y , thenỸ = v ·X + ε̃with ε̃ white noise takes us back to the previous situation. But if ||ỹ||2 = ỹT ỹ,then we have ||y||2 = yTCov(ε)y which shows that the noise covariance indeeddefines the norm that should be used as a minimization criterion if we wantto perform a maximum likelihood estimation of “true” data.This kind of approach may therefore provide arguments for use of a specificmetric, but the point is: methods such as Isomap are not really concerned bythe issue of the noise. In fact, their simplest examples involve data withoutany noise. Thus the question is: what is that metric whose preservation is themain virtue of a ”good” dimension reduction algorithm? This is discussed forinstance in [52] from the perspective of standard tasks to be fulfilled in imageprocessing. We propose here a somewhat different viewpoint.Metrics based on invarianceSince images are a classical example where dimension reduction is often re-quired, we could search as a starting point for arguments leading to considerspecific metrics on image manifolds. For that purpose, we will distinguish be-tween the continuous field amplitude I(x, y) that represents the informationwith which sensors interact, and Iij the pixel array delivered by the sensorsof a camera. The idea is that the pixel representation should be consideredmerely as a choice of coordinates for the physical entity I(x, y) facing the sen-sors. This coordinate system is obviously not neutral on general grounds: first,it will determine the kind of information loss induced by noise on these repre-sentations, and second, if computations performed by modules receiving suchrepresentations are limited then this representation will imply specific possi-bilities and impossibilities as regards what can be performed by such modules.While these two points are very important they are not usually mentioned ex-plicitly in the framework of dimension reduction. From the viewpoint of this106 Philipona and O’Reganframework, coordinate systems should be neutral, and the choice of a met-ric has no reason to be based on it. This means that we must search withinexternal constraints for ideas about what metric to use.We might first try to find whether there is a natural metric on amplitudefields I(x, y). As for infinitely extended fields, a presumably important re-quirement is that this metric should yield a notion of distance between fieldsindependent of a frame of reference in the 2D plane, otherwise we would needto take into account some ”absolute” position and orientation of our ownviewfield to estimate the distance between two fields. Note p = (x, y) forconvenience, then I observed in a frame of reference with change of positiont ∈ R2 and change of orientation r ∈ SO(1) is the field (t, r)∗I : p 7→ I(r·p+t),and therefore we wantd(I, J) = d((t, r) ∗ I, (t, r) ∗ J) ∀t ∈ R2, r ∈ SO(1) (3.1)If d is a norm on square integrable functions, then it derives from a scalarproduct of the form〈I, J〉 =∫k(p, q)I(p)J(q)dpdqand we can easily show the constraint 3.1 to be equivalent tok(p, q) = k̃(||p− q||)where ||p|| is the Euclidean norm on the 2D plane. Hence the norm||I||2 =∫I(p)2dpis a simple case of such invariant norms obtained by using a delta function asa kernel: k̃(v) = δ(v − 0). This suggests that this norm on pixel arrays yieldsan intuitive notion of approximation (for instance when used to perform aPCA) because cameras have been built so that this norm satisfies invarianceproperties that may underlie our own perceptual analysis.The previous simple formalization is instructive on three points:• First, it shows that the standard norm used on pixel arrays has the verydesirable property of being approximately invariant to changes of view-point on fields. This is however so only because of the regularity of thesensor array, and because all sensors are assumed identical and linear withrespect to fields1 2.1 This shows that the physical architecture of a device is part of its computationalabilities.2 As a consequence of this invariance we have the following result. Take T = (dt, dr)a tangent vector to the identity in SE(2), and put I(s) = exp(Ts)∗I a trajectoryin the space of pictures obtained by the action of the exponential of T : then||İ||2 = const.3 Dimensionality Reduction 107• Second it suggests a whole class of metrics on fields that are invariant andtherefore could be considered on pixel arrays. It is easy to construct awhole family of invariant norms once we have one. This can be done usinglinear transformations A such that A((t, r)∗I) = (t, r)∗A(I). Convolutionswith kernels of the form h(||p − q||) satisfy this, hence isotropic blurringor banks of linear invariant filters for instance obviously do not destroyinvariance.• And third, it suggests a generic criterion, invariance, to deal with the casewhere sensors no longer satisfy favorable constraints of regular layout, sim-ilarity and linearity of sensors. In the case of humans, it must be recalledfor instance that photoreceptors are organized on the retina in very inho-mogenous way, and furthermore that blood vessels in the eye obstruct alarge number of photoreceptors. In the case of an artificial agent, the useof an inhomogeneous optical filter is equivalent to a distortion of sensorlayout and sensor responsiveness. If such a filter is added to the camera,then the pixel norm and invariant norm on physical fields will not matchanymore. The invariance property of the pixel norm will break down andthe distance between pairs of pictures will for instance vary when bothpictures are translated by the same amount. We may wonder why opticalfilters should be added to the camera: the point is merely to emphasze thatimportant assumptions about the sensory device are made when consider-ing the Euclidean norm on arrays of pixels. The role of these assumptionshas to be kept in mind when conceiving an unsupervised approach to sen-sory processing.The goal of section 3.4.1 will be to provide a more general formalizationfor the notion of invariance, so as to provide the definition in the most releventway for a given sensorimotor system.3.4 Dimension reduction in the context of sensorimotorinteractionsHaving considered a general overview of known dimension reduction methodsand acknowledged the need for them to provide arguments for the choice of ametric, we try in this section to put forward sensorimotor arguments. We willfirst focus on the construction of a mathematical conception for what can bemeant by a sensorimotor system. We need a picture that is constrained enoughso that we avoid getting lost in wide-ranging considerations about dynamicalsystems, and yet a picture that is general enough so that we leave substantialroom for applicability and unsupervised learning: we want to carefully weighconstraints to be added and put them in a general form so that they can beinstantiated by an agent involved in sensorimotor interactions that may notresemble our own.108 Philipona and O’ReganFor the reasons we have already discussed, we deliberately make the choiceof considering a continuous and dynamical picture of a sensorimotor system.Further we make the choice of a geometrical picture, that is: we want to thinkin a coordinate-free way about the mathematical objects considered in themodel. This means in particular that we do not want models that dependon a specific encoding of sensory data and motor commands. This is thecondition to find device independent algorithms, but it is also motivated byconsiderations at the foundations of the sensorimotor approach to perceptiondeveloped in [1]: namely the assumption of a functional, rather than dualistor reductive physicalist explanation for perceptual experience.3.4.1 Finding a mathematical frameworkTo ground the intuition of this framework we will often go back to a standardexample of robotic system interacting with space: that of a two wheeled vehicleon a 2D plane (Figure 3.2).Fig. 3.2. Two wheeled vehicle on a 2D plane with various variables defining itsphysical stateModel for sensorimotor dynamicsA quite general model for sensorimotor interactions of an embodied agentcould be  ω̇ = F (ω,m)s = G(ω)m = H(s, a)(3.2)where ω are physical states of the agent-and-environment, m are motor com-mands, s are sensory inputs, and a are action commands imposed by higher3 Dimensionality Reduction 109level cognitive functions. F describes the possible dynamics of the physicalstate under different commands, G essentially describes perceptual aliasing(the fact that a given sensory input most often can result from a whole set ofphysical states), and H describes feedback control laws associated with differ-ent actions a. This is for instance a model equivalent to that of [53], apart forone important point: actions a will be here considered a priori to constitutea continuous set rather than a finite set.Our motivation to consider a continuous set of actions is that we want toinvestigate sensorimotor arguments for abstracting a finite number of actionsso that the global sensorimotor interactions of the organism driven by cogni-tive functions still exploit the whole range of low level abilities3. This actuallymeans that we will overview some arguments for turning the previous systeminto a hybrid system [54] without losing too many of the possibilities offeredby the underlying continuous dynamics. As we will see, the issue of isolatinga finite number of actions will be related, in the algorithm that we suggest,to the question of finding a metric on sensory input. But more essentially theissue of the interface between a continuous dynamical system and cognitivefunctions (e.g. algorithms for Turing machines) is at the heart of our generalapproach in the scope of this work.Notation: We will refer to the first equation of the model (3.2) as thesystem dynamic or external dynamic. We will refer to the dynamic involvingfeedback control as the low level sensorimotor dynamic, that is:ω̇ = F (ω,H(G(ω, a))) = ϕ(ω, a) (3.3)The term low level is in contrast to the global dynamic which also involvesslower cognitive feedback. This gives the picture of a fast dynamic (the lowlevel sensorimotor dynamic) driven by a slow one (the cognitive dynamic), asalso suggested in synergetics [55] and hybrid systems [54].Overview of structural aspectsModel (3.3) is typically far too general to provide much insight about thepossible structure of low level sensorimotor interactions. Constraints to con-sider in order to obtain a mathematically tractable framework are suggestedby viewpoints that have arisen in the mathematical study of dynamical sys-tems. We evoke these viewpoints as they can provide hints to ground selectionof actions by specifying ‘structures” that we may want to see preserved bycognitive processing.3 For instance, it seems intuitive that most of the time neighboring actions willlead to similar sensorimotor loops and therefore only one representative actionfor the whole set needs to be retained by cognitive functions. But there will alsobe some actions leading to very different sensorimotor loops, or bearing specificrelationships with other loops, and these will need to be differentiated by cognitivefunctions if the overall structure of the sensorimotor dynamics is to be preserved.110 Philipona and O’ReganCatastrophe theory.An example of a possible and natural constraint to consider on the model, ifit is to refer to a physical system, is the existence of a potential from whichthe dynamic derives. If the sensorimotor dynamic depends on a smooth po-tential then catastrophe theory details conditions under which slight changesof a given action will result in a very different organization of the associatedlow level sensorimotor dynamic: changes in the number of fixed points, thestructure of stable, unstable and repelling sets, structure of cycles, etc. Thiscould be used to partition the action space into areas associated with qualita-tively similar dynamics, and identify low level sensorimotor dynamics that aresingular in the sense that they require a very precise tuning of the parame-ters to occur. This would be the viewpoint of structural stability [56]: identifyand represent dynamics that are robust to slight perturbations in the choiceof action. While such general aspects of dynamical systems must be kept inmind, they do not seem to express sufficiently strong constraints to accountfor the kind of physical regularities that we need to ground a notion of spaceand objects.Geometrical nonlinear systems.Another possible constraint is the one considered in standard approaches tononlinear control theory [57], namely: a vector space structure for the set ofall possible actions, and external dynamics affine with respect to actions i:ω̇ = ϕ0(ω) +∑iϕi(ω) · ai (3.4)This model can be applied to many robotic systems, and is further compatiblewith physiological work about the linear combination of motor primitives inhumans [58]. The affine assumption is a very natural one from a geometricalperspective, as will be sketched later, and it will be retained in the followingapproach.Example. To fix ideas, we can slightly detail the case of an idealvehicle on the 2D plane. Putω = (x, y, θ, ψ1, ψ̇1, ψ2, ψ̇2)where these variables describe, respectively, the position of the middleof the two wheels’ contact points, the vehicle’s heading direction, andthe angles of a fixed point on each wheel with respect to the vertical.Then standard mechanical principles show that(ẋ, ẏ, θ̇) = f0(θ, ψ̇1, ψ̇2)(ψ̈1, ψ̈2) = J1 · τ1 + J2 · τ23 Dimensionality Reduction 111where control variables τ1 and τ2 are the torques applied to drive thewheels. We then see that the dynamic is indeed linear in the motorcommands.In the context of nonlinear control theory, reachability is the issue of de-termining which configurations of the system can be reached from a givenstarting configuration. This is an issue well understood in the geometricalpicture of equations (3.4), and it naturally suggests that an important andtractable requirement to be considered on the choice of a finite set of actions isthat this choice should not reduce reachability, at least in the limit of infinitesequences of such actions. Here we will consider that issues about observabil-ity and stability are about available low level sensorimotor abilities, and sowe will not discuss them.Mechanical and geometrical symmetries.A last important constraint that may be considered is the case where thephysical world possesses a very strong regularity, namely the existence of asymmetry group for system dynamics. This is the case we detail in the nextsection.The existence of a symmetry group can be interpreted as the situationwhere an underlying space imposes that system dynamics are invariant whenphysical states are merely moved in this space. Consider lifting up the vehiclefrom the 2D plane then putting it at another place: There is no change in thedynamic ruling the system, there is no way to infer from the dynamic wherethe vehicle is.If we take as our goal to investigate algorithms allowing a system to locateitself in space, we will need to define a constraint that will let us consider mod-els of sensorimotor interactions in which there is indeed a space to find. Theexistence of a symmetry group will allow us to introduce on general groundsnotions such as displacement, and relative configuration. This provides furtherideas to select specific actions: we can expect some actions to bear very spe-cific relationships with all other actions, and these specific relationships couldprovide a simplification for the set of policies to be considered4. In any casethe presence of symmetries is known to simplify the complexity of optimalplanning in the paradigm of machine learning [59].In this chapter the symmetry group will suggest constraints on metricsto be used on sensory inputs: we wish to search for metrics respecting thissymmetry.4 Consider the case of commutative actions a and b. The set of possible actionsequences to consider for planning reduce to {anbm, n ∈ N, m ∈ N} ∼ N2 while ifactions do not commute then the set of possible action sequences to consider isN{a,b} ∼ R.112 Philipona and O’ReganSpace: system symmetries and sensorimotor symmetriesSymmetries.In order to formulate the assumption of the existence of a symmetry group,we make a slight shift of viewpoint towards a geometric framework: we rewritethe first equation of model (3.2) asω̇ = Fm(ω)and we view Fm as a family of vector fields on the state manifold Ω parameter-ized by motor outputs m. We can see here that the assumption of a dynamicwhich is affine in the motor commands is natural, since the family Fm can beseen as a submanifold of the vector space of vector fields: the affine assumptioncan therefore be seen either as a local approximation or as a general frameinside which smooth constraints will be considered later.The existence of a symmetry groupΣ onΩ for the dynamics can be writtenT ∗Fm = FmHere T ∗ denotes the pull-back application for the diffeomorphism T of Ω (seefor instance [60]).We already have cited the common example of a vehicle moving on a 2Dplane. Other examples could include a vehicle moving on 2D manifolds ofconstant curvature5. To fit with the physics of our world, we will assume theexistence of a smooth symmetry group, i.e. a Lie group.Relative configurations.Under suitable assumptions of regularity for the action of symmetries6, thesymmetry group induces a partitioning of the state manifold Ω into equiva-lence classes defining a manifold of relative configurations7 Ω/Σ. This sug-gests a quite general aspect of an agent body in model (3.2), expressed in5 The case of manifolds of non-constant curvature intuitively calls for a distinctionbetween a sensorimotor agent in a 3D symmetrical world constrained to move for awhile on an arbitrary manifold (physical symmetries still exist, but are symmetriesof the 3D world), and an agent in some sense living ”inside” this manifold (thereare no symmetries there as the agent could distinguish some places on the basisof a different relation between sequences of movement controlled by identicalsequences of commands).6 The action must be properly continuous.7 We prefer to use this term rather than the one more classically used in this contextof shape, because in our framework relative configuration could a priori includemore than shape in the classical sense: for instance changes in the sensitivity of thesensors. Furthermore, if there are objects in the environment their configurationrelative to the agent enters into Ω/Σ3 Dimensionality Reduction 113an abstract framework: the agent’s own configuration resides in what remainsamong physical states when symmetries of the system dynamics have beentaken into account.Example. In the case of the two wheeled vehicle, physical statesequivalent up to Euclidean symmetries of the 2D space are stateswith equal wheel configurations, hence relative configurations can bewritten c = (ψ1, ψ̇1, ψ2, ψ̇2).If further there is no physical state left invariant by symmetries8 (anyEuclidean displacement of the vehicle can be identified straightforwardly byreading out changes in ω), then the geometrical picture of model (3.2) is thatof a principal bundleπ : Ω → Ω/Σwith group structure Σ.Locomotion.This was for the constraints on the external dynamic. As for the sensorimotormodel, we will take into account the control law of the agent in order to defineits displacement abilities. Here also we operate a slight shift toward geometryby writing sensorimotor dynamics asω̇ = ϕa(ω)and view them as a family of vector fields on the state manifold parameterizedby actions.Sensorimotor dynamics such that Tωπϕa(ω) = 0, ∀ω (i.e. vertical vectorfields) define sliding locomotion: this means that such dynamics correspondto a displacement of the agent that occurs without any change of relativeconfiguration. This is hardly expected to happen in our physical world9, butis actually often considered as an approximation when abstracting details ofa sensorimotor system.Example. If we forget about the wheel configuration of the vehicleon the 2D plane, then the relative configuration manifold is reducedto a point (no change other than position and orientation is left in themodel) and locomotion will therefore necessarily be sliding locomo-tion. Indeed, assuming high gain feedback controllers that let wheelrotation velocity quickly converge to specified velocities a = (v1, v2),we can write8 In other words: if symmetries act freely.9 We mean here: controlled sliding locomotion is hardly expected to happen, sinceany system left in outer space with an initial nonzero momentum will suffer slidinglocomotion.114 Philipona and O’Regan(ẋ, ẏ, θ̇) ≈ ρ2l(l(v1 + v2) cos θ, l(v1 + v2) sin θ, v1 − v2) (3.5)which provides a sensorimotor model with sliding locomotion. Notethe requirement to involve control schemes to obtain such a model: onphysical grounds the configuration of the wheels cannot be abstractedfrom the model.On the other hand, some sensorimotor dynamics will correspond to loco-motion associated with changes of relative configuration. This suggests focus-ing on the link between changes in relative configuration and locomotion. Itturns out that both geometrical constraints of the form ”no slipping contact”and mechanical constraints (that is, stemming from a Lagrangian approach),can be expressed as a connection on the principal bundle π : Ω → Ω/Σ[61, 62]. This has been shown to be valid for various artificial systems on arigid ground [63] or in an ideal fluid [64], as well as for swimming microscopicorganisms when viscous forces dominate inertial forces [65]. The fact thatthe physics can be written as a connection implies that only the trajectoryof relative configurations affect the agent’s displacement, not the velocity atwhich the trajectory is traversed. This is a strong constraint on the externaldynamics of a sensorimotor system. It may actually be valid only piecewise,for instance when considering manipulation [66] or legged locomotion [67]. Atthe level of the external dynamics this connection allows a holonomy groupto be defined, that is, the group of displacements for that dynamic10. It alsoallows a notion of curvature to be defined on the principal bundle π that canthen be used to highlight a distinction between stationary changes of relativeconfiguration (whose sensorimotor dynamics are within flat submanifolds ofΩ and correspond to changes of relative configuration that do not interferewith locomotion) and other changes that can be called locomotive changes ofrelative configuration.3.4.2 Back to dimensionality reductionConstraints on a spatial metric for sensory dataAn invariant metric g on the state space Ω is a metric that is invariant underthe action of symmetries:T ∗g = gIt must therefore satisfy, using the symmetry of the metric :g|ω(Fm(ω), Fn(ω)) = T ∗g|ω(Fm(ω), Fn(ω))= g|T (ω)(DpTFm(ω), DpTFn(ω))10 Note that this group need not be a Lie group: in particular it might be a discretesubgroup of the symmetry group (even though the whole framework is continu-ous). We will however assume a Lie group.3 Dimensionality Reduction 115and using the symmetry of system dynamics:g|ω(Fm(ω), Fn(ω)) = g|T (ω)(Fm(T (ω)), Fn(T (ω))) (3.6)In particular, we find the intuitive result:Land survey lemma: The norm of tangent state change ω̇ associatedwith a given motor command m is the same for all states associatedwith the same relative configuration.In the case of system dynamics that are linear in motor commands wehaveg(Fm, Fm) =∑i,jmi ·mj · g(Fmi , Fmj )Therefore if for each state ω the system dynamics span the tangent space TωΩthen the choice of an invariant metric reduces to the choice for each relativeconfiguration of a scalar product on the finite dimensional vector space ofsystem dynamics. Note that there are a priori several invariant metrics, stillleaving room for the consideration of other constraints such as biinvarianceor energetic costs of the different dynamics.No local aliasing.Now of course the point is that physical states are not accessible to the agent,and we would like to actually define a metric on sensory inputs. If there isno local perceptual aliasing then we can still apply the land survey lemma:the norm of tangent sensory input change ṡ associated with a given motorcommand m must be the same for all sensory inputs associated with the samerelative configuration.This will be the case considered in section 3.5.Local aliasing: a suggestion.In the more difficult case where there is local aliasing, then the previousviewpoint could be used to search for a metric on sensory inputs that satisfythe land survey lemma “as much as possible”. We will not follow this linehere, but we present the suggestion anyway: gradient descent could be usedto minimize over a parameterized set of metrics gθ the invariance criterion:ε2(θ) =∑i,j,k[gθ(Ṡk,i, Ṡk,i)− gθ(Ṡk,j , Ṡk,j)]2where for each k, Sk,i(t), i = 1 . . . n is a set of sensory evolutions resultingfrom motor command mk such that the relative configuration of the agentat time t = 0 is the same for all evolutions. The minimizer will be a metricsuch that, in expectation, the norm of tangent sensory inputs associated withidentical spatial displacements is the same.116 Philipona and O’ReganNote that the use of the metric obtained could be interpreted as Bayesianinference, since the metric learned could then be read as agent beliefs onphysical states associated with the sensory inputs used to infer the distancebetween the those physical states.Nonlinear sensory manifoldsSubmanifold of Euclidean spaces.Nonlinear approaches to dimensionality reduction usually define a metric ona data manifold by considering the metric induced on a submanifold of somespace Rd by a norm on that space. Yet it is not clear on general groundswhy we should consider the sensory manifold as a submanifold of some vectorspace. Of course, this makes sense from the computational viewpoint sinceenforcing such a hypothesis allows use of linear algebra and limits the num-ber of parameters to estimate, therefore leading to simple algorithms withfast generalization. But it imposes a constraint that may reduce the possiblenumber of distance-preserving transformations compatible with such a model.Invariance to spatial displacements suggests for instance considering a met-ric that is preserved when pictures are zoomed, so that the distance betweentwo object pictures does not depend on the depth from which the picturesare taken11. Yet when taken together with invariance to 2D translations androtations, this cannot be enforced using a norm. This therefore suggests inves-tigating more general models for the metric than those deriving from a normin a Euclidean embedding space. This will be the framework of the algorithmpresented in section 3.5.Unfolding and beyond.As already discussed, the standard framework of nonlinear dimension reduc-tion focuses on the unfolding of data manifolds and therefore essentially dealswith flat manifolds such as the swiss roll. The issue that data manifolds canpossess intrinsic curvature (such as a sphere that cannot be flattened outwithout introducing large distortions like those observed on earth maps) isactually often discarded as a second order problem because the metric withrespect to which this curvature has been defined anyway did not rely on strongjustifications. In our case however, since we are searching for metrics groundedon sensorimotor interactions, we expect curvature to reflect important differ-ences between sensory data variations with respect to these interactions andwe do not want to discard this information.The goal of unfolding the data however usually stems from the use ofdimension reduction algorithms to visualize high dimensional data in a 2D11 That is, the same change of depth for two pictures should not change the distancebetween them. Independent changes of depths for the two pictures may (will) stillaffect the distance.3 Dimensionality Reduction 117vector space. But in the case of an embodied agent this is not a naturalrequirement and we only want to exploit the geometrical structure as it is,therefore leaving room for a more open framework. For instance, we may wantto compute distances between sensory inputs to know whether two sensoryconfigurations are more or less similar so as to perform quantization on thisbasis. Or, we may want to know what actions can link two neighboring sen-sory inputs so that we can actively try to perform alignment in a navigationcontext. Or we may want to identify points of highest or lowest curvature ina given sensory trajectory, so as to consider a possible segmentation of sen-sorimotor interactions. All these tasks can be performed without having toflatten out the data.3.5 An embodied algorithm for dimension reductionHere we finally present a general attempt to build an invariant metric forsensory inputs delivered by an unknown sensorimotor device (satisfying acertain linearity constraint described in 3.5.1). To ground the intuition andexplain simulations run in section 3.5.2, here is the example we have in mind.Consider an agent exploring a 2D world that is an infinitely extended 2Dpicture. Assume that the agent is able to move above the picture, driftingahead, sideways, and turning to different extents so that its action space iscontinuous. Assume that the agent’s sensory inputs are constituted by a small,say 10×10 pixel, snapshot taken of the picture below the agent. We would likethe agent to build a metric on its sensory inputs that is invariant with respectto symmetries of its sensorimotor interactions in this world. We would alsolike the agent to discover that there are some actions (namely translations)that bear a very special relationship with other possible types of actions.We will make the assumption that there is no relative configuration change(see section 3.4.1) and that there is no local perceptual ambiguity (i.e. per-ceptual aliasing). As for ambiguities, we will then show with examples thatthe algorithm does not suffer significantly from them.3.5.1 Description of the algorithmInvariant sensorimotor metricsTransformation based metrics.With the example of our 2D agent in mind, we note that each displacementof the agent corresponds to modifications of sensory inputs that are approx-imately12 linear in sensory inputs. The reason is that sensory inputs result12 only approximately, as modifications at the edges of view field cannot be predictedfrom what is in the view field.118 Philipona and O’Reganfrom sampling of a field I on space and that the viewpoint transformation(t, r) ∗ I : p 7→ I(r · p + t) (see section 3.3.3) is linear in I. Therefore if eachsensor output is linear in the field (but without requiring that all sensors areidentical nor that they have a uniform layout), then we can expect that eachmotor command resulting in a displacement (t, r) is associated with a globallinear transformation P (m) of sensory inputs:ṡ = P (m) · s (3.7)In this case, the Land survey lemma suggests the invariance criterion:g|s(P (m) · s, P (m′) · s) = constfor all points s of the sensory submanifold and all motor commands m andm′.We recall that a simpler way to describe metrics than providing an explicitformula is by specifying an orthonormal basis {Xi(s)} at each point of thesensory manifold. Evaluation of the metric is then performed by computingtangent vector coordinates in this basis:ṡ =∑ixiXi(s) ⇒ ||ṡ||2 =∑ix2i (3.8)As a consequence, if we consider the case where there is a finite numberof displacements triggered by commands mi and that the family of vectorsXi(s) = P (mi) ·s forms a basis of the tangent space of the sensory manifold ateach s, then we can use transformations P (mi) to define a norm by positingthat the family of Xi(s) should be an orthonormal basis. Then obviously wewill have g|s(P (mi) · s, P (mj) · s) = δij which is indeed independent of s.The specific example of the 2D agent in a picture world therefore suggeststhat invariant metrics can be built from linear predictors of sensory changesinduced by actions. The whole problem then is: first choose appropriate ac-tions, i.e. appropriate predictors, and second compute geodesic distance. Thefirst point will be explained in section 3.5.1, here we present the approximationscheme that will be used for computing geodesics.Approximation scheme for local distances.If the motor command m is held fixed for a duration ∆t, then equation 3.7can be integrated using the matrix exponential:s2 = exp (P (m)∆t) s1 ⇒ s2 − s1 = [exp (P (m)∆t)− Id] s1Therefore the approximation scheme in time for linear predictors P̄ (m) ofsensory changes resulting from holding motor command m fixed for a time ∆treads:3 Dimensionality Reduction 119s2 − s1 = P̄ (m)s1, P̄ (m) ≈ P (m)∆tHerein the notation P will refer to the differential transformation of sensoryinputs expressed in equation 3.7, while the notation P̄ will refer to the numer-ical estimation of this transformation (i.e. to a predictor) based on sensorychanges between a time t and a time t+∆t.Given a finite family of predictors P̄ (mi), a linear approximation for theevaluation of distances between neighboring points (i.e. equation 3.8) is con-sequently given bys2 − s1 =∑ixi · P̄ (mi)s1 + ε⇒ d(s2, s1)2 =∑ix2iThis approximation is depicted in Figure 3.3.a.To provide a more robust estimation for the case of strong nonlinearities,we however consider in the algorithm a slight improvement of this scheme.This improvement requires to have, in addition to the family of actions mi,a family of actions ni such that sequences of commands (mi, ni) yield nochange of sensory inputs. The way to learn such a family will be explainedin the section on action choice, here we give the expression for the improvedapproximation scheme depicted in Figure 3.3.b:s2 − s1 =∑i xi · P̄ (mi)s1 +∑i yi · P̄ (ni)s1 + εwith xi ≥ 0 , yi ≥ 0⇒ d(s2, s1)2 =∑ix2i +∑iy2i(3.9)Justification for this scheme stems from the fact that if we have a familyof commands ni such that:exp (P (mi)∆t) exp (P (ni)∆t) s = s ∀sthen to first orderId+ P (mi)∆t+ P (ni)∆t = Idand therefore P (mi) = −P (ni), which can be used in equation 3.8 to split∑i xiP (mi) · s into terms with positive and negative coordinates, substituteP (mi) with negative coordinates into −P (ni) with positive coordinates, andfind the scheme 3.9 involving only positive coordinates.Computation of geodesic distances.Using a set of predictors P̄ (mi) and P̄ (ni), the algorithm analyses sets of datapoints {st} with the following procedure. The algorithm considers each point,computes the distance with its k nearest neighbors using the scheme 3.9 (pos-itive coordinates xi and yi are computed using Matlab’s routine lsqnonnegfor linear least squares with nonnegativity constraints), and finally computesshortest path distances between points that are far apart using Floyd’s algo-rithm, in exactly the same way Isomap does. In this way, the algorithm is able120 Philipona and O’ReganFig. 3.3. Approximation scheme for geodesic distance between neighbors. Greenaxis shows the tangent space at s1, and P (m)s1 shows a normal basis for the trans-formation based metric. Assume the length of the segment joining s1 to P̄ (m)s1 hasbeen normalized to 1, then red and blue segments show the coordinate of s2 in thebasis defined by predictor P̄ (m), and in this one dimensional example the length ofthese segments actually is the approximation for the geodesic distance d(s2, s1). a)Approximation of geodesic distances based on the sole predictor P̄ (m) using bothpositive and negative coordinates of s2 on P̄ (m)s1. There is a big distortion in theestimation of distances depending on the sign of the coordinate. b) approximationof geodesic distances based on predictors P̄ (m) and inverse predictors P̄ (n) withpositivity constraint: the previous distortion is removed (see text)to compute a distance between sensory data that is dependent of its actionabilities and not on the sensory code. Basically, the distance d(s1, s2) gives anindication of the time the agent has to hold the appropriate motor commandthat will take it from sensory configuration s1 to sensory configuration s2.If the transformation based metric on the sensory manifold is flat, thenthe matrix Mij = d(si, sj) of geodesic distances can be used with MDS torepresent the data in a vector space having the intrinsic dimensionality of thedata.Choice of actionsUltimately the choice of motor commands to be isolated should be based on ageneral analysis of the holonomy group, such as provided by the analysis of thegroup root system (see for instance [68]). So far however we limit ourselves toa much simpler approach, namely: identify commuting displacements, choosea generating family for this set, then complete this family into a basis for theagent’s displacement abilities. Within these constraints, we will choose actionsassociated with changes of sensory inputs as different as possible with respectto the noise-norm on sensory inputs.3 Dimensionality Reduction 121Identification of commuting displacements.The identification of commuting displacements is done through a piecewisebilinear approximation for the composition of predictors resulting from twosuccessive motor commands, as explained now.In the first stage the system draws randomly a number of motor commandsmi, i = 1 . . . N (N = 30). It must be noted that if the model describedby the equation 3.7 were exact, then we could write the expression for thepredictors associated with successive motor commands from transformationsP (mi) because integrating the equation withm = m1 for∆t and thenm = mjfor another ∆t would yields2 = exp (P (mj)∆t) exp (P (mi)∆t) s1But to be robust to underdetermination of the sensory system, which implythat equation 3.7 only holds approximately, it is better to build an estima-tion of the predictor associated with successive motor commands using directexperimentation of sequences of motor commands. The algorithm thereforeperforms a linear regression over a sufficient number of pairs of {after/before}sensory inputs to learn predictors Q̄ij = Q̄(mi,mj) associated with sensorychanges resulting from each sequence of two moves.If the mapping Q̄ is smooth, then we can consider a piecewise bilinearapproximation Q̂ of Q̄ based on the sample of learnt predictors Q̄ij . Q̂ isdefined by the constraints to have value Q̄ij at grid nodes (mi,mj) and to belinear along each edge of grid cells (see Figure 3.4).In equations, this means that we putQ̂(m,m′) =∑i∈I,j∈JαiβjQ̄ijwhere the αi and βj express the linear relationship between m, m′ and theexperimented commands closest to them {mi, i ∈ I} and {mj , j ∈ J}, i.e.m =∑i∈I αimi and m′ =∑j∈J βjmj . Q̂ can then be used to infer that some(not experimented) motor commands should lead to commuting predictors:this is done by minimizing the criterion∆I,J(α, β) = ||Q̂(m,m′)− Q̂(m′,m)||2 = ||∑i∈I,j∈Jαiβj[Q̄ij − Q̄ji]||2for neigborhoods (I, J) of each motor command sequences (mi,mj) (see Fig-ure 3.4). In this equation, we consider the norm on matrices defined by||A||2 = Tr(ATA) (recall that Q̄ij are predictors, i.e. matrices).Choose a generating family for translations and complete it.The algorithm retains a defined number of linearly independent (α, β) hav-ing the smallest ∆ value (commuting commands, i.e. translations) and largest122 Philipona and O’ReganFig. 3.4. Computing commuting predictors (illustrative sketch). Transparent graysurface is the predictor mapping Q̄, black wireframe is the piecewise bilinear approxi-mation Q̂ from sampling Q̄ on a grid with nodes (mi, mj). Red cell correspond to thechoice of a neigborhood for motor sequences (m, m′), blue cell shows neighborhoodfor the motor sequences preformed in reverse order (m′, m), green arrows shows asequence that is expected from the approximation Q̂ to yield the same predictor inboth orders.∆ value (non-commuting values, i.e. rotations). For now, the number of com-mands retained is given to the algorithm and not estimated. Predictors P̄ (mk)associated with the corresponding motor commands mk are then learnt to beused in the scheme 3.9.Inference of inverse actionsFor an improved approximation scheme for local distances, we also rely on theinference of a family of actions nk such that sequences of motor commands(mk, nk) do not yield any change of sensory inputs. This inference can be donefrom Q̄ by looking for commands nk such that Q̄(mk, nk) = 0, and thereforean approximation can be computed from Q̂ by minimizing for each k thecriterion:∆k(β) = ||Q̂(mk, n)||2 = ||∑i∈I,j∈JαiβjQ̄ij ||23 Dimensionality Reduction 123where mk =∑i∈I αimi and n =∑j∈J βjmj . Predictors P̄ (nk) associatedwith the corresponding motor commands nk are then learnt, to be used in thescheme 3.9.3.5.2 ResultsDimension reduction for a flat sensory manifoldTo test the whole procedure, we consider first the case where the agent in a2D picture world is limited to translational moves (Figure 3.5). The reason todo so is that this should theoretically correspond to a case where the sensorymanifold is flat with respect to the predictor metric, and therefore geodesicdistances computed by the algorithm can be checked by embedding sensorydata in a two dimensional vector space using MDS.Note, as already emphasized several times, that in this system there actu-ally is local aliasing since sensory change resulting for instance from forwardtranslation cannot be predicted without errors from the content of the patch.The algorithm however shows no excessive sensitivity to this underdetermi-nation.Fig. 3.5. The data is constituted with a grid of translated patches, whose coor-dinates in the 2D-world are of the form {x0 + n, y0 + m} with n and m positiveintegers. x0 and y0 are randomly chosen. The camera does not rotate at all. Right,a sample of such patches is shown, each patch being translated slightly with re-spect to its neighbors. Left, the output: the embedding of the data points, whichis two-dimensional since the only existing dimensions are vertical and horizontaltranslations. The curving is a border effect, due to the small quantity of data. Ouralgorithm successfully exhibits an approximatively square grid. Note that the algo-rithm does not necessarily find axes corresponding to orthonormal translations.124 Philipona and O’ReganComparison with IsomapThe previous check done, Figure 3.6 shows the effect of a linear modificationof sensory encoding. This naturally has a large impact on Isomap since thisalteration induces a change in the metric (hence of geodesic distances) thatIsomap aims to preserve in the embedding procedure. By comparison, ourmethod is by design independent of such alterations.Fig. 3.6. Again, the camera producing the data does not rotate, and the grid ofpatches is similar to the one in Fig. 3.1. Left, our algorithm is not affected andoutputs a regular embedding. Right, Isomap, tested with different radial parameters(number of neighbors=4,5,7,8), is affected by the distortion introduced in the image.Curved sensory manifoldTo conclude, we consider the initial example where the agent can freely trans-late and rotate (Figure 3.7). This will theoretically result in a sensory mani-fold that is curved with respect to the transformation based metric, and theMDS procedure cannot therefore avoid a fundamental distortion of distances.Nonetheless, if we consider only a limited area of the sensory manifold thendistortions should be kept reasonnable (as in the case for maps of limitedareas of the earth).This additionally allows us to check that the algorithm is indeed able tolearn by itself the difference between commuting and noncommuting motorcommands (i.e. flat and curved directions in the sensory manifold).3 Dimensionality Reduction 125Fig. 3.7. Data is obtained by the camera with a variation of the three parameters(x, y, θ). For clarity, we represent only a subset of this data: points whose initialcoordinates in the 2D world are of type (i, j, α) where i and j are integers andα = 15 or 60. The resulting embedding shows as expected the appearance of a”millefeuilles”, where the vertical axis perfectly relates to the rotation angle and thehorizontal axis to x and y.3.5.3 Related workNon-trivial metrics on dataIn recent work oriented towards MRI, [52] and [69] hint that classical methodsfor dimensionality reduction produce parameters without intuitive meaning.Starting with pictures of an object undergoing unknown deformations, theyaim at characterizing these deformations and tracing back the journey ofthe object. They propose to introduce an appropriate distance measure inIsomap using complex Gabor filters applied on image samples. Argumentsgiven for the use of such a distance measure however only revolve arround thespecific task of image processing, while we have tried to build a more generalperspective. It could be interesting to make the link with this approach, asthe metric based on Gabor filters actually turns out to be invariant.In [70], it is shown how a discrete-valued auxiliary variable dependent onthe original data induces a statistical metric on these data (namely: Fisher dis-tance between the conditional probabilities of the auxiliary variable). Thereis an obvious similarity with the information bottleneck method [45]. Thisauxiliary variable could however be argued to constitute an indirect way ofsupervising these algorithms, as it is not clear to what they could correspond126 Philipona and O’Reganin the general framework of an embodied agent. Nonetheless, this is poten-tially of considerable interest if considered for instance in interaction with thecommunication system: linguistic interaction with a user may indeed providesuch auxiliary variables. This approach is therefore complementary to ours,as we may envision the existence of several metrics on sensory data dependingon the task the agent is involved in: a sensorimotor metric for self insertionin space and basic sensorimotor tasks, a side-information related metric fortasks requiring to match sensory data with auxiliary data.More generally, information theory, Bayesian frameworks, statistical con-siderations on the input space also provide ways to build or improve a metric(see for instance [71]) and define a distance relevant to the problem.Lie groups on sensory data[72] propose an algorithm able to estimate the generator matrix G of a onedimensional Lie group, and already suggested that this could be the groundfor object recognition methods robust to change of viewpoint. Their workis extended by Bengio et al. in [73] where they develop a method for non-local manifold learning. Their unsupervised algorithm is designed to learnand infer, even at a large scale, the transformation on which it is trained.In our case such large scale predictions turned out to be unreliable, which isthe reason why we based our work on direct learning (rather than inference)of predictors Q̄ij for action sequences. The reason for the unreliability is theexistence of a fundamental undetermination for sensory data evolution: in thecase of pictures this comes from the fact that incoming pixels on the borderof patches cannot be fully predicted from the content of the patch before themove. [72] solves the question of large-scale predictions using an implicit cyclichypothesis (i.e. by assuming the signal to be periodic). Our work suggests anew way to tackle (or actually obviate) this problem.The idea of the Land survey lemma is implicit in previous work of ours[74], and was already considered as a way to estimate the group structureof compensable transformations for an agent-environment system, i.e. trans-formation of the whole system leaving the agent’s sensory inputs invariant.The most important difference with this work is that the notion of symmetrygroup for the agent’s sensorimotor interactions presented here is more gen-eral and realistic than that of compensable transformations. To temper thisimprovement, it must be recalled however that we did not provide in thismore general context an algorithmic way to distinguish between the differentnotions of locomotion depicted in section 3.4.1: it is only because we consid-ered an example with no possible change in relative configuration that thetransformation based metric at the basis of the algorithm is invariant to thesymmetry group.3 Dimensionality Reduction 127Theory of invariants in artificial visionThe importance of the notion of invariance, and especially invariance to asymmetry group, is a standard of artificial vision. Indeed, the problem of rec-ognizing objects, whose shape undergoes a geometric viewing transformation,often arises in machine vision tasks. For instance research done on struc-ture from motion, i.e. reconstruction of three dimensional information fromsequences of two dimensional projections, makes extensive use of the math-ematical theory of invariants [75]. Much work has also been devoted to thedevelopment of multiscale geometric representations of shape which are in-variant to a number of the typical viewing transformations in vision [76]. Themain difference between these approaches and ours, apart from taking theperspective of cognitive science rather than that of artificial vision, is thekind of objects manipulated. [75] abstract from images points of interest thatare represented as pair of viewfield coordinates, and [76] manipulate objectssuch as planar curves. We try to address the issue of sensory processing inthe raw format in which sensory data are provided to the agent, without us-ing an interpretation (e.g. in terms of geometric points or curves) of sensorydata (e.g. of pictures). Actually, our aim precisely is ultimately to come to aninterpretation of sensory data.Algorithms for uninterpreted sensors and effectorsMuch work has been devoted to the development of algorithms for embodiedagents that do not assume programmer’s knowledge of the agent body. Forinstance in the domain of map learning, [77] present methods to learn in-creasingly complex interfaces to control in space a robot whose sensorimotorapparatus and environment are initially unknown. In the domain of develop-mental robotics, more precisely of the aquisition of a body schema, [78] takeinspiration from earlier work to implement an algorithm aiming to learn itsbody map by measuring statistical dependence between its various sensors.The main difference between these approaches and ours is that the met-rics used do not measure the same aspects of sensorimotor interactions: eventhough spatial proximity of sensors presumably implies some form of statisti-cal dependence between sensor outputs, the metric induced by the symmetrygroup of system dynamics and the metric induced by sensors’ statistical de-pendence [79] a priori measure two different things. It would be interesting tofurther investigate the link between the two, but it seems at first glance that asystematic link can only be expected between sensors’ statistical dependenceand topological sensor relationships, and not between sensors’ statistical de-pendence and sensors’ spatial metrical relationships. As for side-informationmetrics, it must be stressed that there is no reason to be exclusive about theuse of only one metric: depending on the task and objectives, different metricsmay be appropriate and therefore the approaches are complementary ratherthan competing.128 Philipona and O’ReganThere is an additional difference between the previous approaches andours: we deliberately take a holistic approach with respect to sensory inputs,i.e. we do not make the assumption that individual coordinates of sensory dataare meaningful13 since this actually is an assumption about the format of thesensory code. Consider for instance the case of color vision. The relevance oftackling sensory processing from the viewpoint of each different kind of pho-toreceptor is not clear: color information is an intrinsically multidimensionalobject. Apart from possible changes in subsequent computational abilities, thebasis used by the agent’s sensory system to represent this color informationshould not affect sensory abilities of the agent.3.6 ConclusionMany technical issues remain to be solved so as to implement the algorithmrobustly enough and efficiently enough for real robotic systems. It is no sur-prise that the generality of the approach implies a large increase in computa-tional load as compared to methods optimized for specific devices and tasks.This is an artifical version of the altricial-precocial trade-off that biologicalorganisms have to face: adaptability versus time constraints in learning andprocessing [80]. In the simple examples provided in section 3.5.2, dealing withc × c patches and a actions requires estimating on the order of O(a2 · c4)parameters (for instance to learn the Q̄ij matrices), which roughly requireshandling a learning dataset on the order of O(a2 · c2) patches. Operations oflinear algebra used to perform the estimation of predictors will then requireon the order of O(a2 · c6) elementary operations, based on the fact that thecomplexity of matrix multiplication and inversion is at least cubic in the di-mension of matrix entries [81]. Such a high price however results from therequirement of designing an adaptive system with centralized computationalabilities, i.e. possibly implemented in a Turing machine. As for biological sys-tems, the computational notion of complexity is not so clear, and one must becautious when trying to apply similar frameworks to understand the abilitiesof biological systems and to conceive artificial agents mimicking such abilities.It must also be stressed that an important aspect of the current approachis to constrain the choice of a metric over sensory inputs corresponding to thesame relative configuration of the agent-environment system, without so faraddressing the point of a constraint across different relative configurations.This is a critical point that calls for more theoretical work and is presumablylinked with the notion of objects, as a change of relative configuration occursin particular as a result of displacements of objects in the environment relativeto each other and relative to the agent.13 While such an assumption is made when sensors are implicitly defined as coordi-nates of sensory inputs.3 Dimensionality Reduction 129Beside the previous warnings, many interesting developments can be en-visioned. In addition to displacements, a large variety of transformations onsensory inputs could be considered. In the case of a visual system, blurring,lighting, or focus are target examples, provided that the linear model 3.7holds. The structure of transformations with respect to sequences of actionswill then be more complex than that of a displacement group, yielding newcriteria to distinguish between actions. The coordination of several metrics, asmentioned in the overview of related work, is another possible line of develop-ment. In particular this could be used to measure distances between sensoryinputs that cannot in any way result from agent actions: this would providea necessary complement to our work.In conclusion, we have suggested that dimension reduction could be a firststep towards an unsupervised interfacing between an agent’s sensory systemand its cognitive function, therefore a first step towards an unsupervised self-insertion of the agent in space. We started by giving a survey of standard ap-proaches to dimensionality reduction, and noted their (unjustified) reliance onan a priori metric on sensory inputs. We then proposed a general mathemati-cal framework to describe an embodied agent and showed that this frameworksuggested the use of specific metrics grounded on sensorimotor interactions.We finally described an algorithm aiming at learning such metrics, and putthis algorithm to use on some simple examples.References1. J. K. O’Regan, A. Noë, A sensorimotor account of vision and visual conscious-ness, Behavioral and Brain Sciences 24 (5) (2001) 939–973.2. M. Lungarella, O. Sporns, Mapping information flow in sensorimotor networks,PLoS Computational Biology 2 (10) (2006) 1301–1312.3. R. Bajcsy, Active perception, IEEE Proceedings 76 (8) (1988) 996–1006.4. J. Aloimonos, I. Weiss, A. Bandyopadhyay, Active vision, International Journalof Computer Vision 1 (4) (1988) 333–356.5. D. Ballard, Animate vision, Artificial Intelligence 48 (1) (1991) 1–27.6. J. Eklundh, K.Pahlavan, Head, eye and head-eye system, in: SPIE Applicationsof AI X: Machine Vision and Robotics, Orlando, Fla., 1992.7. I. Findlay, J. M. Gilchrist, Active vision: The psychology of looking and seeing,Oxford: Oxford University Press, 2003.8. D. Philipona, J. K. O’Regan, Color naming, unique hues, and hue cancellationpredicted from singularities in reflection properties, Visual Neuroscience 23 (3-4) (2006) 331–339.URL http://cognitivesystems.org/cosybook/chap3.asp#Philipona069. A. Sloman, Image interpretation: The way ahead?, in: O. Braddick, A. Sleigh.(Eds.), Physical and Biological Processing of Images (Proceedings of an in-ternational symposium organised by The Rank Prize Funds, London, 1982.),Springer-Verlag, Berlin, 1982, pp. 380–401.10. L. P. Kaelbling, M. L. L.ittman, A. Cassandra, Planning and acting in partiallyobservable stochastic keywords, Artificial Intelligence 101 (1998) 99–134.130 Philipona and O’Regan11. P. Beeson, M. MacMahon, J. Modayil, J. Provost, F. Savelli, B. Kuipers, Exploit-ing local perceptual models for topological mapping, in: Proc. IJCAI Workshopon Reasoning with Uncertainty in Robotics (RUR 2003, Acapulco, Mexico),2003, pp. 15–22.12. R. Reiter, Knowledge in action : logical foundations for specifying and imple-menting dynamical systems, Cambridge, Mass.: MIT Press, 2001.13. B. Kuipers, The spatial semantic hierarchy, Artificial Intelligence 119 (2000)191–233.14. H. von Helmholtz, Selected Writings of Hermann Helmholtz, Wesleyan Univer-sity Press, 1878, Ch. The Facts in Perception, pp. 115–147.15. H. Poincaré, La Science et l’Hypothese, Flammarion, 1902.16. J. Piaget, The origins of intelligence in children, W.W. Norton & Company,Inc., 1936.17. J. Gibson, The senses considered as perceptual systems, Boston, MA: HoughtonMifflin, 1966.18. D. Chalmers, The conscious mind: In search of a fundamental theory, New York:Oxford University Press, 1996.19. S. Zeki, A. Bartels, The autonomy of the visual systems and the modularity ofconscious vision, Philosophical Transactions of the Royal Society B: BiologicalSciences 353 (1377) (1998) 1911–1914.20. G. Rees, G. Kreiman, C. Koch, Neural correlates of consciousness in humans,Nature Reviews Neuroscience 3 (2002) 261–270.21. F. Crick, C. Koch, A framework for consciousness, Nature Neuroscience 6 (2)(2003) 119–126.22. J. B. Tenenbaum, V. de Silva, J. C. Langford, A global geometric frameworkfor nonlinear dimensionality reduction, Science 290 (2000) 2319–2323.23. G. C. Reinsel, R. P. Velu, Multivariate reduced-rank regression: Theory andapplications (lecture notes in statistics), Springer-Verlag Telos, 1998.24. M. Tipping, C. Bishop, Probabilistic principal component analysis, Tech. rep.,Neural Computing Research Group, Aston University, Birmingham, UK. (1997).25. M. Artac, M. Jogan, A. Leonardis, Incremental pca for on-line visual learningand recognition., in: C. S. R. Kasturi, D. Laurendeau (Ed.), Proceedings 16thinternational conference on pattern recognition, 2002, pp. 781–784.26. T. Voegtlin, Recursive principal components analysis, Neural Networks 18 (8)(2005) 1051–1063.27. J. H. Friedman, J. W. Tukey, A projection pursuit algorithm for exploratorydata analysis, IEEE Transactions on Computers 23 (1974) 881–890.28. J. H. Friedman, Exploratory projection pursuit, Journal of the American Sta-tistical Association 82 (1987) 249–266.29. M. Collins, S. Dasgupta, R. Schapire, A generalization of principal componentsanalysis to the exponential family, in: T. G. Dietterich, S. Becker, Z. Ghahramani(Eds.), Advances in neural information processing systems, Vol. 14, Cambridge,MA: MIT Press, 2002.30. N. Kambhatla, T. Leen, Dimension Reduction by Local Principal ComponentAnalysis, Neural Computation 9 (7) (1997) 1493–1516.31. J. J. Verbeek, N. Vlassis, B. Kröse, Coordinating mixtures of probabilistic Prin-cipal Component Analyzers, Tech. rep., Computer Science Institute, Universityof Amsterdam, The Netherlands, iAS-UVA-02-01 (feb 2002).3 Dimensionality Reduction 13132. S. Vijayakumar, S. Schaal, Local dimensionality reduction for locally weightedlearning, IEEE International Symposium on Computational Intelligence inRobotics and Automation (1997) 220–225.33. B. Scholkopf, A. Smola, K. Muller, Nonlinear Component Analysis as a KernelEigenvalue Problem, Neural Computation 10 (5) (1998) 1299–1319.34. K. Chang, J. Ghosh, A unified model for probabilistic principal surfaces, IEEEtransactions on pattern analysis and machine intelligence 23 (2001) 22–41.35. P. Fletcher, C. Lu, S. Joshi, Statistics of Shape via Principal Geodesic Analysison Lie Groups, in: IEEE Computer Society Conference on Computer Vision andPattern Recognition, Vol. 1, IEEE Computer Society; 1999, 2003.36. W. Torgerson, Multidimensional scaling: I. Theory and Method, Psychometrika17 (1952) 401–419.37. R. Shepard, The analysis of proximities: Multidimensional scaling with an un-known distance function. II, Psychometrika 27 (3) (1962) 219–246.38. J. B. Kruskal, Multidimensional scaling by optimizing goodness of fit to a non-metric hypothesis, Psychometrika 29 (1964) 1–27.39. J. Sammon, A nonlinear mapping for data structure analysis, IEEE Transactionson Computers 18 (5) (1969) 401–409.40. M. Belkin, P. Niyogi, Laplacian Eigenmaps for Dimensionality Reduction andData Representation, Neural Computation 15 (6) (2003) 1373–1396.41. T. Kohonen, Self-organizating maps., Springer-Verlag, 1997.42. P. Demartines, J. Hérault, Cca: Curvilinear component analysis, in: 15th work-shop GRETSI, 1995.43. T. Martinetz, S. Berkovich, K. Schulten, Neural-gas’ network for vector quanti-zation and its application totime-series prediction, IEEE Transactions on NeuralNetworks 4 (4) (1993) 558–569.44. B. Fritzke, A growing neural gas network learns topologies, in: D. S. T. G.Tesauro, T. K. Leen (Eds.), Advances in neural information processing systems,Vol. 7, MIT Press, Cambridge MA., 1995, pp. 625–632.45. D. Chigirev, W. Bialek, Optimal manifold representation of data: an informationtheoretic approach, Advances in Neural Information Processing Systems 16.46. S. T. Roweis, L. K. Saul, Nonlinear dimensionality reduction by locally linearembedding, Science 290 (2000) 2323–2326.47. J. A. Lee, A. Lendasse, M. Verleysen, Curvilinear distance analysis versusisomap, in: European symposium on artificial neural networks, 2002, pp. 185–192.48. M. Balasubramanian, E. Schwartz, J. Tenenbaum, V. de Silva, J. Langford, TheIsomap Algorithm and Topological Stability, Science 295 (5552) (2002) 7–7.49. J. Lee, M. Verleysen, Nonlinear dimensionality reduction of data manifolds withessential loops, Neurocomputing 67 (2005) 29–53.50. D. S. Broomhead, M. Kirby, A new approach to dimensionality reduction: The-ory and algorithms, SIAM Journal of Applied Mathematics 60 (6) (2000) 2114–2142.51. V. de Silva, J. Tenenbaum, Global Versus Local Methods in Nonlinear Dimen-sionality Reduction, Advances in Neural Information Processing Systems (2003)721–728.52. R. Pless, Differential structure in non-linear image embedding functions, in: Pro-ceedings of the 2004 Conference on Computer Vision and Pattern RecognitionWorkshop (CVPRW’04) Volume 1, IEEE Computer Society, Washington, DC,USA, 2004, pp. 10–17.132 Philipona and O’Regan53. B. Kuipers, Consciousness: Drinking from the firehose of experience, in: Pro-ceedings of the National Conference on Artificial Intelligence, Vol. 20, 2005, pp.1298–1305.54. A. Van der Schaft, H. Schumacher, An introduction to hybrid dynamical sys-tems, Lecture notes in control and information sciences.55. H. Haken, Synergetic computers and cognition: a top-down approach to neuralnets, Springer-Verlag, New York., 1991.56. R. Thom, Structural stability and morphogenesis, Benjamin-Addison Wesley.57. A. Isidori, Nonlinear Control Systems, Springer Communications and ControlEngineering Series, 1995.58. K. Thoroughman, R. Shadmehr, Learning of action through adaptive combina-tion of motor primitives, Nature 407 (2000) 742–747.59. M. Zinkevich, T. Balch, Symmetry in markov decision processes and its im-plications for single agent and multiagent learning, Eighteenth InternationalConference on Machine Learning (2001) 632640.60. I. Kolar, J. Slovak, P. W. Michor, Natural operations in differential geometry.,Berlin, Heidelberg, New York: Springer-Verlag, 1993.61. R. Montgomery, Optimal control of deformable bodies and its relation to gaugetheory, in: T. Ratiu (Ed.), The geometry of hamiltonian systems (Vol. 22), MSRIPublications, Springer-Verlag, 1991.62. J. E. Marsden, J. Ostrowski, Symmetries in motion:geometric foundations ofmotion control, Nonlinear Sci. Today.63. S. D. Kelly, R. M. Murray, Geometric phases and robotic locomotion, Tech. rep.,California institute of technology (1994).64. R. Mason, J. W. Burdick, Propulsion and control of deformable bodies in anideal fluid, in: Proceedings IEEE International Conference on Robotics and Au-tomation, Vol. 1, 1999, pp. 773–780.65. A. Shapere, F. Wilczek, Geometry of self-propulsion at low Reynolds number,Journal of Fluid Mechanics Digital Archive 198 (2006) 557–585.66. R. M. Murray, S. S. Sastry, L. Zexiang, A mathematical introduction to roboticmanipulation., Boca Raton, FL, USA: CRC Press, Inc., 1994.67. B. Goodwine, J. Burdick, Trajectory generation for kinematic legged robots,Proc. IEEE Int. Conf. Robotics and Automation 3 (1997) 2689–2696.68. D. Bump, Lie groups, Springer, 2004.69. R. Souvenir, R. Pless, Isomap and Nonparametric Models of Image Deformation,IEEE Workshop on Motion and Video Computing 2 (2005) 195–200.70. J. Peltonen, A. Klami, S. Kaski, Improved learning of riemannian metrics forexploratory data analysis, Neural Networks 17 (2004) 1087–1100.71. E. Xing, A. Ng, M. Jordan, S. Russell, Distance Metric Learning with Appli-cation to Clustering with Side-Information, Advances in Neural InformationProcessing Systems 15 (2003) 521–528.72. R. Rao, D. Ruderman, Learning Lie groups for invariant visual perception, Pro-ceedings of the 1998 Conference on Advances in Neural Information ProcessingSystems II (1999) 810–816.73. Y. Bengio, O. Delalleau, N. Le Roux, J.-F. Paiement, P. Vincent, M. Ouimet,Spectral dimensionality reduction, in Cirano working papers. (2004).74. D. Philipona, J. K. O’Regan, J.-P. Nadal, O.-M. Coenen, Perception of thestructure of the physical world using unknown sensors and effectors, Advancesin Neural Information Processing Systems 16 (2004) 945–952.3 Dimensionality Reduction 13375. P. Bazin, M. Boutin, Structure from Motion: A New Look from the Point ofView of Invariant Theory, SIAM JOURNAL ON APPLIED MATHEMATICS64 (4) (2004) 1156–1174.76. J. P. Olver, G. Sapiro, A. Tannenbaum, Differential invariant signatures andflows in computer vision:a symmetry group approach, Tech. rep., University ofMinnesota and MIT (1993).77. D. Pierce, B. Kuipers, Map learning with uninterpreted sensors and effectors,Artificial Intelligence 92 (1997) 169–229.78. V. Hafner, F. Kaplan, Interpersonal maps and the body correspondence prob-lem, in: Y. Demiris, K. Dautenhahn, C. Nehaniv (Eds.), Proceedings of the thirdinternational symposium on imitation in animals and artifacts, Herderfordshire,UK., 2005, pp. 48–53.79. J. P. Crutchfield, Information and its metric, Nonlinear Structures in PhysicalSystems - Pattern formation, Chaos and Waves (1990) 119–130.80. A. Sloman, J. Chappell, The Altricial-Precocial Spectrum for Robots, Interna-tional Joint Conference on Artificial Intelligence 19 (2005) 1187–1192.81. P. Giorgi, C. Jeannerod, G. Villard, On the complexity of polynomial matrixcomputations. issac03, philadelphia, usa, in: ISSAC, ACM Press, 2003, pp. 135–142.4Categorical PerceptionMario Fritz1, Mykhaylo Andriluka1, Sanja Fidler2, Michael Stark1, AlesLeonardis2, Bernt Schiele11 Technische Universität Darmstadt, Darmstadt, Germany,lastname@cs.tu-darmstadt.de2 VICOS Lab, University of Ljubljana, Slovenia,firstname.lastname@fri.uni-lj.siThe ability to recognize and categorize entities in its environment is a vitalcompetence of any cognitive system. Reasoning about the current state of theworld, assessing consequences of possible actions, as well as planning futureepisodes requires a concept of the roles that objects and places may possiblyplay. For example, objects afford to be used in specific ways, and places areusually devoted to certain activities. The ability to represent and infer theseroles, or, more generally, categories, from sensory observations of the world, isan important constituent of a cognitive system’s perceptual processing (Sec-tion 1.3 elaborates on this with a very visual example).In the CoSy project, a substantial amount of work has been conducted onthe advancement of methods that recognize and categorize objects and placesby using different modalities, namely, vision, language, and laser range data.Our progress contributes to our effort to build systems that evolve throughinteraction with its environment in an ultimately live-long learning process.While this chapter describes our contribution to modeling, learning andrepresenting of visual categories, Chapter 7 shows how to combine the vi-sual information with other modalities in a multi-modal learning process (e.g.speech/language as detailed in Chapter 8). Finally, Chapter 9 and 10 showshow we integrated these concepts in a autonomous systems to understandthe implications of our progress in categorization on an interactive evolvingsystem.4.1 IntroductionRecently there has been significant progress in visual class recognition rangingfrom visual category modeling (e.g. [1]), learning (e.g. [2]), robustness (e.g. [3])and scalability to more classes (e.g. [4]). Interestingly, there is a surprising di-versity in the proposed approaches. E.g. the employed features representationsrange from local to global and object representations range from generative136 Fritz et al.to discriminant models. In the following we start with a discussion of themost important design and paradigm choices highlighting the implied meritsand draw-backs. This leads to the conclusion, that no single paradigm aloneis powerful enough to deal with the implicit challenges of object categoriza-tion. Instead we argue in this chapter that the object categorization problemrequires a wide variety of approaches. This chapter therefore summarizes var-ious approaches that have been explored during the CoSy project. For eachof these approaches we will discuss the respective strengths and weaknessand will discuss how these approaches advance the state of the art in objectcategorization.Fig. 4.1. Illustration of some challenges encountered in visual categorization. We ob-serve high intra-class variation and low inter-class variation. Observed effects occurat local or global scale with respect to members of the same category or contrastedto other categories.Figure 4.1 illustrates one of the fundamental challenges in visual catego-rization: Many object classes (such as dogs) have high intra-class variabilitybut may share similar features and appearance across different classes (e.g.dogs, cows and horses). Such similarities then lead to low inter-class varia-tion. This challenge taken together with other challenges related to objectcategorization (robustness, viewpoint changes, partial occlusion, light condi-tions, etc.) have lead to a wide variety of approaches. In the following wediscuss three of the most prominent design choices in more detail, namelygenerative vs discriminative modeling, local vs. global object features, andthe importance of the learning paradigm.Generative vs. Discriminative Paradigms:There is a fundamental difference whether an approach models all that is com-mon to all category members (intra-class variation as illustrated in Figure 4.1by the dogs) or all that distinguishes themselves with respect to others (inter-class variation as illustrated in Figure 4.1 by the other animals). Technicallyspeaking – in the generative paradigm – the object model represents all prop-erties or object instances of an object class. In contrast – in the discriminative4 Categorical Perception 137paradigm – the model retreats to a simpler mapping from the observed prop-erties to the predicted category label (discriminant or conditional learning).Both views have a multitude of benefits and drawbacks. As the generativemodel of an object class is typically learned independently of other objectclasses the generative paradigm lends itself to parallelism and incrementallearning. Generative methods also enable principled ways of handling missingdata (e.g. due to occlusion) while inference is typically computationally ex-pensive. Discriminant models on the other hand are designed to capture thedifference between different categories which often leads to higher precision.Typically, this comes at the price of a large training set including a substantialamount of background data.Local vs. Global Object Representation Paradigms:Another important design choice for object category representation is the ap-propriate level of locality used in the feature representation. Approaches rangefrom local interest point-based approaches (like the popular SIFT-feature overedge-segments proposed by [5]) to fully global representations (like the globalHOG-representation proposed by [6]). On the one hand, local features can bemade robust to scale and rotation changes and therefore enable robustnessto partial occlusion. On the other hand, they require sophisticated models todescribe the geometric and topological structure of object classes. In contrast,global feature descriptors inherently describe the geometric and topologicalstructure by describing the global appearance of an object. On the downside,however, they do not expose the desired robustness to partial occlusion andit becomes more difficult to generalize across-instance.Different Learning Paradigms:Over the years various approaches have been proposed for the recognition ofobject categories often based on models learned directly from image data.The approaches, however, vary greatly in the amount of supervision providedfor the training data. The types of annotation varies from pixel-level segmen-tations, (e.g. [7]) over bounding-box annotations (e.g. [8]) and image levelannotation (e.g. [2, 9]) to unsupervised methods (e.g. [10, 11, 12]), which donot even require the information which category is presented in which im-age.While approaches using more supervision tend to require less trainingdata, there is a clear desire to use less supervision typically at the price to usemore unlabeled training data.Central Role of Representation:As motivated in Section 1.4.2, representation plays a central role in a cognitivesystem. The way categories are stored, organized and related to each otherdetermines the capabilities to learn, evolve and therefore to interact of theoverall system. In Chapter 7 on multi-modal learning and in particular in our138 Fritz et al.integration efforts in Chapter 9 and 10, we’ll detail how representation affectsthe capabilities of the overall system.The above discussion leads to the conclusion that no single combination ofthese paradigms is powerful and versatile enough for the general problem of ob-ject categorization. It is commonly believed that relying on a single paradigmof addressing the category phenomenon won’t do justice to the overwhelmingdiversity encountered in categorial perception. Throughout this chapter wetherefore promote learning-based approaches, that are capable to adapt tothe particularities of each categories. In this sense we have explored – duringthe course of the CoSy-project – a variety of different object representationintegrating and advancing the state of the art for different of the above men-tioned paradigms.We perceive the representation of visual categories as a central elementof each vision system, as it influences scalability in the number of categories,capability to evolve over time, addressable categories and it is the startingpoint of categorial reference in the interplay with other components in anintegrated system. The following therefore presents the different approachesexplored both w.r.t. its underlying representations as well as w.r.t. the abovementioned paradigms. In particular, we pursue this idea to progress towardsscalable representations in Section 4.2.1, representations for affordance-basecategorization in Section 4.2.2, representation by generative decompositionsin Section 4.3 and representation of dynamics in Section 4.4. To conclude thisintroduction, we outline how these 4 directions progress over the previousstate-of-the-art in categorial perception.4.1.1 Towards hierarchical scalable representationsAs discussed above one of the important design choices is the locality of thefeature representations. The first approach described in section 4.2.1 aims tobridge the gap between purely local to more global representations by auto-matically learning a hierarchy from local to more global representations. Sucha hierarchical representations are believed to make approaches more scalableand allow for successive evidence aggregation. The proposed hierarchical rep-resentation is learned bootom-up and in an unsupervised fashion leading fromlocal to more global representations.Also computational considerations suggest that matching should be per-formed hierarchically, in order to gradually and coherently limit the otherwisecomputationally prohibited search space [13, 14, 15, 16, 17, 18, 19, 20]. Wepropose a novel approach to representing object categories within an index-able, hierarchical compositional framework.4.1.2 Towards Representations for affordance-based categorizationA second approach (Section 4.2.2) explores the possibility to represent objectsrelated to their visual affordances. In this case we explore and analyze a variety4 Categorical Perception 139of local region descriptors in a generative object model that is trained in asupervised fashion.Most local region descriptors, such as the popular SIFT descriptor, ex-plicitly encode local appearance, and have hence shown impressive results onobjects with sufficient local appearance statistics, such as cars and motorbikes[2, 21]. However, for many object categories that might be important for in-teractions between a cognitive agent and its environment, local appearancemight not be the most useful cue. Tools, cutlery, things in office and kitchenenvironments, and other man-made artifacts seem to require features thatcapture the respective shape and geometry rather than local appearance. Inparticular, categorizing those objects according to supported functions andaffordances demands an explicit representation of geometric properties.4.1.3 Representations and discovery of object classes bygenerative decompositionsThe third approach combines different paradigms ranging from unsupervisedlearning of generative object models to supervised learning for discriminantobject detection. Object representations for categorization tasks should be ap-plicable to a wide range of objects, scaleable to handle large numbers of objectclasses, and at the same time learnable from a few training samples. Whilesuch a scalable representation is still illusive today, it has been argued thatsuch a representation should have at least the following properties: it shouldenable sharing of features [22], it should combine generative models with dis-criminative models [23, 24] and it should combine both local and global aswell as appearance- and shape-based features [3]. Additionally, we argue thatsuch object representations should be applicable both for unsupervised learn-ing (e.g. visual object discovery) as well as supervised training (e.g. objectdetection). We achieve such a representation by learning a generative decom-position of localized oriented gradients that exploit the co-occurence statisticin the presented data. The obtain components range from local to global andlend themselves to unsupervised learning learning as well as supervised state-of-the-art detection.4.1.4 Representations of object dynamicsAn important type of object classes is articulated such as people and animals.Therefore our fourth and last approach presented in this chapter considers thechallenging problem of modeling the dynamics of articulated objects. For thiswe explore a generative object model that represents a person as the topol-ogy of individual body parts. Furthermore a generative model is learned torepresent the dynamics of the human walking cycle. The aim is to improve de-tection and tracking in cluttered scenes using a monocular, potentially movingcamera. Probably the most fundamental difficulty in detection and trackingmany people in cluttered scenes is that many people will be partially and also140 Fritz et al.fully occluded for longer periods of times. Consequently, both the detectionof people in individual frames as well as the data-association between peopledetections in different frames are highly challenging and ambiguous. The de-veloped motion model lends to a more detailed interpretation of the observedsequences.4.2 Low-Level Features and Hierarchical RepresentationLearning4.2.1 Towards Scalable Representations for Visual CategorizationMotivationHierarchical representations can derive and organize the features at multiplelevels that build on top of each other by exploiting the sharability of featuresamong more complex compositions or objects themselves [25, 26, 27, 17]. Thisendows them with high computational efficiency and expressive power thatsurpasses the capabilities of flat representations. A number of hierarchicalrecognition systems have been proposed and confirmed the success of suchrepresentations in object categorization tasks [28, 29, 27, 16, 25, 30, 21, 31,32, 33, 26]. However, an automatic construction of the visual hierarchy thatwould scale well with the number of image categories (that are in nature inthe order of tens of thousands) is still an open issue.We propose a novel approach to representing object categories within anindexable, hierarchical compositional framework. We develop a bottom-up, sta-tistical approach that makes use of simple atomic features, i.e. oriented edges,to gradually learn more complex contour compositions. The learned libraryof features, i.e. parts, is organized with accordance of principles of efficientindexing which ensures that local retrieval of models during the online imageprocessing stage will run in a roughly constant time despite the exponentialincrease in size along the hierarchical layers. The learned compositions canthen be combined into objects with minimal human supervision, whereby thehierarchical sharability of features and the efficient indexability constraintscould be a step towards scalable representations of object categories.We substantiate our choice of design principles proposed to devise a plau-sible representation of object categories with respect to the shape modality:Hierarchical compositionality. Compositionality refers to hierarchi-cal representations defined in terms of parts and their spatial relationswhere the hierarchically constructed entities are built from a relatively smallnumber of lower-level constituents [34]. Computational benefits of composi-tionality in terms of storage, processing demands and the exponential ex-pressive power have long been emphasized in the computer vision litera-ture [18, 16, 35, 20, 36]. Since each hierarchical unit is shared among many4 Categorical Perception 141more complex higher layer compositions, the computational cost is highly re-duced compared to searching for each complex interpretation in isolation. Inaddition, while the receptive field sizes increase with the level of hierarchy,formed compositions are designed to respond to only smaller spatial subsetsin their receptive fields. This offers much higher robustness and faster process-ing over the traditional neural networks approaches, which mainly integrateover the complete receptive fields both spatially as well as all feature valuesin the layer below.A part of the biological evidence could potentially support such a lineof architecture [37, 38]. Additionally, attempts have been made to map themathematical theory onto the neuronal structure of the visual cortex [34].Computational plausibility. Hierarchical representations do not nec-essarily imply computational efficiency. Our main motivation for building ahierarchical visual representation is to enable fast indexing and matching ofimage features against hierarchically organized stored prototypes in order toavoid the computationally prohibitive linear search in the number of higher-level compositions and objects themselves [36, 13].Statistics driven learning. Features and their higher level combinationsshould be learned in an unsupervised manner (at least in the first stages of thehierarchy) in order to avoid hand-labelling of massive image data as well asto capture the regularities within the visual data as effectively and compactlyas possible [39, 40, 41, 19, 20]. Once the visual building blocks are learned,learning of novel objects can proceed mainly in the higher hierarchical layersand can thus operate fast and with no or minimal human supervision.Robust and repeatable detection. To achieve robustness against noiseand clutter, the features comprising the individual hierarchical layers shouldbe manifested as models to enable a robust verification of the presence of theirunderlying components [17, 13, 18]. Models should incorporate loose geometricrelations to achieve the spatial binding of features [41, 15], yet encode enoughflexibility to ensure repeatability and gain discrimination gradually – throughcomposition within the hierarchy.Learning a Hierarchy of Parts (LHOP)The compositional library of partsLet Ln denote the n-th Layer of the hierarchical library. Parts within thehierarchy are defined recursively in the following way. Each part in Ln codesspatial relations between its constituent subparts from a layer below. Formally,each composite part in Ln is characterized by a central subpart and a list ofremaining subparts with their positions relative to the center:Pn` =(Pn−1central, {(Pn−1j ,µj , Σj)}j),where µj = (xj , yj) denotes the relative position of subpart Pn−1j , while Σjdenotes the variance of its position around (xj , yj).142 Fritz et al.The hierarchy starts with a fixed L1 composed of a set of arbitrary filters.Here we choose a set of Gabor filters that best respond to oriented edges. Anexample of a L3 composition is depicted in Figure 4.2.Fig. 4.2. Left: Example of a L3 composition. Middle: Indexing – evoking higherlevel composite hypotheses. Right: Matching – verification of a composite librarypart.Hierarchical processing: Indexing and matching schemeEach image is first processed with a fixed Layer 1, the procedure that differsfrom the hierarchical processing further on. The hierarchical processing stepsare all general in its traversal from one layer to the next, thus described in itsgeneral form.Processing with Layer 1. For any given image, the process starts bydescribing the image in terms of discrete points denoting local oriented edges.This is done on every scale – each rescaled version of the original image (aGaussian pyramid with two scales per octave) is processed separately. First,each image in the pyramid is filtered by 11× 11 Gabor filters. By extractinglocal maxima of the Gabor energy function that are above a low threshold, animage (on each scale) is transformed into a list of L1 parts; {π1k}i. In general,let πnk stand for a realization of the Ln part Pn with a corresponding locationat which it was detected in an image; πnk = {Pn, xk, yk} (here k denotes thesuccessive number of the found part).Hierarchical processing. In order to find a higher level image interpre-tation, the local neighborhoods around the detected Ln−1-parts are comparedagainst the composite, Ln-parts stored in the hierarchical library. Each partπn−1k = (Pn−1k , xk, yk) in the image under consideration is subjected to theindexing and matching procedure – efficient local search for higher level com-positions.The part Pn−1k encoded in πn−1k plays the role of the central part in onlya subset of all compositions at layer Ln of the library. This list is an internalpart of the library and can be accessed in constant time during the onlineprocessing of images – the process referred to as indexing. The matching step4 Categorical Perception 143demands comparing the local spatial neighborhood of πn−1k against the al-lowable (retrieved in the indexing step) prototypical compositions within thehierarchical library. Matching of one such composition demands checking forthe presence of all subparts pertaining to the composition at hand at theirrelative locations, (x, y), and positioned within the allowed variances, Σ, withrespect to the position of the central part πn−1k . The indexing and matchingprocedure is schematically depicted in Figure 4.2.Unsupervised learning of part compositionsThe basic idea behind the learning procedure is to extract statistically salientcompositions that encode spatial relations between the constituent parts fromthe layer below. Each modeled relation between components allows also forsome displacement (variance) in spatial position.The learning algorithm is in principle general – proceeding in the samemanner when building each additional layer. It will thus be described in itsgeneral form.The learning process consists of three stages, namely, (1) the local inhibi-tion performed around each image feature (part), followed by (2) the statisticalupdating of the so-called spatial maps that capture pairwise geometric rela-tions between parts, and finally, (3) learning the higher order compositionsby tracking co-occurrence of spatial pairs. We must emphasize that each finalcomposition can have a varying number of subcomponents (the number canbe anything from 2 and larger).Learning is performed by gathering statistics over a large body of naturalimage data processed up to the last (learned) layer in the hierarchical library,e.g. Ln−1. Each image is thus represented by a list of parts with correspondinglocations, {πn−1k }k. A small local neighborhood around each πn−1k will beinspected in a two-stage process. The first, most crucial step aims to reducethe unnecessary redundancy coded in neighboring parts, referred to as localinhibition. Since each πn−1k is an (n − 1)-th order composition, it is in facta set union of a subset of L1 image parts. Within the inhibition step weremove all neighboring parts around πn−1k that have a large set intersectionwith respect to the L1 image parts. This step removes all features that codea large portion of edge structure already coded by πn−1k . In the next step,learning is performed by tracking frequent co-occurrences of part types andtheir relative locations.The learning process commends by forming a set of all allowable pairs ofpart identities. The list is accompanied by a set of empty matrices, where thedimensions correspond to the spatial extent of the local neighborhoods. Theprepared set thus contains information of type: Cnk,j := (Pn−1k ,Pn−1j ,Vk,j),where Vk,j represents a local spatial voting space for the corresponding com-bination of pairs of parts.Structure of small neighborhoods in terms of part locations is inspectedaround each part, πn−1k . The philosophy of local receptive field processing144 Fritz et al.is the following: the location of each part πn−1j = (Pn−1j , xj , yj) within theneighborhood of and relative to πn−1k will update the voting space Vk,j in Ck,jaccordingly:x := cx+ xj − xk, y := cy + yj − ykVk,j(x, y) = Vk,j(x, y) + 1,where (cx, cy) denotes the center of the spatial map Vk,j .After all images are processed, we detect voting peaks in the learned spatialmaps Vk,j , and for each peak, a spatial surrounding area is formed – modeledby a Gaussian distribution, (µj , Σj).In the final step, the local image neighborhoods are checked once againby projecting the learned spatial pairs and repeating the learning process byincreasing the number of subparts modeled in a composition (the reader isreferred to [20] for details) or by tracking the most frequent co-occurrences ofthe projected spatial pairs.The final selection of composite parts follows the indexibility constraint,i.e., each part of the lower, (n − 1)-th Layer, must not index into too manyhigher layer compositions. Thus the compositions acquired in the learningprocedure are sorted according to their decreasing probabilities and only anumber of statistically most salient compositions consequently define the nextlayer. We set the upper bound to the order of 10 − 20 times the number ofparts in the previous, (n − 1)-th Layer, meaning that on average each partin Ln−1 indexes into 10 to 20 composite parts in Ln. The thresholds usedare chosen to comply with the available computational resources and affectonly the number of finally selected parts and therefore the efficiency of therepresentation.Category-specific higher layersLearning the lower-layer sharable parts in a category-independent way canonly get so far - the overall statistical significance drops, while the numberof parts reaches its critical value for learning. Thus, learning of higher layersproceeds only on a subset of parts - the ones that are the most repeatablein a specific category. Specifically, the learning of higher layers is performedin images of individual categories, whereby the final categorical layer thencombines the most repeatable parts through the object center to form therepresentation of a category.Experimental resultsWe applied our method to a collection of 1500 images containing a number ofdiverse categories (cars, faces, mugs, dogs, etc.). The learned compositionalhierarchy consisted of 160 parts on Layer 2 and 553 Layer 3 parts (a few exam-ples from both layers are depicted in Fig. 4.3). The complete learning process4 Categorical Perception 145took approximately 5 hours on one core of an Intel Core-2 CPU 2.4 Ghz com-puter.To put the proposed hierarchical concept in relation to other hierarchicalapproaches as well as other categorization methods, which focus primarily onshape information, the approach was tested on the Caltech 101 database [42].The Caltech 101 dataset contains images of 101 different object categorieswith the additional background category. The number of images varies from31 to 800 per category, with the average image size of roughly 300×300 pixels.Each image was processed on 3 different scales, spaced apart by√2. Theaverage processing times per image per layer obtained with our C++ imple-mentation are reported in Table 4.1. Most of the processing time is spentfiltering an image with 6 Gabor filters (L1), which has not been optimizedfor performance. The features were combined with a linear SVM for multi-class classification. For this experiment we used 15 images for training and 15images for testing, disjunct from the training set. The results, averaged over8 random splits, are reported in Table 4.1 with classification rates of otherhierarchical approaches shown for comparison.Classification was also tested by varying the number of training examples.For testing, 50 examples were used for categories where this was possible andless otherwise. The classification rate was normalized accordingly. In all cases,the result was averaged over 8 random splits. The results are presented andcompared with other categorization methods in Table 4.1. We must emphasizethat the proposed model focuses on shape information only.Layer 1Layer 2 Layer 3Fig. 4.3. L1 (fixed), and learned L2 and L3 parts (only a subset is shown) used inthe Caltech 101 experiments.DiscussionWe presented a novel approach to building a representation of object cate-gories. The method learns a hierarchy of flexible compositions in an unsuper-vised manner in lower, category-independent layers, while requiring minimalsupervision to learn higher, categorical layers.146 Fritz et al.Table 4.1. Left top and right: Average classification rate (in percentage) on Caltech101. Left bottom: Average processing times per layer per image.Ntrain = 15 Ntrain = 30Serre et al. [29] 44 /Mutch et al. [43] 51 56Ranzato et al. [33] / 54Ommer et al. [26] / 61.3Wolf et al. [44] 51.18 /our method 60.5 66.5Processing time per 300× 300 imageLayer 1 1.6 sLayer 2 0.54 sLayer 3 0.66 sFig. 4.4. Top left two images: learned 3−layer hierarchy for the Caltech experiment;learned hierarchy for faces with compositional links shown. Examples of detectionsof categories cars, cups, and faces, where the first three layers in the library arecommon to all three categories.Furthermore, the design of parts is incremental, where new categories canbe continuously added to the hierarchy. Since the hierarchy is built as anefficient indexing machine, the system can computationally handle an expo-nentially increasing number of parts with each additional layer. The resultsshow that only a small number of higher layer parts are needed to representindividual categories, thus the proposed scheme would potentially allow foran efficient representation of a large number of visual categories.4 Categorical Perception 147(a)(b)(c)Fig. 4.5. Example images from the TUD-Shape ((a), (b)), and TUD-Shape2 (c)data sets, depicting the categories cup, fork, hammer, knife, mug, pan, pliers, pot,saucepan, and scissors.4.2.2 Representations for Functional and Affordance-BasedCategorizationMotivationRecent work in object categorization often uses interest operators in con-nection with local region descriptions to sparsely represent objects in im-ages. Typically, local regions are sampled around interest points (e.g ., fromHarris-Laplace, Hessian-Laplace, or Salient Regions detectors) at characteris-tic scales, and then described by local region descriptors. The resulting sparserepresentation is then vector-quantized using a codebook of visual words, andeither used directly for recognition (bag-of-words approaches [45]), or enrichedby information about the spatial layout of feature occurences.While these local feature-based representations have been successfully usedfor the recognition of object classes with sufficient local appearance statistics,they have rarely been employed in the context of functional or affordance-based categorization of objects, where, intuitively, an object’s shape and ge-ometry should be represented explicitly. Historically, the recognition of geo-metric objects such as cups and tables has been an important focus of objectrecognition [46]. In recent work however, it is largely underrepresented withsome notable exceptions [47, 48]. We therefore compare in [49], on a noveldata collection of 10 geometric object classes3 (see Figure 4.5), various shape-based features with appearance-based descriptors, such as SIFT. The analysisincludes a direct comparison of feature statistics as well as results withinstandard recognition frameworks, incorporating varying degrees of spatial in-formation.Appearance-Based vs. Shape-Based FeaturesWe briefly introduce the features and interest point detectors used in our com-parison. The shape-related features are k-Adjacent Segments [47], Geometric3 http://www.mis.informatik.tu-darmstadt.de/data148 Fritz et al.Blur [50], and Shape Context [51]; the appearance-based region descriptorsare SIFT [5] and GLOH [52]. As interest point detectors, we employ Harris-Laplace [53], Hessian-Laplace [54], and Salient Regions [55].k-Adjacent Segments (k-AS)k-Adjacent Segments have been proposed as an extension to contour segmentnetworks, a graph-based method for template-matching hand-drawings to im-age databases [56]. [47] demonstrates how k-AS can be incorporated into ageneral object recognition framework. We extract k-AS features using theoriginal implementation4. First, edgels are detected via the Berkeley naturalboundary detector [57]. Second, neighboring edgels are chained, and furtherlinked to form L-, T- and higher-order junctions. Last, edgel-chains are re-placed by straight line approximations (contour segments), and joined intoa global contour segment network for the image. A k-AS descriptor then de-scribes the geometric layout of a group of k adjacent segments in that networkw.r.t. relative position, orientation, and length. The dimensionality of k-ASfeatures is n = 4 ∗ k − 2. For the typical choices of k ∈ {2, 3}, n is 6 or 10,rendering k-AS a comparably low dimensional descriptor.Local Region DescriptorsGeometric Blur(GB). We use the original implementation5 of the Geomet-ric Blur [50] region descriptor from [58]. Geometric Blur first extracts c = 4channels of oriented edge energy [59] to obtain a sparse signal S. In S, theregion centered at interest point location x0 is blurred with a spatially-varyingGaussian kernel Gd to obtain the Geometric Blur Bx0(x) = S∗Gαx+β(x0−x).Bx0(x) is then sub-sampled over all channels at n distinct locations in a cir-cular grid. The final descriptor is the concatenation of all c × n samples.Throughout all experiments, we use the standard values for α = 0.5, β = 1and n = 51, resulting in a descriptor of length 204.Shape Context(SC). Shape Context [51] is originally based on edgeinformation. For a given interest point location, it accumulates the relativelocations of nearby edge points in a coarse log-polar histogram. We computea histogram containing 9 spatial bins over 4 edge orientation channels. Binsize increases w.r.t. distance from the interest point center. Note that this issimilar in spirit to spatially varying blur, but results in a smaller descriptor(length 36).SIFT. The Scale Invariant Feature Transform [5] descriptor is a 3D his-togram over local gradient locations and orientations, weighted by gradientmagnitude. It uses 4× 4 location and 8 orientation bins, i.e., 128 in total.GLOH. Gradient Location Orientation Histograms [52] is an extension ofthe SIFT descriptor. It uses 17 bins for location and 16 bins for orientation in a4 http://www.vision.ee.ethz.ch/˜ferrari/release-kas.tgz5 http://www.cs.berkeley.edu/˜aberg/demos/gb demo.tar.gz4 Categorical Perception 149histogram over a log-polar location grid, and reduces descriptor dimensionalityto 128 by PCA. We use the implementation6 of [52] for SC, SIFT and GLOH.All descriptors are made invariant to in-plane rotation by aligning the regionto the dominant gradient direction before descriptor computation.Interest Point DetectorsWe compute local region descriptors based on detections of the following in-terest point detectors: Harris-Laplace (HarLap) [54] is an extension to Har-ris corners [60]. It selects corners at locations where a Laplacian attains anextremum in scale-space. The Hessian-Laplace (HesLap) [54] detector re-sponds to blob-like structures. It searches for local maxima of the Hessiandeterminant, and selects a characteristic scale via the Laplacian as for Harris-Laplace. The Salient Regions (SalReg) detector [55] identifies local imageregions that are non-predictable across scales by measuring entropy over lo-cal intensity histograms. We use publicly available implementations for Har-Lap/HesLap7, and SalReg8.Feature EvaluationWe evaluate the combined performance of feature detectors and descriptorsat three different levels. First, we compute statistics over clusterings of lo-cal feature descriptors (codebooks). Second, we represent objects by means ofoccurrence statistics over codebook matches, and analyze classification perfor-mance in a Bayesian framework. Third, we investigate the impact of graduallyadding location information to that object representation, by jointly boostinglocalized histograms of codebook matches over all object categories.Cluster PrecisionWe follow the argumentation of [61] and base our evaluation on a mid-levelrepresentation of image features common to many computer vision techniques.In particular, we analyze the statistics of clusterings of feature descriptors.In order to quantify how well a clustering of feature descriptors reflects theseparation of object classes, we introduce a refinement to cluster precision [61].Intuitively, we want to measure to what extent features of a given class aregrouped together by clustering. High scores should be obtained by big clusterswith features from many instances of a single object class, and low scoresby small clusters with few features, but from multiple classes. In a secondexperiment, we evaluate the generalization capabilities of a given feature typeover an unseen test set. Features are extracted from an independent set ofimages, and matched against a codebook built from training images. For each6 http://www.robots.ox.ac.uk/˜vgg/research/affine/descriptors.html7 http://www.robots.ox.ac.uk/˜vgg/research/affine/detectors.html8 http://www.robots.ox.ac.uk/˜timork/salscale.html150 Fritz et al.test feature belonging to class a, we determine the precision PCa over allmatched clusters with respect to class a. The average precision over all classesis called matching precision.Results. We measure cluster precision for 9 different compression ratios(#Features/#Clusters) over codebooks generated from 200 training images(see Figures 4.6 (a) to (c)). We observe that cluster precision changes substan-tially if we vary detectors for a given descriptor, while it remains relativelystable over varying descriptors for a given detector. Appearance-based (SIFT,GLOH) and shape-based descriptors (GB, SC) do not show great differencesin cluster precision. Both perform comparably well. k-AS are worst, but stillcomparable to DoG-SIFT. We report that cluster precision results transfer tolarge extents to matching precision over previously unseen data, but omit theplots for brevity.Näıve BayesThe second level of our evaluation represents objects in terms of occurrencestatistics (counts of nearest-neighbor matches) over codebook entries andtrains a multi-class-classifier on a training set of such representations. We usean analogous approach to Multinomial Näıve Bayes [62] for text classifica-tion, and model the posterior distribution of an object class, given occurrencestatistics over a codebook, as a multinomial distribution.Results. We measure classification accuracy over fixed numbers of clustersfrom n = 50 to 1600, increasing by powers of 2 (see Figures 4.6 (d) to (f)).Dependent on the detector, n = 1600 corresponds to compression ratios of9 (HarLap), 52 (HesLap), 21 (SalReg), 5 (DoG), 5 (2-AS) and 16 (3-AS).For each feature type, we train a Näıve Bayes classifier on a training set of200 images (20 per category), and test on an independent test set of 100images (10 per category). The differentiation between descriptors with fixeddetectors is more pronounced for Näıve Bayes than for cluster precision. Inparticular, appearance-based features lead on average. GB and SC performon a comparable level to SIFT and GLOH, but only for individual detectors(SalReg for GB, HesLap for SC). k-AS exhibit relatively weak discriminativepower for Näıve Bayes classification. SC offers a good compromise betweenstrong (GB) and weak (k-AS) discrimination.Localized Bag-of-WordsWe measure the impact of adding location information in terms of classifica-tion accuracy in a Joint Boosting [63] framework. The object representationon the third level of our evaluation is based on histograms of feature occur-rences over a codebook, and inspired by [56]. We divide a rectangular imageregion into a grid of cells. For each cell, a local histogram over soft-matchedcodebook entries is computed, and concatenated to form the object represen-tation. A Joint Boosting algorithm is trained from object representations ofa set of training images using a fixed number of boosting rounds, and tested4 Categorical Perception 151against an independent set of test images. We use decision stumps [63] overhistogram bins as weak classifiers for boosting. By varying the number of gridcells g, we regulate the tradeoff between rich feature statistics (small g) andmore accurate localization (large g).Results. We measure classification accuracy as for Näıve Bayes for varyingnumbers of grid cells g ∈ {1, 4, 9}. We assume known bounding boxes for train-ing and test, and use them to anchor histogram grids. We fix the number ofclusters to n = 200, and obtain histograms of length g×n ∈ {200, 800, 1800}.While for TUD-Shape, boosting over localized bag-of-words lifts the discrim-inant power of all feature types to a comparable level for g = 9, shape-basedfeatures win on TUD-Shape2 (see Figures 4.6 (g) to (i)): HesLap-GB androtation invariant 2-AS are best (42% respectively 44% accuracy). The bestperforming SIFT and GLOH combinations obtain 33% (HesLap-SIFT) and31% (HesLap-GLOH). HarLap-SC obtains 35%.4 6 8 10 12 14 16 18 2000.10.20.30.40.50.60.70.80.91Compression ratioCluster precision Harris−Laplace + SIFTHessian−Laplace + SIFTSalient Regions + SIFTDoG + SIFTHessian−Laplace + GLOHHessian−Laplace + Shape Context4 6 8 10 12 14 16 18 2000.10.20.30.40.50.60.70.80.91Compression ratioCluster precision Harris−Laplace + Geometric BlurHessian−Laplace + Geometric BlurSalient Regions + Geometric Blur4 6 8 10 12 14 16 18 2000.10.20.30.40.50.60.70.80.91Compression ratioCluster precision 2AS2AS, rotation invariant3AS3AS, rotation invariant(a) Cluster precision, TUD-Shape, SIFT (b) Cluster precision, TUD-Shape, GB (c)Cluster precision, TUD-Shape, k-AS200 400 600 800 1000 1200 1400 160000.10.20.30.40.50.60.70.80.91# ClustersClassification accuracy Harris−Laplace + SIFTHessian−Laplace + SIFTSalient Regions + SIFTDoG + SIFTHessian−Laplace + GLOHHessian−Laplace + Shape Context200 400 600 800 1000 1200 1400 160000.10.20.30.40.50.60.70.80.91# ClustersClassification accuracy Harris−Laplace + Geometric BlurHessian−Laplace + Geometric BlurSalient Regions + Geometric Blur200 400 600 800 1000 1200 1400 160000.10.20.30.40.50.60.70.80.91# ClustersClassification accuracy 2AS2AS, rotation invariant3AS3AS, rotation invariant(d) Näıve Bayes, TUD-Shape, SIFT (e) Näıve Bayes, TUD-Shape, GB (f) Näıve Bayes, TUD-Shape, k-AS1 2 3 4 5 6 7 8 900.10.20.30.40.50.60.70.80.91# Grid cellsClassification accuracy Harris−Laplace + SIFTHessian−Laplace + SIFTSalient Regions + SIFTDoG + SIFTHessian−Laplace + GLOHHarris−Laplace + Shape Context1 2 3 4 5 6 7 8 900.10.20.30.40.50.60.70.80.91# Grid cellsClassification accuracy Harris−Laplace + Geometric BlurHessian−Laplace + Geometric BlurSalient Regions + Geometric Blur1 2 3 4 5 6 7 8 900.10.20.30.40.50.60.70.80.91# Grid cellsClassification accuracy 2AS2AS, rotation invariant3AS3AS, rotation invariant(g) Localized BOW, TUD-Shape2, SIFT (h) Localized BOW, TUD-Shape2, GB (i) Localized BOW, TUD-Shape2, k-ASFig. 4.6. First row: cluster precision, second row: näıve Bayes classification accuracy,third row: localized BOW classification accuracy for SIFT, GB, and k-AS. SIFT-plotsinclude the best GLOH and SC curves.152 Fritz et al.DiscussionLocal shape- and appearance-based features do not show great differencesin terms of feature statistics over our shape-based data set. On average, thechoice of detector is more important on average than the choice of descrip-tor. Hessian-Laplace with SIFT and GLOH is best on average. Shape-basedfeatures (Geometric Blur, k-Adjacent Segments) perform mostly worse thanappearance-based ones for classification based on simple occurrence statistics.In particular, k-AS capture generic local shape properties rather than discrim-inant information for an object category. They hence benefit more from addedlocation information than appearance-based features, and can even overtakeappearance-based features on shape-based data. We make use of this resultin Section 7.5, where we demonstrate how to incorporate k-AS features into aframework for the detection of functional object categories based on learnedaffordance cues.4.3 Mid-Level Representation & Detection4.3.1 Towards Adaptive RepresentationsThe main focus of this section is therefore a new object representation thataims to combine the above mentioned properties to make a step towards moreflexible and adaptive object representations applicable to a wide range of ob-jects and suited both for unsupervised as well as supervised learning. There-fore, we propose a novel approach that allows to learn a low-dimensionalrepresentation of object classes by building a generative decomposition ofobjects. These learned decompositions of objects contain both local appear-ance features as well as global silhouette features shared across object classes.This generative model of objects is directly applicable to unsupervised learn-ing tasks such as visual object class discovery. Second, we combine the low-dimensional and generative decomposition of objects with a discriminativelearning framework to enable supervised training and competitive object classdetection. Third, we provide empirical evidence that shows the properties ofthe approach (local vs. global features, feature sharing, unsupervised vs. su-pervised learning) and compares the approach with the state-of-the-art. In-terestingly, the approach outperforms both unsupervised techniques as wellas supervised techniques on various tasks on common databases.4.3.2 Learning of Generative DecompositionsIn this section we describe our approach to decomposition of multiple vi-sual categories by combining dense gradient representations and topic mod-els. Starting from the image, we first present our data representation. Thenwe describe how we apply the topic model to this representation and provide4 Categorical Perception 153w684 . . . w692Fig. 4.7. Dense gradient histogram representation. The loupeshows the 9 possible edge orientation of the histogram bins thatare interpreted as words.visualizations and insights for the obtained model as well as a quantitativeevaluation on an unsupervised learning task. Inspired by [6], we compute gra-dients on each color channel of the images and use the maximum response toobtain a grid of histograms that overlays the image. Each histogram in thegrid has 9 orientation bins equally spaced from 0◦ to 180◦ to represent theunsigned gradient orientation. An example of such an encoding is visualizedin Figure 4.3.2. In each cell, the 9 possible edge orientations associated withthe orientation bins are displayed by short lines. The grayscale value encodesthe accumulated gradient magnitude in each bin. The size of the cells in thegrid is 8×8 pixels.As the following topic models operate on discrete word counts, we nor-malize the histograms to have a constant sum of discrete entries. We decidednot to compute a redundant coding like the blocks in the HOG descriptor [6]as we believe that the introduced non-linearities by local normalization wouldhinder the fitting of the probabilistic model.Topic ModelsTo define a generative process for our data representation, we employ proba-bilistic topic models [64, 65, 66] which were originally motivated in the con-text of text analysis. As it is common habit we adopt the terminology ofthis domain. In the following, a document d refers to a sequence of words(w1, w2, . . . , wNd), where each wi is one word occurrence. The underlying ideaof these models is to regard each document as a mixture of topics. This meansthat each word wi of the total Nd words in document d is generated by firstsampling a topic zi from a multinomial topic distribution P (z) and then sam-pling a word from a multinomial topic-word distribution P (w|z). Thereforethe word probabilities for the combined model are:P (wi) =T∑j=1P (wi|zi = j)P (zi = j) (4.1)where T is the number of topics and P (wi|zi = j) as well as P (zi = j) areunobserved . According to the notation of [66], we will abbreviate154 Fritz et al.DNdT! !(z) wz!(d)!Fig. 4.8. LDA model as formulated by [66].θ(d): topic distribution P (z) for document dφ(j): topic-word distribution P (wi|z = j) for topic jThe particular topic models differ on the one hand in which additional hy-perparameters/priors they introduce and on the other hand in how inferenceand parameter estimation is performed. We will discuss the Latent DirichletAllocation model [65] in some more detail focusing on the version presentedin [66] that uses Gibbs sampling for inference and estimation. The graphicalrepresentation of this model is depicted in Figure 4.8. It visualizes the processthat generates a total of D documents d, where each document has Nd words.Above we already described how each word wi of a particular document isgenerated. In the full model, there are 2 additional hyperparameters, α andβ, which place symmetric dirichlet priors on the topic distribution of each doc-ument θ(d) and the topic-word distributions φ(j) respectively. As the settingfor α and β is common to all documents, these act as forces that impose globaltendencies on these distributions. Intuitively, the prior α for the topic distri-bution θ favors co-activation (sharing) of multiple topics for each documentfor values larger than 1, whereas smaller values result in sparser topic distribu-tion - ultimately having single topics explaining whole documents (clustering).Consequently, the sparseness of the topic-word distribution φ(j) is affected bythis choice. The second parameter β, has a direct smoothing effect on thetopic distributions.For more details on the models, inference and estimation, we refer to [65]and [67]. The idea behind the employed Gibbs sampling procedure is that alltopic assignments zi are initialized (typically randomly) and then iterativelyupdated in a random order. To perform such a single update, a topic is drawnfrom the conditional distribution P (zi|Ω \ zi) and assigned to zi, where Ω \ zidenotes all observed and unobserved variables but zi. This is repeated for afixed number of iterations.4 Categorical Perception 155Fig. 4.9. First row: example topics that were learned by the proposed approachacross categories and viewpoints for the 10 classes of the PASCAL’06 data. Belowfirst row: training images that activated the topic above most. The topics modellocal structures, line segments as well as silhouette-like structures. The topics aredistinctive enough to separate several category members and even view-points. Onthe other hand they are general enough to be shared across categories and view-points.To extend our findings to the detection task that we are aiming for inSection 4.3.4, we extract our representation on the multi-category, multi-viewPASCAL’06 dataset [68], in order to obtain a decomposition that is sharedacross categories.In the first row of Figure 4.9 13 of 100 topic distributions are visualizedthat were trained on the bounding box annotations of the training and valida-tion data of the PASCAL’06 challenge. The rows below display the examplesthat activated this particular topic most. We observe that the topics capturedifferent levels of objects, ranging from global silhouettes (car rear in column10 and side view in column 13) over localized parts (legs in column 3, bicycleframe in column 8 and bicycle wheels in column 12) to line segments and cor-ners (corner in column 1 and line segments in column 2 and 4) . The modeldiscovers distinctive parts that even separate several examples of differentcategories and their viewpoints although no such information was availableto the system during training. Importantly, we can see that other topics likethose that got activated on legs are shared across several categories, which isa desirable property of a compact decomposition in order to be scalable [22].4.3.3 Generative/Discriminative Hybrid Model for DetectionBased on these promising results, we describes a complete system for super-vised multi-category detection that leverages the learned representation.156 Fritz et al.Combining Generative and Discriminant ModelsRecently, the combinations of generative approaches with discriminative oneshave shown to be very effective [23, 24]. The idea is that generative modelscan easily incorporate prior information to support learning from small sam-ples, have increased robustness to noise and generally have more principledways of dealing with missing data. Discriminative models on the other handhave shown to give superior performance for well posed learning tasks and asufficient number of training examples. We also follow this idea and comple-ment the generative topic model by a discriminative SVM classifier with anRBF kernel [69]. In particular we train an SVM to discriminate between thetopic distributions θ(d) which are inferred for images containing the categoryof interest and others that do not contain these. By doing so, we seek to profitfrom the above mentioned benefits of the generative model combined with thediscriminative classifier.Sliding Window Approach to DetectionAs proposed in [6] a sliding window approach can be done efficiently in thissetting if the sliding window is always shifted by exactly one cell in x or ydirection. In this case, the gradient histograms of the cell grid are computedonce and for each sliding window the relevant sub grid is used.4.3.4 Results on Visual Category DetectionWe evaluate our approach on the competition 3 of the PASCAL challenge2006 [68] that poses a challenging detection task as 10 visual categories are tobe detected from multiple viewpoints over a large scale range. Many imagesin training and test suffer from degradations due to occlusion, bad lightingconditions and show high variability in object appearance and backgroundclutter.We outperform all other competitors in the 3 categories bicycle, bus andcar by improving the state-of-the-art [68] on this dataset by 5.75%, 9.14%and 5.67% in average precision respectively. In particular we surpass the fullyglobal approach [6] that our method was motived by. Compared to [70] weimprove on bicycles and bus only by 0.65% and 0.93%, but again significantlyon cars with 8.87%. However, in contrast to [70] we do not use the viewpointannotations to train our approach. For the other categories, we perform aboutaverage, but also showed some inferior results on the highly articulated cate-gories. We are currently investigating means to make the approach less rigidand carry over the good results from the first 3 categories to the other ones.For a more detailed analysis in different settings that show the merits ofthis approach supervised as well as unsupervised setting in comparison tolocal, global and shape-based approaches we refer to [71].4 Categorical Perception 1570 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 100.10.20.30.40.50.60.70.80.91PASCAL challenge 2006Recall1 − Precision bicyclebuscarcatcowdoghorsemotorbikepersonsheepFig. 4.10. Results on the PASCAL VOC challenge 2006. Precision-Recall curvesand example detections.4.3.5 DiscussionThis section described an approach to learn generative decomposition of visualcategories from data that yield and effective representation for unsupervisedas well as supervised categorization tasks. We argue that the flexibility andadaptivity we obtain through this learning-based approach results in the ob-served performance improvements over traditional, hand-crafted feature rep-resentations.Additionally the learned decompositions add a certain structure to theobserved objects and puts object into relation to each other. This is highlyinteresting to obtain object part reference and share knowledge across theobserved object instances. In this sense, our model offers a high-level of intro-spection, that can be leveraged in tutor-based learning setting.To overcome the limitations due to the rigid nature of the underlyingcoding, we currently investigate how to augment the approach with a defor-mation model to add more flexibility. However, it is yet unclear how to obtaina beneficial tradeoff between invariance and discriptiveness that results in animproved performance of the system. Progress in this direction would enablethe system to generalize across more complex deviations in shape and appear-ance.4.4 High-Level Representations and Dynamic ModelsAn important type of object classes is articulated such as people and ani-mals. We study the issues associated to modeling of these articulated objectsclasses by addressing the task of tracking people in cluttered environments. Toachieve reliable extraction of people-tracks as well as data-association acrosslong periods of occlusion, the proposed approach combines recent advancesin people detection with the power of dynamical models for tracking. Ratherthan to simply determine the position and scale of a person as is common forstate-of-the-art people detectors [6, 72], we also extract the position and artic-ulation of the limbs. This allows us to use a more powerful dynamical model158 Fritz et al.that extends people detection to the problem of reliably extracting people-tracklets – people detections consistent over a small number of frames. Inparticular, we use a hierarchical Gaussian process latent variable model (hG-PLVM) [73] to model the dynamics of the individual limbs. As we will show inthe experiments this enables us to detect people more reliably than it wouldbe possible from single frames alone. We combine this with a hidden Markovmodel (HMM) that allows to extend the people-tracklets, which cover onlya small number of frames at a time, to possibly longer people-tracks. Thesepeople-tracks identify individuals over longer sequences of consecutive frameswhen that is appropriate, such as between major occlusion events. Trackingpeople over even longer periods of time is then achieved by associating people-tracks across potentially long periods of occlusion using both the dynamicalmodel and an extracted appearance model, which allows identifying specificindividuals throughout the sequence.4.4.1 Appearance model for single-frame detection and poseestimationFollowing the general idea of part-based object detection, we formulate theproblem of locating an object from a specific class in a test image as searchof the modes of the posterior probability distribution p(L|E) of the objectconfiguration L given the image evidence E [74].In our model, the configuration is described as L = {xo,x1, . . . ,xN}, wherexo is the position of the object center and its scale, and xi is the position andscale of part i. The image evidence, which here is defined as a set of localfeatures observed in the test image, will be denoted as E = {eappk , eposk |k =1, . . . ,K}, where eappk is an appearance descriptor, and eposk is the position andscale of the local image feature with index k. We will denote the combinationof position, scale, and appearance of a local feature as ek = (eappk , eposk ).Assuming that positions of object parts are independent of each othergiven person’s position xo and articulation a, it follows thatp(L|a,E) ≈ p(xo)∏ip(xi|a,E)p(xi|xo, a). (4.2)If we assume that a particular image feature ek belongs to part i of an objectinstance in the image with probability α, then it holds thatp(xi|a,E) = c0 + c1∑ekp(xi|a, ek) +O(α2), (4.3)where c0 and c1 depend only on the image features E [75]. If α is sufficientlysmall, which is true for street scenes in which a particular person usuallyrepresents only a small portion of the image, we obtainp(L|a,E) ≈∏ip(xi|xo, a)[β +∑ekp(xi|a, ek)], (4.4)4 Categorical Perception 159where β can be seen as a regularizer for the evidence obtained from the indi-vidual image features, and we have additionally assumed uniform p(xo).As is common in models based on local feature representations, we intro-duce an object-specific codebook denoted as C = {cj |j = 1, . . . , J}. The partposterior with respect to a single image feature is computed by marginaliza-tion over the codebook entries:p(xi|a, ek) =∑cjp(xi|a, cj , eposk )p(cj |eappk ). (4.5)p(cj |eappk ) is discrete distribution over codebooks based on a Gaussiansimilarity measure, and p(xi|a, cj , eposk ) is learned from training data.4.4.2 Representing the dynamics of the human walking cycles withlatent variable modelPeople-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008 8YConfiguration!200 !150 !100 !50 0 50 100050100150200250yiY = [yi ! RD]Latent space Z Z = [zi ! Rq]Time (frame #) T T = [ti ! R], q ! DFig. 4.11. Representation of articulations in the latent space.In order to integrate the evidence from several subsequence frames, it isnecessary to incorporate prior knowledge about plausible human motions intothe recognition framework. Such prior knowledge can be expressed in the formof prior distribution on the sequences of possible configurations of the person.Modelling such prior directly is difficult due to the high dimensionality ofthe space of pose sequences. Instead, several authors [76, 77, 78] have arguedand shown that a low-dimensional representation is sufficient to approximatethe pose dynamics. In the following we describe a Gaussian process latentvariable model (GPLVM) which we use to obtain such a low-dimensionalrepresentation and discuss how it can be applied to reliable people detectionsin image sequences.Let Y = [y1,y2, . . . ,ym]T be a sequence of D-dimensional observations(here describing the relative joint angles of body limbs). GPLVMs model theD-dimensional observation space as the output of D Gaussian processes withan input space of dimensionality q, where q < D. Each observation yi is160 Fritz et al.associated with a q-dimensional latent point zi. The likelihood of the obser-vation sequence Y given the latent sequence Z = [z1, z2, . . . , zm]T and modelparameters θ is given by [79]:p(Y|Z, θ) =D∏i=1N (Y:,i|0,Kz), (4.6)where Y:,i is the vector of values of feature i across all observations, and Kz isthe covariance matrix with elements given by a covariance function k(zi, zj).In this paper we use a squared exponential covariance function augmented byGaussian white noise.For a given Y we can find the positions of the latent points Z along withthe model parameters θ by maximizing their likelihood from Eq. (4.6).In addition to the low-dimensional latent representation of the data,GPLVMs provide a probabilistic mapping from the latent space to the ob-servation space. One possibility to define a dynamic model in the latent spaceis to place a suitable prior on the elements of Z. Such a prior can be givenby a Gaussian process with time as input variable [73]. The structure ofthis prior is shown on the Fig. 4.11. Given the sequence of points in time,T = [t1, t2, . . . , tm]T at which the observations Y were made, the prior overZ is given byp(Z|T) =q∏i=1N (Z:,i|0,KT) (4.7)where KT is the covariance matrix of the time points. The covariance functionin the time space can again be taken as squared exponential, which ensuressmoothness of the trajectories.We now combine this prior with the likelihood from Eq. (4.6), and maxi-mize w.r.t. Z and θ. Fig. 4.11 shows the 2 dimensional latent space obtainedby applying this model to 11 walking sequences of different subjects, eachcontaining one complete walking cycle. Walking cycles in each sequence aremanually aligned so that we can interpret the frame number in each sequenceas phase of the walking cycle. This hierarchical approach to GPLVM dynam-ics has several advantages over the auto-regressive prior proposed in [77]. Inparticular, it allows us to evaluate the likelihood of a sequence of poses, even ifthe poses occurred at unequally spaced time intervals. This arises, e.g ., whenthe subject was occluded or not detected for several frames. Additionally, fora given pose the model allows us to hypothesize both successive and previousposes, which we use to produce good initial hypotheses for the whole imagesequence from a few good detections.4.4.3 Robust detection and tracking of people in image sequencesThe person detector which we have described in Sec. 4.4.1 provides hypothe-ses for position, scale, and body articulation in single frames based on the4 Categorical Perception 161Fig. 4.12. Detection and tracking of multiple people on “TUD-Crossing” dataset.detection of individual body parts or limbs. To further improve the detec-tion performance in image sequences it is desirable to incorporate temporalconsistency among subsequent frames. In contrast to frequently used motionmodel based on position and velocity of the person, we propose to use a moreexpressive kinematic limb model thereby leveraging the articulated trackingliterature (e.g . [80, 81, 82, 76, 83, 78]).Given the image evidence E = [E1, E2, . . . , Em]T in a sequence of m subse-quent frames, we would like to recover the positions Xo∗ = [xo∗1 ,xo∗2 , . . . ,xo∗m ]Tof the person as well as the configurations of the limbs in each frameY∗ = [y∗1,y∗2, . . . ,y∗m]T with y∗j denoting the recovered limb orientations inthe j-th frame. Assuming independence of the detections in each frame, theposterior factorizes as:p(Y∗,Xo∗|E) ∝ p(Y∗)p(Xo∗)p(E|Y∗,Xo∗) (4.8)∝ p(Y∗)p(Xo∗)m∏j=1p(Ej |y∗j ,xo∗j ).p(Ej |y∗j ,xo∗j ) is the likelihood of the image evidence Ej , and is given by thedetection model described in the previous section. p(Xo∗) corresponds to aprior of human body speed, which we model as a broad Gaussian and p(Y∗)is given by the Gaussian process latent variable model.Given limb likelihoods and the hGPLVM prior, we can maximize Eq. (4.8)to find the best pose sequence. This is equivalent to jointly solving the inversekinematics in each frame of the sequence under soft constraints given by limblikelihoods and is similar to [84], except that in our case hints about limbpositions are provided by a person detector instead of being manually givenby the user. If we denote the training observations, their latent representationand model parameters by M = [Y,T,Z,θ], the probability of the unknownpose sequence Y∗, its latent representation Z∗, and the person positions Xo∗162 Fritz et al.is given byp(Y∗,Xo∗,Z∗|M, E,T∗) ∝ (4.9)p(E|Y∗,Xo∗)p(Y∗|Z∗,M)p(Z∗|T∗,M)p(Xo∗).The first term is the detection likelihood from single-frame detections (seeEq. (4.8)). The second term is given byp(Y∗|Z∗,M) =D∏i=1p(Y∗:,i|Z∗,Y:,i,Z), (4.10)where p(Y∗:,i|Z∗,Y:,i,Z) is a Gaussian process prediction of the pose sequencegiven a sequence of latent positions. The third term is given by the dynamicsprior on the latent space:p(Z∗|T∗,M) =q∏i=1p(Z∗:,i|T∗,Z:,i,T). (4.11)In our formulation, detecting people in a series of m frames therefore corre-sponds to finding pose sequences Y∗ and people positions Xo∗ that maximizeEq. (4.9). The gradients of the second and third terms in 4.9 can be com-puted analytically, while we use a finite difference approximation for gradientof p(Y∗|F ). Subsequently the local maxima can be found using any of thestandard non-linear optimization methods using positions and poses of theperson estimated by the single-frame detector as initialization.Given people hypothesis jointly computed using evidence accumulated overseveral subsequent frames we can compute longer people tracks by consideringthese hypothesis as states in the hidden Markov model and computing theiroptimal sequence with Viterbi algorithm (see [85] for details). Fig. 4.12 showsseveral frames from a street sequence with multiple people, in which our sys-tem can detect and track most of the people in spite of reoccuring parial andfull occlusions.4.5 Outlook & DiscussionWe have argued that that no single combination of these paradigms is pow-erful and versatile enough for the general problem of object categorization.It is commonly believed that relying on a single paradigm of addressing thecategory phenomenon won’t do justice to the overwhelming diversity encoun-tered in categorial perception. In the previous sections we have motivated anddemonstrated the merits of learning-based approaches that integrate differentparadigms of representing visual categories.A hierarchical approach has been presented that learns a representationthat aggregates evidence on multiple levels. Local features have been evalu-ated to reach beyond purely visual categories. An approach to learning gener-ative decompositions was developed to bridge across different representations4 Categorical Perception 163paradigms from local to global. Finally, a even more high-level view on thedata was proposed that manages to address and represented dynamics of ob-jects like pedestrians in a learning based manner.Despite the integration of different views and approaches to the phenomenaof visual categories, we are aware of the limitations, that still have to beovercome. Although the shown approach manage to add flexibility and rangeover larger portion of the design space than previous approaches did, theystill address particular limitations of previous approaches and yet have notintegrated all our lessons learnt into one single approach.A common theme in our approaches is to fit statistical models to the ob-served data, which use little to no supervision to reconstruct the unobservedhidden structure in the data. This leads to higher-level representations whichwe successfully use to solve more complex tasks such as multi-viewpoint mod-eling and temporal modeling. While hierarchical statistical models have beenused for a long time, many of your results are founded on the more recentsuccess of robustly learning such complex model with minimal intervention.To sum up, we want to emphasize the importance these learning-basedapproaches have for moving towards a larger number of addressable cate-gories. We contributed to achieving the desired adaptivity and flexibility, webelieve to be a key ingredient for scalable vision system for object categoriza-tion. Although there are still unresolved computational as well as conceptionalchallenges, we are confident that the shown improvements on learning of hi-erarchies, features and dynamics will help us to reach this goal.Ultimately, the way categories are stored, organized and related to eachother determines the capabilities to learn, evolve and therefore to interact ofthe overall system. In Chapter 7 on multi-modal learning and in particular inour integration efforts in Chapter 9 and 10, we elaborate on how representationaffects the capabilities of the overall system.References1. M. C. Burl, P. Perona, Recognition of planar object classes, in: IEEE Confer-ence on Computer Vision and Pattern Recognition (CVPR’96), IEEE ComputerSociety, San Francisco, CA, USA, 1996, p. 223.2. R. Fergus, P. Perona, A. Zisserman, Object class recognition by unsupervisedscale-invariant learning, in: CVPR’03, pp. 264–271.3. B. Leibe, E. Seemann, B. Schiele, Pedestrian detection in crowded scenes, in:IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05) [86].URL http://cognitivesystems.org/cosybook/chap4.asp#leibe05cvpr4. M. Varma, D. Ray, Learning the discriminative power-invariance trade-off, in:IEEE International Conference on Computer Vision (ICCV’07), IEEE Com-puter Society, Rio de Janeiro, Brazil, 2007.5. D. G. Lowe, Distinctive image features from scale-invariant keypoints, IJCV60 (2) (2004) 91–110.6. N. Dalal, B. Triggs, Histograms of oriented gradients for human detection, in:IEEE International Conference on Computer Vision (ICCV’05) [87].164 Fritz et al.7. B. Leibe, A. Leonardis, B. Schiele, Robust object detection with interleaved cat-egorization and segmentation, International Journal of Computer Vision (IJCV)77 (1-3) (2008) 259–289.URL http://cognitivesystems.org/cosybook/chap4.asp#Leibe05c8. P. Viola, M. Jones, Rapid object detection using a boosted cascade of simplefeatures, in: IEEE Conference on Computer Vision and Pattern Recognition(CVPR’01), IEEE Computer Society, Kauai, HI, USA, 2001, pp. 511–518.9. J. M. Winn, N. Jojic, Locus: Learning object classes with unsupervised segmen-tation, in: IEEE International Conference on Computer Vision (ICCV’05) [87],pp. 756–763.10. M. Weber, M. Welling, P. Perona, Unsupervised learning of object models forrecognition, in: D. Vernon (Ed.), European Conference on Computer Vision(ECCV’00), Vol. 1843 of Lecture Notes in Computer Science, Springer, Dublin,Ireland, 2000.11. J. Sivic, B. C. Russell, A. A. Efros, A. Zisserman, W. T. Freeman, Discoveringobjects and their locations in images, in: IEEE International Conference onComputer Vision (ICCV’05) [87].12. R. Fergus, L. Fei-Fei, P. Perona, A. Zisserman, Learning object categories fromgoogle’s image search, in: IEEE International Conference on Computer Vision(ICCV’05) [87].13. G. J. Ettinger, Hierarchical object recognition using libraries of parameterizedmodel sub-parts, Tech. rep., MIT (1987).14. J. K. Tsotsos, Analyzing vision at the complexity level, Behavioral and BrainSciences 13 (3) (1990) 423–469.15. B. W. Mel, J. Fiser, Minimizing binding errors using learned conjunctive fea-tures, Neural Computation 12 (4) (2000) 731–762.16. Y. Amit, D. Geman, A computational model for visual selection., Neural Comp.11 (7) (1999) 1691–1715.17. Y. Amit, 2d Object Detection and Recognition: Models, Algorithms and Net-works, MIT Press, Cambridge, 2002.18. S. Geman, D. Potter, Z. Chi, Composition systems., Quarterly of Applied Math-ematics 60 (4) (2002) 707–736.19. S. Fidler, G. Berginc, A. Leonardis, Hierarchical statistical learning of genericparts of object structure., in: CVPR, 2006, pp. 182–189.URL http://cognitivesystems.org/cosybook/chap4.asp#s:fidler0620. S. Fidler, A. Leonardis, Towards scalable representations of visual categories:Learning a hierarchy of parts., in: CVPR’07, 2007.URL http://cognitivesystems.org/cosybook/chap4.asp#s:fidler0721. K. Mikolajczyk, B. Leibe, B. Schiele, Multiple object class detection with a gen-erative model, in: IEEE Conference on Computer Vision and Pattern Recogni-tion (CVPR’06) [88], pp. 26–36.URL http://cognitivesystems.org/cosybook/chap4.asp#Mikolajczyk06c22. A. Torralba, K. P. Murphy, W. T. Freeman, Sharing visual features for multiclassand multiview object detection, IEEE Trans. Pattern Analysis and MachineIntelligence 29 (5).23. T. S. Jaakkola, D. Haussler, Exploiting generative models in discriminative clas-sifiers, in: Proceedings of the 1998 conference on Advances in neural informationprocessing systems II, MIT Press, Cambridge, MA, USA, 1999, pp. 487–493.4 Categorical Perception 16524. M. Fritz, B. Leibe, B. Caputo, B. Schiele, Integrating representative and discrim-inant models for object category detection, in: IEEE International Conferenceon Computer Vision (ICCV’05) [87].URL http://cognitivesystems.org/cosybook/chap4.asp#Fritz0525. E. Sudderth, A. Torralba, W. Freeman, A. Willsky, Learning hierarchical modelsof scenes, objects, and parts., in: ICCV’05, 2005, pp. 1331–1338.26. B. Ommer, J. M. Buhmann, Learning the compositional nature of visual ob-jects., in: CVPR’07, 2007.27. F. Fleuret, D. Geman, Coarse-to-fine face detection., IJCV 41 (1/2) (2001) 85–107.28. K. Fukushima, S. Miyake, T. Ito, Neocognitron: a neural network model for amechanism of visual pattern recognition., IEEE Systems, Man and Cybernetics13 (3) (1983) 826–834.29. T. Serre, L. Wolf, S. Bileschi, M. Riesenhuber, T. Poggio, Object recognitionwith cortex-like mechanisms., PAMI 29 (3) (2007) 411–426.30. F. Scalzo, J. H. Piater, Statistical learning of visual feature hierarchies, in:Workshop on Learning, CVPR, 2005”.31. S. Ullman, B. Epshtein, Visual Classification by a Hierarchy of Extended Fea-tures., Towards Category-Level Object Recognition, Springer-Verlag, 2006.32. A. Agarwal, B. Triggs, Hyperfeatures - multilevel local coding for visual recog-nition, in: ECCV (1), 2006, pp. 30–43.33. M. A. Ranzato, F.-J. H., Y.-L. Boureau, Y. LeCun, Unsupervised learning of in-variant feature hierarchies with applications to object recognition., in: CVPR’07,2007.34. E. Bienenstock, S. Geman, Compositionality in neural systems., in: M. Arbib(Ed.), The Handbook of Brain Theory and Neural Networks, MIT Press, 1995,pp. 223–226.35. S. Zhu, D. Mumford, Quest for a stochastic grammar of images., Foundationsand Trends in Computer Graphics and Vision 2 (4) (2007) 259–362.36. A. Califano, R. Mohan, Multidimensional indexing for recognizing visualshapes., Pattern Analysis and Machine Intelligence 16 (4) (1994) 373–392.37. K. Tsunoda, Y. Yamane, M. Nishizaki, M. Tanifuji, Complex objects are repre-sented in macaque inferotemporal cortex by the combination of feature columns.,Nature Neuroscience (4) (2001) 832–838.38. S. Brincat, C. Connor, Dynamic shape synthesis in posterior inferotemporalcortex., Neuron 49 (1) (2006) 17–24.39. H. B. Barlow, Cerebral cortex as a model builder., in: D. Rose, V. Dobson (Eds.),Models of the Visual Cortex, John Wiley: Chichester, 1985, pp. 37–46.40. E. T. Rolls, G. Deco, Computational Neuroscience of Vision., Oxford Univ.Press, 2002.41. S. Edelman, N. Intrator, Towards structural systematicity in distributed, stati-cally bound visual representations, Cognitive Science 27 (2003) 73–110.42. L. Fei-Fei, R. Fergus, P. Perona, Learning generative visual models from fewtraining examples: an incremental bayesian approach tested on 101 object cate-gories., in: IEEE CVPR’04, Workshop on Generative-Model Based Vision, 2004.43. J. Mutch, D. G. Lowe, Multiclass object recognition with sparse, localized fea-tures., in: CVPR06, 2006, pp. 11–18.44. L. Wolf, S. Bileschi, E. Meyers, Perception strategies in hierarchical vision sys-tems, in: CVPR’06, 2006, pp. 2153–2160.166 Fritz et al.45. G. Csurka, C. Dance, L. Fan, J. Willarnowski, C. Bray, Visual categorizationwith bags of keypoints, in: SLCV, 2004.46. J. L. Mundy, Object recognition in the geometric era: A retrospective., in: To-ward Category-Level Object Recognition, 2006, pp. 3–28.47. V. Ferrari, L. Fevrier, F. Jurie, C. Schmid, Groups of adjacent contour segmentsfor object detection, Rapport De Recherche Inria.48. A. Opelt, A. Pinz, A. Zisserman, A boundary-fragment-model for object detec-tion, in: ECCV, 2006, pp. II: 575–588.49. M. Stark, B. Schiele, How good are local features for classes of geometric objects,in: ICCV, 2007.URL http://cognitivesystems.org/CoSyBook/chap4.asp#stark07iccv50. A. C. Berg, J. Malik, Geometric blur for template matching, in: CVPR, 2001,pp. 607–614.51. S. Belongie, J. Malik, J. Puzicha, Shape context: A new descriptor for shapematching and object recognition, in: NIPS, 2000, pp. 831–837.52. K. Mikolajczyk, C. Schmid, A performance evaluation of local descriptors.,PAMI 27 (10) (2005) 1615–1630.53. K. Mikolajczyk, C. Schmid, Scale & affine invariant interest point detectors,IJCV 60 (1) (2004) 63–86.54. K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffal-itzky, T. Kadir, L. J. V. Gool, A comparison of affine region detectors, IJCV65 (1-2) (2005) 43–72.55. T. Kadir, A. Zisserman, M. Brady, An affine invariant salient region detector,in: ECCV’04.56. V. Ferrari, T. Tuytelaars, L. J. V. Gool, Object detection by contour segmentnetworks, in: ECCV’06.57. D. R. Martin, C. Fowlkes, J. Malik, Learning to detect natural image boundariesusing local brightness, color, and texture cues, PAMI 26 (5) (2004) 530–549.58. A. C. Berg, T. L. Berg, J. Malik, Shape matching and object recognition usinglow distortion correspondences, in: CVPR’05.59. M. Morrone, D. Burr, Feature detection in human vision: a phase dependentenergy model, Proc. Royal Soc. London Bulletin (1988) 221–245.60. C. Harris, M. J. Stephens, A combined corner and edge detector, in: AlveyConference, 1988, pp. 147–152.61. K. Mikolajczyk, B. Leibe, B. Schiele, Local features for object class recognition,in: ICCV [87], pp. 1792–1799.URL http://cognitivesystems.org/cosybook/chap4.asp#conf/iccv/MikolajczykLS0562. A. McCallum, K. Nigam, A comparison of event models for naive bayes textclassification, in: AAAI, Workshop on Learning for Text Categorization, 1998.63. A. Torralba, K. Murphy, W. Freeman, Sharing features: efficient boosting pro-cedures for multiclass object detection, in: CVPR, 2004.64. T. Hofmann, Unsupervised learning by probabilistic latent semantic analysis,Machine Learning.65. D. M. Blei, A. Y. Ng, M. I. Jordan, Latent dirichlet allocation, Journal ofMachine Learning Research.66. T. L. Griffiths, M. Steyvers, Finding scientific topics., PNAS USA.67. M. Steyvers, T. L. Griffiths, Probabilistic topic models, in: Handbook of LatentSemantic Analysis, Lawrence Erlbaum Associates, 2007.4 Categorical Perception 16768. M. Everingham, A. Zisserman, C. K. I. Williams, L. Van Gool, The PASCALVisual Object Classes Challenge 2006 (VOC2006) Results, http://www.pascal-network.org/challenges/VOC/voc2006/results.pdf (2006).69. C.-C. Chang, C.-J. Lin, LIBSVM: a library for support vector machines, softwareavailable at http://www.csie.ntu.edu.tw/~cjlin/libsvm (2001).70. O. Chum, A. Zisserman, An exemplar model for learning object classes, in:IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07),IEEE Computer Society, Minneapolis, Minnesota, USA, 2007.71. M. Fritz, B. Schiele, Decomposition, discovery and detection of visual cate-gories using topic models, in: IEEE Conference on Computer Vision and PatternRecognition (CVPR’08) [89], to appear.URL http://cognitivesystems.org/cosybook/chap4.asp#fritz08cvpr72. B. Leibe, E. Seemann, B. Schiele, Pedestrian detection in crowded scenes, in:IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05) [86].URL http://cognitivesystems.org/cosybook/chap4.asp#leibe05cvpr73. N. D. Lawrence, A. J. Moore, Hierarchical Gaussian process latent variablemodels, in: ICML’07, 2007.74. P. F. Felzenszwalb, D. P. Huttenlocher, Pictorial structures for object recogni-tion, IJCV 61 (2007) 55–79.75. C. K. I. Williams, M. Allan, On a connection between object localization with agenerative template of features and pose-space prediction methods, Tech. Rep.EDI-INF-RR-0719, University of Edinburgh (2006).76. R. Urtasun, D. J. Fleet, P. Fua, 3D people tracking with Gaussian process dy-namical models, in: IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR’06) [88].77. J. M. Wang, D. J. Fleet, A. Hertzmann, Gaussian process dynamical models,in: NIPS’05, 2005.78. C. Sminchisescu, A. Kanaujia, D. N. Metaxas, BM3E: Discriminative densitypropagation for visual tracking, PAMI 29 (2007) 2030–2044.79. N. D. Lawrence, Probabilistic non-linear principal component analysis withGaussian process latent variable models, JMLR 6 (2005) 1783–1816.80. J. Deutscher, I. Reid, Articulated body motion capture by stochastic search,IJCV 61 (2005) 185–205.81. D. Demirdjian, L. Taycher, G. Shakhnarovich, K. Grauman, T. Darrell, Avoid-ing the ”streetlight effect”: Tracking by exploring likelihood modes, in: IEEEInternational Conference on Computer Vision (ICCV’05) [87].82. L. Sigal, M. J. Black, Measure locally, reason globally: Occlusion-sensitive artic-ulated pose estimation, in: IEEE Conference on Computer Vision and PatternRecognition (CVPR’06) [88].83. D. Ramanan, D. A. Forsyth, A. Zisserman, Tracking people by learning theirappearance, PAMI 29 (2007) 65–81.84. K. Grochow, S. L. Martin, A. Hertzmann, Z. Popovic, Style-based inverse kine-matics, in: SIGGRAPH, 2004.85. M. Andriluka, S. Roth, B. Schiele, People-tracking-by-detection and people-detection-by-tracking, in: IEEE Conference on Computer Vision and PatternRecognition (CVPR’08) [89], to appear.86. IEEE Computer Society, San Diego, CA, USA, 2005.87. IEEE Computer Society, Beijing, China, 2005.88. IEEE Computer Society, New York, NY, USA, 2006.89. IEEE Computer Society, Anchorage, Alaska, USA, 2008, to appear.5Semantic Modelling of SpaceAndrzej Pronobis1, Patric Jensfelt1, Kristoffer Sjöö1, HendrikZender2, Geert-Jan M. Kruijff2, Oscar Martinez Mozos3, WolframBurgard31 Royal Institute of Technology (KTH), Centre for Autonomous Systems,Stockholm, Sweden {pronobis, patric, krsj}@csc.kth.se2 DFKI GmbH, Saarbrücken, Germany {zender, gj}@dfki.de3 Albert-Ludwigs-Universität Freiburg, Department of Computer Science,Freiburg, Germany {omartine, burgard}@informatik.uni-freiburg.de5.1 IntroductionA cornerstone for robotic assistants is their understanding of the space theyare to be operating in: an environment built by people for people to live andwork in. The research questions we are interested in in this chapter concernspatial understanding, and its connection to acting and interacting in indoorenvironments. Comparing the way robots typically perceive and represent theworld with findings from cognitive psychology about how humans do it, it isevident that there is a large discrepancy. If robots are to understand humansand vice versa, robots need to make use of the same concepts to refer to thingsand phenomena as a person would do. Bridging the gap between human androbot spatial representations is thus of paramount importance.A spatial knowledge representation for robotic assistants must addressthe issues of human-robot communication. However, it must also provide abasis for spatial reasoning and efficient planning. Finally, it must ensure safeand reliable navigation control. Only then can robots be deployed in semi-structured environments, such as offices, where they have to interact withhumans in everyday situations.In order to meet the aforementioned requirements, i.e. robust robot con-trol and human-like conceptualization, in CoSy, we adopted a spatial repre-sentation that contains maps at different levels of abstraction. This stepwiseabstraction from raw sensory input not only produces maps that are suitablefor reliable robot navigation, but also yields a level of representation that issimilar to a human conceptualization of spatial organization. Furthermore,this model provides a richer semantic view of an environment that permitsthe robot to do spatial categorization rather than only instantiation.This approach is at the heart of the Explorer demonstrator (cf. Chap-ter 10), which is a mobile robot capable of creating a conceptual spatial map170 Pronobis et al.of an indoor environment. In the present chapter, we describe how we usemulti-modal sensory input provided by a laser range finder and a camera inorder to build more and more abstract spatial representations.5.1.1 Related WorkResearch in spatial representations for mobile robots has yielded differentmulti-layered environment models. Vasudevan et al. [1] suggest a hierarchicalprobabilistic representation of space based on objects. The work by Galindo etal. [2] presents an approach containing two parallel hierarchies, spatial andconceptual, connected through anchoring. Inference about places is based onobjects found in them. Furthermore, the Hybrid Spatial Semantic Hierarchy(HSSH), introduced by Beeson et al. [3], allows a mobile robot to describe theworld using different representations, each with its own ontology.Other different cognitively inspired approaches to robot navigation con-vey route descriptions from a technically näıve user to a mobile robot. Theseapproaches need not necessarily rely on an exact global self-localization, butrather require the execution of a sequence of strictly local, well-defined be-haviors in order to iteratively reach a target position. Kuipers [4] presents theSpatial Semantic Hierarchy (SSH). Alternatively, the Route Graph model isintroduced by Krieg-Brückner et al. [5]. Both theories propose a cognitivelyinspired multi-layered representation of the map in the head, which is at thesame time suitable for robot navigation.Additionally, several approaches on mobile robotics extend metric maps ofindoor environments with semantic information. The work by Diosi et al. [6]creates a metric map through a guided tour. The map is then segmentedaccording to the labels given by the instructor. Martinez Mozos et al. [7] ex-tract a topological semantic map from a metric one using supervised learning.Alternatively, Friedman et al. [8] use Voronoi Random Fields for extractingthe topologies. Although these works use range measurements as main inputdata, other sensors have been used for similar tasks. Torralba et al. [9] useprocessed images to distinguish between different place categories in the envi-ronment. Pronobis et al. [10] also use vision to recognize the different placesthat form an indoor environment. Finally, the combination of different sen-sory modalities can improve the recognition, as shown in Rottmann et al. [11]and Pronobis et al. [12]. More detailed review of different approaches to placeclassification can be found in Section 5.8.The multi-layered representation presented in this chapter differs from theprevious work primarily in the level of integration achieved. First, each of thelayers of the representation advances the state of the art in its correspondingarea. Second, the advanced techniques are combined into a single, coherentmodel, representing the world at various levels of abstraction (e.g. metric,topological, semantic, conceptual) based on information coming from multi-ple sources (vision, range sensors, verbal cues etc.). In particular, the modelintegrates the approaches of [7] and [12] for the semantic classification of places5 Semantic Modelling of Space 171with visual object search algorithms [13] and the metric mapping based onthe M-Space representation [14]. Moreover, the representation is designed forhuman-robot interaction and the models generated using the aforementionedtechniques are combined with a common-sense ontology of an indoor environ-ment. This bridges the gap in spatial understanding between the robot andhumans and allows to include information extracted from verbal cues into therepresentation.5.1.2 OutlineThe rest of this chapter is organized as follows. First, we highlight the back-ground for the research on spatial representations for mobile robots (Sec-tion 5.2). Then, we provide an overview of our spatial model (Section 5.3)and describe each of the levels of representation in detail (Sections 5.4–5.6).Finally, we present the algorithms used to augment the representation withsemantic object and place information (Sections 5.7 and 5.8, respectively)and report results of performed experiments (Section 5.9). We conclude thechapter with a brief summary in Section 5.10.5.2 BackgroundAn approach to endowing autonomous robots with a human-like conceptual-ization of space inherently needs to take into account research in sensor-basedmapping and localization for robots as well as findings about human spatialcognition.Research in cognitive psychology addresses the inherently qualitative na-ture of human spatial knowledge. In accordance with experimental studies,it is nowadays generally assumed that humans adopt a partially hierarchicalrepresentation of spatial organization [15, 16]. The basic units of such a qual-itative spatial representation are topological regions [17], which correspondto more or less clearly bounded spatial areas. The borders may be definedphysically, perceptually, or may be purely subjective to the human. It hasbeen shown that even in natural environments without any clear physical orperceptual boundaries, humans decompose space into topological hierarchiesby clustering salient landmarks [18].Aside from the functionality of the cognitive map, another relevant ques-tion from cognitive science is how people categorize spatial structures. Cat-egories determine how people can interact with, and linguistically refer toentities in the world. Basic-level categories represent the most appropriatename for a thing or an abstract concept. The basic-level category of a referentis assumed to provide enough information to establish equivalence with othermembers of the class, while distinguishing it from non-members [19, 20]. Wedraw from these notions when categorizing the spatial areas in the robot’sconceptual map. We are specifically concerned with determining appropriate172 Pronobis et al.properties that allow us to meaningfully refer to spatial entities in a situateddialogue between the robot and its user.5.3 Overview of the Spatial ModelThis section gives an overview of the multi-layered conceptual spatial modeladopted in the CoSy project. The model forms a basis for spatial understand-ing, reasoning, navigation, and human-robot interaction in the integratedrobotic system. In this framework, the space is modeled at different levelsof abstraction that range from low-level metric maps for robot localizationand navigation to a conceptual layer that provides a human-like space decom-position and categorization. An illustration of the model and the main levelsof representation is presented in Figure 5.1.The lower layers of the model are derived from sensory input. These layerscombine a metrical, line-based representation of spatial structure modeling oc-cupied space, and a navigation graph of virtual markers modeling free space.Different methods are used to gradually construct more abstract representa-tions. On higher levels, we regard topological regions and spatially situatedobjects as the primitive entities of spatial conceptualization. The robot mustbe able to assign human concepts to such spatial entities in order to meaning-fully act in, and talk about, an environment. Many places in indoor environ-ments are designed in a way that makes their structure, general appearance,and spatial layout afford specific actions; corridors and staircases are exam-ples of this. Other places afford more complex actions provided by objects thatare located there. For instance, the concept of a living room applies to roomsthat are suited for resting. Having a rest, in turn, can be afforded by certainobjects, such as couches or TV sets. The representation allows for combin-ing cues provided by the basic geometrical shape, general visual appearance,perceived objects, and possibly situated dialogue to provide reliable semanticdescriptions of space.The rest of this section provides an overview of each of the layers of thespatial representation.5.3.1 Metric MapThe lowest level of the spatial model is represented by a metric map. Themap encodes spatial boundaries in the environment using lines as basic spatialprimitives and supports self-localization of the robot. It is anchored to a metricworld co-ordinate system, which is also used as a basis for the higher levelrepresentations. The positions of lines as well as of a robot on the metric mapare established and maintained by a module for Simultaneous Localizationand Mapping (SLAM) [14]. Section 5.4 gives more details about the appliedSLAM algorithm and other approaches that have been investigated.5 Semantic Modelling of Space 173Fig. 5.1. The multi-layered structure of the conceptual spatial model.In comparison to a representation based on an occupancy grid [21], the line-based map does not directly provide a description of the free space but ratherof the surfaces in the environment that can be described by lines. However,since the global co-ordinate system of the metric map is purely internal tothe robot and humans are not able to easily evaluate quantitative spatialdescriptions, the metric map alone is not sufficient to support human-robotdialogues.174 Pronobis et al.5.3.2 Navigation MapThe navigation map provides the next layer of representation, which estab-lishes a model of free space and its connectivity, i.e. reachability. It is based onthe notion of a roadmap of virtual free-space markers as described in [22, 23]and implemented as a graph of nodes that are anchored to the metric map. Asthe robot navigates through the environment, a marker or navigation node isdropped whenever the robot has traveled a certain distance from the closestexisting node. Nodes are connected following the order in which they werevisited. More information about the navigation graph can be found in Sec-tion 5.5.It is also in the navigation graph that the spatial representation is aug-mented with semantic information about the environment. First, the semanticcategory of a place is extracted using a place classification algorithm [12] andstored in the nodes. Doors detected in the environment are represented asdoorway nodes and added to the graph. Finally, objects detected by an objectsearch component [13] are also stored on this level of the map. Section 5.7 andSection 5.8 present the algorithms used to detect objects and extract semanticplace information from the geometry and appearance of the environment.5.3.3 Topological MapThe navigation map provides a basis for further, topological abstractions.A topological map consisting of connected areas is built by segmenting thenavigation graph into interconnected sets of nodes separated by recognizeddoors (doorway nodes). This layer of abstraction corresponds to human-likequalitative segmentation of an indoor space into distinct regions (e.g. rooms).On this level, semantic place information extracted and accumulated overentire regions is evaluated to determine appropriate semantic categories forareas in the topological graph. More information about this process can befound in Section 5.5.5.3.4 Conceptual MapOn the highest level of abstraction, the system is endowed with a concep-tual map. The conceptual map builds up a further interpretation of spatialorganization. The topological areas together with their place categories formthe basic spatial entities. A description logic-based reasoner is used to infermore fine-grained semantic information for the areas. The reasoner integratesknowledge about areas and observed objects with a common-sense ontologyof an indoor environment. This ontology represents a taxonomy of areas andobjects and the relations between objects and areas. Since there is a strongconnection between typical objects found in an area and the semantic cate-gory of the area, this layer can also be used to constrain expectations aboutwhich objects are likely to be observed, given that the basic-level concept5 Semantic Modelling of Space 175of an area is known (for example through a situated dialogue with a humanuser). Section 5.6 provides more details about the conceptual map.5.4 Metric MappingThis section gives details about the metric mapping algorithms that wereinvestigated in CoSy and used to maintain the metric representation in thespatial model. Metric maps can be represented in many ways. The two mostcommon approaches are based on occupancy grids [24, 25, 26, 27, 28, 29] andfeatures [30, 31, 32, 33, 14]. Occupancy grids discretize the world into cells.Each grid cell holds a value representing the probability that the correspond-ing area in the environment is occupied. Feature-based maps on the otherhand abstract the sensor data into a set of features. In a structured envi-ronment – of which most office environments are examples – lines, cornersand edges are common features. The features can be parameterized by, forinstance, their color, length, width, position, etc. One of the main advantagesof this type of representation is that it requires very few assumptions aboutthe world, whereas one has to settle on a set of features to parameterize themap beforehand.Mapping (building a model of the environment) and localization (find-ing the position in the environment) are often treated as two separate prob-lems. Maps were made assuming that the position is known and the positionis calculated given a map. However, for an autonomous agent that exploresan unknown environment these two task are intrinsically linked, and form achicken-and-egg problem; to perform mapping one needs the position and toperform position one needs the map. This leads to Simultaneous Localizationand Mapping (SLAM) which has been a thriving research area for more thana decade.In this chapter we will focus on feature-based representations. A feature-based map can in general be writtenM = {fj | j = 1, . . . ,M}, (5.1)where fj is a feature and M is the number of features in the map.5.4.1 M-SpaceA number of different types of features have been used for mapping. Depend-ing on the type of application the model of the environment is 2D or 3D. Formost indoor applications a 2D representation is used; navigation in clutteredenvironments often requires a 3D representation. When taking the step out-doors the world is less structured and it becomes increasingly likely that theground is not flat which also calls for a 3D model.To motivate the work with the so called M-Space representation, let usfirst consider how to represent a line segment. Such a line segment could176 Pronobis et al.for example offer a 2D abstraction of a wall in an indoor environment. Fourparameters are needed to fully specify the line segment. On a real robot, thesensors may not be able to detect its end points; even if the end points arewithin the range of the sensors they are often hard to detect due to occlusions.This implies that a measurement typically only constrains the position of thesensor to a certain distance and relative angle with respect to the wall. Inother words, all dimensions of a wall are typically not constrained by onesingle measurement.There are a number of ways to represent a line segment. To name a few;slope and intersection (y = kyx + my), end points, distance and direction(infinite line [34]), center point, length and orientation. All of these suffer oneor more problems, some of which are addressed by the so called SP-model [35].A characteristic property of SP-model is that each feature element hasits own local reference frame. The frame of reference is chosen with the axesalong the directions of symmetry. A line, for example, has the x-axis alongthe direction of the line. A plane will have a normal that coincides with thez-axis. The main advantage of using a local frame is that the description of theuncertainty can be made independent of the global position of the features.This avoids lever-arm effects that can result when for example using directionand orientation to represent a line. The local frames also help to make frametransformations and differentiations thereof more standardized. Another keyconcept in the SP-model is the so called binding matrix, B. The bindingmatrix is a row-selection matrix. The self-binding matrix selects the DOFsthat are not part of the motion symmetry, i.e. the DOFs of a feature that areconstrained and have probabilistic information attached to them. The bindingmatrices offer a machinery for making partial observations of a feature. Thisis useful, for example, when observing a single point on a line. A limitationwith the SP-model is that one has to attach a frame to all features. For sometypes, such as lines, it is difficult to model the extent, e.g. the length, in aprobabilistic way within the SP-model framework. In [36], the length of linesis estimated and modeled but it relies on always detecting both end points atthe same time and making a direct measurement of the length. An indirectmeasurement is not possible as the origin of the reference frame cannot beobserved, not being attached to anything observable, but just defined to bein the middle of the line.The so called M-Space representation builds on the SP-model4. It alsoattaches a local frame to each feature element and allows for a generic treat-ment of many types of features. The measurement subspace, or M-space, isan abstraction of the measured subspace of the feature space that reflectssymmetries and constraints. The idea is that the features are parameterizedto fully specify their location and extent (the feature space) but that they canbe initialized in a subspace corresponding to the information provided by thesensors.4 For a detailed description see [14] from which this text is derived.5 Semantic Modelling of Space 177For example, when representing a line segment the extent is accommodatedfor in the representation even though only the distance to and the orientationof the line is known initially. We cannot represent the uncertainty with regardto changes in the coordinates along the length of the line by a Gaussiandistribution. However, the uncertainty regarding changes perpendicular to theline and regarding the orientation can be approximated by a Gaussian. Letδxp denote the M-space corresponding to a small change in feature coordinatesδxf . Here the subscript, p, stands for small perturbations in the M-space. Theactual values of the M-space coordinates, xp, are never needed or considered.It is only the changes to them that enter into the estimates. These changesare used to make adjustments to the feature coordinates xf . The uncertaintyestimate is an estimate of the distribution of δxp values around a mean ofzero. The adjustments to the feature coordinates are made to maintain thiszero-mean. No re-centering step like in the SP-model is required with thisview of the uncertainty. The uncertainty is defined in a frame attached to thefeature and can be projected into the global frame using the current globalcoordinates of the feature. The statistics are represented in an analytic wayrather than in the strict geometric sense of the SP-model. In most cases, thedifferences are in the second order corrections to the covariances.The relation between the feature space coordinates and the M-space co-ordinates is defined by a projection matrix, B(xf ), similar to the bindingmatrix in the SP-model. The projection matrix relates small changes δxp tosmall changes δxf . An important difference to the binding matrix is that theprojection matrix is a function of the individual feature and changes withtime. The rather involved re-centering step in the SP-model is replaced byre-evaluating the projection matrices.A common issue in feature-based SLAM is that one cannot initialize afeature after the first observation. A single observation typically does notcontain enough information to do so reliably. Among the reasons behind thiswe find for example• The entire feature is not detected at once.– In the case of a line segment, the end points might not have beendetected if the line is partially occluded or long.– When using monocular vision, only the bearing to the feature can beinitialized from a single image.• Measurements are noisy. Even though a feature is fully observed it is goodpractice to get a second opinion from new measurement data to reject falsemeasurements.The M-space representation offers a solution to these problems by allow-ing the M-space dimensionality to change over time. Features are typicallyinitialized with zero M-space dimensions and with time, as more informationis gathered, more dimensions will be added. Consider mapping a wall as a linesegment. The life cycle of the line segment might be1. First detection: feature initialized with 0 M-space dimensions.178 Pronobis et al.Fig. 5.2. A few stable features are identified and used to define the location oflandmarks. The rest of the features are used to improve the matching/recognitionof these.2. Re-observed N times: the wall’s existence and quality are confirmed. Thedistance and orientation of the wall added to the M-space.3. Start point detected: M-space dimensionality goes up to 34. End point detected: the feature reaches full dimensionality of 4.The importance of the ability to let the dimensions of a feature grow over timeis well illustrated by a horizontal line feature observed by a camera. A singleimage does not contain information to pinpoint the location of a feature. Theassumption that the line feature is horizontal implies that a single observationwill be enough to provide information about the relative orientation of therobot. That is, even if the robot moves parallel under the line and is unableto use triangulation to fix the position of the line in space the observations ofthe line can help reduce the angular uncertainty of the robot. This is usefulin, for example, a corridor where the motion often is parallel to the linearstructures found in the ceiling.5.4.2 Single Camera Bearing Only SLAMRange sensors such as laser scanners are still the most common sensors forsystems that perform mapping and localization in settings where robustness iskey e.g. in industrial applications. However, cameras are becoming increasinglyinteresting as the performance keeps increasing while the price keeps goingdown due to the large demand from the consumer market e.g. for mobilephones. One of the main advantages of a camera over a range sensor is thatthe information that it provides is so much richer and not limited to a fewhundred distance measurements typically lying in a plane. The hard part isto get the information out from the image.There is a rich literature on single camera bearing-only SLAM. Most ofthese use point features extracted from the image to define landmarks in the5 Semantic Modelling of Space 179map [37, 38]. Given standard feature detectors such as SIFT [39], there can beseveral hundreds of features and thus potential landmarks per frame. A singleSIFT descriptor is not discriminative enough in itself, especially in man-madeenvironments where structures like corners give raise to many SIFT pointswith very similar descriptors. When used for object recognition [39] it is acombination of descriptors extracted from the object that provide the dis-criminative strength. This idea is used in the vSLAM approach [40] wherethe SIFT points are used to recognize places. In [41], we present a frame-work where a few stable (over time and in space) SIFT features are identifiedand used as landmarks (location features). The rest of the SIFT features areused to strengthen the matching of features (recognition features). This isillustrated in Figure 5.2. When matching against a landmark, the matchingis performed not only with the feature defining the position of the landmarkbut also the rest of the feature extracted from the two frames. This greatlyimproves the robustness of the matching.5.4.3 Using Visual Attention for SLAMChoosing useful landmarks which are easy to track, stable over several frames,and easily re-detectable when returning to a previously visited location isimportant in order to get a visual SLAM system working. Getting few butgood, rather than many and bad landmarks reduces the issue of complexity.In [42, 43, 44], we suggest the application of a biologically motivated attentionsystem [45] to find salient regions in images. Attention systems are designedto favor regions with a high uniqueness such as a red fire extinguisher on awhite wall. Such regions are especially useful for visual SLAM because theyare discriminative by definition and easy to track and re-detect. We show thatsalient regions have a considerably higher repeatability than Harris-Laplaciansand SIFT key-points. Active gaze control is also used and has been shownto enhance the performance over using a statically mounted camera. Thestrategy to steer the camera consists of three behaviors: a tracking behavioridentifies the most promising landmarks and prevents them from leaving thefield of view. A re-detection behavior actively searches for expected landmarksto support loop-closing. Finally, an exploration behavior investigates regionswith no landmarks, leading to a more uniform distribution of landmarks. Theadvantage of the active gaze control is to obtain more informative landmarks(e.g. with a better baseline), a faster loop closing, and a better distributionof landmarks in the environment. Figure 5.3 shows the difference in the robotpose uncertainty when driving the same trajectory with active camera controland without. This example illustrates that the active camera control allows therobot to reduce the uncertainty by seeing landmarks that it could otherwisenot have seen. For more details please refer to [43, 44].180 Pronobis et al.Fig. 5.3. A comparison of the robot pose uncertainty with camera control (active)or without (passive).5.4.4 Visual ScansOne of the more popular and successful ways to do metric mapping is touse scan matching [46, 47, 29]. In a feature-based setting the scan can beconsidered to be a feature which is defined by the scan distances themselves.Building the map boils down to finding the position from which the scanswere acquired in such a way that the laser scans align and for a consistentmap.In [48] we present an idea to use so called visual scans in much the sameway as laser scans are used in scan matching. Using a stereo camera, a 3Dpoint cloud is calculated by extracting and matching SIFT features in eachimage. This 3D point cloud forms the visual scan. Aside from the position,each point also has a descriptor saying something about the appearance whichcan be used for matching.The map is defined by a number of reference scans. A reference scan isadded when there is not enough overlap between the current visual scan andany of the other visual scans. The advantage of this representation is that itgives a very rich description of the environment (a dense point cloud whichcan have hundreds or even thousands of points) while the estimation problemonly needs to deal with the parameters defining the position of the sensor(3 parameters in 2D and 6 in 3D, compared to 3N with N points treatedindependently). An example is shown in Figure 5.45 Semantic Modelling of Space 181Fig. 5.4. A map with 126 visual reference scans. Together these scans contain 8333points. This shows how the visual scans help define a dense representation at thesame time as providing a low-dimensional estimation problem.5.5 Navigation and Topological MapsAbove the bottom layer – the metric map – the spatial model contains thenavigation and topological maps. As explained above, the metric map en-codes boundaries in the environment and is used to ensure safe and reliablenavigation and obstacle avoidance. In contrast, the navigation and topolog-ical layers encode more abstract information about the space accessible tothe robot, particularly important from the functional point of view. This in-formation is encoded in the form of graph-like structures in which the linksrepresent connectivity between spatial entities at different spatial scales. Thegraph constituting the navigation map consists of nodes representing smallunbounded free space regions in the environment. The topological represen-tation, in its turn, models the indoor environment in terms of larger boundedareas connected by detected doors.The representations play two main roles in the system. First, they dis-cretize the continuous free space into a finite number of spatial units. Theseunits are then used for tasks such as planning or interaction with the robot.For example, the high level task “go to the office”, can be translated to thelow level action “go to the closest navigation node attached to the topologicalnode representing the office”. If this position is occupied the robot can chooseto go to the next navigation node. Discretization of space drastically reducesthe number of combinations that have to be considered during the planningprocess.The second important function of the navigation and topological repre-sentations is preserving additional information about the surroundings. Here,semantic information about places extracted from the sensory input is accu-182 Pronobis et al.Fig. 5.5. An example of a navigation map overlayed on a metric map. The free spacenavigation nodes are represented by circles and are assigned to different topologicalareas based on the separation established by the doorway nodes. The colors ofthe nodes indicate the functional category of the areas as recognized by a placeclassification algorithm.mulated and stored together with navigation and topological nodes. Moreover,information about objects found in the environment is tied to nodes in thenavigation graph. This is used to link the representations with the conceptualmap and allows to refer to places in terms of their functional category or de-tected objects. As an example, the order “go to the TV”, can be processedand translated into “got to the closest node from which you saw the TV”.The rest of this section focuses on the process of generating the naviga-tion and topological representations based on the information encoded in themetric map and additional cues extracted from the sensory input.5.5.1 Building the Navigation GraphThe navigation map provides the first discretization of the continuous spacedescribed by the model. It is represented in the form of a graph built as therobot explores the environment and is based on the notion of a roadmap ofvirtual free space markers [22, 23]. A free space navigation node is droppedwhenever the robot has traveled a certain distance from the closest existingnode (approximately 1 meter). Each node is anchored to the metric mapand is assigned (x, y) co-ordinates. Nodes are connected following the order inwhich they were generated. This order is given by the trajectory that the robotfollows during the map acquisition process. The final graph serves for planningand autonomous navigation in the already visited part of the environment.An example of a navigation graph overlayed on a metric map is presented5 Semantic Modelling of Space 183in Figure 5.5. This simple representation proved to be very powerful duringreal-world experiments with the integrated system.As the robot navigates through the environment, additional informationabout the surroundings collected on the way is assigned to the closest navi-gation nodes. Moreover, special doorway nodes are added to the navigationgraph at the points where doors are detected in the environment (see Fig-ure 5.5). As explained below, these nodes play an important role in buildingthe spatial model.5.5.2 Space Segmentation and Topological GraphThe structure of indoor environments allows for introducing larger scale andmore abstract representations than the navigation graph. In such environ-ments a room is an important concept. Different rooms can be associatedwith different owners and functionalities. Moreover, rooms are spatial enti-ties commonly referred to in the natural language. The ability to segmentspace into rooms becomes crucial for an artificial mobile cognitive system.Therefore, as another layer, the spatial model builds a topological graph con-sisting of areas and links which represents rooms in the environment and theirconnectivity.The structure of the topological graph is built based on the assumptionthat the transition between two areas happens through a door. This creates ahuman-like qualitative segmentation of an indoor space into distinct regions. Adoor detection algorithm is used to generate doorway nodes in the navigationgraph whenever the robot passes through a narrow opening. The width of theopening is selected so that it matches typical doorways in the environment.Information about the door opening, such as width and orientation, is storedalong with the detected position of the doorway in the doorway node. Thedoorway nodes are used to segment the navigation graph and assign navigationnodes to areas in the topological graph.More complex door models such as those in [49, 50] can be used for morerobust door detection. However, such models put additional constraints onhow doors have to look to be recognized. The only assumption in the modeldescribed here is that the door is a narrow opening which the robot passesthrough. No assumptions are made regarding the door leaf (e.g. swinging orsliding) or special structure around the door. This can be beneficial for arobot that has to operate in different environments. An alternative would beto use a learning approach, such as in [51], where both visual features and themotion of the door are taken into account.5.5.3 Adding Object InformationObjects and landmarks play an important role in understanding spatial struc-ture. They are important cues that often determine the actions that can be184 Pronobis et al.performed in a particular area. Moreover, objects are nameable features com-monly used in describing spatial locations. For this reason, visual search algo-rithms are used in the system to perform autonomous exploration and detec-tion of objects typically found in indoor environments. Detailed informationabout the algorithms applied for this purpose can be found in Section 5.7.The presented spatial model incorporates information about objects. Thisinformation is later used by the last conceptual layer. First, however, theobjects must be tied to their spatial locations. This is the role of the objectnodes which are connected with the navigation nodes of the navigation graph.The object nodes store information about the type of the recognized objectand its metric location. The nodes are then linked to the closest navigationnode.5.5.4 Adding Semantic Place InformationMany places in an indoor environment can be characterized by semantic cate-gories corresponding to their inherent functionality. Rooms constitute a goodexample as they can be categorized as offices, kitchens, meeting rooms etc.However, semantic descriptions can also be assigned to smaller regions such asa printer area in a corridor. The semantic place category is usually reflectedin the objects located in that place, but also in the general appearance andgeometrical layout.One of the roles of the navigation nodes and topological areas in the spa-tial model is to store semantic information about the places to which theycorrespond. This information is used to link the lower layers modeling spa-tially allocated regions with the spatial concepts of the conceptual layer. Aspecialized component performing multi-modal place classification is used toextract semantic descriptions from the sensory input of a robot. Visual andlaser range sensory data acquired in an environment are analyzed and com-pared to place models in order to produce beliefs about the place categories. Inthe simplest case, the component can be used to distinguish between two ba-sic place categories: a corridor or a room (e.g. based on the clutteredness andgeometric layout). Further specialization can be performed in the conceptuallayer based on object information or situated dialogue. However, more spe-cific place categories can also be recognized directly by the place classificationsystem. More information about this process can be found in Section 5.8.As will be shown through experiments in Section 5.9, the place classifica-tion system is able to classify a place with high accuracy given a single datasample (e.g. one image and laser scan) corresponding to only one viewpoint.However, in this case, the task is to provide a reliable and stable label for thewhole region covered by a navigation node or a topological area. Since thesensors employed are not omnidirectional, it is necessary to accumulate andfuse the incoming information. However, the data that the robot gathers arenot evenly spread over different viewpoints. On the contrary, in many casesthe sensors receive a continuous stream of non-informative data (e.g. when the5 Semantic Modelling of Space 185Fig. 5.6. Generating semantic labels for navigation nodes and topological areasusing multi-modal place classification.robot is parked close to a wall blocking the view). The system must be ableto deal with such problems as temporary lack of informative cues, long-termocclusions or large variability affecting certain viewpoints.For this reason, the place information produced by the place classificationalgorithm is accumulated over time and space as presented in Figure 5.6. Foreach multi-sensory data sample, place classification provides a set of beliefsencoded as a vector of real-valued outputs (see Section 5.8 for details). Theconfidence of the final decision is also measured and provided by the clas-sification component. The beliefs are fused using a confidence-based spatio-temporal accumulation algorithm. The algorithm relies on information aboutthe current position on the navigation and topological map provided by thelocalization and mapping system. The spatio-temporal accumulation processis performed within the region covered by the current node and area. Whenthe robot moves to a different node, the collected information is used to up-date the semantic label attached to the map and saved as a future prior. Whenthe robot enters a location which was already explored, the previously storedbeliefs are loaded and can be refined by further exploration.The principle behind the confidence-based spatio-temporal accumulationalgorithm is illustrated in Figure 5.7. As the robot explores the environment,it moves with a varying speed. The robot has information about its own move-ment provided by the wheel encoders (odometry). As errors accumulate overtime, this information can only be used to estimate relative movement ratherthan absolute position. Although the accurate metric information could beused instead, odometry is sufficient for our application. The spatio-temporalaccumulation process creates a sparse histogram along the robot pose trajec-tory described by the metric position (x, y) and heading (θ). The size of thehistogram bins is adjusted so that each bin roughly corresponds to a singleviewpoint. Then, as the robot moves, the beliefs about the current semanticcategory accumulate within the bins. An average of the outputs is calculatedin a manner similar to the Discriminative Accumulation Scheme (DAS, [10])186 Pronobis et al.Fig. 5.7. Illustration of the spatio-temporal accumulation process. As the robotexplores the environment, the beliefs collected on the way accumulate over timewithin the bin corresponding to the current pose (x, y, θ) and over space in differentbins.used in the framework of cue integration. This is what we call the temporalaccumulation. It prevents a single viewpoint from becoming dominant dueto long-term observation. Since each viewpoint observed by the robot willcorrespond to a different bin, performing accumulation across the bins (thistime spatially) allows for generating the final outputs to which each viewpointcontributes equally. In order to exclude most of the misclassifications beforethey get accumulated, the decisions are filtered based on the confidence valueprovided by the place classification component. Finally, the best hypothesisis calculated. It is assigned to the navigation node and topological area rep-resenting the spatial region over which the accumulation was performed.5.6 Conceptual MapThe conceptual map provides the link between the low-level maps and thecommunication system used for situated human-robot dialogue by groundinglinguistic expressions in representations of spatial entities, such as instances ofrooms or objects. It is also in this layer that knowledge about the environmentstemming from other modalities, such as object recognition and dialogue, isanchored to the metric and topological maps.Based on the work by Zender [52], our system is endowed with a common-sense OWL ontology of an indoor environment (see Figure 5.8) that describestaxonomies (is-a relations) of room types and typical objects found thereinthrough has-a relations. These conceptual taxonomies have been handcraftedand cannot be changed online. However, instances of the concepts are addedto the ontology during run-time. Through fusion of acquired and assertedknowledge gathered in an interactive map acquisition process [53] and throughthe use of the innate conceptual knowledge, a reasoner can infer information5 Semantic Modelling of Space 187AreaRoomCorridorObjectLabObjectOfficeObjectKitchenObjectLivingroomObjectBookcaseFlowerpotLabOfficeKitchenLivingroomConferenceroomhasObjecthasObjecthasObjecthasObjectWhiteboardDeskComputerWorkbenchPhoneOvenMicrowaveCookerCoffeemachineTeamakerFridgeTVSetCouchFig. 5.8. Illustration of a part of the commonsense ontology of an indoor officeenvironment. Solid arrows denote the taxonomical is-a relation.about the world that is neither given verbally nor actively perceived. Thisway linguistic references to spatial areas can be generated.Acquired KnowledgeWhile the robot moves around constructing the metric and topological maps,our system derives higher-level knowledge from the information in these lay-ers. Each topological area, for instance, is represented in the conceptual mapas an ontological instance of the type Area. Furthermore, as soon as reliableinformation about the semantic classification of an area is available, this is re-flected in the conceptual map by assigning the area’s instance a more specifictype. Information about recognized objects stemming from the vision subsys-tem is also represented in the conceptual map. Whenever a new object in theenvironment is recognized, a new instance of the object’s type, e.g. Couch, isadded to the ontology. Moreover, the object’s instance and the instance of thearea where the object is located are related via the hasObject relation. Thisprocess is shown in Figure 5.1.Asserted KnowledgeDuring a guided tour with the robot, the user typically names areas andcertain objects that he or she believes to be relevant for the robot. Typical188 Pronobis et al.assertions in a guided tour include “You are in the corridor,” or “This is thecharging station.” Any such assertion is stored in the conceptual map, eitherby specifying the type of the current area or by creating a new object instanceof the asserted type and linking it to the area instance with the hasObjectrelation.Innate Conceptual KnowledgeWe have handcrafted an ontology (Figure 5.8) that models conceptual com-monsense knowledge about an indoor office environment. On the top level ofthe conceptual taxonomy, there are the two base concepts Area and Object.Area can be further partitioned into Room or Corridor. The basic-level sub-concepts of Room are characterized by the instances of Object that are foundthere, as represented by the hasObject relation.Inferred KnowledgeBased on the knowledge representation in the ontology, our system uses a de-scription logic-based reasoning software that allows us to move beyond a purelabeling of areas. Combining and evaluating acquired and asserted knowledgewithin the context of the innate conceptual ontology, the reasoner can infermore specific categories for known areas. For example, combining the acquiredinformation that a given topological area is classified as a room and contains acouch with the innate conceptual knowledge given in our commonsense ontol-ogy, it can be inferred that this area can be categorized as being an instanceof LivingRoom. Conversely, if an area is classified as a corridor and the usershows the robot a charging station in that area, no further inference can bedrawn. The most specific category the area instantiates will still be Corridor.Our method allows for multiple possible classifications of any area becausethe main purpose of the reasoning mechanisms in our system is to facilitatehuman-robot interaction. The way people refer to the same room can differfrom situation to situation and from speaker to speaker, as reported by Topp etal. [54]. For example, what one speaker prefers to call the kitchen might bereferred to as the recreation room by another person. Since our aim is to beable to resolve all such possible referring expressions, our method supportsambiguous classifications of areas.5.7 Object Detection and RecognitionIn this section, we discuss how the robot can use active vision for perceivingobjects and landmarks in the environment. The process is active in that it isbased on active search, primed by interpretations established at other levels ofspatial representation. Active vision provides information about objects in the5 Semantic Modelling of Space 189environment. It covers object recognition and determines object pose relativeto the world coordinate system adopted by the metric layer in which all otherrepresentations are grounded.The rest of this section gives details about the approach adopted in CoSyfor object search and localization (Section 5.7.1) and presents results of anexperiment evaluating different methods for object distance estimation (Sec-tion 5.7.2).5.7.1 Object Search and LocalizationIn our early work [55], the search for objects was performed while exploring theenvironment to cover space or when guided by the user. This is not sufficient.The user is able to show the robot all objects, but this is a tedious task anddoes not have the right level of autonomy expected from an autonomous agent.When object search runs in parallel with exploration, it is driven by the laserscanner which has a 180◦ field of view compared to the camera having abouta fourth of that. This means that the camera is not guaranteed to see all partsof the environment.View planning as a research area is well established. The so called artgallery problem [56] is defined as finding a minimal set of viewpoints fromwhich all the parts of the environment can be observed. This problem is akinto the problem of planning for finding objects in the environment. The maindifference is that one also needs to take into account the limitations of theobserver, i.e. the camera. One of the most important limitations comes fromthe finite resolution of the camera and the fact that objects have differentsizes. Even if a small object is in the field of view it will not be detectedif the camera is too far away. Similarly, a large object can typically not bedetected if the camera is too close. A system taking these constraints intoaccount is presented in [57, 13]. This system uses a combination of a visualattention mechanism in the form of the RFCH algorithm [58], camera zoomand SIFT recognition [39] for finding the objects. View planning is carriedout by selecting views from the nodes in the navigation graph. Briefly, whensearching for objects the system first analyzes the map of the environmentand performs the view planning. The robot then visits each view point andperforms the visual search. The visual attention mechanism tells the systemswhat parts of the image to investigate further and the system does so byzooming in, thus gathering more pixels from the potential object. When theobject is close, SIFT recognition is used to verify the identity of the object.The distance to the object is estimated from visual cues. The distance is usedto control the zooming and to estimate the position of the object.This method relies on the visual attention system not to produce too manyviews to investigate further. In [59] a method for accumulating over time theavailable visual evidence for the presence of objects was investigated. Thiswould allow the object recognition algorithm to run in parallel to the ex-190 Pronobis et al.Fig. 5.9. An example of a planned path for object search. In this example, one ofthe nodes is not used in the plan.ploration and could also handle object detection/recognition algorithms thatprovide information that is too weak to act on immediately.The input to the view planning is, besides the objects to search for, a gridrepresentation of the world and a navigation graph. The grid resolution is0.5m [13]. The grid cells represent possible object locations, and the plan isconstructed such that each grid cell is observed from a distance appropriatefor each object. Figure 5.9 shows an example of a plan from the view planning.There may be several views associated with a node corresponding to differentviewing directions.To represent an object for recognition, the system uses one segmentedimage of each object in a close-up view. Two examples of training images areshown in Figure 5.10. To be able to plan for detecting the objects and todetermine the distance to a detected object, the real world and image size ofthe object are also stored in the object database.As a consequence of using a single view for each object, the system canonly recognize the object from one side. Using a multi-view representation ofthe object is a natural extension to this work.In [60], the distance estimate used to determine zoom levels was baseddirectly on the robot’s laser sensor. However, the distance provided by thelaser is often misleading, as Figure 5.11 shows: the laser sensor is placed about30cm above the floor and if an object is not at that height, the estimate maybe wrong. The approach works only for objects that are placed on the flooror are located close to walls (for example, in a bookshelf). If the distance5 Semantic Modelling of Space 191Fig. 5.10. Two of the training images (coffee machine and rice package) providedto the robot beforehand to learn the appearance of the objects.Fig. 5.11. Distance estimation provided by the laser may not be reliable: insteadof the distance to the object on the table, the distance to the shelf is measured.estimate is wrong, the final zoom may either not be sufficient to make theobject occupy enough of the image, or otherwise may be too large causingonly a small part of the object to be seen. Furthermore, even if the objectis recognized, its estimated position might be inaccurate. To address theseissues, we have looked at two alternative ways for distance estimation.Using the Vote MatrixUsing the RFCH vote matrix for distance estimation consists of measuringhow many cells are part of the object and treating the area they occupy in theimage as an approximation of the object’s size. Here, cells are considered to beassociated with a hypothesis if their degree of match is above the threshold andif there is an 8-connected path to the hypothesis with cells of monotonicallyincreasing value. Only the strongest hypothesis and its associated 8-connectedcells are taken into account, because it is likely to be the most reliable.Given the object’s actual size stored in the training database, the distanceis then computed as:D =WrealWim2Dvotetan(α2)192 Pronobis et al.where D stands for the estimated distance (meters); Wreal , for the real widthof the object (meters); Wim , for the width in pixels of the camera image;Dvote , for the width in pixels of the bounding box of the cells associated witha hypothesis and α, the horizontal viewing angle. This procedure is fast andapproximate, but sufficiently accurate to allow the object search algorithm toassign a valid zoom.Using SIFTSIFT produces a scale parameter for each key point extracted. For eachmatched pair of key points in the training and recognition image, the quotientof the keys’ scale parameter gives an estimate of their relative apparent sizeand hence their distance, according to:D =WrealWim2WtrStrSrealtan(α2)where Str denotes the scale of the point extracted from the training image;Sreal , the scale of the point extracted from the recognition image, and Wtr ,the width of the object in the training image in pixels.As mismatched key point pairs can produce incorrect scale parameters,the final estimate of the object distance is taken as the median of the distanceestimates from all matches. Experiments indicate that an adequate estimate isobtained given 10 or more SIFT matches. With 4 matches or more a passablerough estimate is typically obtained (within about 30%). If there are fewerthan 4 matches, the result is likely to be very poor (most likely based on someother structure than the object) and is not used.The drawback of the above method is that extracting SIFT features froman image is computationally expensive, and using it to guide the zoom processmay take too long to be feasible. Another problem is the number of SIFTfeatures required to obtain a robust estimation; when the object is small inthe image (i.e. resolved by few pixels), it is unlikely that enough matches willbe available.5.7.2 Object Distance EstimationFigure 5.12 presents the results of distance estimation using RFCH and SIFTwithout magnification, performed on five different test objects. As expected,performance deteriorates for both methods at long range, due to the decreasedsize of the object in the image, and for RFCH also partly to the discretizationof the vote cells.It is notable that the values obtained through both methods tend towardsthe low end. The reason for this are mainly outliers, erroneously assigned val-ues of 0.5–1m, caused by large background structures being mistaken for a5 Semantic Modelling of Space 193(a) RFCH distance estimates(b) SIFT distance estimatesFig. 5.12. Distance estimation results; all objects. Top image RFCH, bottom SIFT.Boxes signify one standard deviation about the average for each distance; linessignify the most extreme values.close-up object. Compared to RFCH, SIFT exhibits a far more accurate anddependable estimate at short range. However, its quality rapidly deterioratesat longer distances, as can be seen by inspecting the average value of the es-timates beyond 2.5m in Figure 12(b). This is because a certain level of detailis needed to extract SIFT keys. In contrast, RFCH, though most reliable atmedium ranges (as demonstrated by the standard deviations in Figure 12(a)),retains the ability at long range to provide very rough approximations, gen-erally adequate for the purpose of selecting a zoom level for the next step.For the final distance estimate, it should be pointed out that SIFT is used –but the magnification of the image will correspond to shifting the diagram inFigure 12(b) into the 0.5m–1m region where the method is most effective.Figure 5.13 highlights the differences between RFCH and SIFT in distanceestimation. Here, for each test image, the absolute error of the distance esti-mate is compared between the two methods and the percentage of cases where194 Pronobis et al.Fig. 5.13. Proportion of instances in which RFCH and SIFT provide the bestestimate.each of the methods gives better estimate is plotted. The graph shows thatRFCH becomes more reliable at 2m range or above.5.8 Place ClassificationThis section presents a multi-modal place classification algorithm able to iden-tify places and recognize semantic place categories. The method effectivelyutilizes information from different robotic sensors by fusing multiple visualcues and laser range data [12]. The presented approach was used for real-timesemantic labeling of the spatial entities represented by the conceptual spatialmodel.Place classification, as considered in this section, can be described as a su-pervised pattern recognition problem of assigning a region in an environmentto one of predefined place classes based on multi-modal sensory input anda set of place models. First, the place models are build from a collection oflabeled data samples acquired in places belonging to the modeled classes. Themodels store intrinsic visual and geometric properties of the classes. Then, thealgorithm is presented with data samples acquired in one of the same placesor in a novel place belonging to one of the same categories, possibly underdifferent conditions and after some time (where the time range goes from someminutes to several months). The goal is to classify correctly as much of thesensory data samples as possible.The ability to classify places based on their visual and geometric propertiesis an important competence for a mobile cognitive agent in two fundamen-tal scenarios. First, place classification can be used to recognize previouslyvisited places. In this scenario, place classification becomes a key element of5 Semantic Modelling of Space 195topological and hybrid localization systems, providing them with means forglobal localization and loop closing [61, 62, 12]. Second, place classificationcan be used to assign novel places to semantic categories, and thus augmentspace representations with semantic information [9, 7, 63]. In both cases, thisis a challenging classification problem due to large variability and dynamicsof real-world environments. First, viewpoint variations cause the sensors tocapture different aspects of the same place, which often can only be learnedif enough training data are provided. Moreover, real-world environments areusually dynamic and their appearance changes over time. The recognition sys-tem must be robust to variations introduced by changing illumination (e.g.during sunny days and at night) and due to human activity (people mightappear in the images, objects and furniture can be relocated).Place classification is a widely researched topic. Purely geometric solutionsbased on laser range data have proven to be successful for certain tasks [7,64, 61]. However, the limitations of such solutions inspired many researchersto turn towards vision which nowadays is becoming tractable in real-timeapplications. The proposed methods employed either perspective [9, 65, 66, 10,62] or omnidirectional cameras [67, 68, 69, 70]. The main differences betweenthe approaches relate to the way the scene is perceived. Several approachesemploy local features, computed from distinct parts of an image [66, 69, 70].Other use global features, derived from the whole image [67, 68, 9]. Recently,several authors observed that robustness and efficiency can be improved bycombining information provided by different visual cues [10, 62] or differentsensors, such as a camera and a laser range finder [11, 71, 12].The algorithm presented here is able to perform robust place classificationunder different types of variations that occur in indoor environments over aspan of time of several months. The method relies on robust descriptors [72, 39,7] and discriminative classifiers [73, 74] known for their superior generalizationabilities. The reliability is further improved by integrating multiple cues andmodalities. The system uses different types of visual information provided byglobal and local image descriptors and geometric cues derived from laser rangescans. The cues are combined using a high-level cue integration scheme thatlearns how to optimally weight each cue [12]. The system is able to measureits own level of confidence and fuse information over time and space in orderto provide a reliable decision. Finally, in case of dynamic environments, wherethe long-term variability cannot be handled by the generalization abilities ofthe algorithm, the internal representation can be incrementally updated tomaintain a stable performance as proposed in [75].The rest of this section motivates the choice of modalities (Section 5.8.1),provides an overview of the architecture of the place classification system (Sec-tion 5.8.2) and gives details about the algorithms used to extract and classifythe geometric and visual cues (Section 5.8.3 and Section 5.8.4). Then, themethod used for cue integration is described in Section 5.8.5. The section con-cludes with a discussion on the need for adaptive models for place classificationin dynamic environments (Section 5.8.6). All the algorithms were experimen-196 Pronobis et al.tally evaluated on robotic platforms operating in realistic environments. Theexperiments and obtained results are presented in Section Section 5.9.5.8.1 Multiple Cues and Modalities for Place ClassificationNowadays, robots are usually equipped with several sensors, typically a laserrange finder and a camera (or cameras), providing both geometrical and vi-sual information about the environment. The ability to effectively integratemultiple cues, possibly extracted from multiple sensory modalities, becomesan important feature of a place classification system. First of all, as each sen-sor usually captures a different aspect of the environment, using multiple cuesallows for obtaining more descriptive representation. A laser range scannercan be a valuable source of geometrical information, while vision is neces-sary if a robot requires a notion of human-like appearance-based concepts.Good descriptive and discriminative abilities along with robustness are thetwo crucial features of a place classification system with a great influence onits overall performance. The visual sensor is an irreplaceable source of distinc-tive information about a place. However, this information tends to be noisyand difficult to analyze due to the susceptibility to variations introduced bychanging illumination and everyday activities in the environment. At the sametime, laser range finders provide much more stable and robust geometric cues.These cues, however, are unable to uniquely represent the properties of dif-ferent places. This leads to the problem of perceptual aliasing [76]. Clearly,each modality has its own characteristics. Interestingly, the weaknesses of oneoften correspond to the strengths of the other.It is important to note that even alternative interpretations of the informa-tion obtained by the same sensor can be valuable. In this work we concentrateon two different types of visual cues based on global and local image features.Global features are derived from the whole image and thus can capture gen-eral properties of the whole scene. In contrast, local features are computedlocally, from distinct parts of an image. This makes them much more robustto occlusions and viewpoint variations, but requires a costly matching processin order to find feature correspondences.The different properties of the cues result in different performance and er-ror patterns on the place classification task. This is illustrated in Figure 5.14which shows distributions of errors made by three single-cue place classifica-tion algorithms for five different place classes (see Section 5.9.1 for details). Itis apparent that each of the cues makes errors according to a different pattern.The cue integration scheme should exploit this fact in order to increase theoverall performance. The experimental results reported in Section 5.9.2 showthat the performance of a place classification system can indeed be boosted bycombining the stability of geometrical solutions with the versatility of differentvisual cues.5 Semantic Modelling of Space 197Predicted ClassActual Class1pO CR 2pO KT PR1pOCR2pOKTPRPredicted ClassActual Class1pO CR 2pO KT PR1pOCR2pOKTPRPredicted ClassActual Class1pO CR 2pO KT PR1pOCR2pOKTPR00.20.40.60.81(a) Global visual cues (b) Local visual cues (c) Laser range cuesFig. 5.14. Distributions of errors made by three single-cue place classification algo-rithms for five different place classes (1pO - one person office, CR - corridor, 2pO -two persons office, KT - kitchen, PR - printer area). Bright colors indicate the classesmost often confused with the actual class. The diagonal elements were removed.Fig. 5.15. Architecture of the multi-modal place classification system.5.8.2 Architecture of the Place Classification SystemThe architecture of the place classification system described in this section isillustrated in Figure 5.15. The system relies on two visual cues correspondingto two different types of image features (local based on the SIFT descriptor [39]and global based on the Composed Receptive Field Histograms [72]) as wellas simple geometrical cues extracted from laser range scans [7]. The cues areprocessed independently. For each cue, there is a separate path in the systemwhich consists of two main building blocks: a feature extractor and a classifier.Each classifier produces a set of outputs indicating its soft decision for allplace classes. These outputs can be used directly to obtain the final decisionseparately for each cue. In cases when several cues are available, the single-cueoutputs are combined using a high-level discriminative accumulation schemeproducing integrated outputs from which the final decision is derived. As wasdescribed in Section 5.5.4, the integrated outputs can be accumulated overtime and space if the system is used on a mobile platform. Since each of the198 Pronobis et al.Fig. 5.16. An occupancy grid map built on the ground floor of the building 52 atthe University of Freiburg. Some natural divisions can be extracted from this mape.g. corresponding to rooms, doorways and a corridor.cues is treated independently, the system can decide to acquire and processadditional information only when necessary e.g. only in difficult cases. Thisscheme is referred to as Confidence-based Cue Integration [10].5.8.3 Laser-based Place ClassificationThis section presents our approach to place classification based on geometricfeatures extracted from laser range data. Many places in indoor environmentscan be distinguished due to their different structure. This structure can beunique for an instance of a place, but can also be characteristic for a wholesemantic place category. For example, the bounding box of a corridor is usu-ally longer than that of rooms or hallways. At the same time, rooms aretypically smaller than hallways, and also more cluttered than corridors andhallways. As an example, Figure 5.16 shows a typical hand-labeled division ofan environment into three categories of places.As illustrated in Figure 5.15, the place classification algorithm first extractsa set of simple geometrical features from the scan acquired by the range sensor.Figure 5.17 shows an example of a scan taken by a mobile robot in a corridor.Each feature is represented by a numerical value computed from the beamsof the scan or from a polygon representing the covered area. Single featuresalone are not sufficient for reliable places classification. Here, the AdaBoostalgorithm is used to boost the simple features into a strong classifier. Asis shown in Section 5.9.1, different classification algorithms, such as SupportVector Machines [73], can also be used to successfully derive a place class fromthe geometrical features. A brief description of the features and the AdaBoostalgorithm is given below.5 Semantic Modelling of Space 199Fig. 5.17. A range scan covering the complete 360o field of view acquired by amobile robot in a corridor.Fig. 5.18. A decision list built for K classes using binary classifiers. The output ofeach binary classifier is the probability zk that the classified example belongs to thek-th class.Classification Using AdaBoostThe AdaBoost algorithm, introduced in [74], is one of the most popular boost-ing algorithms. This algorithm takes as an input a training set of positive andnegative examples. On each round, AdaBoost calls a weak learning algorithmrepeatedly to select a weak hypothesis. The key idea is to maintain a weightdistribution over the training examples. This distribution indicates the im-portance of the examples at the beginning of the training process and lateris controlled by the algorithm. Below, a modified version of the original al-gorithm is described which outputs a confidence value for each positive andnegative classification [77].The original AdaBoost algorithm was designed for binary classificationproblems. However, to label places in the environment, we need the abilityto handle multiple classes. One way to construct a multi-class classifier is toarrange several binary classifiers into a decision list. Each element of such alist represents one binary classifier which determines if an example belongsto one specific class. In addition, each binary classifier outputs a confidencevalue C+k for a positive classification of its class k. Figure 5.18 illustrates thestructure of the probabilistic decision list.200 Pronobis et al.In the decision list, each test example is fed into the first binary classifier,which outputs a confidence value C+1 for a positive classification. Then theexample is passed to the next binary classifier. This process is repeated untilthe last element in the list. The complete output of the decision list is repre-sented by a histogram z. In this histogram, the k-th bin stores the probabilitythat the classified location belongs to the k-th class according to the sequenceof classifiers in the decision list. This probability can be computed as follows:zk = C+kk−1∏j=1(1− C+j ), (5.2)where the confidence value for the last K-th bin is equal to 1. In the multi-cue framework, these probability values are used as the outputs which areintegrated by the cue integration function (see Figure 5.15).One important question in the context of a sequential classifiers is theorder in which the individual binary classifiers are arranged. A good strategyis to order the classifiers in increasing order according to their training errorrate. Compared to the optimal order, the classifier generated by this heuristicperformed only 1.3% worse on average for an application with several classesdemonstrated in [78]. In several cases, the sequence generated by this heuristicturned out to be the optimal one.Simple Features from Sensor Range DataLet’s assume that the mobile robot is equipped with a range sensor coveringthe 360o field of view. Each laser observation z = {b0, ..., bM−1} contains a setof beams bi. Each beam bi consists of a tuple (αi, di) where αi is the angleof the beam relative to the robot and di is the length of the beam. Eachtraining example for the AdaBoost algorithm is just one observation z and itsclassification y. Thus, the set of training examples is given asE = {(zi, yi) | yi ∈ Y = {Room,Corridor, . . .}} . (5.3)In this approach, each laser observation is represented by a set of simplegeometric features expressed using single real values. All features are rotation-ally invariant to make the classification dependent only on the (x, y)-positionof the robot and not of its orientation. Most of the features are standard geo-metrical characteristics often used in shape analysis and pattern recognition.We define a feature f as a function that takes as argument one observationand returns a real value: f : Z → <, where Z is the set of all possible ob-servations. Figure 5.19 shows graphically some of these features used. Thecomplete list of features, together with their mathematical definition, can befound in [77].Common configurations on real mobile robots have only one laser scannercovering the 180o in front of the robot. In these cases the values correspond-ing to the rear laser scan can be set to zero. A more advanced solution is5 Semantic Modelling of Space 201Fig. 5.19. Examples of features generated from laser range data, namely the averagedistance between two consecutive beams, the perimeter of the area covered by a scan,and the mayor axis of the ellipse that approximates the polygon described by thescan. Here, the laser beams cover a 360o field of view.to maintain a local map around the robot. This local map can be updatedduring the movements of the robot, and then used to simulate the rear laserbeams [77]. This approximation have shown good results in several indoorexperiments [7, 63].5.8.4 Vision-based Place ClassificationThis section describes the visual place classification algorithms proposedin [12] that constitute the two paths of the vision-based channel in the multi-modal system presented in Figure 5.15. Each of the paths is built around aSupport Vector Machine (SVM) classifier [79] and a different type of visualfeature, global or local, extracted from the same image frame (see Section 5.8.1for the distinction between the feature types). The global features are repre-sented using a rich global descriptor, Composed Receptive Field Histograms(CRFH, [72]). The local features are based on the Scale Invariant FeatureTransform (SIFT, [39]). Both have already been proved successful in the do-main of vision-based localization [10, 37, 66].The rest of the section describes the feature extraction algorithms andsketches the theory behind SVMs which will also form a basis for the cueintegration scheme presented in Section 5.8.5.Global Visual Features: Composed Receptive Field HistogramsCRFH is a multi-dimensional statistical representation of the occurrence ofresponses of several image descriptors applied to the image. This idea is illus-trated in Figure 5.20. Each dimension corresponds to one descriptor and thecells of the histogram count the pixels sharing similar responses of all descrip-tors. This approach allows to capture various properties of the image as well asrelations that occur between them. We tested a wide variety of combinationsof image descriptors with several scale levels. On the basis of an evaluation ofperformance and computational cost, we build the histograms from either firstorder or second order Gaussian derivative filters applied to the illumination202 Pronobis et al.Fig. 5.20. The process of generating multi-dimensional receptive field histogramsshown on the example of the first-order derivatives computed at the same scale t = 4from the illumination channel.channel at two scales. This resulted in either 4- or 6-dimensional histograms.Multi-dimensional histograms can grow extremely fast if the number of dimen-sions grows. However, most of the cells are usually empty [72]. Storing onlythose that are non-zero allows for reducing the amount of required memoryand performing operations such as histogram accumulation and comparisonefficiently.In case of SVMs, special care must be taken in choosing an appropriatekernel function which acts as a similarity measure between the feature vectors.In this work, the χ2 kernel [80] was used for the CRFH descriptors. The χ2kernel belongs to the family of exponential kernels, and is given byK(x,y) = exp{−γχ2(x,y)}, χ2(x,y) =∑i||xi − yi||2||xi + yi||. (5.4)Local Features: Scale Invariant Feature TransformThe process of local feature extraction consists of two stages: interest pointdetection and description. The interest point detector identifies a set of char-acteristic points in the image that could be re-detected even in spite of varioustransformations. The role of the descriptor is to extract robust features fromthe local patches located at the detected points. Here, we used the scale,rotation, and affine invariant interest point detector based on the difference-of-Gaussians (DoG) operator [81] and the SIFT descriptor [39]. Figure 5.21presents local patches located at the interest points detected in three typicalimages acquired in an indoor environment.In case of local features, the similarity between two images is measuredby solving the correspondence problem. Thus, in order to couple the local5 Semantic Modelling of Space 203Fig. 5.21. Local patches at the interest points detected in three typical imagesacquired in an indoor environment. The size of the patches illustrate the scale atwhich the points were detected.descriptors with the SVMs, the match kernel proposed in [82] was used. Thematch kernel is given byK(Lh,Lk) =1nhnh∑jh=1maxjk=1,...,nk{Kl(Ljhh ,Ljkk )}, (5.5)where Lh,Lk are local feature sets and Ljhh ,Ljkk are two single local features.The sum is always calculated over the smaller set of local features and onlysome fixed amount of best matches is considered in order to exclude outliers.The local feature similarity kernel Kl can be any Mercer kernel. Here, theRBF kernel based on the Euclidean distance was used for the SIFT features:Kl(Ljhh ,Ljkk ) = exp{−γ||Ljhh −Ljkk ||2}. (5.6)Support Vector MachinesSupport Vector Machines are a binary discriminative classifier known for theirsuperior generalization abilities. Consider the problem of separating the setof labeled training data (x1, y1), (x2, y2), . . . , (xn, yn) into two classes, wherexi ∈ . However, in order to satisfy thiscondition, the robot will have to plan a sense-door sensing action which,when executed, will provide the robot with the information really needed, i.e.the real value of doorstate(?d)?a, which it can then use in the next planningphase to concretise its plan for exploring the appartment.6.4 Probabilistic Monitoring of Dynamic ProcessesIn this section, we introduce our approach to monitoring low-level processes inthe robotic system. This provides the basis for robustly verifying and refutingassertions made by the planning system and it provides (symbolic) statusinformation about processes and low-level tasks to the planner allowing it toreact appropriately.In our approach, the system to be monitored is modelled as a dynamicBayesian network (DBN) [2] and its state is estimated sequentially usingsampling-based estimators in conjunction with learned sampling functions.As the main contribution, which goes beyond the application to the processmonitoring task, we present a novel way of improving the generation of statesamples in particle filters. This step has to be performed frequently and it hasa major influence on the efficiency and robustness of the filter. Specifically,we show how Gaussian processes (GPs), that is, a state-of-the-art Bayesianapproach to regression and classification, can be used to incorporate priorknowledge into the state estimation process.DBNs are widely used to model the dynamic behavior of artifical and natu-ral systems. In a DBN, the variables of interest are indexed by time and relatedto one another using conditional probability distributions (CPDs), which ingeneral span multiple time frames to describe the dynamics. The result is a(typically sparse) network of local dependencies for which efficient learningand inference mechanisms have been developed. A particularly flexible and,238 Brenner et al.thus, often-used inference technique are the so called particle filters (PFs).They represent the state of the system by a finite set of weighted samplesand perform inference by updating the sample set iteratively using the DBNstructure, the conditional models, and the set of observed (thus constant)variables.Our primary goal is to improve the generation of state samples in theparticle filter algorithm [3], which is a step that has to be performed fre-quently and that has a major influence on the efficiency and robustness of thefilter. In particular, we consider data-driven sampling, i.e. the generation ofstate hypotheses at locations that are most promising given the latest sensormeasurements. Our major contribution is, that the sampling policy is learnedfrom past experience, using Gaussian process (GP) regression for continuousstate variables and classification for discrete ones. Given the GPs ability toalso estimate the uncertainty of its predictions, we can derive a sound wayof integrating these informed proposals into the particle filter framework. Asa result, the estimation process is more robust since the sampling density ishigher in the important parts of the state space while it is also more efficientbecause less samples are “wasted” in unlikely regions.As a running example and test-bed for our approach, we consider theproblem of online failure detection and recovery for a mobile robot. Concretely,we face the task of (a) localizing an autonomous, wheeled robot relative toan environment, (b) to detect collisions with (unseen and movable) obstaclesonline and (c) to estimate their physical parameters such that they can bedealt with safely.In the following, we first formalize the sequential state estimation problemand describe how to solve it using particle filters. We then introduce theconcept of Gaussian process proposals and specifically focus on the mobilerobot localization and collision detection problem outlined above.6.4.1 Sequential State EstimationOne of the fundamental problems in robotics and engineering is to estimatethe state of a system given a sequence of acquired sensor measurements andaction commands that have been executed. In order to perform this tasksequentially, i.e. in an iterative way whenever new information is available,the system is typically modeled using a dynamic Bayesian network. Assuminga discretization of time—either by modeling iterations of equal duration or bytaking the finite set of measurement and action events—, there is a finite set oftime-indexed variables that has to be modeled. The DBN factorizes the jointposterior over these variables to a (typically sparse) network of dependentvariables and their corresponding conditional probability distributions. Here,these ’local models’ can be limited to a certain time frame or they can spanseveral to model the behavior of the system over time.6 Planning and Failure Detection 239Fig. 6.2. DBN for the mobile robot domain.Mobile Robot LocalizationIn the case of mobile robot localization, for instance, the (dynamic) state ofthe system is the three-dimensional pose st = xt = (xt, yt, θt) of the robot rela-tive to its environment. Additional variables include the sensor measurementsz1:t as well as the action commands u0:t−1 that have been issued—w.l.o.g. as-suming time-synchronous state, action, and measurement variables for ease ofnotation. A typical DBN for such a system is composed of a prior distributionover poses p(s0), a transition model p(st+1|st, ut) and an observation modelp(zt|st). Figure 6.2 shows how these CPDs are connected to yield the typicalDBN for state estimation problems. It can be read from the graph, that eachvariable is independent from all others given its direct predecessors (see [2] fora general discussion of graphical models). Given the modeled independenceassumptions, the state posterior can be formulated recursively asp(st | z1:t, u0:t−1) =∫p(st | st−1, zt, ut−1) p(st−1 | z0:t−1, u0:t−2) dst−1 (6.1)∝ p(zt | st)︸︷︷︸obs. model∫p(st | st−1, ut−1)︸︷︷︸transition modelp(st−1 | z0:t−1, u0:t−2)︸︷︷︸recursive termdst−1 .In order to achieve a tractable approximation of this equation, one has to makeadditional assumptions about the distributions involved. Important classes ofsuch approximations includeDiscrete filters (DF) where the state space is discretized—typically using aregular grid—which turns the integral in the equation above into a sumover possible predecessor states,Kalman filters (KF) for which the distributions over states p(st| . . .) and theobservation model are assumed Gaussian and the transition model is lin-ear, the extended Kalman filter (EKF) for which the assumption of lin-earity is relaxed, andParticle filters (PF) which represent the distributions over states by a weightedset of state hypotheses which are manipulated individually according tothe transition and observation models.Particle filters are flexible in terms of distributions that can be represented,they are easy to implement, and they can convert heuristics into provablycorrect algorithms through the concept of proposal distributions (see Chapter 5in [2]).240 Brenner et al.6.4.2 Particle Filters for Nonparametric Bayesian FilteringParticle filters seek to approximate the integral in Eq. (6.1) using Monte Carlointegration. Concretely, they represent distributions over states p(st| . . .) bysets of weighted samples Xt = {〈s[i]t , w[i]t 〉}ni=1. Starting from a prior distri-bution X0 ∼ p(s0), the particle set is updated sequentially by executing thefollowing steps in each iteration:1. Sampling (or motion update): The next generation of particles {s[i]t } isobtained from the generation {s[i]t−1} by sampling from a proposal distri-bution π.2. Importance Weighting (or observation update): Importance weights w[i]tare assigned to the individual particles according tow[i]t =p(s[i]1:t | z1:t, u1:t)π(s[i]1:t | z1:t, u1:t)∝p(zt | s[i]t )p(s[i]t | s[i]t−1, ut)π(st | s[i]1:t−1, z1:t, u1:t)· w[i]t−1 (6.2)The weights account for the fact that the proposal distribution π is ingeneral not equal to the target distribution of successor states. It can beshown that for all proposal functions π with p(x) > 0 ⇒ π(x) > 0 andthe weight update (6.2), the PF algorithm is guaranteed to approximatethe correct posterior [3].3. Resampling: Particles are drawn with replacement proportional to theirimportance weight.In the special case of π ≡ p(st|st−1, ut), that is if sampling is performed ac-cording to the transition model, the relationship between Eq. (6.1) and thefiltering procedure can directly be seen and the recursive weight update (6.2)simplify to a multiplication of all particle weights with the respective obser-vation likelihoods p(zt|s[i]t ). For a formal derivation of the particle filteringprinciple, see for example [12] or [3]. Note that resampling exchanges ’likeli-hoods’ by ’frequencies’ without altering the represented distribution. In thelimit of infinitively many and densely-sampled particles, the representation ofa distribution by non-constant importance weights is equivalent to the onewhere all weights are equal, but the particle density varies accordingly. Thus,the purpose of the resampling step is to distribute more particles to the prob-able areas of the state space and less to the improbable ones.The robustness and efficiency of particle filters strongly depends on theproposal distribution π that is used to sample the new state hypotheses inthe sampling step. If the proposal distribution differs too much from the trueposterior, there is a high risk of filter divergence. On the other hand, if π istoo flat (e.g., uniform in the extreme case) many particles are wasted.In the following section, we extend the mobile robot DBN to also include adiscrete failure mode variable and additional continuous variables for the fail-ure’s parameters. As this increases the dimensionality of the state space andalso adds highly unlikely transitions (failures occur only rarely), the transition6 Planning and Failure Detection 241Fig. 6.3. Extended DBN to also include the discrete failure mode f and the con-tinuous failure parameters o. With st = (ft, ot, xt), we denote the entire state of thesystem to highlight the relation to the standard model depicted in Fig. 6.2.model becomes a sub-optimal choice as proposal distribution. For this reason,we then introduce an informed proposal distribution, which utilizes informa-tion about the failure mode contained in the most recent sensor measurementto yield better pose estimates.6.4.3 Modeling the Influence of Failures using Hybrid DBNsIn realistic environments there typically exist external influences that cancause drastic changes to the behavior of a dynamic system. Consider for ex-ample a mobile robot colliding with an undetected obstacle while executing amotion command. Such situations can be modeled by extending the DBN by adiscrete failure mode variable ft and additionally continuous failure-dependentstate variables ot (see Fig. 6.3). With st = (ft, ot, xt), we denote the entirestate of the system to highlight the relation to the standard model depicted inFig. 6.2. We assume the commonly used constant failure rate model (see [13]),in which failure events do not depend on the other state variables, but ratheroccur according to an individually specified prior distribution. Since the ob-servations are independent of the failure state (ft, ot) given the pose xt of therobot, the observation model remains p(zt | st). The state transition modelcan be factorized as p(xt, ft, ot | xt−1, ft−1, ot−1, ut−1) =p(ft | ft−1)︸︷︷︸failure event model· p(ot | ft, xt−1, ot−1)︸︷︷︸failure parameter model· p(xt | ot, xt−1, ut−1)︸︷︷︸transition model. (6.3)Note that this is a direct translation of the graphical model depicted in Fig. 6.3and that we denote with st = (ft, ot, xt) the state of the system includingfailure variables and the pose of the robot.The constant failure rate model states that failure events are distributedexponentially depending on a failure rate parameter λ, i.e.242 Brenner et al.p(ft) = 1− e−λ·(t−t̃) , (6.4)where t̃ denotes the time of the last failure event. For such a model, the meantime between failures becomes MTBF = 1λ . For realistic failure rates λ, thismodel results in extremely low failure probabilities per filter iteration. Assume,for example, a mean time of 30 minutes between collisions of a service robotwith unseen obstacles. This implies λ = 11800s = 0.0005 and with a filterfrequency of δt = 0.1 seconds yields a failure probability of p(ft | ¬ft−1) ≈0.000056 within one iteration. For such a small value, just one of 20, 000particles would be sampled to a failure mode on average, if the transitionmodel was used directly as proposal distribution for this variables. Thus, onewould either need an extremely large particle set or would risk that failuresremain undetected. This problem is amplified by the fact that not only thediscrete failure mode has to be sampled, but also the unknown continuousfailure parameters. Since in general, there is no prior knowledge about theparameters of randomly occurring failures, we assume a uniform distributionp(ot | ft, xt−1, ot−1) = U[omin,omax](ot) (6.5)over a certain interval. Note that this model applies only to the case wherethe system transitions into a failure state. The evolution of failure parameterswithin a failure state is typically governed by a much more peaked distributionsimilar to the motion model of the robot. In Section 6.6, we describe our modelfor the evolution of collision parameters based on rigid body dynamics. Thismodel is able to track collisions sufficiently accurate, if the initial collisionparameters have been estimated well enough. To achieve this, we introduce aninformed proposal distribution in the next section, which utilizes informationabout the failure mode contained in the most recent sensor measurement.6.5 Gaussian Processes Proposals for Failure EventsIf system models containing states with extremely low a-priori probabilitiesare to be estimated using particle filters, it is obvious that the state transi-tion model should not be employed as the proposal distribution directly (seethe numerical example in the previous section). To address the problem oflow sampling probabilities for important parts of the state space, [14] intro-duced the risk sensitive particle filter (RSPF) that incorporates a learned riskfunction to force the filter into less likely but important states. While this ap-proach ensures a reasonable amount of samples in the important failure modes,it cannot adapt to the specific situation the robot is in when the samplingdecision is made. In contrast, we propose to use learned proposal distributionsthat provide informed guesses about what the states with higher a-posteriorprobabilities are.6 Planning and Failure Detection 2436.5.1 Data-driven Proposal DistributionsIn sequential importance sampling [15], an arbitrary proposal distributioncan be used to sample the relevant areas of the state space as long as (a)all possible states have a non-zero possibility of being chosen and (b) theimportance weights of the particles are adjusted appropriately. Proposal dis-tributions that depend on the most recent sensor measurements or on featuresextracted by separate algorithms are typically denoted as data-driven propos-als or detector-mediated proposals [16]. Such proposals aim at approximatingthe optimal proposal p(st | st−1, zt, ut−1) which includes the most currentsensor measurement zt. It can be shown [2] that such a fully informed pro-posal minimizes the variance of the importance weights making it the optimalchoice.Assuming that the proposal π(st | st−1, zt, ut−1) in the failure detec-tion scenario also factorizes according to our transition model (6.6), i.e.,π(xt, ft, ot | xt−1, ft−1, ot−1, ut−1) =πf (ft | ft−1)︸︷︷︸failure event proposal· πo(ot | ft, xt−1, ot−1)︸︷︷︸failure parameter proposal·π(xt | ot, xt−1, ut−1)︸︷︷︸state proposal, (6.6)then the weight for particle i at time t becomesw[i]t =p(s[i]1:t | z1:t)π(s[i]1:t | z1:t)=p(zt | s[i]1:t, z1:t−1) p(s[i]1:t | z1:t−1)p(zt | z1:t−1)︸︷︷︸=:1/ηπ(s[i]1:t | z1:t)= η ·p(zt | s[i]t ) p(s[i]t | s[i]t−1)π(s[i]t | s[i]1:t−1, z1:t)·p(s[i]1:t−1 | z1:t−1)π(s[i]1:t−1 | z1:t−1)︸︷︷︸=w[i]t−1= η · w[i]t−1 · p(zt | s[i]t ) ·p(x[i]t | o[i]t , x[i]t−1)π(x[i]t | s[i]1:t−1, z[i]1:t)·p(f [i]t | f[i]t−1)πf (f[i]t | s[i]1:t−1, z[i]1:t)·p(o[i]t | f[i]t , x[i]t−1, o[i]t−1)πo(o[i]t | s[i]1:t−1, z[i]1:t). (6.7)Here, we left out the control variables u for brevity. The normalizing factorη is constant for all particles i. πf denotes the proposal for the failure eventand πo the one for the failure parameters.We now propose to learn πf and πo from data. Formally, the task is tolearn a mapping from a feature vector Ft extracted from 〈s[i]1:t−1, z[i]1:t)〉 to thevariables ft and ot. Any feature vector Ft as well as any learned models πfand πo can be used as a proposal as long as the assumptions• πf (f | Ft) 6= 0 for all f with p(f | s1:t, z1:t−1) 6= 0244 Brenner et al.• πo(o | ft, Ft) 6= 0 for all o with p(o | s1:t, z1:t−1) 6= 0.hold, which means that all possible failure states are assigned a non-zero prob-ability of being chosen. Visually speaking, Equation 6.7 states that after eachfilter iteration, the particle weights have to be multiplied with the currentobservation likelihood p(zt | s[i]t ) and with two correction terms for the twolearned proposal distributions. To calculate these correction terms for a spe-cific sample s[i]t , we divide the probabilities defined in Equations 6.4 and 6.5by the likelihoods according to which the state variables f [i]t and o[i]t havebeen drawn from πf and πo. Another precondition for the learned propos-als therefore is the availability of likelihoods for sampled values. As we willsee in the next section, Gaussian processes always ensure the non-zero prob-ability assumptions, can easily be sampled from, and provide the transitionprobabilities needed for the weight correction described above.6.5.2 Learning Sampling Models from DataA major benefit of using Gaussian processes for learning and representingπf and πo is that by the availability of predictive uncertainties, the learnedproposal distributions can directly be used in the sampling-based estimationscheme. The estimated state variables can be sampled directly from the Gaus-sian Processes. In the classification as well as the regression case.In our concrete scenario, we found that the discrepancy between the cur-rent motion estimate (velocity and heading) w.r.t. an additional source (e.g.a laser scan-matcher or an inertial sensor) provides a good feature for learn-ing proposals for the hidden failure states. Concretely, we take the extractedfeatures Ft as input vectors x and failure parameters ot as targets y. Givena training set of such quantities and a properly chosen covariance functionk, we can compute the predictive distribution πo(ot | ft, Ft) := N (µ, σ2) asdetailed, for example, in [17]. This (normal) distribution naturally meets therequirements for proposal distributions named in the previous section. It canbe sampled from directly, it supplies the likelihoods of sampled values, it hasan infinite support, and it therefore only assigns non-zero likelihoods.An important aspect that has been left out so far is the choice of covariancefunction and how its parameters can be set. The covariance function playsan important role in the Gaussian process framework as it represents theprior knowledge about the underlying function g. By changing its form andparameters, one can control the generalization behavior and smoothness ofthe predictor. A common choice of covariance function (see [18]) that is alsoused in this work isk(x,x′) := a0 + v0 · exp(−12m∑d=1wd(xd − x′d)2)with a constant component and a nonlinear, stationary term, which dependsonly on the distance between input points. The parameters of the covariance6 Planning and Failure Detection 245-0.15-0.1-0.05 0 0.05 0.1 0.15 0.2 0.25 0.3-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5Contact point [m]Diff vr [m/s]Regression resultTraining data (diff vt = -0.4 +/- 0.1)Fig. 6.4. The collision event training set, in which two feature values are mappedto class instances (top left), the learned class probabilities πf (Ft) using Gaussianprocess classification (top right). The learned regression model for the collision pa-rameters visualized by a cut through the two-dimensional pdf that maps velocitydeviations to contact points for a collision (bottom).function are called hyperparameters of the Gaussian process. They can eitherbe fixed by maximizing the likelihood of the given data points or, for fullyBayesian treatment, can be integrated over using parameter-specific prior dis-tributions. For our experiments reported in Section 6.6, we employed the latterstrategy with Gamma priors on the hyperparameters. We set these priors tofavor smooth regression functions to avoid overfitting to the training data.The analytically intractable integration over the hyperparameters is approxi-mated using Markov Chain sampling and the prediction results are cached ona fine-grained grid.Binary classification problems can consistently be modeled in this frame-work by including for every binary target ti a real-valued latent variable li,such thatp(ti = 1) =11 + e−li, (6.8)which is known as the logistic model that links class probabilities to real val-ues, see [19]. The latent variables can now be given a Gaussian process prior asin the regression setting and predictions of class probabilities can be performed246 Brenner et al.by predicting the corresponding latent variables and evaluating Equation 6.8.For the failure detection problem, we again use feature vectors Ft as inputsand binary failure labels ft as targets. The predicted class probabilities fornew features then directly define the failure event proposal πf (ft | Ft).6.5.3 Predicting Collision Events and ParametersAs discussed in Section 6.4.3, typical failure rates result in extremely lowfailure event probabilities. We therefore propose to train a Gaussian processclassifier πf for predicting failure events online. The learning task can besimplified by not taking the raw sensor measurements and state variables asinputs to the learner directly, but choosing lower dimensional features of theseinstead. In our failure detection setting, where we aim at detecting collisionswith unseen obstaclesIn this section, we apply the sequential failure detection approach de-scribed above to the problem of collision detection for mobile robots undernoisy sensor measurements and without additional hardware like bumpers orinertial sensors. For learning the collision event proposal and the proposal forthe contact point of the robot with the obstacle, we achieve excellent resultswith simple features based on the rotational and the translational velocityof the robot. Thus, we use as input to the Gaussian process classifier thetwo-dimensional feature vector Ft = (∆vt,∆vr), where ∆vt is the differencebetween the translational velocity estimated by the particle filter and the oneestimated by local laser scan matching. Furthermore, ∆vr is the differenceof the rotational velocities respectively. A training set of 500 automaticallylabeled trajectories was generated by simulating random collisions with dif-ferent obstacles using the 3D simulator Gazebo [20]. The top left diagram inFig. 6.4 shows the gathered data points, the middle diagram in the same figureshows the learned class probabilities depending on the two velocity differencesdescribed above. As can be seen from the left diagram, ∆vt is negative fornearly all “collision” data points, which corresponds to the fact that the robotis slowed down when a collision occurs. The data points for “no collision” arespread widely and do not separate well from the “collision” data points dueto noisy sensor measurements and imperfect labeling of the collision events.This makes the classification problem a hard one. It should be stressed, thatwe use this classifier as a proposal distribution for collisions rather than asa collision detector directly, because the features are too ambiguous to allowfor perfect instantaneous classification. Experiments with a real robot (seeSection 6.6) showed that this yields high detection rates with a low numberof false alarms.Given a collision event, the continuous collision parameters o have to beestimated to simulate the effects on the system and to continue the trackingprocess. Since the task is not to fully track the pushed obstacle over time,a simple model that abstracts from the obstacle’s geometry and exact posehas proven sufficient to describe the effects on the robot. A collision with an6 Planning and Failure Detection 247unseen obstacle is represented by the obstacle mass m and the contact pointc on the front of the robot. Therefore, the collision parameters are o = (m, c).We learn the proposal distribution πo(ot|Ft) for the parameters ot using thesame velocity-based features and simulated training set as described aboveand the Gaussian process regression technique. The lower diagram of Fig. 6.4depicts a cut through the learned 2-dimensional distribution for the collisionparameter c, which is the contact point of the obstacle on the front of therobot. The point of contact is measured in meters from the center of therobot’s front to the right. It can be seen from the diagram that unexpectedclockwise rotations (∆vr < 0) of the robot are mapped to positive values forthe contact point, which corresponds to a collision on the righthand side ofthe robot.6.6 Implementation of Sensor-level MonitoringOur failure detection system described in the has previous two sections hasbeen implemented on a real robot and was tested in an office environment.Before presenting experimental results, we describe the motion model we im-plemented for our localization system. The most widely used motion modelfor mobile robots is based on the wheel encoder measurements (see [12]). Thisinformation, rather than the actual control command, is taken as control in-put ut−1, which under normal circumstances results in accurate predictionsof the performed movement. Under the influence of failures like collisions orwheel slip, however, the motion of the wheels is not consistent with the wholerobot’s motion any more. A more appropriate model for such situations thatstill is efficient enough to be evaluated online is based on simple rigid body dy-namics (see [21]). We model the robot as a rigid body in the (two-dimensional)plane, represented by a set of constant values and the variable state vectorxt = (posx, posy, posθ, velt, velr) which includes the translational velocity veltand the rotational velocity velr. In each filter iteration, the wheel thrusts arecalculated from the actual velocity command that was sent to the motors.From this, the next state vector is computed by numerical simulation us-ing the physical relationships between forces, acceleration, and speed. Due tospace limitations, we refer to [22] for details about rigid body physics. Withthis model, collisions with another rigid object at a given point of contactcan be simulated using the same type of physical abstraction, namely com-puting the impulse, the resulting forces, and ultimately the influence on therobot’s state vector. At the same time, this model describes how the point ofcontact between the robot and the obstacle changes over time and thereforedefines the transition model for failure parameters of Equation 6.6. From ourexperience, this physical model achieves the best balance between accuracyand efficiency. Simpler models fail to handle important test cases while morecomplex models have too many free parameters to be evaluated in real time.248 Brenner et al.Fig. 6.5. Results of our experimental evaluation with a real robot colliding with un-detected obstacles. Left: Two correctly detected collisions (triangles), the estimatedtrajectory (solid line), and the ground truth (dotted). Middle: Average deviation ofthe estimated trajectory from the ground truth in meters for a varying number ofparticles. Right: Estimation details around a correctly detected collision includingthe manually labeled true collision event. One filter iteration corresponds to 0.1seconds.6.6.1 EvaluationTo quantitatively evaluate the usefulness of our approach to failure detection,we compared it to a particle filter that implements the same process modelwith the standard uninformed proposals described in Section 6.4. The param-eters of the standard filter were optimized for best tracking performance andfailure detection rates to ensure comparability. We recorded data by manuallysteering the robot through the environment and arranged for two collisions,one with boxes of milk and the other one with a box of lemonade bottles.Both obstacles were placed at arbitrary positions and the obstacle heightswere too low for the laser sensor to detect them. The left plot in Fig. 6.5 de-picts a typical test run, in which our system successfully tracked the pose ofthe robot and detected the two collisions. On the recorded data set, we testedour improved particle filter with Gaussian process proposals (GP-PF) as wellas the standard particle filter (Std PF) for different parameter settings. Eachfilter was executed 50 times for each parameter setting.Figure 6.6 gives the failure detection performance of the different filters.The detection rate is defined as the number of correctly identified failuresrelative to the full number. The false positives rate is the amount of falsealarms relative to the number of detections. The ground truth collision eventswere manually entered and a collision was counted as correctly detected, whenthe marginal failure likelihood exceeded a threshold Θ after a maximum ofsix filter iterations (0.6 seconds) after the true failure event. The threshold Θwas optimized independently for each filter to ensure unbiased comparison.The middle diagram in Fig. 6.5 gives the average deviation of the trackedposes of the robot compared to the ground truth trajectory. The ground truth6 Planning and Failure Detection 249 0 10 20 30 40 50 60 70 80 90 100[Percent]STD-PF150 particlesSTD-PF950 particlesGP-PF50 particlesGP-PF300 particlesDetection RateFalse PositivesFig. 6.6. Detection results with the optimized standard particle filter (STD-PF)and our approach that uses Gaussian process proposals (GP-PF).trajectory was computed using a scan matcher. The results visualized in thisdiagram show that our system stays around ten centimeters closer to the truetrajectory and produces less variance in these estimates than the standardapproach. This is mainly due to the fact that the failure parameter (here, thepoint of contact with the obstacle) is estimated more accurately. To give animpression about the accuracy with which our filter estimates the point ofcontact and thereby the path of the robot, one failure event is depicted indetail in the diagram of Figure 6.7. It can be seen, that the estimated failurelikelihood increases shortly after the labeled failure event and that the headingangle of the robot is correctly estimated.The detection rates as well as the tracking results show that learned Gaus-sian process proposals can indeed increase the reliability and efficiency ofonline state estimation approaches. The time requirements for the improvedparticle filter are around 10% to 15% higher than for the standard implemen-tation without Gaussian process proposals. Nevertheless, the implementedsystem with 200 particles still processes one minute of recorded data in lessthan 23 seconds on a PC with a 2800 MHz CPU.6.7 Continual Collaborative PlanningIn the previous sections, we have considered the dynamics of the world out-side the agent mainly as a source of unforeseeable, uncontrollable events thatrequire careful monitoring, failure diagnosis and, possibly, replanning. How-ever, quite often the dynamics of an environment is due to other agents. Inthis case, an agent may, instead of just trying to react to their actions, try toactively interact with these agents to learn more about the current world state250 Brenner et al.-0.2 0 0.2 0.4 0.6 0.8 1 1.2 75 80 85 90 95 100 105Filter iterationEstimated headingTrue headingCollision likelihoodTrue collisionEstim. contact pointFig. 6.7. Measurements and filter estimates during a typical collision event. Betweeniteration 90 and 95, the robot hits an undetected obstacle.or even negotiate future plans with them. Such proactive multiagent planningis similar in spirit to the techniques for active infomation gathering presentedin the earlier sections of this chapter.For Continual Collaborative Planning (CCP) we must incorporate the Con-tinual Planning algorithm (Algs. 2 and 3) into a distributed algorithm thatallows communication between agents. The basic CCP algorithm is shownin Alg. 4. As is customary in Distributed Systems, the current state of thealgorithm depends on the messages received from other agents [23].As long as no message is sent or received the basic Continual Planningalgorithm is executed as discussed above. However, during his individual CPprocess the agent may now also ask others to achieve subgoals. This arisesnaturally whenever an agent devises a plan that includes actions by anotheragent. Instead of executing such an action himself, the planning agent mustsend an appropriate request to the other agent. The planning agent will keepa whitelist for requests that have been accepted and a blacklist for those whichhave been denied.To incorporate the generation of speech acts into continual planning, weslightly extend Alg. 3 to Alg. 5. The only visible changes occur in steps 4-5 in which requests are determined and communicated. Note, however, thatalready step (3), i.e. the execution of a planned action, may refer to a commu-nicative action. For example, MAPL domains generally define tell-val actions,i.e. actions for informing another agent about a state variable value. If anagent receives such a speech act (step 4 of Alg. 4) it adds this information toits knowledge base, i.e. CCP assumes the sincerity of other agents.The specific request is computed by the function SelectBestRequestin steps 4 of Alg. 5. This is a black-box function whose realisation may differ6 Planning and Failure Detection 251Algorithm 4 CCP Agent(S,G)P = ∅Received no message:if S satisfies G doreturn “goal reached”P = MonitoringAndReplanning(S, G, P )if P = ∅ thenreturn “cannot achieve goal G”(S, P ) = ExecutionAndStateEstimation(S, P )Received request(e) from agent a:sg = TranslateRequestToGoal(e) (1)P = MonitoringAndReplanning(S, sg, ∅) (2)if P = ∅ thensend “cannot execute request e” to aelsesend “accept request e” to aadd sg to G as temporary subgoal (3)Received (tell-val vx) from agent a:add v = x to S (4)Received “cannot execute request e” from agent a:add e to blacklist (5)Received “accept request e” from agent a:add e to whitelist (6)Algorithm 5 ExecutionAndStateEstimation(S, P )e = choose a first level event from PS′ = app(S, e) (1)exp = ExpectedPerceptions(S′, Esense) (2)if agt(e) = self thenexecute(e) (3)elsee′ = SelectBestRequest(e, P, blacklist,whitelist) (4)send request(e′) to agt(e) (5)perc = GetSensorData() (6)if perc ⊇ exp then (7)remove e from PS = Fuse(S′, perc) (8)return (S,P)between CCP implementations and applications. SelectBestRequest cantake into account the whitelist and blacklist to determine requests that havenot already been accepted or refused or which would be subsumed by suchearlier requests.252 Brenner et al.SelectBestRequest also takes into account not only the event in ques-tion itself, but also its context in the agent’s plan: If the plan contains severalactions by another agent, i.e. a whole subplan, it is often best not to requestexecution of the actions individually, but to ask for the achievement of its endresult. This will give the executing agent the liberty to find the best way tocombine execution of the request with its individual goals. In other situationsor applications, in contrast, it might be crucial for the requesting agent thatits original plan is executed as precisely as possible.Since the decision about the most appropriate request is highly dependenton the domain of application and, in particular, the form of cooperativityamong agents, the general CCP framework does not commit to a specificalgorithm for SelectBestRequest. Our standard implementation uses asimple algorithm that determines the maximal asynchronous subplan of Pthat uses only one specific agent. Then SelectBestRequest will requestall actions on the final level of this plan. However, we are currently investi-gating variants that determine where it is reasonable to “outsource” largersubproblems to groups of other agents.CCP process flow.CCP agents are first and foremost individual CP agents. Collaboration amongagents arises only when one CP agent includes actions by another agent intohis plan, i.e. when it wants to be helped. Whether collaboration is generallydesirable or only a last resort largely depends on the domain of application:sometimes collaboration can raise a plan’s utility dramatically, e.g . becauseof concurrent execution, but often the need for synchronisation and negotia-tion outweighs these advantages. We abstract from this issue here and assumean appropriate cost model that Planner can exploit exists for each plan-ning domain, e.g . one that assigns additional costs to actions intended to beperformed by other agents.When an agent receives a request to perform an action, it first translatesthis request to a new subgoal (step 1 of Alg. 4). If the request correspondsdirectly to a grounded MAPL action, the subgoal is determined by the effectsof this action. However, we want to allow for other forms of requests, too; inparticular we want to be able to model requests posed by human users, e.g .during Human-Robot-Interaction. The reason for translating such requeststo goals is twofold: What matters to the requesting agent is usually not theexact action, but the end result achieved, i.e. the achievement of a goal or ofa precondition for a subsequent action of the requesting agent.4 Additionally,4 This is also where assertions play a central role again. By referring to an assertionin a request, an agent a can ask another agent b for some behaviour even withoutknowing what b knows about the current situation, i.e. a does not need to reasonabout the abstraction level of b’s ensuing continual planning process. Instead, btranslates the request to a goal corresponding to the effects of the assertion andwill then use the requested assertion, another one or none at all, depending onits own degree of knowledge.6 Planning and Failure Detection 253agents may want to use referring expressions in their requests, because theycannot make an unique name assumption for objects they talk about. Thisis of particular importance when interaction happens in natural language,e.g . in Human-Robot-Interaction. Referring expressions can be modelled asconstraints on the goal state to achieve [24]. The translation process has beendescribed in our previous work [24]. In a nutshell, it maps a request to theeffect of the corresponding operator scheme. If referring expressions are used,a more complex formula describing the constraints on the operator parametersis produced.In the basic form of CCP described here, agents will check whether theycan achieve the new subgoal (step 2 of Alg. 4) and, if possible, are alwayswilling to adopt it (step 3). If the request is conflicting with their previousgoals, they will reject it. The requesting agent will then add this request toa blacklist which is taken into consideration when requests are chosen again(step 4 of Alg. 5).In practice often a more complex strategy of subgoal acceptance will benecessary. For instance, we have experimented with fixed and dynamic hierar-chies among agents where higher-ranking agents will not accept requests fromlower-ranking ones. Evaluating the resulting CCP variants is an importanttopic for future work. Our aim in this book, however, is to introduce CCPas a general framework for collaborative planning; we therefore restrict ourattention to the setting with maximal cooperativity among agents.Temporary subgoalsA request is adopted as a temporary subgoal (step 3 of Alg. 4), i.e. the agentcommits to achieving it at some point during the CCP run, but it need notnecessarily stay true later on. Temporary subgoals (TSG) are not first-classconstructs in MAPL; instead we exploit the fact that new goals and actionscan be added to the planning domain on-the-fly during CP. Essentially, foreach TSG sg a unique constant csg is generated and (achieved csg) is addedto G as a conjunct. Then an artificial action (achieve csg) is created whoseprecondition is sg. Since only (achieve csg) can achieve (achieved csg) this en-forces the planner to satisfy the precondition, i.e. sg. However, once (achievedcsg) has been made true sg can become false again without preventing G frombecoming satisfied. Thus sg is only enforced to be temporarily true.Requests and TSGs add another important element to CCP: just as thebeliefs of an agent are continually revised during the CP process, the use ofTSGs lead to continual goal revision. Although Alg. 4 only adds new TSGs toG, we can in practice remove TSGs that are already satisfied from G. DuringCCP, agents thus try to achieve continually expanding and shrinking goal sets.254 Brenner et al.6.8 MAPSIMPlan execution and plan monitoring are essential parts of Continual Planningalgorithms. Therefore continual planning must, in contrast to classical plan-ning, be implemented and tested in some sort of “reality”. Providing such areality, e.g . a simulation, is not a trivial task. Being able to provide realitiesin a domain-independent way, such that continual planning can be evaluatedfor arbitrary planning domains, is even harder. This might be a reason forthe comparatively few systematic approaches to continual planning in theliterature (cf. Sec. 6.10).For evaluating CCP across varied MAPL domains and scenarios, we havedeveloped a simulation testbed for multiagent planning, called MAPSIM.MAPSIM is a simulation generator that automatically transforms MAPL do-mains into multiagent simulations. MAPSIM parses and analyses a MAPLdomain and turns it into perception, action, and communication models forCCP agents. During the simulation, MAPSIM maintains and updates theglobal world state and it uses the sensor models to compute individual andjoint perceptions of agents. In other words, MAPSIM interprets the planningdomain as an executable model of the environment. Thus, MAPSIM allowsdesigners of DCP algorithms to evaluate their approaches on various domainswith minimal effort.Agents interact with MAPSIM by sending commands that directly de-rived from MAPL actions selected for execution during CCP. The simulatorthen executes the action, i.e. it checks the preconditions and applies effectsas specified in the MAPL domain. If the controlling agent of a command isnot identical to the agent who sent it to the simulator this is interpretedas a request which, of course, is not directly executed but passed on to thecorresponding agent. MAPSIM also accepts some specific commands for ac-knowledging subgoal acceptance and subgoal achievement. When a commandcorresponds to a speech act, it is “heard” by the addressees, i.e. its effectsare added to the beliefs of the addressees. In short, MAPSIM allows agentsto directly implement Alg. 4 without having to care about the internals of ofthe “world” they are acting in. However, MAPSIM does not enforce the useof CCP. Agents can use arbitrary deliberative or reactive methods to deter-mine their behaviour and their reactions to requests. We believe that this canmake MAPSIM a valuable evaluation tool even when a (distributed) continualplanning algorithm is used that differs significantly from CCP.Interaction is very common during CCP runs in MAPSIM: agents planand execute speech acts in order to gather information, negotiate requestsand ensure self-synchronised plan execution by informing each other aboutstate changes. Thus, CCP can be regarded not only as multiagent planningapproach, but also as a model of collaborative, situated dialogue. Fig. 6.8shows parts of one such interaction in a scenario where one agent, Anne,wants another, R2D2, to bring her coffee. Since it is hard to study dialoguebehaviour in different variants of CCP based only on the logging format shown6 Planning and Failure Detection 255Fig. 6.8. A MAPSIM interaction in the Household domain.MAPSIM run starts. There are 2 agents: Anne and R2D2.(1) Anne: request R2D2 ’give R2D2 coffee Anne’.(2) R2D2: accept request ’give R2D2 coffee Anne’.(3) R2D2: request Anne ’tell val Anne R2D2 pos(coffee)’.(4) Anne: execute ’tell val Anne R2D2 pos(coffee)’.(5) R2D2: ack achieved ’tell val Anne R2D2 pos(coffee)’.(6) ...in Fig. 6.8, we have extended MAPSIM with a verbalisation module, calledthe reporter. The reporter observes all physical and communicative events inthe simulation and verbalises them in English. All dialogues shown in theremainder of the chapter are unaltered outputs of this module. Fig. 6.9 showsthe beginning of the MAPSIM run of Fig. 6.8 using the reporter.Fig. 6.9. Verbalisation of the interaction of Fig. 6.8 by the MAPSIM reporter agent.MAPSIM run starts. There are 2 agents: Anne and R2D2.(1) Anne: ”Please bring me the coffee, R2D2.”(2) R2D2: ”Okay.”(3) R2D2: ”Where is the coffee, Anne?”(4) Anne: ”The coffee is in the kitchen.”(5) R2D2: ”Thanks, Anne.”(6) ...The reporter is a simple template engine that first determines an appro-priate pattern depending on the command type currently executed, then re-cursively replaces templates with concrete arguments until a template-freesentence is generated. Base values for arguments are generated directly fromanalysing the MAPL domain. For example, operator names are assumed to di-rectly correspond to verbs. Standard templates can be overridden by domain-specific patterns, but, surprisingly, this is often not even necessary to gener-ate fairly natural English phrases. While, compared to “real” natural-languageprocessing systems, this is a simplistic approach with obvious limits, the mini-mal effort needed to give MAPSIM basic linguistic capabilities is a noteworthyindication of the similarity between the MAPL representation and language.Currently, dialogues are only verbalised by the reporter; the “real” com-munication between agents is performed as direct update of mental statesas described by the MAPL speech acts. See Sec. 7.4.6 for a discussion ofhow agents can be enabled to communicate completely in a natural language,thereby enabling human-in-the-loop collaboration and dialogues.256 Brenner et al.MAPSIM is implemented in Python. The generic planner currently usedby the CCP agents is a modified version of Axioms-FF [25] that prevents theuse of assertions enabled in the current state, as required by definition3 andallows to express belief introspection with derived predicates. To enable theuse of a classical PDDL planner like FF, MAPL is first compiled to PDDL:induced state variables are explicitly generated; speech acts that use statevariables as parameters in MAPL are translated to a specific PDDL actionfor each such state variable; derived predicates for introspection of belief statevariables are added to the PDDL domain; and MVSVs are translated to PDDLpropositions.6.9 Situated Dialogue as Continual CollaborativePlanningIn this section, we will explain the use of CCP for scenarios that interleaveplanning, acting and sensing as well as action and interaction between multi-ple agents, i.e. for scenarios where agents engage in situated dialogue [26, 27].In this chapter, we use examples from MAPSIM to discuss planning of in-teractions. However, the same approach has been used for interaction plan-ning in our integrated robot systems (see Chapters 9 and 10). There, CCPis integrated with a specialised component for situated dialogue processing,presented in Chapter 8. CCP is used for high-level pragmatic reasoning, i.e.for planning how and why to use communication for achieving a (possiblynon-communicative) goal. Linguistic situation-appropriate realisation of theplanned “verbal behaviour”, as well as the situation-aware interpretation ofspeech acts by other agents, are handled by the specialised dialogue compo-nent.Figs. 6.10, 6.11, and 6.12 show situated dialogues between MAPSIM agentsin different MAPL domains, generated using the CCP algorithm and auto-matically verbalised by the MAPSIM reporter. This shows how, due to thedomain independence of both the CCP algorithm and the MAPSIM imple-mentation, it is easily possible to evaluate different CCP variants and dialoguestrategies across different domains and scenarios.It is important to realize that none of the sample runs shows the executionof a single multiagent plan, but a series of plans, devised, partly executed andrevised several times according to Alg. 4.In Fig. 6.10 the necessity for collaboration stems from the fact that onlyMrChips can move to the kitchen to get coffee, but only MrData can openthe kitchen door. At the beginning, both agents have different goals: MrDatawants to have coffee and MrChips wants to be at MrData’s service. Basically,this is a master-slave scenario; however, since both agents have individualgoals, incidentally it is MrChips that takes the initiative. MrChips’ originalgoal is very simple: he wants to achieve (has-goal MrChips). This goal isdirectly achieved by step 1 and 2 of the dialogue. However, step 2 already is6 Planning and Failure Detection 257Fig. 6.10. Mixed-initiative dialogue between two artificial agents in MAPSIM(Household domain).MAPSIM run starts. There are 2 agents: MrChips and MrData.(1) MrChips: ”What can I do for you, MrData?”(2) MrData: ”Please bring me the coffee, MrChips!”(3) MrChips: ”Where is the coffee, MrData?”(4) MrData: ”The coffee is in the kitchen, MrChips!”(5) MrChips: ”Please open the kitchen door, MrData!”(6) MrData opens the kitchen door.(7) MrChips moves to the kitchen.(8) MrChips takes the coffee.(9) MrChips moves to the livingroom.(10) MrChips: ”I have the coffee, MrData!”(11) MrChips: ”Please take the coffee, MrData!”(12) MrData takes the coffee.(13) MrData: ”Thanks for the coffee, MrChips!”MAPSIM terminates successfully.a request by MrData that provides MrChips with a new temporary subgoalthat he will try to achieve during the rest of the dialogue.To see why and how MrData generated this request, consider MrData’sindividual planning process: knowing that the coffee is in the kitchen, hecan generate a plan that involves MrChips going to the kitchen and bringingback the coffee. Alg. 4 uses the function SelectBestRequest to determinethe appropriate subgoal that MrData will request MrChips to achieve. Ourstandard implementation uses a simple algorithm FindIndividualSubplan(omitted from this chapter) that determines the largest subplan of P thatuses only one specific agent. Then SelectBestRequest chooses an actionon the final level of this plan as the best request. In our example, MrChipscan directly ask for the last action in his plan, namely MrData giving him thecoffee.The strategy of requesting long-term effects rather than immediately pos-sible actions is also exemplified in Fig. 6.11. Its main advantage is that itabstracts from the level of detail on which the individual agents can plan(due to the differences in their knowledge). In Fig. 6.11, the “Boss” agentmay not know whether the preconditions for the “put” action are already sat-isfied, i.e. his own plan may have included knowledge-gathering actions andassertions. However, by asking for the expected result, he gives the robot theopportunity to find its own appropriate solution.In the household scenario, MrChips adopts the new goal to provide Mr-Data with coffee (according to step 3 of Alg. 4). However, since he does notknow where the coffee is his next plan must resort to a fairly abstract as-sertion, (fetch-A MrChips coffee), that completely hides the position of the258 Brenner et al.(1) Boss: ”Please put obj1 on obj2, Robot.”(2) Robot picks up obj1.(3) Robot puts obj1 on obj2.Fig. 6.11. Long-term request in the object manipulation domain.target object as well as the possible complex planning necessary to reach thatposition. Although very abstract, the assertion guides MrChips’ planning bymeans of its replanning condition (KIF MrChips (pos coffee)) which, in words,says that in order to fetch the coffee he must at least find out where it is. Thisleads to the following multiagen plan by MrChips. Here, MrChips takes theinitiative again by asking MrData about where the coffee is:MrChips: request MrData ’tell val MrData MrChips pos(coffee)’MrData: execute ’tell val MrData MrChips pos(coffee)’MrChips: execute ’fetch-A MrChips coffee’MrChips: execute ’give MrChips MrData coffee’Alg. 2 declares MrChips’ plan as valid, so Alg. 3 executes the first action:MrChips requests MrData to tell him the position of the coffee. The domain-specific verbalisation template maps a request for telling the value of statevariable pos to the wh-question “where?” as shown in step 3 of the Fig. 6.10.Note that in the further course of the dialogue both agents switch seam-lessly between communicative and physical actions. However, unnecessary ver-balisation of action effects is avoided, because both agents also reason abouttheir mutual perceptions. For example, MrData does not verbalise havingopened the door, because in his plan he can apply a sensor model for MrChips,thereby deducing that MrChips will perceive the opening of the door him-self. In this manner, CCP both enforces knowledge preconditions, but alsoavoids unnecessary communication about them. In many applications, how-ever, communication is necessary for grounding the collaborative process orsimply acknowledging understanding [28]. Fig. 6.12 shows an example (froma Search and Rescue planning domain) where CCP was run with enforced ac-knowledgements upon subgoal adoption and achievement. Especially in thoseCollaborative Planning environments that include both artificial and humanagents such acknowledgements are crucial.6.10 Related WorkThis work integrates ideas from several subfields of AI, in particular Classicaland Distributed Planning, Multiagent Systems, Epistemic Logic, and Reason-ing about Actions and Change.Planning in dynamic, incompletely known domains can be modelled as aconformant, contingent or probabilistic planning task. All of These approaches6 Planning and Failure Detection 259(1) Ambulance: ”Please extinguish house1, Firebrigade.”(2) Firebrigade: ”Okay.”(3) Firebrigade refills watertank.(4) Firebrigade extinguishes house1.(5) Ambulance: ”Thanks for extinguishing house1, Firebrigade.”Fig. 6.12. Acknowledgements for subgoal adoption and achievement.compute conditional plans or policies for possible contingencies such that theagent can react adequately when faced with them. Unfortunately, this in-creased flexibility comes at the cost of being computationally much harderthan classical planning [29, 30]. Thus, these approaches scale badly in dy-namic multiagent environments with large numbers of unobservable featuresand exogenous events. Therefore, Continual Planning is often advocated asa practical approach to planning in such environments [31]. In practice, thisoften amounts to not more than repeatedly switching between planning andexecution. Previous work that more tightly integrates planning, monitoring,execution and information gathering includes [32, 33, 34, 35, 36]. Similarlyto our work, these approaches explicitly model knowledge and knowledge-gathering actions. Instead of the concept of assertions which enables us topostpone parts of the planning process, yet reason about its outcomes, theseapproaches use runtime variables to represent unknown sensing results. Run-time values can be used as action parameters in the remainder of plan and thusallow for reasoning about unknown future knowledge, although this reasoningis heavily limited because nothing is known about the variable beside the factthat has been sensed. Because of the limitations of runtime variables, MAPLdoes not yet support them. Instead, our use of MVSVs enables a planner tonon-deterministically guess one of the possible value of the MVSVs domain ifthis is desired by the domain modeller.Our asynchronous plans are similar to Boutilier and Brafman’s [37] multi-actuator plans. They model interacting effects of concurrent actions by spe-cific kinds of conditional effects of the individual agents. A plan must providesimultaneity constraints ensuring that the interaction really takes place asplanned. The authors assume that an external synchronisation mechanismwill ensure that during execution the constraints are met by the agents. Coxand Durfee’s [38] and Clement’s [39] coordination algorithms provide suchmechanisms. Our approach, however, rests on the assumption that executingagents are truly autonomous and there is no external instance to synchronisethem. Therefore it must allow agents to synchronise plan execution on theirown. This is achieved by explicitly including the knowledge and perceptionsnecessary for coordinated execution into the plans of agents. Since only plansthat include these information are valid multiagent plans, the planning algo-rithm itself can (and is forced to) ensure synchronised execution. It is worthstudying, however, how special-purpose coordination algorithms like those of260 Brenner et al.Cox or Clement could be integrated with our Continual Planning approach,thereby potentially simplifying the actual planning process.Planning with sensing actions has often been described in the planningliterature [40, 41, 42, 10, 43]. To our knowledge, none of these models extendsplanning for sensing to the concept of copresence [44]. Copresence has beenmuch discussed in literature on pragmatics, e.g. by Clark and colleagues [45].To our knowledge, ours is the first work that allows agents to explicitly reasonabout copresence in order to self-coordinate multiagent plan execution withoutexplicit communication.The explicit inclusion of beliefs and mutual beliefs in our planning ap-proach follows BDI models of multiagent planning, e.g . the SharedPlans modelof Grosz and Kraus [46] that describes the role of (mutual) beliefs as neces-sary conditions for planful MA behaviour. Our formalism and implementationdoes not cover all aspects of these models (yet); in particular, we do not modelintentions explicitly (yet). However, by explicitly modelling perception andcopresence, our approach complements existing BDI approaches to MA plans,since it can explain how knowledge conditions for joint behaviour can beachieved during plan execution.The need for Distributed Continual Planning (DCP), i.e. Continual Plan-ning in multiagent settings, was pronounced clearly by desJardins and col-leagues in [1]. Most work within this field is based on hierarchical represen-tations of multiagent plans [47, 48, 49, 50, 51]. Indeed, the expansion of as-sertions is similar to the decomposition of HTN schemata [52, 53, 54]. In ourapproach, however, the abstraction hierarchy need not be explicitly given bythe domain designer, but is resolved by the planner itself. Also the purposeof the abstraction is different from HTN planning: while HTN decomposi-tions embody knowledge about how to solve subtasks, assertions essentiallyrepresent a way to postpone parts of the planning process. Thus ContinualPlanning with assertions produces a series of non-hierarchical plans, whereasHTN produces one abstraction hierarchy. Note also that while it has beenproposed in textbooks that HTN planners may leave parts of the plan hierar-chy unexpanded until a plan has been partially executed, we are not aware ofwork that describes how exactly an HTN planner should make such decisions.Our CCP approach to dialogue is close in spirit to existing frameworksfor collaborative dialogue, in particular the one of Lochbaum [55]. In contrastto other approaches to dialogue planning that use formal state descriptionsmainly for the specification and validation of the computational approaches,e.g . [28, 56, 57], we directly reason on the formal logical representations ofthe agents’ beliefs (i.e. the MAPL states). In this respect, our work mostlyresembles Sadek’s approach to dialogue planning [58]. Due to our use of ageneral-purpose planning method, other approaches can deal with more elab-orate linguistic phenomena. However, CCP seems to be able to explain prag-matic aspects of a dialogue better (or at least more explicitly) than otherapproaches, because of the inherently causal reasoning underlying all dialoguecreated. In particular, CCP is suited for situated dialogue planning, because6 Planning and Failure Detection 261the generated dialogues directly depend on the agent’s knowledge about thecurrent situation and its physical actions in the world.Common approaches to failure detection are model-based, that is, theyinvolve reasoning about potential failures based on an explicit system modeldescribing the structural and behavioral properties of the system. This hasbeen approached (a) from within the AI community using symbolic reasoningwith a focus on large systems with many interacting components and (b)from the control theory community concentrating on fewer components withcomplex dynamics and higher noise levels. Most of the established approachesfound in the literature have originally been designed for discrete, static, andnoise-free domains. More recently, they have typically been extended to alsohandle continuous variables—if just by discretizing the space—and to respectthe dynamic and uncertain nature of the processes. It is, nevertheless, commonagreement that open questions in this area include the handling of noise,diagnosis of hybrid systems, and model building without excessive humanengineering.The close coupling between a system and its environment makes it hard ingeneral to detect abnormal behavior using instantaneous statistical tests onlywithout tracking possible failure modes over time [59]. This is especially thecase for mobile systems, such as service robots. By modeling faults as specialstates in a (typically hybrid) state space model, any state estimation techniquecan—in principle—be used for estimating their posterior likelihoods given asequence of observations. Approaches found most often in the FDI literatureinclude multi-hypothesis tracking [60, 61], single model tracking [62], or onestep look-ahead particle filtering [63]. In particle filter based approaches tofault diagnosis, the system is typically modeled by a non-linear Markov jumpprocess [64] or a dynamic mixture of linear processes [63]. Verma et al. [65]introduce the variable resolution particle filter for failure detection. Their ap-proach is similar to ours in that they build an abstraction hierarchy of systemmodels. They model the robot as a hybrid system with discrete modes ofoperation and corresponding continuous state variables. They assume thatthe probabilities of failure modes can be estimated from the state trackingperformance of their associated system models. As described in Sec. 6.4, wealso approach the problem using sequential state estimation in hybrid modelsand—in contrast to previous work—apply Gaussian process learning for sam-pling the failure mode more efficiently and robustly based on prior experience.Approaches that deal with the time efficiency of particle filters include [66]in which real-time constraints are considered for single system models or tech-niques in which a Rao-Blackwellized particle filter is used to coordinate mul-tiple models for tracking moving objects [67]. Related work addressing theefficiency of particle filtering also includes the regularized particle filter [68]and the Parzen particle filter [69]. Early references to the general idea in-clude [70], in which the authors propose to exchange the delta-Dirac kernelsof a particle filters for Gaussian kernels in order to estimate fixed systemparameters and dynamic state variables jointly.262 Brenner et al.6.11 ConclusionActing deliberatively is hard in realistic dynamic environments that cognitiveagents can only partially observe and influence. For all practical purposes,deliberation must be combined with reactive behaviour that takes newly per-ceived changes into account quickly. Often, however, dynamic environmentsdemand an even more proactive behaviour: agents must actively try to gathernew or reconfirm outdated information before they can determine appropriatesolutions to their problems. To that end, they must be able to reason abouthow and when to extend or update their knowledge, and then plan their owncycle of planning, acting and replanning.In this chapter, we have investigated the interplay between planning, act-ing and behaviour monitoring in dynamic multiagent environments. We haveintroduced several theoretical and practical tools that can be used for buildingpractical cognitive agents for such environments:The Multiagent Planning Language MAPL Intelligent agentsin dynamic multiagent environments must be able to explicitly reason abouttheir own and others’ sensory and communicative capabilities, their beliefs andmutual beliefs, and about the necessary conditions for joint behaviour. Mostformal languages for AI Planning, notably the de facto standard PDDL, donot permit this. We have therefore developed the Multiagent Planning Lan-guage MAPL that extends PDDL and thus also enables extending existingplanning methods for multiagent planning. MAPL plans can freely interleavephysical action, sensing and communication, and thus forms the basis for ourCP and CCP algorithms. In particular, MAPL plans guarantee that duringexecution all agents are provided (by perception or communication) with thenecessary knowledge to autonomously, i.e. without a central scheduler or syn-chronisation component, execute their parts of a MA plan.Continual Planning with Assertions We have developed a newalgorithmic framework for continual planning, i.e. integrated planning, planexecution, and plan monitoring. Previous approaches to continual planninghave mostly regarded it as just a case of replanning in light of new infor-mation. However, such a purely reactive approach is insufficient in dynamicenvironments where knowledge is limited from the start. Such environmentsdemand more proactive behaviour: agents must actively try to gather new (orreconfirm outdated) information before they can determine appropriate solu-tions to their problems. To that end, they must be able to reason about howand when to extend or update their knowledge, and then plan their own cycleof planning, acting and replanning. In CoSy, we have introduced a principledapproach for this kind of continual planning. Based on the novel concept ofassertions agents can decide autonomously which parts of of planning prob-lem they can already solve in detail and for which they must gather additionalinformation.Failure Detection and Robust Monitoring To be able to checkassertions and to reason about the outcome of actions, the robot requires the6 Planning and Failure Detection 263ability to monitor lower-level dynamic processes. We developed a novel ap-proach to process-monitoring based on particle filtering that enables one touse prior knowledge about failure modes learned from experience or in simu-lation. We derived a general framework for integrating learned failure modelsinto algorithms for sequential state estimation. The system was implemented,tested, and demonstrated on the task of online collision detection for a mobilerobot.Continual Collaborative Planning Other agents are the majorsource of exogenous change in most dynamic environments. However, theirpresence is not only a source of uncertainty, but can also be exploited: byinteracting with others, an agent can reduce its uncertainty about the presentand constrain possible contingencies in the future effectively. Again, sharinginformation and negotiating plans with others is a continual process that is in-terleaved with physical action (e.g. to enable communication in the first place)and sensing (e.g. to establish a context for situated references). Therefore, wehave extended our continual planning approach to Continual CollaborativePlanning (CCP), which integrates planning, acting, sensing and communica-tion. During CCP, agents will not only continually update their beliefs, butalso revise their goals as a result of negotiations with other agents. This leadsto particularly dynamic collaborative behaviour.The Multiagent Planning Simulator MAPSIM Continual plan-ning approaches can only be tested in environments where agents can actuallyinterleave planning, execution and sensing. The CoSy demonstration scenar-ios provide realistic settings for such an evalutation; however, we also wantedto provide a generic way to evaluate CCP and other distributed continualplanning approaches across a wide range of multiagent planning domains andproblems. We have therefore developed MAPSIM, a software environmentthat can automatically generate MA simulations from MAPL domains. Inother words, MAPSIM interprets a formal MAPL domain as an executablemodel of the environment in which agents can perceive, plan, act and engagein negotiations for task-orientated collaboration.Situated Dialogue as Continual Collaborative Planning Whenseveral agents are situated in a common environment they usually interactphysically as well as verbally. Verbal interaction in such environments, i.e.situated dialogue, both reflects the past and influences the future physicalbehaviour of the agents. Thus situated dialogue can often be regarded ascontinual collaborative planning. Interestingly, the role of communication inCCP is twofold: A dialogue move can be part of the collaborative planningprocess; however, it is also the execution of a communicative action and,just like the execution of a physical action, it changes the “world” in waysthat may lead to previously unforeseen changes in plans and, consequently,additional interactions. Since goals and plans of agents are continually revised,dialogues generated by CCP naturally include mixed-initiative subdialoguesand interleaved physical and communicative actions, as exemplified in theCoSy demonstrator systems.264 Brenner et al.Continual Planning in a Complex Cognitive Architecture Largecognitive systems, e.g. the CoSy demonstrators that are built on the CASTarchitecture and include many substantially different subarchitectures, do notonly act in dynamic environments, but can themselves be considered complexmultiagent systems. We have therefore applied CCP not only for planninginteractions with humans or other robots, but also for planning internal pro-cesses flows of a cognitive system. Each of the “agents”, i.e. the subarchitec-tures, in such a system can provide a plethora of information. One importantrealisation made while integrating our continual planner into the CAST ar-chitecture was that the abundance of information potentially available to theplanner is obstructive to its success. Instead, what is needed is a demand-driven way to provide information to the planner. Again, this need turns outto be a need that can be satisfied by a continual planning approach: by includ-ing less information in the state initially given to the planner, but extendingit with meta-level and self-referential knowledge (e.g. knowledge about whichsubarchitecture may have or may provide additional knowledge), the continualplanner can determine on its own which information it additionally requiresfor solving a problem. This may go as far as the planner requiring more de-tailed information about the planning ontology or the planning operators, allduring the continual planning process.Summary During the CoSy project, we have developed a new frame-work for proactive continual planning and execution monitoring. The frame-work has been successfully used in simulation as well as in complex robotsystems (c.f . Chapters 9 and 10) for a variety of tasks ranging from monitor-ing hardware failures to engaging in human-robot dialogues. We believe thatour approach of encouraging information gathering on all levels of a cognitivearchitecture is a significant step towards building agents that show intelli-gent proactive, yet robust behaviour, i.e. agents that can rightfully be calledautonomous.References1. M. DesJardins, E. Durfee, J. C. Ortiz, M. Wolverton, A survey of research indistributed, continual planning, The AI Magazine.2. M. Kevin, Dynamic bayesian networks: Representation, inference and learning,Ph.D. thesis, UC Berkeley (2002).3. A. Doucet, N. de Freitas, N. Gordan (Eds.), Sequential Monte-Carlo Methodsin Practice, Springer Verlag, 2001.4. M. Brenner, B. Nebel, Continual planning and acting in dynamic multiagentenvironments, Journal of Autonomous Agents and Multiagent SystemsAcceptedfor publication.URL http://www.cognitivesystems.org/cosybook/chap6.asp#Brenner/etal:20085. M. Fox, D. Long, PDDL 2.1: an extension to PDDL for expressing temporalplanning domains, JAIR.6 Planning and Failure Detection 2656. M. Helmert, The Fast Downward planning system, Journal of Artificial Intelli-gence Research 26 (2006) 191–246.7. M. Brenner, B. Nebel, Continual planning and acting in dynamic multiagentenvironments, in: PCAR ’06: Proceedings of the 2006 international symposiumon Practical cognitive agents and robots, ACM, New York, NY, USA, 2006,pp. 15–26. doi:http://doi.acm.org/10.1145/1232425.1232431.URL http://www.cognitivesystems.org/cosybook/chap6.asp#Brenner/etal:2006a8. M. Yokoo, K. Hirayama, Algorithms for distributed constraint satisfaction: areview, Autonomous Agents and Multi-Agent Systems 3 (2).9. C. Bäckström, Computational aspects of reordering plans, JAIR 9 (1998) 99–137.10. R. Petrick, F. Bacchus, A knowledge-based approach to planning with incom-plete information and sensing., in: Proc. AIPS-02, 2002.11. J. Hoffmann, R. Brafman, Contingent planning via heuristic forward searchwith implicit belief states, in: S. Biundo, K. L. Myers, K. Rajan (Eds.), ICAPS,AAAI, 2005, pp. 71–80.12. S. Thrun, W. Burgard, D. Fox, Probabilistic Robotics, MIT Press, 2005.13. B. Ng, A. Pfeffer, R. Dearden, Continuous time particle filtering, in: Proceedingsof the 19th IJCAI, Edinburgh, 2005.14. S. Thrun, J. Langford, V. Verma, Risk sensitive particle filters., in: NIPS, 2001.15. A. Doucet, On sequential simulation-based methods for bayesian filtering, Tech.rep., Signal Processing Group, Departement of Engeneering, University of Cam-bridge (1998).16. Z. Khan, T. R. Balch, F. Dellaert, An mcmc-based particle filter for trackingmultiple interacting targets., in: ECCV (4), 2004, pp. 279–290.17. C. E. Rasmussen, C. K. Williams, Gaussian Processes for Machine Learning,The MIT Press, Cambridge, Massachusetts, 2006.18. D. J. C. MacKay, Introduction to Gaussian processes, in: C. M. Bishop (Ed.),Neural Networks and Machine Learning, NATO ASI Series, Kluwer AcademicPress, 1998, pp. 133–166.19. R. Neal, Monte carlo implementation of gaussian process models for bayesianregression and classification, Tech. rep., Dept. of Computer Science, Universityof Toronto. (1997).20. N. Koenig, A. Howard, Design and use paradigms for gazebo, an open-sourcemulti-robot simulator. technical report, Tech. rep., USC Center for Roboticsand Embedded Systems, CRES-04-002 (2004).21. C. Plagemann, C. Stachniss, W. Burgard, Efficient failure detection for mobilerobots using mixed-abstraction particle filters, in: H. Christiensen (Ed.),European Robotics Symposium 2006, Vol. 22 of STAR Springer tracts inadvanced robotics, Springer-Verlag Berlin Heidelberg, Germany, 2006, pp.93–107.URL http://www.cognitivesystems.org/cosybook/chap6.asp#Plagemann/etal:200622. A. Witkin, D. Baraff, An introduction to physically based modeling, in: SIG-GRAPH’97 Course Notes, 1997.23. N. Lynch, Distributed Algorithms, Morgan Kaufmann, San Francisco, CA, 1996.24. M. Brenner, Situation-aware interpretation, planning and execution of usercommands by autonomous robots, in: Proceedings of IEEE RO-MAN 2007,266 Brenner et al.2007.URL http://www.cognitivesystems.org/cosybook/chap6.asp#Brenner:2007a25. S. Thiebaux, J. Hoffmann, B. Nebel, In defense of axioms in PDDL, in: Proc.IJCAI, 2003.26. G. Kruijff, M. Brenner, Modelling spatio-temporal comprehension in situatedhuman-robot dialogue as reasoning about intentions and plans, in: Proceedingsof the Symposium on Intentions in Intelligent Systems, AAAI Spring Sympo-sium Series 2007, Stanford University, Palo Alto, CA, 2007.URL http://www.cognitivesystems.org/cosybook/chap6.asp#Kruijff/Brenner:200727. M. Brenner, I. Kruijff-Korbayová, A continual multiagent planning approachto situated dialogue, in: Proceedings of the 12th Workshop on the Semanticsand Pragmatics of Dialogue (Semdial), London, UK, 2008.URL http://www.cognitivesystems.org/cosybook/chap6.asp#Brenner/etal:2008a28. D. Traum, A computational model of grounding in natural language conversa-tion, Ph.D. thesis, Univ. of Rochester (1994).29. M. Littman, J. Goldsmith, M. Mundhenk, The computational complexity ofprobabilistic planning, JAIR.30. J. Rintanen, Constructing conditional plans by a theorem-prover, JAIR 10(1999) 323–352.31. S. Russell, P. Norvig, Artificial Intelligence: A Modern Approach, 2nd Edition,Prentice-Hall, Englewood Cliffs, NJ, 2003.32. J. A. Ambros-Ingerson, S. Steel, Integrating planning, execution and monitoring,in: Proc. AAAI-88, Saint Paul, MI, 1988, pp. 83–88.33. O. Etzioni, S. Hanks, D. Weld, D. Draper, N. Lesh, M. Williamson, An approachto planning with incomplete information, in: Proc. KR-92, 1992, pp. 115–125.34. C. A. Knoblock, Planning, executing, sensing, and replanning for informationgathering, in: C. Mellish (Ed.), Proceedings of the Fourteenth InternationalJoint Conference on Artificial Intelligence, Morgan Kaufmann, San Francisco,1995, pp. 1686–1693.35. O. Etzioni, K. Golden, D. S. Weld, Sound and efficient closed-world reasoningfor planning, Artificial Intelligence 89 (1-2) (1997) 113–148.36. K. Golden, Leap before you look: Information gathering in the puccini planner,in: Proc. AIPS-98, 1998, pp. 70–77.37. C. Boutilier, R. Brafman, Partial order planning with concurrent interactingactions, JAIR.38. J. S. Cox, E. H. Durfee, An efficient algorithm for multiagent plan coordination,in: Proc. AAMAS ’05, 2005.39. B. Clement, A. Barrett, Continual coordination through shared activities, in:Proc. AAMAS ’03, 2003.40. H. J. Levesque, What is planning in the presence of sensing?, in: Proc. AAAI-96,MIT Press, 1996, pp. 1139–1146.41. K. Golden, D. Weld, Representing sensing actions: The middle ground revisited.,in: Proc. KR ’96, 1996.42. D. S. Weld, C. R. Anderson, D. E. Smith, Extending graphplan to handle un-certainty and sensing actions, in: AAAI/IAAI, 1998, pp. 897–904.6 Planning and Failure Detection 26743. R. P. A. Petrick, F. Bacchus, Extending the knowledge-based approach to plan-ning with incomplete information and sensing., in: Proc. ICAPS 2004, 2004, pp.2–11.44. D. Lewis, Convention. A Philosophical Study, Harvard University Press, Cam-bridge, Massachusetts, 1969.45. H. H. Clark, C. R. Marshall, Definite reference and mutual knowledge, in: Ele-ments of discourse understanding, Cambridge University Press, 1981.46. B. J. Grosz, S. Kraus, Collaborative plans for complex group action, ArtificialIntelligence 86.47. E. Durfee, T. Montgomery, Coordination as distributed search in hierarchicalbehavior space, IEEE Transactions on Systems, Man, and Cybernetics.48. K. L. Myers, Cpef: A continuous planning and execution framework, The AIMagazine 20 (4).49. E. H. Durfee, Distributed continual planning for unmanned ground vehicleteams, AI Magazine 20 (4) (1999) 55–61.50. M. DesJardins, M. Wolverton, Coordinating a distributed planning system, TheAI Magazine 20 (4).51. B. Clement, E. Durfee, Top-down search for coordinating the hierarchical plansof multiple agents, in: Proc. AGENTS ’99, 1999.52. Q. Yang, Intelligent Planning: A decomposition and abstraction based approach,Springer-Verlag, 1997.53. K. Erol, J. Hendler, D. Nau, Complexity results for hierarchical task-networkplanning, Annals of Mathematics and Artificial Intelligence 18 (1996) 69–93.54. D. Nau, Y. Cao, A. Lotem, H. Munoz-Avila, SHOP: Simple hierarchical or-dered planner, in: T. Dean (Ed.), Proceedings of the 16th International JointConference on Artificial Intelligence (IJCAI-99), Morgan Kaufmann, Stockholm,Sweden, 1999.55. K. E. Lochbaum, A collaborative planning model of intentional structure, Com-putational Linguistics.56. C. Rich, C. L. Sidner, Collagen: A collaboration manager for software interfaceagents, User Modeling and User-Adapted Interaction Special Issue on Compu-tational Models for Mixed Initiative Interaction.57. N. Blaylock, J. Allen, G. Ferguson, Managing communicative intentions withcollaborative problem solving, in: Current and New Directions in Dialogue,Kluwer, 2003.58. M. D. Sadek, Dialogue acts are rational plans, in: Proceedings of the ESCA/ETRWorkshop on Multi-Modal Dialogue, Maratea, Italy, 1991.59. R. Dearden, D. Clancy, Particle filters for real-time fault detection in planetaryrovers, in: Proceedings of the Thirteenth International Workshop on Principlesof Diagnosis, 2002, pp. 1–6.60. M. W. Hofbaur, B. C. Williams, Hybrid estimation of complex systems, IEEETransactions on Systems, Man, and Cybernetics, Part B 34 (5) (2004) 2178–2191.61. R. Mehra, C. Rago, S. Seereeram, Autonomous failure detection, identificationand fault-tolerant estima tion with aerospace applications, in: IEEE AerospaceConference, Aspen, CO, Proceedings. Vol. 2; CO; UNI TED STATES, 1998.62. R. Washington, On-board real-time state and fault identification for rovers., in:Proceedings of the IEEE International Conference on Robotics and Automation(ICRA), 2000, pp. 1175–1181.268 Brenner et al.63. N. de Freitas, R. Dearden, F. Hutter, R. Morales-Menendez, J. Mutch, D. Poole,Diagnosis by a waiter and a mars explorer, in: Invited paper for Proceedings ofthe IEEE, special issue on sequential state estimation, 2003.64. J. Driessen, Y. Boers, An efficient particle filter for nonlinear jump markovsystems, in: IEEE Sem. Target Tracking: Algorithms and Applications, Sussex,UK, 2004.65. V. Verma, S. Thrun, R. Simmons, Variable resolution particle filter., in: Proc ofIJCAI, 2003.66. C. Kwok, D. Fox, M. Meila, Real-time particle filters., in: Advances in NeuralInformation Processing Systems 15 (NIPS), 2002, pp. 1057–1064.67. C. Kwok, D. Fox, Map-based multiple model tracking of a moving object., in:RoboCup 2004: Robot Soccer World Cup VIII, 2004, pp. 18–33.68. C. Musso, N. Oudjane, L. G. F., Improving regularized particle filters, in:A. Doucet, N. de Freitas, N. Gordon (Eds.), Sequential Monte Carlo Meth-ods in Practice, New York, Statistics for Engineering and Information Science,Springer?Verlag, 2001, Ch. 12, p. 247?271.69. T. Lehn-Schiøler, D. Erdogmus, J. C. Principe, Parzen particle filters, in:ICASSP, Vol. 5, 2004, pp. 781–784.70. J. Liu, M. West, Combined parameter and state estimation in simulation-basedfiltering, Sequential Monte Carlo Methods in Practice.7Multi-modal LearningDanijel Skočaj1, Matej Kristan1, Alen Vrečko1, Aleš Leonardis1, MarioFritz2, Michael Stark2, Bernt Schiele2, Somboon Hongeng3, Jeremy L.Wyatt31 VICOS Lab, University of Ljubljana, Ljubljana, Slovenia,first.last@fri.uni-lj.si2 Technische Universität Darmstadt, Darmstadt, Germany,lastname@informatik.tu-darmstadt.de3 Intelligent Robotics Laboratory, School of Computer Science, University ofBirmingham, Birmingham, UK, {jlw,sxh}@cs.bham.ac.uk7.1 IntroductionThe main topic of this chapter is learning, more specifically, multimodal learn-ing.In biological systems, learning occurs in various forms and at various de-velopmental stages facilitating adaptation to the ever changing environment.Learning is also one of the most fundamental capabilities of an artificial cogni-tive system, thus significant efforts have been dedicated in CoSy to researchinga variety of issues related to it.Learning involves numerous deep problems that stretch far beyond straight-forward applications of current statistical methods. In particular, learningshould take place in a rich interaction of an artificial cognitive system withthe environment or with a human (tutor) exploiting multiple modalities to ar-rive at new concepts and/or extend the current ontologies or merely to adaptto different circumstances.One of the main topics which was addressed within the CoSy project wasthe selection of appropriate representations that allow for their efficient cre-ation and modification as the new data is acquired, and the levels and types ofsupervision that guide the learning processes. In this context, the exploitationof multiple modalities is crucial as they provide means for robust and efficientlearning that is not possible within a single modality. For example, the dataof one modality may act as a means of supervision for learning within anothermodality, or, through a simple dialogue inherent ambiguities present in a vi-sual signal can be resolved. Similarly, an artificial cognitive system can learnimportant insights into how the environment is structured, including causalrelations, by observing activities or exercising certain actions.270 Skočaj et al.Therefore, the learning we are addressing in this chapter involves informa-tion from multiple modalities. One modality helps another modality by pro-viding additional information, i.e., one modality supervises another modality.In Sections 7.2 and 7.3, we address the multimodal learning in the context ofinteraction between the visual and communication modalities. The PlayMatesystem we have developed has the capability of relating the visual informa-tion and the information obtained through the communication subsystem. Inthis way, the learning of visual concepts can be supervised by language. InSections 7.4 and 7.5, the pure visual information is interacting with the infor-mation produced by performing certain actions (pushing, grasping). Broadlyspeaking we can view visual information and the information obtained fromactions as two separate sub-modalities. In this view, sections 7.4 and 7.5 ad-dress interaction between the visual sub-modality and the action sub-modality.Again, visual cues are learnt that could not have been learnt without havingaccess to information from different origins (sub-modalities). Note that in theliterature, the multimodal learning, as defined here, is also frequently referredto as cross-modal learning. Both terms emphasize the interaction between dif-ferent modalities during the learning process and are often used interchange-ably; so they are in this book.In terms of the levels of supervision, a variety of different modes have beenapplied, ranging from fully supervised, weakly supervised, to unsupervised. Asfor the types of supervision they were achieved through dialogues, perceptionof affordance cues and exploratory strategies. The approaches presented inSections 7.2 and 7.3 focus on analysing different levels of supervision, whichis achieved through a dialogue with a human teacher. The integrated systemwe have developed facilitates such kind of research and provides means forapplying different learning strategies in an interactive learning settings.The remainder of the chapter contains four sections which address theissues mentioned above in different contexts.Section 7.2 presents an interactive framework for continuous learning ofvisual concepts. The main goal that was set is to learn associations betweenautomatically extracted visual features and words describing the scene (visualattributes and spatial relations) in a user friendly, natural, open-ended, andcontinuous manner. The system facilitates interactive learning of basic visualconcepts in a dialogue with a human tutor. The learning can be performedwith different levels of supervision. The developed framework also supportsunlearning, which enables recovery from any errors accumulated in the learnedmodels. The entire learning process is fully integrated with other parts of thesystem that provide information needed by the learner and use the informa-tion produced by the learner. The entire system thus provides visual inputand enables verbal and non-verbal communication with a tutor facilitatingcontinuous and interactive cross-modal learning.In Section 7.3, crossmodal learning of visual categories is performed in away that combines supervised and unsupervised training methods. While su-pervised methods tend to produce more accurate results, unsupervised meth-7 Multi-modal Learning 271ods are highly attractive due to their potential to use masses of unlabeledtraining data. The proposed novel method uses unsupervised training to ob-tain visual groupings of objects and a cross-modal learning scheme to over-come the inherent limitations of purely unsupervised training. The methoduses a unified and scale-invariant object representation by Scale-Invariant Pat-terns (SIPs) that allows us to handle labeled as well as unlabeled informationin a coherent way. One of the potential settings is to learn object categorymodels from many unlabeled observations and a few dialogue interactionsthat can be ambiguous or even erroneous. Initial experiments demonstratethe ability of the system to learn meaningful generalizations across objectsalready from a few dialogue interactions.Section 7.4 addresses supervised-learning of human actions and activities.A cognitive robot that interacts with humans needs not only to detect mo-tion patterns, but also to put them in the context of understanding humanintention. Towards this goal, a new representation of intentional actions hasbeen proposed, which models the causal relations between hand motion, ob-ject states, and the effects of actions. It has been shown, by reproducing a keyresult from the literature on action learning in children, that the proposed rep-resentation has properties in common with action learning in humans. Humanmovement analysis has also been put in the context of activity understand-ing. This extension allows the robot to relate the concept of objects to thesituations that the agent tends to perform certain kinds of actions, and alsoto learn about the reliability of a class of execution styles.Section 7.5 addresses multimodal learning in the context of an embodiedcognitive agent. In this study, the challenge of functional object categorizationis approached from a completely different angle. Namely, the functional cate-gory representations are acquired by observing few prototypical human-objectinteractions rather than explicitly modeling physical object properties. Nat-urally, the set of functional categories that the system’s local feature-basedvision module is able to represent is restricted to those that are characterizedby distinct visual features, e.g., the bent shape of a mug handle suggests howto grasp the mug in a specific way. Such distinct visual features are termedaffordance cues, and the functional object category detection is based on therecognition of these cues. The results are reported for the detection of twofunctional object categories learned by the system, which demonstrate theirgeneralization capabilities across and beyond basic level categories. In fact,it is shown that the system supports the interpretation of these categories ascomposite functional ones.7.2 Continuous learning framework7.2.1 IntroductionAn important characteristic of a system that operates in a real-life environ-ment is the ability to expand its current knowledge. The system has to create272 Skočaj et al.and extend concepts by observing the environment – and has to do so contin-uously, in a life-long manner. An integral part of such a continuous learningsystem is a method for incremental updating of previously learned represen-tations, which has to fulfill several requirements: (i) the learning algorithmshould be able to update the current representations (and create new onesif necessary), (ii) it should not require access to old (previously processed)original data, (iii) the representations should be kept compact; they shouldallow improving the information content while not requiring an increase ofmemory requirements with each observation, and (iv) the computational ef-fort needed for a single update should not depend on the amount of previouslyobserved data. Furthermore, models should allow for error-recovery (unlearn-ing) in cases when erroneous information gets incorporated into them. Thelearning process should thus create, extend, update, delete, and modify mod-els of real-world concepts in a continuous, life-long manner, while still keepingthe representations of the environment compact and efficient.In this subsection we present an algorithm that satisfies these requirementswhen learning associations between low-level visual features and higher-levelconcepts. In particular, we address the problem of continuous learning of visualproperties (such as colour or shape) and spatial relations4(such as ‘to theleft of’ or ‘far away’). The main goal is to find associations between wordsdescribing these concepts and simple visual features extracted from images.We tackle this problem by using a continuous learning paradigm in a cross-modal interaction between the system and the tutor. This interaction plays acrucial role in the entire learning process, since the tutor provides very reli-able information about the scenes in question. This information can also beinferred by the system itself, reducing the need for tutor supervision, how-ever also increasing the risk of false updates and degradation of the currentknowledge. In this section we introduce and analyse several different learningmodes requiring different levels of tutor supervision.In our setting, the inputs in the learning process are partial descriptions ofthe scene; descriptions of the objects and relationships between them. An ob-ject can be labeled with several concept labels (e.g., object A can be ‘yellow’and ‘round’ and positioned ‘in the middle’ of the image and ‘to the left of’object B). These descriptions serve as learning examples. There are no nega-tive examples (e.g., the information that ‘yellow’ is not ‘red’ is not provided).Therefore, the algorithm should build reconstructive representations withoutrelying on discriminative information, which would discriminate between dif-ferent classes (i.e., concepts) [1].Similar problems have often been addressed by researchers from variousfields, from psychology, to computational linguistics, artificial intelligence, andcomputer as well as cognitive vision. Our problem is closely related to the sym-bol grounding problem [2], which has been very often addressed by a plethora4 In the current implementation we only address a simplified geometrical aspect ofspatial relations, without considering functional aspect and context.7 Multi-modal Learning 273of researchers [3, 4, 5, 6, 7, 8, 9]. Our work is closely related to that of Roy[5, 6], in that our framework focuses on learning qualitative linguistic descrip-tions of visual object properties and scene descriptions. His system in [5] wasdesigned to learn word forms and visual attributes from speech and videorecordings, and subsequently, he extended this work for generating spokendescriptions of scenes [6]. Several papers have also been published addressingonline learning, particularly with regard to object recognition [10, 11, 12].In our system, however, we utilize, in a unified framework, continuous onlinelearning of qualitative object properties and spatial relations in a setting withno negative examples where every sample can be labelled with multiple con-cept labels. Furthermore, our system facilitates unlearning and error-recoveryas well. The system is completely integrated in a robot that on one hand pro-vides useful multimodal information for the learning subsystem, and on thesecond hand, uses the knowledge and information provided by the learningsubsystem to accomplish given tasks.Fig. 7.1 shows a sample dialogue between the tutor and the system, whichindicates the process of learning of object visual properties (colour, size, andshape). The tutor is describing the scene and verifying the state of the robot’sknowledge by posing questions. The robot also asks the tutor when presentedwith an object whose visual properties it does not recognise. This exampleshows how the system can extend its knowledge in a dialogue with the tutor.This is one of our motivating scripts for the PlayMate scenario.’H: This is a blue, large, and square object.’’R: OK.’...’H: What do you see?’’R: I don’t know this object.’’H: It is green, small, and triangular.’’R: OK.’...’H: What do you see?’’R: I see a small and square object.’’H: Yes, that’s correct.’’R: OK.’...’H: What do you see?’’R:I see a green object. I think it is also large.’’H: Yes, that’s correct.’’R: OK.’’H: It is also rectangular.’’R: OK.’...’R: I don’t know this object. What is it like?’’H: It is blue, small, and rectangular.’’R: OK.’...’R: Is this object large?’’H: Yes, that’s correct.’’R: OK.’’H: It is also yellow and circular.’’R: OK.’...’R: Is this object red?’’H: No, it is not red.’’R: OK.’’H: It is yellow.’’R: OK.’...’H: What do you see?’’R: I see a red, small, and triangular object.’Fig. 7.1. An example dialogue between the tutor (H) and the system (R).The integration aspect of the PlayMate is described in Chapter 9, whilein this section we focus on the problem of learning. We first present dif-ferent modes of learning requiring different levels of human interaction andthen introduce the main incremental learning algorithm and the underlayingtechniques for managing reconstructive representations. We then present the274 Skočaj et al.evaluation of the proposed methods. Finally, we summarize and discuss someopen issues.7.2.2 Different modes of learningThe interaction between a tutor and an artificial cognitive system plays animportant role in a continuous learning framework. In this subsection we defineseveral learning modes which alter the behaviour of the system and requiredifferent levels of tutor involvement.When implementing a continuous learning mechanism, two main issueshave to be addressed. Firstly, the representation, which is used for modelingthe observed world, has to allow for updates when presented with newly ac-quired information, considering the requirements described in Section 7.2.1.Secondly, a crucial issue is the quality of the updating, which highly dependson the correctness of the interpretation of the current visual input. With thisin mind, several learning strategies can be used, ranging from completely su-pervised to completely unsupervised. Here we discuss three such strategies:• Tutor-driven approach (TD). The correct interpretation of the visualinput is always correctly given by the tutor.• Tutor-supervised approach (TS). The system tries to interpret thevisual input. If it succeeds to do this reliably, it updates the current model,otherwise asks the tutor for the correct interpretation.• Exploratory approach (EX ). The system updates the model with theautomatically obtained interpretation of the visual input. No interventionfrom the tutor is provided.We further divide tutor-supervised learning into two sub-approaches:• Conservative approach (TSc). The system asks the tutor for the cor-rect interpretation of the visual input whenever it is not completely surethat its interpretation is correct.• Liberal approach (TSl). The system relies on its recognition capabilitiesand asks the tutor only when its recognition is very unreliable.Similarly, we also allow for conservative and liberal exploratory sub-app-roaches (EXc, EXl).It is obvious that the system is supposed to have a certain level of self-understanding; it should be able to estimate whether its current knowledgesuffices to interpret the current scene, or it should ask a tutor for help. There-fore, it should have a recognition capability, i.e., the ability to interpret thevisual input to some extent. And even more importantly, the system shouldbe able to evaluate the reliability of this recognition process.To formalise the above descriptions, let us assume that the recognitionalgorithm is able to provide information about the reliability of the recogni-tion by giving one of the following five answers when asked to confirm theinterpretation of the visual scene (e.g., the question may be: “Is this object7 Multi-modal Learning 275circular?”): ‘yes’ (YES), ‘probably yes’ (PY), ‘probably no’ (PN), ‘no’ (NO),and ‘don’t know’ (DK). Table 7.1 presents actions that are taken after ananswer is obtained from the recognition process. The system can either askthe tutor for the correct interpretation of the scene (or the tutor provides itwithout being asked), update the model with its interpretation, or do noth-ing. As it is stated in Table 7.1, the system can communicate with the tutorall of the time (TD learning), often (TSc), occasionally (TSl) or even never(EX learning). This communication is only initiated by the tutor in the tutor-driven approach, while in other approaches the dialogue and/or the learningprocess is initiated by the system itself.Table 7.1. Update table.YES PY PN NO DKTD ask ask ask ask askTSc update ask ask / askTSl update update / / askEXc update / / / /EXl update update / / /To speed up the initial phase of the learning process and to enable devel-opment of consistent basic concepts, one could start with mainly tutor-drivenlearning with many user interactions. These concepts would then be used forupdating with limited help from the user in tutor-driven or even exploratorymanner.7.2.3 Learning algorithmThere are two major parts of the interactive learning framework discussed inthis section. The first part is the update algorithm, which continually updatesthe representations of the visual concepts. The second is the recognition algo-rithm, which, using the representations, can produce quantitative answers tothe queries from the user.Our system does not have a priori access to the negative examples of thelearnt concepts. Furthermore, we allow multiple concept labels for each in-put instance, which generally prohibits the update algorithm to exploit thediscriminative information to improve the concepts. For those reasons, theupdate algorithm builds and maintains reconstructive representations of theobserved visual features in a form of generative models. Each visual conceptis associated with a visual feature which best models the observed visual dataaccording to two criteria: consistency and specificity. The algorithm automat-ically determines which of the extracted visual features are consistent over allobservations of the same visual concept; at the same time, the features alsohave to be specific for that particular concept.276 Skočaj et al.The learning algorithm thus selects from a set of one-dimensional features5the feature whose values are most consistent and specific over all of the ob-served images representing the same visual concept (e.g., all images of largeobjects, or circular objects, or pairs of objects far apart etc.). Note that thisprocess is performed incrementally, considering the current image (or a veryrecent set of images) and only the learnt representations of the concepts with-out accessing the previously processed images. For the purposes of establish-ing the concept-feature associations, the distribution each feature is initiallymodelled by a simple generative model for each concept separately. In ourimplementations, a single Gaussian is used for such simple generative model.These models (Gaussians) are then continually updated as new observationsarrive. After each observation, the algorithm associates each concept with asingle feature. Once an association with a particular feature is established, anadditional, more detailed generative model is created for that feature. It isinitialized by the simple model of that feature and then updated to a moreaccurate and complex one as new observations arrive. In our implementationswe use kernel density estimates (KDE) for the detailed generative models,which will be described in more detail in the next subsection. After moreand more observations arrive, it may turn out that some other feature agreesbetter with a given concept than the one to which it was initially associated.In that event, the concept-feature association is reset to a new feature andthe generative model for the concept is reinitialized. The algorithm fulfills theconsistency and specificity criteria by associating a concept with a featurewhose generative model for that concept is most distinctive from the genera-tive models of the other concepts. To measure the distinctiveness we requirea measure of distance between the generative models. In our implementationswe used the Hellinger distance [13] since it is a metric and returns the dis-tance value on a constrained interval between zero and one, which makes itconvenient for interpretation how much two distributions are similar (in con-trast to Kullback-Liebler divergence). Therefore an analytic Hellinger distancewas applied to evaluate distance between single Gaussians (simple generativemodels) and a numerical generalization of the Hellinger distance for the moredetailed generative models (see, eg., [14]).In the recognition stage, the algorithm first extracts feature vectors fromthe visual data and evaluates it against every concept (i.e., its generativemodel). For each concept, the algorithm returns a value between zero and one,which reflects the confidence that the observation agrees with that concept.Since our generative models, as we will see later, are in fact probability densityfunctions (pdf), the confidence is evaluated as the integral over the featurevalues whose probability is smaller than that of the observed value. Thisresult is then quantized and transformed into a form which is suitable for5 These features may be, for example, the median hue value of an object, area ofsegmented region, coordinates of the object center, distance between two objects,etc.7 Multi-modal Learning 277communication with the human operator, or into a form which is appropriatefor any other particular mode of continuous learning presented in the previoussubsection.In online algorithms like the one presented in this section, the noise in theinput data has a detrimental effect on the learnt representations. In an onlinesemi-autonomous and exploratory framework discussed here, this problem iseven more pronounced. If, for example, the recognition algorithm fails at somepoint to correctly interpret the visual scene and overestimates the reliability ofits recognition, the generative models of the concepts tend to degrade and theperformance of the system will typically decrease severely. However, since thesystem interacts with a human user, the user can help the system to recoverfrom the errors through interaction, by, e.g., indicating to a system that itsbelief about a certain concept is wrong. This unlearning is a crucial elementof the learning system since it affects the choice of the generative models usedto model the concepts – apart from allowing learning from positive examples,these generative models have to support learning from negative examples aswell. This latter process is what is called unlearning. In the following sectionwe discuss a particular form of the generative models which are used in ourimplementations and allow flexible learning from positive, as well as from thenegative examples.7.2.4 Reconstructive representations for interactive/onlinelearningA popular approach to building a generative model from the observed datais to estimate the underlying probability density function (pdf), which gen-erated the data. In particular, Gaussian Mixture Models (GMM) have beenshown to approximate well probability density functions which may even befar from Gaussian [15]. Standard approaches to GMM estimation includeParzen estimators [15, 16, 17, 18], expectation-maximization-based (EM) al-gorithms [19, 20, 21], and variational-Bayes methods [22, 23], to name a few.However, extending the above techniques to online estimation is nontrivial,since they require all the data in advance. Most of the recent solutions foronline estimation thus either impose temporal constraints on the incomingdata [24, 25], assume a known prior over the model parameters [26], assumeeach observed sample is approximated by a single Gaussian with a knowncovariance [27], or assume the modes of the underlying pdf are well separatedand Gaussian [28]. Recently, Kristan et al. [29, 14] have developed a frame-work for online estimation of the GMMs, which can be viewed as an onlineextension of the batch kernel density estimation [15], (OKDE). Among theexisting approaches to online estimation of mixture models, OKDE makesthe least assumptions about the incoming data and the shape of the approxi-mated distributions. As a marked contrast to existing approaches, this is theonly framework which also allows refining a GMM generative model from thenegative examples through the process of unlearning. In this section we will278 Skočaj et al.briefly overview the learning/unlearning methodologies of the OKDE; for adetailed treatment, however, the reader is referred to [29, 14].The online estimation in OKDE from positive observations proceeds asfollows. A Gaussian kernel is assigned to each new observed sample and addedto the existing mixture model. The variance of the kernel is determined usingan online extension of the plug-in rule [15] for the bandwidth estimation.To maintain a low complexity, the mixture model is compressed from timeto time into an equivalent mixture with a smaller number of components.This is achieved by iterating between a reduction step and an optimizationstep. The reduction step uses a reduced-set density estimation [30] to removecomponents from the mixture and the optimization step applies a Levenberg-Marquardt optimization to minimize the error between the original mixturemodel and its compressed equivalent. In this way, the OKDE produces avariable-bandwidth kernel density estimate of the underlying distribution witha low complexity.Figure 7.2 shows results of an experiment in which a set of samples wassampled from a reference distribution which was a mixture of a uniformand a Gaussian distribution. These samples were then used one at a timein OKDE to approximate the reference pdf. For comparison, Figure 7.2 alsoshows the results for two standard batch kernel-density estimators: an optimaland a suboptimal. The optimal estimator used the solve-the-equation plug-inmethod [31] to estimate the kernels, while the suboptimal estimator used theSilverman’s rule-of-thumb ([15], page 60). In terms of the integrated squarederror (ISE) between the approximation an the underlying reference distribu-tion, the OKDE outperformed the suboptimal batch KDE, and approachedthe accuracy of the optimal batch KDE (Figure 7.2b). Note from Figure 7.2cthat the complexity of the mixture model produced by the OKDE remainslow even after many samples have been observed, while the complexity of themixtures produced by the optimal and the suboptimal KDE increases linearlywith the number of samples.As we have mentioned in the previous section, being able to learn fromall-positive examples is not flexible enough for continuous learning in cog-nitive agents. Indeed, in a noisy environment, false examples will often getaccidentally incorporated into the estimated mixture model and a mechanismfor unlearning is required to remove them from the model. We will illustratethe unlearning framework of OKDE with an example of interactive learningof the concept of a red color (Figure 7.3); for the details the reader is referredto [14]. Assume that we present a robot with an object to which we referas a red fork (see Figure 7.3a). The concept of a red color is constructed inform of a mixture model pred(x) (Figure 7.3b) from the sampled hue values(green dots in Figure 7.3a). By backprojecting pred(x) to the original image,we obtain the belief map in Figure 7.3c. Note that high beliefs are assigned tothe color of the fork’s handle and even higher to the color of its head. This isbecause the right-most mode in the pred(x) (Figure 7.3e) corresponds to thehue values from the forks yellow head. The agent would thus wrongly believe7 Multi-modal Learning 279200 400 600 80000.10.20.30.40.50.60.70.8number of samplesISE200 400 600 800101102103number of samplesnum.ofcomponents(a) (b) (c)Fig. 7.2. Comparison between approximations of a pdf using the OKDE andtwo batch KDE methods. Approximations after observing 1000 samples (a). TheISE between the approximations and the reference model (b), and the number ofcomponents in the approximations (c), w.r.t. the number of samples. Results of theOKDE, the suboptimal batch KDE and the optimal batch KDE are shown in redfull lines, blue dotted lines and green dashed lines, respectively. The reference pdfis depicted in (a) by black dash-dotted line.that the yellow as well as the red hues make up the concept of the red color.To rectify this, we then present a yellow ball (Figure 7.3b) and say that itscolor is yellow and NOT red. As before, hue values are sampled from theball and another mixture model, a negative example, pyell(x) is constructed(Figure 7.3f). The complement of this negative example pyell(x) (Figure 7.3g)can now be considered as an additional positive example, which assigns a lowprobability to those values which are likely to present the negative examples.Assuming that pyell(x) is independent from pred(x), the complement is fusedwith pred(x) by multiplication to yield a refined model p̂red(x). The refinedp̂red(x), after an additional compression step, is shown in Figure 7.3h. Notethat the mode corresponding to the yellow color has been attenuated, whichis verified in the resulting belief image (Figure 7.3d). We see that now onlythe pixels on the fork’s handle are believed to correspond the concept of thered color. This shows the flexibility of learning with OKDE: The agent wasable to learn the concept of a red color from a highly ambiguous data undersupervision in only two steps of interaction.7.2.5 Experimental resultsThe system presented in this section was primarily designed to work in in-teraction with a user and it forms a part of the PlayMate system. Fig. 7.1shows a sample dialogue between the tutor and the system. The first partof the dialogue took place in the tutor-driven learning mode, when the tu-tor was teaching the system about the objects in the scene, while the secondpart of the dialogue took place using the tutor-supervised modes of learning,when the system took the initiative and asked the tutor for clarification when280 Skočaj et al.0 0.05 0.1 0.15 0 0.05 0.1 0.15 0 0.05 0.1 0.15010 0.05 0.1 0.15a b c de f g hFig. 7.3. Hue values are sampled from (a) to initialize the mixture model pred(x)(e). The mixture pyell(x) built from the hue values of a yellow ball (b) is shownin (f). The complement of pyell(x) is shown in (g) and the unlearnt compressedmodel p̂red(x) is shown in (h). The belief images corresponding to pred(x) before andafter unlearning, (e) and (h), are shown in (c) and (d), respectively. White colorscorrespond to high beliefs, while dark colors correspond to low beliefs. The mixturesin (b,d,f) are show in bright (red) lines, while their components are depicted in blacklines.needed. Additional examples and more detailed description of architecturalissues are described in Chapter 9.To comprehensively analyse the proposed learning modes, we have alsobuilt a simulation experimental setup. We performed experiments on imageswith known ground truth, and simulated the answers of the tutor. In this wayextensive tests could be automatically performed and a reliable evaluation ofthe proposed methods were obtained. In the following we only briefly presentthe results; the detailed descriptions of the experiments can be found in [32,33, 34, 29].First, we present the results on learning visual attributes of objects. Ba-sic shapes of various different colours and sizes were selected as test objects.Some of them are depicted in Fig. 7.4(a). We considered three visual attributes(colour, size and shape), and ten values of these visual attributes altogether(red, green, blue, yellow; small, large; square, circular, triangular, and rectan-gular). The main goal was to find associations between these attribute valuesand six extracted features (like the median of the hue over all pixels in thesegmented region, or the area of this region, etc).We incrementally updated the representations with the training imagesusing different learning strategies. At each step, we evaluated the currentknowledge by recognising the visual properties of test images. The evaluationmeasure we used is recognition score, which rewards successful recognition(true positives and true negatives) and penalises incorrectly recognised visualproperties (false positives and false negatives).The results (the curves of the evolution of the recognition score throughtime) are presented in Fig. 7.4(c), while Fig. 7.4(d) plots the frequency of7 Multi-modal Learning 281communication between the system and the user, i.e. how often the user isrequired to provide information.All the different learning strategies presented in subsection 7.2.2 weretested. It can be seen that Tutor driven learning (TD) performs best andyields almost perfect recognition results. However, in this case the user hasto completely describe the object at every learning step. Tutor-supervisedlearning (TSc, TSl) proved to be quite successful as well (in the beginningthe system does not have a lot of knowledge, which would enable reliable au-tonomous recognition, so the first 10 training images were added in a tutordriven mode). Tutor supervised learning required significantly less interactionwith the tutor; the average number of questions drops to app. 2 in the con-servative and even to almost 0 in the liberal case. The exploratory approach(EXc, EXl), which does not involve interaction with the tutor, does not im-prove the model. So, as expected, there is a trade-off between the quality ofthe results and the autonomy of the system.Exactly the same system was also used for learning simple spatial relations.We only changed the features that were to be extracted from the image (posi-tion of the object centers in the scene, distance between the objects, etc.). Us-ing five such features, the learning framework was able to learn eleven spatialrelationships (like ’to the left of’, ’far from’, or ’near’). The correctly assignedassociations, along with the previously learned visual attributes, enabled theautomatic detection of objects and the production of scene descriptions suchas those presented in Fig. 7.4(b).Next, we present the results of the experiment performed on a set of ev-eryday objects (some of them are depicted in Fig.7.5(a)).Fig. 7.5(b) shows the evolution of the overall accuracy of the conceptrecognition, which increases by adding new samples (green line). The growthof the accuracy is very rapid at the beginning when new models of newlyintroduced concepts are being added, but still remains positive even after allmodels are formed due to refinement of the corresponding representations.Fig. 7.5(c) plots the average number of Gaussian components in the KDEmodels. One can observe that after a while this number does not grow anymore, thus the model size remains limited. The models do not improve byincreasing their complexity, but rather due to refinement of the underlyingrepresentations.To demonstrate the unlearning algorithm, we incorrectly labeled every 10-th training sample in the first half of the incremental learning process. As aresult, the underlaying KDE representations were corrupted and modelled alsothe feature values, which were not supposed to be encompassed. Therefore,the recognition results degraded (red line in Fig. 7.5(b)). However, by applyingunlearning to the corrupted representations, they were successfully corrected,which resulted in a significant improvement of the recognition performance(blue line). The recognition results after unlearning were very similar to thoseobtained in an error free learning process.282 Skočaj et al.ABA is yellow, small, and circular.B is blue, large, and square.A is on the left.A is near.B is on the right.B is far away.A is to the left of B.A is closer than B.A is far from B.B is to the right of A.B is further away than A.B is far from A.(a)0 10 20 30 40 50 600100200300400500600700no of added imagesrecognition score TDTScTSlEXcEXl0 10 20 30 40 50 60012345678910no of added imagesnumber of questions (b)(c) (d)TDTScTSlEXcEXlFig. 7.4. (a) Input shapes. (b) Automatically generated scene description. (c)Recognition score. (d) Frequency of communication.These results are further demonstrated on one particular example. Figs. 7.5(d-i) depict the evolution of the underlying KDE model of the feature ‘Hue’,which was associated with the concept ’Blue’, over time. It shows how theinitially simple model improves through time Figs. 7.5(d-f). It also shows theefficiency of the compression algorithm; the compressed model (Figs 7.5(g))resembles the original one (Fig. 7.5(f)) at a very high degree of accuracy. Fi-nally, Figs. 7.5(h,i) demonstrate the performance of the unlearning algorithm,which was able to undo the noise introduced into the model.7.2.6 Discussion and outlookIt this section we presented an interactive framework for continuous learning ofvisual concepts. The main goal is to learn associations between automaticallyextracted visual features and words describing the scene (visual attributes,spatial relations) in a user friendly, natural, open-ended, and continuous man-ner. The system facilitates interactive learning of basic visual concepts in adialogue with a human tutor. The learning can be performed on different lev-els of supervision. The developed framework also supports unlearning, whichenables recovery from the errors accumulated in the learned models.In the previous subsection the methods proposed were evaluated in isola-tion by explicitly providing the ground truth information, which is supposed to7 Multi-modal Learning 283(a)(b)0.523 0.603 0.684HuBl0.367 0.544 0.721HuBl0.367 0.544 0.721HuBl0.367 0.539 0.710HuBl−0.050 0.329 0.708HuBl0.367 0.538 0.710HuBl(d) (e) (f ) (g) (h)(i)0 50 100 1500102030405060708090100image numberrecognition rate correct labelscorrupted labelsunlearning0 50 100 15012345678image numbernumber of components (c)Fig. 7.5. Learning of basic object properties: (a) Seven everyday objects from thedatabase. Results of incremental learning: (b) evolution of accuracy through timefor incremental learning on correct data (green line), on partially incorrect data (redline), and after unlearning (blue line), (c) average number of components throughtime. Evolution of one KDE model representing the object property ‘blue’ throughtime: model after (d) 15, (e) 85, and (f) 120 added training samples. (g) Compressedfinal model. (h) Final model learned with incorrect labels. (i) Model after unlearning.be provided by other parts of a cognitive system setup. However, these meth-ods are fully integrated in the overall system based on CAST, and play anintegral role of the PlayMate system. Other parts of the system thus provideinformation needed by the learner (e.g., linguistic input) and use the infor-mation produced by the learner (e.g., recognition results). The entire systemthus provides visual input and enables verbal and non-verbal communicationwith a tutor facilitating continuous, and interactive cross-modal learning.The continuous learning framework we have developed is able to learnbasic visual concepts (like colours, shapes and spatial relations). A naturalextension of this system would be to make it fully scalable to enable it to learnmore complex concepts and hierarchies of concepts. Furthermore, the systemcould also consider other modalities in a larger part as well. A very interestingresearch issue is introspection and detection of ignorance, and planning of newactions that the system should undertake to gather the knowledge which wouldbe used to fill these gaps. This is a prerequisite for more efficient exploratorylearning, which would require minimal supervision of a human tutor. Thecurrent framework and the overall system we have developed is a solid basefor such research and may serve as a good starting point and a tool in furtherinvestigations and development.284 Skočaj et al.Fig. 7.6. System overview.7.3 Cross-modal learning of visual categoriesSection 4.1 already outlined a rough topology of existing work on visual cat-egorization. One axis along which such methods can be aligned is how theyare trained and which level of supervision they require. Interestingly, mosttoday’s object categorization methods use either supervised or unsupervisedtraining methods. While supervised methods tend to produce more accurateresults, unsupervised methods are highly attractive due to their potentialto use far more and unlabeled training data. This section proposes a novelmethod that uses unsupervised training to obtain visual groupings of objectsand a cross-modal learning scheme to overcome inherent limitations of purelyunsupervised training. The method uses a unified and scale-invariant objectrepresentation by Scale-Invariant Patterns (SIPs) that allows us to handle la-beled as well as unlabeled information in a coherent way. One of the potentialsettings is to learn object category models from many unlabeled observationsand a few dialogue interactions that can be ambiguous or even erroneous.First experiments demonstrate the ability of the system to learn meaningfulgeneralizations across objects from only a few dialogue interactions.System overview.Figure 7.6 shows an overview of the presented system, which is tighly re-lated to the structure of this section. Section 7.3.1 describes how the visionsystem represents the visual input. This representation allows for an unsu-pervised visual grouping step as described in Section 7.3.3. To overcome theinherent limitations of unsupervised as well as fully supervised approaches,we describe how we combine different levels of supervision in Section 7.3.3.Section 7.3.4 describes the language system that parses an utterance to a log-ical form. Section 7.3.5 explains the spatial reasoning processes that associatethe expressions with the visual observation. Finally we present two scenarios,in which the system successfully resolved ambiguous information and evenrecovered from erroneous beliefs.7 Multi-modal Learning 285C1...Cn...C1Cn...C1Cnbag-of-wordsspatialbag-of-wordsScale-InvariantPattern (SIP)Fig. 7.7. The scale-invariant representation we use is derived from the common bag-of-words representation. Recently a spatial resolution was added to each bin. We im-prove the robustness of these type of representations by adding a scale-normalizationstep yet retaining the spatial information which yields what we call a Scale-InvariantPattern (SIP). The figure visualizes how local featuers (blue dots) are entered in aspatially discretized histogram.7.3.1 Object Representation by Scale-Invariant PatternsIn this section we describe how we encode visual patterns and objects as whatwe call Scale-Invariant Pattern (SIP). SIPs are a key ingredient in our systemsas they provide an embedding of visual patterns and objects into a vector spacethat facilitates clustering and recognition in a cheap yet effective manner.First, we describe the low-level feature description we base our representationon and then derive the actual representation.When a new image is grabbed from the camera, SIFT descriptors [35]are extracted at Hessian-Laplace interest points [36]. While there exists awide range of interest point and descriptor combinations, we opted for thisparticular combination based on evaluations on different categorization tasks[37]. Following a common philosophy in the field to visual categorization [38,39, 40, 41, 42, 43], we first generate a visual codebook based on clustering ofdetected features. In our experiments we use a codebook with 1000 entriesobtained by k-means clustering.Based on this feature representation we derive one of the important in-gredients of our system: the Scale-Invariant Pattern (SIP) representation.This representation is the basis not only for discovering objects in the scene,but also for visual grouping and object categorization. We adapt the scale-invariant representation from [43] which already has shown its versatility inweakly-supervised learning and ranking tasks. As visualized in Figure 7.7, thisrepresentation can be seen as a extension of the bag-of-words representation[38] by adding two spatial dimensions to each bin [41]. In order to obtain ascale-invariant representation, the feature positions (posx, posy) are normal-286 Skočaj et al.Fig. 7.8. Visual grouping of objects by clustering.ized with respect to the center of the pattern (cx, cy):(pos′x, pos′y) =(posx − cxσ,posy − cyσ)(7.1)where σ is the detected scale of the feature. For efficiency, we store these pat-terns as sparse vectors Ψ . The sparsity of the SIPs is one key to the efficiencywe are aiming for while still maintaining a descriptive representation of thevisual input.7.3.2 Data-Driven Visual GroupingSimilarily to [44], we use an agglomerative clustering scheme (average link-age) to group object instances in an unsupervised manner. The objects arerepresented as SIPs (Sec. 7.3.1) which we will denote as sparse vectors Ψ inthe following formulas. As described before we normalize these to unit lengthand we use the scalar-product to measure similarity between these objects.We prefer to use agglomerative clustering over k-means as we do not want tospecify the number of visual clusters apriori. The threshold required for theagglomerative clustering scheme is set empirically to a constant value for allour experiments. Figure 7.8 visualizes the clusters C1 to CN obtained by oursystem given the observed objects displayed on the left. Although there aresome confusions, we observe a good generalization across category instances.In order to obtain representatives ΨCl for each cluster Cl, we compute aweighted sum of the observed patterns Ψk:ΨCl =∑kp(Cl|Ψk)Ψk (7.2)In our implementation, we have chosen to use hard assignment of the SIPs tothe clusters which renders the probability p(Cl|Ψk) of assigning pattern Ψk tocluster Cl binary.7.3.3 Combining Unsupervised and Supervised LearningIn this section, we present a model for visual category recognition that com-bines different levels of supervision in a joint model. The key ingredient is7 Multi-modal Learning 287the SIP representation from Secion 7.3.1, which we use throughout. The lastsection formulated an unsupervised grouping process using this common rep-resentation.Supervised CategorizationTo provide basic functionality for our system, we describe how supervisedcategorization is implemented. Similarly to clustering in Section 7.3.2, wemodel each category Ai by a single representative ΨSAi (superscript S denotesthe supervised model). This is done by summing over all training patternsavailable for that categoryΨSAi =∑j∈SAiΨj , (7.3)where SAi denotes the indices of the SIPs that are labeled with category Aiin a supervised manner (e.g. ”This is a bottle”).Incorporating Semi-Supervision and Unsupervised Information in oneconsistent FrameworkWe formulate the fusion of information obtained from supervised to unsuper-vised sources as an extension of the supervised case by assuming uncertaintyabout the correct labeling of the clusters Cl and their representatives ΨClfrom the unsupervised visual grouping step (Sec. 7.3.2):ΨAi = ΨSAi︸︷︷︸supervised+∑lp(Ai|Cl)unsupervised︷︸︸︷ΨCl︸︷︷︸semi-supervised(7.4)p(Ai|Cj) encodes the belief that cluster Cl contains instances of category Ai.How this probability is computed from a few interactions and updated byassociating spatial expression with visual observations is described in Section7.3.5.To perform classification in the supervised and semi-supervised case, weevaluate the proposed model ΨAi as well as ΨSAifor an observed SIP Ψ̄ byusing histogram intersection. Intuitively, the intersection measures to whichpercentage the model explains the observation, which we interpret as the prob-ability of belonging to the same class. In order to make models and observa-tions comparable we normalize both to one. Bayes’ rule is applied afterwardsto obtain the model posterior:p(Ψ̄ |ΨAi) =∑min(Ψ̄ ,ΨAi) (7.5)p(ΨAi |Ψ̄) =p(Ψ̄ |ΨAi)p(ΨAi)∑A p(Ψ̄ |ΨAi)p(ΨAi)(7.6)288 Skočaj et al.p(ΨAi) is the category prior, which we assume to be uniform. We decide forthe category label with the highest posterior:Âi = argmaxAip(ΨAi |Ψ̄) (7.7)7.3.4 Language SystemIn human-assisted visual learning, a human tutor provides the system withdescriptions of the current visual scene. To relate these descriptions to thevisual input, the system constructs a representation of the meaning of an ut-terance. While this section outlines the language system and how it interfaceswith the other components in this cross-modal learning task, a more detaileddescription is postponed to Capter 8 in the context of the full dialogue system.For this analysis of the utterances that accompany the visually perceivedscenes, we use a Combinatory Categorial Grammar[45] parser6. The parseruses a CCG grammar to relate the syntactic structure of an utterance to thepropositional meaning it expresses. Meaning is represented as an ontologicallyrichly sorted, relational structure similar to a description logic formula [46],which makes it possible to use ontologies to mediate between linguisticallyexpressed meaning, and the categories formed in the visual system. Using thehierarchical structure of ontologies, and the possibility to perform ontologi-cal inference over instances on these ontologies, provides a more general andbetter scalable approach to ”visual grounding” of language than provided bythe string-based approach proposed in e.g. [47], or previous ontology-basedapproaches such as [48, 49].In our scenario, utterances are typically predicative copulative sentencesin indicative mood (i.e. ”X is Y”), which assert that a given predication (”Y”)holds for the subject of the sentence (”X”). In our examples, the predicationconsists of a phrase that encodes a spatial relation (e.g. ”left of the bottle”or ”below the apple”). In the logical form, the subject is represented as the of the state description that is denoted by the utterance, whereasthe predication is represented as .We can thus easily derive the spatial configuration asserted in an utter-ance from its logical form representation. The following example shows sucha logical form that is the result of the parsing process of the utterance ”themobile is left of the bottle”:@b1:state(be ^ind ^(m1:thing ^ mobile ^unique ^sg ^specific_singular) ^(l1:region ^ left ^6 http://openccg.sourceforge.net7 Multi-modal Learning 289horizontal ^static ^(b2:thing ^ bottle ^unique ^sg ^+ ^specific_singular)))For entering utterances into the system, we connect the parser with a speechrecognizer as well as a keyboard interface. In the experiments, we mostly usethe keyboard interface as well as scripted input for larger evaluations.7.3.5 Scene ReasoningModeling spatial relations as perceived by the human is a challenge in it-self, as issues like reference frame and context have to be handled appropri-ately in situated dialogue systems [50]. Considering the scenarios and mainfocus of this section, we restricted ourselves to modeling four basic spatialrelations R ∈ {”leftof”, ”rightof”, ”above”, ”below”}. We employ triangularshaped distributions p(pos(Ψi),pos(Ψj)|R) defined in 2d image coordinates,where objects are referenced by their patterns Ψi and pos(Ψi) denotes theirposition in image coordinates. Although these distributions are representedas non-parametric kernel densities which lend themselves to online updating,we don’t explore this option here and keep them fixed as pre-defined in theexperiments.Spatial Reasoning.We formulate the association of a spatial expression E extracted from anutterance (see Sec. 7.3.4) with two patterns Ψi and Ψj with positions pos(Ψi)and pos(Ψj) observed in scene Sk, as the problem of finding the most likelypair P̂i,j of patterns: P̂(k)i,j = argmaxPi,j p(Pi,j |E,Sk),wherep(Pi,j |E,Sk) = p(Ψi, Ψj ,pos(Ψi),pos(Ψj)|E,Sk)= p(Ψi|E,Sk) p(Ψj |E,Sk) p(pos(Ψi),pos(Ψj)|E,Sk), (7.8)withp(Ψ |E,Sk) =∑hp(Ψ |Ah)p(Ah|E,Sk). (7.9)As we don’t model a complete category system yet, leave out contextual effectsand assume certainty about the expression E referring to the categories Ae1and Ae2 and the relation R, the equation simplifies top(Pi,j |E,Sk) = p(Ψi|Ae1)p(Ψj |Ae2)p(pos(Ψi),pos(Ψj)|R) (7.10)290 Skočaj et al.Finally, we insert the visual model from Eq. 7.4 to obtain a computationalmodel:p(Pi,j |E,Sk) = p(Ψi|Ψe1)p(Ψj |Ψe2)p(pos(Ψi),pos(Ψj)|R) (7.11)This formulation facilitates incorporating information and belief from previousinteractions as well as learning from scratch. If no information about the visualcategories is available p(Ψi|Ψe1) and p(Ψj |Ψe2) become uninformative and thesystem relies only on its notion of spatial relations p(pos(Ψi),pos(Ψj)|R). Thiscan lead to wrong associations. In the next section we present an example andshow that the system can successfully deal with this issue.7.3.6 Label Propagation and Conflict ResolutionIn the following, we describe two scenarios, that show the capabilities of oursystem to propagate information, resolve ambiguities and recover from errors.Scenario 1 - Label forward propagation.In the first scenario, one annotated example each for banana and mobile ispresented to the system. Then the system observes the scene as shown inthe screenshot in Figure 9(a) and the utterance ”the can is above the mo-bile” is parsed. The red lines visualize the observed relations between objectsin the scene. Very unlikely ones have already been pruned away by the sys-tem. By generalizing across category instances, system identifies ”mobile” and”banana” correctly (with probabilities 0.69 and 0.66 respectively) while eval-uating ”mobile” model for the banana results in a low probability of 0.15.Consequently the most likely relation is inferred correctly and displayed inlight green. A model for the category ”can” is created and the observed mo-bile is added to the existing model for ”mobile”. In fact, the figure shows thestate in which the acquired ”can” model is already used for detection. The canis detected correctly, but also the bottle gets a high score for the ”can” model,as it’s the best explanation given the learned categories (banana, mobile, can).Scenario 2 - Label backward propagation.In the second scenario, we show how the system can recover from erroneousbeliefs and update its models accordingly. The system starts without anyknowledge about visual categories. Figures 7.9 (b) show a screenshot dis-playing the scene as observed by the system, which is accompanied by theutterance ”the can is above the mobile”. Using the same visualization as inthe previous scenario, it can be seen in the left image that the most likelyrelation inferred by the system is wrong. Now we provide the system withsupervised knowledge of the visual categories bottle and pen. Revisiting thescene in memory, the object probabilities get updated and the belief aboutthe associated relation gets changed to the correct one as shown in the rightimage.7 Multi-modal Learning 291(a) (b)(c)Fig. 7.9. (a) shows an example scenario for propagation of labels from knowncategories to unknown ones. (b) and (c) Show a scenario where the system updatesassociations (light green) to recover form an incorrect belief7.4 Learning complex actions7.4.1 IntroductionIn this section, we describe the development of an action analysis system fora cognitive robot. Our goal is to make the robot aware of the activities thatoccur in its environment so that it can react in an intelligent manner. Thisscenario has a different set of requirements from typical video analysis ap-plications. Detecting and labeling motion patterns in videos is not sufficientand the robot needs to reason about the intention of the agent and the func-tional specification of the concepts of objects and situations. A more suitablerepresentation is hence required in order to solve these cognitive tasks. Wepropose a novel action representation that addresses these issues based on thefollowing principles.Psychologically realistic modelling of intentional actionsA general approach to inferring the intention of an agent is to match theobserved motion patterns against a set of typical behaviours known to the ob-server. The challenge here is to account for uncommon motion patterns. Hu-man observers will first try to account for the variation by drawing additionalevidence from the scene to check if something could have forced the agent to292 Skočaj et al.perform the action in a different manner. When the collected evidence fails toaccount for the uncommon movement, the inference is that the agent actuallyintended to perform the action in a different manner. In order to achieve thiskind of reasoning in a robot, we propose to model intentional actions usinga factorized hidden Markov model (FMM). By explicitly modelling the cor-relation between hand movements, object states and environmental context,FMM allows the robot to quickly generate an alternative explanation for un-common hand movement, which is exactly the kind of reasoning we performin almost any everyday event.Integrating probabilistic action rules with Markov networksIn order to put object manipulation in the context of human activities, wepropose to upgrade probabilistic action rules to model the correlation betweenthe motion patterns, the properties of actions, its start situations and the ac-tion outcomes with probability distributions. These rules allows for functionalspecification of concepts that enable the robot to reason about the conceptof objects (or situations) on which (or in which) the agent tends to performcertain kinds of actions, and the class of execution styles that lead to theaction being executed successfully.The remainder of this section proceeds as follows. We first describe our rep-resentation of actions and behaviours in Section 7.4.2. Inference and learningalgorithms are discussed in Sections 7.4.3 and 7.4.4. Section 7.4.5 summarizesour experiment results, and we conclude in Section 7.4.6.7.4.2 Action representationLet us consider a toy domain of color and shape games. In this domain, theagent sits near a table on top of which two colored flags are placed, one onthe right and the other on the left. A colored geometrical shape (circular orsquared) is then placed in front of the agent. In the color game, the agentmoves the shape to the flag that has the same color as the shape. In the shapegame, the agent moves the circular shape to the left flag and the squared shapeto the right flag. In general, the object manipulation/movement varies fromtrial to trial, and may not be observed in its entirety. The agent may need toreach around an obstacle, or may fail to push the shape because the objectsurface is slippery. In order to infer the agent’s intention, the observer thereforeneeds to take into account the surrounding context and make hypothesesabout things that are ambiguous. Below, we present our action representationthat supports these kinds of reasoning. We first describe our representationof object manipulation/movement using a hierarchical model, as shown inFig. 7.10. We then describe how to put these movements in the context ofgame playing.7 Multi-modal Learning 293(a) (b)Fig. 7.10. (a) System Overview, (b) Factorized HMMFactorized, hidden Markov modelsA complex manipulation consists of a sequence of simple actions, (e.g., “reachfor the colored shape”, “push the color shape to the left flag”) as shown inFig. 7.10 (a). Each simple action is associated with a goal (g), which is thedesired pose of the target object. A common approach to modelling thesemotion sequences is based on hidden Markov models (HMM) [51, 52], wherean action corresponds to a node in the HMM. Associated with a HMM nodeis a joint distribution of motion features that characterize the action. Thesemotion features are computed at time t as Et = {fh, fo}, where fh denotehand motion features and fo denote the features of object motion and pose.In [53], we argue that reasoning about intentions is more realistically mod-elled by a factorized hidden Markov model (FMM), where we take a viewpointof Markov Decision Process (MDP) [54]. In FMM, an action node is factoredinto hand maneuver (h) and object state (o), which are associated with fhand fo respectively. The links between h, o, and g inside the rectangular boxon the left of Fig. 7.10 (a) represent the fact that the agent who aims toachieve gi = g1 chooses ht at time t based on ot−1, such that ot−1 transformsinto ot after the execution. This repeats until g1 has been achieved (i.e., untilot−1 becomes similar to g1). The agent then aims for another goal (e.g. g2).In Fig. 7.10 (b), an upper triangle node in Fig. 7.10 (a) is expanded to showthat the value of gt is conditioned on the value of the et node, which checkswhether ot−1 has satisfied the current goal. We note that we use the trianglenodes g1 and g2 in Fig. 7.10 (a) instead of expanding them because the goalshave not evolved during the execution.294 Skočaj et al.Probabilistic action rulesIn the game scenario, the evolution of the agent’s behavior will depend onthe situation that the agent is in. If the agent is playing a color game, thefollowing might occur: “get the shape”, “push the shape to the flag that hasthe same color as the shape”, then “remove the shape”. If we represent eachof these steps by a rule that maps the initial state to the outcome state, actionrecognition can be reduced a constraint satisfaction problem, where the mostlikely sequence of actions is the one that satisfies the most constraints (asimposed by the rules). However, executing these actions may not lead to theexpected outcomes. The agent may fail to push the shape because the shapeis made of slippery material. A robot may execute “pick-and-place” moresuccessfully than “pushing”, where as it may be the other way around for ayoung child who has not mastered grasping. Reasoning about actions thereforedepends on the situations. In order to handle the stochastic nature of anaction, we represent an action by a probabilistic action rule that allows us toassociate the transition dynamics of world states with probability distributionsP (s′|s, a), where s, s′ ∈ S (a world state described in terms of visual propertiesand relational features) and a ∈ A (a set of actions). A typical probabilisticaction rule a(x̄) that involves a set of objects x̄ has the form:∀x̄.Ψ(x̄) ∧ a(x̄) → • p1 Ψ ′1(x̄)· · · · · ·pn Ψ′n(x̄)(7.12)where Ψ(x̄) is the state before (antecedent) executing the action, and Ψ ′i(x̄)is a possible outcome which is associated with a probability pi, subject to aconstraint that∑i=0,...,n pi = 1. By default each rule has an empty outcome(no change) that represents action failure and noise. We say that a rule coversa state Γ (C) and action a(C) if there exists a substitution σ mapping thevariables in x̄ to C.In [55], P. Domingos et al. represent actions by probabilistic logic rules,which correlate (action) outcome states to the action properties specified inthe antecedent. Our probabilistic rules upgrade their logic rules by integratinginformation about action processes into the rules, which enables the observerto make inferences about the kinds of execution styles the agent tends to per-form, and the situations in which they are likely to succeed. We can restructurethe terms in the antecedent into three groups: deictic references, actioncontext (AC), and outcome context (OC). Each deictic reference consistsof a variable Vi and restriction ρi which is a set of features that define Vi withrespect to the variables x̄ in the action, as well as to the other Vj such thatj < i. In order to determine that an example is covered by a rule, we mustensure that these deictic variables can be resolved. Action context consistsof features that condition the decision to act using a. These features impose7 Multi-modal Learning 295restrictions on the properties of deictic variables; e.g. the types of objects,their colors and shapes. Outcome context consists of features that may causevariations in the outcome of the action rule. The following is an example ofan action rule that describes an agent playing a color game. The rule containsthree deictic variables: X (a geometrical shape), A (an agent) and Y (a flag).It also contains a restriction that X is located at the area R0 (front) of A.The AC part of the rule contains two features. The first represents the factthat there is no color variation between X and Y (i.e., they are of similarcolor). The second feature contains information that is provided by anothermodality (e.g., speech), which states that A is engaged in a color game. TheOC and Outcome parts indicate that the agent pushes X to the front (R0)of the flag Y with a success rate of 80%.playColor(X) : {X : shape(X)A : agent(A), at(X, A.R0)Y : flag(Y )}: AC → colorV ariation(X, Y, 0), engageIn(A, colorgame): OC → manipulatedWith(X, push):Outcome → 0.8 : at(X, Y.R0)0.2 : nochange7.4.3 InferenceGiven a set of action rules ri, i ∈ {0, ..., R}, the system can make inferencesabout unknown parameters by computing their likelihood from the observa-tion. For example, in the game scenario, we can make inference about whichgame the agent is playing by comparing the likelihood of ri = playColor(X)and rj = playShape(X).Computing the likelihood of action rulesFor each rule ri, we obtain a set of possible substitutions, each of which mapsthe deictic references of the rule to the objects in the scene. For substitutionσs, the rule ri is instantiated by substituting all variables that appear in therule according to σs. We then compute the joint probability of the terms inAC to obtain P (AC) (i.e. the likelihood of the precondition(s) of the rule).If P (AC) is larger than a threshold δ, we say that the rule can be instanti-ated with σs. For the instantiated rule, we verify that the agent executes theaction according to the type of manipulation that would lead to the resultsspecified in Outcome. This is done by matching the observed motion patternswith the appropriate FMM (factorized HMM). In our case, the manipulationtype and outcome state specified in the OC and Outcome parts of the ruleprovide a template for creating this FMM. For playColor(X), an FMM of296 Skočaj et al.type “push X to Y.RO” is created. At time T , the system can check if theaction rule has been completed, by computing P (Outcome) in the same waythat it computes P (AC). If the agent has not completed the action, we cancompute the likelihood of the rule as P (AC)P (OC), where P (OC) is the like-lihood of the FMM. The outcome can then be predicted with the likelihoodof P (AC)P (OC) ∗ 0.8.Computing the likelihood of FMMsAs shown in Fig. 7.10 (a), object manipulation may consist of multiple actions.Since we only consider goal-based actions, let us represent an action i bygi ∈ G, where G is the set of m actions known to the observer. To “pushX to Y.RO”, the agent may first reach for the object X (goal-based actiongi), then push it to Y.R0 (gj), and retract the hand (gk). This behaviour isencoded in a FMM by am×m transition matrix Ω that models the probabilitydistributions over the transitions between actions. Below, we first discuss thecomputation of P (gi0:t|E0:t), or the likelihood of action i being observed fromtime frames 0 to t. We then discuss the computation of the likelihood of actionsequences based on Ω. Let E0:t be {E0, E1, . . . , Et}, where Et = (fh, fo)t. Asdescribed in detail in [53, 56], we compute P (gi0:t|E0:t) using Eq. 7.13.P (gi0:T |E0:T ) = P (gi0:T )P (E0|o0)TYt=1Xht=hu∈n(h)P (Et|ht)P (Et|ot)P (ot|ot−1ht)P (ht|ot−1, git)(7.13)We characterize action i by the hand maneuver types hu ∈ Hi that areconditioned on the states of object ov ∈ Oi as illustrated in Fig. 7.10 (b).In Eq. 7.13, the product of probabilities after∑is the likelihood that theagent is performing action i by executing a maneuver type hu in response tothe observed state of object at time t. Assuming that the state transition isdeterministic, we only combine the likelihood of object state P (Et|ot) withP (ht|ot−1, git) and the likelihood of the hand maneuver P (Et|ht). The com-bined probability is then propagated to the next time frame. The likelihoodof action i therefore gradually increases when the observed hand maneuvermatches the expectation, otherwise it decreases.For a multi-step manipulation p (fmmp), we characterize the transi-tion patterns among m actions by a transition matrix Ω, where the el-ement at row i and column j corresponds to ωij = P (gt = i|gt−1 =j, et−1 = 1). We can then derive the most likely sequence of actions fromE0:T subject to Ω. This is done by searching for ti, tj , ...tk that maximizesP (gi0:tigjti+1:tj ...gktk′+1:T|E0:ti:tj :...:T , Ω), where gi0:ti means action i (i.e. gi) oc-curs from t = 0 to t = ti, and k′ is the action that occurs prior to k. FromFig. 7.10 (b), this can be computed as follow:7 Multi-modal Learning 297P (gi0:tigjti+1:tj...gktk′+1:T|E0:ti:tj :...:T , Ω) = P (gi0:ti |E0:ti)P (eti = 1|Eti)ωjiP (gjti+1:tj|Eti+1:tj )P (etj = 1|Etj )...ωkk′P (gktk′+1:T|Etk′+1:T )(7.14)We compute P (gjti+1:tj |Eti+1:tj ) using Eq. 7.13 and P (etj = 1|Etj ) is theprobability of the goal state of gj being true at time tj . The computationcomplexity of Eq. 7.14 (for short, we call this P (fmmp)) can be exponentialin time frame T , because ti, tj , .., tk can take any value from 0 to T . In [57],we present a variant of Viterbi’s algorithm that efficiently computes Eq. 7.14in linear time. Putting this into the context of action rule recognition, we letP (OC) = P (fmmp) for the rule that has p specified in the OC part. If theobserved actions match the pattern of fmmp, P (fmmp) will be high, whichincreases P (AC)P (OC) accordingly.7.4.4 LearningIn order to compute Eq. 7.13, we need the models for hand maneuversP (hu|E), object states P (ov|E) and the policy for choosing hand maneuversP (hu|ov, gi). In [53, 56] we present a supervised learning algorithm for thesemodels. The user provides a set of segmented training video sequences, fromwhich fh and fo are computed. Most discriminative features are then selectedand clustered into groups based on a co-occurrence criteria. A classifier forhu is then learned for each group of hand movement features (and similarlyfor ov) based on the Bayes’ assumption that all selected features f are inde-pendent given hu. Finally, a noisy-OR generative causal network with linksfrom ov (cause nodes) to hu (evidence nodes) is learned, and P (hu|ov, gi) iscomputed from the causal power associated with the links of this network, asshown in [58].For a multi-step manipulation p, we need to learn Ω that is associated withthe goal state of each step. In [57], we present an iterative algorithm similarto the learning algorithm in HMMs for learning Ω from a training data setQ, where each q ∈ Q contains a single demonstration of behaviour. In HMMsthe probability of a transition from one state to another is learned, whereasin our case, the probability of a transition from one goal state to another alsodepends on the kinds of movements that are associated with the goal states.In other words, the probability of “push X to Y then retract the hand” isdifferent from “pick up X and move it to Y then retract the hand”.7.4.5 ExperimentsIn this section we present experimental results to test the performance of ourFMM models.Psychologically realistic modelling of intentional actionsSimilar to our modelling of the correlation between (h, o, g), it has been shownby psychological experiments that a human infant appears to maintain a rep-resentation of actions that relate goal states, actions, and situation constraints298 Skočaj et al.a) Habituation 1 b) Habituation 2 c) Test 1 d) Test 2Fig. 7.11. The experimental events to test the understanding of goal-directed ac-tions by one-year old infants.to one another via the rationality principle. In [59], infants were separated intotwo groups. Infants in the first group were habituated to events depicted inFig. 7.11 (a) (“jumping over an obstacle”), while those in the second groupwere habituated to events depicted in Fig. 7.11 (b) (“jumping over nothing”).After habituation, they were shown events in Figures 7.11 (c) and (d), and thelooking time was measured. The results have shown that on average the infantshabituated to Fig. 7.11 (a) spent almost 5 seconds longer looking at Fig. 7.11(c) than Fig. 7.11 (d). In contrast, the infants habituated to Fig. 7.11 (b)spent equal amount of time (around 1 sec difference) looking at Figures 7.11(c) and (d). To the infants in the second group, “jumping over nothing” isquickly associated with the intention of the executor, whereas to the infantsin the first group, the movement contradicts their expectation. In [53, 56], wehave repeated this experiment by training two FMM-based action models torepresent the two groups of infants, and use −log(P (gi|E1:T ) as a measureof the degree of surprise on observing the test sequence E1:T . Our resultsshow that by taking into account the correlation between environmental con-text and hand movements, our action model has common properties with theaction learning in human infants.Recognition of complex actionsWe have validated our complex action recognition approach on three typesof action sequences. In a “putting” sequence, the agent puts a new objectin front of itself. In a “getting” sequence, the agent moves an object that isalready in the scene in front of it. In a “scrubbing” sequence, the agent scrubs(or pretends to scrub) a surface by pushing and pulling the object. We askeda user to perform each behaviour five times and use them to learn the transi-tion matrices described in Section 7.4.4. In each trial, we changed the initiallocation of the object and the direction of scrubbing. In order to evaluate thelearned behaviours, we collect two test videos of each behaviour. We segmentactions in the videos, and compute the likelihood of each behaviour usingEq. 7.14. We compared the most likely behaviour with the ground-truth andobtained a 83.33% detection rate (i.e., five out of six behaviours are correctlydetected). The failure occurs in one of the “putting” videos, which is causedby errors in object tracking.7 Multi-modal Learning 299Based on our probabilistic action rules, we have put these manipulationpatterns in the context of color and shape games (Section 7.4.2). We tested ourinference mechanism by asking a user to play color and shape games five timeseach. At the beginning of some trials, we put the left flag to the right and viceversa, and choose a different colored shape to play with. At the end of the trial,we compute the log ratio of P (AC)P (Outcome) of the two rules to determinewhich game the agent has played. During the trials where P (Outcome) islow, we compute, at each time frame, the log ratio of P (AC)P (OC) of thetwo rules instead. We have obtained 100% accuracy at the end of all trials.During the trial, the performance has dropped to 78.57% due to tracking noiseand the ambiguity in the detection of motion patterns (e.g., when the handpauses). While being less reliable, reasoning about movement provides usefulinformation for learning about failure.7.4.6 ConclusionIn this section, we have proposed an integration of factorized hidden Markovmodels (FMMs) into probabilistic action rules as a representation of humanbehaviours. This integration allows us to reason about motion patterns aspart of understanding about human intentions, as well as to relate functionalconcepts of an object to its properties. Our action learning and recognitionsystem (ALRS) can be embedded into a reasoning apparatus of any intelligentsystem that requires information about the behaviour of agents. In the CoSyproject, we embedded ALRS into a robotic system and used it to represent andreason about object manipulation during a game activity. Such informationis then used by a planner to plan the robot’s actions. We believe that ourprobabilistic action rules are also suitable for representing both the robot’sand the agent’s actions. Having a common representation would allow therobot to ground the action models to their own actions, and to bootstrap themodels and acquire models that are specific to its configuration.7.5 Functional object class detection based on learnedaffordance cuesIn recent years, computer vision has made tremendous progress in the fieldof object category detection. Diverse approaches based on local features, suchas simple bag-of-words methods [38] have shown impressive results for thedetection of a variety of different objects. More recently, adding spatial infor-mation has resulted in a boost in performance [41], and combining differentcues has even further pushed the limits. One of the driving forces behindobject category detection is a widely-adopted collection of publicly availabledata sets [60, 61], which is considered an important instrument for measur-ing and comparing the detection performance of different methods. The basisfor comparison is given by a set of rather abstract, basic level categories [62].300 Skočaj et al.These categories are grounded in cognitive psychology, and category instancestypically share characteristic visual properties.Michael Stark - TU DarmstadtCoSy Review 2007From Abstract to Functional Object Categories2Sittable Handle-graspablePourableSidewall-graspableChair SpoonMug Watering can HammerSalt shakerMichael Stark - TU DarmstadtCoSy Review 2007From Abstract to Functional Object Categories2Sittable Handle-graspablePourableSidewall-graspableChair SpoonMug Watering can HammerSalt shakerFig. 7.12. Basic level (left) vs. functional (right) object categories.In the context of embodied cognitive agents, however, different criteriafor the formation of categories seem more appropriate. Ideally, an embodied,cognitive agent (an autonomous robot, e.g .), would be capable of categorizingand detecting objects according to potential uses, and w.r.t. their utility inperforming a certain task. This functional definition of object categories isrelated to the notion of affordances pioneered by [63].Figure 7.12 exemplifies the differentiation between functional and basiclevel categories, and highlights the following two key properties: 1) functionalcategories may generalize across and beyond basic level categories (both amug and a watering-can are handle-graspable, and so is a hammer), and 2)basic level categories can be recovered as composite functional categories (amug is both handle-graspable, sidewall-graspable, and can be poured from).7.5.1 Related WorkAttempts to detect objects according to functional categories date back to theearly days of computer vision. [64] was among the first to suggest functionalcharacterizations of objects as consequences of basic geometric properties. [65]pioneered a body of work on functional categorization of CAD-inspired face-vertex object descriptions by geometric reasoning, and was later extended byvisual input for recognizing primitive shapes from range images of idealisticscenes [66]. [67] introduced an explicit mapping between geometric and corre-sponding functional primitives and relations, again restricted to a small classof parametric shapes. [68] added force feedback for distinguishing among dif-ferent tools that afford piercing other objects. Only recently, [69] stepped intothe direction of more realistic settings, recognizing previously unseen, realworld objects, but specifically tailored towards grasp point prediction. Theapproach is based on training a logistic regression model on annotated syn-thetic images, combining 2D filter responses with 3D range data in a dense,multi-scale image representation.In [70], we approach the challenge of functional object categorization froma completely different angle. First, we build our system on robust and well-established grounds in the field of object recognition, applicable to real-worldimages of cluttered scenes. We explore the capabilities of a widely adopted de-tection framework, based on a suitable geometric local feature representation.7 Multi-modal Learning 301Second, we choose to acquire functional category representations by observ-ing few prototypical human-object interactions rather than explicitly model-ing physical object properties. Naturally, the set of functional categories thatour local feature-based vision system will be able to represent is restrictedto those that are characterized by distinct visual features. As an example,consider the bent shape of a mug handle, which suggests to grasp the mugin a specific way. We call such distinct visual features affordance cues, andbase our system for functional object category detection on the recognition ofthese cues. We report first results for the detection of two functional objectcategories learned by our system, and demonstrate their generalization ca-pabilities across and beyond basic level categories. We show that our systemsupports the interpretation of these categories as composite functional ones.7.5.2 Affordance Cue AcquisitionGiven an observed human-object interaction featuring a single affordance cue(a video sequence plus tutor guidance), the purpose of the affordance cueacquisition sub-system is to obtain a visual feature-based representation ofthat cue. It proceeds by first estimating an accurate per-pixel segmentation ofthe interaction region (the region where tutor and object pixels overlap duringinteraction), and then extracting features in a local neighborhood around thatregion. Tutor guidance informs the system about the beginning and the endof an interaction. Figure 7.5.2 gives an overview of affordance cue acquisition,which is detailed in the following.Michael Stark - TU DarmstadtICVS 2008, Santorini, GreeceAffordance Cue Acquisition8FG/BGSegmentationSkin Labeling Region MatchingFeatureExtractionObservation Tutor Guidance- “Robot, this is a jug.”- “Robot, this is grasping the jug.”...Scene SegmentationAffordance CueIdentificationAffordance Cue Acquisition8Fig. 7.13. Affordance cue acquisition overview.Foreground/Background Segmentation and Skin LabelingWe employ the Background Cut [71] algorithm for foreground/backgroundsegmentation, combining global and local Gaussian mixture color modelswith a data-dependent discontinuity penalty in a Conditional Random Fieldmodel [72]. In order to distinguish human tutor and manipulated object,we apply a likelihood ratio test on all pixels labeled as foreground by fore-ground/background segmentation. We build the ratio between the likelihood302 Skočaj et al.of a pixel originating from object color and the corresponding likelihood forskin color, using a pre-trained skin color model [73]. Figure 7.5.2 includes anexample labeling (black denotes background, white object, and gray skin).Region MatchingWe determine the interaction region as the set of object pixels that has beenoccluded by the human tutor in the course of an interaction. We identify thosepixels by choosing two frames from the interaction sequence, i) one during theinteraction, and ii) one after (but with the object still visible). Then, the setof occluded object pixels is computed as the intersection of all skin-labeledpixels of frame i) with all object-labeled pixels of frame ii), transformed inorder to equalize object pose differences between the two frames. The trans-formation is obtained by estimating the homography between frames i) andii), using RANSAC. Initial point-to-point correspondences are established byRobust Nearest-Neighbor Matching of SIFT descriptors [74] on Harris-Laplaceinterest points [75].Feature ExtractionOur representation of affordance cues is based on geometric local featurescalled k-Adjacent Segments (k-AS) [76], and justified by the results of ourevaluation of local features presented in Section 4.2.2. k-Adjacent Segmentshad initially been proposed in the context of shape-matching line drawingsto real images [77]. k-AS detect distinct edge segments in an image, formgroups of k such segments, and then encode their relative geometric layoutin a low dimensional, scale-invariant shape descriptor. In our experiments,we consistently use k = 2, since 2-AS features have shown a good discrim-ination/repeatability tradeoff [78]. We augment the groups returned by the2-AS detector by additional pairs of edge segments according to perceptualgrouping criteria in the spirit of [79].7.5.3 Functional Object Category DetectionA variant of the Implicit Shape Model (ISM) [80] serves as the basis for ourfunctional object category detection system. We extend the original model inorder to allow for independent training of several different affordance cues, andflexible combination for detecting composite functional categories. Figure 7.14gives an overview of the ISM. Training an ISM for an affordance cue amountsto matching acquired affordance cue features to a previously built codebook,and storing the relative position (x, y) and size (scale) of the object w.r.t. thefeature occurrence along with matched codebook entries. For detecting an af-fordance cue in a previously unseen image, all features in a test image are againmatched to the codebook. For every matched codebook entry, each stored fea-ture occurrence probabilistically votes for a hypothesized object position ina generalized three-dimensional Hough voting space (x, y, scale). The modes7 Multi-modal Learning 303Michael Stark - TU DarmstadtICVS 2008, Santorini, Greece• Implicit Shape ModelAffordance Cue-Based Object Category Detection12Generic kAS-codebookAgglomerativeClustering...CB Matching /OccurrenceCollection Affordance CueFeatures[Leibe, Leonardis, Schiele. Toward Category-Level Object Recognition. 2006]CB Matching /Density EstimationFeature ExtractionMode Estimation /Back-ProjectionTest Image12Fig. 7.14. Implicit Shape Model Overview.of the resulting vote distribution constitute hypotheses on possible affordancecue positions and scales, enabling edge-level localization by back-projectingthe corresponding features into the image plane (see Figure 7.16).7.5.4 ExperimentsWe report qualitative results for the detection performance of our system ona subset of the ETHZ Shape Classes data set [77], and a series of images fromthe PlayMate scenario.The handle-graspable category.We begin by giving results for the handle-graspable functional category (rows(a) to (c) of Figure 7.16), learned from affordance cue features of single imagesgiven in column (1). We observe that the models learned from either of thethree mugs perform comparably well in detecting handle-like structures inthe given test images, despite apparent appearance differences between theobjects used for training and testing, and considerable background clutter.Row (d) highlights the generalization of a handle-graspable model learnedfrom mug (1)(c) over other object categories such as coffee pot, vase, andelectric water jug. Image (5)(d) indicates the limitations of our approach.While the detector mistakenly fires on a circular sign in the background (falsepositive), it misses an obvious handle-graspable affordance cue on the whiteThermos bottle (false negative). While the false positive can be explained bythe limited information encoded by the 2-AS features, the false negative maybe attributed to predominant background structures.304 Skočaj et al.The sidewall-graspable category.Rows (e) and (f) of Figure 7.16 show the detection results for a secondcategory, sidewall-graspable, again learned from single images. In row (e),a model has been learned from a bottle, and from a mug in row (f). Thesidewall-graspable detector exhibits remarkable performance in the detectionof sidewall-like structures in cluttered images, although it is slightly more sus-ceptible to false positives than the handle-graspable detector, again due to thelimitations of the employed features (see (e)(5)).The handle-graspable/sidewall-graspable category.We now combine both handle-graspable and sidewall-graspable affordance cuesby training two independent ISM models, one for each cue, and joining theirpredictions for detection. In fact, the combination of both cues improves thedetection performance of our system (example detections are given in row(g)). In particular, the sidewall-graspable affordance cue compensates for in-accuracies in the localization of handle-graspable features. By back-projectingfeatures, the joint detector is able to to distinguish and accurately localizeboth of the two affordance cues, shown in yellow (handle-graspable) and red(sidewall-graspable).Grasping.Figure 7.15 depicts an attempt to grasp a jug at its handle by a robot armmounted onto our agent, after the system has acquired the corresponding af-fordance cue. Although actual grasping fails due to limited visual servoingcapabilities, the agent manages to touch the jug at the correct position. Weapplied affordance cue acquisition independently for two cameras of a cali-brated stereo rig, and obtained 3D coordinates by triangulation.Michael Stark - TU DarmstadtCoSy Review 2007 6Left camera Right camera Grasping attemptFig. 7.15. Binocular affordance cue acquisition and resulting grasping attempt.7.6 Conclusion and outlookIn this chapter we have explored several issues related to multi-modal learningfor recognition and interaction of cognitive agent with its environment. In7 Multi-modal Learning 305(a)(b)(c)(d)(e)(f)(g)(1) (2) (3) (4) (5)Fig. 7.16. Example detections. Each row corresponds to a single experiment, unlessotherwise stated. For each row (a) to (g), columns (2) to (5) give example detectionsfor a system that has been trained solely on the highlighted affordance cue featuresin column (1). Line segments are plotted in yellow, and pairs selected as 2-AS featureare connected by a blue line. Row (d) continues example detections of row (c), androw (g) depicts detections from a system trained on affordance cue features (1)(a)and (1)(f). Back-projected edges from the handle-graspable detector are plotted inyellow, those from the sidewall-graspable detector in red.306 Skočaj et al.particular we have focused on (1) representations for different modalities,(2) levels of supervision in interactive learning, (3) exploitation of differentmodalities to improve learning and recognition, and (4) learning in embodiedagent.In Section 7.2 we have considered the problem of online, open-ended, learn-ing of visual concepts. The goal was to learn associations between low-levelvisual features such as color, shape and spatial relations and the words describ-ing the scene. Therefore, the developed system associates words, descriptionsof the scene provided by the tutor, with the representations of the low-levelvisual features. Four general requirements for an open-ended learning systemwere defined and generative models called the online Kernel Density Estimateswere explored in the context of online learning. The learning framework wasembedded into the PlayMate system, within which we have studied differentlevels of supervision in tutor-agent learning: tutor driven, tutor supervised andexplorative approach. Not surprisingly, experiments have shown that, with thetutor-driven learning, the robot achieved an almost perfect recognition score.However, this came at a cost of tutor’s constant involvement – at each learningstep, the tutor had to describe the visual input (e.g., image with objects) com-pletely and without errors. In tutor-supervised learning, the robot would askfor the tutor’s assistance only when it could not recognize the visual inputwith sufficient confidence. When unaware of its ignorance, the robot wouldoften incorrectly recognize the visual data and then incorrectly learn (up-date its representations) from that. The incorrect updates of representationswould eventually lead to the robot’s reduced confidence about the recogni-tion/classification of visual data. At that point the robot would start askingthe tutor about the visual data more frequently and the tutor would providethe correct information through which the robot’s representations would becorrected. Thus in a tutor-supervised framework, the tutor would act as aloose feedback control loop for the robot’s learning. While the robot’s recog-nition ability would decrease a bit in comparison to the tutor-driven learning,the tutor-supervised learning required significantly less interaction with thetutor and made the robot more autonomous in two ways: in the initiative tolearn as well as in the initiative to engage verbal communication with the tutorwhen asking for clarification of the observed scene. The explorative approach,which did not involve any interaction with the tutor, produced the poorest re-sults. This clearly indicates a trade-off between the quality of the results andthe autonomy of the system – a learning autonomous system which knowstoo little and passively observes the environment, is likely to correctly learnonly little or nothing. These results indicate a need for the system’s ability tointrospect, be more aware of its ignorance and plan ways to interact with theenvironment as a prerequisite for more efficient and exploratory learning.A combination of supervised and unsupervised learning was also exploredin Section 7.3 where a tutor-driven approach to cross-modal learning wastreated probabilistically. Visual categories were encoded through a scale-invariant object representation called Scale-Invariant Patterns (SIPs). This7 Multi-modal Learning 307representation allowed a coherent handling of labelled as well as unlabelledinformation. An important feature of SIPs is that they provide an embeddingof visual patterns and objects into a vector space that facilitates clusteringand recognition in an effective manner, which accounts for some of the gen-eral requirements for the open-ended learning system, which were defined inSection 7.2. The spatial relations (”left of”, ”right of”, ”below”, ”above”) be-tween objects were encoded a priori using triangular distributions which weremodelled using Kernel-Density Estimates. We then considered a problem oflearning an object category models from many unlabelled observations anda small number of dialogue interactions with the tutor. We allowed the in-formation provided by the tutor to be at least ambiguous or even erroneous.Experiments have shown that the robot was able to generate meaningful gen-eralizations across objects already from a few dialogue interactions. Since theprocess of learning was posed probabilistically, the uncertainty about cate-gories was contained within distributions. During learning, the robot was thusable to propagate information, resolve ambiguities as well as recover from theerrors. These results substantiate the findings about the tutor-driven learningdiscussed in Section 7.2.Sections 7.2 and 7.3 indicate that for successful learning, constant tutorinvolvement is not required, however, the robot does require means of activeinteraction with the tutor or the environment to resolve ambiguities and torecover from errors. Since a natural way for the tutor to communicate withthe robot is through actions (e.g., moving a hand and pointing to the objects,pushing objects), the robot does not only have to detect motion patterns, buthas also be able to put them in the context of understanding human inten-tion. This issue was addressed in Section 7.4, where learning of human actionsand activities was studied in the context of tutor-driven learning. Intentionalactions were presented by modelling the causal relations between the handmotion, object states, and the effects of actions, which were presented usingfactorized hidden Markov models. The particular representations, the prob-abilistic action rules, allowed the robot to reason about motion patterns aspart of understanding about human intentions, as well as to relate functionalconcepts of an object to its properties. The developed action learning andrecognition system was used within the PlayMate scenario to represent andreason about object manipulation during a human-computer interaction. Theaction recognition approach was validated on several action sequences. It hasbeen shown, by reproducing a key result from the literature on action learn-ing in children, that the proposed representation has properties that at leaston some level of abstraction resemble action learning in humans. We believethat the developed probabilistic action rules are also suitable for representingboth the robot’s and the agent’s actions. An attractive property of having acommon representation would allow the robot to ground the action modelsto its own actions, and to bootstrap the models and acquire models that arespecific to its embodiment.308 Skočaj et al.A multi-modal learning in the context of embodied cognitive agent wasaddressed in Section 7.5. Ideally, an embodied, cognitive agent (e.g., an au-tonomous robot), would be capable of categorizing and detecting objects ac-cording to potential uses, and w.r.t. their utility in performing a certain task.In Section 7.5 we have therefore addressed learning of functional object cate-gories from the perspective of state-of-the-art object detection. The functionalcategory representations were acquired by observing prototypical interactionsof a human with an object. In contrast to Section 7.4, where the aim wasto translate the visual motion patterns into intentional actions, here the goalof observing an interaction was to characterize distinct visual features whichdetermine a functional category of an object. For example, a human wouldgrasp a mug by the handle, which would suggest that the bent shape of thehandle characterizes the mug’s functional category. Such distinct visual fea-tures are termed affordance cues, and the functional object category detectionis thus based on the recognition of these cues. The detection system was builton robust and well established techniques of object recognition, applicable toreal-world images of cluttered scenes. We have explored the capabilities ofa widely adopted detection framework, based on a suitable geometric localfeature representation. The functional object category detection was based onthe Implicit Shape Model, which was extended to allow for independent train-ing of several different affordance cues, and flexible combination for detectingcomposite functional categories. A result was an approach for tutor-driven ac-quisition, learning, and recognition of affordance cues in real-world, clutteredimages. The approach was successfully applied to the task of functional-objectdetection from a cluttered scene. Clearly, the currently explored methodologyis limited by the 2D nature of the used local features, however, it demonstratedanother view of cross-modal learning thorough tutor interaction.To summarize, in this chapter we have addressed several issues related tocross-modal learning, including the extent of tutor’s involvement, relationsbetween human actions and intentions, and the problem of online open-endedlearning. In our quest to study these problems we have proposed differentsolutions, whose strength we have substantiated with extensive experiments.However, several open questions still remain. One is the scalability of theapproaches—how to structure the complexity of the concepts? Is there a nat-ural hierarchy of concepts that can lead to an increasing number of layeredabstractions and sophistication? At which levels should different modalitiesinteract and form associations? How to apply the proposed learning strategiesto such representations? Can we improve the robustness of learning throughcombination of reconstructive and discriminative models of representation?An important research issue is also the question how can an artificial cogni-tive system perform introspection and detection of ignorance, which wouldsubsequently lead to planning and execution of new actions that would helpthe system to gather information to fill these knowledge gaps. However, suchcapabilities, which represent a necessary prerequisite for more efficient ex-ploratory learning, possibly with a minimal supervision of a human tutor, can7 Multi-modal Learning 309only be researched, explored, and tested within an integrated artificial cogni-tive system for which the CoSy project provided an essential initial theoreticaland experimental substrate.References1. S. Fidler, D. Skočaj, A. Leonardis, Combining reconstructive and discriminativesubspace methods for robust classification and regression by subsampling, IEEETransactions on Pattern Analysis and Machine Intelligence 28 (3) (2006) 337–350.URL http://cognitivesystems.org/CoSyBook/chap7.asp#fidlerPAMI062. S. Harnad, The symbol grounding problem, Physica D: Nonlinear Phenomena42 (1990) 335–346.3. E. Ardizzone, A. Chella, M. Frixione, S. Gaglio, Integrating subsymbolic andsymbolic processing in artificial vision, Journal of Intelligent Systems 1(4) (1992)273–308.4. A. Chella, M. Frixione, S. Gaglio, A cognitive architecture for artificial vision,Artificial Intelligence 89 (1–2) (1997) 73–111.5. D. K. Roy, A. P. Pentland, Learning words from sights and sounds: a computa-tional model, Cognitive Science 26 (1) (2002) 113–146.6. D. K. Roy, Learning visually-grounded words and syntax for a scene descriptiontask, Computer Speech and Language 16(3) (2002) 353–385.7. L. Steels, P. Vogt, Grounding adaptive language games in robotic agents, in:Proceedings of the Fourth European Conference on Artificial Life, ECAL’97,Complex Adaptive Systems, 1997, pp. 474–482.8. P. Vogt, The physical symbol grounding problem, Cognitive Systems Research3 (3) (2002) 429–457.9. C. Bauckhage, G. Fink, J. Fritsch, F. Kummert, F. Lömker, G. Sagerer,S. Wachsmuth, An integrated system for cooperative man-machine interaction,in: IEEE International Symposium on Computational Intelligence in Roboticsand Automation, 2001, pp. 328–333.10. S. Kirstein, Wersing, E. H., Körner, Rapid online learning of objects in a biolog-ically motivated recognition architecture, in: 27th DAGM, 2005, pp. 301–308.11. L. Steels, F. Kaplan, AIBO’s first words. the social learning of language andmeaning, Evolution of Communication 4 (1) (2001) 3–32.12. A. Arsenio, Developmental learning on a humanoid robot, in: IEEE InternationalJoint Conference On Neural Networks, 2004, pp. 3167–3172.13. D. E. Pollard, A user’s guide to measure theoretic probability, Cambridge Uni-versity Press, 2002.14. M. Kristan, D. Skočaj, A. Leonardis, Online kernel density estimation forinteractive learning, submitted for publication.URL http://cognitivesystems.org/CoSyBook/chap7.asp#KristanIMAVIS200815. M. P. Wand, M. C. Jones, Kernel Smoothing, Chapman & Hall/CRC, 1995.16. D. W. Scott, W. F. Szewczyk, From kernels to mixtures, Technometrics 43 (3)(2001) 323–335.17. J. Goldberger, S. Roweis, Hierarchical clustering of a mixture model, in: NeuralInf. Proc. Systems, 2005, pp. 505–512.310 Skočaj et al.18. K. Zhang, J. T. Kwok, Simplifying mixture models through function approxi-mation, in: Neural Inf. Proc. Systems, 2006.19. G. J. Mc Lachlan, T. Krishan, The EM algorithm and extensions, Wiley, 1997.20. M. A. F. Figueiredo, A. K. Jain, Unsupervised learning of finite mixture models,IEEE Trans. Patter. Anal. Mach. Intell. 24 (3) (2002) 381–396.21. Z. Živkovič, F. van der Heijden, Recursive unsupervised learning of finite mix-ture models, IEEE Trans. Patter. Anal. Mach. Intell. 26 (5) (2004) 651 – 656.22. A. Corduneanu, C. M. Bishop, Artificial Intelligence and Statistics, MorganKaufmann, Los Altos, CA, 2001, Ch. Variational Bayesian model selection formixture distributions, pp. 27–34.23. C. A. McGrory, D. M. Titterington, Variational approximations in Bayesianmodel selection for finite mixture distributions, Comput. Stat. Data Analysis51 (11) (2007) 5352–5367.24. M. Song, H. Wang, Highly efficient incremental estimation of gaussian mixturemodels for online data stream clustering, in: SPIE: Intelligent Computing: The-ory and Applications, 2005, pp. 174–183.25. O. Arandjelović, R. Cipolla, Incremental learning of temporally-coherent gaus-sian mixture models, in: British Machine Vision Conference, 2005, pp. 759–768.26. W. F. Szewczyk, Time-evolving adaptive mixtures, Tech. rep., National SecurityAgency (2005).27. A. Declercq, J. H. Piater, Online learning of gaussian mixture models - a two-level approach, in: Intl.l Conf. Comp. Vis., Imaging and Comp. Graph. Theoryand Applications, 2008, pp. 605–611.28. B. Han, D. Comaniciu, Y. Zhu, L. S. Davis, Sequential kernel density approx-imation and its application to real-time visual tracking, IEEE Trans. Patter.Anal. Mach. Intell. 30 (7) (2008) 1186–1197.29. M. Kristan, D. Skočaj, A. Leonardis, Incremental learning with Gaussian mix-ture models, in: Computer Vision Winter Workshop CVWW 2008, Moravsketoplice, Slovenia, 2008, pp. 25–32.URL http://cognitivesystems.org/CoSyBook/chap7.asp#kristanCVWW0830. M. Girolami, C. He, Probability density estimation from optimally condenseddata samples., IEEE Trans. Patter. Anal. Mach. Intell. 25 (10) (2003) 1253–1264.31. M. C. Jones, J. S. Marron, S. J. Sheather, A brief survey of bandwidth selectionfor density estimation, J. Amer. Stat. Assoc. 91 (433) (1996) 401–407.32. D. Skočaj, G. Berginc, B. Ridge, A. Štimec, M. Jogan, O. Vanek, A. Leonardis,M. Hutter, N. Hewes, A system for continuous learning of visual concepts, in:International Conference on Computer Vision Systems ICVS 2007, Bielefeld,Germany, 2007.URL http://cognitivesystems.org/CoSyBook/chap7.asp#skocajICVS0733. D. Skočaj, B. Ridge, G. Berginc, A. Leonardis, A framework for continuouslearning of simple visual concepts, in: Computer Vision Winter Workshop 2007,St. Lambrecht, Austria, 2007, pp. 99–105.URL http://cognitivesystems.org/CoSyBook/chap7.asp#skocajCVWW0734. D. Skočaj, M. Kristan, A. Leonardis, Continuous learning of simple visual con-cepts using incremental kernel density estimation, in: International Conferenceon Computer Vision Theory and Applications, Funchal, Madeira, Portugal,2008, pp. 598–604.URL http://cognitivesystems.org/CoSyBook/chap7.asp#skocajVISAPP087 Multi-modal Learning 31135. D. Lowe, Object recognition from local scale invariant features, in: ICCV’99,1999.36. K. Mikolajczyk, C. Schmid, A performance evaluation of local descriptors, in:CVPR’03, 2003.37. K. Mikolajczyk, B. Leibe, B. Schiele, Local features for object class recognition,in: ICCV’05, Beijing, China, 2005.38. G. Csurka, C. Dance, L. Fan, J. Willarnowski, C. Bray, Visual categorizationwith bags of keypoints, in: SLCV, 2004.39. B. Leibe, E. Seemann, B. Schiele, Pedestrian detection in crowded scenes, in:CVPR’05, San Diego, CA, USA, 2005.40. J. Sivic, B. C. Russell, A. A. Efros, A. Zisserman, W. T. Freeman, Discoveringobjects and their locations in images, in: ICCV’05, Beijing, China, 2005.41. S. Lazebnik, C. Schmid, J. Ponce, Beyond bags of features: Spatial pyramidmatching for recognizing natural scene categories, in: CVPR’06, 2006, pp. 2169–2178.42. A. Agarwal, B. Triggs, Hyperfeatures - multilevel local coding for visual recog-nition, in: ECCV’06, Springer, 2006.43. M. Fritz, B. Schiele, Towards unsupervised discovery of visual categories, in:DAGM’06, Berlin, Germany, 2006.44. K. Grauman, T. Darrell, Unsupervised learning of categories from sets of par-tially matching image features, in: CVPR’06, IEEE Computer Society, Wash-ington, DC, USA, 2006, pp. 19–25.45. J. Baldridge, G.-J. M. Kruijff, Multi-modal combinatory categorial grammar,in: EACL ’03, Morristown, NJ, USA, 2003.46. J. Baldridge, G.-J. M. Kruijff, Coupling ccg and hybrid logic dependency se-mantics, in: ACL ’02, Morristown, NJ, USA, 2001.47. D. Roy, Learning words and syntax for a scene description task, ComputerSpeech and Language 16 (3).48. G.-J. M. Kruijff, J. D. Kelleher, G. Berginc, A. Leonardis, Structural descrip-tions in Human-Assisted robot visual learning, in: Proceedings of 1st AnnualConference on Human-Robot Interaction, 2006.49. G.-J. M. Kruijff, J. D. Kelleher, N. Hawes, Information fusion for visual referenceresolution in dynamic situated dialogue, in: PIT 2006, Kloster Irsee, Germany,2006.50. J. Kelleher, G.-J. Kruijff, F. Costello, Proximity in context: an empiricallygrounded computational model of proximity for processing topological spatialexpression, in: Coling-ACL ’06, Sydney Australia, 2006.51. M. Brand, N. Oliver, A. Pentland, Coupled hidden markov models for com-plex action recognition, in: IEEE Proceedings of Computer Vision and PatternRecognition, Puerto Rico, USA, 1997.52. C. Wren, A. Pentland, Dynamic modeling of human motion, in: Proceedingsof the Third IEEE International Conference on Automatic Face and GestureRecognition, Nara, Japan, 1998.53. S. Hongeng, J. Wyatt, Learning causality and intention in human actions,in: Proceedings of IEEE-RAS International Conference on Humanoid Robots,Genoa, France, 2006.URL http://cognitivesystems.org/CoSyBook/chap7.asp#hong0654. R. S. Sutton, A. G. Barto, Reinforcement learning : An introduction, MIT Press,Cambridge, Massachusetts, 1998.312 Skočaj et al.55. P. Domingos, M.Richardson, Markov logic: A unifying framework for statisticalrelational learning, in: Proceedings of the ICML 2004 Workshop on StatisticalRelational Learning and its Connection to Other Fields, Banff, Canada, 2004.56. S. Hongeng, J. Wyatt, Learning Causality and Intentional Actions, LNAI: To-wards Affordance-Based Robot Control, Springer, 2008.URL http://cognitivesystems.org/CoSyBook/chap7.asp#hong08a57. S. Hongeng, J. Wyatt, Learning goal-based motion sequences of object ma-nipulation, Tech. Rep. CSR-08-02, School of Computer Science, University ofBirmingham (2008).58. C. Glymour, Learning causes : Psychological explanations of causal explanation,Minds and Machines 8 (1998) 39–60.59. G. Gergely, G. Csibra, Teleological reasoning in infancy: the naive theory ofrational action, Trends in Cognitive Sciences 7(7) (2003) 287–292.60. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, A. Zisserman, ThePASCAL Visual Object Classes Challenge 2007 (VOC2007) Results.61. G. Griffin, A. Holub, P. Perona, Caltech-256 object category dataset, Tech. Rep.7694, California Institute of Technology (2007).62. E. Rosch, C. B. Mervis, W. D. Gray, D. M. Johnson, P. B. Braem, Basic objectsin natural categories, Cognitive Psychology.63. J. J. Gibson, The theory of affordance, in: Percieving, Acting, and Knowing,Lawrence Erlbaum Associates, Hillsdale, NJ, 1977.64. P. H. Winston, B. Katz, T. O. Binford, M. R. Lowry, Learning physical descrip-tions from functional definitions, examples, and precedents, in: AAAI’83.65. L. Stark, K. Bowyer, Achieving generalized object recognition through reasoningabout association of function to structure, PAMI 13 (10) (1991) 1097–1104.66. L. Stark, A. Hoover, D. Goldgof, K. Bowyer, Function-based recognition fromincomplete knowledge of shape, in: WQV93, 1993, pp. 11–22.67. E. Rivlin, S. J. Dickinson, A. Rosenfeld, Recognition by functional parts, Com-puter Vision and Image Understanding: CVIU 62 (2) (1995) 164–176.68. L. Bogoni, R. Bajcsy, Interactive recognition and representation of functionality,CVIU 62 (2) (1995) 194–214.69. A. Saxena, J. Driemeyer, A. Y. Ng, Robotic grasping of novel objects usingvision, IJRR.70. M. Stark, P. Lies, M. Zillich, J. Wyatt, B. Schiele, Functional object class de-tection based on learned affordance cues, in: 6th International Conference onComputer Vision Systems (ICVS), 2008.URL http://cognitivesystems.org/CoSyBook/chap7.asp#stark08icvs71. J. Sun, W. W. Zhang, X. Tang, H. Y. Shum, Background cut, in: ECCV, 2006,pp. II: 628–641.72. J. Lafferty, A. McCallum, F. Pereira, Conditional random fields: Probabilisticmodels for segmenting and labeling sequence data, in: ICML’01.73. M. J. Jones, J. M. Rehg, Statistical color models with application to skin de-tection, in: CVPR, IEEE Computer Society, 1999, pp. 1274–1280.74. D. G. Lowe, Distinctive image features from scale-invariant keypoints, IJCV60 (2) (2004) 91–110.75. K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffal-itzky, T. Kadir, L. J. V. Gool, A comparison of affine region detectors, IJCV’05.76. V. Ferrari, L. Fevrier, F. Jurie, C. Schmid, Groups of adjacent contour segmentsfor object detection, Rapport De Recherche Inria.7 Multi-modal Learning 31377. V. Ferrari, T. Tuytelaars, L. J. V. Gool, Object detection by contour segmentnetworks, in: ECCV, 2006.78. M. Stark, B. Schiele, How good are local features for classes of geometric objects,in: ICCV, 2007.URL http://cognitivesystems.org/CoSyBook/chap7.asp#stark07iccv79. M. Zillich, Incremental Indexing for Parameter-Free Perceptual Grouping, in:31st Workshop of the Austrian Association for Pattern Recognition, 2007.80. B. Leibe, A. Leonardis, B. Schiele, An implicit shape model for combined objectcategorization and segmentation, in: Toward Category-Level Object Recogni-tion, Springer, 2006.URL http://cognitivesystems.org/CoSyBook/chap7.asp#Leibe06b8Situated Dialogue Processing forHuman-Robot InteractionGeert-Jan M. Kruijff, Pierre Lison, Trevor Benjamin, Henrik Jacobsson,Hendrik Zender, Ivana Kruijff-Korbayová1DFKI GmbH, Saarbrücken Germany gj@dfki.de8.1 IntroductionIn CoSy, our robots were to be able to interact with human. These interactionsserved to help the robot learn more about its environment, or to plan and carryout actions. For a robot to make sense of such dialogues, it needs to understandhow a dialogue can relate to, and refer to, “the world” – local visuo-spatialscenes, as in the Playmate scenario (9), or the spatial organization of an indoorenvironment in the Explorer scenario (10).This is not a trivial problem. Language presents a powerful system toexpress meaning. Also, perception provides a cognitive system with rich ex-periences of the world. The fundamental problem for (situated) dialogue pro-cessing is how to relate linguistically expressed meanings to these experiences.If we look at this problem from the viewpoint of dialogue as a means of com-munication, then we can pose first of all the following two requirements. Anysolution needs to be efficient, to allow a cognitive system to respond in atimely fashion, and effective, so that the system arrives at those meaningswhich are indeed likely to be correct in the given ”context.”But, as a communication channel, or rather an information channel, spo-ken dialogue poses further requirements than just efficiency and effectiveness.Spoken input is typically noisy, with utterances in a spoken dialogue oftenbeing incomplete, or grammatically incorrect. We may only be able to con-struct partial representations of a meaning for an utterance. Furthermore,if we consider meaning to be about something, i.e. about a ”referent” in adialogue- or situated context, then this partiality goes beyond the boundaryof a single utterance as meaning in a dialogue is typically contributed to sucha referent over several utterances. A solution to modeling meaning in situ-ated dialogue thus needs to allow for a gradual resolution and construction ofmeaning representations, (with the possibility to ”postpone” the resolutionof ambiguities), and provide for meaning to persist over utterances, i.e. overthe course of a dialogue. Finally, these requirements are based on a view ofmeaning being referential to a context, i.e. a dialogue context and a situated316 Kruijff et al.context. Meaning representations will need to indicate, and represent, thisreferentiality : how they purport to refer, and to what.There exist several approaches to deal with each one, or several, of theseissues. Roy & Reiter present a comprehensive overview of existing approachesto relating language and the world (up to 2005) in [1], and identify severalimportant issues very similar to the requirements we raised above. One of theearliest systems which connected built utterance analyses to a visual worldwas Winograd’s SHRDLU [2]. SHRDLU built linguistic analyses in an incre-mental, “left-to-right” fashion, and connected those to visuo-spatial represen-tations of local scenes to help prune analyses. Among more recent approaches,the most developed are those by Gorniak & Roy, and Steels et al. Gorniak &Roy [3, 4] present an approach in which utterance meaning is probabilisticallymapped to visual and spatial aspects of objects in the current scene. Recently,they have extended their approach to include action-affordances [5]. Their fo-cus has primarily been on the aspect of how to relate language and informationabout the world, just like SHRDLU and the work by Steels et al [6, 7, 8].The latter have developed an approach where the connection between wordmeaning and percepts is modeled as a semiotic network, in which abstractcategories mediate between language and the visual world. Although manyof these approaches use some form of incremental processing to graduallyconstruct utterance meaning, connecting meanings to the social and physi-cal context as they are construed, the (im)possibility to connect alternativemeanings does not feed back into the incremental process to prune inviableanalyses. This is where Scheutz et al [9, 10] take matters a step further. Theypresent an approach for incremental utterance processing in which the analy-ses are referentially interpreted, and pruned if it is impossible to connect themin the situated contexts referred to.The approach we describe in this chapter primarily advances on the aboveby addressing the requirements we raised earlier in a systematically integratedfashion. We postulate bi-directionality hypothesis, a close coupling betweenhow a cognitive system processes situated dialogue, and how it processes andrepresents experiences associated with a situated context to which dialoguerefers. The purpose of this coupling is for these different modalities to inter-change and interconnect information, to help guide processing. (Such a closecoupling is inspired by how humans appear to process visually situated dia-logue, see e.g. [11, 12, 13, 14].) More specifically, based on this hypothesis weaddress the requirements as follows.Gradual construction Utterance meanings are constructed incrementally,using Combinatory Categorial Grammar [15, 16]. Meaning is representedas an ontologically richly sorted, relational structure (using a decidablefragment of modal logic, [17, 18, 19, 20]). The ontological sorting is usedas a mediating basis for binding linguistic content to content in othermodalities, including visuo-spatial scenes, spatial maps, and actions ([21,22, 23] and Chapters 2 and 7). In comparison to approaches as mentioned8 Situated Dialogue Processing for HRI 317above we thus extend category-based mediation (the ”what” dimension)with a spatio-temporal dimension of interpretation (the ”where/when”and ”how”) covering several levels of spatial organization. In our approachwe are not restricted only to visuo-spatial information about the currentlyperceivable scene.Referentiality The propositional meaning of an utterance is complementedwith information on how that meaning is presented as referring to alarger context. Such referentiality is established directly, based on forms ofanaphoric or deictic references, or more indirectly using a basic form of in-formation structure (see Section 8.4). Structured discourse representationmodels [24] are used to model dialogue context, and represent referential-ity. These models are complemented with a temporal representation ofhow referred-to events can be related ([25, 23], Chapter 6). Dedicated al-gorithms are used to resolve linguistic references to situated contexts suchas visuo-spatial scenes (collection of visuo-spatial information on Bindingworking memory, [26]) and large-scale spatial organization [27] (subset ofrelevant locations in a map, see Section 8.5).Persistence The dialogue context model provides a persistent basis for main-taining how linguistic content has been resolved to content in a situatedcontext. Structures relating different linguistic references to a specific ref-erent can be combined to determine how information about that referenthas been communicated over the course of a dialogue (see e.g. [28], andChapter 7 for use in visual learning).Efficiency & Effectiveness Based on information about the current situ-ated context (whether local, or at a larger scale, cf. also Section 8.5), andthe current dialogue context, incremental comprehension of spoken utter-ances can prune unlikely word- and meaning-hypotheses. (In comparison,Scheutz et al only guide parsing.) The result is a combined performanceof speech recognition and parsing of close to 90% correct interpretationsof free speech on our domains (against a base-line performance of 68%).Bi-directionality has a fundamental impact on how we conceive of lin-guistic meaning and its processing. Linguistic meaning arises as a reflectionof how a cognitive system experiences, and structures its knowledge of, theworld [29, 30]. Processing meaning is based on a continuous interchange ofinformation between functions involved in computing aspects of meaning, bethose functions related to experience or dialogue. The resulting view on mod-ularity implies dialogue processing to be more like a “permeable glass box”than the traditional “black box.” Interconnectivity is based on a model akinto pass-by-reference, to allow different functions to keep track of how contentchanges over time. Adopting this hypothesis, we show in this chapter howintegration with information about situated contexts helps us to achieve bet-ter robust processing of spoken, situated dialogue processing on the Playmateand Explorer domains than if no situated information would have been used.318 Kruijff et al.An overview of the chapter is as follows. In Section 8.2 we briefly discussobservations that have been made on how humans appear to process situ-ated dialogue. These observations have in part inspired the functionality weconsider in our approach. Then, Section 8.3 we first discuss the basics for con-textualized language processing, showing how bi-directionality influences firstof all the design of the processes and representations we adopt. We continue inSection 8.4 with looking into dialogues about what the robot can see, connect-ing dialogue meaning with an understanding of local visuo-spatial scenes. Wediscuss how we can talk about those scenes, for example when a human triesto teach the robot more about the objects it sees, and how bi-directionalityhelps focusing speech recognition, utterance analysis, reference resolution, andproducing references to objects. The Explorer scenario provides us with a set-ting in which we go beyond the current scene, often discussing places we canvisit, or where we can find objects in the world. Important is that we do notneed to be in those places to discuss them. In Section 8.5 we present howwe can go beyond the current situated context, and use information aboutthe larger world around the robot to talk about other places. One interestingchallenge bi-directionality helps addressing is in resolving and producing ref-erences to such places. Finally, in Section 8.6 we take meaning beyond havingmostly a referential, indexical nature. We look into how particularly speechacts like questions and commands express intentionality , and how we canintentionalityrelate that to processes for motivation and planning. Bi-directionality entersthe dialogue processing picture again by indicating which potential utteranceinterpretations correspond to possible plans, and which ones do not.8.2 BackgroundLanguage provides us with virtually unlimited ways in which we can com-municate meaning. This, of course, raises the question of how precisely wecan then understand an utterance as we hear it. Empirical studies in vari-ous branches of psycholinguistics and cognitive neuroscience have investigatedwhat information listeners use when trying to understand spoken utteranceswhich are about visual scenes. An important observation across these stud-ies is that interpretation in context plays a crucial role in the comprehensionof utterance as it unfolds [11]. Following [13] we can identify two importantdimensions of the interaction between the purely linguistic, dialogue context,and the situated context. One is the temporal dimension. The ways our vi-sual attention are guided appear to be timed closely with how we proceedwith understanding an utterance. In empirical studies we can witness this byfor example eye movements. The second is the information dimension. Thisindicates that listeners not only use linguistic information during utterancecomprehension, but also scene understanding and ”world knowledge.” Belowwe discuss aspects of these dimensions in more detail.8 Situated Dialogue Processing for HRI 3198.2.1 Multi-level integration in language processingUntil the early 1990s, an often-adopted model of language comprehension wasthat of a modular, stage-like process. On this model, a language user wouldsequentially construct each level of linguistic comprehension – from auditoryrecognition all the way to pragmatic, discourse-level interpretation. As [31] ob-serve, two hypotheses followed from this view. One hypothesis is that peoplefirst construct a local, context-independent representation of the communi-cated meaning. Only once this meaning has been completely constructed, itis interpreted against the preceding dialogue context (if any). Secondly, andrelated, is the hypothesis that dialogue context-related processing only entersthe process of language comprehension at a relatively late stage.Opposing these hypotheses is the view that language comprehension isan incremental process. In such a process, each level of linguistic analysis isperformed in parallel. Every new word is related to representations of the pre-ceding input, across several levels – with the possibility for using the interpre-tation of a word at one level to co-constrain its interpretation at other levels.(This may result in ambiguities to be resolved only at a later point.) A naturalprediction that follows from this view is that interpretation against dialoguecontext can in principle affect utterance comprehension as the utterance isincrementally analyzed, assisting in restricting the potential for grammaticalforms of ambiguity. [32, 11] phrased this as a principle of parsimony : thosegrammatical analyses are selected that for their reference resolution imposethe least presuppositional requirements on a dialogue context.Since then, various studies have investigated further possible effects of di-alogue context during utterance comprehension. Methodologically, psycholin-guistic studies have primarily investigated the effects of dialogue context bymeasuring saccadic eye movements in a visual scene, based on the hypothesisthat eye movements can be used as indications of underlying cognitive pro-cesses [33, 34]. Alternatively, cognitive neuroscience-based studies use event-related brain potentials (ERPs) to measure the nature and time course of theeffects of dialogue context on human sentence comprehension [35].Both lines of study have found that lexical, semantic and discourse-level in-tegrative effects occur in a closely time-locked fashion, starting already at thephoneme or sub-word level; see [36], and [37, 31, 38]. Particularly, a range ofdialogue-level integrative effects have observed. Referential binding has beenshown to play a role in the constraining various types of local syntactic ambi-guities, like garden path-constructions [32, 11, 39], and relative clauses [40, 41];[42, 37, 31]. These effects primarily concern a disambiguation of already builtstructures. Integrating semantic and dialogue-level information during utter-ance comprehension also has important anticipatory effects. [43, 44]; [45] ob-serve how contextual information influences what lexical meanings can be an-ticipated, priming phonological understanding and lexical access. Contextualinformation can even override disprefered lexical meaning [46].320 Kruijff et al.Anticipatory effects indicate that utterance comprehension is thus not onlyan incremental process of constructing and then disambiguating. Anticipa-tion enables context-dependent phonological recognition, lexical retrieval, andsyntactic construction - without there being a need to generate and test allcombinatory possible constructions. Incrementality and anticipation based onmulti-level integration appears to give rise to a process in which comprehen-sion arises through a convergence based on constraining and co-activation.Dialogue context and the interpretative contexts which are delineated duringutterance comprehension converge to become functionally identical [31]. As aresult, ambiguity need not even arise, or is at least being much more limiteda priori through context.An important issue in all of the above remains of course the degree towhich integrative effects indeed should commit to a certain understanding.Garden path sentences are a good example. They show that overcommitmentrisks the need for re-interpretation – an issue for cognitive control [47, 48, 49].8.2.2 Language processing and situational experienceWe already noted before that humans integrate linguistic and non-linguisticinformation when processing an utterance. Below we discuss studies whichinvestigate how categorical and contextual information from situation aware-ness can effect utterance comprehension. These studies use eye-trackers tomonitor where people look at in a scene, and when.[50] present a study revealing that listeners focus their attention on objectsbefore these objects are referred to in the utterance. For example, considera scene with a cat, a mouse, and a piece of cheese. When someone hears”The cat chases the mouse”, her gaze already moves to the mouse in thescene before she has actually heard that word; similarly for ”The mouse eatsthe cheese.” Knowing that cats typically chase mice (not cheese), and thatthe argument structure of chase reflects this, the listener expects that thenext object to be mentioned will be the mouse, and directs gaze to thatobject. We thus see an anticipatory effect arising from the online integration oflexico-semantic information (verbal argument structure), situational context(the present objects, and the reported action), and categorical knowledge(prototypical object-action relations).Not only world knowledge can influence online utterance comprehension,also scene understanding can. For example, consider the situation in Fig-ure 8.1. [33] show that, once the listener has heard ”Put the apple on thetowel ...” she faces the ambiguity of whether to put the (lone) apple onto the(empty) towel, or to take the apple that is on the towel and put it somewhereelse. The ambiguity is revealed as visual search in the scene. Only once shehas heard the continuation ”... into the box” this ambiguity can be resolved.Interestingly, in [33] the listener cannot directly manipulate the objects. If thisis possible (cf. Figure 8.1), [51] show that also reachability plays a role in com-prehending the utterance. Because only one apple is reachable, this is taken8 Situated Dialogue Processing for HRI 321Fig. 8.1. Put, apple, towel, boxas the preferred referent, and as such receives the attention. This underlinesthe effect physical embodiment may have on language comprehension.Scene understanding also concerns the temporal projection towards pos-sible future events [52]. [12, 53] show how such projection can also affectutterance comprehension. These studies used a scene with a table, and besideit a glass and a bottle of wine. Investigated was where listeners look whenthey hear ”The woman will put the glass on the table. Then, she will pick upthe wine, and pour it carefully into the glass.” It turns out that after hearingthe ”pouring” phrase, listeners look at the table, not the glass. Listeners thusexplicitly project the result of the picking action into the scene, imagining thescene in which the glass is on the table.These studies reveal that the interaction between vision and language isnot direct, but mediated [12]. Categorical understanding plays an importantrole in the sensorimotoric grounding of language. This is further underlined bystudies like [54, 55], following up on the idea of category systems as mediat-ing between perceptual modalities and language [56, 57]. These studies showhow categorical understanding gives rise to expectations based on affordances,influencing comprehension of spatial or temporal aspects of action verbs.In conversational dialogue [14, 58] gaze has been shown to be automaticallyaligned in simple collaborative interaction. The time intervals between eye-fixations during production and comprehension of a referring expression areshorter than in monologue. This is further evidence for the relevance of visualcommon ground of interlocutors and how that accelerates the activation ofjointly relevant concepts.8.3 TalkingWhat does it take to make a robot talk? Specifically, what does it take tomake a robot process situated dialogue?In the context of CoSy, for a robot to talk it first of all needs to be able toprocess a speech signal, turning it into (possible) sequences of words. Then,turn those words into utterances, and assign a meaning representation tothem. Depending on the referential status of content in these meaning repre-sentations, links are made to a model of the preceding dialogue, so that it can322 Kruijff et al.be established how meaning refers to things that were already said before (ifat all), and how it moves the dialogue along.That’s one part – listening, comprehending utterances against a modelof the dialogue context, and updating that context model as the dialoguecontinues. The other part is the talking part. Based on how the dialogue hasdeveloped so far, the robot should decide how to continue. Then, followingup on this decision, it should see how to formulate the utterances to achievethat ”goal”, and formulate them such that they refer to the situations in acontextually appropriate way. It should be clear to the listener what the robotis referring to, talking about. And once the robot has said what it decided tosay, it should of course again update the model of the dialogue context.In this section we would like to focus on the comprehension side, andsketch how the production side is structured. We would like to start simplehere, explaining the basics behind the approach we take to making robotstalk. Explain the design decisions as they result from the bi-directionalityhypothesis, why we do the things the way we propose to do them. Comethe next sections, we will delve into more detail, or where necessary providereferences to more technical discussions.Adopting the bi-directionality hypothesis poses requirements both on howwe design our processes, and our representations.We consider the processes involved in comprehending situated dialogue tobe ”permeable glass boxes.” While processing, it should be possible to takepartial results, connect them with information from other processes, and thenuse the results to guide how to continue processing. One way to design suchprocesses so is to make them incremental. In incremental processing , a pro-incremental processingcess proceeds from the ”beginning” towards the ”end” of a representation itis to process, in a step-wise fashion. After each step, bi-directionality can beused to guide how to take the next step. Because linguistic representationsare typically sequential, we can process them incrementally. Much of the ben-efits of bi-directionality consist therein that they can help processes focus onsensible analyses, discarding those which are not supported by the context.Each process typically maintains several concurrent hypotheses. Particu-larly if we look at things from an efficiency point of view, there are severalrequirements bi-directionality raises for the nature of representations. Firstof all, we are looking at dialogue, a context in which interpretations developover time. Utterances refer to the preceding context, adding to or correctingprevious information. Connecting information across processes can be moreefficient if these relations are clear. It identifies a history of what previouslywas already interconnected. The way we will address this requirement is byusing discourse referents as permanent hooks to relate information to, bothdiscourse referentsat utterance- and at dialogue-level.Secondly, even though a process may maintain multiple hypotheses, thisdoes not imply that they need not share certain similarities in how they in-terpret something. Representations should identify how alternative analysesare different, and where there are similarities, so that we can avoid having to8 Situated Dialogue Processing for HRI 323check each hypothesis individually. We will address this requirement by pack-ing multiple hypotheses into a single, possibly underspecified graph structure, packingand determining preference orders over alternatives. preference ordersBelow we will explain these processes and representations in more detail.We start with representations, to make it clear what we are working with,and towards.Representing an utteranceLoosely speaking, a dialogue is an exchange of utterances between two or dialoguemore ”interlocutors.” Usually, this exchange serves a particular purpose. In thecontext of human-robot interaction, that purpose usually relates to performingtasks in the real-world.The thing is, whereas sentences in a text are usually complete, and gram-matically well-formed, this need not be the case with utterances in spokendialogue. Utterances are often incomplete or grammatically incorrect, andmay include self-corrections. ”Take the red uh ... no put that green one nextto the ... you know, yes, the pyramid.” This of course raises the question,what we should consider an utterance to be. utteranceMost dialogue systems (still) consider an utterance to be like a sentence,and have a definable beginning and end. We adopt a more flexible notion thanthat. What we ultimately consider to be an utterance, depends on the contextin which linguistically conveyed content is being used. As far as processingwithin our system is concerned, an utterance is a stream. There are markedpoints at which it can be further interpreted, either within the dialogue systemor beyond it. At such ”points,” the representation of the utterance providesenough meaning to start off further forms of processing. Each further inter-pretation modality is thus free in considering when it works with meaning,and thus –ultimately– what it considers an ”utterance” to be.Which brings us to how we represent meaning. We represent meaning asan ontologically richly sorted, relational structure – a logical form [18, 20] in logical forma decidable fragment of modal logic [17, 19]. The following is an example of alogical form:324 Kruijff et al.@w1 :cognition(want ∧ 〈Mood〉 ind ∧ 〈Tense〉 pres ∧〈Actor〉 (i1 : person ∧ I ∧ 〈Num〉 sg) ∧〈Event〉 (p1 : action−motion ∧ put ∧〈Actor〉 y1 : person ∧〈Patient〉 (m1 : thing ∧ mug ∧〈Delimitation〉 unique ∧ 〈Num〉 sg ∧ 〈Quantification〉 specific ∧〈Modifier〉 (r1 : q− color ∧ red)) ∧〈Result〉 (t1 : m− whereto ∧ to ∧〈Anchor〉 (r2 : e− region ∧ right ∧〈Delimitation〉 unique ∧〈Num〉 sg ∧〈Quantification〉 specific ∧〈Owner〉 (b1 : thing ∧ ball ∧〈Delimitation〉 unique ∧ 〈Num〉 sg ∧ 〈Quantification〉 specific)))) ∧〈Patient〉 (y1 : person ∧ you ∧ 〈Num〉 sg) ∧〈Subject〉 i1 : person)Each node has a unique identifier with an associated ontological sort (e.g.p1 : action −motion means that the handle p1 is of sort action −motion),and a proposition (e.g. put for p1). Nodes are connected through namedrelations. These indicate how the content of a single node contributes to themeaning of the whole expression. For example, ”you” (y1) both indicates theone whom something is wanted of (Patient-relation from w1), and the one whois to perform the put action (Actor-relation from p1). Nodes carry additionalfeatures, e.g. i1 identifies a singular person.Propositions and relations in such a representation are instances of con-cepts. This makes it possible for us to interpret logical forms further usingontological reasoning. We use this possibility in reference resolution, and inrelating meaning representations to interpretations formed outside the dia-logue system.The relational nature of our representations provides us with several conse-quences. We build up our representations from elementary propositions as weillustrated above – sorted identifiers and propositions, features, and relations.An interpretation is thus simply a conjunction of such elementary proposi-tions, and the more we can connect those elementary propositions, the morecomplete our interpretation becomes. This makes it relatively straightforwardto represent partial interpretations. For example, for ”take the red ...” receivesthe following interpretation:8 Situated Dialogue Processing for HRI 325@t1 :action−motion(take ∧ 〈Mood〉 imp ∧ 〈Tense〉 pres ∧〈Actor〉(a1 : entity ∧ addressee) ∧〈Patient〉 (m1 : thing ∧〈Delimitation〉 unique ∧ 〈Num〉 sg ∧ 〈Quantification〉 specific ∧〈Modifier〉 (r1 : q− color ∧ red))〈Subject〉 a1 : entity)The interpretation shows more than just the content for the three words.It also shows that ”red” is expected to be the color of the ”thing” which issupposed to be taken.Characteristic for language is that it presents many ways in which we cansay things – and interpret them. This inevitably means that we will usuallyget not just one, but multiple alternative interpretations for an utterance.To keep ambiguity to a minimum, we should look at to what extent theseinterpretations are indeed different. Where they show overlaps, we shouldideally have to deal with those identical parts only once.Using relational structure and elementary propositions enables us to doso. We represent alternative interpretations as alternative ways in which wecan connect content, whereas identical content across interpretations is rep-resented once. The procedure to create such ”condensed” representations iscalled packing , after [59, 60]. Figure 8.2 illustrates the development of the packingpacked packed representation for ”here is the ball”. At the first step (”take”),9 logical forms are packed together, with two alternative roots, and severalpossible ontological sorts for the word “here”. The second step reduces thenumber of alternative interpretations to one single logical form, rooted on theverb “be” with a “presentational” ontological sort. The possible meanings forthe determiner is expressed at the dependent node of the “Presented” relation.At this point we have an overspecified meaning. Although the delimination isunique, we cannot tell at this point whether we are dealing with a singularobject, or a non-singular (i.e. plural) object – all we know it has to be one orthe other. This becomes determined in the fourth step (”here is the ball”).In the appendix to this chapter we present a detailed technical discussionof packing.Representing the interpretation of an utterance in contextThe meaning of an utterance goes well beyond what is expressed just bythe individual words that make it up. Meaning is about how the utterancerelates to the context – the situation (indexically) and to the actions (to be)performed therein (intentionally). How it can be taken to refer to things wealready talked about, to beliefs we have, to expectations which may be raisedon the basis of what we are saying. How the utterance helps us to further thedialogue, helping to reach a goal – or not.Referent resolution is the first step we take to relate content from the cur-rent utterance, to that of previous utterances in the dialogue context. The326 Kruijff et al.Fig.8.2.Exampleofincrementalparsingandpackingoflogicalforms,fortheutterance“Hereistheball”.Fourstepsareshown.8 Situated Dialogue Processing for HRI 327purpose here is to establish co-reference relations: relations between mentionsreferring to the same object(s) or event(s). Examples of references to previ-ous objects are pronouns (e.g. ”it”), or anaphoric expressions (e.g. ”the redmug”). We are using a (simple) algorithm based on referent resolution in thesegmented dialogue representation theory of [24].For each index in a logical form, the algorithm determines potential an-tecedents in the preceding dialogue, using the model of the dialogue contextthe system maintains. There are two simple cases. One, we may be talkingabout a something new. We then create a new (unique) referent identifier, sayantn, and represent this as a reference structure [NEW : {antn}]. Two, thereis a unique antecedent referent anti. We represent this as [OLD : {anti}],meaning there is a ”discourse old” antecedent anti. In both cases we relatethe index in the logical form (which only has naming uniqueness within thescope of the logical form) to the built structure.Complications arise if a reference cannot be ambiguously resolved. A goodexample of such a situation arises when resolving deictic pronouns like ”this”.How a deictic pronoun needs to be resolved, depends on the dialogue- andthe situated context. If the utterance is not accompanied by a gesture, thepreference is to resolve the reference to a preceding antecedent in the dialogue.However, if the utterance is accompanied by a gesture, then this preferencemay be overridden. It may be that the gesture refers to an object which wasmentioned before, just not most recently; or it may refer to an object whichhas not been talked about at all. To capture these possibilities, we allowfor reference structures to specify preference orders over sets of old and newreferents. For example, if a deictic pronoun can be resolved to several oldantecedents, with anti the most preferred, or to a new referent antn, then weget[OLD : anti < {antj , ..., antk} < NEW : {antn}].Subsequently, information about grounding the utterance in the situatedcontext then can help resolving this ambiguity (e.g. by providing support for anew referent). The example of deictic pronoun nicely illustrates the principlebi-directional nature of situated dialogue processing as implemented here.There is no strict pipeline of interpretation processes, invoked at incrementalsteps. Instead, interpretation processes interact to mutually constrain andcomplement the interpretations they form.Another aspect of dialogue-level interpretation regards ”speech acts”, ordialogue moves. A dialogue move specifies how an utterance ”functions in”, i.e.contributes to furthering the dialogue. We determine an utterance’s possibledialogue move(s) on the basis of the shape of the logical form, and expectationsabout possible moves to extend the current dialogue. Figure 8.3 illustrates adecision tree used to map logical form features to dialogue moves.Once the dialogue move for an utterance has been determined, the utter-ance content, and its referent- and event structures are added to the dialogue328 Kruijff et al.Fig. 8.3. Example of a decision tree for determining dialogue moves from LF formcontext model maintained by the system. Figure 8.4 shows a snapshot of sucha model. (We will elaborate on event structures in Section 8.6.)Comprehending an utterance in contextWhen we try to comprehend an utterance, we analyze it at several linguisticlevels. As we are dealing with spoken dialogue, the first step is the automaticspeech recognition [ASR], which takes an audio signal stream as input andautomatic speechrecognition produces a word recognition lattice as output. This step is known to be partic-ularly error-prone [61], for several reasons. The first one is the inherent noisepresent in the real-world environments in which our robots are deployed. Sincewe require the speech recognition system to be speaker-independent, we alsohave to deal with the wide variety of voices, accents and styles of speech ofhuman speakers. And finally, natural spoken dialogue is also characterised bya high proportion of disfluencies (filled pauses, speech repairs, corrections,repetitions), and the production of many partial or ill-formed utterances, allof which negatively affect the performance of the speech recognition.Our strategy for addressing this issue is to exploit contextual knowledgeabout the situated environment and the dialogue history to prime the utter-ance recognition. This knowledge is represented in the cognitive architectureas a cross-modal salience model of the situated context. It integrates bothsalience modelvisual salience (objects perceived in the physical scene) and linguistic salience(previously referred-to objects within the current dialogue). The model isdynamically updated as the environment evolves, and is used to establish ex-pectations about uttered words which are most likely to be heard given thecontext. The update is realised by continuously adapting the word probabil-ities specified in the statistical language model of the speech recognizer. Wehave shown that this approach yields a statistically significant improvementof the ASR performance compared to a baseline, non context-sensitive model[62].As soon as the speech recognizer is able to suggest a (partial) recognitionhypothesis for the utterance, a word recognition lattice is created and insertedinto the working memory for subsequent analysis. A word recognition lat-tice is a packed representation for the set of potential recognition hypothesis,8 Situated Dialogue Processing for HRI 329Fig.8.4.Systemvisualizationofdialoguecontextmodel:Utterancecontent,coreference,eventstructures,anddialoguemoves.330 Kruijff et al.combined with their respective confidence scores. The set of recognition hy-potheses can be easily retrieved by traversing the lattice. Figure 8.5 illustratesa typical example of word lattice.Fig. 8.5. A typical word recognition latticeThis word recognition lattice is then further processed incrementally – thelowest, incremental level being that of grammatical analysis. For modeling nat-ural language grammar we use the Combinatory Categorial Grammar (CCG)framework [15, 16]. CCG is a lexicalized framework: For each word, there areone or more lexical entries specifying a syntactic category, and a correspond-ing lexical meaning. A syntactic category defines how the word can be usedin forming a larger, grammatical expression. The lexical meaning specifies theword meaning. The meaning of an expression is built up compositionally, inparallel to its syntactical derivation.Fig. 8.6. Incremental analysis of ”take the mug”Figure 8.6 illustrates how meaning is built up in parallel to a grammaticalderivation. The verb ”take” has a syntactic category s : e/np : p. This meansthat it will yield a sentence s if it is combined to the right / with a noun8 Situated Dialogue Processing for HRI 331phrase np. The indices e and p relate the syntactic material to the meaningbeing built: e provides a handle to the index for the verbal meaning, whereasp indicates that the noun phrase will provide the meaning for the Patient[20, 18].The words ”take” and ”the” can be combined incrementally into an expres-sion ”take the”, using function composition (the B rule in CCG, cf. [15]). Theresulting syntactic category specifies that this expression requires a noun n toits right to yield a complete sentence. The meaning of the determiner ”the”circumscribes that of the noun. Finally, ”take the” and ”mug” are combinedinto a complete expression, ”take the mug”.We have factorized (incremental) grammatical analysis into several, inter-connected functions: the incremental parsing process itself, packing/unpackingand pruning of incrementally construed analyses of utterance meaning, andcontext-sensitive lexical retrieval. Figure 8.7 illustrates the interactions be-tween these different functions.Fig. 8.7. Context-sensitive utterance interpretation at grammatical level: interac-tive processes for parsing and lexical retrieval, which can be primed by contextualinformation.Parsing begins by retrieving the lexical entries for the first word, andinitializing the chart. A chart is a data structure in which all active and com-pleted analysis are stored, marking for each analysis what part of the utterance(from beginning to some position x) it covers. Maintaining partial analysesmakes it possible to re-use them at a later point, when constructing analysesthat span more of the utterance. (This principle of re-using partial analyses332 Kruijff et al.sets chart-based parsing apart from e.g. backtracking, in which analyses areconstrued every time anew.) The chart is subsequently updated with the lex-ical entries for the first word, and a parsing process starts. Parsing is basedon a bottom-up Early chart parser built for incrementally parsing Combi-natory Categorial Grammar. Its implementation relies on basic functionalityprovided by OpenCCG1.Incremental chart parsing creates partial, and integrated analyses for astring in a left-to-right fashion. After each increase in position, the parserchecks whether it has reached a frontier. A frontier is specified as a type ofcomplete grammatical structure at the right branch of a grammatical deriva-tion. This enables us to specify whether the parser should return after everyword, or e.g. after every phrase. At each frontier check, the chart is pruned us-ing a category scorer. This scorer ranks the categories for the partial analysesconstrued so far, possibly pruning them if they are guaranteed not to lead toa complete analysis. (For example, in an incremental analysis, any categoryrequiring an argument to the left \ preceding the beginning of the utterancewill never be completed.)Once incremental parsing stops, a packed logical form is construed, andprovided to working memory. This packed representation of possible gram-matical interpretations of an utterance provides the basis for further interpre-tation steps – for example, referent resolution. Depending on the exact fashionin which these processes are synchronized, the next phase of incremental pars-ing is triggered by the becoming available of further information on workingmemory (e.g. referents). In this case, the chart is retrieved, and updated withthe lexical entries for the current word, and incremental parsing continues asdescribed above.The advantage of factorizing grammatical analysis into separate inference-and lexical retrieval processes is that the system can use information aboutthe situated- and task-context to prime attention in both processes, possiblyasynchronously (i.e. ”opportunistically”). Activated categories for objects andevents can help to restrict what lexical meanings are retrieved (”activated”) fora word. Furthermore, based on what (partial) interpretations can be groundedin the context, unsupported interpretations (analyses) can be removed fromthe chart.Picking up the right interpretationEven with the help of these contextual priming/pruning techniques, the out-come of the utterance comprehension process will nevertheless remain in manycases severely ambiguous and underspecified. This is not surprising: ambiguityis known to be extremely pervasive in natural language, at all processing lev-els (lexical, syntactic, semantic, pragmatic), and contextual priming/pruningalone cannot be expected to resolve all ambiguities. This means that most1 http://openccg.sf.net8 Situated Dialogue Processing for HRI 333utterance will still yield tens, if not hundreds, of possible analyses. With-out mechanisms for interpretations selection/filtering at our disposal, theseambiguities are inevitably going to hinder any further interpretation.We therefore implemented a robust parse selection system able to deter- parse selectionmine the most probable analysis among a set of alternative interpretations.The parse selection is based on a statistical linear model which explores a setof relevant acoustic, syntactic, semantic and contextual features of the parses,and is applied to compute a likelihood score for each of them.Our approach can therefore be seen as a discriminative approach to ut-terance interpretation: we first generate the possible analyses, and then dis-criminate amongs them according to various features.The parameters of this linear model are estimated against an automaticallygenerated corpus of 〈utterance, logical form〉 pairs. The learning algorithm isan averaged perceptron, a simple and efficient technique for parameter estima-tion which is known to give very good results for this task [63].The parse selection can be formalised as a function F : X → Y where thedomain X is the set of possible input utterances2, and the range Y is the setof parses. We assume:1. A function GEN(x) which enumerates all possible parses for an input x.In our case, this function simply represents the set of parses of x whichare admissible according to the CCG grammar.2. A d -dimensional feature vector f(x, y) ∈

Edinburgh Research Explorer

Abstract

Extracted data

Edinburgh Research Explorer

Abstract

Extracted data

Related items

Related items