Agents with limited modeling abilities: Implications on collaborative problem solving

Abstract. Collaboration plays a critical role when a group is striving for goals which are difficult or impossible to achieve by an individual. Knowledge about collaborators ’ contributions to a task is important when solving problems as a team. However, a problem in many collaboration scenarios is the uncertainty and incompleteness of such knowledge. To investigate this problem, we present a collaboration framework where team members use models of collaborators ’ performance to estimate contributions to a task, and propose agents for tasks based on these estimations. We conducted a simulation-based study to assess the impact of modeling limitations on task performance. The main results of our simulation are that maintaining models of agents improves task performance, but exhaustive model maintenance is not essential. Additionally, we found that the ability of agents to update their models has a large impact on task performance. We then extended our framework to support more refined agent models, and performed additional simulated studies. Our results indicated that task performance is improved by the availability of additional reasoning resources and the use of probabilistic models that represent variable agent performance.