Learning situation-specific control in multi-agent systems

The work presented in this thesis deals with techniques to improve problem solving control skills of cooperative agents through machine learning. In a multi-agent system, the local problem solving control of an agent can interact in complex and intricate ways with the problem solving control of other agents. In such systems, an agent cannot make effective control decisions based purely on its local problem solving state. Effective cooperation requires that the global problem-solving state influence the local control decisions made by an agent. We call such an influence cooperative control. An agent with a purely local view of the problem solving situation cannot learn effective cooperative control decisions that may have global implications, due to the uncertainty about the overall state of the system. This gives rise to the need for learning more globally situated control knowledge. An agent needs to associate appropriate views of the global situation with the knowledge learned about effective control decisions. We call this form of knowledge situation-specific control. This thesis investigates learning such situation-specific cooperative control knowledge. Despite the agreement among researchers in multi-agent systems about the importance of the ability for agents to learn and improve their performance, this work represents one of the few attempts at demonstrating the utility and viability of machine learning techniques for learning control in complex heterogeneous multi-agent systems. More specifically, this thesis empirically demonstrates the effectiveness of learning situation-specific control for three aspects of cooperative control: (1) Organizational roles. Organizational roles are policies for assigning responsibilities for various tasks to be performed by each of the agents in the context of global problem solving. This thesis studies learning organizational roles in a multi-agent parametric design system called L-TEAM. (2) Negotiated search. One way the agents can overcome the partial local perspective problem is by engaging in a failure-driven exchange of non-local requirements to develop the closest possible approximation to the actual composite search space. This thesis uses a case-based learning method to endow the agents with the capability to approximate non-local search requirements in a given situation, thus avoiding the need for communication. (3) Coordination strategies. Coordination mechanisms provide an agent with the ability to behave more coherently in a particular problem solving situation. The work presented in this thesis deals with incorporating learning capabilities into agents to enable them to choose a suitable subset of the coordination mechanisms based on the present problem solving situation to derive approximate coordination strategies