This paper summarizes the evolution of our research on the Cooperative Intelligent Real-Time Cont... more This paper summarizes the evolution of our research on the Cooperative Intelligent Real-Time Control Architecture (CIRCA), one of the first AI architectures designed specifically for hard-real-time environments and architecturally-enforced performance guarantees. Beginning with the objective of providing reliable real-time execution of automatically-generated plans, CIRCA research progressed to define a rigorous link between planning models, execution semantics, and performance guarantees. Formal verification techniques and automatic abstraction methods were then incorporated to improve the rigor and performance of the planning system. Multi-agent negotiation and coordination capabilities were developed to demonstrate performance guarantees spanning distributed CIRCA agents. As the limitations of CIRCA's fully-guaranteed semantics became clear, the research grew to include probabilistic versions of the problem and new solution methods. Versions of CIRCA are capable of reasoning about durative concurrent actions, exogenous events and adversaries, nondeterministic actions, and probabilistic actions and events.
The work reported in this paper was supported in part by the National Science Foundation under Gr... more The work reported in this paper was supported in part by the National Science Foundation under Grants DMC-8721492 and IRI-9158473, and by a NSF Graduate Fellowship. The opinions, ndings, and recommendations expressed in this publication are those of the author, and do not necessarily re ect the views of the NSF. 1 Note the duality of time as a resource and timeliness as measure of output quality.
Artificial intelligence, on the one hand, and behavioral and social science on the other, are ine... more Artificial intelligence, on the one hand, and behavioral and social science on the other, are inextricably related because the ability to coordinate in multiagent environments is fundamental to intelligence. Distributed artificial intelligence (DAI) has used several different social metaphors as the basis for building computational theories of intelligent coordination, with significant success. However, the real challenge in developing general theories of coordination is in bridging between the different perspectives implicit in the various social metaphors. In this paper, I describe a line of research that has led to a view of coordination as a distributed search problem, where the search space provides a common representation for organizations, plans, and schedules. The evolving approach is thus a candidate foundation for a truly interdisciplinary study of coordination from both computational and social perspectives.
Today's service-oriented systems realize many ideas from the research conducted a decade or so ag... more Today's service-oriented systems realize many ideas from the research conducted a decade or so ago in multiagent systems. Because these two fields are so deeply connected, further advances in multiagent systems could feed into tomorrow's successful service-oriented computing approaches.This article describes a 15year roadmap for service-oriented multiagent system research.
In an economy in which a producer must learn the preferences of a consumer population, it is face... more In an economy in which a producer must learn the preferences of a consumer population, it is faced with a classic decision problem: when to explore and when to exploit. If the producer has a limited number of chances to experiment, it must explicitly consider the cost of learning (in terms of foregone profit) against the value of the information acquired. Information goods add an additional dimension to this problem; due to their flexibility, they can be bundled and priced according to a number of different price schedules. An optimizing producer should consider the profit each price schedule can extract, as well as the difficulty of learning of this schedule. In this paper, we demonstrate the tradeoff between complexity and profitability for a number of common price schedules. We begin with a one-shot decision as to which schedule to learn. Schedules with moderate complexity are preferred in the short and medium term, as they are learned quickly, yet extract a significant fraction of the available profit. We then turn to the repeated version of this one-shot decision and show that moderate complexity schedules, in particular two-part tariff, perform well when the producer must adapt to nonstationarity in the consumer population. When a producer can dynamically change schedules as it learns, it can use an explicit decision-theoretic formulation to greedily select the schedule which appears to yield the greatest profit in the next period.
The term multi-agent system is currently in vogue, and has been generally applied to any system t... more The term multi-agent system is currently in vogue, and has been generally applied to any system that is, or can be considered to be, composed of multiple interacting agents. In the various multi-agent (or, more properly, multiagent) systems that have been proposed developed, a wide variety of "agents" have been considered, ranging from fully autonomous intelligent agents (such as people) down to relatively simple entities (such as rules or clusters of rules). Thus, as it has become progressively used, "multiagent systems" has come to encompass an increasingly broad variety of issues, approaches, and phenomena, to the point
Figure 2 illustrates different scenarios possible during plan execution. If the executing plan ha... more Figure 2 illustrates different scenarios possible during plan execution. If the executing plan handles the current state, CIRCA has ensured failure avoidance. However, if the system leaves this planned-for state set, then the current plan may not maintain safety, and CIRCA must ...
Allocating scarce resources among agents to maximize global utility is, in general, computational... more Allocating scarce resources among agents to maximize global utility is, in general, computationally challenging. We focus on problems where resources enable agents to execute actions in stochastic environments, modeled as Markov decision processes (MDPs), such that the value of a resource bundle is defined as the expected value of the optimal MDP policy realizable given these resources. We present an algorithm that simultaneously solves the resource-allocation and the policy-optimization problems. This allows us to avoid explicitly representing utilities over exponentially many resource bundles, leading to drastic (often exponential) reductions in computational complexity. We then use this algorithm in the context of self-interested agents to design a combinatorial auction for allocating resources. We empirically demonstrate the effectiveness of our approach by showing that it can, in minutes, optimally solve problems for which a straightforward combinatorial resource-allocation tec...
Abstract. Distributed problem solving involves the collective effort of multiple problems solvers... more Abstract. Distributed problem solving involves the collective effort of multiple problems solvers to combine their knowledge, information, and capabilities so as to develop solutions to problems that each could not have solved as well (if at all) alone. The challenge in distributed problem solving is thus in marshalling the distributed capabilities in the right ways so that the problem solving activities of each agent complement the activities of the others, so as to lead efficiently to effective solutions. Thus, while working together leads to distributed problem solving, there is also the distributed problem of how to work together that must be solved. We consider that problem to be a distributed planning problem, where each agent must formulate plans for what it will do that take into account (sufficiently well) the plans of other agents. In this paper, we characterize the variations of distributed problem solving and distributed planning, and summarize some of the basic techniqu...
From the 9th of November to the 14th of November 2008 the Dagstuhl Seminar 08461 Planning in Mult... more From the 9th of November to the 14th of November 2008 the Dagstuhl Seminar 08461 Planning in Multiagent Systems was held in Schloss Dagstuhl Leibniz Center for Informatics. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The rst section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available.
Approximate models of world state transitions are necessary when building plans for complex syste... more Approximate models of world state transitions are necessary when building plans for complex systems operating in dynamic environments. External event probabilities can depend on state feature values as well as time spent in that particular state. We assign temporally-dependent probability functions to state transitions. These functions are used to locally compute state probabilities, which are then used to select highly probable goal paths and eliminate improbable states. This probabilistic model has been implemented in the Cooperative Intelligent Real-time Control Architecture (CIRCA), which combines an AI planner with a separate real-time system such that plans are developed, scheduled, and executed with real-time guarantees. We present flight simulation tests that demonstrate how our probabilistic model may improve CIRCA performance. 1 INTRODUCTION To control a complex system, an agent must build and execute a plan that is capable of recognizing state changes due to its own actio...
Decentralized MDPs provide powerful models of interactions in multi-agent environments, but are o... more Decentralized MDPs provide powerful models of interactions in multi-agent environments, but are often very difficult or even computationally infeasible to solve optimally. Here we develop a hierarchical approach to solving a restricted set of decentralized MDPs. By forming commitments with other agents and modeling these concisely in their local MDPs, agents effectively, efficiently, and distributively formulate co-ordinated local policies. We introduce a novel construction that captures commitments as constraints on local policies and show how Linear Programming can be used to achieve local optimality subject to these constraints. In contrast to other commitment enforcement approaches, we show ours to be more robust in capturing the intended commitment semantics while maximizing local utility. We also describe a commitment-space heuristic search algorithm that can be used to approximate optimal joint policies. A preliminary empirical evaluation suggests that our approach yields fas...
appears in Procs KSCO-2002 Military Coalitions are examples of large-scale multi-faceted virtual ... more appears in Procs KSCO-2002 Military Coalitions are examples of large-scale multi-faceted virtual organizations, which sometimes need to be rapidly created and flexibly changed as circumstances alter. The Coalition Agents eXperiment (CoAX) aims to show that multi-agent systems are an effective way of dealing with the complexity of real-world problems, such as agile and robust Coalition operations and enabling interoperability between heterogeneous components including legacy and actual military systems. CoAX is an international collaboration carried out under the auspices of DARPA’s Control of Agent-Based Systems (CoABS) program. Building on the CoABS Grid framework, the CoAX agent infrastructure groups agents into domains that reflect real-world organizational, functional and national boundaries, such that security and access to agents and information can be governed by policies at multiple levels. A series of staged demonstrations of increased complexity are being conducted in a re...
Abstract: There is more to scaling up agent-based systems than simply increasing the number of ag... more Abstract: There is more to scaling up agent-based systems than simply increasing the number of agents involved. Many of the challenges to making agent-based systems work in more realistic settings arise from the characteristics of the agents ’ tasks and environment, and the expectations of the systems ’ users. In this chapter, my goal is thus to emphasize this broader array of challenges to coordinating agent-based systems, as a step both towards extending our understanding of scale-up issues as well as towards developing richer metrics for evaluating the degree to which coordination strategies for agent-based systems can apply to more demanding applications. Key words: Multi-agent systems; coordination 1.
In multi-agent MDPs, it is generally necessary to consider the joint state space of all agents, m... more In multi-agent MDPs, it is generally necessary to consider the joint state space of all agents, making the size of the problem and the solution exponential in the number of agents. However, often interactions between the agents are only local, which suggests a more compact problem representation. We consider a subclass of multi-agent MDPs with local interactions where dependencies between agents are asymmetric, meaning that agents can affect others in a unidirectional manner. This asymmetry, which often occurs in domains with authority-driven relationships between agents, allows us to make better use of the locality of agents ’ interactions. We present and analyze a graphical model of such problems and show that, for some classes of problems, it can be exploited to yield significant (sometimes exponential) savings in problem and solution size, as well as in computational efficiency of solution algorithms. 1.
We address situations in which an agent with uncertainty in rewards can selectively query another... more We address situations in which an agent with uncertainty in rewards can selectively query another agent/human to improve its knowledge of rewards and thus its policy. When there is a time delay between posing the query and receiving the response, the agent must determine how to behave in the transient phase while waiting for the response. Thus, in order to act optimally the agent must jointly optimize its transient policy along with its query. In this paper, we formalize the aforementioned joint optimization problem and provide a new algorithm called JQTP for optimizing the Joint Query and Transient Policy. In addition, we provide a clustering technique that can be used in JQTP to flexibly trade performance for reduced computation. We illustrate our algorithms on a machine configuration task.
Knobtock and Korf have determined that abstraction can reduce search at a single agent from expon... more Knobtock and Korf have determined that abstraction can reduce search at a single agent from exponential tolinear complexity (Knoblock 1991; Korf 1987). We extend their results by showing how concurrent problem solving among multiple agents using abstraction can further educe search to logarithmic omplexity. We empirically validate our formal analysis by showing that it correctly predicts performance for the Towers of Hanoi problem (which meets all of the assumptions of the analysis). Furthermore, a powerful form of abstraction for large multiagent systems is to group agents into teams, and teams of agents into larger teams, to form an organizational pyramid. We apply our analysis to such an organization ofagents and demonstrate the results in a delivery task domain. Our predictions about abstraction's benefits can also be met in this more realistic domain, even though assumptions made in our analysis are violated, Our analytical results thus hold the promise for explaining in g...
This paper summarizes the evolution of our research on the Cooperative Intelligent Real-Time Cont... more This paper summarizes the evolution of our research on the Cooperative Intelligent Real-Time Control Architecture (CIRCA), one of the first AI architectures designed specifically for hard-real-time environments and architecturally-enforced performance guarantees. Beginning with the objective of providing reliable real-time execution of automatically-generated plans, CIRCA research progressed to define a rigorous link between planning models, execution semantics, and performance guarantees. Formal verification techniques and automatic abstraction methods were then incorporated to improve the rigor and performance of the planning system. Multi-agent negotiation and coordination capabilities were developed to demonstrate performance guarantees spanning distributed CIRCA agents. As the limitations of CIRCA's fully-guaranteed semantics became clear, the research grew to include probabilistic versions of the problem and new solution methods. Versions of CIRCA are capable of reasoning about durative concurrent actions, exogenous events and adversaries, nondeterministic actions, and probabilistic actions and events.
The work reported in this paper was supported in part by the National Science Foundation under Gr... more The work reported in this paper was supported in part by the National Science Foundation under Grants DMC-8721492 and IRI-9158473, and by a NSF Graduate Fellowship. The opinions, ndings, and recommendations expressed in this publication are those of the author, and do not necessarily re ect the views of the NSF. 1 Note the duality of time as a resource and timeliness as measure of output quality.
Artificial intelligence, on the one hand, and behavioral and social science on the other, are ine... more Artificial intelligence, on the one hand, and behavioral and social science on the other, are inextricably related because the ability to coordinate in multiagent environments is fundamental to intelligence. Distributed artificial intelligence (DAI) has used several different social metaphors as the basis for building computational theories of intelligent coordination, with significant success. However, the real challenge in developing general theories of coordination is in bridging between the different perspectives implicit in the various social metaphors. In this paper, I describe a line of research that has led to a view of coordination as a distributed search problem, where the search space provides a common representation for organizations, plans, and schedules. The evolving approach is thus a candidate foundation for a truly interdisciplinary study of coordination from both computational and social perspectives.
Today's service-oriented systems realize many ideas from the research conducted a decade or so ag... more Today's service-oriented systems realize many ideas from the research conducted a decade or so ago in multiagent systems. Because these two fields are so deeply connected, further advances in multiagent systems could feed into tomorrow's successful service-oriented computing approaches.This article describes a 15year roadmap for service-oriented multiagent system research.
In an economy in which a producer must learn the preferences of a consumer population, it is face... more In an economy in which a producer must learn the preferences of a consumer population, it is faced with a classic decision problem: when to explore and when to exploit. If the producer has a limited number of chances to experiment, it must explicitly consider the cost of learning (in terms of foregone profit) against the value of the information acquired. Information goods add an additional dimension to this problem; due to their flexibility, they can be bundled and priced according to a number of different price schedules. An optimizing producer should consider the profit each price schedule can extract, as well as the difficulty of learning of this schedule. In this paper, we demonstrate the tradeoff between complexity and profitability for a number of common price schedules. We begin with a one-shot decision as to which schedule to learn. Schedules with moderate complexity are preferred in the short and medium term, as they are learned quickly, yet extract a significant fraction of the available profit. We then turn to the repeated version of this one-shot decision and show that moderate complexity schedules, in particular two-part tariff, perform well when the producer must adapt to nonstationarity in the consumer population. When a producer can dynamically change schedules as it learns, it can use an explicit decision-theoretic formulation to greedily select the schedule which appears to yield the greatest profit in the next period.
The term multi-agent system is currently in vogue, and has been generally applied to any system t... more The term multi-agent system is currently in vogue, and has been generally applied to any system that is, or can be considered to be, composed of multiple interacting agents. In the various multi-agent (or, more properly, multiagent) systems that have been proposed developed, a wide variety of "agents" have been considered, ranging from fully autonomous intelligent agents (such as people) down to relatively simple entities (such as rules or clusters of rules). Thus, as it has become progressively used, "multiagent systems" has come to encompass an increasingly broad variety of issues, approaches, and phenomena, to the point
Figure 2 illustrates different scenarios possible during plan execution. If the executing plan ha... more Figure 2 illustrates different scenarios possible during plan execution. If the executing plan handles the current state, CIRCA has ensured failure avoidance. However, if the system leaves this planned-for state set, then the current plan may not maintain safety, and CIRCA must ...
Allocating scarce resources among agents to maximize global utility is, in general, computational... more Allocating scarce resources among agents to maximize global utility is, in general, computationally challenging. We focus on problems where resources enable agents to execute actions in stochastic environments, modeled as Markov decision processes (MDPs), such that the value of a resource bundle is defined as the expected value of the optimal MDP policy realizable given these resources. We present an algorithm that simultaneously solves the resource-allocation and the policy-optimization problems. This allows us to avoid explicitly representing utilities over exponentially many resource bundles, leading to drastic (often exponential) reductions in computational complexity. We then use this algorithm in the context of self-interested agents to design a combinatorial auction for allocating resources. We empirically demonstrate the effectiveness of our approach by showing that it can, in minutes, optimally solve problems for which a straightforward combinatorial resource-allocation tec...
Abstract. Distributed problem solving involves the collective effort of multiple problems solvers... more Abstract. Distributed problem solving involves the collective effort of multiple problems solvers to combine their knowledge, information, and capabilities so as to develop solutions to problems that each could not have solved as well (if at all) alone. The challenge in distributed problem solving is thus in marshalling the distributed capabilities in the right ways so that the problem solving activities of each agent complement the activities of the others, so as to lead efficiently to effective solutions. Thus, while working together leads to distributed problem solving, there is also the distributed problem of how to work together that must be solved. We consider that problem to be a distributed planning problem, where each agent must formulate plans for what it will do that take into account (sufficiently well) the plans of other agents. In this paper, we characterize the variations of distributed problem solving and distributed planning, and summarize some of the basic techniqu...
From the 9th of November to the 14th of November 2008 the Dagstuhl Seminar 08461 Planning in Mult... more From the 9th of November to the 14th of November 2008 the Dagstuhl Seminar 08461 Planning in Multiagent Systems was held in Schloss Dagstuhl Leibniz Center for Informatics. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The rst section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available.
Approximate models of world state transitions are necessary when building plans for complex syste... more Approximate models of world state transitions are necessary when building plans for complex systems operating in dynamic environments. External event probabilities can depend on state feature values as well as time spent in that particular state. We assign temporally-dependent probability functions to state transitions. These functions are used to locally compute state probabilities, which are then used to select highly probable goal paths and eliminate improbable states. This probabilistic model has been implemented in the Cooperative Intelligent Real-time Control Architecture (CIRCA), which combines an AI planner with a separate real-time system such that plans are developed, scheduled, and executed with real-time guarantees. We present flight simulation tests that demonstrate how our probabilistic model may improve CIRCA performance. 1 INTRODUCTION To control a complex system, an agent must build and execute a plan that is capable of recognizing state changes due to its own actio...
Decentralized MDPs provide powerful models of interactions in multi-agent environments, but are o... more Decentralized MDPs provide powerful models of interactions in multi-agent environments, but are often very difficult or even computationally infeasible to solve optimally. Here we develop a hierarchical approach to solving a restricted set of decentralized MDPs. By forming commitments with other agents and modeling these concisely in their local MDPs, agents effectively, efficiently, and distributively formulate co-ordinated local policies. We introduce a novel construction that captures commitments as constraints on local policies and show how Linear Programming can be used to achieve local optimality subject to these constraints. In contrast to other commitment enforcement approaches, we show ours to be more robust in capturing the intended commitment semantics while maximizing local utility. We also describe a commitment-space heuristic search algorithm that can be used to approximate optimal joint policies. A preliminary empirical evaluation suggests that our approach yields fas...
appears in Procs KSCO-2002 Military Coalitions are examples of large-scale multi-faceted virtual ... more appears in Procs KSCO-2002 Military Coalitions are examples of large-scale multi-faceted virtual organizations, which sometimes need to be rapidly created and flexibly changed as circumstances alter. The Coalition Agents eXperiment (CoAX) aims to show that multi-agent systems are an effective way of dealing with the complexity of real-world problems, such as agile and robust Coalition operations and enabling interoperability between heterogeneous components including legacy and actual military systems. CoAX is an international collaboration carried out under the auspices of DARPA’s Control of Agent-Based Systems (CoABS) program. Building on the CoABS Grid framework, the CoAX agent infrastructure groups agents into domains that reflect real-world organizational, functional and national boundaries, such that security and access to agents and information can be governed by policies at multiple levels. A series of staged demonstrations of increased complexity are being conducted in a re...
Abstract: There is more to scaling up agent-based systems than simply increasing the number of ag... more Abstract: There is more to scaling up agent-based systems than simply increasing the number of agents involved. Many of the challenges to making agent-based systems work in more realistic settings arise from the characteristics of the agents ’ tasks and environment, and the expectations of the systems ’ users. In this chapter, my goal is thus to emphasize this broader array of challenges to coordinating agent-based systems, as a step both towards extending our understanding of scale-up issues as well as towards developing richer metrics for evaluating the degree to which coordination strategies for agent-based systems can apply to more demanding applications. Key words: Multi-agent systems; coordination 1.
In multi-agent MDPs, it is generally necessary to consider the joint state space of all agents, m... more In multi-agent MDPs, it is generally necessary to consider the joint state space of all agents, making the size of the problem and the solution exponential in the number of agents. However, often interactions between the agents are only local, which suggests a more compact problem representation. We consider a subclass of multi-agent MDPs with local interactions where dependencies between agents are asymmetric, meaning that agents can affect others in a unidirectional manner. This asymmetry, which often occurs in domains with authority-driven relationships between agents, allows us to make better use of the locality of agents ’ interactions. We present and analyze a graphical model of such problems and show that, for some classes of problems, it can be exploited to yield significant (sometimes exponential) savings in problem and solution size, as well as in computational efficiency of solution algorithms. 1.
We address situations in which an agent with uncertainty in rewards can selectively query another... more We address situations in which an agent with uncertainty in rewards can selectively query another agent/human to improve its knowledge of rewards and thus its policy. When there is a time delay between posing the query and receiving the response, the agent must determine how to behave in the transient phase while waiting for the response. Thus, in order to act optimally the agent must jointly optimize its transient policy along with its query. In this paper, we formalize the aforementioned joint optimization problem and provide a new algorithm called JQTP for optimizing the Joint Query and Transient Policy. In addition, we provide a clustering technique that can be used in JQTP to flexibly trade performance for reduced computation. We illustrate our algorithms on a machine configuration task.
Knobtock and Korf have determined that abstraction can reduce search at a single agent from expon... more Knobtock and Korf have determined that abstraction can reduce search at a single agent from exponential tolinear complexity (Knoblock 1991; Korf 1987). We extend their results by showing how concurrent problem solving among multiple agents using abstraction can further educe search to logarithmic omplexity. We empirically validate our formal analysis by showing that it correctly predicts performance for the Towers of Hanoi problem (which meets all of the assumptions of the analysis). Furthermore, a powerful form of abstraction for large multiagent systems is to group agents into teams, and teams of agents into larger teams, to form an organizational pyramid. We apply our analysis to such an organization ofagents and demonstrate the results in a delivery task domain. Our predictions about abstraction's benefits can also be met in this more realistic domain, even though assumptions made in our analysis are violated, Our analytical results thus hold the promise for explaining in g...
Uploads
Papers by Edmund Durfee