Virtual Marketplaces
Decision Processes
in Agent-Based
Automated
Contracting
The Magnet system meets many of the challenges of
modeling decision making for customer agents in
automated contract negotiation.
John Collins, Corey Bilot,
and Maria Gini
University of Minnesota
Bamshad Mobasher
DePaul University
B
usiness-to-business e-commerce is
expanding rapidly, letting manufacturers both broaden their customer base and increase their pool of
potential suppliers. However, negotiating
supplier contracts for the multiple components that often make up a single product
is a complicated process. Because component parts must be assembled and time
dependencies often exist among operations, scheduling is a major challenge.
Currently, there are no existing mechanisms or frameworks for automated negotiation and contracting among manufacturers, part suppliers, and specialized
subcontractors. As the sidebar, “Related
Work on Market-Based Architectures,”
describes, current e-commerce systems typically rely instead on either fixed-price catalogs or auctions. However, such systems
focus only on cost, which is just one factor
in the complicated buyer-supplier relationship. For meaningful contract negotiations,
IEEE INTERNET COMPUTING
systems must take into account other key
factors, including schedule, quality, delivery performance, and flexibility.1
The University of Minnesota has developed the Multi-Agent Negotiation Testbed
(Magnet) system,2 which is designed to
support multiple agents in negotiating contracts for tasks with temporal and precedence constraints. Magnet has been under
development since early 1998. Currently,
its distributed market infrastructure is in
place, as are the principal elements of its
customer agents, which we focus on here.
Magnet’s customer agents have two
key tasks. First, they must determine the
specific contents of a Request for Quote
(RFQ), which they send out at the start of
negotiations to solicit supplier bids. The
content of the RFQ determines how
much time suppliers have to submit bids,
and constrains the start and end times
for the tasks. Next, once bids have been
received, the customer agents review and
1089-7801/01/$10.00 ©2001 IEEE
http://computer.org/internet/
MARCH • APRIL 2001
61
Virtual Marketplaces
Related Work on Market-Based Architectures
Markets play an essential role in the economy, and market-based
architectures are a popular choice for multiple agents.1-3 However,
most existing market architectures limit agent interactions to
manual negotiations, direct agent-to-agent negotiation,4,5 or various
types of auctions.6
Auctions are becoming the predominant mechanism for agentmediated electronic commerce.7 AuctionBot6 and eMEDIATOR8
are among the most well-known examples of multiagent auction
systems. Determining the winners of combinatorial auctions is
difficult. Dynamic programming9 works for small sets of bids, but
does not scale, and it imposes significant restrictions on the bids.
Algorithms such as Bidtree8 and CASS10 have been proposed to
reduce the search complexity, but price is their only criterion for
bid selection. Our bids include a time window for each task, which
ties bid selection to scheduling.
Also, existing architectures for multiagent virtual markets
typically rely on the agents themselves to manage their interaction
details rather than providing explicit facilities and infrastructure
for managing multiple negotiation protocols. In our work, agents
interact with each other through a market. The market
infrastructure provides a common vocabulary; collects statistical
information that helps agents estimate costs, schedules, and risks;
and acts as a trusted intermediary during the negotiation process.
Most work in supply-chain management is limited to strict
hierarchical modeling of the decision-making process. This is
inadequate for distributed supply-chains, since each organization is
self-interested, not cooperative.A notable exception is the Mascot
system.11 Mascot agents coordinate scheduling with the user, but
there is no explicit notion of payments or contracts, and the criteria
for accepting or rejecting a bid are not explicitly stated.The system’s
major objective is to show the advantage of using lateral coordination
policies that focus on optimizing schedules locally through temporalconstraint exchange.12 Our objective is to negotiate contracts with
suppliers that optimize the customer’s utility.
select bids. They do this using an algorithm we
developed that operates on combinatorial problems with multiple tasks and attributes including
cost, supplier reputation and reliability, time constraints, and other risk factors.
To model decision making in the uncertain environment of contract negotiation, we use the
Expected Utility Theory. We describe that theory
here, followed by more detailed descriptions of the
customer agents’ key decision-making processes
and our search algorithm. First, we offer an
overview of the Magnet environment.
The Magnet System
Figure 1 shows an overview of the Magnet system.
The system’s market infrastructure supplies cus62
MARCH • APRIL 2001
http://computer.org/internet/
References
1. K. Sycara and A.S. Pannu,“The Retsina Multiagent System:Toward Integrating
Planning, Execution, and Information Gathering,” Proc. Second Int’l Conf. on
Autonomous Agents, IEEE CS Press, Los Alamitos, Calif., 1998, pp. 350-351.
2. M.P.Wellman and P.R.Wurman,“Market-Aware Agents for a Multiagent World,”
Robotics and Autonomous Systems, vol. 24, 1998, pp. 115-125.
3. M.Tsvetovatyy et al.,“Magma:An Agent-Based Virtual Market for Electronic
Commerce,” J.Applied Artificial Intelligence, vol. 11, no. 6, 1997, pp. 501-524.
4. T.W. Sandholm,“Negotiation Among Self-Interested Computationally Limited
Agents,” doctoral thesis, Dept. of Computer Science, University of
Massachusetts,Amherst, 1996.
5. P. Faratin, C. Sierra, and N.R. Jennings, “Negotiation Decision Functions for
Autonomous Agents,” Int’l J. Robotics and Autonomous Systems, vol. 24, no. 3-4,
1997, pp.159-182.
6. P.R.Wurman, M.P.Wellman, and W.E.Walsh,“The Michigan Internet Auction Bot:
A Configurable Auction Server for Human and Software Agents,” Second Int’l
Conf. on Autonomous Agents, IEEE CS Press, Los Alamitos, Calif., 1998, pp. 301-308.
7. R.H. Guttman,A.G. Moukas, and P. Maes,“Agent-Mediated Electronic Commerce:
A Survey,” Knowledge Engineering Review, vol. 13, no. 2, June 1998, pp.143-152.
8. T. Sandholm,“Approaches to Winner Determination in Combinatorial Auctions,”
Decision Support Systems, vol. 28, no. 1-2, 2000, pp. 165-176.
9. M.H. Rothkopf, A. Pekeuc, and R.M. Harstad, “Computationally Manageable
Combinatorial Auctions,” Management Science, vol. 44, no. 8, 1998, pp. 1131-1147.
10. Y. Fujishjima, K. Leyton-Brown, and Y. Shoham, “Taming the Computational
Complexity of Combinatorial Auctions: Optimal and Approximate
Approaches,” Proc. 16th Joint Conf. on Artificial Intelligence, Morgan Kaufman, San
Francisco, 1999.
11. N.M. Sadeh et al.,“Mascot:An Agent-Based Architecture for Coordinated MixedInitiative Supply Chain Planning and Scheduling,” Workshop on Agent-Based
Decision Support in Managing the Internet-Enabled Supply-Chain/Agents ‘99.
12. D. Kjenstad, "Coordinated Supply Chain Scheduling," doctoral dissertation,
Dept. of Production and Quality Engineering, Norwegian University of Science
and Technology,Trondheim, Norway, 1998.
tomer agents with statistics to help with decision
making, connects customer and supplier agents,
and helps the agents communicate.
Market Infrastructure
Magnet mediates agent interactions through the
market infrastructure,2 which provides a domain
ontology, a contracting protocol, and authentication services. It also tracks the requests, commitments, and progress toward task completion.
The market’s ontology describes the types of
tasks or goods available in the market. Along with
item descriptions, the ontology contains statistics,
including details such as how many suppliers typically bid on an item and how long a task typically takes. The market also keeps a registry of supIEEE INTERNET COMPUTING
Automated Contracting
Customer agent
Top-level
goal
Supplier agent
Planner
Task
network
Replan
Domain
model
Market
Market
statistics
Market
ontology
Domain
model
Commitments
Bid
manager
Rebid
Task
assignment
Execution
manager
Bid
management
Availability
Bid
protocol
Market
session
Bid
protocol
Resource
manager
Events and
responses
Figure 1. Overview of the Magnet system. Magnet mediates interactions among customer and
supplier agents and provides customer agents with market statistics.
pliers who typically participate in market activities,
and it maintains performance statistics to help customer agents (or the agent’s human user) make
decisions. Market services are dynamically delivered to participating agents through a market session. The session encapsulates a market transaction
and also serves as a persistent repository for the
current state of the transaction throughout the life
of the contract.
Agent Interactions
In Magnet, agents function in three basic phases: planning, negotiation, and execution monitoring. Here, we focus on decisions made in the
bidding cycle, which occurs in the negotiation
phase. We distinguish between two agent roles:
the customer and the supplier. A customer is an
agent who has a goal and needs resources outside its direct control to achieve it. The goal’s
value can vary over time. A supplier is an agent
who has resources or services and, in response to
a customer agent’s RFQ, might offer to provide
them to the customer for a specified price over a
specified time period.
The bidding cycle consists of several steps:
■
■
The customer agent issues an RFQ, which
includes a specification of each task and a set of
precedence relations among tasks. For each task,
the agent specifies a time window that gives the
task’s earliest start time and latest end time.
Suppliers send bids, which include the total
cost, estimates on how long each task will take,
and a time window for startup on each task.
IEEE INTERNET COMPUTING
■
■
■
When the customer agent awards a bid, it pays
the supplier a deposit (refundable only if the
supplier fails) and specifies a start time within
the supplier’s time window.
When the supplier completes a task, the customer agent pays the outstanding balance,
minus the deposit.
If the supplier fails to complete a task, it forfeits the price and returns the deposit to the
customer agent (the customer might also levy
a penalty for nonperformance, though we
ignore this possible complication here).
Once the customer agent awards bids, a secondary
protocol lets agents negotiate schedule changes. This
helps prevent outright failure and reduces risk for
both parties, at the cost of complicating the agents’
behavioral requirements during plan execution.
An Example
Assume Acme Widgets asks its agent to find the
resources to prepare a trade-show display within
two weeks. Acme’s sales department has estimated
that if the display is ready on time, it can book
sales during the show that will earn the company
US$10,000 in profit (minus the cost of the display).
The agent has three tasks to accomplish, and
there is uncertainty as to whether the suppliers
can deliver on time (we ignore the uncertainty in
the profit number). Figure 2 (next page) shows the
customer agent’s financial situation as the plan
progresses. The customer agent pays deposits on
all tasks (d1, d2, and d3) when it awards bids at the
end of the bidding cycle. Once suppliers complete
http://computer.org/internet/
MARCH • APRIL 2001
63
Virtual Marketplaces
Activity
Bidding
Task s1
Task s2
Task s3
Gain
Payoff V = 10,000
d1 + d2 + d3
c1 − d 1
c2 − d 2
c3 − d3
Loss
Time
Figure 2. The financial position of the Acme Widget
customer agent. The agent's financial position changes
as it awards bids, makes deposits and payments, and
accrues value for completed tasks.
the tasks, the customer agent makes payment on
each task, minus the deposit (c1 − d1, for example).
If a task, n, is not completed by the supplier, the
supplier agent returns the deposit dn to the customer agent. When the plan is complete, the value
V of the goal accrues.
Assuming that the customer agent receives multiple bids that specify different costs, deposits, and
time parameters, the goal of the decision process
is to award a combination of bids that maximizes
the expected utility—that is, the subjective value to
the user—at the project’s end. Because that utility
involves some probability of both loss and gain,
we must assess the risk posture of the person or
organization on whose behalf the agent is acting.
To do this, we model the agent’s decision making
based on marginal expected utility: the expected
change in the decision maker’s overall utility due
to some decision.
where pi is the probability of outcome i, and Wi
is the decision maker’s resulting wealth if outcome i is realized. Basically, decision makers
weigh the utility of each outcome within the
opportunity. To make a decision, decision makers
compare the expected utility, E(U), to their current utility, U(W0), where W0 represents their current wealth. If the opportunity’s expected utility
exceeds the current utility, the decision maker
will pursue the opportunity. Similarly, when faced
with multiple opportunities, decision makers can
decide which (if any) they will pursue by comparing the expected utilities of each opportunity
with their current utility.
Computing Utility
EUT guides our customer agents in situations
where there is a trade-off between a plan’s overall cost and its likelihood of success. For example, the agent might need to choose between suppliers who are very reliable but charge a high
price, and others who charge less but are less likely to complete the task. By computing the expected utility of different scenarios, the agent can
choose from among them.
To compute the expected utility for a plan being
executed by several supplier agents, we treat the
plan as a set of ordered task-completion events.
Each event has a probability of succeeding, and, at
the time of each event, the customer agent must
pay some supplier agent. When the last task is
complete, the customer agent gains the benefit of
plan completion. If any task fails to complete, we
assume the customer agent will abandon the plan
and forfeit the deposits paid to downstream suppliers (suppliers who have yet to begin processing
their respective tasks). For n tasks, this gives
Expected Utility Theory
To produce plans that are acceptable to users, an
automated agent must be able to handle decision
making in an uncertain environment. To model
this, we use the Expected Utility Theory (EUT),3
which models decision making under uncertainty
using probabilities and a utility curve, U(W), that
maps a level of “wealth” to a level of utility.
According to EUT, decision makers faced with
an opportunity consisting of a set of n wealthbased outcomes will calculate the expected utility over the set of outcomes:
E(U ) =
64
MARCH • APRIL 2001
n
∑U (W )p
i
i
i =1
http://computer.org/internet/
(1)
E(U ) = U (W0 ) +
n
∑
i =1
n
M ( − z i )(1 − pi )
∏p
+ M (V )
i −1
∏ p
j
j
j =1
(2)
j =1
where M(x) is the change in utility due to a financial gain of x, the pi are the success probabilities
of the successive tasks, the zi are the cumulative
“debits” resulting from each task completion (the
di and ci in Figure 2), and V is the net “credit” that
accrues on plan completion.
We call the function M(x) the marginal
expected utility of a gain of x. This notion lets us
simplify thinking about situations in which we
IEEE INTERNET COMPUTING
Automated Contracting
s2
are concerned only about changes in wealth due
to some decision. Denoting with ∆ Wi the
change in wealth relative to W0, for outcome i,
Equation 1 becomes
E(U ) = U (W0 ) +
n
∑ M (∆W )p
i
i
i =1
(3)
For our purposes, M(∆ W) is really a qualitative
concept; we do not expect to compute it exactly.
Although many functions have been proposed,4
little is known about how to elicit preferences from
users or an organization that will yield an accurate utility function. Instead, we recognize its existence and its general shape. To compute values, we
will bound M(∆ W) with a linear function as an
upper bound (risk neutral), which is fairly close to
reality for small gambles.
We define successful plan execution as “completed by the deadline” and successful task completion as “completed without violating the plan’s
temporal constraints.” A task can be completed
successfully even if it is not finished within the
bidder’s proposed time frame, as long as the schedule has sufficient slack to absorb the overrun. A
plan fails if it is completed after its deadline, and
we ignore any of the completed work’s residual
value to the customer. These definitions form our
starting point; we now extend our analysis to more
complicated cases.
Application Example
Given the example of Figure 2, the expected utility E(U) for Acme Widgets is:
E(U )
= U (W0 ) + M ( −d2 − d3 )(1 − p1 )
+ M ( −c1 − d3 ) p1 (1 − p2 )
+ M ( −c1 − c 2 ) p1 p2 (1 − p3 )
+ M ( −c1 − c 2 − c 3 ) p1 p2 p3
+ M (V ) p1 p2 p3
(4)
where V is the US$10,000 profit, dn is the deposit
paid when bid n is awarded, cn is the price paid
when task n is completed, and pn is the probability that task n will be completed by the agreedupon deadline.
To compute the expression in Equation 4, the
customer agent estimates, for each task and supplier, the probabilities that the tasks will be completed on time. The agent then computes its marginal utility M(x) for each possible outcome.
Although statistical information about tasks and
IEEE INTERNET COMPUTING
s1
s3
s5
s6
s4
Figure 3. A branching task network.
Tasks can be scheduled in qualitatively
different ways, yielding different marginal utility values.
suppliers is available from the market, the utility
function depends on the user. Users who trust their
agents to make autonomous decisions can specify
an analytical form for their own utility function.
Other users might let the agent do some computations and present alternatives, and then make the
final decision themselves.
When a set of tasks includes potentially parallel activities, the analysis is more complex. Different possible schedules might have different marginal utility values, depending on the relative costs
and success probabilities of the individual tasks.
Once a task starts, the user is liable for its full cost
at completion, regardless of whether the plan is
abandoned due to a failure on some other branch
of the plan.
Consider the plan in Figure 3, for example.
Depending on expected task durations, it might be
possible to complete task s2 before starting s5, or
to delay s2’s start until either or both of s3 and s4
are complete. It might even be possible to serialize
s3 and s4 in either order if the plan has sufficient
slack. Each of these orderings will yield a different
E(U) value. For example, if s2 is expensive relative
to tasks s3 and s4, it should be delayed until after
both s3 and s4 have completed, if possible. This
reduces the number of times the cost of s2 appears
in Equation 2.
How sensitive is a risk estimate to uncertainty
about supplier reliability? One way to answer this
is to look at the partial derivatives of E(U) with
respect to pk in Equation 2:
∂E(U )
∂pk
k −1
∏ pj
= − M(− dk )
j =1
M(− zi )(1 − pi )
+
i = k +1
n
∑
∏ j =1 p j
i −1
pk
(5)
∏i =1 pi .
n
+ M(V )
pk
If the upper bound of any derivative is negative,
http://computer.org/internet/
MARCH • APRIL 2001
65
Virtual Marketplaces
we are better off abandoning the plan because a
higher success probability for that element will
lead to a worse overall outcome. If any of the
derivatives are especially large and the confidence
in the corresponding probability estimates are low,
the customer agent should warn the user about the
uncertainty and its potential impact, and consider
adding slack to the schedule to allow for recovery.
Because of the temporal constraints among
tasks, one supplier’s failure to accomplish a task
does not necessarily mean the overall goal will fail;
recovery is possible if another supplier assumes the
task without invalidating the schedule. However,
permitting such flexibility complicates bid selection. Selecting the cheapest bids and the tightest
possible schedule is likely to create a brittle combination, and is thus not always the best choice.
We can reduce risk by consolidating tasks with
single suppliers who can bid on “packages” composed of task subsets from the RFQ. In general,
we’re better off accepting such packages from a
risk standpoint, assuming that the supplier will let
us pay for the whole package when it’s complete.
In some cases, we might be willing to pay a premium over the individual task prices to reduce risk.
This is most advantageous toward the end of the
plan. To see why, we restructure Equation 4 to consolidate tasks 2 and 3, without changing the costs
and deposit amounts:
E(U )
= U( W0 ) + M (− d2 − d3 )(1 − p1)
+ M (− c 1) p1(− p23 )
+ M (− c 1 − c 2 − c 3 + V ) p1p23 .
(6)
The fourth term from Equation 4 is now missing, and the third term represents a smaller outlay.
Also, the last term’s probability factor might be
larger if the supplier’s probability of delivering on
the consolidated task (p23) is nearly the same as
delivering on a single task (p2 or p3).
■
■
For each factor, the market maintains the sample
mean, sample size, and variance. This lets agents
compute a confidence interval and issue risk
reports to the user in a standard form: “There is
an n percent probability that your risk is less
than x.”
To estimate risk, we compute a lower bound of
the risk R. This is the absolute value of the negative part of the expected value computation, not
including the payoff for plan completion:
n
R=
■
66
MARCH • APRIL 2001
Performance to commitment (Pc): the ratio of
successes to attempts, where the task was comhttp://computer.org/internet/
i −1
∑ −zi (1 − pi )∏ p j .
i =1
(7)
j =1
A risk estimate must be made in the context of
a particular schedule for two reasons. First, specific start times determine the schedule slack
available for recovery if a supplier misses a deadline. Second, scheduling decisions affect the
ordering of parallel tasks.
Clearly, minimizing risk by adjusting task
start times is a nonlinear combinatorial optimization problem, since the amount of slack
available for failure recovery influences individual completion probabilities. We have several options, such as:
■
■
Risk and Utility
For agents to make appropriate autonomous decisions, we must estimate how risks affect the marginal expected utility. First, however, we must
determine which elements contribute to risk, and
how to estimate and decrease risk.
To support risk evaluation, Magnet’s market
maintains several types of data on each supplier
and task type.
pleted within the promised duration. It does
not include bid awards that the customer agent
abandoned before starting the task.
Performance with overruns (Pl): the percentage
of attempts completed successfully but late.
Overrun duration (tl): the lateness of completions with respect to bid durations.
Estimate marginal completion probabilities
given additional time. We can estimate performance with overrun and overrun duration
data, and assume that the improvement in
completion probability is linear in time.
Initialize each task’s start time to be as early
as possible, consistent with precedence constraints and bid specifications. This is a heuristic based on our observation that later tasks
tend to be riskier because of the larger outlays
later in the plan.
Another option is to present users with the choices, risk data, and sensitivities from Equation 5, and
let them decide.
Decision Process in Customer Agents
The customer agent takes utility and risk into
IEEE INTERNET COMPUTING
Automated Contracting
account at two points during the bidding cycle:
during RFQ composition, when it specifies the
tasks and time windows; and during bid evaluation, when it makes decisions about which bids to
accept based on such things as price, timing, and
supplier and data reliability.
Composing the RFQ
The customer agent’s goal in composing the RFQ
is to maximize the plan’s expected marginal utility at completion time. The agent can’t schedule
tasks directly; instead, it issues an RFQ to garner
a set of bids from potential suppliers. It then composes a schedule based on these bids.
The agent generates the RFQ from a task network that consists of a set of tasks, the temporal
constraints among them, and possibly nonzero
delays between tasks to cover communication and
transportation delays. Operations in task networks
need not be linearized with respect to time, since
multiple agents can execute operations in parallel.
Figure 3 shows an example task network.
Time allocation. When composing an RFQ, the
agent must decide how to allocate time in three
ways:
■
■
■
between bidding and execution of the plan;
within the bidding cycle, between suppliers and
the customer agent; and
among the tasks in the plan.
Supplier agents need time to evaluate their
resource availability and compose bids, and the
customer agent needs time to evaluate bids. When
we allocate additional time to the bidding process,
we reduce the available execution time and
increase the risk of plan failure. If we decrease time
for the bidding process, supplier and customer
agents have less time to consider their options.
Our principal strategy in time allocation
between the customer agent and supplier agents is
to allocate just enough time for the customer agent
to make a decision. The reason is that suppliers
will likely either not bid or will raise prices if they
have to reserve resources while speculating on
outstanding bids. Also, any extra time the customer agent spends on decision making reduces
the available execution time. For further guidance
on this time-allocation problem, see our detailed
characterization of the bid-evaluation process.5
Scheduling. The RFQ includes early start and late
finish times for each task. The customer agent
IEEE INTERNET COMPUTING
must set these “time windows” prior to soliciting
bids. Once the bidding cycle concludes, the agent
composes the bids into a feasible schedule. But
first, suppliers must return bids that satisfy the
plan’s precedence constraints. The customer agent
attempts to influence the availability of attractive
bids by setting the relative time allocation among
the tasks and the extent to which the time windows of adjacent tasks—those connected by precedence relations—can overlap.
To make these decisions, the customer agent
uses three kinds of task data from the Magnet
market’s ontology:
■
■
■
the number of bidders likely to submit bids,
the expected duration, and
the amount of variability in the duration data.
The customer agent builds an initial schedule using
the expected duration data. It then sets the initial
time windows using the Critical Path algorithm.6
This algorithm first walks the directed graph of tasks
and precedence constraints forward to compute the
earliest start times for each task, and then backward
from the goal time to compute the latest possible
finish and start times for each task. The entire plan’s
minimum duration is called the makespan; the difference between the goal time and the latest early
finish time is called the plan’s total slack.
In Figure 4 (next page), the green bars show the
expected duration of the tasks in Figure 3’s task network. The customer agent selected a total slack of 5
units for a 35-unit makespan, or about 14 percent.
The yellow bars’ durations are one standard deviation below the expected values, while the blue bars
show an alternative formulation in which the total
slack is apportioned among the tasks on the critical
path to produce an RFQ with no overlap among
adjacent tasks. The yellow-bar option is likely to
produce more bids and potentially lower-cost bids
because it offers suppliers additional scheduling
flexibility. The blue option will make it easier for the
customer to compose a feasible plan, assuming bids
are received to cover all the tasks. In this case, the
bid evaluation problem is reduced to a version of the
combinatorial auction-winner-determination problem, for which there are reasonably efficient optimal solution methods.7-9
Discussion. We can make an additional adjustment
using bidder population and variability data. We’re
still investigating the detailed relationships
between the bidder count and variability data, and
the optimal adjustments that must be made to the
http://computer.org/internet/
MARCH • APRIL 2001
67
Sigma
Duration
Task
Virtual Marketplaces
0
10
20
30
40
10
S1
10
2
8
10
16
S2
16
11
5
17.5
10
S3
10
6
4
11.7
8
S4
8
3
5
10.8
12
S5
12
9
3
12.5
5
S6
5
3
2
5.8
Expected
Expected-sigma
Safe
Figure 4. A timeline showing a set of tasks and alternative time allocations. The colored bars show
how the critical path algorithm can be used to reduce task durations and increase time windows.
RFQ schedule. Our current approach is to increase
the relative time allocation when duration data is
more variable, and increase the overlap when there
are many bidders.
There is a tension between issuing an RFQ that
guarantees the feasibility of any plan constructed
with the resulting bids and issuing one that will
solicit the maximum number of bids. We assume
that supplier bids are based on an evaluation of their
current resource commitments, and therefore that
larger time windows will result in more bids. Suppliers will know that more time flexibility in their
bids will give them a competitive advantage.
Evaluating and Awarding Bids
Once suppliers send their bids, the customer
68
MARCH • APRIL 2001
http://computer.org/internet/
agent’s goal is to find and schedule a bid combination that maximizes E(U), then award the bids.
The challenge is to find a good mapping of bids to
tasks, then schedule the bids so that there is a low
risk of unrecoverable failure.5 A “good” mapping
covers all tasks (each task is mapped to exactly one
bid), satisfies temporal constraints, and is relatively low-cost. Also, in our system, bids are exclusive:
When a single supplier agent submits multiple
bids, only one can be accepted (though other bid
semantics are possible).8
Search algorithm. To decide on bids, our customer
agents use our adaptive, anytime search algorithm
shown in Figure 5. We based the algorithm on a simulated annealing10 framework with modular selecIEEE INTERNET COMPUTING
Automated Contracting
1. Initialize search:
1.1 Pre-process bids: For each task, generate a list of the bids that include the task and its average bid price.
1.2 Coverage test: If any task lacks a bid, exit (coverage cannot be achieved).
1.3 Single bid test: If any task has a single bid, the bid must be part of any solution. The bid might contain
one or more tasks. Create node(s) that map the bid, compute their value V, and add them to the queue.
1.4 Initialize queue: If there are no singletons, create a node mapping all tasks to no bids and add it to the queue.
1.5 Set initial annealing temperature.
2. While not time out and improving do:
2.1 Select a node N for expansion: Select a random number R = Vmin − T ln(1−r)(Vmax − Vmin), where
■ r is a random number uniformly distributed between 0 and 1,
■ T is the current annealing temperature, and
■ nodes in the queue are sorted by increasing values; Vmin is the value of the first node, Vmax the value of
the last node.
Choose node N as the last node in the queue for which VN ≤ R.
2.2 Select a bid B:
Discard all bids that appear on the tabu list of N.
Discard all bids that appear on the tried list of N.
Choose a bid according to the current bid selection policy.
2.3 Expand node N with bid B, producing node N′: For each task mapped by bid B that is already mapped
in N to some other bid B′, remove bid B′.
If B′ is a singleton bid (see 1.3 above), abandon the expansion.
Add B to the expansions-tried list of node N.
Copy the tabu list of node N into node N′ and add bid B in front.
Truncate the tabu list in node N′ to the tabu size limit.
2.4 Evaluate node N′:
VN = CostN + RiskN + FeasN + CovN where
■ CostN is the sum of bid prices (uncovered tasks are assigned an average price),
■ RiskN is the expected cost of recovering from plan failure times a weighting factor,
■ FeasN is the weighted sum of schedule overlaps, and
■ CovN is the number of tasks that are not mapped to a bid, times a weighting factor.
2.5 Update best-node statistics.
2.6 Adjust the annealing temperature T.
3. Return the best node found.
Figure 5. The search algorithm for bid selection. We based the adaptive, anytime search algorithm
on a simulated annealing framework.
tors and evaluators. Simulated-annealing is characterized by two elements: the annealing temperature
T, which is periodically reduced by a factor ε; and
the stochastic node-selection procedure. Nodes represent sets of bids mapped to their respective tasks;
we keep them in an ordered queue of fixed maximum length, sorted by their value. Step 2.4 in the
algorithm shows how we compute the node value,
which is a combination of factors and includes a
risk component. Intuitively, risk increases whenever an agent accepts a bid that increases either (a) the
probability of missing the goal deadline or (b) the
probability of missing the latest possible start time
for some subsequent task.
As Figure 5 shows, we maintain a tabu list and
IEEE INTERNET COMPUTING
Table 1. Solutions found.
Selector
Random
Cov
FeasCov
Combined
Small-bid problem
Large-bid problem
2
3
2
6
2
0
0
1
a tried list on each node. The tabu list is a list of
bids recently used in the history of node expansions leading to the target node. By prohibiting
expansion by bids on the tabu list, we prevent
http://computer.org/internet/
MARCH • APRIL 2001
69
Virtual Marketplaces
x 104
We have implemented and tested several bidselection methods (see Step 2.2 in Figure 5):
11
Random
Cov
FeasCov
Combined
10
■
Node value
9
■
8
■
7
6
5
4
0
2
4
6
8
10
12
14
16
■
x 105
Time (msec)
Figure 6. Improvement curves for the small-bid problem.
Averages are shown for 20 runs.
9
Clearly, significant complexity costs are associated with both the Feasibility Improvement and Cost
Improvement methods.
To generate focused improvements for a given
node, you can compose these selectors into different configurations. In our experiments, we used
the following selectors:
8
■
x 104
11
Random
Cov
FeasCov
Combined
Node value
10
■
7
6
■
5
4
0
2
4
6
8
10
12
Time (msec)
14
16
x 105
Figure 7. Improvement curves for the large-bid problem.
Averages are shown for 20 runs.
short cycles in the search space and thereby
encourage exploratory behavior. The tried list is
simply the list of bids that have already been used
to expand the node in question; we don’t try the
same expansion twice.
70
Random Bid: Choose a bid at random, and
attempt to add it to the node.
Coverage Improvement: Choose a bid that covers a task that is not mapped in the node.
Feasibility Improvement: Scan mapping to find
tasks with negative slack. Of those, move tasks
constrained by their bids (rather than by predecessors or successors) to relieve negative slack.
Sort tasks by their potential to reduce infeasibility and save them. Choose the untried bid
with the highest potential to reduce infeasibility. Note that with this selector, it’s possible to
choose a bid that introduces other infeasibilities.
Cost Improvement: Choose the (untried) bid that
is responsible for the maximum positive deviation from the average (or expected) price, and
replace it with a lower bid that covers at least
the task with the highest positive cost deviation.
This selector is useful only if you have average
or expected cost data on each task.
MARCH • APRIL 2001
http://computer.org/internet/
■
Random (described above).
FeasCov: If the node is infeasible, use the Feasibility Improvement selector; otherwise, if it is
not fully covered, use the Coverage Improvement selector; otherwise, use the Random selector.
CostFeasCov: If the cost of the node’s covered
portion is above average, attempt to reduce its
cost; otherwise, use the FeasCov selector.
Combined: Run the Random selector until it
stops producing improvement, then switch to
Feasibility Improvement until it stops producing improvement, then switch back to Random, then to Coverage Improvement, then back
to Random, then to CostFeasCov, and finally
back to Random.
We performed numerous studies of the algorithm’s
performance.5 In particular, we compared the
Random bid selector with the more focused bid
selectors. Our interests are in both the ability to
find solutions and the rate of improvement. The
IEEE INTERNET COMPUTING
Automated Contracting
latter is important for setting time limits in an
anytime search.
To probe a range of problem complexity factors, we ran the Random, Coverage, FeasCov, and
Combined selectors against two problem types of
the same size, but with different complexity levels. Both sets had 50 tasks and 100 bidders, all of
which are generated with the same random number sequences. Total slack was 10 percent, and we
set task durations to 60 percent of their expected
values to relax the RFQ time windows. The average bid size (the number of tasks included in a bid)
was 5 in the small-bid problem and 15 in the
large-bid problem. Table 1 shows the number of
solutions that the different selectors found for
each problem. The actual number of such solutions is not known.
Figures 6 and 7 show the improvement curves
for the four bid selectors on the small- and largebid problems, respectively. Error bars show σ/√
n
where σ is the standard deviation across runs, and
n is the number of runs. The Combined selector
clearly gives the best overall performance, both in
terms of consistency and solution quality.
integer programming for bid evaluation.
Many challenging problems remain before a
fully automated agent can function effectively in
this environment. A vocabulary must be developed to allow unambiguous specification of tasks
and bids. Many domains require more complex
relationships among tasks than we currently support, such as staged delivery over time, or task
splitting across multiple suppliers. On the supplier side, we must develop an understanding of how
to use the bidding process to maximize the value
of supplier resources.
Our bid-evaluation test framework will be
released to the research community in 2001. Other
parts of the Magnet system will be released over
the following two years. Further information is
available at http://www.cs.umn.edu/Research/
airvl/magnet.
Acknowledgments
We gratefully acknowledge partial support for our research
from the National Science Foundation, award NSF/
IIS-0084202.
References
Conclusions and Future Work
Many business-to-business interactions are moving to the Internet and to auction-based pricing
mechanisms. Although some businesses have
achieved success trading individual goods, many
problems remain in trading bundles of goods or
coordinated tasks. If an agent is to act on behalf
of a human decision-maker in such a market, it
must be able to evaluate risk factors in ways that
the person will find reasonable. We believe
Expected Utility Theory offers a good framework
for doing this.
A major problem in markets with coordinated
tasks is determining the winning bids. It is not
sufficient to pick the lowest prices. Our stochastic approach, based on simulated annealing,
offers good, anytime performance. Our experimental results show that excessive focus on
improvement leads to faster improvement early
on, at the price of a lowered likelihood of eventually finding a solution that satisfies all constraints. We’ve reported on additional experiments elsewhere.5 Among our other studies, we
compared our simulated annealing search
method to a purely systematic search. As expected, the systematic approach reliably finds the
optimum bid assignment, at the cost of exponential scalability and anytime controllability.
We are currently studying the possibility of using
IEEE INTERNET COMPUTING
1. J. Collins et al., “Mixed-Initiative Decision Support in
Agent-Based Automated Contracting,” Proc. Fourth Int’l
Conf. on Autonomous Agents, IEEE CS Press, Los Alamitos, Calif., 2000, pp. 247-254.
2. J. Collins et al., “A Market Architecture for Multi-Agent
Contracting,” Proc. Second Int’l Conf. on Autonomous
Agents, IEEE CS Press, Los Alamitos, Calif., 1998, pp.
285-292.
3. T. Biswas, Decision-Making Under Uncertainty, St. Martin’s Press, New York, 1997.
4. L. Eeckhoudt and C. Gollier, Risk Evaluation, Management, and Sharing, Harvester Wheatsheaf, Hemel Hempstead, Hertfordshire, UK, 1995.
5. J. Collins et al., “Bid Selection Strategies for Multi-Agent
Contracting in the Presence of Scheduling Constraints,”
in Agent Mediated Electronic Commerce, Vol. LNAI1788,
Springer Verlag, New York, 2000.
6. F.S. Hillier and G.J. Lieberman, Introduction to Operations
Research, McGraw Hill, New York, 1990.
7. A. Andersson, M. Tenhunen, and F. Ygge, “Integer Programming for Combinatorial Auction Winner Determination,” Proc. 4th Int’l Conf on Multi-Agent Systems, IEEE
CS Press, Los Alamitos, Calif., 2000, pp. 39-46.
8. N. Nisan, “Bidding and Allocation in Combinatorial Auctions,” Proc. ACM Conf. on Electronic Commerce (EC’00),
ACM Press, New York, 2000, pp. 1-12.
9. T. Sandholm, “Approaches to Winner Determination in
Combinatorial Auctions,” Decision Support Systems, vol.
28, no. 1-2, 2000, pp. 165-176.
http://computer.org/internet/
MARCH • APRIL 2001
71
Virtual Marketplaces
10. C.R. Reeves, Modern Heuristic Techniques for Combinatorial Problems, John Wiley & Sons, New York, 1993.
John Collins is currently finishing a PhD in computer science
at the University of Minnesota. His research interests are in
decision-theoretic areas of artificial intelligence, including
planning and scheduling, resource management, and decision support. He is the principal developer of the Magnet
system. Collins is a member of the IEEE Computer Society,
ACM, AAAI, and INFORMS.
research focuses on methods to distribute intelligence
among robots or software agents. Her major contributions
include multiple agents, motion planning, and navigation.
Gini has coauthored more than 100 technical papers. She
is on the editorial boards of the journals Autonomous
Robots and Integrated Computer-Aided Engineering. She is
the general co-chair for the International Conference on
Autonomous Agents, 2002.
Corey Bilot currently works in the corporate database systems
department at 3M and is finishing a masters' degree in
computer science at the University of Minnesota. He
received a bachelors degree in economics from the University of Wisconsin-Eau Claire in 1991. Bilot was a casualty
actuary for four years and received his associateship in the
Casualty Actuarial Society in 1995 before leaving the profession to pursue computer science.
Bamshad Mobasher is an assistant professor of computer science at DePaul University, where he also serves as the
director of the Center for Web Intelligence. Previously, he
was on the faculty at the University of Minnesota. He
received his PhD from Iowa State University in 1994. He
is active in several research areas, including Web data
mining, autonomous software agents, and multiagent systems for e-commerce. He has authored more than 30
papers and book chapters, and has served on program
committees of numerous conferences and workshops.
Maria Gini is a professor in the Department of Computer Science and Engineering at the University of Minnesota. Her
Readers can contact the authors at gini@cs.umn.edu.
cluster computing
collaborative computing
dependable systems
distributed agents
distributed databases
distributed multimedia
grid computing
middleware
mobile & wireless systems
operating systems
real-time systems
security
IEEE
Distributed Systems Online
c o m p u t e r . o r g / d s o n l i n e
72
MARCH • APRIL 2001
http://computer.org/internet/
IEEE INTERNET COMPUTING