What Is Simulated Annealing?

Simulated annealing is a method for solving unconstrained and bound-constrained optimization problems. The
method models the physical process of heating a material and then slowly lowering the temperature to decrease
defects, thus minimizing the system energy.

At each iteration of the simulated annealing algorithm, a new point is randomly generated. The distance of the
new point from the current point, or the extent of the search, is based on a probability distribution with a scale
proportional to the temperature. The algorithm accepts all new points that lower the objective, but also, with a
certain probability, points that raise the objective. By accepting points that raise the objective, the algorithm
avoids being trapped in local minima, and is able to explore globally for more possible solutions. An annealing
schedule is selected to systematically decrease the temperature as the algorithm proceeds. As the temperature
decreases, the algorithm reduces the extent of its search to converge to a minimum.

Simulated Annealing Terminology

Objective Function
The objective function is the function you want to optimize. Global Optimization Toolbox algorithms attempt to
find the minimum of the objective function. Write the objective function as a file or anonymous function, and pass
it to the solver as a function handle. For more information, see Compute Objective Functions and Create
Function Handle.

Temperature
The temperature is a parameter in simulated annealing that affects two aspects of the algorithm:

 The distance of a trial point from the current point (See Outline of the Algorithm, Step 1.)
 The probability of accepting a trial point with higher objective function value (See Outline of the Algorithm, Step
2.)
Temperature can be a vector with different values for each component of the current point. Typically, the initial
temperature is a scalar.

Temperature decreases gradually as the algorithm proceeds. You can specify the initial temperature as a positive
scalar or vector in the InitialTemperature option. You can specify the temperature as a function of iteration
number as a function handle in the TemperatureFcn option. The temperature is a function of the Annealing
Parameter, which is a proxy for the iteration number. The slower the rate of temperature decrease, the better the
chances are of finding an optimal solution, but the longer the run time. For a list of built-in temperature functions
and the syntax of a custom temperature function, see Temperature Options.

Annealing Parameter
The annealing parameter is a proxy for the iteration number. The algorithm can raise temperature by setting the
annealing parameter to a lower value than the current iteration. (SeeReannealing.) You can specify the
temperature schedule as a function handle with the TemperatureFcn option.

Reannealing
Annealing is the technique of closely controlling the temperature when cooling a material to ensure that it
reaches an optimal state. Reannealing raises the temperature after the algorithm accepts a certain number of
new points, and starts the search again at the higher temperature. Reannealing avoids the algorithm getting
caught at local minima. Specify the reannealing schedule with the ReannealInterval option.

How Simulated Annealing Works

Outline of the Algorithm
The simulated annealing algorithm performs the following steps:
1. The algorithm generates a random trial point. The algorithm chooses the distance of the
trial point from the current point by a probability distribution with a scale depending on
the current temperature. You set the trial point distance distribution as a function with
the AnnealingFcn option. Choices:
 @annealingfast (default) — Step length equals the current temperature, and direction is
uniformly random.
 @annealingboltz — Step length equals the square root of temperature, and direction is
uniformly random.
 @myfun — Custom annealing algorithm, myfun. For custom annealing function syntax,
see Algorithm Settings.
2. The algorithm determines whether the new point is better or worse than the current point.
If the new point is better than the current point, it becomes the next point. If the new point
is worse than the current point, the algorithm can still make it the next point. The
algorithm accepts a worse point based on an acceptance function. Choose the acceptance
function with theAcceptanceFcn option. Choices:
 @acceptancesa (default) — Simulated annealing acceptance function. The probability of
acceptance is
( )
,
Δmax(T)
11+exp
where
Δ = new objective – old objective.
T = initial temperature of component i
0

T = the current temperature.

Since both Δ and T are positive, the probability of acceptance is between 0 and 1/2.
Smaller temperature leads to smaller acceptance probability. Also, larger Δ leads to
smaller acceptance probability.
 @myfun — Custom acceptance function, myfun. For custom acceptance function syntax,
see Algorithm Settings.
3. The algorithm systematically lowers the temperature, storing the best point found so far.
The TemperatureFcn option specifies the function the algorithm uses to update the
temperature. Let kdenote the annealing parameter. (The annealing parameter is the same
as the iteration number until reannealing.) Options:
 @temperatureexp (default) — T = T * 0.95^k. 0

 @temperaturefast — T = T / k. 0

 @temperatureboltz — T = T / log(k). 0

 @myfun — Custom temperature function, myfun. For custom temperature function syntax,
see Temperature Options.
4. simulannealbnd reanneals after it accepts ReannealInterval points. Reannealing sets the
annealing parameters to lower values than the iteration number, thus raising the
temperature in each dimension. The annealing parameters depend on the values of
estimated gradients of the objective function in each dimension. The basic formula is
( )
( )
ki=log ,
T0Timaxj sj si
where
 k = annealing parameter for component i.
i

T = initial temperature of component i.

0

T = current temperature of component i.

i

s = gradient of objective in direction i times difference of bounds in direction i.

i

simulannealbnd safeguards the annealing parameter values against Inf and other
improper values.
5. The algorithm stops when the average change in the objective function is small relative
to FunctionTolerance, or when it reaches any other stopping criterion. See Stopping
Conditions for the Algorithm.
For more information on the algorithm, see Ingber [1].
Stopping Conditions for the Algorithm
The simulated annealing algorithm uses the following conditions to determine when to stop:
 FunctionTolerance — The algorithm runs until the average change in value of the objective
function in StallIterLim iterations is less than the value of FunctionTolerance. The default
value is 1e-6.
 MaxIterations — The algorithm stops when the number of iterations exceeds this maximum
number of iterations. You can specify the maximum number of iterations as a positive integer
orInf. The default value is Inf.
 MaxFunctionEvaluations specifies the maximum number of evaluations of the objective
function. The algorithm stops if the number of function evaluations exceeds the value
ofMaxFunctionEvaluations. The default value is 3000*numberofvariables.
 MaxTime specifies the maximum time in seconds the algorithm runs before stopping. The
default value is Inf.
 ObjectiveLimit — The algorithm stops when the best objective function value is less than or
equal to the value of ObjectiveLimit. The default value is -Inf.
Bibliography
[1] Ingber, L. Adaptive simulated annealing (ASA): Lessons learned. Invited paper to a
special issue of the Polish Journal Control and Cybernetics on "Simulated Annealing
Applied to Combinatorial Optimization." 1995. Available
from http://www.ingber.com/asa96_lessons.ps.gz

Simulated Annealing Options

Set Simulated Annealing Options at the Command Line
Specify options by creating an options structure using the optimoptions function as follows:
options = optimoptions(@simulannealbnd,'Param1',value1,'Param2',value2, ...);

See Set Options for simulannealbnd at the Command Line for examples.
Each option in this section is listed by its field name in the options structure. For
example, InitialTemperature refers to the corresponding field of the options structure.
Plot Options
Plot options enable you to plot data from the simulated annealing solver while it is running.
PlotInterval specifies the number of iterations between consecutive calls to the plot
function.
 To display a plot when calling simulannealbnd from the command line, set the PlotFcn field
of options to be a function handle to the plot function. You can specify any of the following
plots:
 @saplotbestf plots the best objective function value.
 @saplotbestx plots the current best point.
 @saplotf plots the current function value.
 @saplotx plots the current point.
 @saplotstopping plots stopping criteria levels.
 @saplottemperature plots the temperature at each iteration.
 @myfun plots a custom plot function, where myfun is the name of your function. See Structure
of the Plot Functions for a description of the syntax.
For example, to display the best objective plot, set options as follows
options = optimoptions(@simulannealbnd,'PlotFcn',@saplotbestf);

To display multiple plots, use the cell array syntax

options = optimoptions(@simulannealbnd,'PlotFcn',{@plotfun1,@plotfun2, ...});

where @plotfun1, @plotfun2, and so on are function handles to the plot functions.
If you specify more than one plot function, all plots appear as subplots in the same window.
Right-click any subplot to obtain a larger version in a separate figure window.
Structure of the Plot Functions

The first line of a plot function has the form

function stop = plotfun(options,optimvalues,flag)

The input arguments to the function are

 options — Options structure created using optimoptions.
 optimvalues — Structure containing information about the current state of the solver. The
structure contains the following fields:
o x — Current point
o fval — Objective function value at x
o bestx — Best point found so far
o bestfval — Objective function value at best point
o temperature — Current temperature
o iteration — Current iteration
o funccount — Number of function evaluations
o t0 — Start time for algorithm
o k — Annealing parameter
 flag — Current state in which the plot function is called. The possible values for flag are
o 'init' — Initialization state
o 'iter' — Iteration state
o 'done' — Final state
The output argument stop provides a way to stop the algorithm at the current
iteration. stop can have the following values:
 false — The algorithm continues to the next iteration.
 true — The algorithm terminates at the current iteration.
 Temperature Options
Temperature options specify how the temperature will be lowered at each iteration over the
course of the algorithm.
 InitialTemperature — Initial temperature at the start of the algorithm. The default is 100.
The initial temperature can be a vector with the same length as x, the vector of
unknowns.simulannealbnd expands a scalar initial temperature into a vector.
 TemperatureFcn — Function used to update the temperature schedule. Let k denote the
annealing parameter. (The annealing parameter is the same as the iteration number until
reannealing.) The options are:
o @temperatureexp — The temperature is equal to InitialTemperature * 0.95^k. This is the
default.
o @temperaturefast — The temperature is equal to InitialTemperature / k.
o @temperatureboltz — The temperature is equal to InitialTemperature / ln(k).
o @myfun — Uses a custom function, myfun, to update temperature. The syntax is:
temperature = myfun(optimValues,options)

where optimValues is a structure described in Structure of the Plot Functions. options is

either created with optimoptions, or consists of default options, if you did not create an
options structure. Both the annealing parameter optimValues.k and the
temperature optimValues.temperature are vectors with length equal to the number of
elements of the current point x. For example, the function temperaturefast is:
temperature = options.InitialTemperature./optimValues.k;

Algorithm Settings
Algorithm settings define algorithmic specific parameters used in generating new points at
each iteration.
Parameters that can be specified for simulannealbnd are:
 DataType — Type of data to use in the objective function. Choices:
o 'double' (default) — A vector of type double.
o 'custom' — Any other data type. You must provide a 'custom' annealing function. You
cannot use a hybrid function.
 AnnealingFcn — Function used to generate new points for the next iteration. The choices are:
o @annealingfast — The step has length temperature, with direction uniformly at random. This
is the default.
o @annealingboltz — The step has length square root of temperature, with direction uniformly
at random.
o @myfun — Uses a custom annealing algorithm, myfun. The syntax is:
newx = myfun(optimValues,problem)

where optimValues is a structure described in Structure of the Output Function, and problem is a
structure containing the following information:

 objective: function handle to the objective function

 x0: the start point
 nvar: number of decision variables
 lb: lower bound on decision variables
 ub: upper bound on decision variables
 For example, the current position is optimValues.x, and the current objective function value
is problem.objective(optimValues.x).
 ReannealInterval — Number of points accepted before reannealing. The default value
is 100.
 AcceptanceFcn — Function used to determine whether a new point is accepted or not. The
choices are:
o @acceptancesa — Simulated annealing acceptance function, the default. If the new objective
function value is less than the old, the new point is always accepted. Otherwise, the new point
is accepted at random with a probability depending on the difference in objective function
values and on the current temperature. The acceptance probability is
( )
,
Δmax(T)
11+exp
where Δ = new objective – old objective, and T is the current temperature. Since both Δ
and T are positive, the probability of acceptance is between 0 and 1/2. Smaller temperature
leads to smaller acceptance probability. Also, larger Δ leads to smaller acceptance
probability.
o @myfun — A custom acceptance function, myfun. The syntax is:
acceptpoint = myfun(optimValues,newx,newfval);

where optimValues is a structure described in Structure of the Output Function, newx is the point
being evaluated for acceptance, and newfval is the objective function at newx.acceptpoint is a
Boolean, with value true to accept newx, and false to reject newx.

Hybrid Function Options

A hybrid function is another minimization function that runs during or at the end of iterations
of the solver. HybridInterval specifies the interval (if not never or end) at which the hybrid
function is called. You can specify a hybrid function using the HybridFcn option. The choices
are:
 [] — No hybrid function.
 @fminsearch — Uses the MATLAB® function fminsearch to perform unconstrained
minimization.
 @patternsearch— Uses patternsearch to perform constrained or unconstrained
minimization.
 @fminunc — Uses the Optimization Toolbox™ function fminunc to perform unconstrained
minimization.
 @fmincon — Uses the Optimization Toolbox function fmincon to perform constrained
minimization.
Note: Ensure that your hybrid function accepts your problem constraints. Otherwise, simulannealbnd throw
You can set separate options for the hybrid function. Use optimset for fminsearch,
or optimoptions for fmincon, patternsearch, or fminunc. For example:
hybridopts = optimoptions('fminunc','Display','iter','Algorithm','quasi-
newton');

Include the hybrid options in the simulannealbnd options structure as follows:

options = optimoptions(@simulannealbnd,options,'HybridFcn',
{@fminunc,hybridopts});
 hybridopts must exist before you set options.

See Include a Hybrid Function for an example.

Stopping Criteria Options
Stopping criteria determine what causes the algorithm to terminate. You can specify the
following options:
 FunctionTolerance — The algorithm runs until the average change in value of the objective
function in StallIterLim iterations is less than FunctionTolerance. The default value is 1e-6.
 MaxIterations — The algorithm stops if the number of iterations exceeds this maximum
number of iterations. You can specify the maximum number of iterations as a positive integer
or Inf.Inf is the default.
 MaxFunctionEvaluations specifies the maximum number of evaluations of the objective
function. The algorithm stops if the number of function evaluations exceeds the maximum
number of function evaluations. The allowed maximum is 3000*numberofvariables.
 MaxTime specifies the maximum time in seconds the algorithm runs before stopping.
 ObjectiveLimit — The algorithm stops if the best objective function value is less than or
equal to the value of ObjectiveLimit.
Output Function Options
Output functions are functions that the algorithm calls at each iteration. The default value is
to have no output function, []. You must first create an output function using the syntax
described inStructure of the Output Function.
Using the Optimization app:
 Specify Output function as @myfun, where myfun is the name of your function.
 To pass extra parameters in the output function, use Anonymous Functions.
 For multiple output functions, enter a cell array of output function
handles: {@myfun1,@myfun2,...}.
At the command line:
 options = optimoptions(@simulannealbnd,'OutputFcn',@myfun);
 For multiple output functions, enter a cell array:
options = optimoptions(@simulannealbnd,'OutputFcn',{@myfun1,@myfun2,...});

To see a template that you can use to write your own output functions, enter
edit saoutputfcntemplate

at the MATLAB command line.

Structure of the Output Function

The output function has the following calling syntax.

[stop,options,optchanged] = myfun(options,optimvalues,flag)

The function has the following input arguments:

 options — Options structure created using optimoptions.
 optimvalues — Structure containing information about the current state of the solver. The
structure contains the following fields:
o x — Current point
o fval — Objective function value at x
o bestx — Best point found so far
o bestfval — Objective function value at best point
o temperature — Current temperature, a vector the same length as x
o iteration — Current iteration
o funccount — Number of function evaluations
o t0 — Start time for algorithm
o k — Annealing parameter, a vector the same length as x
 flag — Current state in which the output function is called. The possible values for flag are
o 'init' — Initialization state
o 'iter' — Iteration state
o 'done' — Final state
Passing Extra Parameters in the Optimization Toolbox documentation explains how to
provide additional parameters to the output function.
The output function returns the following arguments:
 — Provides a way to stop the algorithm at the current iteration. stop can have the
stop
following values:
o false — The algorithm continues to the next iteration.
o true — The algorithm terminates at the current iteration.
 options — Options structure modified by the output function.
 optchanged — A boolean flag indicating changes were made to options. This must be set
to true if options are changed.
Display Options
Use the Display option to specify how much information is displayed at the command line
while the algorithm is running. The available options are
 off — No output is displayed. This is the default value for an options structure created
using optimoptions.
 iter — Information is displayed at each iteration.
 diagnose — Information is displayed at each iteration. In addition, the diagnostic lists some
problem information and the options that have been changed from the defaults.
 final — The reason for stopping is displayed. This is the default.
Both iter and diagnose display the following information:
 Iteration — Iteration number
 f-count — Cumulative number of objective function evaluations
 Best f(x) — Best objective function value
 Current f(x) — Current objective function value
 Mean Temperature — Mean temperature function value