1

Introduction

Optimization algorithms are becoming increasingly popular in multi-

engineering design activities, primarily because of the availability and
affordability of high speed computers. They are extensively used in those
engineering design problems where the emphasis is on maximizing or
minimizing a certain goal. For example, optimization algorithms are routinely
used in aerospace design activities to minimize the overall weight, simply
because every element or component adds to the overall weight of the
aircraft. Thus, the minimization of the weight of aircraft components is of
major concern to aerospace designers. Chemical engineers, on the other hand,
are interested in designing and operating a process plant for an optimum rate
of production. Mechanical engineers design mechanical components for the
purpose of achieving either a minimum manufacturing cost or a maximum
component life. Production engineers are interested in designing optimum
schedules of various machining operations to minimize the idle time of
machines and the overall job completion time. Civil engineers are involved
in designing buildings, bridges, dams, and other structures in order to achieve
a minimum overall cost or maximum safety or both. Electrical engineers are
interested in designing communication networks so as to achieve minimum
time for communication from one node to another.
All the above-mentioned tasks involve either minimization or maximiza-
tion (collectively known as optimization) of an objective. It is clear from the
spectrum of the above problems that it is difficult to discuss the formulation
of various engineering optimization problems in a single book. Fortunately,
a designer specialized in a particular design is usually more informed about
different factors governing that design than anyone else. Thus, as far as the
formulation of the optimal problem is concerned, the designer can acquire it
with some practice. However, every designer should know a few aspects of
the formulation procedure which would help him or her to choose a proper
optimization algorithm for the chosen optimal design problem. This requires

1
2 Optimization for Engineering Design: Algorithms and Examples

a knowledge about the working principles of different optimization methods.

In subsequent chapters, we discuss various optimization methods which would
hopefully provide some of that knowledge. In this chapter, we demonstrate the
optimal problem formulation procedures of four different engineering optimal
design problems.

1.1 Optimal Problem Formulation

In many industrial design activities, a naive optimal design is achieved by
comparing a few (limited up to ten or so) alternative design solutions created
by using a priori problem knowledge. In such an activity, the feasibility
of each design solution is first investigated. Thereafter, an estimate of the
underlying objective (cost, profit, etc.) of each design solution is computed
and the best design solution is adopted. This naive method is often followed
because of the time and resource limitations. But, in many cases this method
is followed simply because of the lack of knowledge of the existing optimization
procedures. But whatever may be the reason, the purpose of either achieving
a quality product or of achieving a competing product is never guaranteed
to be fulfilled with the above naive method. Optimization algorithms
described in this book provide systematic and efficient ways of creating
and comparing new design solutions in order to achieve an optimal design.
Since an optimization algorithm requires comparison of a number of design
solutions, it is usually time-consuming and computationally expensive. Thus,
the optimization procedure must only be used in those problems where there
is a definite need of achieving a quality product or a competitive product. It is
expected that the design solution obtained through an optimization procedure
is better than other solutions in terms of the chosen objective—cost, efficiency,
safety, or others.
We begin our discussion with the formulation procedure by mentioning
that it is almost impossible to apply a single formulation procedure for all
engineering design problems. Since the objective in a design problem and
the associated design parameters vary from product to product, different
techniques need to be used in different problems. The purpose of the
formulation procedure is to create a mathematical model of the optimal design
problem, which then can be solved using an optimization algorithm. Since
an optimization algorithm accepts an optimization problem in a particular
format, every optimal design problem must be formulated in that format. In
this section, we discuss different components of that format.
Figure 1.1 shows an outline of the steps usually involved in an optimal
design formulation process. As mentioned earlier, the first step is to realize
the need for using optimization in a specific design problem. Thereafter, the
designer needs to choose the important design variables associated with the
design problem. The formulation of optimal design problems involves other
considerations, such as constraints, objective function, and variable bounds.
As shown in the figure, there is usually a hierarchy in the optimal design
Introduction 3

process; although one consideration may get influenced by the other. We

discuss all these aspects in the following subsections.

Figure 1.1 A flowchart of the optimal design procedure.

1.1.1 Design Variables

The formulation of an optimization problem begins with identifying

the underlying design variables, which are primarily varied during the
optimization process. A design problem usually involves many design
parameters, of which some are highly sensitive to the proper working of
the design. These parameters are called design variables in the parlance of
optimization procedures. Other (not so important) design parameters usually
remain fixed or vary in relation to the design variables. There is no rigid
guideline to choose a priori the parameters which may be important in a
problem, because one parameter may be more important with respect to
minimizing the overall cost of the design, while it may be insignificant with
respect to maximizing the life of the product. Thus, the choice of the important
parameters in an optimization problem largely depends on the user. However,
it is important to understand that the efficiency and speed of optimization
algorithms depend, to a large extent, on the number of chosen design variables.
In subsequent chapters, we shall discuss certain algorithms which work very
efficiently when the number of design variables is small, but do not work that
well for a large number of design variables. Thus, by selectively choosing the
design variables, the efficacy of the optimization process can be increased. The
first thumb rule of the formulation of an optimization problem is to choose as
4 Optimization for Engineering Design: Algorithms and Examples

few design variables as possible. The outcome of that optimization procedure

may indicate whether to include more design variables in a revised formulation
or to replace some previously considered design variables with new design
variables.

1.1.2 Constraints
Having chosen the design variables, the next task is to identify the constraints
associated with the optimization problem. The constraints represent some
functional relationships among the design variables and other design
parameters satisfying certain physical phenomenon and certain resource
limitations. Some of these considerations require that the design remain in
static or dynamic equilibrium. In many mechanical and civil engineering
problems, the constraints are formulated to satisfy stress and deflection
limitations. Often, a component needs to be designed in such a way that
it can be placed inside a fixed housing, thereby restricting the size of the
component. There is, however, no unique way to formulate a constraint in
all problems. The nature and number of constraints to be included in the
formulation depend on the user. In many algorithms discussed in this book,
it is not necessary to have an explicit mathematical expression of a constraint;
but an algorithm or a mechanism to calculate the constraint is mandatory. For
example, a mechanical engineering component design problem may involve
a constraint to restrain the maximum stress developed anywhere in the
component to the strength of the material. In an irregular-shaped component,
there may not exist an exact mathematical expression for the maximum stress
developed in the component. A finite element simulation software may be
necessary to compute the maximum stress. But the simulation procedure and
the necessary input to the simulator and the output from the simulator must
be understood at this step.
There are usually two types of constraints that emerge from most
considerations. Either the constraints are of an inequality type or of an
equality type. Inequality constraints state that the functional relationships
among design variables are either greater than, smaller than, or equal to,
a resource value. For example, the stress (σ(x)) developed anywhere in a
component must be smaller than or equal to the allowable strength (Sallowable )
of the material. Mathematically,
σ(x) ≤ Sallowable .
Most of the constraints encountered in engineering design problems are of this
type. Some constraints may be of greater-than-equal-to type: for example,
the natural frequency (ν(x)) of a system may required to be greater than
2 Hz, or mathematically, ν(x) ≥ 2. Fortunately, one type of inequality
constraints can be transformed into the other type by multiplying both sides
by −1 or by interchanging the left and right sides. For example, the former
constraint can be transformed into a greater-than-equal-to type by either
−σ(x) ≥ −Sallowable or Sallowable ≥ σ(x).
Introduction 5

Equality constraints state that the functional relationships should exactly

match a resource value. For example, a constraint may require that the
deflection (δ(x)) of a point in the component must be exactly equal to 5 mm,
or mathematically,
δ(x) = 5.
Equality constraints are usually more difficult to handle and, therefore, need
to be avoided wherever possible. If the functional relationships of equality
constraints are simpler, it may be possible to reduce the number of design
variables by using the equality constraints. In such a case, the equality
constraints reduce the complexity of the problem, thereby making it easier
for the optimization algorithms to solve the problem. In Chapter 4, we
discuss a number of algorithms which are specially designed to handle equality
constraints. Fortunately, in many engineering design optimization problems,
it may be possible to relax an equality constraint by including two inequality
constraints. The above deflection equality constraint can be replaced by two
constraints:
δ(x) ≥ 4,

δ(x) ≤ 6.

The exact deflection requirement of 5 mm is relaxed by allowing it to deflect

anywhere between 4 mm to 6 mm. Although this formulation is inexact as
far as the original requirement is concerned, this flexibility allows a smoother
operation of the optimization algorithms. Thus, the second thumb rule in the
formulation of optimization problems is that the number of complex equality
constraints should be kept as low as possible.

1.1.3 Objective Function

The third task in the formulation procedure is to find the objective function
in terms of the design variables and other problem parameters. The common
engineering objectives involve minimization of overall cost of manufacturing,
or minimization of overall weight of a component, or maximization of net
profit earned, or maximization total life of a product, or others. Although
most of the above objectives can be quantified (expressed in a mathematical
form), there are some objectives that may not be quantified easily. For
example, the esthetic aspect of a design, ride characteristics of a car suspension
design, and reliability of a design are important objectives that one may be
interested in maximizing in a design, but the exact mathematical formulation
may not be possible. In such a case, usually an approximating mathematical
expression is used. Moreover, in any real-world optimization problem, there
could be more than one objective that the designer may want to optimize
simultaneously. Even though a few multi-objective optimization algorithms
exist in the literature (Chankong and Haimes, 1983), they are complex and
computationally expensive. Thus, in most optimal design problem, multiple
objectives are avoided. Instead, the designer chooses the most important
6 Optimization for Engineering Design: Algorithms and Examples

objective as the objective function of the optimization problem, and the

other objectives are included as constraints by restricting their values within
a certain range. For example, consider an optimal truss structure design
problem. The designer may be interested in minimizing the overall weight of
the structure and simultaneously be concerned in minimizing the deflection of
a specific point in the truss. In the optimal problem formulation, the designer
may like to use the weight of the truss (as a function of the cross-sections
of the members) as the objective function and have a constraint with the
deflection of the concerned point to be less than a specified limit. In general,
the objective function is not required to be expressed in a mathematical form.
A simulation package may be required to evaluate the objective function. But
whatever may be the way to evaluate the objective function, it must be clearly
understood.
The objective function can be of two types. Either the objective function
is to be maximized or it has to be minimized. Unfortunately, the optimization
algorithms are usually written either for minimization problems or for
maximization problems. Although in some algorithms, some minor structural
changes would enable to perform either minimization or maximization, this
requires extensive knowledge of the algorithm. Moreover, if an optimization
software is used for the simulation, the modified software needs to be
compiled before it can be used for the simulation. Fortunately, the duality
principle helps by allowing the same algorithm to be used for minimization
or maximization with a minor change in the objective function instead of
a change in the entire algorithm. If the algorithm is developed for solving a
minimization problem, it can also be used to solve a maximization problem by
simply multiplying the objective function by −1 and vice versa. For example,
consider the maximization of the single-variable function f (x) = x2 (1 − x)
shown by a solid line in Figure 1.2. The maximum point happens to be
at x∗ = 0.667. The duality principle suggests that the above problem is
equivalent to minimizing the function F (x) = −x2 (1 − x), which is shown by
a dashed line in Figure 1.2. The figure shows that the minimum point of the
function F (x) is also at x∗ = 0.667. Thus, the optimum solution remains the
same. But once we obtain the optimum solution by minimizing the function
F (x), we need to calculate the optimal function value of the original function
f (x) by multiplying F (x) by −1.

1.1.4 Variable Bounds

The final task of the formulation procedure is to set the minimum and the
maximum bounds on each design variable. Certain optimization algorithms
do not require this information. In these problems, the constraints completely
surround the feasible region. Other problems require this information in order
to confine the search algorithm within these bounds. In general, all N design
variables are restricted to lie within the minimum and the maximum bounds
as follows:
(L) (U )
xi ≤ xi ≤ xi for i = 1, 2, . . . , N .
Introduction 7

Figure 1.2 Illustration of the duality principle. The maximum point of f (x) is the
same as the minimum point of F (x).

(L)
In any given problem, the determination of the variables bounds xi and
(U )
xi may be difficult. One way to remedy this situation is to make a guess
about the optimal solution and set the minimum and maximum bounds so
that the optimal solution lies within these two bounds. After simulating the
optimization algorithm once, if the optimal solution is found to lie within
the chosen variable bounds, there is no problem. On the other hand, if any
design variable corresponding to the optimal solution is found to lie on or
near the minimum or the maximum bound, the chosen bound may not be
correct. The chosen bound may be readjusted and the optimization algorithm
may be simulated again. Although this strategy may seem to work only with
linear problems, it has been found useful in many real-world engineering
optimization problems.
After the above four tasks are completed, the optimization problem
can be mathematically written in a special format, known as nonlinear
programming (NLP) format. Denoting the design variables as a column
vector1 x = (x1 , x2 , . . . , xN )T , the objective function as a scalar quantity
f (x), J inequality constraints as gj (x) ≥ 0, and K equality constraints as

hk (x) = 0

1
The representation of the design variables in the above column vector helps
to achieve some matrix operations in certain multivariable optimization methods
described in Chapters 3 and 4.
8 Optimization for Engineering Design: Algorithms and Examples

we write the NLP problem:




Minimize f (x) 







subject to 


gj (x) ≥ 0, j = 1, 2, . . . , J; 
(1.1)




hk (x) = 0, k = 1, 2, . . . , K; 





(L) (U ) 

xi ≤ xi ≤ xi , i = 1, 2, . . . , N.

Note that the above formulation can represent a formulation for

maximization problems by using the duality principle and can represent
a formulation for problems with the lesser-than-equal-to type inequality
constraints by using the techniques described earlier. However, the
optimization algorithm used to solve the above NLP problem depends on
the type of the objective function and constraints. It is important to note
that the constraints must be written in a way so that the right-side of the
inequality or equality sign is zero.
It is worth mentioning here that all the above four tasks are not
independent of each other. While formulating the objective function, the
designer may decide to include or delete some constraints. In many problems,
while the constraints are being formulated, it is necessary to add some
artificial design variables, which make the overall formulation easier. The
update of the design variables, the constraints, the objective function, and
the variable bounds may continue for a few iterations until the designer is
finally satisfied with a reasonable formulation. Certain possible iterations are
shown in Figure 1.1. We may mention here that this update also depends on
the knowledge of the optimization algorithms to be used to solve the problem.
But this requires some practice of using optimization algorithms before such
input may be incorporated into the formulation procedure. Nevertheless, after
the optimization problem is formulated, an optimization algorithm is chosen
and an optimal solution of the NLP problem is obtained. We now illustrate
the above four steps of the formulation procedure in four engineering optimal
design problems.

1.2 Engineering Optimization Problems

Optimization problems can be found in most engineering disciplines. Because
of the variety of engineering design problems, it is not possible to discuss the
formulation of every optimization problem that is encountered in engineering
design in a single book. However, in this section, we categorize different
engineering problem-solving tasks in which an optimization method can be
applied to find a solution.