Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893 www.elsevier.

com/locate/cma

Response surface approximation of Pareto optimal front in multi-objective optimization

a,*

, Rajkumar Vaidyanathan a,1, Raphael T. Haftka a, Wei Shyy Nestor V. Queipo b, Kevin Tucker c

a,2

Department of Mechanical and Aerospace Engineering, University of Florida, P.O. Box 116250, Gainesville, FL 32611-6250, United States b Applied Computing Institute, Faculty of Engineering, University of Zulia, Venezuela c NASA Marshall Space Center, MS/TD64, MSFC, AL 35812, United States Received 31 March 2005; received in revised form 6 February 2006; accepted 6 July 2006

Abstract A systematic approach is presented to approximate the Pareto optimal front (POF) by a response surface approximation. The data for the POF is obtained by multi-objective evolutionary algorithm. Improvements to address drift in the POF are also presented. The approximated POF can help visualize and quantify trade-os among objectives to select compromise designs. The bounds of this approximate POF are obtained using multiple convex-hulls. The proposed approach is applied to study trade-os among objectives of a rocket injector design problem where performance and life objectives compete. The POF is approximated using a quintic polynomial. The compromise region quanties trade-os among objectives. 2006 Elsevier B.V. All rights reserved.
Keywords: Pareto optimal front; Response surface approximation; Multi-objective evolutionary algorithms; Rocket injector design; Pareto drift

1. Introduction Practical engineering design often involves multiple disciplines and lacks the benet of closed form analytical solutions. The design scope is frequently dened by multiple and sometimes conicting design objectives, along with a substantial number of design variables. Multi-objective optimization problems usually have many optimal solutions, known as Pareto optimal solutions [1]. Each Pareto optimal solution represents a dierent compromise among design objectives. Hence, the designer is interested in nding many Pareto optimal solutions in order to select a design compromise that suits his preference structure. There are a number of dierent methods available for solving multi-objective
Corresponding author. Tel.: +1 352 392 6780; fax: +1 352 392 7303. E-mail address: tusharg@u.edu (T. Goel). 1 Currently with General Motors, Bangalore, India. 2 Present address: Department of Aerospace Engineering, The University of Michigan, Ann Arbor, MI 48109, United Sates. 0045-7825/$ - see front matter 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.cma.2006.07.010
*

optimization problems. One popular approach is condensing multiple objectives into a single, composite objective function by methods like weighted sum, geometric mean, perturbation, Tchybeshev, minmax, goal programming, and physical programming [13]. Another approach is to optimize one objective while treating other objectives as constraints [4]. These approaches give one Pareto optimal solution in each simulation. There are numerous multi-objective evolutionary algorithms (MOEAs) [5] that can be made to nd multiple Pareto optimal solutions in a single simulation run [5]. Some of the latest ones include strength Pareto evolutionary algorithm (Zitzler and Thiele) [6], Pareto archived evolutionary strategies (PAES by Knowles and Corne) [7], elitist nondominated sorting genetic algorithm (NSGA-II by Deb et al.) [8], and controlled elitist non-dominated sorting genetic algorithm (Deb and Goel) [9]. Most of the popular evolutionary algorithms are based on the concept of Pareto dominance and involve a nite size of population at each generation [5]. Due to the nite

880

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893

size of the population, some of the good solutions make way for the other solutions. If the lost solutions are the Pareto optimal solutions, the loss may not be repaired and sub-Pareto optimal solutions are obtained as the nal solution. This problem is named here Pareto drift. A remedy for this problem is suggested in the form of maintaining an archive of the Pareto optimal solutions. To solve multi-objective optimization problems one can couple EAs with exact function evaluations or with surrogate models of computationally expensive function evaluators. With reference to the former scenario, Sasaki et al. [10,11] used EAs coupled with CFD analysis to design the supersonic wings for multiple objectives. Ma kinen et al. [12] used EAs coupled with CFD analysis to solve aircraft wing design problems. Deb and Goel [1315] used evolutionary algorithms and a posterior local search to obtain Pareto optimal solution set of simple mechanical component shapes. Ishibuchi et al. [16] used a combination of evolutionary algorithms and local search for nding Pareto optimal solutions in owshop scheduling test problems. Obayashi et al. [17] used EAs coupled with CFD to design supersonic wings. In all these works, evolutionary algorithms are directly coupled with exact evaluation of designs. For computationally expensive problems, direct coupling of function evaluators with EAs will be impractical because multi-objective evolutionary algorithms require many analyses. Therefore, surrogates such as response surface approximations are often adopted [1835]. Once such surrogate models are available, the computational burden of performing optimization and generating a multitude of trade-o solutions is substantially reduced. For example, Madsen et al. [18], Vaidyanathan et al. [19,20], Shyy et al. [21] and Papila et al. [22,23] used polynomial- or neural network-based surrogate models as design evaluators for the optimization of propulsion components such as turbulent ow diuser, supersonic turbine and swirl coaxial injector element. Dornberger et al. [24] used neural networks and polynomial response surfaces to approximate the design objectives and a modication of genetic algorithm to nd the Pareto optimal solutions for the design of turbine blades. Bramanti et al. [25] used neural network models to approximate the design objectives and then coupled these models with evolutionary algorithm to nd multiple trade-o solutions to electromagnetic problems. Wilson et al. [26] and Cappelleri et al. [27] used surrogate modeling (response surface approximations and kriging) for approximating the objectives while designing piezomorph actuators. Farina et al. [28,29] used evolutionary strategies along with multiquadrics interpolation-based response surface approximations to optimize the shape of electromagnetic components like C-core and magnetizer for single and multiple objectives. Emmerich et al. [30,31] proposed using local metamodel (Kriging approximation based on a few nearest neighbors) to evaluate objectives required for optimization of analytical test functions and an airfoil shape design problem using evolutionary algorithms. Ong et al. [32] used radial basis

functions to approximate the objective function and constraints and then used a combination of evolutionary algorithm and sequential quadratic programming to nd optimal solutions of an aircraft wing design problem with single objective. Recently, Nain and Deb [33] combined articial neural networks and evolutionary algorithms to reduce the computational cost while nding trade-o solutions for standard multi-objective optimization test problems. In a latest eort, Knowles and Hughes [34], and Knowles [35] combined a Gaussian process based global optimizer EGO with evolutionary multi-objective optimization algorithms to signicantly reduce the computational cost in optimization of analytical test problems. As previously stated, a major advantage of using surrogate models is that function evaluation becomes inexpensive, making it feasible to evaluate a large number of Pareto optimal designs (e.g. [26]). Once many Pareto optimal solutions are obtained, the Pareto optimal front (POF) can be represented by its own response surface and the domain of application can be identied. Though the designer has many Pareto optimal solutions, he still faces the problem of selecting the compromise solution. The response surface approximation of the Pareto optimal front would allow the designer to visualize and assess trade-os among the objectives, to explore compromise solutions, and to take decisions based on realistic goals. This paper presents a methodology to construct a response surface approximation of the Pareto optimal front based on surrogate models. The methodology is demonstrated using a four-objective single element gasgas injector design problem motivated by liquid-rocket injector applications. Major interests here are to probe the interactions and trade-os among objectives, assessing correlations and conicts among them. The objective function evaluations require the solution of a uid dynamics problem involving turbulence and combustion. The solution is obtained by numerically solving the NavierStokes equations, aided by the turbulence closure and chemical kinetic schemes. A pressure-based, nite dierence, NavierStokes solver, FDNS500-CVS [3638], is used in this study. Steady state is considered for all problems. The objectives of this paper can be summarized as follows: 1. demonstrate the construction of the Pareto optimal front response surface approximation (Pareto optimal front RSA), addressing the problem of its region of applicability in function space; 2. discuss the Pareto-drift problem and suggest a remedy in the context of dominance based evolutionary algorithms and 3. illustrate the objective function trade-os for liquidrocket injector design problem. The paper is organized as follows: Section 2 gives the general description of terms used in the paper. Section 3 briey describes the NSGA-II algorithm. Section 4

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893

881

discusses the problem of Pareto drift and proposes one possible remedy. A method for constructing a response surface approximation for the Pareto optimal front is outlined in Section 5 and illustrated with the help of a single element liquid-rocket injector design problem in Section 6. Finally, Section 7 summarizes the main conclusions of the paper. 2. Basic terminology This section denes some of the terms used in this paper. More precise denitions of the terms can be found in Refs. [1,5,39,40]. 2.1. Multi-objective optimization and Pareto optimality 2.1.1. Search space Search space or design space is the set of all possible combinations of the design variables. If all design variables are real, the design space is given as x 2 RN (N is the number of design variables). The feasible domain S is the region in design space where all constraints are satised. 2.1.2. Multi-objective optimization problem formulation Multi-objective optimization problem is formulated as Minimize Fx; where F fj : 8j 1; M ; x xi : 8i 1; N Subject to : Cx 6 0; where C cp : 8p 1; P ; Hx 0; where H hk : 8k 1; K : 1 2.1.3. Domination criteria A feasible design x(1) dominates another feasible design (2) x (denoted as x(1) < x(2)), if both of the following conditions are true: The design x(1) is no worse than x(2) in all objectives, i.e., fj (x(1)) k fj (x(2)) for all j = 1, 2, . . . , M objectives x1 x2 ) 8j 2 Mfj x1 fj x2 or 8j 2 Mfj x1 6 fj x2 : 2

2.1.5. Pareto optimal set All the designs x (x 2 S) which are non-dominated with respect to any other design in set S, comprise a set known as Pareto optimal set. 2.1.6. Pareto optimal front (POF) The function space representation of the Pareto optimal set is the Pareto optimal front. When there are two objectives, the Pareto optimal front is a curve, when there are three objectives, the Pareto optimal front is represented by a surface and if there are more than three objectives, it is represented by a hyper-surface. 2.2. Response surface approximation In this study, response surface approximations are employed in two aspects, namely, to generate surrogates of the computationally expensive simulation-based models and to approximate the Pareto optimal front by representing one objective in terms of others. A brief description of the response surface methodology [39] is given in Appendix A. 3. Elitist non-dominated sorting genetic algorithm (NSGA-II) One of the multi-objective evolutionary algorithms (MOEA) that has been eective in nding the Pareto optimal solutions is the elitist non-dominated sorting genetic algorithm (NSGA-II) developed by Deb et al. [8]. The algorithm is described as follows: 1. Randomly initialize population (designs in the variable space) of size npop. 2. Compute objectives and constraints for each design. 3. Rank the population using non-domination criteria (many individuals can have same rank and rank-1 is the best). 4. Compute crowding distance (this distance nds the relative closeness of a solution to other solutions in the function space and is used to dierentiate between the solutions on same rank). 5. Employ genetic operators selection, crossover and mutation to create intermediate population of size npop. 6. Evaluate objectives and constraints for this intermediate population. 7. Combine the two (parent and intermediate) populations, rank them and compute the crowding distance. 8. Select new population of npop best individuals based on the rank and crowding distance. 9. Go to step 3 and repeat till termination criteria is reached, which in the current study is chosen to be the number of generations. In this study, the real-coded version of NSGA-II was used that is, crossover and mutation operations were conducted in the real space rather than the binary space. For

The design x(1) is strictly better than x(2) in at least one objective, or fj (x(1)) < fj (x(2)) for at least one j 2 {1, 2, . . . , M} x1 < x2 ) ^j 2 Mfj x1 < fj x2 : 3

2.1.4. Non-dominated solutions [41] If two designs are compared, then the designs are nondominated with respect to each other if neither design dominates the other. A design x 2 S (S is the set of all feasible designs) is non-dominated with respect to a set A  S, if 9 =a 2 A:a < x. Such designs in function space are called non-dominated solutions. Moreover, any design x is Pareto optimal if x is non-dominated with respect to S.

882

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893

all simulations a tournament selection operator with tournament size of two was used. An extensive parametric study was conducted to select the parameters used in NSGA-II algorithm [42]. Based on the above mentioned studies, following parameters were set for the simulations: Population size (npop) 100 Generations 250 Crossover probability (Pcross) 1.00 Distribution parameter (for crossover) 20 Mutation probability (Pmut) 0.25 Distribution parameter (for mutation) 200

4. Pareto drift in MOEAs Most of the popular multi-objective evolutionary algorithms (MOEAs) are based on dominance criteria. These algorithms investigate the POF using a nite population size and implement diversity preserving mechanisms such as niching, clustering, crowding distances etc. [5, Chapters 56] to nd the complete POF. The solutions obtained at each generation are characterized into dominated and non-dominated solutions. These non-dominated solutions are non-dominated with respect to the current set of solutions and may include the Pareto optimal solutions as well as sub-optimal solutions. In non-elitist MOEAs, the genetic operators may destroy some of these solutions to explore the design space. Introducing elitism in MOEAs alleviates this problem to some extent, but when the number of non-dominated solutions in the combined population exceeds the population size, as happens commonly in elitist MOEAs, some of the non-dominated solutions have to be dropped. If the solution thus lost is Pareto optimal, it may not be recovered during the course of the optimization and a suboptimal solution can appear to be a non-dominated solution. This problem of losing Pareto optimal solutions is dened as Pareto drift. Fig. 1 shows one instance of this behavior for a twoobjective optimization problem while using NSGA-II [8]. All the solutions evaluated so far are shown by the dots and the non-dominated solutions at the nal generation of a NSGA-II simulation are shown by asterisks. Most of the solutions obtained after the nal generation of a NSGA-II simulation are apparently non-dominated with respect to all the evaluated solutions, but a few solutions in the population at the nal generation were dominated. This is clear in the zoomed view in Fig. 2, where most of the nal generation solutions were dominated by the solutions evaluated during the search. This problem is common to all dominance based methods. One remedy to this problem is to maintain and continuously update an (unbounded-sized) archive of all the non-dominated solutions obtained so far, similar to some evolutionary strategies (e.g., PAES [7]) which maintain a bounded-sized archive. The NSGA-II algorithm is augmented with the archiving strategy and referred as

Fig. 1. Demonstration of Pareto-drift problem: solutions at last generation of NSGA-II simulation (reported Pareto optimal front) vs. all evaluated solutions. (For zoom in region, see Fig. 2.)

Fig. 2. Pareto-drift problem: loss of Pareto optimal solutions during NSGA-II simulations (zoomed in view). Full Pareto optimal front is shown in Fig. 1.

archiving NSGA-II (NSGA-IIa). The implementation can be summarized as follows: 1. Initialize an archive with all the non-dominated solutions after Step 3 in the NSGA-II algorithm. 2. After Step 7 in the NSGA-II algorithm, update the archive as follows: Compare archive solutions with rank-1 solutions in the combined population. Remove all dominated solutions from the archive. Add all rank-1 solutions in the current population which are non-dominated with respect to the archive. The nal Pareto optimal set is the archive solutions set. This archiving of the Pareto optimal solutions retains the

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893
Optimization problem Min F={f1, f2,,fm} fi = f(X) s. t. C(X) 0 H(X) = 0

883

Evaluate F, H, C fm = f(x1, x2,, xn) cp = c(x1, x2,, xn) hk = h(x1, x2,, xn)

x1 x2 xn

Response surface method

F H C

Global Pareto optimal response surface

Perform correlation check

F* = F*(f1, f2, , fm)

f2
f3

f2 f1 f3 f1

Fig. 3. Flowchart of proposed method of constructing Pareto optimal front response surface approximation.

information of the good solutions obtained so far and improves the convergence to the Pareto optimal front. On the negative side, the time and memory requirements increase substantially due to continuous increase in the archive size and the need to compare all the archive solutions with the current generation non-dominated solutions. 5. Generating the Pareto optimal front response surface approximation Fig. 3 illustrates the methodology used for generating the Pareto optimal front RSA. The starting point is the identication of design variables and their allowable ranges, performance criteria and constraints. Once the problem is dened, the designs are evaluated (objectives and constraints) through an experiment or numerical simulation as required by the problem. Because the cost of design evaluation is often very high, computationally inexpensive surrogate models are developed for both objectives and constraints. Polynomial response surface approximations were used in the context of this work. On the other hand, for multi-objective problem, identifying correlated objectives may help reduce the complexity of the design problem. Highly correlated objectives may be dropped3 or a representative objective can be used for all the correlated objectives (principal component analysis) hence reducing the dimensionality of the problem [42]. After dening the multi-objective optimization problem, the Pareto optimal solutions are generated. In this paper, a MOEA based hybrid method (NSGA-IIa + e-constraint strategy [4], Appendix B) [42] is suggested as the multiobjective optimizer to generate Pareto optimal solutions. The Pareto optimal front is then approximated by a response surface t to the available Pareto optimal solutions. Since the quality of the response surface approximaOnly the objectives which are strongly correlated to another should be dropped to reduce the dimensionality. If the objectives are weakly correlated, dropping objectives will cause loss of information.
3

tion depends on the number of data points, it is imperative to generate a large number of the Pareto optimal solutions to ascertain good quality. The Pareto optimal front RSA represents one objective as a function of the other objectives, so it is important to choose this objective judiciously. Also note that the equation developed for the Pareto optimal front RSA is valid only in a limited region of function space. To identify the region where this response surface represents the Pareto optimal front, convex hull(s) is (are) tted to the approximating data [44]. When the response surface approximation to Pareto optimal solutions results in non-convex Pareto optimal front, identication of the boundary of the Pareto optimal front RSA using a single convex hull, may not be appropriate. Then it is more apt to t several convex hulls to the subsets of Pareto optimal solutions such that the non-convex boundary of the Pareto optimal front RSA is properly approximated. The multiple subsets (clusters) of the Pareto optimal set can be eectively identied using clustering. Details of the particular clustering method used here are given in Appendix C. It is important to note that a sucient number of subsets (clusters) should be selected to adequately capture the non-convex boundary. This Pareto optimal front RSA shows the interactions of the dierent objectives for the global optimal values and helps understand the physics of the problem. Once the close form solution is available, the designer can visualize and assess trade-os among dierent objectives. This information can be used to rene the utility functions (importance associated with dierent objectives) to come up with more useful and practical designs. 6. An application: liquid-rocket single element injector design The proposed methodology was applied to design a single element injector of liquid-rocket engine. A schematic diagram of the hybrid Boeing element injector under consideration is shown in Fig. 4.

884

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893

Fig. 4. Schematic of hybrid Boeing element injector (US patent 6,253,539) and design variables.

6.1. Problem modeling The injector design has two primary objectives: improvement of performance and life. As discussed by Vaidyanathan et al. [20], the performance of the injector is indicated by the axial length of the thrust chamber, while the survivability of the injector is associated with the thermal eld inside the thrust chamber. A visual representation of the objectives is shown in Fig. 5. High temperatures induce high thermal stresses on the injector and the thrust chamber and thus reduce the life of the components but improve the performance of the injector. Consequently, the objectives under consideration are: 1. Combustion length (Xcc) is the distance from the inlet, where 99% of the combustion is complete. It is desirable to keep the combustion length as small as possible as this directly aects the size and eciency of the combustor. 2. Face temperature (TFmax) is the maximum temperature of the injector face. It is desirable to reduce temperature to increase the life of the injector. 3. Wall temperature (TW4) is the wall temperature at 3 in. (fourth probe) from the injector face. Higher values of the wall temperature reduce the life of the injector, so this objective is minimized. 4. Tip temperature (TTmax) is the maximum temperature on the post tip of the injector. It is desirable to keep this temperature low to maximize life.

It can be seen that the dual goal of maximizing the performance and the life is now cast as a four-objective design problem. As discussed by Vaidyanathan et al. [20] these objectives pose dierent and some times conicting requirements on the design scenarios, hence there cannot be a single optimal solution for this problem. There are four design variables for the injector design problem shown in Fig. 4. These design variables, their baseline values, and ranges are given as follows. 1. Hydrogen ow angle (a) the maximum angle varies between 0 to 20. The baseline hydrogen ow angle is 10. 2. Hydrogen area (DHA) the increment with respect to the baseline cross-section area (0.0186 in2) of the tube carrying hydrogen. The increment varies from 0% to 25% of the baseline hydrogen area. 3. Oxygen area (DOA) the decrement with respect to the baseline cross-section area (0.0423 in2) of the tube carrying oxygen. The area varies between 0% and (40)% of the baseline area. 4. Oxidizer post tip thickness (OPTT) varies between X00 to 2X00 . The baseline value of tip thickness X00 is 0.01 in. All the variables were linearly normalized between 0 and 1. More information about the design variables can be found in the work by Vaidyanathan et al. [20]. The boundary conditions applied in all CFD simulations are as follows. Both fuel and oxidizer ow in, through the inlet (west) boundary where the mass ow rate is xed for both streams. The nozzle exit, at the east boundary, is modeled by an outlet boundary condition. The south boundary is modeled with the symmetry condition. All walls (both sides of the oxygen post, the outside of the fuel annulus, the outside chamber wall, and the faceplate) are modeled with the no slip adiabatic wall boundary condition. This is a computationally expensive simulation-based problem so for optimization purposes it is advisable to develop surrogate models for the objective functions. It was shown by Vaidyanathan et al. [19,20], Shyy et al. [21], and Papila et al. [22,23] that accurate response surfaces for complex problems like single element injectors can be developed.

Fig. 5. Performance measures in single element liquid-rocket injector.

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893

885

6.2. Response surface approximation of objective functions For constructing the surrogate models, Vaidyanathan et al. [20] conducted a design of experiments (orthogonal arrays [46]) to pick the design data points. CFD simulations were carried out for 54 data points. Simulations for two designs failed to produce valid results. Response surfaces for all the objectives but TTmax were approximated in normalized variable space using 38 design data points and tested using the remaining 14 designs. For the objective TTmax all 52 designs were used to t the response surface. The response surfaces were t using standard least-squares regression with a quadratic polynomial using JMP [47]. A reduced cubic response surface was tted to the objective TTmax and this was cross-validated using PRESS [39]. Vaidyanathan et al. [20] obtained the following relations between design variables and objective functions TFmax 0:692 0:477a 0:687DHA 0:080DOA 0:0650OPTT 0:167a 0:0129DHAa 0:0796DHA 0:0634DOAa 0:0257DOADHA 0:0877DOA2 0:0521OPTTa 0:00156OPTTDHA 0:00198OPTTDOA 0:0184OPTT ; 4 X cc 0:153 0:322a 0:396DHA 0:424DOA 0:0226OPTT 0:175a2 0:0185DHAa 0:0701DHA 0:251DOAa 0:179DOADHA 0:0150DOA 0:0134OPTTa 0:0296OPTTDHA 5 0:0752OPTTDOA 0:0192OPTT ; TW4 0:758 0:358a 0:807DHA 0:0925DOA 0:0468OPTT 0:172a2 0:0106DHAa 0:0697DHA2 0:146DOAa 0:0416DOADHA 0:102DOA 0:0694OPTTa 0:00503OPTTDHA 0:0151OPTTDOA 0:0173OPTT2 ;
2 2 2 2 2 2 2 2

Table 1 Accuracy of response surface approximation of objectives (refer to Appendix A for denition of accuracy measures) TFmax # of observations R2 adj ra Mean r (14 pts) PRESS 38 1.000 0.00566 0.495 0.00460 Xcc 38 0.995 0.0205 0.497 0.0178 TW4 38 1.000 0.00803 0.514 0.00669 TTmax 52 0.989 0.0303 0.591 0.0388

The quality of the response surface approximations is given in Table 1 [20]. The response surfaces for all the objectives had very high adjusted coecient of multiple determination which indicate good prediction capabilities. The rms error on the training and the test data points for response surfaces tted to objectives TFmax and TW4 were very low while the errors for the objectives Xcc and TTmax (PRESS error instead of rms error) were relatively higher but the values were reasonable. For the chosen application, side constraints do not require any modeling; however, in a general case when closed form solution of objectives and constraints is not available, one can develop surrogate models for both objectives and constraints and use these surrogates in optimization as was done by Mack et al. [48] for design of radial turbine used in liquid-rocket engine. 6.3. Dene multi-objective optimization problem Using correlation analysis, Goel et al. [42] found that the objectives TFmax and TW4 were strongly correlated in the design space (Appendix D). Hence, TW4 was dropped from the objectives list4 and a multi-objective optimization problem was formulated with the remaining three objectives TFmax, Xcc, and TTmax (Eqs. (4), (5) and (7), respectively). The constraints on this problem were simple bounds on the variables (between 0 and 1). 6.4. Generate Pareto optimal solutions The three-objective optimization problem was solved using NSGA-IIa and e-constraint strategy. At the end of a NSGA-IIa simulation, the total number of optimal solutions in the archive was 5724 and the population at the nal generation had 100 non-dominated solutions. For this problem, the archive corresponds to less than 350 KB of memory (using double precision type for all variables and functions) which is modest for current computational capabilities. This indicates that the size of archive may not be a critical issue in terms of memory requirements for complex problems with more objectives, variables and constraints. Future improvements in computer hardware will further
Choice of dropping TW4 instead of TFmax was arbitrary since the correlation was very strong. Principal component analysis to identify three orthogonal vectors would have been more appropriate.
4

TTmax 0:370 0:205a 0:0307DHA 0:108DOA 1:019OPTT 0:135a 0:0141DHAa 0:0998DHA2 0:208DOAa 0:0301DOADHA 0:226DOA2 0:353OPTTa 0:0497OPTTDOA 0:423OPTT2 0:202DHAa2 0:281DOAa2 0:342DHA2 a 0:245DHA DOA 0:281DOA DHA 0:184OPTT a 0:281DHAaDOA: 7
2 2 2

886

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893

render this issue of memory size trivial. The computational expense of comparing and replacing non-dominating solutions in the archive can be reduced by implementing ecient search algorithms. A comparison of the solutions in archive with the solutions at the nal iteration is shown in Fig. 6. The majority (64 out of 100) of the non-dominated solutions at nal iteration were dominated by the optimal solutions in the archive. To improve robustness of the results, the NSGAIIa simulations were repeated with ten random initial seeds. Results from the NSGA-IIa and the NSGA-II were compared for each simulation. It was observed that on an average 63 out of 100 (range = [53, 68], standard deviation = 5) non-dominated solutions in the NSGA-II nal iteration were dominated by the corresponding archive solutions. This demonstrates the eectiveness of the archiving strategy in preventing the loss of potential Pareto optimal solutions. To further improve the convergence to the Pareto optimal front, the archives from the ten NSGA-IIa simulations were combined. The dominated and duplicate designs were removed from the combined archive to get 29,650 candidate optimal designs. To assess the improvements by using multiple simulations, each archive from the NSGA-IIa simulation was compared with the combined archive. It was observed that on an average 2346 solutions

Fig. 6. Comparison of the solutions in the archive with the solutions at nal iteration (it demonstrates that the archive alleviates the Pareto drift). This result is shown for one representative NSGA-II/a simulation.

(range = [2169, 2620], standard deviation = 137) from the individual archives were dominated by the combined archive of the solutions. Goel et al. [42] demonstrated that local search using econstraint strategy can further improve the solutions obtained using MOEA. Following the procedure outlined in Appendix B, 118,600 solutions were obtained in a combined pool of solutions from local search and the NSGA-IIa simulations. There were 88,553 non-dominated solutions. After removing the duplicates, 87,149 Pareto optimal solutions were found. This nal solution set dominated 29,483 solutions out of 29,650 solutions from the combined pool of the solutions obtained after multiple NSGA-IIa simulations. The results manifest the improvements using the hybrid approach of using MOEA and local search [42]. The improvements in the solutions quality (given as the distance between the dominated solution and the solution which dominates this solution) from each step (archiving, multiple simulations and hybridization) are quantied by computing average and maximum improvements and are tabulated in Table 2. The results of comparison between the NSGA-II and the NSGA-IIa (archiving), and between individual archive and combined archive (multiple simulations) are averaged over 10 simulations. For all steps, the average improvements were much smaller than the maximum improvements. The eect of subsequent steps on convergence to the Pareto optimal front (the average and maximum improvements in the solution quality) reduced which demonstrated the convergence to the Pareto optimal front. Relatively smaller reduction in maximum improvements demonstrates that dierent steps are eective in converging the far-from-optimal solutions to Pareto optimal front. To elucidate the impact of this three step method to achieve converged Pareto optimal solution, a simple example is presented as follows. Suppose a designer decides to give equal importance to all objectives (typical preference structure), the objective function f to minimize is TFmax + Xcc + TTmax. If she uses only NSGA-II simulations results, the best solution obtained is f = 0.8106 (TFmax = 0.3790, Xcc = 0.3431, TTmax = 0.0885). After implementing the archiving strategy, her best result is f = 0.7692 (TFmax = 0.3811, Xcc = 0.3518, TTmax = 0.0363). With multiple simulations her result improves to f = 0.7670 (TFmax = 0.3819, Xcc = 0.3503,

Table 2 Average and maximum improvements in solution quality (distance between dominated solution and the solution which dominates this solution) by (a) archiving, (b) multiple simulations and (c) hybridization (MOEA + local search) Average improvement Mean* Archiving Multiple simulations Hybridization
*Based

Maximum improvement Min 0.090 0.032 Max 0.249 0.083 Mean* 0.560 0.466 0.324 Std dev.* 0.073 0.041 Min 0.436 0.415 Max 0.690 0.527

Std

dev.*

0.156 0.050 0.008

0.039 0.015

on 10 simulations.

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893

887

TTmax = 0.0348) and after hybridization, her nal solution is f = 0.7667 (TFmax = 0.3817, Xcc = 0.3504, TTmax = 0.0346). Solving this single objective optimization problem using sequential quadratic programming gives the same result as the nal solution. 6.5. Representative designs The procedure discussed previously resulted in a Pareto optimal solution set with 87,149 solutions. In order to illustrate alternative representative design concepts, a hierarchical clustering algorithm (given in Appendix C) was used in the function space. Nine compromise solutions were ` -vis Pareto optimal solution selected. These solutions vis-a set are shown in Fig. 7. Fig. 7 shows that the representative solutions were uniformly selected from the Pareto optimal front. It was observed that small values of the face temperature TFmax in general were accompanied by longer combustion length Xcc and higher tip temperature TTmax. However, a small compromise in the face temperature can substantially reduce the combustion length and tip temperature. Also it was observed that the combustion length can be substantially reduced by allowing a high thermal environment in the combustion chamber. Selected designs represent these distinct regions. Objective function values and design parameter values for the nine representative designs are given in Table 3. Some interesting results about the physics of the problem were observed from Table 3. In general, the smaller values of the ow angle yield low face temperature, but higher temperature on the injector tip and longer combustion length. Increase in the ow angle of the injector causes an increase in the temperature of the face of injector and a reduction in the length of the combustor. It is also seen that if the cross-section area of the oxygen tube is reduced, for low values of ow angle and high value of cross-section area of hydrogen tube, the temperature at the face increases and the combustion length reduces. There is a strong eect of cross-interactions among dierent variables, which

Table 3 Objective functions and design variables for nine representative Pareto optimal designs (refer to Figs. 4 and 5) a 0.000 0.000 0.000 0.017 0.246 0.624 0.300 0.472 1.000 DHA 1.000 1.000 1.000 1.000 1.000 0.665 0.104 0.090 0.018 DOA 0.629 0.202 0.308 0.897 0.163 0.000 0.000 0.000 0.562 OPTT 0.803 0.669 0.281 0.000 0.000 0.000 0.260 0.000 0.000 TFmax 0.015 0.033 0.044 0.067 0.171 0.498 0.730 0.818 0.936 Xcc 0.957 0.655 0.689 1.023 0.503 0.260 0.122 0.076 0.117 TTmax 0.937 0.955 0.703 0.393 0.365 0.218 0.562 0.250 0.013

impact the optimal design. This does not allow drawing conclusion about the eect of individual parameters. A more detailed analysis of the results is given by Vaidyanathan et al. [49]. 6.6. Pareto optimal front response surface approximation Visualization of the Pareto optimal front in threedimensions is cumbersome and only qualitative inferences about dierent design domains and function trade-os can be made. This problem is alleviated by approximating the Pareto optimal front with a polynomial-based response surface. All 87,149 Pareto optimal solutions were used to t the response surface. It was unknown which of the objectives can be represented as a function of the remaining objectives more accurately. Hence, each objective function was represented in terms of the other two objective functions considering dierent order polynomials. Since the cost of tting a polynomial response surface model is very small compared to the cost of identifying Pareto optimal solutions, it is recommended to try all (Nobj) combinations before selecting the nal form of relationship between objectives. The quality indicators of dierent response surface approximations are presented in Table 4. Several observations can be made from Table 4. First, choosing the proper objective to be represented as a function of the remaining two objectives inuences the quality of the approximation substantially. In this case, TFmax = TFmax(Xcc, TTmax) gave the best quality of the response surface and TTmax = TTmax(TFmax, Xcc) gave the worst results. Second, as expected, increasing the order of the polynomial gives a better t. With increasing order of the polynomial, R2 adj increased and rms error and number of points with high errors (>0.1) reduced. The maximum error for Xcc = Xcc(TFmax, TTmax) reduced with increase in the order of polynomial but the eect was unclear for the remaining two approximations. Finally, a quintic response surface approximation of TFmax = TFmax(Xcc, TTmax) was selected as Pareto optimal front RSA. The R2 adj was very good. The Pareto optimal front RSA ts the data very well. The coefcients of the Pareto optimal front RSA and the corresponding t-statistics are given in Table 5. High values of

Fig. 7. Nine representative trade-o solutions obtained using clustering.

888

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893

Table 4 Accuracy of response surface approximations of the 87,149 Pareto optimal solutions (refer to Appendix A for denition of accuracy measures) TFmax = f(Xcc, TTmax) Quadratic Mean response R2 adj RMS error Max error # of points with error > 0.1 R2 adj RMS error Max error # of points with error > 0.1 R2 adj RMS error Max error # of points with error > 0.1 R2 adj RMS error Max error # of points with error > 0.1 0.475 0.977 0.0560 0.236 5512 0.984 0.0462 0.210 4001 0.991 0.0349 0.245 1974 0.993 0.0316 0.262 1304 Xcc = f(TFmax, TTmax) 0.357 0.883 0.0980 0.400 25,840 0.926 0.0783 0.356 18,762 0.969 0.0506 0.300 4964 0.983 0.0371 0.280 1806 TTmax = f(TFmax, Xcc) 0.369 0.754 0.161 0.452 55,157 0.899 0.103 0.298 27,287 0.937 0.0815 0.555 17,712 0.956 0.0679 0.273 11,967

Cubic

Quartic

Quintic

t-statistics for all the terms showed statistical signicance of the coecients. The contour plot of the Pareto optimal front RSA TFmax = TFmax(Xcc, TTmax) along with the Pareto optimal solutions illustrates its application domain in Fig. 8. It is important to note that Table 5 represents the Pareto optimal front for limited ranges of Xcc and TTmax. Only the regions bounded by the Pareto optimal solutions shown

Table 5 Coecients of response surface representing Pareto optimal front TFmax = TFmax(Xcc, TTmax) Term Intercept Xcc TTmax X2 cc XccTTmax TT2 max X3 cc X2 cc TTmax X cc TT2 max TT3 max X4 cc X3 cc TTmax 2 X2 cc TTmax X cc TT3 max TT4 max X5 cc X4 cc TTmax 2 X3 cc TTmax 3 X2 cc TTmax X cc TT4 max TT5 max Coecient 1.03 0.27 1.09 13.34 0.62 2.53 33.41 2.99 4.47 0.95 30.74 2.38 2.68 8.46 2.17 10.84 5.57 14.32 12.16 1.76 0.58 t-ratio 1632.7 23.5 127.1 166.6 68.15 10.07 130.4 12.2 10.84 23.22 90.15 5.22 7.09 23.77 16.6 79.23 13.08 16.53 14.29 3.75 5.33

Fig. 8. Contour plot of the Pareto optimal front response surface approximation TFmax = TFmax(Xcc, TTmax) with Pareto optimal solutions.

as dots in Fig. 8 represent the Pareto optimal front. This means that the zones with extreme values of the objective TFmax (>1.00 or <0.00) are not part of the Pareto optimal front. The analysis of the response surface contours tells the designer that TFmax decreases with increase in Xcc and TTmax. For every value of Xcc an optimal value of TTmax can be found such that TFmax is minimized and vice-versa. For Xcc < 0.40, increase in Xcc is more eective in reducing TFmax as compared to the increase in TTmax. This trend is reversed in the region Xcc > 0.40. The insight gained in the present investigation has further elucidated the interactions of the objectives in specic ranges, besides those observed by Vaidyanathan et al. [20]. It is also evident from Fig. 8 that for a good compromise among all the objectives, the values of normalized Xcc should be between 0.350.60 and TTmax should be between

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893

889

0.00 and 0.30 of its normalized value. For this range of values, TFmax varies between 0.10 and 0.40 of its normalized value. This information is very useful for a designer, in selecting the trade-os among all the objectives or improving the already existent preference structure. For example, the optimization with equal importance to all the objectives had led to a design with TFmax = 0.3817, Xcc = 0.3504, TTmax = 0.0346. But following the information about the Pareto optimal front RSA, he may allow an increase in objective TTmax to reduce the other two objectives. Once the close form relation is available to the designer relating dierent objectives, a sensitivity analysis can be done to nd a more exact quantication of the trade-os for the selected design. At the optimal design when all objectives have equal importance, the derivatives using the response surface are given as oTFmax/ oXcc = 1.26; oTFmax/oTTmax = 1.04. This indicates that increasing Xcc by a unit amount keeping TTmax constant is more (21%) eective in reducing TFmax than reducing TTmax while keeping Xcc constant. However, since objectives Xcc and TTmax are not independent, one must ensure that changes in the design objectives do not violate the boundary of the Pareto optimal front. The boundary of the application regions of this nonconvex response surface was identied using multiple convex hulls of the Pareto optimal solutions. One convex hull was identied in each cluster of points. The convex hull identies the set of points on the boundary such that all the points in the cluster are within the bounded region. For this problem, convex hulls using 9 clusters were not sucient to represent the boundary of the Pareto optimal response surface. A signicant non-Pareto optimal region was bounded by the convex hulls. Hence convex hulls were tted to the data from 30 clusters. These 30 clusters, centroids of the clusters and the corresponding bounding con-

Table 6 Data points in function space dening the boundary of one representative convex hull Xcc 0.093 0.108 0.098 0.069 0.066 0.055 0.055 0.055 0.073 0.093 TTmax 0.937 0.937 1.035 1.094 1.094 0.951 0.942 0.940 0.938 0.937 TFmax 0.387 0.380 0.384 0.388 0.389 0.405 0.406 0.406 0.397 0.387

vex hulls are shown on the contour plot of the response surface in Fig. 9. The points dening the boundary of one representative convex hull are given in Table 6. The boundary of the response surface is identied adequately with the convex hulls in most of the regions, but there were some small regions where the convex hulls bounded nonPareto optimal front. The non-Pareto optimal front region can be further reduced by using a large number of clusters; however there is a trade-o between the number of clusters (and convex hulls) and the non-Pareto optimal front region for non-convex problems. In order to select a combination of Xcc and TTmax, where the Pareto optimal front RSA is valid, the selected point must be bounded by at least one convex hull. 6.7. Comparison of 3-objective vs. 4-objective optimization problem results To verify the eect of reducing the objectives, a single NSGA-IIa simulation based hybrid strategy was used to solve optimization problems with all 4-objectives and 3objectives, which resulted in 18,812 and 16,467 unique non-dominated solutions. These two non-dominated solution sets were compared. Most of the solutions obtained were non-dominated with respect to one another. Four solutions from the 4-objective NSGA-IIa simulation results were dominated by solutions obtained for the 3-objective optimization, and eight solutions obtained from the 3objective simulation were dominated by the 4-objective simulation results. The number of dominated solutions for both populations was very small when the total number of solutions is considered. The diversity of the solutions in objective function space was also comparable for the two cases. This veries that there were no adverse eects introduced by reducing the number of objectives. 7. Concluding remarks The Pareto optimal front in multi-objective optimization problems is useful to visualize and assess trade-os among dierent design objectives. In addition to identify compromise solutions, this also helps the designer set realistic design goals. In this paper, a systematic approach for

Fig. 9. Convex-hulls to bound the domain where the Pareto optimal front RSA TFmax = TFmax(Xcc, TTmax) is valid. Dots show the centers of the clusters for each convex hull and thick lines show the boundary of each cluster.

890

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893

providing an approximate closed form solution to the Pareto optimal front and associated issues are presented. This approach comprise of the following steps: (i) evaluation of design by direct simulation or surrogate models, (ii) determination of the Pareto optimal solutions by multi-objective optimizer, (iii) approximation of Pareto optimal front using response surface and (iv) approximation of the boundary using convex-hull(s) of the Pareto optimal solutions data. The selection of the objective to be represented as a function of the other objectives has a great bearing on the accuracy of t of the Pareto optimal front RSA and hence should be carefully selected by trying the dierent possibilities. The Pareto optimal front RSA does not represent the Pareto front for all combinations of the objectives so the relevant domain should be properly bounded. Convex hulls t to the Pareto optimal solutions data can be used to locate the boundary of the domain where the response surface represents the Pareto optimal front. For a non-convex Pareto optimal front, the bounds should be identied using convex hulls tted to the multiple clusters of the Pareto optimal solutions. The centers of the clusters characterize alternative optimal design concepts which can be selected for further assessment. The problem of Pareto drift (losing Pareto optimal solutions during course of optimization) in the dominance based MOEAs is identied. Implementation of an archiving strategy to preserve all good solutions is suggested as a remedy. Specically, NSGA-II algorithm is modied to implement the archiving strategy and the improvements in the quality of the solutions and convergence to Pareto optimal front were demonstrated. On the negative side, the computational cost of nding the Pareto optimal solutions has increased due to continuous update of the archive. The proposed approach is exemplied using a fourobjective liquid-rocket single-element injector design problem. The response surface method was employed in two aspects. First, response surfaces based on the CFD solutions were used to evaluate the design objectives [20]. This response surface related the design variables and the objectives. Second, the Pareto optimal front was approximated by response surface. This response surface represented one objective in terms of others objectives. The complexity of the multi-objective optimization problem was reduced by removing one (TW4) of the two correlated objectives (TFmax and TW4). The resulting multi-objective optimization problem was solved using a combination of NSGAIIa and e-constraint strategy. The solution of the multiobjective optimization problem was the Pareto optimal solutions data which represented a non-convex Pareto front. The Pareto optimal front was approximated by a quintic response surface TFmax = TFmax(Xcc, TTmax). The domain of application of this Pareto optimal front RSA was bounded by convex hulls tted to multiple clusters of the Pareto optimal solutions data. The analysis of Pareto optimal solutions revealed important information about the eect of design variables on the

objectives. For example, smaller values of the ow angle yields low face temperature, but higher temperature on injector tip and longer combustor length. Interactions among dierent objectives as identied by the Pareto optimal response surface manifested the importance of each design objective in dierent regions of the function space. For example, the face temperature can be reduced more eectively by allowing small increase in combustion length for small combustors, whereas allowing small rise in tip temperature is more eective for moderate length combustors. The Pareto optimal response surface also helped in the visualization of trade-os among designs. It was observed that a good compromise region with equal importance to all objectives, will have 0.35 < Xcc < 0.60, 0.00 < TTmax < 0.30 and 0.10 < TFmax < 0.40. In summary, the proposed systematic approach was found useful when solving a complex multi-objective liquid-rocket injector design problem and holds promise to be eective in broader contexts. Acknowledgement The present eort has been supported by the NASA Constellation University Institute Program (CUIP), Ms. Claudia Meyer program monitor. Appendix A. Response surface methodology [39] The response surface method ts a function to a set of experimentally or numerically evaluated design data points. There are various response surface approximation methods available in the literature [39], with polynomialbased approximations being the most popular among practitioners. In this technique, an appropriate order polynomial is tted to a set of data points, such that the adjusted rms (root mean square) error ra [39] is minimized. The adjusted rms error ra [39] is dened as following: Let Np be the number of data points, Nc be the number of coecients, and error ei at any design point i being dened as yi; ei y i ^ 8

where yi is the actual value of the function at the design point and y i is the predicted value. Hence v , u N uX ra t e2 9 N p N c : i
i 1

If Nt additional test data are used to test the quality of the approximation, the rms error r is given as v u M , uX rt 10 e2 N t: i
i 1

Prediction capability of the response surface is given by the coecient of multiple determination R2 adj dened as,

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893

891

R2 adj

1 r2 a N p

,N p X
i1

y i  y where  y

Np X i1

, yi N p: 11

1. For any candidate optimal solution n, formulate constrained single objective optimization problem Minimize Subject to: Cx 6 0; Hx 0; f m x 6 em where C cp : 8p 1; P ; where H hk : 8k 1; K ;
n 8m 1; M ; m 6 j; em fm :

f j x;

where x xi : 8i 1; N

For a good t, should be closer to 1. When the number of data points is not adequate enough to spare enough data for testing the response surface (RS), PRESS statistic is used to estimate the performance of the RS. A residual is obtained by tting a RS over the design space after dropping one design point from the training set and then comparing the RS predicted value for that point with the expected value. PRESS is the square sum of the residuals and is given by v , u Np uX t 2 PRESS y i ^ yi 12 Np
i1

R2 a

13 Note that the last constraint was not present in the original problem. 2. Solve the constrained optimization problem given in Eq. (13) for "j = 1, M using current design x(n) as initial guess. 3. Sequential quadratic programming algorithm in MATLAB [43] was used in this paper. 4. If the optimal solution to this problem is found, this solution is accepted else original solution is taken as optimal solution. 5. Repeat Step 1 for all the candidate optimal solutions. 6. For each candidate optimal solution, M new solutions are obtained using local search. 7. Combine all the optimal and candidate solutions in a set SPOF. 8. Remove sub-optimal solutions from the set SPOF by performing a non-dominated search. 9. Remove duplicate designs from set SPOF. The solution set SPOF represents the Pareto optimal set. Appendix C. Clustering algorithm Clustering of the Pareto optimal solution set data serves two purposes: rst, to identify the subsets which can be used to identify the boundary of the Pareto optimal response surface and second, to reduce the large number of Pareto optimal solutions data to a manageable number of trade-o designs which can be evaluated by the designer. If Nclust designs are required, a clustering algorithm groups the entire solution set into Nclust distinct clusters and picks one representative solution from each clusters. This representative solution is the closest to the mean of the parent cluster. Each representative solution of the cluster can be used by the designer to evaluate the design and the solutions in each cluster represent the subset which can be used to nd the boundary of the Pareto optimal response surface by tting the convex hulls. The clustering procedure, similar to hierarchical clustering method [45], followed in this paper is given as follows: 1. Reduce the large number of Pareto optimal solutions to a more manageable number by selecting the designs with high crowding distance. In this study,

where ^ y i is the value predicted by the RS for the ith point which is excluded while generating the RS. If this value is close to ra then the model performs well. More details on the polynomial response surface approximation can be found in the classical text by Myers and Montgomery [39]. Appendix B. MOEA + local search strategy [42] After running the evolutionary algorithm, a set of approximate Pareto optimal points was obtained. It was shown by Goel and Deb [1315], and Ishibuchi et al. [16] that using a local search method on this approximate Pareto optimal solution set can further improve the solutions and help better converge to the Pareto optimal set. There are a number of dierent ways of using the local search in conjunction with EA. Goel and Deb [15] studied two extremes of hybridization, a posteriori hybrid method where local search starts from the EA simulation results, and an online hybrid method which uses local search after every generation of the EA simulation. They found that the posteriori hybrid method performs better than the online hybrid method in terms of convergence to the Pareto optimal front and ensuring diversity on the Pareto optimal front. In the current study, the posteriori hybrid method is used and the sequential quadratic programming method implementation in MATLAB [43] is used as the local optimizer. When using a local search method, the multi-objective optimization problem has to be converted to a single objective optimization problem. The weighted sum strategy where multiple objectives are combined in a single objective by associating a weight to each objective is very popular and simple to implement. However, it was shown by Goel et al. [42] that this method does not always improve the solutions. Alternatively, the e-constraint strategy gives good results for two-objective optimization problems [1,40]. Goel et al. [42] extended this idea to the multi-objective optimization problems with more than two objectives. The procedure of using the e-constraint strategy to improve the convergence to Pareto optimal front is given as follows:

892

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893

2.

3. 4. 5.

6. 7. 8. 9. 10. 11.

12.

3000 designs out of 89,147 designs with highest crowding distance were selected.5 Since crowding distance favors lonely solutions, this set of solutions used for clustering algorithm is augmented by selecting random solutions which favor the regions with high density. Thus compensating the loss of the solutions in the crowded regions. Another 2000 designs were selected randomly. Initialize all the solutions in this set as independent clusters. Identify the power p such that npop  2pNclust. For p/2 iterations, a. Compute distances between each cluster. b. Merge all the neighboring clusters such that each cluster is merged with only one cluster. c. Find the new centroid of the cluster. d. Repeat step 5a. This step merges many neighbors at one time, reducing the cost of computation in clustering. Now compute the distance between the centroids of each cluster. Merge the two clusters with smallest distance between the centroids. Re-compute the centroid of the new cluster. Repeat step 6 till the number of clusters is reduced to Nclust. Find the data point which was closest to the centroid of the cluster. Now for each data point in the Pareto optimal set, a. Compute the distance from the centroid of Nclust clusters. b. Move the data point to the cluster whose centroid is closest to the data point. Repeat step 11 for all the points in the Pareto optimal set.

one. It also has the minimum fuel ow area and the thinnest post tip thickness. This design performs well, but has very high wall and injector face temperatures. Goel et al. [42] systematically investigated the correlations among the objectives in entire design space before solving the optimization problem. A uniform grid data (114 = 14,641 designs) was generated and evaluated using the response surface approximations for the objective functions. The correlation matrix Cdes and corresponding p-values were computed6 using MATLAB with proper corrections for the boundary points 3 2 X cc TW4 TTmax TFmax 7 6 0:28 7 6 TFmax 1:00 0:12 0:96 7 6 14 C des 6 0:12 1:00 0:03 0:57 7 7: 6 X cc 7 6 0:96 0:03 1:00 0:30 5 4 TW4 TTmax 0:28 0:57 0:30 1:00 The correlation matrix (Cdes) showed that the objective functions TFmax and TW4 were strongly correlated in the design space. p-values and 95% condence intervals for the correlation coecients also established the statistical signicance of the results. The objectives Xcc and TTmax were found to be weakly correlated. These ndings are in agreement with the results observed by Vaidyanathan et al. [20]. References
[1] K.M. Miettinen, Nonlinear Multiobjective Optimization, Kluwer, Boston, 1999. [2] A. Messac, Physical programming: eective optimization for computational design, AIAA J. 34 (1) (1996) 149158. [3] P. Sen, J.-B. Yang, Multiple Criteria Decision Support in Engineering Design, Springer-Verlag, London, 1998. [4] V. Chankong, Y.Y. Haimes, Multiobjective Decision Making Theory and Methodology, Elsevier Science, New York, 1983. [5] K. Deb, Multi-objective Optimization Using Evolutionary Algorithms, Wiley, Chichester, UK, 2001. [6] E. Zitzler, L. Thiele, An evolutionary algorithm for multi-objective optimization: The strength Pareto approach, Technical report no. 43, Zurich: Computer engineering and networks laboratory Switzerland, 1998. [7] J. Knowles, D. Corne, Approximating the non-dominated front using the Pareto archived evolution strategy, Evol. Comput. 8 (2) (2000) 149172. [8] K. Deb, S. Agrawal, A. Pratap, T. Meyarivan, A fast and elitist multiobjective genetic algorithm for multi-objective optimization: NSGAII, in: Proceedings of the Parallel Problem Solving from Nature VI Conference, Paris, 2000, pp. 849858. [9] K. Deb, T. Goel, Controlled elitist non-dominated sorting genetic algorithms for better convergence, in: Proceedings of the First International Conference on Evolutionary Multi-criterion Optimization, Zurich, 2001, pp. 6781.

This method reduces the cost of clustering substantially while identifying the representative solution of the cluster and the members in each cluster. Appendix D. Reduced optimization problem after correlation analysis [42] Vaidyanathan et al. [20] pointed out that all four objectives, based on the designs obtained during their individual minimization largely fall into two groups: (i) TFmax and TW4 and (ii) Xcc and TTmax. Minimizing TFmax and TW4 leads to a design with a equal to zero (making the design a shear coaxial type), maximum fuel ow area and thickest post tip. This design yields less desirable performance due to slower mixing across the shear layer. Minimizing TTmax and Xcc, results in an impinging-like design with a equal to

This step is required to limit the computational cost of clustering operation. If the number of Pareto optimal solutions is small, Steps 1 and 2 can be ignored.

6 Here quadrature is used to numerically compute correlations. One can also resort to Monte-Carlo simulations or even analytical integrals to compute correlations. However since the correlations are computed in entire design space, the error in computation of correlations should be no more than small numerical error irrespective of the choice of sampling points.

T. Goel et al. / Comput. Methods Appl. Mech. Engrg. 196 (2007) 879893 [10] D. Sasaki, M. Morikawa, S. Obayashi, K. Nakahashi, Aerodynamic shape optimization of supersonic wings by adaptive range multiobjective genetic algorithms, in: Proceedings of the First International Conference on Evolutionary Multi-criterion Optimization, Zurich, 2001, pp. 639652. [11] D. Sasaki, S. Obayashi, K. Nakahashi, NavierStokes optimization of supersonic wings with four objectives using evolutionary algorithms, J. Aircraft 39 (4) (2002) 621629. [12] R.A.E. Ma kinen, J. Periaux, J. Toivanen, Multidisciplinary shape optimization in aerodynamics and electromagnetics using genetic algorithms, Int. J. Numer. Methods Fluids 30 (1999) 149159. [13] K. Deb, T. Goel, Multi-objective evolutionary algorithms for engineering shape design, in: R. Sarker, M. Mohammadin, X. Yao (Eds.), Evolutionary Optimization, Kluwer, 2000, pp. 147176. [14] K. Deb, T. Goel, A hybrid multi-objective evolutionary approach to engineering shape design, in: Proceedings of Evolutionary Multicriterion Optimization Conference, Zurich, 2001, pp. 385399. [15] T. Goel, K. Deb, Hybrid methods for multi-objective evolutionary algorithms, in: Proceedings of the 4th Asia-Pacic Conference on Simulated Evolution and Learning Computational Intelligence for the e-age SEAL02, Singapore, 2002, pp. 188192. [16] H. Ishibuchi, T. Yoshida, T. Murata, Balance between genetic search and local search in memetic algorithms for multi-objective permutation owshop scheduling, IEEE Trans. Evol. Comput. 7 (2) (2003) 204223. [17] S. Obayashi, D. Sasaki, A. Oyama, Finding tradeos by using multiobjective optimization algorithms, Trans. Jpn. Soc. Aeronaut. Space Sci. 47 (155) (2004) 5158. [18] J.I. Madsen, W. Shyy, R.T. Haftka, Response surface techniques for diuser shape optimization, AIAA J. 38 (2000) 15121518. [19] R. Vaidyanathan, N. Papila, W. Shyy, K.P. Tucker, L.W. Grin, R.T. Haftka, N. Fitz-Coy, Neural network and response surface methodology for rocket engine component optimization, in: 8th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Paper No. 2000-4480, Long Beach, CA, 2000. [20] R. Vaidyanathan, K.P. Tucker, N. Papila, W. Shyy, CFD based design optimization for a single element rocket injector, J. Propulsion Power 20 (4) (2004) 705717 (also presented in 41st Aerospace Sciences Meeting and Exhibit, Reno, NV, 2003). [21] W. Shyy, N. Papila, R. Vaidyanathan, K. Tucker, Global design optimization for aerodynamics and rocket propulsion components, Progr. Aerospace Sci. 37 (2001) 59118. [22] N. Papila, W. Shyy, L.W. Grin, F. Huber, K. Tran, Preliminary design optimization for a supersonic turbine for rocket propulsion, in: 36th AIAA/ASME/SAE/ASEE Joint Propulsion Conference and Exhibit, Huntsville, AL, 2000. [23] N. Papila, W. Shyy, L.W. Grin, D.J. Dorney, Shape optimization of supersonic turbines using response surface and neural network methods, J. Propulsion Power 18 (2001) 509518. [24] R. Dornberger, D. Bu che, P. Stoll, Multidisciplinary optimization in turbomachinery design, in: European Congress on Computational Methods in Applied Sciences and Engineering, Barcelona, 2000. [25] A. Bramanti, P. Di Barba, M.B. Farina, A. Savini, Combining response surfaces and evolutionary strategies for multi-objective Pareto-optimization in electromagnetics, Int. J. Appl. Electromagn. Mech. 15 (2001) 231236. [26] B. Wilson, D. Cappelleri, T.W. Simpson, M. Frecker, Ecient Pareto frontier exploration using surrogate approximations, Optim. Engrg. 2 (2001) 3150. [27] D.J. Cappelleri, M.I. Frecker, T.W. Simpson, A. Snyder, A metamodel based approach for optimal design of a PZT bimorph actuator for minimally invasive surgery, ASME J. Mech. Des. 124 (2) (2002) 354357. [28] M. Farina, J.K. Sykulski, Comparative study of evolution strategies combined with approximation techniques for practical electromag-

893

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38] [39]

[40] [41] [42]

[43] [44]

[45] [46] [47] [48]

[49]

netic optimization problems, IEEE Trans. Magn. 37 (5) (2001) 3216 3220. M. Farina, P. Di Barba, A. Bramanti, A GRS method for Paretooptimal front identication in electromagnetic multi-objective synthesis, IEEE Proc. Sci. Manage. Technol. 149 (5) (2002) 207213. M. Emmerich, A. Giotis, M. Ozdenir, T. Baeck, K. Giannakoglou, Metamodel-assisted evolution strategies, in: Proceedings of Parallel Problem Solving from Nature (PPSN VII), Springer LNCS, 2002, pp. 361370. M. Emmerich, K. Giannakoglou, B. Naujoks, Single and multiobjective optimization assisted by Gaussian random eld metamodels, IEEE Trans. Evol. Comput. (Special Issue on Evolutionary Computation in the Presence of Uncertainty) 10 (4) (2006) 421439. Y.S. Ong, P.B. Nair, A.J. Keane, Evolutionary optimization of computationally expensive problems via surrogate modeling, AIAA J. 41 (4) (2003) 687696. P.K.S. Nain, K. Deb, A multi-objective optimization procedure with successive approximation methods, KANGAL Technical Report 2005002, Kanpur Genetic Algorithms Laboratory, India, 2005. J. Knowles, E.J. Hughes, Multiobjective optimization on a budget of 250 evaluations, in: Proceedings of Evolutionary Multi-Criterion Optimization Conference, 2005, LNCS 3410, pp. 176190. J. Knowles, ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multi-objective optimization problems, IEEE Trans. Evol. Comput. 10 (1) (2006) 5066. Y.S. Chen, Compressible and incompressible ow computation with a pressure-based method, AIAA89-0286, in: AIAA 28th Aerospace Sciences Meeting, 1989. T.S. Wang, Y.S. Chen, A united NavierStokes oweld and performance analysis of liquid rocket engines, AIAA 90-2494, in: AIAA 26th Joint Propulsion Conference, 1990. Y.S. Chen, R.C. Farmer, CFD analysis of bae ame stabilization, AIAA 91-1967, in: AIAA 27th Joint Propulsion Conference, 1991. R.H. Myers, D.C. Montgomery, Response Surface Methodology Process and Product Optimization Using Designed Experiment, Wiley-Interscience, 1995. R.E. Steuer, Multiple Criteria Optimization: Theory, Computation, and Application, Wiley, New York, 1986. D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, 1989. T. Goel, R. Vaidyanathan, R.T. Haftka, N.V. Queipo, W. Shyy, K. Tucker, Response surface approximation of Pareto optimal front in multi-objective optimization, in: Proceedings of the 10th AIAA/ ISSMO Multidisciplinary Analysis and Optimization Conference, Albany, NY, 2004, AIAA2004-4501. MATLAB, The language of technical computing, Version 6.5 Release 13. 19842002, The MathWorks, Inc. P. Ramu, S. Missoum, R.T. Haftka, A convex hull approach for the reliability-based optimization of transient dynamic problems, in: Proceedings of the 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Albany, NY, 2004, AIAA 2004-4618. A.K. Jain, R.C. Dubes, Algorithms for Clustering Data, Prentice Hall College, New Jersey, 1988. A. Owen, Orthogonal arrays for: computer experiments, integration and visualization, Statist. Sin. 2 (2) (1992) 439452. JMP, The statistical discovery softwareTM, Version 5, Copyright 19892002, SAS Institute Inc., Cary, NC, USA. Y. Mack, T. Goel, W. Shyy, R.T. Haftka, Surrogate model based optimization framework: a case study in aerospace design, in: S. Yang, Y.S. Ong, Y. Jin (Eds.), Evolutionary Computation in Dynamic and Uncertain Environments, accepted for publication. R. Vaidyanathan, T. Goel, W. Shyy, R.T. Haftka, N.V. Queipo, K. Tucker, Global sensitivity and trade-o analyses for multi-objective liquid rocket injector design, in: 40th AIAA/ASME/SAE/ASEE Joint Propulsion Conference and Exhibit, Ft. Lauderdale, FL, 2004.