article

The complex-step derivative approximation

Authors:

Joaquim R. R. A. Martins,

Juan J. AlonsoAuthors Info & Claims

ACM Transactions on Mathematical Software (TOMS), Volume 29, Issue 3

Pages 245 - 262

https://doi.org/10.1145/838250.838251

Published: 01 September 2003 Publication History

Abstract

The complex-step derivative approximation and its application to numerical algorithms are presented. Improvements to the basic method are suggested that further increase its accuracy and robustness and unveil the connection to algorithmic differentiation theory. A general procedure for the implementation of the complex-step method is described in detail and a script is developed that automates its implementation. Automatic implementations of the complex-step method for Fortran and C/C++ are presented and compared to existing algorithmic differentiation tools. The complex-step method is tested in two large multidisciplinary solvers and the resulting sensitivities are compared to results given by finite differences. The resulting sensitivities are shown to be as accurate as the analyses. Accuracy, robustness, ease of implementation and maintainability make these complex-step derivative approximation tools very attractive options for sensitivity analysis.

References

[1]

Anderson, W. K., Newman, J. C., Whitfield, D. L., and Nielsen, E. J. 1999. Sensitivity analysis for the Navier--Stokes equations on unstructured meshes using complex variables. AIAA Paper 99--3294.

[2]

Bartholomew-Biggs, M. 1995. OPFAD---A Users Guide to the OPtima Forward Automatic Differentiation Tool. Numerical Optimization Centre, University of Hertfordsshire.

[3]

Beck, T. 1994. Automatic differentiation of iterative processes. J. Comput. Appl. Math. 50, 109--118.

[4]

Beck, T. and Fischer, H. 1994. The if-problem in automatic differentiation. J. Comput. Appl. Math. 50, 119--131.

[5]

Bendtsen, C. and Stauning, O. 1996. FADBAD, A flexible C++ package for automatic differentiation---using the forward and backward methods. Tech. Rep. IMM-REP-1996-17, Technical University of Denmark, DK-2800 Lyngby, Denmark.

[6]

Berz, M. 1987. The differential algebra FORTRAN precompiler DAFOR. Tech. Rep. AT--3: TN--87--32, Los Alamos National Laboratory, Los Alamos, N.M.

[7]

Bischof, C., Carle, A., Corliss, G., Grienwank, A., and Hoveland, P. 1992. ADIFOR: Generating derivative codes from Fortran programs. Sci. Prog. 1, 1, 11--29.

[8]

Bischof, C., Corliss, G., and Grienwank, A. 1991. ADIFOR exception handling. Tech. Rep. MCS-TM-159, Argonne Technical Memorandum.

[9]

Bischof, C. H., Roh, L., and Mauer-Oats, A. J. 1997. ADIC: An extensible automatic differentiation tool for ANSI-C. Softw. --- Prac. Exp. 27, 12, 1427--1456.

[10]

Brown, S. 1995. OPRAD---A Users Guide to the OPtima Reverse Automatic Differentiation Tool. Numerical Optimization Centre, University of Hertfordsshire.

[11]

Corliss, G., Faure, C., Griewank, A., Hascoet, L., and Naumann, U., Eds. 2001. Automatic Differentiation: From Simulation to Optimization. Springer.

[12]

Faure, C. and Papegay, Y. 1997. Odyssée Version 1.6. The Language Reference Manual. INRIA. Rapport Technique 211.

[13]

Giering, R. 1997. Tangent Linear and Adjoint Model Compiler Users Manual. Max-Planck Institut für Meteorologie.

[14]

Gill, P. E., Murray, W., and Wright, M. H. 1981. Practical Optimization, Chapter 2, pp. 11--12. Academic Press, San Diego, Calif.

[15]

Griewank, A. 2000. Evaluating Derivatives. SIAM, Philadelphia, Pa.

[16]

Griewank, A., Bischof, C., Corliss, G., Carle, A., and Williamson, K. 1993. Derivative convergence for iterative equation solvers. Optim. Meth. Softw. 2, 321--355.

[17]

Griewank, A., Juedes, D., and Utke, J. 1996. Algorithm 755: ADOL-C: A package for the automatic differentiation of algorithms written in C/C++. ACM Trans. Math. Softw. 22, 2 (June), 131--167.

[18]

Horwedel, J. E. 1991. GRESS Version 2.0 User's Manual. ORNL. TM-11951.

[19]

Kubota, K. 1996. PADRE2---FORTRAN Precompiler for Automatic Differentiation and Estimates of Rounding Errors. SIAM, Philadelphia, Pa., pp. 367--374.

[20]

Lutz, M. 1996. Programming Python. O'Reilly & Associates, Inc., Cambridge, Mass.

[21]

Lyness, J. N. 1967. Numerical algorithms based on the theory of complex variable. In Proceedings of the ACM National Meeting (Washington, D.C.). ACM New York, pp. 125--133.

[22]

Lyness, J. N. and Moler, C. B. 1967. Numerical differentiation of analytic functions. SIAM J. Numer. Anal. 4, 2 (June), 202--210.

[23]

Martins, J. R. R. A. 2002. A Coupled-Adjoint Method for High-Fidelity Aero-Structural Optimization. Ph.D. dissertation. Stanford University, Stanford, CA 94305.

[24]

Martins, J. R. R. A. 2003. A Guide to the Complex-Step Derivative Approximation. http://mdolab.utias.utoronto.ca.

[25]

Martins, J. R. R. A., Alonso, J. J., and Reuther, J. J. 2002. Complete configuration aero-structural optimization using a coupled sensitivity analysis method. AIAA Paper 2002-5402 (Sept.).

[26]

Martins, J. R. R. A., Kroo, I. M., and Alonso, J. J. 2000. An automated method for sensitivity analysis using complex variables. AIAA Paper 2000-0689 (Jan.).

[27]

Martins, J. R. R. A., Sturdza, P., and Alonso, J. J. 2001. The connection between the complex-step derivative approximation and algorithmic differentiation. AIAA Paper 2001-0921 (Jan.).

[28]

Newman, J. C., Anderson, W. K., and Whitfield, L., D. 1998. Multidisciplinary sensitivity derivatives using complex variables. Tech. Rep. MSSU-COE-ERC-98-08 (July), Computational Fluid Dynamics Laboratory.

[29]

Olver, F. W. J. 1983. Error Analysis of Complex Arithmetic. Reidel, Dordrecht, Holland, pp. 279--292.

[30]

Pryce, J. D. and Reid, J. K. 1998. AD01, A Fortran 90 code for automatic differentiation. Report RAL-TR-1998-057, Rutherford Appleton Laboratory, Chilton, Didcot, Oxfordshire, OX11 OQX, U.K.

[31]

Reuther, J., Alonso, J. J., Jameson, A., Rimlinger, M., and Saunders, D. 1999a. Constrained multipoint aerodynamic shape optimization using an adjoint formulation and parallel computers: Part I. J. Aircraft 36, 1, 51--60.

[32]

Reuther, J., Alonso, J. J., Martins, J. R. R. A., and Smith, S. C. 1999b. A coupled aero-structural optimization method for complete aircraft configurations. AIAA Paper 99--0187.

[33]

Reuther, J., Alonso, J. J., Vassberg, J. C., Jameson, A., and Martinelli, L. 1997. An efficient multiblock method for aerodynamic analysis and design on distributed memory systems. AIAA Paper 97--1893 (June).

[34]

Rhodin, A. 1997. IMAS---Integrated Modeling and Analysis System for the solution of optimal control problems. Comput. Phys. Commun. 107 (Dec.), 21--38.

[35]

Shiriaev, D. 1996. ADOL--F automatic differentiation of Fortran codes. In Computational Differentiation: Techniques, Applications, and Tools, M. Berz, C. H. Bischof, G. F. Corliss, and A. Griewank, Eds., SIAM, Philadelphia, Pa., pp. 375--384.

[36]

Squire, W. and Trapp, G. 1998. Using complex variables to estimate derivatives of real functions. SIAM Rev. 40, 1 (Mar.), 110--112.

[37]

Sturdza, P., Manning, V. M., Kroo, I. M., and Tracy, R. R. 1999. Boundary layer calculations for preliminary design of wings in supersonic flow. AIAA Paper 99--3104 (June).

[38]

Yao, J., Alonso, J. J., Jameson, A., and Liu, F. 2001. Development and validation of a massively parallel flow solver for turbomachinery flow. J. Prop. Power 17, 3 (June), 659--668.

Cited By

Majidian H(2024)A comparative study of Filon-type rules for oscillatory integralsJournal of Numerical Analysis and Approximation Theory10.33993/jnaat531-138053:1(132-145)Online publication date: 6-Mar-2024
https://doi.org/10.33993/jnaat531-1380
Moni AYao WMalekmohamadi H(2024)Data-Driven Nonintrusive Model-Order Reduction for Aerodynamic Design OptimizationAIAA Journal10.2514/1.J06308062:7(2638-2658)Online publication date: Jul-2024
https://doi.org/10.2514/1.J063080
Navarro JVelasquez-Gonzalez JAristizabal MJarmer GKessler SMontoya AMillwater HRestrepo D(2024)Arbitrary-Order Sensitivity Analysis in Wave Propagation Problems Using Hypercomplex Spectral Finite Element MethodAIAA Journal10.2514/1.J06283462:4(1447-1460)Online publication date: Apr-2024
https://doi.org/10.2514/1.J062834
Show More Cited By

Index Terms

The complex-step derivative approximation

Recommendations

Using Multicomplex Variables for Automatic Computation of High-Order Derivatives

The computations of the high-order partial derivatives in a given problem are often cumbersome or not accurate. To combat such shortcomings, a new method for calculating exact high-order sensitivities using multicomplex numbers is presented. Inspired by ...
Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/C++

The C++ package ADOL-C described here facilitates the evaluation of first and higher derivatives of vector functions that are defined by computer programs written in C or C++. The resulting derivative evaluation routines may be called from C/C++, ...
A methodology for the development of discrete adjoint solvers using automatic differentiation tools

A methodology for the rapid development of adjoint solvers for computational fluid dynamics (CFD) models is presented. The approach relies on the use of automatic differentiation (AD) tools to almost completely automate the process of development of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Mathematical Software

ACM Transactions on Mathematical Software Volume 29, Issue 3

September 2003

107 pages

ISSN:0098-3500

EISSN:1557-7295

DOI:10.1145/838250

Issue’s Table of Contents

Copyright © 2003 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2003

Published in TOMS Volume 29, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

496
Total Citations
View Citations
5,020
Total Downloads

Downloads (Last 12 months)230
Downloads (Last 6 weeks)23

Reflects downloads up to 28 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Majidian H(2024)A comparative study of Filon-type rules for oscillatory integralsJournal of Numerical Analysis and Approximation Theory10.33993/jnaat531-138053:1(132-145)Online publication date: 6-Mar-2024
https://doi.org/10.33993/jnaat531-1380
Moni AYao WMalekmohamadi H(2024)Data-Driven Nonintrusive Model-Order Reduction for Aerodynamic Design OptimizationAIAA Journal10.2514/1.J06308062:7(2638-2658)Online publication date: Jul-2024
https://doi.org/10.2514/1.J063080
Navarro JVelasquez-Gonzalez JAristizabal MJarmer GKessler SMontoya AMillwater HRestrepo D(2024)Arbitrary-Order Sensitivity Analysis in Wave Propagation Problems Using Hypercomplex Spectral Finite Element MethodAIAA Journal10.2514/1.J06283462:4(1447-1460)Online publication date: Apr-2024
https://doi.org/10.2514/1.J062834
Gade JGeiger FKemmler RBischoff M(2024)A form-finding method for adaptive truss structures subject to multiple static load casesInternational Journal of Space Structures10.1177/0956059923121270739:2(93-114)Online publication date: 30-Jan-2024
https://doi.org/10.1177/09560599231212707
Peng ZYang YShao TJiang CZhou K(2024)X-SLAM: Scalable Dense SLAM for Task-aware Optimization using CSFDACM Transactions on Graphics10.1145/365823343:4(1-15)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658233
Oshima K(2024)The Impact of Second-Order Analytical Derivatives on Regularized Direct Multiple Shooting Method for Impulsive Spacecraft Trajectory OptimizationGuidance, Navigation and Control10.1142/S273748072450001804:01Online publication date: 15-Mar-2024
https://doi.org/10.1142/S2737480724500018
Shinglot VZweck J(2024)Floquet Stability of Periodically Stationary Pulses in a Short-Pulse Fiber LaserSIAM Journal on Applied Mathematics10.1137/23M159810684:3(961-987)Online publication date: 10-May-2024
https://doi.org/10.1137/23M1598106
Rubensson E(2024)A Unifying Framework for Higher Order Derivatives of Matrix FunctionsSIAM Journal on Matrix Analysis and Applications10.1137/23M158058945:1(504-528)Online publication date: 8-Feb-2024
https://doi.org/10.1137/23M1580589
Jongeneel WYue MKuhn D(2024)Small Errors in Random Zeroth-Order Optimization Are ImaginarySIAM Journal on Optimization10.1137/22M151026134:3(2638-2670)Online publication date: 19-Jul-2024
https://doi.org/10.1137/22M1510261
Najmon JTovar A(2024)Evaluation of Neural Network-Based Derivatives for Topology OptimizationJournal of Mechanical Design10.1115/1.4064243146:7Online publication date: 29-Jan-2024
https://doi.org/10.1115/1.4064243
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents