Introduction to Derivative-Free Optimization

The absence of derivatives, often combined with the presence of noise or lack of smoothness, is a major challenge for optimization. This book explains how sampling and model techniques are used in derivative-free methods and how these methods are designed to efficiently and rigorously solve optimization problems. Although readily accessible to readers with a modest background in computational mathematics, it is also intended to be of interest to researchers in the field. Introduction to Derivative-Free Optimization is the first contemporary comprehensive treatment of optimization without derivatives. This book covers most of the relevant classes of algorithms from direct search to model-based approaches. It contains a comprehensive description of the sampling and modeling tools needed for derivative-free optimization; these tools allow the reader to better analyze the convergent properties of the algorithms and identify their differences and similarities.

Preface; 1. Introduction; Part I. Sampling and Modeling: 2. Sampling and linear models; 3. Interpolating nonlinear models; 4. Regression nonlinear models; 5. Underdetermined interpolating models; 6. Ensuring well poisedness and suitable derivative-free models; Part II. Frameworks and Algorithms: 7. Directional direct-search methods; 8. Simplicial direct-search methods; 9. Line-search methods based on simplex derivatives; 10. Trust-region methods based on derivative-free models; 11. Trust-region interpolation-based methods; Part III. Review of Other Topics: 12. Review of surrogate model management; 13. Review of constrained and other extensions to derivative-free optimization; Appendix: software for derivative-free optimization; Bibliography; Index.

The first contemporary comprehensive treatment of optimization without derivatives. This text explains how sampling and model techniques are used in derivative-free methods and how they are designed to solve optimization problems. It is designed to be readily accessible to both researchers and those with a modest background in computational mathematics.

Andrew R. Conn is a research staff member at the IBM T. J. Watson Research Center, Yorktown Heights, NY. In 1994 he was (with N. I. M. Gould and Ph. L. Toint) a joint recipient of the Beale/Orchard-Hays Prize for Computational Excellence in Mathematical Programming and with Chandu Visweswariah he received an IBM Corporate Award in 2002 for contributions to circuit tuning. Currently his major application projects are in the petroleum industry.
Katya Scheinberg is a research staff member in the Business Analytics and Mathematical Sciences Department at the IBM T. J. Watson Research Center. She obtained her PhD in 1997 from Columbia University in New York. She has been working in the area of derivative-free optimization for over ten years and is the author of multiple papers on the subject as well as the open source widely known DFO software.
Luis Nunes Vicente is a Professor of Mathematics at the University of Coimbra, Portugal. He obtained his PhD from Rice University, TX in 1996 under a Fulbright scholarship and was among the three finalists of the 94-96 A. W. Tucker Prize of the Mathematical Programming Society. His research has been strongly supported by the European Union and the European Space Agency. He is a member of several editorial boards including SIAM Journal on Optimization and Journal of Global Optimization and he recently ended a six year term as editor of the SIAM SIAG/Optimization Views-and-News.