Explaining the Saddlepoint Approximation
Constantino GOUTIS and George CASELLA
Saddlepoint approximations are powerful tools for obtain-
ing accurate expressions for densities and distribution func-
tions. We give an elementary motivation and explanation of
approximation techniques, starting with Taylor series ex-
pansions and progressing to the Laplace approximation of
integrals. These approximations are illustrated with exam-
ples of the convolution of simple densities. We then turn to
the saddlepoint approximation and, using both the Fourier
inversion formula nd Edgeworth expansions, we derive the
saddlepoint approximation to the density of a single ran-
dom variable. We next approximate the density of the sam-
ple mean of iid random variables, and also demonstrate the
technique for approximating the density of a maximum like-
lihood estimator in exponential families.
KEY WORDS: Edgeworth expansions; Fourier trans-
form; Laplace method; Maximum likelihood estimators;
Moment-generating functions; Taylor series.
1. INTRODUCTION
The saddlepoint approximation has been a valuable tool
in asymptotic analysis. Various techniques of accurate ap-
proximations, relying on it in one way or another, have been
developed since the seminal article by Daniels (1954). Reid
(1988, 1991) gave a comprehensive review of the applica-
tions and a broad coverage of the relevant literature.
The number of applications of the saddlepoint approxi-
mation is quite impressive, as warrants this extremely pow-
erful approximation (see Section 5 for a partial list). Typ-
ically, derivations and implementations of saddlepoint ap-
proximations rely on tools such as exponential tilting, Edge-
worth expansions, Hermite polynomials, complex integra-
tion, and other advanced notions. Although these are impor-
tant tools for researchers in the area, they may obscure the
fundamental idea of the saddlepoint approximation. A goal
of this article is to illustrate that there is a simple basic idea
behind this useful technique. Namely, write the quantity one
wishes to approximate as an integral, expand the integrand
with respect to the dummy variable of integration, keep the
first few terms and integrate. The integral can be over the
complex plane, corresponding to the inversion formula of a
Fourier transform, but this is a secondary point.
We start with an elementary motivation of the technique,
stressing familiar Taylor series expansions. At the begin-
ning we will somewhat ignore the statistical applications,
because saddlepoint approximations are general techniques,
and quite often references to random variables and distri-
butions may be more confusing than illuminating. Once
the approximation is developed, however, we will examine
some statistical applications. Throughout he article, we as-
sume that the functions are as regular as needed. In other
words, when we write a derivative or an integral, we as-
sume that they exist. Furthermore, we develop the methods
in the univariate case. This is almost without loss of gen-
erality as the multivariate case is essentially the same but
with a somewhat more complicated notation.
To keep the technical evel reasonable, we avoid any rig-
orous asymptotic analysis, but a few remarks are in or-
der. The accuracy of an approximation is assessed by ex-
amining the size of the error of approximation. We use
the notation 0(l) which denotes a function that satisfies
1imt<O tO (') = constant, where for a random sample of
size n, standard techniques typically give approximations
of order 0( ). The saddlepoint can improve this to 0( n)
and even 0(n-3/2) in some circumstances. Unfortunately,
calculation of these error terms often requires detailed tech-
nical arguments, and we do not include them here (see Field
and Ronchetti 1990 or Kolassa 1994). This omission does
not speak to the importance of such calculations. Indeed, in
application, the size of the approximation error is perhaps
the most important concern, and this error must be assessed
either through analytical or numerical means.
In Section 2 we introduce approximation techniques from
the point of Taylor series and Laplace approximations, and
give some examples. Section 3 is an attempt o explain the
original derivation of the saddlepoint, which has its roots
in Fourier transforms and complex analysis, along with an-
other derivation based on Edgeworth expansions. Those un-
willing to wade through these derivations need only look at
formula (25), which gives the formula for the density ap-
proximation of a sample mean. Section 4 treats the case of
the MLE in exponential families, with the important for-
mulas being (34) and (36). Section 5 contains a short dis-
cussion.
2. FIRST EXPANSIONS
We begin by looking at some basic principles of approx-
imation, using the familiar tool of the Taylor expansion. As
we will see, the underlying strategy of this approximation
carries through to more sophisticated approximations. Note,
however, that the approximations in this section can have
large errors. As we are working with densities of single
George Casella is Liberty Hyde Bailey Professor of Biological Statistics,
Department of Biometrics, Cornell University, 434 Warren Hall, Ithaca,
NY 14853. The authors thank Luis Tenorio for useful discussions, and
the reviewers for providing detailed comments on earlier versions of this
article, which resulted in a much improved presentation. This research was
supported by NSF Grants DMS 9305547 and DMS 9625440, and this is
paper BU-131 1-M in the Biometrics Unit, Cornell University, Ithaca, NY
14853. The original version of this article was written in December 1995,
before the tragic death of Costas Goutis in July 1996.
216 The Amnerican Statistician, Autguist 1999, Vol. 53, No. 3 (?) 1999 Americanl Statistical Association