Hard-Constrained versus
Soft-Constrained Parameter
Estimation
A. BENAVOLI
L. CHISCI, Member, IEEE
Universita` di Firenze
Italy
A. FARINA, Fellow, IEEE
L. ORTENZI
SELEX—Sistemi Integrati S.p.A.
Italy
G. ZAPPA
Universita` di Firenze
Italy
The paper aims at contrasting two different ways of
incorporating a priori information in parameter estimation,
i.e., hard-constrained and soft-constrained estimation.
Hard-constrained estimation can be interpreted, in the
Bayesian framework, as maximum a posteriori probability
(MAP) estimation with uniform prior distribution over the
constraining set, and amounts to a constrained least-squares
(LS) optimization. Novel analytical results on the statistics
of the hard-constrained estimator are presented for a linear
regression model subject to lower and upper bounds on a single
parameter. This analysis allows to quantify the mean squared
error (MSE) reduction implied by constraints and to see how
this depends on the size of the constraining set compared
with the confidence regions of the unconstrained estimator.
Contrastingly, soft-constrained estimation can be regarded as
MAP estimation with Gaussian prior distribution and amounts to
a less computationally demanding unconstrained LS optimization
with a cost suitably modified by the mean and covariance of
the Gaussian distribution. Results on the design of the prior
covariance of the soft-constrained estimator for optimal MSE
performance are also given. Finally, a practical case-study
concerning a line fitting estimation problem is presented in order
to validate the theoretical results derived in the paper as well
as to compare the performance of the hard-constrained and
soft-constrained approaches under different settings.
Manuscript received November 18, 2004; revised September 19,
2005; released for publication March 10, 2006.
IEEE Log No. T-AES/42/4/890175.
Refereeing of this contribution was handled by V. Krishnamurthy.
Authors’ current addresses: A. Benavoli, G. Zappa, L. Chisci, DSI,
Universita` di Firenze, via Santa Marta 3, 50139 Firenze, Italy;
E-mail: (chisci@dsi.unifi.it); A. Farina, L. Ortenzi, Engineering
Division, SELEX—Sistemi Integrati S.p.A., Rome, Italy.
0018-9251/06/$17.00 c° 2006 IEEE
I. INTRODUCTION
Consider the classical maximum likelihood
(ML) estimation of a parameter vector, given a set
of measurements. The estimate is provided by the
minimization of the likelihood function; in this
case, the Cramer-Rao lower bound (CRLB) can be
computed by the inversion of the Fisher information
matrix (FIM) and gives the minimum theoretically
achievable variance of the estimate [2, 3]. In the work
presented here, it is investigated how to improve the
estimation accuracy by exploiting a priori information
on the parameters to be estimated.
In [1] the CRLB is computed in presence of
equality “constraints.” The constrained estimation
problem, in this case, can be reduced to an
unconstrained problem by the use of Lagrange
multipliers. Equality constraints [4, 5] express very
precise a priori information on the parameters to
be estimated; unfortunately, in several practical
applications, such precise information is not available.
In most cases, “inequality” constraints are available
[6] with a consequent complication of the estimation
problem and of the computation of the CRLB.
There are clearly many ways of expressing
a priori information. This paper will specifically
address two possible approaches. A first approach,
referred to as “hard-constrained” estimation, assumes
inequality constraints (e.g. lower and upper bounds)
on the parameters to be estimated and formulates,
accordingly, the estimation problem as a constrained
least squares (LS) optimization problem. The
second approach, referred to as “soft-constrained”
estimation, considers the parameters to be estimated
as random variables with a Gaussian probability
density function (pdf), whose statistical parameters
(mean and covariance) are a priori known. In this
case the a priori knowledge on the parameters
simply modifies the LS cost functional and the
resulting estimation problem amounts to a less
computationally demanding unconstrained LS
optimization. Hard-constrained and soft-constrained
estimation can be interpreted, in a Bayesian
framework, as maximum a posteriori probability
(MAP) estimation with a uniform and, respectively,
Gaussian prior distribution. Another alternative
could be constrained minimum mean squared error
(MMSE) estimation which can be numerically
approximated via Monte Carlo methods [7, 8]. This
approach, which involves extensive computations for
numerical integration, is computationally much more
expensive and will, therefore, not be considered in this
work.
A theoretical analysis on the statistics of the
hard-constrained estimator for a linear regression
model subject to hard bounds on a single parameter
will be carried out. The obtained results allow to
quantify the bias and mean squared error (MSE)
1224 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 42, NO. 4 OCTOBER 2006