Berndt–Hall–Hall–Hausman algorithm

The Berndt–Hall–Hall–Hausman (BHHH) algorithm is a numerical optimization algorithm similar to the Newton–Raphson algorithm, but it replaces the observed negative Hessian matrix with the outer product of the gradient. This approximation is based on the information matrix equality and therefore only valid while maximizing a likelihood function.^[1] The BHHH algorithm is named after the four originators: Ernst R. Berndt, Bronwyn Hall, Robert Hall, and Jerry Hausman.^[2]

Usage

If a nonlinear model is fitted to the data one often needs to estimate coefficients through optimization. A number of optimisation algorithms have the following general structure. Suppose that the function to be optimized is Q(β). Then the algorithms are iterative, defining a sequence of approximations, β_k given by

${\displaystyle \beta _{k+1}=\beta _{k}-\lambda _{k}A_{k}{\frac {\partial Q}{\partial \beta }}(\beta _{k}),}$,

where ${\displaystyle \beta _{k}}$ is the parameter estimate at step k, and ${\displaystyle \lambda _{k}}$ is a parameter (called step size) which partly determines the particular algorithm. For the BHHH algorithm λ_k is determined by calculations within a given iterative step, involving a line-search until a point β_k+1 is found satisfying certain criteria. In addition, for the BHHH algorithm, Q has the form

${\displaystyle Q=\sum _{i=1}^{N}Q_{i}}$

and A is calculated using

${\displaystyle A_{k}=\left[\sum _{i=1}^{N}{\frac {\partial \ln Q_{i}}{\partial \beta }}(\beta _{k}){\frac {\partial \ln Q_{i}}{\partial \beta }}(\beta _{k})'\right]^{-1}.}$

In other cases, e.g. Newton–Raphson, ${\displaystyle A_{k}}$ can have other forms. The BHHH algorithm has the advantage that, if certain conditions apply, convergence of the iterative procedure is guaranteed.^{[citation needed]}

See also

• Davidon–Fletcher–Powell (DFP) algorithm
• Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm

References

Further reading

• V. Martin, S. Hurn, and D. Harris, Econometric Modelling with Time Series, Chapter 3 'Numerical Estimation Methods'. Cambridge University Press, 2015.
• Amemiya, Takeshi (1985). Advanced Econometrics. Cambridge: Harvard University Press. pp. 137–138. ISBN 0-674-00560-0.
• Gill, P.; Murray, W.; Wright, M. (1981). Practical Optimization. London: Harcourt Brace.
• Gourieroux, Christian; Monfort, Alain (1995). "Gradient Methods and ML Estimation". Statistics and Econometric Models. New York: Cambridge University Press. pp. 452–458. ISBN 0-521-40551-3.
• Harvey, A. C. (1990). The Econometric Analysis of Time Series (Second ed.). Cambridge: MIT Press. pp. 137–138. ISBN 0-262-08189-X.

• v
• t
• e

Optimization: Algorithms, methods, and heuristics

Unconstrained nonlinear

Functions Golden-section search Interpolation methods Line search Nelder–Mead method Successive parabolic interpolation Gradients Convergence Trust region Wolfe conditions Quasi–Newton Berndt–Hall–Hall–Hausman Broyden–Fletcher–Goldfarb–Shanno and L-BFGS Davidon–Fletcher–Powell Symmetric rank-one (SR1) Other methods Conjugate gradient Gauss–Newton Gradient Mirror Levenberg–Marquardt Powell's dog leg method Truncated Newton Hessians Newton's method

Constrained nonlinear
General Barrier methods Penalty methods Differentiable Augmented Lagrangian methods Sequential quadratic programming Successive linear programming

Convex optimization
Convex minimization Cutting-plane method Reduced gradient (Frank–Wolfe) Subgradient method Linear and quadratic Interior point Affine scaling Ellipsoid algorithm of Khachiyan Projective algorithm of Karmarkar Basis-exchange Simplex algorithm of Dantzig Revised simplex algorithm Criss-cross algorithm Principal pivoting algorithm of Lemke

Combinatorial
Paradigms Approximation algorithm Dynamic programming Greedy algorithm Integer programming Branch and bound/cut Graph algorithms Minimum spanning tree Borůvka Prim Kruskal Shortest path Bellman–Ford SPFA Dijkstra Floyd–Warshall Network flows Dinic Edmonds–Karp Ford–Fulkerson Push–relabel maximum flow

Metaheuristics
Evolutionary algorithm Hill climbing Local search Parallel metaheuristics Simulated annealing Spiral optimization algorithm Tabu search

• Software