Statistical Consultants Ltd |
M-estimatorsStatistical Analysis Techniques, Robust Estimators, Alternatives to OLS The three main classes of robust estimators are M, L and R. Robust estimators are resistant to outliers and when used in regression modelling, are robust to departures from the normality assumption. M-estimators are a maximum likelihood type estimator. M estimation involves minimizing the following: Where ρ is some function with the following properties:
For For OLS: ρ(r) = r^{2} Note that OLS doesn’t satisfy the third property, therefore it doesn’t count as a robust M-estimator. In the case of a linear model, the function to minimise will be: Instead of minimising the function directly, it may be simpler to use the function’s first order conditions set to zero: where: If the ρ function can be differentiated, the M-estimator is said to be a ψ-type. Otherwise, the M-estimator is said to be a ρ-type. L_{p}
subclass
L_{p }is a subclass of M estimators. An L_{p }beta
coefficient would be
one that minimises the following:Where 1≤ p ≤2. If p=1, it is the equivalent of The lower p is, the more robust the L_{p }will be to outliers. The lower p is, the greater the number of iterations would be needed for the sum of |r|^{p} to converge at the minimum. Tukey’s bisquare
M-estimator
Tukey proposed an M-estimator that has the following ρ(z_{i})
function:Where c is a constant and , where s is the estimated scale parameter. Tukey’s bisquare psi
function leaves out any extreme
outliers by giving them a zero weighting. Huber's M-estimator
Where c is a constant and , where s is the estimated scale parameter. It essentially applies an Andrews's M-estimator
Where , where s is the estimated scale parameter. See
also:
Regression
L-estimators R-estimators |
Copyright © Statistical Consultants Ltd 2010 |