Robustifying Conventional Outlier Detection Procedures
Publication: Journal of Surveying Engineering
Volume 125, Issue 2
Abstract
The conventional outlier detection procedures, such as the methods of Baarda and Pope or the t-testing procedure, determine only one outlier reliably. The approach to robustifying these procedures is as follows: (1) To identify outliers by using an estimator that has a high breakdown point and a bounded influence function; (2) to find “good observations” by separating outliers from whole observations; (3) to constitute the reduced samples obtained by systematically adding each single outlier in turn to the good observations; and (4) to apply the conventional outlier detection procedures to each single reduced sample separately. To test the approach, an M-estimator with Andrews weight function is chosen. Then it is studied using a coordinate transformation simulation. Only two outliers are able to be determined reliably.
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
Andrews, D. ( 1974). “A robust method for multiple linear regression.” Technometrics, 16, 523–531.
2.
Baarda, W. ( 1968). “A testing procedure for use in geodetic networks.” Netherlands Geod Comm., Publ. on Geodesy, New Ser., 2(5).
3.
Chatterjee, S., and Hadi, A. S. ( 1988). Sensitivity analysis in linear regression. Wiley, New York.
4.
Chen, Y. Q., Kavouras, M., and Chrzanowski, A. ( 1987). “A strategy for detection of outlying observations in measurements of high precision.” Can. Surveyor, 41, 529–540.
5.
Coakley, C. W., and Hettmansperger, T. P. ( 1993). “A bounded influence, high breakdown, efficient regression estimator.” J. Am. Stat. Assn., 88, 872–880.
6.
Davies, L., and Gather, U. ( 1993). “The identification of multiple outliers.” J. Am. Stat. Assn., 88, 782–792.
7.
Donoho, D. L., and Huber, P. J. ( 1983). “The notion of breakdown point.” A Festschrift for Erich Lehmann, P. J. Bickel, K. A. Doksum, and J. L. Hodges Jr., eds., Wadsworth, Belmont, Calif., 157–184.
8.
Hampel, F. ( 1968). “Contributions to the theory of robust estimation,” PhD dissertation, University of California, Berkeley, Calif.
9.
Hampel, F., Ronchetti, E., Rousseuw, P., and Stahel, W. ( 1986). Robust statistics: The approach based on influence functions. Wiley, New York.
10.
Hadi, A. S., and Siminoff, J. S. ( 1993). “Procedures for the identification of multiple outliers in linear models.” J. Am. Stat. Assn., 88, 1264–1272.
11.
Heck, B. ( 1981). “Der Einfluss einzelner Beobachtungen auf das Ergebnis einer Ausgleichung und die Suche nach Ausreissern in den Beobachtungen.” AVN, 88, 17–34 (in German).
12.
Hekimoğlu, Ş. (1997). “The finite sample breakdown points of the conventional iterative outlier detection procedures.”J. Surv. Engrg., ASCE, 123(1), 15–31.
13.
Huber, P. J. ( 1964). “Robust estimation of a location parameter.” Ann. Math. Statistics, 35, 73–101.
14.
Huber, P. J. ( 1981). Robust statistics. Wiley, New York, N.Y.
15.
Kampmann, G. ( 1994). “Robuste Deformationsanalyse mittels balancierter Ausgleichung.” AVN, 101, 8–17 (in German).
16.
Koch, K. R. ( 1987). Parameter estimation and hypothesis testing in linear models. Springer, New York.
17.
Kok, J. J. ( 1984). “On data snooping and multiple outlier testing.” NOAA Tech. Rep. NOS NGS 30, Nat. Oc. Service, Nat. Geodetic Survey, U.S. Dept. of Commerce, Rockville, Md.
18.
Markatou, M., and Hettmansperger, T. P. ( 1990). “Robust bounded-influence tests in linear models.” J. Am. Stat. Assn., 85, 187–190.
19.
Markatou, M., Stahel, W. A., and Ronchetti, E. ( 1991). “Robust M-type testing procedures for linear models.” Directions in robust statistics and diagnostics, Part 1, W. Stahel and S. Weisberg, eds., Springer, New York, 201–220.
20.
Markatou, M., and He, X. ( 1994). “Bounded influence and high breakdown point testing procedures in linear models.” J. Am. Stat. Assn., 89, 543–549.
21.
Pope, A. J. ( 1976). “The statistics of residuals and the detection of outliers.” NOAA Tech. Rep. NOS 65 NGS 1, U.S. Dept. of Commerce, Rockville, Md.
22.
Rousseeuw, P. J. ( 1984). “Least median of squares regression.” J. Am. Stat. Assn., 79, 871–880.
23.
Rousseeuw, P. J., and Leroy, A. M. ( 1987). Robust regression and outlier detection. Wiley, New York.
24.
Simpson, D. G., Ruppert, D., and Carroll, R. J. ( 1992). “On one-step GM estimates and stability of inferences in linear regression.” J. Am. Stat. Assn., 87, 439–450.
25.
Xu, P. L. ( 1993). “Consequences of constant parameters and confidence intervals of robust estimation.” Anno LII-Bollettino Di Geodesia e Scienze Affini-N.3.
26.
Ylvisaker, D. ( 1977). “Test resistance.” J. Am. Stat. Assn., 72, 551–557.
27.
Yohai, V. J. ( 1987). “High breakdown point and high efficiency robust estimates for regression.” Ann. of Statistics, 15, 642–656.
Information & Authors
Information
Published In
Journal of Surveying Engineering
Volume 125 • Issue 2 • May 1999
Pages: 69 - 86
History
Published online: May 1, 1999
Published in print: May 1999
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.