Optimal Minimum L1-Norm Criteria for Outlier Identification in GNSS and Leveling Networks
Publication: Journal of Surveying Engineering
Volume 149, Issue 4
Abstract
The goal of this paper was to perform an exhaustive search regarding which combination of minimum L1-norm (MinL1) criteria is a better option for outlier identification. By means of Monte Carlo simulations (MCS), we compared the mean success rate (MSR) of combinations of three approaches for weight matrix two approaches for test statistics and two approaches for method of solution = 12 combinations of criteria for outlier identification with MinL1, in six geodetic network configurations. Six of them had never been evaluated via MCS, and four had never been tested in the literature. In general, the optimal combination was the adjustment with the same weights for the observations, normalized residual as the test statistic, and simplex method of solution (MinL1S - - simplex). In each case, we also computed the MSR of the iterative data-snooping (IDS) for reference. IDS presented MSR higher than all 12 MinL1 combinations in the four geodetic networks with mean redundancy numbers greater than or equal to 0.5. However, the MSR of the MinL1 optimal combination were 1.62 and 2.65 times higher than those of the IDS for the leveling and the global navigation satellite system (GNSS) networks with mean redundancy numbers less than 0.5, respectively. These results provide strong evidence that MinL1 with such criteria combinations has the potential to be the new state-of-the-art method for outlier identification in low redundancy GNSS and leveling networks.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or code generated or used during the study are available in a repository or online in accordance with funder data retention policies (https://github.com/stefano998/Optimal-MinL1-criteria).
Acknowledgments
This work was supported by the Departamento de Ciência e Tecnologia do Exército Brasileiro/Brasil. The third author thanks the CNPq—Conselho Nacional de Desenvolvimento Científico e Tecnológico/Brasil for its Research Productivity Grant (Process No. 313699/2021-6).
References
Abdelmalek, N., and W. Malek. 2008. Numerical linear approximation in C. London: CRC Press.
Amiri-Simkooei, A. 2003. “Formulation of L1 norm minimization in Gauss-Markov models.” J. Surv. Eng. 129 (1): 37–43. https://doi.org/10.1061/(ASCE)0733-9453(2003)129:1(37).
Amiri-Simkooei, A. 2018. “On the use of two L1 norm minimization methods in geodetic networks.” Earth Obs. Geomatics Eng. 2 (1): 1–8. https://doi.org/10.22059/eoge.2018.256034.1021.
Baarda, W. 1968. “A testing procedure for use in geodetic networks.” Neth. Geod. Comm. 2 (5): 1–97.
Baselga, S. 2007. “Global optimization solution of robust estimation.” J. Surv. Eng. 133 (3): 123–128. https://doi.org/10.1061/(ASCE)0733-9453(2007)133:3(123).
Baselga, S., I. Klein, S. S. Suraci, L. C. Oliveira, M. T. Matsuoka, and V. F. Rofatto. 2020. “Performance comparison of least squares, iterative and global L1 norm minimization and exhaustive search methods for outlier detection in leveling networks.” Acta Geodyn. Geomater. 17 (4): 425–438. https://doi.org/10.13168/AGG.2020.0031.
Bonimani, M. L. S., V. F. Rofatto, M. T. Matsuoka, I. Klein, M. R. Veronez, and L. G. D. Silveira Jr. 2021. “O Efeito das Covariâncias entre os Componentes de Linha Base sobre a Confiabilidade de Redes GNSS: Resultados para uma Rede com Alta Redundância.” Revista Brasileira de Cartografia 73 (2): 666–684. https://doi.org/10.14393/rbcv73n2-58105.
Dantzig, G. 1963. Linear programming and extensions. Princeton, NJ: Princeton University Press.
Erenoglu, R., and S. Hekimoglu. 2010. “Efficiency of robust methods and tests for outliers for geodetic adjustment models.” Acta Geod. Geophys. Hung. 45 (4): 426–439. https://doi.org/10.1556/AGeod.45.2010.4.3.
Gao, Y., E. J. Krakiwsky, and J. Czompo. 1992. “Robust testing procedure for detection of multiple blunders.” J. Surv. Eng. 118 (1): 11–23. https://doi.org/10.1061/(ASCE)0733-9453(1992)118:1(11).
Gemael, C. 1994. Introdução ao ajustamento de observações: Aplicações geodésicas. Curitiba, Brazil: Universidade Federal do Paraná.
Ghilani, C. D. 2010. Adjustment computations: Spatial data analysis. 5th ed. New York: Wiley.
Harrison, R. L. 2010. “Introduction to Monte Carlo simulation.” In Vol. 1204 of Proc., AIP Conf. Proc., 17–21. College Park, MD: American Institute of Physics.
Hekimoglu, S., and R. C. Erenoglu. 2007. “Effect of heteroscedasticity and heterogeneousness on outlier detection for geodetic networks.” J. Geod. 81 (2): 137–148. https://doi.org/10.1007/s00190-006-0095-z.
Imparato, D., P. Teunissen, and C. Tiberius. 2018. “Minimal detectable and identifiable biases for quality control.” Surv. Rev. 51 (367): 289–299. https://doi.org/10.1080/00396265.2018.1437947.
Inal, C., M. Yetkin, S. Bulbul, and B. Bilgen. 2018. “Comparison of L1 norm and L2 norm minimisation methods in trigonometric levelling networks.” Supplement, Tehnički Vjesnik 25 (S1): 216–221.
Junhuan, P. 2005. “The asymptotic variance–covariance matrix, Baarda test and the reliability of -norm estimates.” J. Geod. 78 (11): 668–682. https://doi.org/10.1007/s00190-004-0433-y.
Khodabandeh, A., and A. R. Amiri-Simkooei. 2011. “Recursive algorithm for L1 norm estimation in linear models.” J. Surv. Eng. 137 (1): 1–8. https://doi.org/10.1061/(ASCE)SU.1943-5428.0000031.
Klein, I. 2014. “Proposta de um novo método para o planejamento de redes geodésicas.” Ph.D. thesis, Programa de Pós-Graduação em Sensoriamento Remoto, Universidade Federal do Rio Grande do Sul.
Klein, I., S. S. Suraci, L. C. Oliveira, V. F. Rofatto, M. T. Matsuoka, and S. Baselga. 2022. “An attempt to analyse Iterative Data Snooping and L1-norm based on Monte Carlo simulation in the context of leveling networks.” Surv. Rev. 54 (382): 70–78. https://doi.org/10.1080/00396265.2021.1878338.
Kubat, M. 2017. An introduction to machine learning. 2nd ed. Cham, Switzerland: Springer.
Lehmann, R. 2012. “Improved critical values for extreme normalized and studentized residuals in Gauss–Markov models.” J. Geod. 86 (12): 1137–1146. https://doi.org/10.1007/s00190-012-0569-0.
Lehmann, R. 2013. “On the formulation of the alternative hypothesis for geodetic outlier detection.” J. Geod. 87 (4): 373–386. https://doi.org/10.1007/s00190-012-0607-y.
Marsaglia, G., and W. W. Tsang. 2000. “The Ziggurat method for generating random variables.” J. Stat. Software 5 (8): 1–7. https://doi.org/10.18637/jss.v005.i08.
Marshall, J. 2002. “L1-norm pre-analysis measures for geodetic networks.” J. Geod. 76 (6): 334–344. https://doi.org/10.1007/s00190-002-0254-9.
Marshall, J., and J. Bethel. 1996. “Basic concepts of norm minimization for surveying applications.” J. Surv. Eng. 122 (4): 168–179. https://doi.org/10.1061/(ASCE)0733-9453(1996)122:4(168).
Matsumoto, M., and T. Nishimura. 1998. “Mersenne twister: A 623-dimensionally equidistributed uniform pseudo-random number generator.” ACM Trans. Model. Comput. Simul. 8 (1): 3–30. https://doi.org/10.1145/272991.272995.
Raychaudhuri, S. 2008. “Introduction to Monte Carlo simulation.” In Proc., 2008 Winter Simulation Conf., 91–100. New York: IEEE. https://doi.org/10.1109/WSC.2008.4736059.
Rofatto, V. F., M. T. Matsuoka, I. Klein, M. L. Bonimani, B. P. Rodrigues, C. C. Campos, M. R. Veronez, and L. G. Silveira Jr. 2022. “An artificial neural network-based critical values for multiple hypothesis testing: Data-snooping case.” Surv. Rev. 54 (386): 440–455. https://doi.org/10.1080/00396265.2021.1968176.
Rofatto, V. F., M. T. Matsuoka, I. Klein, M. R. Veronez, M. L. Bonimani, and R. Lehmann. 2020a. “A half-century of Baarda’s concept of reliability: A review, new perspectives, and applications.” Surv. Rev. 52 (372): 261–277. https://doi.org/10.1080/00396265.2018.1548118.
Rofatto, V. F., M. T. Matsuoka, I. Klein, M. R. Veronez, and L. G. Silveira. 2020b. “A Monte Carlo-based outlier diagnosis method for sensitivity analysis.” Remote Sens. 12 (5): 860. https://doi.org/10.3390/rs12050860.
Schwarz, C. R., and J. J. Kok. 1993. “Blunder detection and data snooping in LS and robust adjustments.” J. Surv. Eng. 119 (4): 127–136. https://doi.org/10.1061/(ASCE)0733-9453(1993)119:4(127).
Suraci, S. S., R. R. Goldschmidt, L. C. Oliveira, and I. Klein. 2021a. “A meta-classifier approach for outlier identification in geodetic networks.” In Proc., 22nd Brazilian Symp. on GeoInformatics, 210–215. São José dos Campos, Brazil: National Institute for Space Research.
Suraci, S. S., and L. C. Oliveira. 2020. “Aplicação das normas L1 e L∞ em redes altimétricas: Identificação de outliers e construção do modelo estocástico.” Revista Cartográfica 101 (Dec): 135–153. https://doi.org/10.35424/rcarto.i101.669.
Suraci, S. S., L. C. Oliveira, and I. Klein. 2019. “Two aspects on L1-norm adjustment of leveling networks.” Revista Brasileira de Cartografia 71 (2): 486–500. https://doi.org/10.14393/rbcv71n2-47697.
Suraci, S. S., L. C. Oliveira, I. Klein, and R. R. Goldschmidt. 2022a. “Critérios ótimos para identificação de outliers com o estimador da norma-L1 mínima: Estudo de caso em uma rede GNSS de baixa redundância.”In Proc., 12th Colóquio Brasileiro de Ciências Geodésicas, 470–472. Curitiba, Brazil: Universidade Federal do Paraná.
Suraci, S. S., L. C. Oliveira, I. Klein, and R. R. Goldschmidt. 2022b. “Introducing covariances of observations in the minimum L1-norm, is it needed?” J. Geod. Sci. 12 (1): 65–74. https://doi.org/10.1515/jogs-2022-0135.
Suraci, S. S., L. C. Oliveira, I. Klein, V. F. Rofatto, M. T. Matsuoka, and S. Baselga. 2021b. “Monte Carlo-based covariance matrix of residuals and critical values in minimum L1-norm.” Math. Probl. Eng. 2021 (Jun): 1. https://doi.org/10.1155/2021/8123493.
Taboga, M. 2021. “Linear combinations of normal random variables.” StatLect. Accessed September 6, 2022. https://www.statlect.com/probability-distributions/normal-distribution-linear-combinations.
Teunissen, P. J. G. 2006. Testing theory: An introduction. 2nd ed. Delft, Netherlands: Delft University Press.
Teunissen, P. J. G. 2018. “Distributional theory for the DIA method.” J. Geod. 92 (Jan): 59–80. https://doi.org/10.1007/s00190-017-1045-7.
Valero, J. L. B., and S. B. Moreno. 2005. “Robust estimation in geodetic networks.” Física de la Tierra 17 (Mar): 7–22.
Witten, I. H., and E. Frank. 2005. Data mining: Practical machine learning tools and techniques. 2nd ed. Amsterdam, Netherlands: Elsevier.
Xu, P. 2005. “Sign-constrained robust least squares, subjective breakdown point and the effect of weights of observations on robustness.” J. Geod. 79 (Jun): 146–159. https://doi.org/10.1007/s00190-005-0454-1.
Yetkin, M., and C. Inal. 2011. “L1 norm minimization in GPS networks.” Surv. Rev. 43 (323): 523–532. https://doi.org/10.1179/003962611X13117748892038.
Information & Authors
Information
Published In
Journal of Surveying Engineering
Volume 149 • Issue 4 • November 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Mar 3, 2023
Accepted: Jun 23, 2023
Published online: Aug 30, 2023
Published in print: Nov 1, 2023
Discussion open until: Jan 30, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Yeqing Tao, Xinchuan Li, Hao Chen, Juan Yang, Solution for the Robust Estimation of Heterogeneous Data Fusion Based on Classification Estimation, Journal of Surveying Engineering, 10.1061/JSUED2.SUENG-1492, 150, 3, (2024).