Data Snooping for the Equality Constrained Nonlinear Gauss–Helmert Model Using Sensitivity Analysis
This article has been corrected.
VIEW CORRECTIONPublication: Journal of Surveying Engineering
Volume 146, Issue 4
Abstract
To develop a universal-outliers processing algorithm under the conditions with equality constraints, the equality-constrained nonlinear Gauss–Helmert (GH) model, which contains the equality-constrained Gauss–Markov (GM) and errors-in-variables (EIV) models as special cases, is selected as the research object in this paper. The least squares solution for the nonlinear GH model with equality constraints is obtained using the Euler–Lagrange approach, and then, it is equivalently formulated as the standard constrained least squares (CLS) problem. To construct the test statistics for the outliers detection, a distinctive sensitivity analysis approach is introduced into this CLS problem. The local sensitivity of the weighted sum of squared residuals to the perturbations of observations in the CLS problem is discussed, and then, the local test statistics are constructed based on these sensitivity indicators. To verify the performance of the sensitivity-based test statistics, the proposed data-snooping algorithm for the equality-constrained nonlinear GH model is applied to a three-dimensional (3D) symmetric similarity transformation. The computational results of the simulated and real examples manifest that the proposed data-snooping algorithm using the sensitivity-based test statistics can effectually decrease the negative impact of the outliers and derive reliable parameters. It should be pointed out that the new algorithm is applicable in various kinds of equality-constrained least squares and total least squares problems.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
We greatly thank two anonymous reviewers for their constructive comments. This research was supported by the National Natural Science Foundation of China (No. 41774009) and the Natural Science Foundation of Jiangsu Province (No. BK20180720). All data, models, and code generated or used during the study appear in the published article.
References
Akyilmaz, O. 2007. “Total least-squares solution of coordinate transformation.” Surv. Rev. 39 (303): 68–80. https://doi.org/10.1179/003962607X165005.
Akyilmaz, O. 2010. A new approach to geodetic datum transformations. [Unpublished Presentation In Turkish.] Istanbul, Turkey: Prof. Dr. Tevfik Ayan Geodesy Colloquium, Faculty of Civil Engineering, Istanbul Technical Univ.
Akyilmaz, O. 2011. “Solution of the heteroscedastic datum transformation problems.” In Proc., Abstracts of the 1st Int. Workshop on the Quality of Geodetic Observation and Monitoring Systems QuGOMS. New York: Springer.
Amiri-Simkooei, A. 2017. “Weighted total least squares with singular covariance matrices subject to weighted and hard constraints.” J. Surv. Eng. 143 (4): 04017018. https://doi.org/10.1061/(ASCE)SU.1943-5428.0000239.
Amiri-Simkooei, A. 2018. “Parameter estimation in 3D affine and similarity transformation: Implementation of variance component estimation.” J. Geod. 92 (11): 1285–1297. https://doi.org/10.1007/s00190-018-1119-1.
Amiri-Simkooei, A., and S. Jazaeri. 2012. “Weighted total least squares formulated by standard least squares theory.” J. Geod. Sci. 2 (2): 113–124. https://doi.org/10.2478/v10156-011-0036-5.
Amiri-Simkooei, A., and S. Jazaeri. 2013. “Data-snooping procedure applied to errors-in-variables models.” Stud. Geophys. Geod. 57 (3): 426–441. https://doi.org/10.1007/s11200-012-0474-2.
Baarda, W. 1967. “Statistical concepts in geodesy.” In Vol. of 2 Publications on Geodesy. Delft, Netherlands: Geodetic Commission.
Baarda, W. 1968. “A testing procedure for use in geodetic networks.” In Vol. of 2 Publications on Geodesy. Delft, Netherlands: Geodetic Commission.
Baselga, S. 2007. “Critical limitation in use of τ test for gross error detection.” J. Surv. Eng. 133 (2): 52–55. https://doi.org/10.1061/(ASCE)0733-9453(2007)133:2(52).
Beck, A., and A. Ben-Tal. 2006. “On the solution of the Tikhonov regularization of the total least-squares.” SIAM J. Optim. 17 (1): 98–118. https://doi.org/10.1137/050624418.
Box, M. J. 1971. “Bias in nonlinear estimation.” J. R. Stat. Soc. B 33 (2): 171–201. https://doi.org/10.1111/j.2517-6161.1971.tb00871.x.
Chang, G. B. 2015. “On least-squares solution to 3D similarity transformation problem under Gauss–Helmert model.” J. Geod. 89 (6): 573–576. https://doi.org/10.1007/s00190-015-0799-z.
Chang, G. B. 2016. “Closed form least-squares solution to 3D symmetric Helmert transformation with rotational invariant covariance structure.” Acta Geod. Geophys. 51 (2): 237–244. https://doi.org/10.1007/s40328-015-0123-7.
Chang, G. B., P. Lin, H. F. Bian, and J. X. Gao. 2018a. “Simultaneous Helmert transformations among multiple frames considering all relevant measurements.” Meas. Sci. Technol. 29 (3): 035801. https://doi.org/10.1088/1361-6501/aaa03a.
Chang, G. B., T. H. Xu, and Q. X. Wang. 2018b. “M-estimator for the 3D symmetric Helmert coordinate transformation.” J. Geod. 92 (1): 47–58. https://doi.org/10.1007/s00190-017-1043-9.
Dowling, E. M., R. D. Degroat, and D. A. Linebarger. 1992. “Total least squares with linear constraints.” In Vol. 5 of Proc., Acoustics Speech Signal Processing (ICASSP-92), 341–344. New York: IEEE.
Fang, X. 2011. “Weighted total least squares solution for application in geodesy.” Ph.D. dissertation, Dept. of Institute of Geodesy, Leibniz Univ.
Fang, X. 2013. “Weighted total least squares: Necessary and sufficient conditions, fixed and random parameters.” J. Geod. 87 (8): 733–749. https://doi.org/10.1007/s00190-013-0643-2.
Fang, X. 2015. “Weighted total least-squares with constraints: A universal formula for geodetic symmetrical transformations.” J. Geod. 89 (5): 459–469. https://doi.org/10.1007/s00190-015-0790-8.
Felus, Y. A., and R. C. Burtch. 2009. “On symmetrical three-dimensional datum conversion.” GPS Solut. 13 (1): 65–74. https://doi.org/10.1007/s10291-008-0100-5.
Golub, G. H., P. C. Hansen, and D. P. O’Leary. 1999. “Tikhonov regularization and total least-squares.” SIAM J. Matrix Anal. Appl. 21 (1): 185–194. https://doi.org/10.1137/S0895479897326432.
Golub, G. H., and C. F. van Loan. 1980. “An analysis of the total least squares problem.” SIAM J. Numer. Anal. 17 (6): 883–893. https://doi.org/10.1137/0717073.
Guo, J. F. 2007. “Theory of model errors and its applications in GPS data processing.” [In Chinese.] Ph.D. thesis, Institute of Geodesy and Geophysics, Chinese Academy of Sciences.
Guo, J. F., J. K. Ou, and H. T. Wang. 2007. “Quasi-accurate detection of outliers for correlated observations.” J. Surv. Eng. 133 (3): 129–133. https://doi.org/10.1061/(ASCE)0733-9453(2007)133:3(129).
Guo, J. F., J. K. Ou, and H. T. Wang. 2010. “Robust estimation for correlated observations: Two local sensitivity-based downweighting strategies.” J. Geod. 84 (4): 243–250. https://doi.org/10.1007/s00190-009-0361-y.
Hekimoglu, S. 1997. “Finite sample breakdown points of outlier detection procedures.” J. Surv. Eng. 123 (1): 15–31. https://doi.org/10.1061/(ASCE)0733-9453(1997)123:1(15).
Huber, P. J. 1964. “Robust estimation of a location parameter.” Ann. Math. Stat. 35 (1): 73–101. https://doi.org/10.1214/aoms/1177703732.
Jazaeri, S., and A. Amiri-Simkooei. 2015. “Weighted total least squares for solving non-linear problem: GNSS point positioning.” Surv. Rev. 47 (343): 265–271. https://doi.org/10.1179/1752270614Y.0000000132.
Klein, I., M. Matsuoka, M. Guzatto, and F. Nievinski. 2017. “An approach to identify multiple outliers based on sequential likelihood ratio tests.” Surv. Rev. 49 (357): 449–457. https://doi.org/10.1080/00396265.2016.1212970.
Koch, K. R. 1999. Parameter estimation and hypothesis testing in linear models. 2nd ed. Berlin: Springer.
Koch, K. R. 2014. “Robust estimations for the nonlinear Gauss Helmert model by the expectation maximization algorithm.” J. Geod. 88 (3): 263–271. https://doi.org/10.1007/s00190-013-0681-9.
Lehmann, R. 2012. “Improved critical values for extreme normalized and studentized residuals in Gauss–Markov models.” J. Geod. 86 (12): 1137–1146. https://doi.org/10.1007/s00190-012-0569-0.
Lehmann, R. 2013. “On the formulation of the alternative hypothesis for geodetic outlier detection.” J. Geod. 87 (4): 373–386. https://doi.org/10.1007/s00190-012-0607-y.
Li, B., M. Wang, and Y. Yang. 2017. “Multiple linear regression with correlated explanatory variables and responses.” Surv. Rev. 49 (352): 1–8. https://doi.org/10.1179/1752270615Y.0000000006.
Mahboub, V., A. R. Amiri-Simkooei, and M. A. Sharifi. 2013. “Iteratively reweighted total least squares: A robust estimation in errors-in-variables models.” Surv. Rev. 45 (329): 92–99.
Mahboub, V., and M. A. Sharifi. 2013. “On weighted total least-squares with linear and quadratic constraints.” J. Geod. 87 (3): 279–286. https://doi.org/10.1007/s00190-012-0598-8.
Mercan, H., O. Akyilmaz, and C. Aydin. 2018. “Solution of the weighted symmetric similarity transformations based on quaternions.” J. Geod. 92 (10): 1113–1130. https://doi.org/10.1007/s00190-017-1104-0.
Neitzel, F. 2010. “Generalization of total least-squares on example of unweighted and weighted 2D similarity transformation.” J. Geod. 84 (12): 751–762. https://doi.org/10.1007/s00190-010-0408-0.
Pope, A. J. 1976. The statistics of residuals and the detection of outliers. Rockville, MD: US Dept. of Commerce, National Geodetic Survey.
Rofatto, V., M. Matsuoka, and I. Klein. 2017. “An attempt to analyse Baarda’s iterative data snooping procedure based on Monte Carlo simulation.” S. Afr. J. Geomatics 6 (3): 416–435. https://doi.org/10.4314/sajg.v6i3.11.
Rofatto, V., M. Matsuoka, I. Klein, M. Veronez, M. Bonimani, and R. Lehmann. 2018. “A half-century of Baarda’s concept of reliability: A review, new perspectives, and applications.” Surv. Rev. 52 (372): 261–277. https://doi.org/10.1080/00396265.2018.1548118.
Rousseeuw, P. J., and A. M. Leroy. 1987. Robust regression and outlier detection. New York: Wiley.
Schaffrin, B., and Y. Felus. 2009. “An algorithmic approach to the total least squares problem with linear and quadratic constraints.” Stud. Geophys. Geod. 53 (1): 1–16. https://doi.org/10.1007/s11200-009-0001-2.
Schaffrin, B., and A. Wieser. 2008. “On weighted total least-squares adjustment for linear regression.” J. Geod. 82 (7): 415–421. https://doi.org/10.1007/s00190-007-0190-9.
Shen, Y. Z., B. F. Li, and Y. Chen. 2011. “An iterative solution of weighted total least squares adjustment.” J. Geod. 85 (4): 229–238. https://doi.org/10.1007/s00190-010-0431-1.
Sima, D. M., S. van Huffel, and G. H. Golub. 2004. “Regularized total least squares based on quadratic eigenvalue problem solver.” BIT Numer. Math. 44 (4): 793–812. https://doi.org/10.1007/s10543-004-6024-8.
Teunissen, P. J. G. 1985. “The geometry of geodetic inverse linear mapping and nonlinear adjustment.” In Vol. of 8 Publications on geodesy. Delft, Netherlands: Geodetic Commission.
Teunissen, P. J. G. 2018. “Distributional theory for the DIA method.” J. Geod. 92 (1): 59–80. https://doi.org/10.1007/s00190-017-1045-7.
Van Huffel, S., and J. Vandewalle. 1991. The total least-squares problem: Computational aspects and analysis. Philadelphia: Society for Industrial and Applied Mathematics.
Wang, B., J. C. Li, and C. Liu. 2016. “A robust weighted total least squares algorithm and its geodetic applications.” Stud. Geophys. Geod. 60 (2): 177–194. https://doi.org/10.1007/s11200-015-0916-8.
Wang, B., J. C. Li, C. Liu, and J. Yu. 2017. “Generalized total least squares prediction algorithm for universal 3D similarity transformation.” Adv. Space Res. 59 (3): 815–823. https://doi.org/10.1016/j.asr.2016.09.018.
Wang, B., J. Yu, C. Liu, M. F. Li, and B. Zhu. 2018. “Data snooping algorithm for universal 3D similarity transformation based on generalized EIV model.” Measurement 119 (Apr): 56–62. https://doi.org/10.1016/j.measurement.2018.01.040.
Xu, P. L., J. N. Liu, and C. Shi. 2012. “Total least squares adjustment in partial errors-in-variables models: Algorithm and statistical analysis.” J. Geod. 86 (8): 661–675. https://doi.org/10.1007/s00190-012-0552-9.
Yang, L., J. Wang, N. Knight, and Y. Shen. 2013. “Outlier separability analysis with a multiple alternative hypotheses test.” J. Geod. 87 (6): 591–604. https://doi.org/10.1007/s00190-013-0629-0.
Yeredor, A. 2006. “On the role of constraints in system identification.” In Proc., 4th Int. Workshop on Total Least-Squares and Errors-in-Variables Modeling, edited by S. van Huffel and I. Markovsky, 46–48. Amsterdam, Netherlands: Elsevier Science.
Zaminpardaz, S., and P. J. G. Teunissen. 2019. “DIA-datasnooping and identifiability.” J. Geod. 93 (1): 85–101. https://doi.org/10.1007/s00190-018-1141-3.
Zhou, Y. J., X. J. Kou, J. Li, and X. Fang. 2017. “Comparison of structured and weighted total least-squares adjustment methods for linearly structured errors-in-variables models.” J. Surv. Eng. 143 (1): 04016019. https://doi.org/10.1061/(ASCE)SU.1943-5428.0000190.
Zhou, Y. J., X. J. Kou, J. J. Zhu, and J. Li. 2014. “A Newton algorithm for weighted total least squares solution to a specific errors-in-variables model with correlated measurements.” Stud. Geophys. Geod. 58 (3): 349–375. https://doi.org/10.1007/s11200-013-0254-7.
Information & Authors
Information
Published In
Journal of Surveying Engineering
Volume 146 • Issue 4 • November 2020
Copyright
©2020 American Society of Civil Engineers.
History
Received: Sep 2, 2019
Accepted: Mar 31, 2020
Published online: Jun 24, 2020
Published in print: Nov 1, 2020
Discussion open until: Nov 24, 2020
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.