Dense Point Cloud Quality Factor as Proxy for Accuracy Assessment of Image-Based 3D Reconstruction
Publication: Journal of Surveying Engineering
Volume 147, Issue 1
Abstract
Photogrammetry using structure from motion (SfM) and multiview stereopsis (MVS) techniques can recover three-dimensional (3D) structure from a set of overlapping, unoriented, and uncalibrated images captured by nonmetric digital cameras. It is possible to generate accurate reconstructions of sparse points using mathematically robust bundle adjustment procedures together with accurate surveying control data. However, MVS, which recovers the dense geometry by matching and expanding between sparse points, is prone to additional error. Miscellaneous constituents such as sensor specifications, data collection, and site conditions can introduce random noise or artifacts that locally degrade the accuracy of the dense point cloud. This paper proposes seven indexes, named dense point cloud quality factors (DPQFs), as proxy indicators of image-based dense reconstruction accuracy. DPQFs include proximity to keypoint features, distance to GCPs, angle of incidence, camera stand-off distances, number of overlapping images, brightness index, and darkness index. The correlation between the DPQFs and the 3D error was investigated in simulated and empirical experiments scenarios with varying factors. The results of this study showed that the DPQFs provide proxy indications for accuracy when the error estimation for the dense point clouds is more challenging than error propagation computations in bundle adjustment (BA). The DPQFs can be defined solely using the SfM-MVS data, without prior knowledge about the error. Inclusion of the factors as additional fields of information and their visualization provide tangible intuitions regarding the factors that influence the accuracy of image-based 3D reconstruction.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code generated or used during the study are available from the corresponding author by request.
Acknowledgments
Funding for this project was provided internally by the School of Civil and Construction Engineering (CCE) at Oregon State University (OSU). The flights for the empirical case study were authorized by the FAA under license 2016-WSA-101-COA issued to OSU. The data set used for experimental data also were used for a conference publication assessing the use of UAS-based photogrammetry for generating civil integrated management (CIM) models (Javadnejad et al. 2017a). The authors thank Chase Simpson, Matthew Gillins, and Brian Weaver for their help with the field work, and Stephanie Taylor and Kevin Hamilton from Linn County Road Department, Oregon, for providing access to the site. The authors also thank Leica Geosystems, David Evans and Associates, and MicroSurvey for providing surveying equipment and/or software utilized in the experimental case study.
References
Abellan, A., M.-H. Derron, and M. Jaboyedoff. 2016. “‘Use of 3D point clouds in geohazards’ special issue: Current challenges and future trends.” Remote Sens. 8 (2): 130. https://doi.org/10.3390/rs8020130.
Agüera-Vega, F., F. Carvajal-Ramírez, and P. Martínez-Carricondo. 2017. “Accuracy of digital surface models and orthophotos derived from unmanned aerial vehicle photogrammetry.” J. Surv. Eng. 143 (2): 04016025. https://doi.org/10.1061/(ASCE)SU.1943-5428.0000206.
Anderson, M., R. Motta, S. Chandrasekar, and M. Stokes. 1996. “Proposal for a standard default color space for the internet—sRGB.” In Proc., Color and Imaging Conf., 238–245. Springfield, VA: Society for Imaging Science and Technology.
Bolkas, D. 2019. “Assessment of GCP number and separation distance for small UAS surveys with and without GNSS-PPK positioning.” J. Surv. Eng. 145 (3): 04019007. https://doi.org/10.1061/(ASCE)SU.1943-5428.0000283.
Calì, M., and R. Ambu. 2018. “Advanced 3D photogrammetric surface reconstruction of extensive objects by UAV camera image acquisition.” Sensors 18 (9): 2815. https://doi.org/10.3390/s18092815.
Carbonneau, P. E., and J. T. Dietrich. 2017. “Cost-effective non-metric photogrammetry from consumer-grade sUAS: Implications for direct georeferencing of structure from motion photogrammetry.” Earth Surf. Processes Landforms 42 (3): 473–486. https://doi.org/10.1002/esp.4012.
Che, E., and M. J. Olsen. 2017. “Fast ground filtering for TLS data via scanline density analysis.” ISPRS J. Photogramm. Remote Sens. 129 (Jul): 226–240. https://doi.org/10.1016/j.isprsjprs.2017.05.006.
Chudley, T. R., P. Christoffersen, S. H. Doyle, A. Abellan, and N. Snooke. 2019. “High-accuracy UAV photogrammetry of ice sheet dynamics with no ground control.” Cryosphere 13 (3): 955–968. https://doi.org/10.5194/tc-13-955-2019.
Clapuyt, F., V. Vanacker, and K. Van Oost. 2016. “Reproducibility of UAV-based earth topography reconstructions based on Structure-from-Motion algorithms.” Geomorphology 260 (May): 4–15. https://doi.org/10.1016/j.geomorph.2015.05.011.
Colomina, I., and P. Molina. 2014. “Unmanned aerial systems for photogrammetry and remote sensing: A review.” ISPRS J. Photogramm. Remote Sens. 92 (Jun): 79–97. https://doi.org/10.1016/j.isprsjprs.2014.02.013.
Crandall, D. J., A. Owens, N. Snavely, and D. P. Huttenlocher. 2013. “SfM with MRFs: Discrete-continuous optimization for large-scale structure from motion.” IEEE Trans. Pattern Anal. Mach. Intell. 35 (12): 2841–2853. https://doi.org/10.1109/TPAMI.2012.218.
Dandois, J., M. Olano, and E. Ellis. 2015. “Optimal altitude, overlap, and weather conditions for computer vision UAV estimates of forest structure.” Remote Sens. 7 (10): 13895–13920. https://doi.org/10.3390/rs71013895.
Eltner, A., A. Kaiser, C. Castillo, G. Rock, F. Neugirg, and A. Abellán. 2016. “Image-based surface reconstruction in geomorphometry—Merits, limits and developments.” Earth Surf. Dyn. 4 (2): 359–389. https://doi.org/10.5194/esurf-4-359-2016.
Faraji, M. R., X. Qi, and A. Jensen. 2016. “Computer vision–based orthorectification and georeferencing of aerial image sets.” J. Appl. Remote Sens. 10 (3): 036027. https://doi.org/10.1117/1.JRS.10.036027.
Fischler, M. A., and R. C. Bolles. 1981. “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography.” Commun. ACM 24 (6): 381–395. https://doi.org/10.1145/358669.358692.
Fonstad, M. A., J. T. Dietrich, B. C. Courville, J. L. Jensen, and P. E. Carbonneau. 2013. “Topographic structure from motion: A new development in photogrammetric measurement.” Earth Surf. Processes Landforms 38 (4): 421–430. https://doi.org/10.1002/esp.3366.
Furukawa, Y., and J. Ponce. 2010. “Accurate, dense, and robust multiview stereopsis.” IEEE Trans. Pattern Anal. Mach. Intell. 32 (8): 1362–1376. https://doi.org/10.1109/TPAMI.2009.161.
Gao, M., X. Xu, Y. Klinger, J. van der Woerd, and P. Tapponnier. 2017. “High-resolution mapping based on an unmanned aerial vehicle (UAV) to capture paleoseismic offsets along the Altyn-Tagh fault, China.” Sci. Rep. 7 (1): 8281. https://doi.org/10.1038/s41598-017-08119-2.
González-Quiñones, J., J. Reinoso-Gordo, C. León-Robles, J. García-Balboa, and F. Ariza-López. 2018. “Variables influencing the accuracy of 3D modeling of existing roads using consumer cameras in aerial photogrammetry.” Sensors 18 (11): 3880. https://doi.org/10.3390/s18113880.
Griffin, G. F. 2014. “The use of unmanned aerial vehicles for disaster management.” Geomatica 68 (4): 265–281. https://doi.org/10.5623/cig2014-402.
Harris, C., and M. Stephens. 1988. “A combined corner and edge detector.” In Vol. 15 of Proc., Alvey Vision Conf. Oxen, UK: Alvey Vision Club.
Harwin, S., and A. Lucieer. 2012. “Assessing the accuracy of georeferenced point clouds produced via multi-view stereopsis from unmanned aerial vehicle (UAV) imagery.” Remote Sens. 4 (6): 1573–1599. https://doi.org/10.3390/rs4061573.
Harwin, S., A. Lucieer, and J. Osborn. 2015. “The impact of the calibration method on the accuracy of point clouds derived using unmanned aerial vehicle multi-view stereopsis.” Remote Sens. 7 (9): 11933–11953. https://doi.org/10.3390/rs70911933.
Hofsetz, C., K. Ng, G. Chen, P. McGuinness, N. Max, and Y. Liu. 2004. “Image-based rendering of range data with estimated depth uncertainty.” IEEE Comput. Graphics Appl. 24 (4): 34–41. https://doi.org/10.1109/MCG.2004.8.
James, M. R., S. Robson, S. D’Oleire-Oltmanns, and U. Niethammer. 2017a. “Optimising UAV topographic surveys processed with structure-from-motion: Ground control quality, quantity and bundle adjustment.” Geomorphology 280 (Mar): 51–66. https://doi.org/10.1016/j.geomorph.2016.11.021.
James, M. R., S. Robson, and M. W. Smith. 2017b. “3-D uncertainty-based topographic change detection with structure-from-motion photogrammetry: Precision maps for ground control and directly georeferenced surveys.” Earth Surf. Processes Landforms 42 (12): 1769–1788. https://doi.org/10.1002/esp.4125.
Javadnejad, F. 2018. “Small unmanned aircraft systems (UAS) for engineering inspections and geospatial mapping.” Ph.D. dissertation, School of Civil and Construction Engineering, Oregon State Univ.
Javadnejad, F., and D. T. Gillins. 2016. “Unmanned aircraft systems-based photogrammetry for ground movement monitoring.” In Proc., Pipelines 2016, 1000–1011. Reston, VA: ASCE.
Javadnejad, F., D. T. Gillins, C. E. Parrish, and R. K. Slocum. 2019. “A photogrammetric approach to fusing natural colour and thermal infrared UAS imagery in 3D point cloud generation.” Int. J. Remote Sens. 41 (1): 211–237. https://doi.org/10.1080/01431161.2019.1641241.
Javadnejad, F., M. N. Gillins, and D. T. Gillins. 2016. “Vertical accuracy assessment of image-based reconstructed 3D point clouds with respect to horizontal ground sampling distance.” In Proc., ASPRS Annual Conf., Imaging & Geospatial Technology Forum (IGTF) 2016. Fort Worth, TX: American Society for Photogrammetry and Remote Sensing.
Javadnejad, F., C. H. Simpson, D. T. Gillins, T. Claxton, and M. J. Olsen. 2017a. “An assessment of UAS-based photogrammetry for civil integrated management (CIM) modeling of pipes.” In Proc., Pipelines 2017, 112–123. Reston, VA: ASCE.
Javadnejad, F., B. Waldron, and A. Hill. 2017b. “LITE flood: Simple GIS-based mapping approach for real-time redelineation of multifrequency floods.” Nat. Hazards Rev. 18 (3): 04017004. https://doi.org/10.1061/(ASCE)NH.1527-6996.0000238.
Javadnejad, F., B. A. Waldron, and F. Alinia. 2013. “GIS-based gold potential mapping in the Muteh Deposit Area, Iran, with respect to a new mineralization concept.” In Proc., 2013 4th Int. Conf. on Computing for Geospatial Research and Application, 148–149. New York: IEEE.
Lague, D., N. Brodu, and J. Leroux. 2013. “Accurate 3D comparison of complex topography with terrestrial laser scanner: Application to the Rangitikei canyon (N-Z).” ISPRS J. Photogramm. Remote Sens. 82 (Aug): 10–26. https://doi.org/10.1016/j.isprsjprs.2013.04.009.
LINZ (Land Information New Zealnd). 2016. “Invercargill 0.1m urban aerial photos (2016).” Accessed January 1, 2017. https://data.linz.govt.nz/layer/88147-invercargill-01m-urban-aerial-photos-2016/.
Lowe, D. G. 1999. “Object recognition from local scale-invariant features.” In Proc., 7th IEEE Int. Conf. on Computer Vision, 1150–1157. New York: IEEE.
Lowe, D. G. 2004. “Distinctive image features from scale-invariant keypoints.” Int. J. Comput. Vision 60 (2): 91–110. https://doi.org/10.1023/B:VISI.0000029664.99615.94.
Mahmoudabadi, H., M. J. Olsen, and S. Todorovic. 2016. “Efficient terrestrial laser scan segmentation exploiting data structure.” ISPRS J. Photogramm. Remote Sens. 119 (Sep): 135–150. https://doi.org/10.1016/j.isprsjprs.2016.05.015.
McCaffrey, K. J. W., R. R. Jones, R. E. Holdsworth, R. W. Wilson, P. Clegg, J. Imber, N. Holliman, and I. Trinks. 2005. “Unlocking the spatial dimension: Digital technologies and the future of geoscience fieldwork.” J. Geol. Soc. 162 (6): 927–938. https://doi.org/10.1144/0016-764905-017.
O’Banion, M. S., M. J. Olsen, C. E. Parrish, and M. Bailey. 2019. “Interactive visualization of 3D coordinate uncertainties in terrestrial laser scanning point clouds using OpenGL shader language.” J. Surv. Eng. 145 (1): 04018012. https://doi.org/10.1061/(ASCE)SU.1943-5428.0000267.
O’Banion, M. S., M. J. Olsen, C. Rault, J. Wartman, and K. Cunningham. 2018. “Suitability of structure from motion for rock-slope assessment.” Photogramm. Rec. 33 (162): 217–242. https://doi.org/10.1111/phor.12241.
Olsen, M., J. Wartman, M. McAlister, H. Mahmoudabadi, M. O’Banion, L. Dunham, and K. Cunningham. 2015. “To fill or not to fill: Sensitivity analysis of the influence of resolution and hole filling on point cloud surface modeling and individual rockfall event detection.” Remote Sens. 7 (9): 12103–12134. https://doi.org/10.3390/rs70912103.
Omasa, K., F. Hosoi, and A. Konishi. 2006. “3D lidar imaging for detecting and understanding plant responses and canopy structure.” J. Exp. Bot. 58 (4): 881–898. https://doi.org/10.1093/jxb/erl142.
Oregon Department of Transportation (ODOT). 2020. “Oregon real-time GNSS network (ORGN).” Accessed September 2, 2020. https://www.oregon.gov/odot/orgn/pages/index.aspx.
OuYang, D., and H.-Y. Feng. 2005. “On the normal vector estimation for point cloud data from smooth surfaces.” Comput.-Aided Des. 37 (10): 1071–1079. https://doi.org/10.1016/j.cad.2004.11.005.
Pajares, G. 2015. “Overview and current status of remote sensing applications based on unmanned aerial vehicles (UAVs).” Photogramm. Eng. Remote Sens. 81 (4): 281–330. https://doi.org/10.14358/PERS.81.4.281.
Rangel, J. M. G., G. R. Gonçalves, and J. A. Pérez. 2018. “The impact of number and spatial distribution of GCPs on the positional accuracy of geospatial products derived from low-cost UASs.” Int. J. Remote Sens. 39 (21): 7154–7171. https://doi.org/10.1080/01431161.2018.1515508.
Remondino, F., M. G. Spera, E. Nocerino, F. Menna, and F. Nex. 2014. “State of the art in high density image matching.” Photogramm. Rec. 29 (146): 144–166. https://doi.org/10.1111/phor.12063.
Sanz-Ablanedo, E., J. Chandler, J. Rodríguez-Pérez, and C. Ordóñez. 2018. “Accuracy of unmanned aerial vehicle (UAV) and SfM photogrammetry survey as a function of the number and location of ground control points used.” Remote Sens. 10 (10): 1606. https://doi.org/10.3390/rs10101606.
Seitz, S. M., B. Curless, J. Diebel, D. Scharstein, and R. Szeliski. 2006. “A comparison and evaluation of multi-view stereo reconstruction algorithms.” In Vol. 1 of Proc., 2006 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR’06), 519–528. New York: IEEE.
Shi, Y., et al. 2016. “Unmanned aerial vehicles for high-throughput phenotyping and agronomic research.” PLoS One 11 (7): e0159781. https://doi.org/10.1371/journal.pone.0159781.
Shum, H.-Y., Q. Ke, and Z. Zhang. 1999. “Efficient bundle adjustment with virtual key frames: A hierarchical approach to multi-frame structure from motion.” In Proc., 1999 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (Cat. No PR00149), 538–543. New York: IEEE.
Slocum, R. K., and C. E. Parrish. 2017. “Simulated imagery rendering workflow for UAS-based photogrammetric 3D reconstruction accuracy assessments.” Remote Sens. 9 (4): 396. https://doi.org/10.3390/rs9040396.
Smith, M. W., and D. Vericat. 2015. “From experimental plots to experimental landscapes: Topography, erosion and deposition in sub-humid badlands from structure-from-motion photogrammetry.” Earth Surf. Processes Landforms 40 (12): 1656–1671. https://doi.org/10.1002/esp.3747.
Snavely, N., S. M. Seitz, and R. Szeliski. 2006. “Photo tourism.” In Proc., ACM SIGGRAPH 2006 Papers on—SIGGRAPH ’06, 835. New York: ACM Press.
Snavely, N., S. M. Seitz, and R. Szeliski. 2008. “Modeling the world from internet photo collections.” Int. J. Comput. Vision 80 (2): 189–210. https://doi.org/10.1007/s11263-007-0107-3.
Sohn, G., and I. Dowman. 2007. “Data fusion of high-resolution satellite imagery and LiDAR data for automatic building extraction.” ISPRS J. Photogramm. Remote Sens. 62 (1): 43–63. https://doi.org/10.1016/j.isprsjprs.2007.01.001.
Szeliski, R. 2010. Computer vision: Algorithms and applications. New York: Springer.
Tomasi, C., and T. Kanade. 1992. “Shape and motion from image streams under orthography: A factorization method.” Int. J. Comput. Vision 9 (2): 137–154. https://doi.org/10.1007/BF00129684.
Tonkin, T., and N. Midgley. 2016. “Ground-control networks for image based surface reconstruction: An investigation of optimum survey designs using UAV derived imagery and structure-from-motion photogrammetry.” Remote Sens. 8 (9): 786. https://doi.org/10.3390/rs8090786.
Triggs, B., P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon. 1999. “Bundle adjustment—A modern synthesis.” In Proc., Int. Workshop on Vision Algorithms, 298–372. Berlin: Springer.
Westoby, M. J., J. Brasington, N. F. Glasser, M. J. Hambrey, and J. M. Reynolds. 2012. “‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications.” Geomorphology 179 (Dec): 300–314. https://doi.org/10.1016/j.geomorph.2012.08.021.
Wood, R. L., D. T. Gillins, M. E. Mohammadi, F. Javadnejad, H. Tahami, M. N. Gillins, and Y. Liao. 2017. “2015 Gorkha post-earthquake reconnaissance of a historic village with micro unmanned aerial systems.” In Proc., 16th World Conf. on Earthquake (16WCEE). Tokyo: International Association for Earthquake Engineering.
Yao, Y., H. Zhu, Y. Nie, X. Ji, and X. Cao. 2014. “Revised depth map estimation for multi-view stereo.” In Proc., 2014 Int. Conf. on 3D Imaging (IC3D), 1–7. New York: IEEE.
Zhu, Z., C. Stamatopoulos, and C. S. Fraser. 2015. “Accurate and occlusion-robust multi-view stereo.” ISPRS J. Photogramm. Remote Sens. 109 (Nov): 47–61. https://doi.org/10.1016/j.isprsjprs.2015.08.008.
Information & Authors
Information
Published In
Journal of Surveying Engineering
Volume 147 • Issue 1 • February 2021
Copyright
© 2020 American Society of Civil Engineers.
History
Received: Jun 23, 2019
Accepted: Jun 9, 2020
Published online: Sep 23, 2020
Published in print: Feb 1, 2021
Discussion open until: Feb 23, 2021
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.