Step-Level Occupant Detection across Different Structures through Footstep-Induced Floor Vibration Using Model Transfer
Publication: Journal of Engineering Mechanics
Volume 146, Issue 3
Abstract
This paper presents a floor-vibration-based step-level occupant-detection approach that enables detection across different structures through model transfer. Detecting the occupants through detecting their footsteps (i.e., step-level occupant detection) is useful in various smart building applications such as senior/healthcare and energy management. Current sensing approaches (e.g., vision-based, pressure-based, radio frequency–based, and mobile-based) for step-level occupant detection are limited due to installation and maintenance requirements such as dense deployment and requiring the occupants to carry a device. To overcome these requirements, previous research used ambient structural vibration sensing for footstep modeling and step-level occupant detection together with supervised learning to train a footstep model to distinguish footsteps from nonfootsteps using a set of labeled data. However, floor-vibration-based footstep models are influenced by the structural properties, which may vary from structure to structure. Consequently, a footstep model in one structure does not accurately capture the responses in another structure, which leads to high detection errors and the costly need for acquiring labeled data in every structure. To address this challenge, the effect of the structure on the footstep-induced floor vibration responses is here characterized to develop a physics-driven model transfer approach that enables step-level occupant detection across structures. Specifically, the proposed model transfer approach projects the data into a feature space in which the structural effects are minimized. By minimizing the structure effect in this projected feature space, the footstep models mainly represent the differences in the excitation types and therefore are transferable across structures. To this end, it is analytically shown that the structural effects are correlated to the maximum-mean-discrepancy (MMD) distance between the source and target marginal data distributions. Therefore, to reduce the structural effect, the MMD between the distributions in the source and target structures is minimized. The robustness of the proposed approach was evaluated through field experiments in three types of structures. The evaluation consists of training a footstep model in a set of structures and testing it in a different structure. Across the three structures, the evaluation results show footstep detection F1 score of up to 99% for the proposed approach, corresponding to a 29-fold improvement compared to the baseline approach, which do not transfer the model.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
This research was partially supported by NSF (CNS-1149611 and CMMI-1653550), Google, Intel, and Highmark. The authors would also like to acknowledge Vincentian and Baptist Nursing Homes for providing deployment sites to conduct experiments and collect data.
References
Alyamkin, S., and S. Eremenko. 2011. “Pedestrian detection algorithms based on an analysis of the autocorrelation function of a seismic signal.” Optoelectronics Instrum. Data Process. 47 (2): 124–129. https://doi.org/10.3103/S8756699011020038.
Baron, M. 2013. Probability and statistics for computer scientists. Boca Raton, FL: CRC Press.
Ben-David, S., J. Blitzer, K. Crammer, and F. Pereira. 2007. “Analysis of representations for domain adaptation.” In Proc., 19th Int. Conf. on Neural Information Processing Systems, 137–144. Cambridge, MA: MIT Press.
Bickel, S., M. Brückner, and T. Scheffer. 2007. “Discriminative learning for differing training and test distributions.” In Proc., 24th Int. Conf. on Machine Learning, 81–88. New York: Association for Computing Machinery.
Blitzer, J., R. McDonald, and F. Pereira. 2006. “Domain adaptation with structural correspondence learning.” In Proc., 2006 Conf. Empirical Methods in Natural Language Processing, 120–128. Stroudsburg, PA: Association for Computational Linguistics.
Brincker, R., L. Zhang, and P. Andersen. 2000. “Modal identification from ambient responses using frequency domain decomposition.” In Proc., 18th Int. Modal Analysis Conf. Aalborg, Denmark: Univ. of Aalborg.
Changyong, F., W. Hongyue, L. Naiji, C. Tian, H. Hua, and L. Ying. 2014. “Log-transformation and its implications for data analysis.” Shanghai Arc. Psychiatry 26 (2): 105.
Dai, W., G.-R. Xue, Q. Yang, and Y. Yu. 2007. “Transferring naive bayes classifiers for text classification.” In Vol. 1 of Proc., 22nd National Conf. on Artificial Intelligence, 540–545. Palo Alto, CA: AAAI Press.
Fagert, J., M. Mirshekari, S. Pan, P. Zhang, and H. Y. Noh. 2017a. “Characterizing left-right gait balance using footstep-induced structural vibrations.” In Vol. 10168 of Proc., Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems. Bellingham, WA: International Society for Optics and Photonics.
Fagert, J., M. Mirshekari, S. Pan, P. Zhang, and H. Y. Noh. 2017b. “Monitoring hand-washing practices using structural vibrations.” In Proc., Structural Health Monitoring 2017. Lancaster, PA: DEStech Publications.
Friedman, J., T. Hastie, and R. Tibshirani. 2001. Vol. 1of The elements of statistical learning. New York: Springer.
Gretton, A. 2013. “Introduction to RKHS, and some simple kernel algorithms.” Adv. Top. Mach. Learn. Lect. Conducted Univ. Coll. London 16.
Gretton, A., K. M. Borgwardt, M. Rasch, B. Schölkopf, and A. J. Smola. 2007. “A kernel method for the two-sample-problem.” In Advances in neural information processing systems, 513–520. London: MIT Press.
Gretton, A., K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola. 2012. “A kernel two-sample test.” J. Mach. Learn. Res. 13 (1): 723–773.
Gretton, A., O. Bousquet, A. Smola, and B. Schölkopf. 2005. “Measuring statistical dependence with hilbert-schmidt norms.” In Proc., Int. Conf. on Algorithmic Learning Theory, 63–77. New York: Springer.
I/O Sensor Nederland bv. 2006. SM-24 geophone element. Drunen, Netherlands: I/O Sensor Nederland bv.
Koç, G., and K. Yegin. 2013. “Footstep and vehicle detection using slow and quick adaptive thresholds algorithm.” Int. J. Distrib. Sens. Netw. 9 (10): 783604. https://doi.org/10.1155/2013/783604.
Lam, M., M. Mirshekari, S. Pan, P. Zhang, and H. Y. Noh. 2016. “Robust occupant detection through step-induced floor vibration by incorporating structural characteristics.” In Vol. 4 of Proc., Dynamics of Coupled Structures, 357–367. New York: Springer.
Li, Y., X. Jing, H. Lv, and J. Wang. 2012. “Analysis of characteristics of two close stationary human targets detected by impulse radio UWB radar.” Prog. Electromagnet. Res. 126: 429–447. https://doi.org/10.2528/PIER12011908.
Ling, X., W. Dai, G.-R. Xue, Q. Yang, and Y. Yu. 2008. “Spectral domain-transfer learning.” In Proc., 14th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 488–496. New York: Association for Computing Machinery.
Madarshahian, R., J. M. Caicedo, and N. Haerens. 2019. “Human activity benchmark classification using multilayer artificial neural network.” In Vol. 2 of Proc., Dynamics of Civil Structures, 207–210. New York: Springer.
MATLAB. 2018. Matlab statistics and machine learning toolbox. Natick, MA: MathWorks.
Mirshekari, M., J. Fagert, A. Bonde, P. Zhang, and H. Y. Noh. 2018a. “Human gait monitoring using footstep-induced floor vibrations across different structures.” In Proc., ACM Int. Joint Conf. and 2018 Int. Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, 1382–1391. New York: Association for Computing Machinery.
Mirshekari, M., S. Pan, J. Fagert, E. M. Schooler, P. Zhang, and H. Y. Noh. 2018b. “Occupant localization using footstep-induced structural vibration.” Mech. Syst. Sig. Process. 112 (Nov): 77–97. https://doi.org/10.1016/j.ymssp.2018.04.026.
Mirshekari, M., S. Pan, P. Zhang, and H. Y. Noh. 2016. “Characterizing wave propagation to improve indoor step-level person localization using floor vibration.” In Vol. 9803 of Proc., Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems. Bellingham, WA: International Society for Optics and Photonics.
Mirshekari, M., P. Zhang, and H. Y. Noh. 2017. “Calibration-free footstep frequency estimation using structural vibration.” In Vol. 2 of Proc., Dynamics of Civil Structures, 287–289. New York: Springer.
Muñoz-Salinas, R., E. Aguirre, and M. Garca-Silvente. 2007. “People detection and tracking using stereo vision and color.” Image Vision Comput. 25 (6): 995–1007. https://doi.org/10.1016/j.imavis.2006.07.012.
Murakita, T., T. Ikeda, and H. Ishiguro. 2004. “Human tracking using floor sensors based on the markov chain monte carlo method.” In Vol. 4 of Proc., 17th Int. Conf. on Pattern Recognition, 917–920. New York: IEEE.
Oberg, T., A. Karsznia, and K. Oberg. 1993. “Basic gait parameters: Reference data for normal subjects, 10-79 years of age.” J. Rehabil. Res. Dev. 30 (2): 210.
Oppenheim, A. V., A. S. Willsky, and S. H. Nawab. 1997. Signals and systems. 2nd ed. Upper Saddle River, NJ: Prentice-Hall.
Pan, S., M. Mirshekari, J. Fagert, C. G. Ramirez, A. J. Chung, C. C. Hu, J. P. Shen, P. Zhang, and H. Y. Noh. 2018. “Characterizing human activity induced impulse and slip-pulse excitations through structural vibration.” J. Sound Vib. 414 (Feb): 61–80. https://doi.org/10.1016/j.jsv.2017.10.034.
Pan, S., C. G. Ramirez, M. Mirshekari, J. Fagert, A. J. Chung, C. C. Hu, J. P. Shen, H. Y. Noh, and P. Zhang. 2017a. “Surfacevibe: Vibration-based tap and swipe tracking on ubiquitous surfaces.” In Proc., 16th ACM/IEEE Int. Conf. on Information Processing in Sensor Networks, 197–208. New York: IEEE.
Pan, S., S. Xu, M. Mirshekari, P. Zhang, and H. Y. Noh. 2017b. “Collaboratively adaptive vibration sensing system for high-fidelity monitoring of structural responses induced by pedestrians.” Front. Built Environ. 3: 28. https://doi.org/10.3389/fbuil.2017.00028.
Pan, S., T. Yu, M. Mirshekari, J. Fagert, A. Bonde, O. J. Mengshoel, H. Y. Noh, and P Zhang. 2017c. “Footprintid: Indoor pedestrian identification through ambient structural vibration sensing.” Proc., ACM Interactive, Mobile, Wearable and Ubiquitous Technol. 1 (3): 89. https://doi.org/10.1145/3130954.
Pan, S. J., J. T. Kwok, and Q. Yang. 2008. “Transfer learning via dimensionality reduction.” In Proc., 23rd AAAI Conf. on Artificial Intelligence, 677–682. Palo Alto, CA: AAAI Press.
Pan, S. J., I. W. Tsang, J. T. Kwok, and Q. Yang. 2011. “Domain adaptation via transfer component analysis.” IEEE Trans. Neural Networks 22 (2): 199–210. https://doi.org/10.1109/TNN.2010.2091281.
Pan, S. J., and Q. Yang. 2010. “A survey on transfer learning.” IEEE Trans. Knowl. Data Eng. 22 (10): 1345–1359. https://doi.org/10.1109/TKDE.2009.191.
Poston, J. D., R. M. Buehrer, and P. A. Tarazaga. 2017a. “A framework for occupancy tracking in a building via structural dynamics sensing of footstep vibrations.” Front. Built Environ. 3: 65. https://doi.org/10.3389/fbuil.2017.00065.
Poston, J. D., R. M. Buehrer, and P. A. Tarazaga. 2017b. “Indoor footstep localization from structural dynamics instrumentation.” Mech. Syst. Sig. Process. 88 (May): 224–239. https://doi.org/10.1016/j.ymssp.2016.11.023.
Reuland, Y., S. G. Pai, S. Drira, and I. F. Smith. 2017. “Vibration-based occupant detection using a multiple-model approach.” In Vol. 2 of Dynamics of civil structures, 49–56. New York: Springer.
Richman, M. S., D. S. Deadrick, R. J. Nation, and S. Whitney. 2001. “Personnel tracking using seismic sensors.” In Aerospace/defense sensing, simulation, and controls, 14–21. Bellingham, WA: International Society for Optics and Photonics.
Saha, M., S. Thakur, A. Singh, and Y. Agarwal. 2014. “Energylens: Combining smartphones with electricity meter for accurate activity detection and user annotation.” In Proc., 5th Int. Conf. on Future Energy Systems, 289–300. New York: Association for Computing Machinery.
Sasaki, Y. 2007. “The truth of the f-measure.” Teach Tutor Mater. 1 (5): 1–5.
Savio, D., and T. Ludwig. 2007. “Smart carpet: A footstep tracking interface.” In Vol. 2 of Proc., 21st Int. Conf. on Advanced Information Networking and Applications Workshops, 754–760. New York: IEEE.
Schölkopf, B., A. Smola, and K.-R. Müller. 1998. “Nonlinear component analysis as a kernel eigenvalue problem.” Neural Comput. 10 (5): 1299–1319. https://doi.org/10.1162/089976698300017467.
Smola, A., A. Gretton, L. Song, and B. Schölkopf. 2007. “A hilbert space embedding for distributions.” In Proc., Int. Conf. on Algorithmic Learning Theory, 13–31. New York: Springer.
Snidaro, L., C. Micheloni, and C. Chiavedale. 2005. “Video security for ambient intelligence.” IEEE Trans. Syst. Man Cybern. Part A Syst. Humans 35 (1): 133–144. https://doi.org/10.1109/TSMCA.2004.838478.
Strang, G., and S. Strang. 2006. Linear algebra and its applications. Belmont, CA: Thomson Brooks/Cole.
Subramanian, A., K. G. Mehrotra, C. K. Mohan, P. K. Varshney, and T. Damarla. 2010. “Feature selection and occupancy classification using seismic sensors.” In Trends in applied intelligent systems, 605–614. New York: Springer.
Succi, G. P., D. Clapp, R. Gampert, and G. Prado. 2001. “Footstep detection and tracking.” In Aerospace/defense sensing, simulation, and controls, 22–29. Bellingham, WA: International Society for Optics and Photonics.
Sugiyama, M., S. Nakajima, H. Kashima, P. V. Buenau, and M. Kawanabe. 2008. “Direct importance estimation with model selection and its application to covariate shift adaptation.” In Proc., 20th Int. Conf. on Neural Information Processing Systems, 1433–1440. New York: Association for Computing Machinery.
Texas Instruments. 2014. OpAmp breakout. Dallas: Texas Instruments.
Uziel, S., T. Elste, W. Kattanek, D. Hollosi, S. Gerlach, and S. Goetze. 2013. “Networked embedded acoustic processing system for smart building applications.” In Proc., 2013 Conf. on Design and Architectures for Signal and Image Processing, 349–350. New York: IEEE.
Visser, H. 1983. “Gait and balance in senile dementia of Alzheimer’s type.” Age Ageing 12 (4): 296–301. https://doi.org/10.1093/ageing/12.4.296.
Wang, C., X. Wang, Z. Long, J. Yuan, Y. Qian, and J. Li. 2016. “Estimation of temporal gait parameters using a wearable microphone-sensor-based system.” Sensors 16 (12): 2167. https://doi.org/10.3390/s16122167.
Yu, T., S. Pan, S. Xu, X. Chen, M. Mirshekari, J. Fagert, P. Zhang, H. Y. Noh, and O. J. Mengshoel. 2018. “ILPC: Iterative learning using physical constraints in real-world sensing data.” In Proc., SmartIoT 2018 Workshop: AI Enhanced IoT Data Processing for Intelligent Applications, 120. Palo Alto, CA: AAAI Press.
Zetik, R., S. Crabbe, J. Krajnak, P. Peyerl, J. Sachs, and R. Thomä. 2006. “Detection and localization of persons behind obstacles using m-sequence through-the-wall radar.” In Vol. 6201 of Proc., Sensors, and Command, Control, Communications, and Intelligence (C3I) Technologies for Homeland Security and Homeland Defense V, 62010I. Bellingham, WA: International Society for Optics and Photonics.
Information & Authors
Information
Published In
Journal of Engineering Mechanics
Volume 146 • Issue 3 • March 2020
Copyright
©2019 American Society of Civil Engineers.
History
Received: Jan 22, 2019
Accepted: Jul 3, 2019
Published online: Dec 21, 2019
Published in print: Mar 1, 2020
Discussion open until: May 21, 2020
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.