Assessing Daily Activity Routines Using an Unsupervised Approach in a Smart Home Environment
Publication: Journal of Computing in Civil Engineering
Volume 37, Issue 1
Abstract
When the mental acuity of older adults deteriorates (e.g., dementia), irregular patterns manifest within their activities of daily living (ADL), which renders an effective opportunity for healthcare providers to monitor patients’ mental status. Although successful, such studies depended on supervised learning approaches to recognize ADLs, which require tedious human observation and manual annotation of data. To establish a more efficient alternative, this study develops an unsupervised data segmentation process by modifying a Superpixels Extracted via Energy Driven Sampling (SEEDS) algorithm and a hierarchical clustering method effective for high-dimensional temporal sensor data. The proposed approaches consider the spatiotemporal features (e.g., start time, duration, location, and sequence) and activity-oriented features (e.g., motion intensity and appliance usages) to identify ADL routines without necessitating predefined rules or limiting the scope of features. The results showed that the proposed approaches have comparable accuracy (0.788) to benchmark models that require a priori knowledge (e.g., ontology). Our proposed methodology can be extended to high-dimensional, nonintrusive sensing data to capture the variability of ADL routines in the future. This study contributes a methodological advance for efficiently assessing ADL routines via high-dimensional sensor data and supports future opportunities for capitalizing on smart home technologies that enable older adults to live alone safely, aging-in-place.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, and code generated or used during the study are available from the corresponding author by request.
Acknowledgments
This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2021R1I1A1A01052305) and by the Institute of Construction and Environmental Engineering (ICEE) at Seoul National University (SNU). Any opinions, findings, conclusions, or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of NRF or SNU.
References
Adams, R., and L. Bischof. 1994. “Seeded region growing.” IEEE Trans. Pattern Anal. Mach. Intell. 16 (6): 641–647. https://doi.org/10.1109/34.295913.
Aghabozorgi, S., A. S. Shirkhorshidi, and T. Y. Wah. 2015. “Time-series clustering—A decade review.” Inf. Syst. 53 (Oct–Nov): 16–38. https://doi.org/10.1016/j.is.2015.04.007.
Alberdi, A., A. Weakley, M. Schmitter-Edgecombe, D. J. Cook, A. Aztiria, A. Basarab, and M. Barrenechea. 2018. “Smart home-based prediction of multidomain symptoms related to Alzheimer’s disease.” IEEE J. Biomed. Health Inf. 22 (6): 1720–1731. https://doi.org/10.1109/JBHI.2018.2798062.
Alcalá, J. M., J. Ureña, Á. Hernández, and D. Gualda. 2017. “Assessing human activity in elderly people using non-intrusive load monitoring.” Sensors 17 (2): 351. https://doi.org/10.3390/s17020351.
Aminikhanghahi, S., and D. J. Cook. 2019. “Enhancing activity recognition using CPD-based activity segmentation.” Pervasive Mob. Comput. 53 (Feb): 75–89. https://doi.org/10.1016/j.pmcj.2019.01.004.
Anderson, L. A., R. A. Goodman, D. Holtzman, S. F. Posner, and M. E. Northridge. 2012. Aging in the United States: Opportunities and challenges for public health. Washington, DC: American Public Health Association.
Barberger-Gateau, P., D. Commenges, M. Gagnon, L. Letenneur, C. Sauvel, and J.-F. Dartigues. 1992. “Instrumental activities of daily living as a screening tool for cognitive impairment and dementia in elderly community dwellers.” J. Am. Geriatrics Soc. 40 (11): 1129–1134. https://doi.org/10.1111/j.1532-5415.1992.tb01802.x.
Blankevoort, C. G., M. J. Van Heuvelen, F. Boersma, H. Luning, J. De Jong, and E. J. Scherder. 2010. “Review of effects of physical activity on strength, balance, mobility and ADL performance in elderly subjects with dementia.” Dementia Geriatric Cognit. Disord. 30 (5): 392–402. https://doi.org/10.1159/000321357.
Bucks, R. S., D. L. Ashworth, G. K. Wilcock, and K. Siegfried. 1996. “Assessment of activities of daily living in dementia: Development of the Bristol Activities of Daily Living Scale.” Age Ageing 25 (2): 113–120. https://doi.org/10.1093/ageing/25.2.113.
Caliński, T., and J. Harabasz. 1974. “A dendrite method for cluster analysis.” Commun. Stat.- Theory Methods 3 (1): 1–27. https://doi.org/10.1080/03610927408827101.
Chalmers, C., P. Fergus, C. A. C. Montanez, S. Sikdar, F. Ball, and B. Kendall. 2020. “Detecting activities of daily living and routine behaviours in dementia patients living alone using smart meter load disaggregation.” IEEE Trans. Emerging Top. Comput. 10 (1): 157–169. https://doi.org/10.1109/TETC.2020.2993177.
Chikhaoui, B., S. Wang, and H. Pigot. 2011. “A frequent pattern mining approach for ADLs recognition in smart environments.” In Proc., 2011 IEEE Int. Conf. on Advanced Information Networking and Applications, 248–255. New York: IEEE.
Chimamiwa, G., M. Alirezaie, F. Pecora, and A. Loutfi. 2021. “Multi-sensor dataset of human activities in a smart home environment.” Data Brief 34 (Feb): 106632. https://doi.org/10.1016/j.dib.2020.106632.
Choi, S.-S., S.-H. Cha, and C. C. Tappert. 2010. “A survey of binary similarity and distance measures.” J. Syst. Cybern. Inf. 8 (1): 43–48.
Civitarese, G., C. Bettini, T. Sztyler, D. Riboni, and H. Stuckenschmidt. 2018. “NECTAR: Knowledge-based collaborative active learning for activity recognition.” In Proc., 2018 IEEE Int. Conf. on Pervasive Computing and Communications (PerCom), 1–10. New York: IEEE.
Cook, D. J. 2010. “Learning setting-generalized activity models for smart spaces.” IEEE Intell. Syst. 2010 (99): 32–38. https://doi.org/10.1109/MIS.2010.112.
Cook, D. J., A. S. Crandall, B. L. Thomas, and N. C. Krishnan. 2012. “CASAS: A smart home in a box.” Computer 46 (7): 62–69. https://doi.org/10.1109/MC.2012.328.
Covinsky, K. E., R. M. Palmer, R. H. Fortinsky, S. R. Counsell, A. L. Stewart, D. Kresevic, C. J. Burant, and C. S. Landefeld. 2003. “Loss of independence in activities of daily living in older adults hospitalized with medical illnesses: Increased vulnerability with age.” J. Am. Geriatrics Soc. 51 (4): 451–458. https://doi.org/10.1046/j.1532-5415.2003.51152.x.
Cramariuc, B., M. Gabbouj, and J. Astola. 1997. “Clustering based region growing algorithm for color image segmentation.” In Proc., 13th Int. Conf. on Digital Signal Processing, 857–860. New York: IEEE.
Davies, D. L., and D. W. Bouldin. 1979. “A cluster separation measure.” IEEE Trans. Pattern Anal. Mach. Intell. PAM-1 (2): 224–227. https://doi.org/10.1109/TPAMI.1979.4766909.
Enshaeifar, S., A. Zoha, A. Markides, S. Skillman, S. T. Acton, T. Elsaleh, M. Hassanpour, A. Ahrabian, M. Kenny, and S. Klein. 2018. “Health management and pattern analysis of daily living activities of people with dementia using in-home sensors and machine learning techniques.” PLoS One 13 (5): e0195605. https://doi.org/10.1371/journal.pone.0195605.
Ester, M., H.-P. Kriegel, J. Sander, and X. Xu. 1996. “A density-based algorithm for discovering clusters in large spatial databases with noise.” In Proc., KDD, 226–231. Palo Alto, CA: Association for the Advancement of Artificial Intelligence.
Felzenszwalb, P. F., and D. P. Huttenlocher. 2004. “Efficient graph-based image segmentation.” Int. J. Comput. Vision 59 (2): 167–181. https://doi.org/10.1023/B:VISI.0000022288.19776.77.
Ferrie, J. E., M. J. Shipley, T. N. Akbaraly, M. G. Marmot, M. Kivimäki, and A. Singh-Manoux. 2011. “Change in sleep duration and cognitive function: Findings from the Whitehall II Study.” Sleep 34 (5): 565–573. https://doi.org/10.1093/sleep/34.5.565.
Ghayvat, H., M. Awais, S. Pandya, H. Ren, S. Akbarzadeh, S. Chandra Mukhopadhyay, C. Chen, P. Gope, A. Chouhan, and W. Chen. 2019. “Smart aging system: Uncovering the hidden wellness parameter for well-being monitoring and anomaly detection.” Sensors 19 (4): 766. https://doi.org/10.3390/s19040766.
Ghayvat, H., S. Mukhopadhyay, B. Shenjie, A. Chouhan, and W. Chen. 2018. “Smart home based ambient assisted living: Recognition of anomaly in the activity of daily living for an elderly living alone.” In Proc., 2018 IEEE Int. Instrumentation and Measurement Technology Conf. (I2MTC), 1–5. New York: IEEE.
Ghods, A., K. Caffrey, B. Lin, K. Fraga, R. Fritz, M. Schmitter-Edgecombe, C. Hundhausen, and D. J. Cook. 2019. “Iterative design of visual analytics for a clinician-in-the-loop smart home.” IEEE J. Biomed. Health Inf. 23 (4): 1742–1748. https://doi.org/10.1109/JBHI.2018.2864287.
Ghods, A., and D. J. Cook. 2019. “Activity2Vec: Learning ADL embeddings from sensor data with a sequence-to-sequence model.” Preprint, submitted July 12, 2019. https://arxiv.org/abs/1907.05597.
Gold, D. A. 2012. “An examination of instrumental activities of daily living assessment in older adults and mild cognitive impairment.” J. Clin. Exp. Neuropsychol. 34 (1): 11–34. https://doi.org/10.1080/13803395.2011.614598.
Gower, J. C. 1967. “A comparison of some methods of cluster analysis.” Biometrics 23 (4): 623–637. https://doi.org/10.2307/2528417.
Gower, J. C. 1971. “A general coefficient of similarity and some of its properties.” Biometrics 27 (4): 857–871. https://doi.org/10.2307/2528823.
Hajihashemi, Z., M. Yefimova, and M. Popescu. 2014. “Detecting daily routines of older adults using sensor time series clustering.” In Proc., 2014 36th Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society, 5912–5915. New York: IEEE.
Hayes, T. L., T. Riley, N. Mattek, M. Pavel, and J. Kaye. 2014. “Sleep habits in mild cognitive impairment.” Alzheimer Dis. Associated Disord. 28 (2): 145–150. https://doi.org/10.1097/WAD.0000000000000010.
Irvine, N., C. Nugent, S. Zhang, H. Wang, and W. W. Ng. 2020. “Neural network ensembles for sensor-based human activity recognition within smart environments.” Sensors 20 (1): 216. https://doi.org/10.3390/s20010216.
Katz, S. 1963. “Studies of illness in the aged. The index of ADL: A standardized measure of biologic and psychologic function.” JAMA 185 (12): 914–919. https://doi.org/10.1001/jama.1963.03060120024016.
Law, M., and L. Letts. 1989. “A critical review of scales of activities of daily living.” Am. J. Occup. Ther. 43 (8): 522–528. https://doi.org/10.5014/ajot.43.8.522.
Lee, B., C. R. Ahn, P. Mohan, T. Chaspari, and H.-S. Lee. 2019a. “Measuring routine variability of daily activities with image complexity metrics.” In Proc., 6th ACM Int. Conf. on Systems for Energy-Efficient Buildings, Cities, and Transportation, 376–377. New York: Association for Computing Machinery.
Lee, B., C. R. Ahn, P. Mohan, T. Chaspari, and H.-S. Lee. 2020. “Evaluating routine variability of daily activities in smart homes with image complexity measures.” J. Comput. Civ. Eng. 34 (6): 04020042. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000924.
Lee, M.-T., Y. Jang, and W.-Y. Chang. 2019b. “How do impairments in cognitive functions affect activities of daily living functions in older adults?” PLoS One 14 (6): e0218112. https://doi.org/10.1371/journal.pone.0218112.
Li, W., Y. Xu, B. Tan, and R. J. Piechocki. 2017. “Passive wireless sensing for unsupervised human activity recognition in healthcare.” In Proc., 2017 13th Int. Wireless Communications and Mobile Computing Conf. (IWCMC), 1528–1533. New York: IEEE.
Liu, Y., Z. Li, H. Xiong, X. Gao, and J. Wu. 2010. “Understanding of internal clustering validation measures.” In Proc., 2010 IEEE Int. Conf. on Data Mining, 911–916. New York: IEEE.
Lyons, B. E., D. Austin, A. Seelye, J. Petersen, J. Yeargers, T. Riley, N. Sharma, N. Mattek, K. Wild, and H. Dodge. 2015. “Pervasive computing technologies to continuously assess Alzheimer’s disease progression and intervention efficacy.” Front. Aging Neurosci. 7 (102): 1–14. https://doi.org/10.3389/fnagi.2015.00102.
Maggi, S., J. A. Langlois, N. Minicuci, F. Grigoletto, M. Pavan, D. J. Foley, and G. Enzi. 1998. “Sleep complaints in community-dwelling older persons: Prevalence, associated factors, and reported causes.” J. Am. Geriatrics Soc. 46 (2): 161–168. https://doi.org/10.1111/j.1532-5415.1998.tb02533.x.
Medina-Quero, J., S. Zhang, C. Nugent, and M. Espinilla. 2018. “Ensemble classifier of long short-term memory with fuzzy temporal windows on binary sensors for activity recognition.” Expert Syst. Appl. 114 (Dec): 441–453. https://doi.org/10.1016/j.eswa.2018.07.068.
Meng, L., C. Miao, and C. Leung. 2017. “Towards online and personalized daily activity recognition, habit modeling, and anomaly detection for the solitary elderly through unobtrusive sensing.” Multimedia Tools Appl. 76 (8): 10779–10799. https://doi.org/10.1007/s11042-016-3267-8.
Minor, B., and D. J. Cook. 2017. “Forecasting occurrences of activities.” Pervasive Mob. Comput. 38 (Jul): 77–91. https://doi.org/10.1016/j.pmcj.2016.09.010.
Mohan, P., B. Lee, T. Chaspari, and C. Ahn. 2020a. “Assessment of daily routine uniformity in a smart home environment using hierarchical clustering.” IEEE J. Biomed. Health Inf. 25 (8): 3197–3208. https://doi.org/10.1109/JBHI.2020.3048327.
Mohan, P., B. Lee, T. Chaspari, and C. R. Ahn. 2019. “Capturing regularity of ADL routines using hierarchical clustering models.” In Proc., 6th ACM Int. Conf. on Systems for Energy-Efficient Buildings, Cities, and Transportation, 373–374. New York: Association for Computing Machinery.
Mohan, P., B. Lee, T. Chaspari, and C. R. Ahn. 2020b. “Capturing occupant routine behaviors in smart home environment using hierarchical clustering models.” In Proc., Construction Research Congress 2020: Computer Applications, 1310–1318. Reston, VA: ASCE.
Moriya, K., E. Nakagawa, M. Fujimoto, H. Suwa, Y. Arakawa, A. Kimura, S. Miki, and K. Yasumoto. 2017. “Daily living activity recognition with echonet lite appliances and motion sensors.” In Proc., 2017 IEEE Int. Conf. on Pervasive Computing and Communications Workshops (PerCom Workshops), 437–442. New York: IEEE.
Mudrazija, S., J. L. Angel, I. Cipin, and S. Smolic. 2020. “Living alone in the United States and Europe: The impact of public support on the independence of older adults.” Res. Aging 42 (5–6): 150–162. https://doi.org/10.1177/0164027520907332.
Nakagawa, E., K. Moriya, H. Suwa, M. Fujimoto, Y. Arakawa, and K. Yasumoto. 2017. “Toward real-time in-home activity recognition using indoor positioning sensor and power meters.” In Proc., 2017 IEEE Int. Conf. on Pervasive Computing and Communications Workshops (PerCom Workshops), 539–544. New York: IEEE.
Nygård, L. 2003. “Instrumental activities of daily living: A stepping-stone towards Alzheimer’s disease diagnosis in subjects with mild cognitive impairment?” Acta Neurol. Scand. 107 (Feb): 42–46. https://doi.org/10.1034/j.1600-0404.107.s179.8.x.
Ordóñez, F., P. De Toledo, and A. Sanchis. 2013. “Activity recognition using hybrid generative/discriminative models on home environments using binary sensors.” Sensors 13 (5): 5460–5477. https://doi.org/10.3390/s130505460.
Rahman, M. A., and Y. Wang. 2016. “Optimizing intersection-over-union in deep neural networks for image segmentation.” In Proc., Int. Symp. on Visual Computing, 234–244. Cham, Switzerland: Springer.
Rezatofighi, H., N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, and S. Savarese. 2019. “Generalized intersection over union: A metric and a loss for bounding box regression.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 658–666. New York: IEEE.
Riboni, D., and M. Murtas. 2019. “Sensor-based activity recognition: One picture is worth a thousand words.” Future Gener. Comput. Syst. 101 (Dec): 709–722. https://doi.org/10.1016/j.future.2019.07.020.
Riboni, D., T. Sztyler, G. Civitarese, and H. Stuckenschmidt. 2016. “Unsupervised recognition of interleaved activities of daily living through ontological and probabilistic reasoning.” In Proc., 2016 ACM Int. Joint Conf. on Pervasive and Ubiquitous Computing, 1–12. New York: Association for Computing Machinery.
Roehrig, B., K. Hoeffken, L. Pientka, and U. Wedding. 2007. “How many and which items of activities of daily living (ADL) and instrumental activities of daily living (IADL) are necessary for screening.” Crit. Rev. Oncol./Hematol. 62 (2): 164–171. https://doi.org/10.1016/j.critrevonc.2006.10.001.
Rosenthal, E., L. Brennan, S. Xie, H. Hurtig, J. Milber, D. Weintraub, J. Karlawish, and A. Siderowf. 2010. “Association between cognition and function in patients with Parkinson disease with and without dementia.” Mov. Disord. 25 (9): 1170–1176. https://doi.org/10.1002/mds.23073.
Salvador, S., and P. Chan. 2004. “Determining the number of clusters/segments in hierarchical clustering/segmentation algorithms.” In Proc., 16th IEEE Int. Conf. on Tools with Artificial Intelligence, 576–584. New York: IEEE.
Sander, J., M. Ester, H.-P. Kriegel, and X. Xu. 1998. “Density-based clustering in spatial databases: The algorithm gdbscan and its applications.” Data Min. Knowl. Discovery 2 (2): 169–194. https://doi.org/10.1023/A:1009745219419.
Scullin, M. K., and D. L. Bliwise. 2015. “Sleep, cognition, and normal aging: Integrating a half century of multidisciplinary research.” Perspect. Psychol. Sci. 10 (1): 97–137. https://doi.org/10.1177/1745691614556680.
Shahid, Z. K., S. Saguna, and C. Åhlund. 2022. “Detecting anomalies in daily activity routines of older persons in single resident smart homes: Proof-of-concept study.” JMIR Aging 5 (2): e28260. https://doi.org/10.2196/28260.
Singla, G., D. J. Cook, and M. Schmitter-Edgecombe. 2009. “Tracking activities in complex settings using smart environment technologies.” Int. J. Biosci. Psychiatry Technol. 1 (1): 25–35.
Sinha, A., and D. K. Lobiyal. 2013. “Performance evaluation of data aggregation for cluster-based wireless sensor network.” Hum.-Centric Comput. Inf. Sci. 3 (1): 1–17. https://doi.org/10.1186/2192-1962-3-13.
Som, A., N. Krishnamurthi, M. Buman, and P. Turaga. 2020. “Unsupervised pre-trained models from healthy ADLs improve Parkinson’s disease classification of gait patterns.” In Proc., 2020 42nd Annual Int. Conf. of the IEEE Engineering in Medicine & Biology Society (EMBC), 784–788. New York: IEEE.
Tilton, J. C. 1998. “Image segmentation by region growing and spectral clustering with a natural convergence criterion.” In Proc., IGARSS’98. Sensing and Managing the Environment. 1998 IEEE Int. Geoscience and Remote Sensing. Symp. Proc. (Cat. No. 98CH36174), 1766–1768. New York: IEEE.
Twomey, N., T. Diethe, I. Craddock, and P. Flach. 2017. “Unsupervised learning of sensor topologies for improving activity recognition in smart environments.” Neurocomputing 234 (Apr): 93–106. https://doi.org/10.1016/j.neucom.2016.12.049.
Urwyler, P., R. Stucki, L. Rampa, R. Müri, U. P. Mosimann, and T. Nef. 2017. “Cognitive impairment categorized in community-dwelling older adults with and without dementia using in-home sensors that recognise activities of daily living.” Sci. Rep. 7 (1): 42084. https://doi.org/10.1038/srep42084.
Van den Bergh, M., X. Boix, G. Roig, and L. Van Gool. 2015. “SEEDS: Superpixels extracted via energy-driven sampling.” Int. J. Comput. Vision 111 (3): 298–314. https://doi.org/10.1007/s11263-014-0744-2.
Wang, Y., Z. Fan, and A. Bandara. 2016. “Identifying activity boundaries for activity recognition in smart environments.” In Proc., 2016 IEEE Int. Conf. on Communications (ICC), 1–6. New York: IEEE.
Wei, M., T. W. Chow, and R. H. Chan. 2015. “Clustering heterogeneous data with k-means by mutual information-based unsupervised feature transformation.” Entropy 17 (3): 1535–1548. https://doi.org/10.3390/e17031535.
WHO (World Health Organization). 2015. World report on ageing and health. Geneva: WHO.
Yu, X., H. Li, Z. Zhang, and C. Gan. 2019. “The optimally designed variational autoencoder networks for clustering and recovery of incomplete multimedia data.” Sensors 19 (4): 809. https://doi.org/10.3390/s19040809.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 37 • Issue 1 • January 2023
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Mar 3, 2022
Accepted: Sep 13, 2022
Published online: Nov 4, 2022
Published in print: Jan 1, 2023
Discussion open until: Apr 4, 2023
ASCE Technical Topics:
- Algorithms
- Buildings
- Business management
- Data collection
- Deterioration
- Energy efficiency
- Energy engineering
- Engineering fundamentals
- Human and behavioral factors
- Materials characterization
- Materials engineering
- Mathematics
- Measurement (by type)
- Methodology (by type)
- Model accuracy
- Models (by type)
- Practice and Profession
- Research methods (by type)
- Sensors and sensing
- Smart buildings
- Structural engineering
- Structures (by type)
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.