An AI-Based System Utilizing IoT-Enabled Ambient Sensors and LLMs for Complex Activity Tracking

Yuan Sun; Jorge Ortiz

doi:10.54097/dj2pt496

Authors

Yuan Sun
Jorge Ortiz

DOI:

https://doi.org/10.54097/dj2pt496

Keywords:

Ambient Sensors; Complex Activity Recognition; Elderly Care; Edge Devices.

Abstract

Complex activity recognition plays an important role in elderly care assistance. However, the reasoning ability of edge devices is constrained by the classic machine learning model capacity. In this paper, we present a non-invasive ambient sensing system that can detect multiple activities and apply large language models (LLMs) to reason the activity sequences. This method effectively combines edge devices and LLMs to help elderly people in their daily activities, such as reminding them to take pills or handling emergencies like falls. The LLM-based edge device can also serve as an interface to interact with elderly people, especially with memory issue, assisting them in their daily lives. By deploying such a system, we believe that the smart sensing system can improve the quality of life for older people and provide more efficient protection.

Downloads

Download data is not yet available.

References

Hamad Ahmed and Muhammad Tahir. 2017. Improving the accuracy of human body orientation estimation with wearable imu sensors. IEEE Transactions on instrumentation and measurement, 66, 3, 535–542.

Luca Arrotta, Gabriele Civitarese, and Claudio Bettini. 2022. Dexar: deep explainable sensor-based activity recognition in smart-home environmentsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 6, 1, 1–30.

Amay J Bandodkar and Joseph Wang. 2014. Non-invasive wearable electrochemical sensors: a review. Trends in biotechnology, 32, 7, 363–371.

Oresti Banos, Juan-Manuel Galvez, Miguel Damas, Hector Pomares, and Ignacio Rojas. 2014. Window size impact in human activity recognition. Sensors, 14, 4, 6474–6499.

Paolo Bernardi, Marta Cavagnaro, Stefano Pisa, and Emanuele Piuzzi. 2000. Specific absorption rate and temperature increases in the head of a cellular-phone user. IEEE transactions on microwave theory and techniques, 48, 7, 1118–1126.

Lu Chi, Borui Jiang, and Yadong Mu. 2020. Fast fourier convolution. Advances in Neural Information Processing Systems, 33, 4479–4488.

Veena Chidurala, Xiaodong Wang, Xinrong Li, and Jesse Hamner. 2022. Iot based sensor system design for real-time non-intrusive occupancy monitoring. In 2022 IEEE International Conference on Internet of Things and Intelligence Systems (IoTaIS). IEEE, 252–257.

Munkhjargal Gochoo, Tan-Hsu Tan, Shih-Chia Huang, Shing-Hong Liu, and Fady S Alnajjar. 2017. Dcnn-based elderly activity recognition using binary sensors. In 2017 international conference on electrical and computing technologies and applications (ICECTA). IEEE, 1–5.

Aftab Khan, Nils Hammerla, Sebastian Mellor, and Thomas Plötz. 2016. Optimising sampling rates for accelerometer-based human activity recognition. Pattern Recognition Letters, 73, 33–40.

Gierad Laput, Yang Zhang, and Chris Harrison. 2017. Synthetic sensors: towards general-purpose sensing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 3986–3999.

Athanasios Lentzas and Dimitris Vrakas. 2020. Non-intrusive human activity recognition and abnormal behavior detection on elderly people: a review. Artificial Intelligence Review, 53, 3, 1975–2021.

M Martinez-Burdalo, A Martin, M Anguiano, and R Villar. 2004. Comparison of fdtd-calculated specific absorption rate in adults and children when using a mobile phone at 900 and 1800 mhz. Physics in Medicine & Biology, 49, 2, 345.

Anzah H Niazi, Delaram Yazdansepas, Jennifer L Gay, Frederick W Maier, Lakshmish Ramaswamy, Khaled Rasheed, and Matthew P Buman. 2017. Statistical analysis of window sizes and sampling rates in human activity recognition. In HEALTHINF, 319–325.

Samir A Rawashdeh, Derek A Rafeldt, and Timothy L Uhl. 2016. Wearable imu for shoulder injury prevention in overhead sports. Sensors, 16, 11, 1847.

Ioan Susnea, Luminita Dumitriu, Mihai Talmaciu, Emilia Pecheanu, and Dan Munteanu. 2019. Unobtrusive monitoring the daily activity routine of elderly people living alone, with low-cost binary sensors. Sensors, 19, 10, 2264.

Xuzhong Yan, Heng Li, Angus R Li, and Hong Zhang. 2017. Wearable imu-based real-time motion warning system for construction workers’ musculoskeletal disorders prevention. Automation in construction, 74, 2–11.

Xin Yu and Panos D Prevedouros. 2013. Performance and challenges in utilizing non-intrusive sensors for traffic data collection.