A Blind Assistance System Based on Real-Time Object Detection, Timing Analysis and Angle Navigation

Abstract

In this paper, we propose an assistive system for the blind based on real-time object detection, temporal analysis, and angle computation. We use the YOLOv5 model for object detection on mobile and combine it with a lightweight multi-target tracking algorithm (DeepSORT) to achieve continuous recognition of the objects in the video stream, thus providing more stable detection results. The system provides directional cues to the blind by calculating the angle of the object relative to the centre of the image, and intuitive environmental descriptions with Text-to-Speech (TTS) speech output. Tests show that (on Android devices), with proper model quantisation and optimisation, YOLOv5 can achieve real-time detection speeds of around 15-20 FPS; and that multi-target tracking does not significantly increase latency. The error range of the angle calculation is about ±5°, which is an acceptable error and in line with the accuracy level of the mobile phone's sensor and human alignment capabilities. The system provides a highly accurate and portable visual perception assistance solution for blind people.