Adaptive Visual Obstacle Avoidance Algorithm for Aerial Drones Integrating Deep Learning and Spatial Geometric Transformations

Haitian Sun

doi:10.54097/aj38p662

Authors

Haitian Sun

DOI:

https://doi.org/10.54097/aj38p662

Keywords:

UAV, Visual Obstacle Avoidance, YOLOv8, PnP Algorithm.

Abstract

This study proposes a new visual obstacle avoidance method for drones based on the YOLO and PnP algorithms. Most existing obstacle avoidance methods are based on sensors such as lidar and ultrasonic sensors, or visual obstacle avoidance methods based on monocular or binocular cameras. In contrast, this study only uses a monocular camera on a two-axis gimbal on an aerial drone to detect faces in the picture, protecting the human head, which is the object that may be the most harmful, with the lowest hardware and algorithm costs, and providing effective obstacle avoidance strategy support for the autonomous flight safety of aerial drones.

Downloads

Download data is not yet available.

References

[1] Moskowitz, E. E., Siegel-Richman, Y. M., Hertner, G., & Schroeppel, T. (2018). Aerial drone misadventure: A novel case of trauma resulting in ocular globe rupture. American Journal of Ophthalmology Case Reports, 10, 35–37. DOI: https://doi.org/10.1016/j.ajoc.2018.01.039

[2] Peng, Y., Qu, D., Zhong, Y., Xie, S., Luo, J., & Gu, J. (2015). The obstacle detection and obstacle avoidance algorithm based on 2-D lidar. 2015 IEEE International Conference on Information and Automation, 1648–1653. DOI: https://doi.org/10.1109/ICInfA.2015.7279550

[3] Borenstein, J., & Koren, Y. (1988). Obstacle avoidance with ultrasonic sensors. IEEE Journal on Robotics and Automation, 4(2), 213–218. DOI: https://doi.org/10.1109/56.2085

[4] Gupta, N., Makkar, J. S., & Pandey, P. (2015). Obstacle detection and collision avoidance using ultrasonic sensors for RC multirotors. 2015 International Conference on Signal Processing and Communication (ICSC), 419–423. DOI: https://doi.org/10.1109/ICSPCom.2015.7150689

[5] Suherman, S., Putra, R. A., & Pinem, M. (2020). Ultrasonic Sensor Assessment for Obstacle Avoidance in Quadcopter-based Drone System. 2020 3rd International Conference on Mechanical, Electronics, Com- puter, and Industrial Technology (MECnIT), 50–53. DOI: https://doi.org/10.1109/MECnIT48290.2020.9166607

[6] Nieuwenhuisen, M., Droeschel, D., Schneider, J., Holz, D., Labe, T., & Behnke, S. (2013). Multimodal obstacle detection and collision avoidance for micro aerial vehicles. 2013 European Conference on Mobile Robots, 7–12. DOI: https://doi.org/10.1109/ECMR.2013.6698812

[7] Ross, S., Melik-Barkhudarov, N., Shankar, K. S., Wendel, A., Dey, D., Bagnell, J. A., & Hebert, M. (2012). Learning Monocular Reactive UAV Control in Cluttered Natural Environments (No. arXiv:1211.1690). arXiv. DOI: https://doi.org/10.1109/ICRA.2013.6630809

[8] Mori, T., Scherer, S. (2013) First results in detecting and avoiding frontal obstacles from a monocular camera for micro unmanned aerial vehicles. In: IEEE International Conference on Robotics and Automation. 1750-1757. DOI: https://doi.org/10.1109/ICRA.2013.6630807

[9] Souhila, K., & Karim, A. (2007). Optical Flow Based Robot Obstacle Avoidance. International Journal of Advanced Robotic Systems, 4(1), 2. DOI: https://doi.org/10.5772/5715

[10] Gosiewski, Z., Ciesluk, J., & Ambroziak, L. (2011). Vision-based obstacle avoidance for unmanned aerial vehicles. 2011 4th International Congress on Image and Signal Processing, 2020–2025. DOI: https://doi.org/10.1109/CISP.2011.6100621

[11] Aguilar, W., Casaliglla, V., & P lit, J. (2017). Obstacle Avoidance Based-Visual Navigation for Micro Aerial Vehicles. Electronics, 6(1), 10. DOI: https://doi.org/10.3390/electronics6010010

[12] Yang, X., Chen, J., Dang, Y., Luo, H., Tang, Y., Liao, C., Chen, P., & Cheng, K.-T. (2021). Fast Depth Prediction and Obstacle Avoidance on a Monocular Drone Using Probabilistic Convolutional Neural Net- work. IEEE Transactions on Intelligent Transportation Systems, 22(1), 156–167. DOI: https://doi.org/10.1109/TITS.2019.2955598

[13] Glenn Jocher, Ayush Chaurasia, Alex Stoken, et al. (2022) ultralytics/yolov5: v7.0 - YOLOv5 SOTA Realtime Instance Segmentation (v7.0). Zenodo.

[14] Sohan, M., Sai Ram, T., & Rami Reddy, Ch. V. (2024). A Review on YOLOv8 and Its Advancements. In I. J. Jacob, S. Piramuthu, & P. Falkowski-Gilski (Eds.), Data Intelligence and Cognitive Informatics. Springer Nature, Singapore. 529–545. DOI: https://doi.org/10.1007/978-981-99-7962-2_39

[15] Makibuchi, N., Kato, H., & Yoneyama, A. (2013). Vision-based robust calibration for optical see-through head-mounted displays. 2013 IEEE International Conference on Image Processing, 2177–2181. DOI: https://doi.org/10.1109/ICIP.2013.6738449

[16] Wang, W., Wang, F., Liu, P., He, Y., & Zhang, X. (2016). A monovision- based 3D pose estimation system for vehicle behavior prediction. 2016 IEEE International Conference on Vehicular Electronics and Safety (ICVES), 1–6. DOI: https://doi.org/10.1109/ICVES.2016.7548177

[17] Deretey, E., Ahmed, M. T., Marshall, J. A., & Greenspan, M. (2015). Visual indoor positioning with a single camera using PnP. 2015 Interna- tional Conference on Indoor Positioning and Indoor Navigation (IPIN), 1–9. DOI: https://doi.org/10.1109/IPIN.2015.7346756

[18] Kubı(´) kov , H., Jedli ka, K., Fiala, R., & Beran, D. (2020). Indoor Positioning Using PnP Problem on Mobile Phone Images. ISPRS Inter- national Journal of Geo-Information, 9(6), 368. DOI: https://doi.org/10.3390/ijgi9060368

[19] Lepetit, V., Moreno-Noguer, F., & Fua, P. (2009). EPnP: An Accurate O(n) Solution to the PnP Problem. International Journal of Computer Vision, 81(2), 155–166. DOI: https://doi.org/10.1007/s11263-008-0152-6

[20] Watanabe, Y., Calise, A., & Johnson, E. (2007). Vision-Based Obstacle Avoidance for UAVs. AIAA Guidance, Navigation and Control Conference and Exhibit. Hilton Head, South Carolina. DOI: https://doi.org/10.2514/6.2007-6829