Real-Time Convolutional Neural Network-Based Method for Detecting and Tracking Human Motion on Quadcopters

Abstract

This paper proposes a convolutional neural network (CNN) method for human motion detection and tracking on a quadcopter. To address the challenges mentioned above, the proposed methodology is designed on computer vision techniques with an object tracking algorithm and a CNN model. The object tracking algorithm is implemented using a proportional integral differential (PID) controller to calculate the control parameters, including the pitch and yaw angles, in real time. These parameters are determined by calculating the offset between the position of the human and the camera coordinate frame. To achieve accurate object detection, a CNN model is designed based on the single shot multibox detector (SSD) architecture, which is crucial for object detection. The model above is integrated with the MobileNet base network, which is responsible for feature extraction of the object. The use of self-collected person data in model training ensures good performance for this specific application. The object detection results demonstrate that the model achieves a high level of accuracy (98%). The proposed methodology is applied to an NVIDIA Jetson NANO computer. To rigorously assess the control system, the proposed methodology was used to conduct outdoor flight tests on a campus. These tests prioritized minimal pedestrian traffic and stable weather conditions, ensuring a controlled environment for evaluation. Analysis of the flight data and signal graphs provided valuable insights into the effectiveness of the system.