Object Detection and Moving Object Tracking by Using Drones (TUBITAK 1001-116E080)

The goal of this project is detection and tracking of objects in a region by using a team of drones (quadrators). The drones will collobarate with each other and share the given tasks. Thus, the project involves visual object detection, tracking, task allocation and execution of given tasks. To the best of our knowledge, the unmanned aerial vehicles (UAVs) (majority of these are made by Israel) used by Turkish military operate manually by an operator and do not share the tasks. Moreover, the UAVs produced by Turkish companies do not have detection and tracking capabilities.

Real-time Object Detection using Polyhedral Conic Classifiers

The demo of the object tracking software delivered to MHAVK Group for Teknofest - Fighting UAVs contest (Teknofest Savasan IHA Yarismasina Katilan MHAVK Grubuna Teslim Ettigimiz Nesne Takip Yaziliminin Demosu)

About Project

Visual object localization in digital images is a hard task and the main difficulty of object localization arises from high variability in appearance, color and texture among objects of the same class. Also changes in scale and viewpoints under various lighting conditions and complex background clutters make the problem even harder. Traditional object localization methods usually treat object localization problem as a binary classification task, namely, distinguishing between the object class and the background class. The trained classifier is turned into a detector by sliding it across the image or image pyramid and classifiying each such local window. In this project, we will use polyhedral conic classifiers that are developed especially for visual object detection tasks.

Visual tracking of moving objects is the task of returning object locations in a video frame. This task is relatively easier when the camera is fixed or the object moves with a constant speed in one direction. But when both the object and camera are moving as in our application, the task is more difficult. The most succesful tracking algorithms use motion cues to return possible locations of the objects in the new frames and an object detector is applied to these regions to find the location of the objects. We will also apply a similar procedure in this project, but we will design the classifier of the detector such that it adapts itself to appearance changes of the moving objects.

Allocation of the tasks among the drones is very important for the overall efficiency. Many factors must be taken into consideration for this goal. For example, tracking of a vehicle carrying an important person may be more important than tracking of other vehicles. Also, the drones may differ in terms of the sensors they carry, their speed and power consumption. So, sometimes it may be more convenient to assign a task to a drone which has better sensors or needs less power to operate. So, task allocation must take many factors into consideration. Market based approaches provide this. Once the tasks are determined, drones bid for the tasks and the tasks must be assigned optimally among the drones. This is also known as auction clearing.

In this project, we will use an algorithm that provides the optimal solution for this market based approach. The optimization will aim to minimize the distance between the location of the task and the drone. On the other hand, other criteria such as velocity of the drones or power consumption could be easily changed with the determined one. Once the tasks are shared, the drones are expected to execute the given task without colliding with other drones or other objects such as trees or buildings. This requires that the drones must estimate their true locations as wells as the locations of other drones, communicate with each other and process the images and have a good action plan.

See our initial results on visual object tracking !

Publications

Hakan Cevikalp, Halil Saglamlar
Polyhedral Conic Classifiers for Computer Vision Applications and Open Set Recognition
IEEE Transactions on PAMI (in review)

Hakan Cevikalp, Emre Cimen, Gurkan Ozturk
The Nearest Polyhedral Convex Conic Regions for High-Dimensional Classification
Machine Learning (in review)

Hakan Cevikalp, Bill Triggs
Polyhedral Conic Classifiers for Visual Object Detection and Classification
CVPR 2017

Hasan Saribas, Hakan Cevikalp, Sinem Kahvecioglu
İnsansız Hava Araçlarından Alınan Görüntülerdeki Araçların Konumlarının Bulunması
2018 26th Signal Processing and Communications Applications Conference (SIU)

Hasan Saribas, Hakan Cevikalp, Sinem Kahvecioglu
Car Localization in Aerial Images Taken from Quadcopter
RTET–2018

A video of quadrotor designed for the project TUBITAK 2209-B Sanayiye Yonelik Lisans Projesi Destegi - Autonomously flying quadrotor design for indoor environments

(Kapalı ortamda bir konumdan başka bir konuma uçan otonom quadrotor tasarımı başlıklı proje kapsamında oluşturulan quadrotor videosu)

Videos of autonomous mobile robot designed for the project TUBITAK 2209-B Sanayiye Yonelik Lisans Projesi Destegi - Autonomous mobile robot design for indoor environments

(Bina içi otonom akıllı kara aracı tasarımı başlıklı proje kapsamında oluşturulan kara aracı videosu)