Knowledge Extraction in Video Through the Interaction Analysis of Activities

Abstract

A video is a growing stream of unstructured data that significantly increases the amount of information transmitted and stored on the Internet. For example, every minute YouTube users upload 72 GB of information. Some of the best applications for video analysis include the monitoring of activities in defense and security scenarios such as the autonomous planes that collect video and images at reduced risk and the surveillance cameras in public places like traffic lights, airports, and schools. Some of the challenges in the analysis of video correspond to implement complex operations such as searching of activities, understanding of scenes, and predicting of new content. Those techniques must transform a video into a collection of more informative structures (features, objects, and models) to be analyzed by a computer by considering the relationships of causality and co-occurrence between hidden and observable variables. The research presented in this dissertation proposes and demonstrates novel algorithms for automatic and semi-automatic analysis of activities in video. The analysis of video is approached with machine learning, databases, and information retrieval algorithms to find repeating patterns and extract co-occurring activities and label them appropriately. The experiments demonstrate that the proposed techniques overcome the limitations associated to each of those operations in a better way than existing techniques