Employing a RGB-D Sensor for Real-Time Tracking of Humans across Multiple Re-Entries in a Smart Environment

Abstract

The term smart environment refers to physical \nspaces equipped with sensors feeding into adaptive algorithms \nthat enable the environment to become sensitive and \nresponsive to the presence and needs of its occupants. People \nwith special needs, such as the elderly or disabled people, \nstand to benefit most from such environments as they offer \nsophisticated assistive functionalities supporting independent \nliving and improved safety. In a smart environment, the key \nissue is to sense the location and identity of its users. In this \npaper, we intend to tackle the problems of detecting and \ntracking humans in a realistic home environment by exploiting \nthe complementary nature of (synchronized) color and depth \nimages produced by a low-cost consumer-level RGB-D \ncamera. Our system selectively feeds the complementary data \nemanating from the two vision sensors to different algorithmic \nmodules which together implement three sequential \ncomponents: (1) object labeling based on depth data \nclustering, (2) human re-entry identification based on \ncomparing visual signatures extracted from the color (RGB) \ninformation, and (3) human tracking based on the fusion of \nboth depth and RGB data. Experimental results show that this \ndivision of labor improves the system\xe2\x80\x99s efficiency and \nclassification performance