Submitted Abstract
Visual surveillance systems are more than ever required to increase their performance in order to better detect, and eventually prevent, criminal as well as terrorist attacks. Indeed, in the aftermath of the Charlie Hebdo and the kosher grocery store terrorist attacks in Paris, it is clear that today no one is immune to such threats. At the local level, a 2012 Eurostat study showed a scary increase in criminality, ranking the City of Luxembourg as “the third most dangerous capital in Europe”. Automatically detecting abnormal or undesirable behaviours would certainly provide a decisive support in case of contingency, and could be a good deterrent. Action recognition systems have been extensively researched by the computer vision community, and interesting results have been achieved using images or videos captured with conventional 2D cameras. However, there is still no system that can robustly and effectively perform under real-world conditions, where there is a constant change in illumination, texture, occlusions and viewpoint. Our goal, in the proposed effort, is to lift these four limitations. By using RGB-D cameras, from which 3D information can be coupled with colour information, sensitivity to illuminations and textures can be largely reduced. Our research problem may then be casted as a 3D action recognition problem. In order to tackle sensor specific properties, and define a system that is invariant to occlusions and viewpoint variations, we will investigate three distinct and complementary research axes:1)3D trajectories for action recognition from RGB-D data,2)Selection and refinement strategies for improved trajectory-based action recognition,3)Selection and refinement strategies for improved skeleton-based action recognition.We will develop theoretical models for each axis, and target a final unified framework to be integrated in one 3D action recognition system. While the main objective of 3D-ACT is to automatically detect abnormal and suspicious behaviour of humans from surveillance videos, it is also necessary to maintain the balance between security in critical infrastructures and privacy of individuals at the same time. We will go after such a balance by directly incorporating privacy in the design of our systems, in line with our work on privacy-preserving pattern recognition. Evaluations and testing of our algorithms will be carried out in the SnT Computer Vision laboratory, partially funded by the FNR under the CORE project FAVE.