Scalable Topological and Geometric Methods for Multimodal Activity Modeling
The Challenge
Computer vision and machine-learning techniques have revolutionized our world in recent years. However, for robust operation, these techniques need very large amounts of data and annotation. This requirement places real limits on their utility in critical situations where access to data may be limited.
The Approach
Our research addresses this problem by taking a physics and mathematics-based approach to modeling difficult sources of variations, without needing large amounts of data. Our approach spans new mathematical techniques from geometry and topology, fused with deep-learning methodologies.
Findings and Impact
Our findings indicate that the robustness and performance of current approaches for computer vision and machine-learning can be significantly improved with a more careful choice of mathematical models, that pay attention to phenomena such as image-formation physics, illumination, and motion. Our approaches are applied to video-based activity analysis problems of relevance to the US Army.