Scalable Topological and Geometric Methods for Multimodal Activity Modeling

Principal Investigator

Pavan Turaga

Award Amount

$450,657

Sponsor

US Department of Defense (DOD)

Grant start date

Grant end date

The Challenge

Computer vision and machine-learning techniques have revolutionized our world in recent years. However, for robust operation, these techniques need very large amounts of data and annotation. This requirement places real limits on their utility in critical situations where access to data may be limited.

The Approach

Our research addresses this problem by taking a physics and mathematics-based approach to modeling difficult sources of variations, without needing large amounts of data. Our approach spans new mathematical techniques from geometry and topology, fused with deep-learning methodologies.

Findings and Impact

Our findings indicate that the robustness and performance of current approaches for computer vision and machine-learning can be significantly improved with a more careful choice of mathematical models, that pay attention to phenomena such as image-formation physics, illumination, and motion. Our approaches are applied to video-based activity analysis problems of relevance to the US Army.