Emerging challenges and opportunities in exploiting mobile photos and videos

The prevalence of mobile cameras and social media sharing services have drastically increased the volume of photos and videos, which pose a new challenge for developing efficient retrieval, mining, and visualization approaches. Meanwhile, the mobile devices have almost been the must in daily life for capturing important moments, browsing the photos and videos shared by the families and friends, or working as Q&A devices for understanding the objects of interests (e.g., landmarks, products, video programs, or restaurant reviews, etc.) by taking a snapshot and searching the related information. In this talk, we will first look into the emerging opportunities and promising solutions to manipulate such large-scale data through mobile devices. We will point the properties that we can leverage from mobile devices and go through the challenges (e.g., small mobile screen size, low bandwidth, limited storage and computing power), which further pose new constraints in photo/video content analysis and retrieval in the emerging mobile platform. Meanwhile, instead of treating the user-generated photos/videos as independent workloads, we will demonstrate the exciting applications in mobile recommendation and human activity mining as, in a holistic manner, analyzing such sheer amount of photos and videos, which are generally to record people activities and those the most memorable for the users. We will also review emerging industrial and academic research problems and the pilot experiments over million-scale photo/video collections.