SeViAnno 2.0: Web-Enabled Collaborative Semantic Video Annotation Beyond the Obvious
•
1 like•1,623 views
This document discusses SeViAnno 2.0, a web-enabled collaborative semantic video annotation system. It presents the motivation and background of multimedia annotation tools. SeViAnno 2.0 has a 3-tier architecture that is cloud-enabled, scalable, and allows rapid development. It utilizes multimedia metadata web services and a user interface for collaborative tagging. Near real-time collaboration is supported through data structure dependencies and update propagation protocols. Future work includes developing a WebRTC infrastructure and operational transformation algorithms for distributed collaborative semantic annotations.
SeViAnno 2.0: Web-Enabled Collaborative Semantic Video Annotation Beyond the Obvious
- 1. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-1 TeLLNet Learning Layers This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. SeViAnno 2.0: Web-Enabled Collaborative Semantic Video Annotation Beyond the Obvious Petru Nicolaescu & Ralf Klamma RWTH Aachen University Advanced Community Information Systems (ACIS) {lastname}@dbis.rwth-aachen.de
- 2. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-2 TeLLNet Learning Layers Responsive Open Community Information Systems Community Visualization and Simulation Community Analytics Community Support WebAnalytics WebEngineering Advanced Community Information Systems (ACIS) Requirements Engineering
- 3. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-3 TeLLNet Learning Layers Agenda Motivation Background A Web information system for video annotation – Collaborative – Near real-time – Cloud-enabled SeViAnno 2.0: design and implementation Conclusions and outlook
- 4. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-4 TeLLNet Learning Layers Motivation or „We kill people based on metadata“ (David Cole) Good tools for automatic multimedia annotation available „for special domains“ Good single user tools for multimedia annotation Beyond the obvious: (Web) tagging the non-visible Collaborative multimedia annotation Decoupled infrastructure + + - - -
- 5. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-5 TeLLNet Learning Layers Related Work SVCAT VideoAnnEx videoANT M-OntoMat 2.0 SeViAnno 2.0 Tool Platform Usage Tagging Method Desktop Web-based Single user Collaborative Manual Automatic ++ ++ + ++ ++ ++ + ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ M-OntoMat 2.0 + Via-tool SVCAT VideoAnnEx videoANT
- 6. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-6 TeLLNet Learning Layers EU FP7 Learning Layers Project How can we scale up video tagging to support informal learning at the work place Objectives – Mobile creation of videos with learning materials (physical-digital world) – Tag the non-obvious for informal learning in communities (social media layer) – Scaffold meaningful learning by exploiting semantic tagging information (social semantic layer) Two regional clusters – Construction (Germany) – Healthcare (UK) http://learning-layers.eu/
- 7. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-7 TeLLNet Learning Layers Decoupled Multimedia Annotation Information System Architecture 3-tier architecture Cloud-enabled solutions Advantages – Separation of concerns – Scalability – Rapid development & deployment – Application customization
- 8. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-8 TeLLNet Learning Layers Multimedia Metadata Web Services Tethys (former i5Cloud) LAS SeViAnno MPEG-7 Metadata Services Connectors (REST, HTTP, Ajax) User Management Object Management Session Management Cloud Video Transcoder Tethys Data Storage Video Transcoding Scaling Management Cloud Video Upload ROLE SDK NRT Collaboration Widget/Space Management Streaming Server
- 9. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-9 TeLLNet Learning Layers Multimedia Metadata User Interface
- 10. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-10 TeLLNet Learning Layers Near Real-Time (NRT) Collaboration Data structure dependent – Tree-like data structures with operational transformations – e.g. XML-based (MPEG-7) NRT updates propagation – Underlying infrastructure and used protocols (e.g. XMPP) Preserving data consistency and user intention – NRT video operations – Concurrent updates
- 11. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-11 TeLLNet Learning Layers Space (shared by multiple users) SeViAnno 2.0 & ROLE Framework Web application (composed of widgets) Widget (collaborative web component) EU FP7 ROLE Project http://role-sandbox.eu/
- 12. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-12 TeLLNet Learning Layers User Interface Distribution Limited space and interaction possibilities Trend to using multiple personal devices Lack of tools and methodologies for Web-based DUIs
- 13. Lehrstuhl Informatik 5 (Information Systems) Prof. Dr. M. Jarke I5-NiKl-0714-13 TeLLNet Learning Layers Conclusion and Future Work Open Source Development and Cloud Deployment (ask for details) Strong and scalable service infrastructure (Distributed) Web widget and mobile interfaces Challenges – WebRTC infrastructure for update propagation – Operational transformation algorithms for collaborative semantic annotations – Parallel video processing in the cloud (MapReduce) – OpenID Connect (OIDC) for security and privacy