Research on Human-Computer Re-Interaction in AR Books

Abstract

With the advancement of technology, the application of Augmented Reality (AR) technology in the field of books is becoming increasingly widespread. The common application of this technology involves presenting different virtual elements by combining them with the real environment. Aimed at enhancing the fun and richness of the reading experience, this paper explores the enrichment of human-computer interaction design through the recognition of reader actions using a camera after the presentation of virtual elements. For action recognition, the color differences in the image are compared, noise and irrelevant information are removed, and the color coding of the effective information area is converted into a binary graphic form, displayed as “bright” (assigned “1”) in the constructed matrix; while the ineffective area is displayed as “dark” (assigned “0”). When actions change, the areas of “1” and “0” in the matrix also adjust accordingly, thus achieving real-time recognition of dynamic actions. Based on this process, human-computer interaction design captures reader actions through the camera and triggers responses related to the invoked virtual elements. This not only provides readers with a more immersive reading experience but also enhances the richness of the displayed virtual reality content. Additionally, it offers new perspectives for interactive book design. The focus of this paper is on exploring the potential application of action recognition technology in interactive design methods within AR books.