Real-world interaction with camera phones

With the integration of cameras, mobile phones have evolved into networked personal image capture devices. Camera phones can perform image processing tasks on the device itself and use the result as an additional means of user input and a source of context data. In this paper we present a system that turns such phones into mobile sensors for 2-dimensional visual codes. The proposed system induces a code coordinate system and visually detects phone movements. It also provides the rotation angle and the amount of tilting of the camera as additional input parameters. These features enable applications such as item selection and interaction with large-scale displays. With the code coordinate system, each point in the viewed image – and therefore arbitrarily shaped areas – can be linked to specific operations. A single image point can even be associated with multiple information aspects by taking different rotation and tilting angles into account.