Abstract
A recent work called I’m Watching You/Me (IMWYM) introduced a technical apparatus for live VR visualization of 360 drawings made on-the-fly. The exhibition [IN]Musicality uses and improves on the techniques presented in this first prototype but focusing on their artistic application. In fact, [IN]Musicality explores the artistic side of Hybrid Immersive Models through the interaction with spherical perspectives drawn to live music.
The article enumerates the issues from the first version of the artefact, presents possible ways to solve them, describes the solutions adopted for [IN]Musicality, and establishes milestones for the future roadmap of developments.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The software Eq A Sketch 360 is a raster-based drawing program that helps the user to learn about equirectangular perspective while drawing. The software uses the translational symmetry of equirectangular perspective to create a snap-to tool that allows the user to draw with precision an equirectangular line by choosing two arbitrary points or a special apex point [12].
- 2.
The collaborations were from professors and students from the course TAC (Technology, Architecture and Communication), Faculty of Architecture, National University of La Plata, Argentina. Professors: Tania Zuccari and Analía Jara. Students: Jimena Junco, Julian Alcalde, Marcelo Vias Alcacer, Agustina Castellini, Maria Pilar Ramos, Nadia Luengo, Catalina Vilas, Katherine Rosas Tantalean, Juan Pablo Simonini, Jimena Sierra, Martin Roque, Rocío Filoni, Pedro Troyano, Rodrigo Antonini, Celina Giustozzi.
- 3.
See dmad.ciac.pt and www.ualg.pt/en/curso/1714 for more information about the program.
- 4.
This material was produced and gathered for the collective curatorship [IN]Tangibildades Digitais (course “Curadoria partilhada”), for the digital logbook or DDB (course “Digital Media Art Project”), and finally also as for individual purposes.
- 5.
- 6.
- 7.
References
Olivero, L.F., Araújo, A.B.: I’m watching you/me. live drawing and VR visualization of spherical perspectives using TouchDesigner. In: Proceedings of ACM ARTECH Conference (ARTECH2021). ACM Press, Aveiro Portugal (2021). https://doi.org/10.1145/3483529.3483778
Araújo, A.B.: Ruler, compass, and nail: constructing a total spherical perspective. J. Math. Arts 12, 144–169 (2018). https://doi.org/10.1080/17513472.2018.1469378
Olivero, L.F.: Hybrid immersive models from cubical perspective drawings - modelli ibridi immersivi da disegni in prospettiva cubica (2021). https://lufo.art/hybrid-immersive-models-from-cubical-perspective-drawings-phd-thesis
Rossi, A., Olivero, L.F., Araújo, A.B.: For representation, a new reality: hybrid immersive models. In: Magnaghi-Delfino, P., Mele, G., Norando, T. (eds.) Faces of Geometry. LNNS, vol. 172, pp. 263–275. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-63702-6_20
Masiero Sgrinzatto, C.: Venice original: a spherical illustration of an imaginary environment. In: Proceedings of ACM ARTECH conference (ARTECH2021), pp. 679–682. ACM Press, New York (2021). https://doi.org/10.1145/3483529.3483754
Flores, M., Araújo, A.B.: Applications of anamorphosis and mixed reality in a classroom setting. In: 10th International Conference on Digital and Interactive Arts, pp. 1–10. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3483529.3483532
Araújo, A.B.: Drawing equirectangular VR panoramas with ruler, compass, and protractor. J. Sci. Technol. Arts 10, 15–27 (2018). https://doi.org/10.7559/citarj.v10i1.471
Araújo, A.B., Olivero, L.F., Rossi, A.: A descriptive geometry construction of VR panoramas in Cubical Spherical Perspective. Diségno. 35–46 (2020). https://doi.org/10.26375/disegno.6.2020.06
Grau, O.: Virtual Art: From Illusion to Immersion. MIT Press, Cambridge (2003)
Jerald, J.: The VR Book: Human-Centered Design for Virtual Reality. Association for Computing Machinery and Morgan & Claypool (2015)
Olivero, L.F., Araújo, A.B.: Desiderata for a performative hybrid immersive drawing platform. i-com. 21, 33–53 (2022). https://doi.org/10.1515/icom-2022-0009
Araújo, A.B.: Eq a sketch 360, a serious toy for drawing equirectangular spherical perspectives. In: Proceedings of the 9th International Conference on Digital and Interactive Arts, pp. 1–8. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3359852.3359893
da Veiga, P.A.: A/r/Cography: art, research and communication. In: Proceedings of the 9th International Conference on Digital and Interactive Arts. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3359852.3359859
Carpentier, N.: Arts-based research in communication and media studies: the example of silencing/unsilencing nature. XVI JORNADAS CIAC., Faro, Universidade do Algarve (2022)
Carpentier, N., Sumiala, J.: Introduction: arts-based research in communication and media studies. Comun. Soc.: J. Media Perform. Arts Cult. Stud.: Nuova Serie: XLIII 1, 2021 (2021). https://doi.org/10.26350/001200_000125
Lee, A.S.: Integrating positivist and interpretive approaches to organizational research. Organ. Sci. 2, 342–365 (1991)
Schultz, C.S., Legg, E.: A/r/tography: at the intersection of art, leisure, and science. Null 42, 243–252 (2020). https://doi.org/10.1080/01490400.2018.1553123
Leavy, P.: Methods Meets Art: Arts-Based Research Practice. The Guilford Press, New York (2020)
Zanela Saccol, A.: Um retorno ao básico: Compreendendo os paradigmas de pesquisa e sua aplicação na pesquisa em administração. Rev. Adm. Univ. Federal Santa Maria 2, 250–269 (2009)
Candy, L., Edmonds, E.: Practice-based research in the creative arts: foundations and futures from the front line. Leonardo 51, 63–69 (2018). https://doi.org/10.1162/LEON_a_01471
Sullivan, G.: Art Practice as Research: Inquiry in the Visual Arts. SAGE (2005)
Düchting, H.: Kandinsky. TASCHEN, Köln (2013)
Borges, J.L.: Borges esencial. Edicion Conmemorativa/Essential Borges: Commemorative Edition. Real Academia Española/Alfaguara, Madrid (2017)
Lopes, M.M., Bastos, P.B., Araújo, A.B., Olivero, L.F., Adérito, F.M. (eds.): Proceedings of ACM ARTECH Conference (ARTECH2021) - 10th International Conference on Digital and Interactive Arts. ACM Press, Aveiro Portugal (2021)
Acknowledgments
The author was supported by FCT national funds through projects UI/BD/150851/2021.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix A – Table of Issues and Solutions
Appendix A – Table of Issues and Solutions
1.1 A.1 Narrative/Development
Issue | Criticality and draft solution |
---|---|
1 - Missing concept. IMWYM 1 focused on a technical advancement. There was neither a clear artistic concept nor a defined narrative. As the goals of the research changed, the new edition focuses on the artistic application. Consequently, both concept and narrative were critical improvements for integrating IMWYM with the new goals. | Critical issues. IMWYM 2 would not have any sense without the definition of a concept and a narrative. Solution. Elaboration of a concept and a narrative for redirecting and integrating the artefact to/with artistic purposes, and for highlighting the particularities of its components. For example, the artefact has a strong focus on the interaction with 360 content, which might open a wide range of possibilities for exploring VR content using gamification. |
2 - Missing narrative. IMWYM 1 was included within the shared exhibition “Contingency” [24], yet neither limited nor oriented towards any specific direction by it. This lack of narrative was both with the general exhibition, and within the prototype. |
1.2 A.2 Interaction/Evolution
Issue | Criticality and draft solution |
---|---|
3 – Lack of a front-end interface. An interface was critically necessary for a more efficient and simplified operation. For example, one characteristic of the artefact was the possibility of switching or adding inputs, and then interacting with them. Yet, visitors would not know how to do it since the available screen was showing pure visual code, something not inviting to interact with. | Critical issue. The interface is the connection between the content and the interaction. As so, it is very important for a simplified operation and for gathering visitors’ feedback. Then, the data is the base for understating the phenomenological reading of the interaction with the artefact. Furthermore, the artist’s guidance is required without the interface (issue 5). Solution. Elaboration of a front-end interface that enables a fluent interaction with the content, gather data from it, and show a menu for visitors to get help and read information. |
4 - Dependence on the artist’s presence. Due to issue 3 and missing additional graphic material (flyers, posters), the interaction was very hard to understand, which required the authors’ presence for operating the artefact and explaining its functioning. | Critical issue. The artefact should stand by itself, with a minimum of explanations. Solution. Addition of supplementary material explaining the artefact distributed either digitally with an external device and/or with printed material |
5 - Fixed FOV. Currently, the field of view can be set in Touchdesigner but remains constant at runtime. It might be useful to add a function for setting it dynamically so as to zoom in and out while looking at details. | Low priority. This issue is not connected with the main functioning but with an advanced fruition and visualisation feature. Solution. Add a button/function on the interface to change the FOV dynamically. |
1.3 A.3 Experience/Fruition
Issue | Criticality and draft solution |
---|---|
6 – Software incompatibility. The first problem arose migrating from Mac to Windows. Using Touchdesigner on both OS, one might have expected a better integration. Yet, I needed to rework some key components. The second incompatibility was with the external camera (IN-5) since the one provided was dated. | Medium priority. Compatibility issues are a big problem if we are using a local computer. Solution. Have access to the local facilities with more anticipation to test extensively the artefact. Otherwise, it can be solved using the original laptop with which the artefact was built. In any case, it would be more opportune to have a web-based solution to avoid any OS’ limitations. |
7 - Navigation glitch. Compass and Gravity sensors determined the camera’s position. A glitch occurs during the navigation: there is a jump of the view under certain rotations. | Low priority. The navigation is less trustable, but still functioning. Solution. Use of Quaternion-based calculations instead of Euler angles. |
8 - Zig Sim app. The app needed to be maximized and onscreen to send the OSC data. Plus, the wireless connectivity was also an issue due to OS’ restrictions. Consequently, the artefact used a wired connection both to keep the phone’s battery loaded, and to simplify the net connection. | Medium-high priority. The problem avoids a smoother interaction with the content. Solution. A mobile application able to send OSC data running on the background. The creation of a dedicated application might be useful for other purposes (e.g., using the phone within a VR headset). |
9 – Limited inputs’ projection. The presented version was limited to the equirectangular projection. | Medium priority. It is important to integrate better the theory of the research. Solution. Add cubical and azimuthal-equidistant perspectives as inputs. |
10 - Calibration of the camera (IN-5). The calibration of the camera was very fragile, which forced a recalibration very often. | Medium-high priority. The live capturing should not rely on a physical calibration, but on a dynamic calibration based on the analysis of the image. Solution. Use of markers on the paper or board and live image-processing tools. |
11 - IN-2 and OU-2 were not live tested. Due to spatial and equipment constraints in the room. | Low-medium priority. The functioning of both components was verified during the programming stage. However, IN-2 might result difficult to whom is not familiarised with the device and spherical drawing. Hence, much likely IN-2 might be only on the interest of dedicated artists or teachers. OU-2 is needed for separating IN-3 and the interface. Solution. Larger space and equipment availability. |
12 - Setup IN-1 (physical drawing)/AR-1 (artist)/IN-5 (camera). This configuration generated a rather obvious problem: being the physical drawing hanged on the wall, the artist casted shadows while drawing and the visual captured by the camera was partially blocked. | Low priority. The problem is more aesthetical since does not block the functioning nor avoids the normal interaction with the artefact. Solution. Consider a setup for drawing onto a glass (or something transparent), and a camera capturing the drawing from the rear side. |
13 - Setup IN-1 (physical drawing)/OU-1. During the first presentation, IN-1 and OU-1 were almost parallel and within the same line. Therefore, it was very complicated for the artist to watch the VR results and having instant visual feedback. | Low priority. The issue is only for the artist using the artefact and not for regular visitors. Solution. Build a flexible setup considering the installation with and without the live drawing performance and pay attention to the perspective that both artist and visitors have. |
Rights and permissions
Copyright information
© 2023 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Olivero, L.F. (2023). [IN]Musicality: A Collection of VR Drawings and Music as an Artistic Application of Hybrid Immersive Models. In: Brooks, A.L. (eds) ArtsIT, Interactivity and Game Creation. ArtsIT 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 479. Springer, Cham. https://doi.org/10.1007/978-3-031-28993-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-28993-4_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28992-7
Online ISBN: 978-3-031-28993-4
eBook Packages: Computer ScienceComputer Science (R0)