It is a challenge to introduce and practice computational modes of visual narrative in art and de... more It is a challenge to introduce and practice computational modes of visual narrative in art and design and computer science classroom environments. Students need to learn, practice and synthesize a collection of multidisciplinary skills when they develop projects (software development, art production, animation, interactivity, game design, electronics, among others), and such efforts often require collaborative work in team-based environments. These challenges are especially acute in domains such as augmented, mixed and virtual reality, where teams are also tasked to innovate in emerging platforms, as standards, affordances and potentials are open-ended and changing rapidly. As students address such design problems with creativity, they practice divergent thinking, and ingenuity, solving problems in inventive ways-such skills are difficult to teach. Emerging curricular strategies sometimes focus on fostering collaborative spaces and challenges that require multidisciplinary problem solving for unorthodox scenarios and the interplay of technology, arts, and design thinking. That is the focus of this paper. We present a project and technology platform called Story-Go-Round. The particulars of the platform are evolving-it was developed over the course of an academic semester, as a studio assignment for art and design students. The project involves a collection of technologies and a flexible, open-source design-a depth camera, actuated physical stage, basic electronics, digitally fabricated cabinet, computer and game controller controls, and a series of software templates and development environments. The students are asked to consider the affordances of the system and consider its potential to be modified, hacked and hybridized, towards the goal of creating a novel, cyclical storytelling experience. Story-Go-Round is a mythotrope, meaning "story turning", a term coined by the authors as a reference to a zoetrope, a historically significant cylindrical pre-film animation device developed in the 19 th century. It is an experimental physical development platform for producing augmented digital experiences. Students physically construct dioramas in an actuated carousel, modify the hardware platform, and then integrate the hand-crafted sets and engineered technical system with digital content, animation, interactivity, gameplay and effects. The platform has a low barrier to entry, it first involves crafting using methods familiar from childhood. It then introduces foundational skills, allowing the students to show off expert skills while at the same time crossing discipline domains into new areas. Along the way students consider the history of animation, video game consoles and digital media. As educators in this highly virtualized era, we are tempted to shift the focus exclusively to digital tools; however, this limits the design that students produce to choices and levels of complexity offered by software, and the results become alike and repetitive. We wanted to create an assignment that functions by combining two made-up worlds: one that is physically produced and another that is digitally animated and controlled. When students try to develop a multi-faceted project like this, which crosses multiple technical and artistic domains, and try to quickly learn the skills they need to implement their vision, they get to experience the remixing, mashup, open-source making culture, which is so prominent in the way we work today. How else can young designers best learn the duct-tape and jerry-rigging skills which will allow them to see potential opportunities in disparate things, and find new ways of hacking them together to create something new, playful and whimsical, even if what they create is also "absurd", and often not practical?
Project-based courses in Art + Design and Computer Science curriculum foster innovation spaces th... more Project-based courses in Art + Design and Computer Science curriculum foster innovation spaces that require creative thinking and multidisciplinary problem solving for unorthodox scenarios through the interplay of technology, arts, and design thinking. “Story-Go-Round” is a physical development platform for producing augmented digital experiences, designed exclusively as an assignment to tell stories through interactive control and gameplay via circular rotating sets. We propose a miniature mechanical and replaceable “carousel-like” stage linked to a game engine, which controls the rotation using a motor as well as streaming live camera feeds of the stage directly into the game environment. The assignment is inspired by the historical aspects of replaceable computer media such as game cartridges, and early animation devices such as the zoetrope and animated concertina. The stage component is open to continuous physical interaction for an evolving set design, like the process used in...
Todd Berreth Todd Berreth is a designer, programmer and architect, experimenting with emerging te... more Todd Berreth Todd Berreth is a designer, programmer and architect, experimenting with emerging technologies for architectural applications, advanced visualization, and novel human-computer interaction. His research interests focus on the pervasive use of display and sensor technologies in the designed environment, as well as potential opportunities for creating intelligent, useful and engaging spaces and experiences, particularly within the contexts of museums and art. Berreth is affiliated with Duke University, as an instructor and research associate in the Art, Art History and Visual Studies Department / Visualization and Interactive Systems Group. He heads his own research-based design practice, Studio Berreth, that explores the dynamic relationship between people and their media, digital technology and the built world. More information can be found at http://studioberreth.io or via email at todd@studioberreth.io.
There is a crisis in our communities about the tributes to a shared civic life represented in exi... more There is a crisis in our communities about the tributes to a shared civic life represented in existing public artwork and monuments. Culture wars are being waged herein and appear increasingly unreconcilable. This paper discusses this moment and describes the range of strategies artists and designers have used to remediate these works. It presents a project description of an interactive artwork that suggests innovative approaches in this realm. The author introduces a conceptual model which served as inspiration for the piece that may be useful when discussing and designing such interventions.
It is a challenge to introduce and practice computational modes of visual narrative in art and de... more It is a challenge to introduce and practice computational modes of visual narrative in art and design and computer science classroom environments. Students need to learn, practice and synthesize a collection of multidisciplinary skills when they develop projects (software development, art production, animation, interactivity, game design, electronics, among others), and such efforts often require collaborative work in team-based environments. These challenges are especially acute in domains such as augmented, mixed and virtual reality, where teams are also tasked to innovate in emerging platforms, as standards, affordances and potentials are open-ended and changing rapidly. As students address such design problems with creativity, they practice divergent thinking, and ingenuity, solving problems in inventive ways-such skills are difficult to teach. Emerging curricular strategies sometimes focus on fostering collaborative spaces and challenges that require multidisciplinary problem solving for unorthodox scenarios and the interplay of technology, arts, and design thinking. That is the focus of this paper. We present a project and technology platform called Story-Go-Round. The particulars of the platform are evolving-it was developed over the course of an academic semester, as a studio assignment for art and design students. The project involves a collection of technologies and a flexible, open-source design-a depth camera, actuated physical stage, basic electronics, digitally fabricated cabinet, computer and game controller controls, and a series of software templates and development environments. The students are asked to consider the affordances of the system and consider its potential to be modified, hacked and hybridized, towards the goal of creating a novel, cyclical storytelling experience. Story-Go-Round is a mythotrope, meaning "story turning", a term coined by the authors as a reference to a zoetrope, a historically significant cylindrical pre-film animation device developed in the 19 th century. It is an experimental physical development platform for producing augmented digital experiences. Students physically construct dioramas in an actuated carousel, modify the hardware platform, and then integrate the hand-crafted sets and engineered technical system with digital content, animation, interactivity, gameplay and effects. The platform has a low barrier to entry, it first involves crafting using methods familiar from childhood. It then introduces foundational skills, allowing the students to show off expert skills while at the same time crossing discipline domains into new areas. Along the way students consider the history of animation, video game consoles and digital media. As educators in this highly virtualized era, we are tempted to shift the focus exclusively to digital tools; however, this limits the design that students produce to choices and levels of complexity offered by software, and the results become alike and repetitive. We wanted to create an assignment that functions by combining two made-up worlds: one that is physically produced and another that is digitally animated and controlled. When students try to develop a multi-faceted project like this, which crosses multiple technical and artistic domains, and try to quickly learn the skills they need to implement their vision, they get to experience the remixing, mashup, open-source making culture, which is so prominent in the way we work today. How else can young designers best learn the duct-tape and jerry-rigging skills which will allow them to see potential opportunities in disparate things, and find new ways of hacking them together to create something new, playful and whimsical, even if what they create is also "absurd", and often not practical?
The proliferation of virtual museums, on line and built inside a museum's collections, stimulates... more The proliferation of virtual museums, on line and built inside a museum's collections, stimulates new challenges for research and public education. This happens through multimodal applications and methods, such as augmented reality, virtual storytelling, virtual reality, computer graphics and haptic applications. This paper discusses a haptic approach with a hybrid solution, able to virtualize tangible objects in a physical space and digital models in a cyber space. The main goal of this approach is to propose alternative ways to experience an object, an artifact, a site or an entire collection. In particular, the hands-on experience is very important, for having tangible feedback from objects and to generate a visual narrative, by touching and combining physical/virtual interaction simultaneously, to customize the visit according to the experience and empathy generated by the unique artifacts. The Tangible Interactive Table for Archeology (TITA) is a open-source prototype of a digital-haptic device, designed for museums and in general for the digital communication of archaeological artifacts, monuments and sites. The intention is to lower the bar of entry for this type of technology in the cultural heritage community, to enable further research and exploration of its affordances, by initiating a community open design project, which provides the digital fabrication files and specifications to produce, test, extend and hybridize such a device, and mechanisms for sharing mutual hardware and software development projects. Such efforts to combine virtual reality and tangible 3d printed artifacts, also entails a rethinking of the production pipeline of digitally reconstructing 3d environments and their constituent elements, where 3d digital models need to be at turns both " lightweight " , visually accurate yet low-poly for real-time, game engine applications, and high-fidelity and well formed as 3d printed artifacts.
This project uses machine learning and computer vision techniques and a novel interactive visuali... more This project uses machine learning and computer vision techniques and a novel interactive visualization tool to provide street-level characterization of urban spaces such as safety and maintenance in urban neighborhoods. This is achieved by collecting and annotating street-view images, extracting objective metrics through computer vision techniques, and using crowdsourcing to statistically model the perception of subjective metrics such as safety and maintenance. For modeling human perception and scaling it up with a predictive algorithm, we evaluate perception predictions across two points in time separated by economic changes in the urban core of Raleigh, North Carolina, in the aftermath of the 2008 Great Recession. We hypothesize specific socioeconomic processes can be substantially reflected in the built environment of cities and, thus, render themselves visible at the street level. This paper describes the process of incorporating subjective visual ratings across two datasets of temporally separated street-view images, an algorithm, and a visualization tool. This work serves as a case study for utilizing AI and visualization techniques in a richer characterization of urban spaces that includes both objective metrics such as income (that operates at a broader scale) and subjective metrics such as perception of individuals (that operates at a narrower scale at specific locations). We outline an interdisciplinary methodology to test this hypothesis in streetscape data from Raleigh, NC, from 2008 to 2020. We describe the results of training algorithms that utilized image features with crowdsourced human perception ratings. We provide a comparison of the results with income data. The analysis and interpretation of this comparison provide insight into the challenges and opportunities for using AI technology in characterizing changes in urban environments. One challenge is the ability of human domain experts to interpret the output of algorithms through manipulation and to integrate these results into their workflow. This is addressed with a novel interface designed for interactive analysis and visualization. We conclude with a discussion of some of the benefits and limitations of integrating AI models in the human expert's decision-making process in the presence of both subjective and objective metrics.
It is a challenge to introduce and practice computational modes of visual narrative in art and de... more It is a challenge to introduce and practice computational modes of visual narrative in art and design and computer science classroom environments. Students need to learn, practice and synthesize a collection of multidisciplinary skills when they develop projects (software development, art production, animation, interactivity, game design, electronics, among others), and such efforts often require collaborative work in team-based environments. These challenges are especially acute in domains such as augmented, mixed and virtual reality, where teams are also tasked to innovate in emerging platforms, as standards, affordances and potentials are open-ended and changing rapidly. As students address such design problems with creativity, they practice divergent thinking, and ingenuity, solving problems in inventive ways-such skills are difficult to teach. Emerging curricular strategies sometimes focus on fostering collaborative spaces and challenges that require multidisciplinary problem solving for unorthodox scenarios and the interplay of technology, arts, and design thinking. That is the focus of this paper. We present a project and technology platform called Story-Go-Round. The particulars of the platform are evolving-it was developed over the course of an academic semester, as a studio assignment for art and design students. The project involves a collection of technologies and a flexible, open-source design-a depth camera, actuated physical stage, basic electronics, digitally fabricated cabinet, computer and game controller controls, and a series of software templates and development environments. The students are asked to consider the affordances of the system and consider its potential to be modified, hacked and hybridized, towards the goal of creating a novel, cyclical storytelling experience. Story-Go-Round is a mythotrope, meaning "story turning", a term coined by the authors as a reference to a zoetrope, a historically significant cylindrical pre-film animation device developed in the 19 th century. It is an experimental physical development platform for producing augmented digital experiences. Students physically construct dioramas in an actuated carousel, modify the hardware platform, and then integrate the hand-crafted sets and engineered technical system with digital content, animation, interactivity, gameplay and effects. The platform has a low barrier to entry, it first involves crafting using methods familiar from childhood. It then introduces foundational skills, allowing the students to show off expert skills while at the same time crossing discipline domains into new areas. Along the way students consider the history of animation, video game consoles and digital media. As educators in this highly virtualized era, we are tempted to shift the focus exclusively to digital tools; however, this limits the design that students produce to choices and levels of complexity offered by software, and the results become alike and repetitive. We wanted to create an assignment that functions by combining two made-up worlds: one that is physically produced and another that is digitally animated and controlled. When students try to develop a multi-faceted project like this, which crosses multiple technical and artistic domains, and try to quickly learn the skills they need to implement their vision, they get to experience the remixing, mashup, open-source making culture, which is so prominent in the way we work today. How else can young designers best learn the duct-tape and jerry-rigging skills which will allow them to see potential opportunities in disparate things, and find new ways of hacking them together to create something new, playful and whimsical, even if what they create is also "absurd", and often not practical?
Project-based courses in Art + Design and Computer Science curriculum foster innovation spaces th... more Project-based courses in Art + Design and Computer Science curriculum foster innovation spaces that require creative thinking and multidisciplinary problem solving for unorthodox scenarios through the interplay of technology, arts, and design thinking. “Story-Go-Round” is a physical development platform for producing augmented digital experiences, designed exclusively as an assignment to tell stories through interactive control and gameplay via circular rotating sets. We propose a miniature mechanical and replaceable “carousel-like” stage linked to a game engine, which controls the rotation using a motor as well as streaming live camera feeds of the stage directly into the game environment. The assignment is inspired by the historical aspects of replaceable computer media such as game cartridges, and early animation devices such as the zoetrope and animated concertina. The stage component is open to continuous physical interaction for an evolving set design, like the process used in...
Todd Berreth Todd Berreth is a designer, programmer and architect, experimenting with emerging te... more Todd Berreth Todd Berreth is a designer, programmer and architect, experimenting with emerging technologies for architectural applications, advanced visualization, and novel human-computer interaction. His research interests focus on the pervasive use of display and sensor technologies in the designed environment, as well as potential opportunities for creating intelligent, useful and engaging spaces and experiences, particularly within the contexts of museums and art. Berreth is affiliated with Duke University, as an instructor and research associate in the Art, Art History and Visual Studies Department / Visualization and Interactive Systems Group. He heads his own research-based design practice, Studio Berreth, that explores the dynamic relationship between people and their media, digital technology and the built world. More information can be found at http://studioberreth.io or via email at todd@studioberreth.io.
There is a crisis in our communities about the tributes to a shared civic life represented in exi... more There is a crisis in our communities about the tributes to a shared civic life represented in existing public artwork and monuments. Culture wars are being waged herein and appear increasingly unreconcilable. This paper discusses this moment and describes the range of strategies artists and designers have used to remediate these works. It presents a project description of an interactive artwork that suggests innovative approaches in this realm. The author introduces a conceptual model which served as inspiration for the piece that may be useful when discussing and designing such interventions.
It is a challenge to introduce and practice computational modes of visual narrative in art and de... more It is a challenge to introduce and practice computational modes of visual narrative in art and design and computer science classroom environments. Students need to learn, practice and synthesize a collection of multidisciplinary skills when they develop projects (software development, art production, animation, interactivity, game design, electronics, among others), and such efforts often require collaborative work in team-based environments. These challenges are especially acute in domains such as augmented, mixed and virtual reality, where teams are also tasked to innovate in emerging platforms, as standards, affordances and potentials are open-ended and changing rapidly. As students address such design problems with creativity, they practice divergent thinking, and ingenuity, solving problems in inventive ways-such skills are difficult to teach. Emerging curricular strategies sometimes focus on fostering collaborative spaces and challenges that require multidisciplinary problem solving for unorthodox scenarios and the interplay of technology, arts, and design thinking. That is the focus of this paper. We present a project and technology platform called Story-Go-Round. The particulars of the platform are evolving-it was developed over the course of an academic semester, as a studio assignment for art and design students. The project involves a collection of technologies and a flexible, open-source design-a depth camera, actuated physical stage, basic electronics, digitally fabricated cabinet, computer and game controller controls, and a series of software templates and development environments. The students are asked to consider the affordances of the system and consider its potential to be modified, hacked and hybridized, towards the goal of creating a novel, cyclical storytelling experience. Story-Go-Round is a mythotrope, meaning "story turning", a term coined by the authors as a reference to a zoetrope, a historically significant cylindrical pre-film animation device developed in the 19 th century. It is an experimental physical development platform for producing augmented digital experiences. Students physically construct dioramas in an actuated carousel, modify the hardware platform, and then integrate the hand-crafted sets and engineered technical system with digital content, animation, interactivity, gameplay and effects. The platform has a low barrier to entry, it first involves crafting using methods familiar from childhood. It then introduces foundational skills, allowing the students to show off expert skills while at the same time crossing discipline domains into new areas. Along the way students consider the history of animation, video game consoles and digital media. As educators in this highly virtualized era, we are tempted to shift the focus exclusively to digital tools; however, this limits the design that students produce to choices and levels of complexity offered by software, and the results become alike and repetitive. We wanted to create an assignment that functions by combining two made-up worlds: one that is physically produced and another that is digitally animated and controlled. When students try to develop a multi-faceted project like this, which crosses multiple technical and artistic domains, and try to quickly learn the skills they need to implement their vision, they get to experience the remixing, mashup, open-source making culture, which is so prominent in the way we work today. How else can young designers best learn the duct-tape and jerry-rigging skills which will allow them to see potential opportunities in disparate things, and find new ways of hacking them together to create something new, playful and whimsical, even if what they create is also "absurd", and often not practical?
The proliferation of virtual museums, on line and built inside a museum's collections, stimulates... more The proliferation of virtual museums, on line and built inside a museum's collections, stimulates new challenges for research and public education. This happens through multimodal applications and methods, such as augmented reality, virtual storytelling, virtual reality, computer graphics and haptic applications. This paper discusses a haptic approach with a hybrid solution, able to virtualize tangible objects in a physical space and digital models in a cyber space. The main goal of this approach is to propose alternative ways to experience an object, an artifact, a site or an entire collection. In particular, the hands-on experience is very important, for having tangible feedback from objects and to generate a visual narrative, by touching and combining physical/virtual interaction simultaneously, to customize the visit according to the experience and empathy generated by the unique artifacts. The Tangible Interactive Table for Archeology (TITA) is a open-source prototype of a digital-haptic device, designed for museums and in general for the digital communication of archaeological artifacts, monuments and sites. The intention is to lower the bar of entry for this type of technology in the cultural heritage community, to enable further research and exploration of its affordances, by initiating a community open design project, which provides the digital fabrication files and specifications to produce, test, extend and hybridize such a device, and mechanisms for sharing mutual hardware and software development projects. Such efforts to combine virtual reality and tangible 3d printed artifacts, also entails a rethinking of the production pipeline of digitally reconstructing 3d environments and their constituent elements, where 3d digital models need to be at turns both " lightweight " , visually accurate yet low-poly for real-time, game engine applications, and high-fidelity and well formed as 3d printed artifacts.
This project uses machine learning and computer vision techniques and a novel interactive visuali... more This project uses machine learning and computer vision techniques and a novel interactive visualization tool to provide street-level characterization of urban spaces such as safety and maintenance in urban neighborhoods. This is achieved by collecting and annotating street-view images, extracting objective metrics through computer vision techniques, and using crowdsourcing to statistically model the perception of subjective metrics such as safety and maintenance. For modeling human perception and scaling it up with a predictive algorithm, we evaluate perception predictions across two points in time separated by economic changes in the urban core of Raleigh, North Carolina, in the aftermath of the 2008 Great Recession. We hypothesize specific socioeconomic processes can be substantially reflected in the built environment of cities and, thus, render themselves visible at the street level. This paper describes the process of incorporating subjective visual ratings across two datasets of temporally separated street-view images, an algorithm, and a visualization tool. This work serves as a case study for utilizing AI and visualization techniques in a richer characterization of urban spaces that includes both objective metrics such as income (that operates at a broader scale) and subjective metrics such as perception of individuals (that operates at a narrower scale at specific locations). We outline an interdisciplinary methodology to test this hypothesis in streetscape data from Raleigh, NC, from 2008 to 2020. We describe the results of training algorithms that utilized image features with crowdsourced human perception ratings. We provide a comparison of the results with income data. The analysis and interpretation of this comparison provide insight into the challenges and opportunities for using AI technology in characterizing changes in urban environments. One challenge is the ability of human domain experts to interpret the output of algorithms through manipulation and to integrate these results into their workflow. This is addressed with a novel interface designed for interactive analysis and visualization. We conclude with a discussion of some of the benefits and limitations of integrating AI models in the human expert's decision-making process in the presence of both subjective and objective metrics.
Uploads
Papers by Todd Berreth
Journal Articles by Todd Berreth