CENTER FOR

IMMERSIVE
EXPERIENCES
6.53

Dr. Patrick Dudas

Institute For Computational and Data Sciences
Associate Director Of The Center For Immersive Experiences
Lead Data Visualization Research and Development Engineer
 WHO AM I?

• Patrick M. Dudas, PhD

• pronouns = [he,him,his]
• Institute for Computational and Data Sciences
• Associate Director of the Center for Immersive
Experiences
• Lead Data Visualization Research and
Development Engineer
• Passions: Data and Storytelling Fluency

2
VISUALIZATIONS

Art

Data Code

UI/UX
3
CENTER FOR IMMERSIVE EXPERIENCES
(CIE)

OSVPR

ICDS

CIE

4
 ICDS INTERDISCIPLINARY INITIATIVES

Predictive Infrastructure Industry Quantum Data Privacy,

Science Applications Computing Policy and Law

AI/Machine Advanced Visualization Resilience & Social Diversity in International

Learning & Immersive Experience Justice (Joint) Computational & Data Partnerships
Science
5
CENTER FOR IMMERSIVE EXPERIENCES
CIE PEOPLES

Jessica Menold - Director

Keerthana
Clayton Colson -Lead
Alex Fatemi - Lead Bart Masters - Lead Govindarazan - Ph.D
Mixed Reality
3D Modeler Programmer for XR Student, Department
Developer
of Architecture

7
SO, WHAT IS THE
TECHNOLOGY?

8
IMMERSIVE TECHNOLOGIES

• Virtual
• Augmented
• Mixed Reality

9
 360 VIDEO

10
Immersive Experiences at Penn State - https://imex.psu.edu/
 JOINT EXPERIENCES
• Multi-user experiences
• Linking different VR platforms
• Remote & local

• Virtual version of actual field trip

• Traditional classroom / lab setup for
higher learning motivation (?)

11
Designing Low-Cost Joint VR Experiences – Wallgrün et al.
 AUGMENTED REALITY

12
Credit: CNET - https://www.cnet.com/ - https://www.youtube.com/watch?v=gZhQCVSvq5E
 MIXED REALITY

13
Credit: Microsoft Hololens - https://www.microsoft.com/en-us/hololens - https://www.youtube.com/watch?v=d3YT8j0yYl0
VR TECHNOLOGIES

14
 It is all about
“presence”. A
peculiar sense of
“being there”
unique to virtual
reality

Why 6-
DOF
Virtual
Reality?
VR is far more With VR, the gap
psychologically between the real
powerful than and mediated
any medium ever experience is
invented very small

15
Credit: Thank you to Pejman Sajjadi for creating the slides
 HOW VR WORKS?
The brain tends to treat VR experiences as real
You can break the rules!
You can do dangerous tasks
You can change your body (e.g., 3 arms), age, etc.
You can distract the brain to not feel pain
Why
You can go back and forth in time Virtual
You can save tremendous amounts of costs Reality?
By breaking the rules, you can teach concepts
impossible in other mediums

16
KEY AFFORDANCES OF VR

Interactivity Spatial Presence Social Presence Embodiment

17
INTERACTIVITY

18
SPATIAL PRESENCE

19
SOCIAL PRESENCE

20
AVATARS IN VR

Considerations Interactions
Realism Branching conversation tree
Uncanny Valley “Man behind the curtain”
Non-verbal Cues AI

21
 EMBODIMENT

Embodied Cognition (EC) Theory:

Cognition is highly dependent on the characteristics of the physical (virtual) body and its interactions
22
with the physical (virtual) world
The “Machine to be another”
uses embodied virtual reality
to allow users to “body swap”
Credit: BeAnotherLab
 EMPATHY

Emergent research
suggests that xR supports Emotional
the development of Empathy
emotional empathy, but
not cognitive empathy
Cognitive
Empathy

Credit: Thank you to Jessica Menold for creating the slides

 FIDELITY

Higher fidelity mixed

reality experiences Higher
increases empathy in Fidelity
participants significantly
more than lower fidelity
experiences Lower
Fidelity
BEYOND BUILDING EMPATHY,
xR TECHNOLOGIES CAN

HELP
DESIGNERS
PROTOTYPE
COMPLE PRODUCTS,
X
SYSTEMS, OR SERVICES
xR technologies can enable more effective virtual
collaborations, enhancing the abilities of remote teams
One significant barrier with
immersive experiences is that
it eliminates many of our
senses during the simulation
These technologies will
continue to evolve, with tactile
and olfactory
features coming online in the
coming years
WHAT IS CIE DOING IN THIS SPACE?
30
CIE’s overarching goal is to catalyze and translate fundamental
research through xR technologies

31
 We are actively supporting
We are actively supporting research exploring the
utility of xR technologies to support surgical training
research exploring the utility
of xR technologies to support
surgical training

In the photo, Dr. Scarlett Miller tests a

mixed-reality simulation of laparoscopic
surgery
32
Credit: Thank you to Jessica Menold for creating the slides
One of the core areas CIE is focusing on is the use of digital twins in
medical fields

Telehealth for Clinical decision Legal and ethical

underserved regions making implications

33
PROJECT TYPES

Human-Subject Research Training Support Quantitative Science​ Digital Twins

HHD VR Lab 360 video & images Centennial Earthquake Machine Learning​
DEI ICDS Grant HoloLens Labs Catalog​ IoT
Implicit Bias in Hiring​ Sensors

34
 ACADEMIC PROJECTS/SUPPORT

Joint, Multi-
user
Marcellus Center for
Immersive
Outreach and Research
Virtual Field
Trip

Educational
Music Education
Experience

Bee
Entomology
Anatomy

35
https://cie.psu.edu/BeeModel/ - Erica Krieger
 RESEARCH PROJECTS
Immersive
Paradigms
for Second Psychology
Language
Learning

Visualizing
Department of
Subsurface
Geoscience
Data

Improving
Health
Health and Human
Through
Development
Immersive
VR

Erica Smithwick - Visualizing Forest Futures Under Climate Uncertainty: Integrating Indigenous 36
Knowledge into Decision-Support Tools for Collaborative Decision Making
CIE can help your faculty use advanced visualization and immersive
tech to explore convergent research areas

Providing xR Collaborating on grant Funding preliminary

development service submissions work through seed
grants
THANK YOU!
Next Workshop: Introduction to
Unity Game Engine
Fri Dec 1, 2023, 2 – 3PM