Sensory Substitution
- 2. Neuroplasticity Adaptive capacities of the central nervous system Its ability to modify its own structural organization and functioning
- 3. Sensory Substitution Using input from one sensory modality to acquire information for another Eg - Persons who become blind do not lose the capacity to see. Usually, they lose the peripheral sensory system (the retina), but retain central visual mechanisms The input from a sensory substitution system can reach many brain structures Offers an opportunity to restore function. In such restoration of lost senses, information from an artificial receptor is coupled to the brain via a human– machine interface (HMI), replacing information usually carried to the brain from an intact sense organ. Sensory substitution is thus only possible because of brain plasticity.
- 5. Auditory-vision sensory substitution: seeing via the ears Meijer (1992) - The vOICe system Webcam, Headphones, Computing Device software subsamples the image into a an array of pixels The image is heard column by column, panning from left to right over 1 s Bright pixels sounding loud (i.e. brightness to loudness mapping) Pixels high in the view being higher in pitch A white ’/’ would be heard as a single tone rising in pitch A white ‘X’ would be heard as two simultaneous sound streams, one ascending in pitch and one descending
- 6. Tactile-vision substitution: seeing via skin receptors Tactile-Visual sensory substitution (TVSS) explorations of Bach-y-Rita and colleagues The original device used an array of tactile pins typically mounted on participants’ backs A later version was based on electrical stimulation of the tongue Participants used a hand-held or head- mounted camera to explore the scene. The visual view of the camera was then converted into an array of pixels, and the brightness levels of the different pixels were mapped to corresponding levels of tactile stimulation by the tactors in a tactile array. Although the tactile array was normally placed on the back, within 5–15 h of training the participants reported an externalization of their sensations in front of them, i.e. in front of the camera.
- 8. Tactile-vestibular sensory substitution: balancing via skin receptors Rehabilitation of subjects with bilateral vestibular loss (BVL) extremely difficult. System includes the following: A miniature accelerometer was mounted on a low-mass plastic hard hat. Anterior-posterior and medial-lateral angular displacement data were fed to a previously developed tongue display unit (TDU) that generates a patterned stimulus on a electrotactile array held against the superior, anterior surface of the tongue. Subjects readily perceived both position and motion of a small ‘target’ stimulus on the tongue display, and interpreted this information to make corrective postural adjustments, causing the target stimulus to become centred.
- 10. Evaluation Strengths Demonstrated the capacity of the brain to adapt to information relayed from an artificial receptor via an auditory or tactile HMI With training and with motor control of the input by the subject, percepts are accurately identified and spatially located Limitation Precise neural mechanisms have not been identified
- 11. Implications Inexpensive implementations of the technology to make it accessible to a wide range of patients suffering sensory loss It should be possible to use the same technology to expand human sensibilities, for example, enabling the use of night vision apparatus without interfering with normal vision The technology enables a whole range of noninvasive low-risk experiments with human subjects to gain a deeper understanding of brain plasticity and cognitive processes Paul Bach-y-Rita