Innovative Art Experience: Merging Sound and Visuals for the Visually Impaired (Full Transcript)

Speaker 1: Tall grass blows in the breeze while a man with a sketchbook sits on a hillside.

Speaker 2: I'm Keith Salmon. I'm a landscape painter. He makes an ink drawing. The paintings I do are based on Scottish mountains, so they're 100% landscape. I found out in 1990 that my sight was deteriorating, and it's gone downhill very quickly, so that I'm now officially registered blind. I just realized I had to adapt and find new ways of working. I started to add sound. Whenever we went walking in the mountains, the recorder went with us, and we would stop, and I would record. And that was the starting process which has led on to the Oregon Project.

Speaker 3: I'm Neil Joshi, and I'm a researcher at Microsoft Research. The Oregon Project is an interactive audiovisual experience. We pose the question, how does one enjoy and consume visual artwork if they have a visual impairment?

Speaker 2: We decided to do this fabulous part of northeast Oregon. The canyonlands just stood out for me. These locations were not only visually very different, but also audibly very different.

Speaker 3: We wanted to mimic the visual experience in sound, so people could have an experience with sound that was also personal. If you move backwards just a little bit... Oh, wow. There's four Kinects, 15 speakers, and three drawings in front of you. As you move through the room, your position is tracked, and you hear a sound recording that's specific to your location in front of the artwork.

Speaker 2: I'd like to think that in the future, we would be able to sort of have something like this, something that gives a sort of just broader audio experience, which would help both visually impaired people, but also fully sighted people. So, yeah, it's very, very exciting. Up to a year ago, I really thought that I might sort of have to start working as a professional artist. And now, with this technology, doing this project, I'm very, very confident that, yeah, even if my sight deteriorates, I will still be working as a professional artist.

Speaker 1: Loved.