Frank Russo helps make music for the deaf.
Working with a team of researchers, the Ryerson psychology professor invented a chair that allows deaf people to feel music through vibrations. He also works with both deaf and hearing musicians to compose music that focuses on vibrations and vision rather than sound.
Prof. Russo, a music cognition expert who also sings and plays guitar, will discuss music without sound at the TEDx Talks in Toronto Thursday. The conference’s tagline is “ideas worth spreading.”
Your talk will be on experiencing music without sound. Tell me more.
I plan to talk about the other modalities – or the other senses – and whether or not we can experience music through these other senses. This is interesting from a scientific perspective. It also has some interesting practical and artistic implications when we’re considering music experienced by the deaf.
Performers do things when they’re performing that convey emotion and these things can be seen. So, for example, when a performer is performing something that is melancholy, their movements are melancholy. By movements, I mean their facial expressions, the way that their body moves, the way that their hands move. There’s really a lot that can be seen that conveys important structural and emotional information about music. There’s [also] a long history of the deaf experiencing music through vibration.
Legend has it that in his later years, a deafened Beethoven cut the legs off his piano to feel the vibrations through the floorboards. How do deaf people experience music and how does this inform your work?
Deaf culture is extremely visual and it also involves the body, more prominently I would say than oral cultures. So their experience of music, maybe not surprisingly, is informed by what they see and what they feel. There’s this long history of feeling music. For example, there’s a famous percussionist, Evelyn Glennie. She’s deaf and she talks about experiencing music through her body. So she’ll perform without shoes so that she can feel the vibration through her body.
You and a team of researchers at Ryerson developed the emoti-chair. What is it and how does it work?
The emoti-chair is a sensory substitution technology that’s designed to take sound and present it to the body as vibration. You can put your hand on a speaker and you can feel the vibration because all sound emanates from some form of vibration. The challenge, though, with touching a speaker or even touching a musical instrument is what we call perceptual masking. Perceptual masking occurs in vibration when the lower frequency vibrations dominate the higher frequency vibrations. So all we feel is the thump, thump, thump. So what we’ve done in the emoti-chair is separate out the frequencies and present them to different parts of the body. We’ll take the high frequencies and we’ll present them to the upper part of the back. We’ll take the lower frequencies in the music signal and we’ll present them to the lower part of your back.
You’ve held a couple dozen concerts for deaf and hard-of-hearing people with the emoti-chair. What are the concerts like?
It’s really evolved. We’ve gone from taking prefabricated music that’s been constructed for hearing ears and have translated it into deaf music. We are now doing something entirely different, where from the conceptualization of the music we’re thinking about this as a vibe track or a piece of music that’s primarily for vibration and vision, not sound. So that opens up all sorts of interesting artistic possibilities for the deaf and hearing community.
It sounds like you’re almost creating a new art form of music without sound.
That’s what we like to think, yeah. And we actually are putting on a series of workshops across the country where we’re exploring this. We did one in Vancouver last June. We’re going to do the next one at the Banff Centre for the Arts next spring. At these workshops, we’re trying to bring together music performers or composers that want to work on this new art form, on developing something that’s music-like but has this reallocation of the sensory priorities so that vibration and vision are in the foreground.
Do people who experience music without sound also experience the emotion that is so much a part of music?
Absolutely. We have been doing some research in the lab along those lines. And yes, there’s a great deal of agreement between the emotion experienced by a deaf individual and a hearing individual.
Your talk will be on experiencing music without sound. Tell me more.
I plan to talk about the other modalities – or the other senses – and whether or not we can experience music through these other senses. This is interesting from a scientific perspective. It also has some interesting practical and artistic implications when we’re considering music experienced by the deaf.
Performers do things when they’re performing that convey emotion and these things can be seen. So, for example, when a performer is performing something that is melancholy, their movements are melancholy. By movements, I mean their facial expressions, the way that their body moves, the way that their hands move. There’s really a lot that can be seen that conveys important structural and emotional information about music. There’s [also] a long history of the deaf experiencing music through vibration.
Legend has it that in his later years, a deafened Beethoven cut the legs off his piano to feel the vibrations through the floorboards. How do deaf people experience music and how does this inform your work?
Deaf culture is extremely visual and it also involves the body, more prominently I would say than oral cultures. So their experience of music, maybe not surprisingly, is informed by what they see and what they feel. There’s this long history of feeling music. For example, there’s a famous percussionist, Evelyn Glennie. She’s deaf and she talks about experiencing music through her body. So she’ll perform without shoes so that she can feel the vibration through her body.
You and a team of researchers at Ryerson developed the emoti-chair. What is it and how does it work?
The emoti-chair is a sensory substitution technology that’s designed to take sound and present it to the body as vibration. You can put your hand on a speaker and you can feel the vibration because all sound emanates from some form of vibration. The challenge, though, with touching a speaker or even touching a musical instrument is what we call perceptual masking. Perceptual masking occurs in vibration when the lower frequency vibrations dominate the higher frequency vibrations. So all we feel is the thump, thump, thump. So what we’ve done in the emoti-chair is separate out the frequencies and present them to different parts of the body. We’ll take the high frequencies and we’ll present them to the upper part of the back. We’ll take the lower frequencies in the music signal and we’ll present them to the lower part of your back.
You’ve held a couple dozen concerts for deaf and hard-of-hearing people with the emoti-chair. What are the concerts like?
It’s really evolved. We’ve gone from taking prefabricated music that’s been constructed for hearing ears and have translated it into deaf music. We are now doing something entirely different, where from the conceptualization of the music we’re thinking about this as a vibe track or a piece of music that’s primarily for vibration and vision, not sound. So that opens up all sorts of interesting artistic possibilities for the deaf and hearing community.
It sounds like you’re almost creating a new art form of music without sound.
That’s what we like to think, yeah. And we actually are putting on a series of workshops across the country where we’re exploring this. We did one in Vancouver last June. We’re going to do the next one at the Banff Centre for the Arts next spring. At these workshops, we’re trying to bring together music performers or composers that want to work on this new art form, on developing something that’s music-like but has this reallocation of the sensory priorities so that vibration and vision are in the foreground.
Do people who experience music without sound also experience the emotion that is so much a part of music?
Absolutely. We have been doing some research in the lab along those lines. And yes, there’s a great deal of agreement between the emotion experienced by a deaf individual and a hearing individual.
No comments:
Post a Comment