Why is music?
Music can move us like nothing else.
But how exactly does it work?
Scientists are seeking the answers.
“Music doesn’t really make any sense, from an evolutionary point of view,” says Jean-Julien Aucouturier, a brain researcher in Paris. “It doesn’t fit with the evolutionary view of why we have emotions.”
Darwin on music
“As neither the enjoyment nor the capacity of producing musical notes are faculties of the least use to man in reference to his daily habits of life, they must be ranked amongst the most mysterious with which he is endowed. They are present, though in a very rude condition, in men of all races, even the most savage; but so different is the taste of the several races, that our music gives no pleasure to savages, and their music is to us in most cases hideous and unmeaning.
“Whether or not the half-human progenitors of man possessed, like the singing gibbons, the capacity of producing, and therefore no doubt of appreciating, musical notes, we know that man possessed these faculties at a very remote period. M. Lartet has described two flutes made out of the bones and horns of the reindeer, found in caves together with flint tools and the remains of extinct animals. The arts of singing and of dancing are also very ancient, and are now practised by all or nearly all the lowest races of man. Poetry, which may be considered as the offspring of song, is likewise so ancient, that many persons have felt astonished that it should have arisen during the earliest ages of which we have any record.”
For some kinds of communication, it’s not difficult to understand the survival value. Screams can frighten a foe or warn others. Language can bond social groups. But why would we evolve to create and appreciate something as elaborate as music?
It’s a long-standing riddle. Since Charles Darwin’s time, the emotional power of music has been a matter of great debate. Darwin himself called the capacity to make or be moved by music one of “the most mysterious” of mankind’s endowments.
These days, big thinkers like Steven Pinker have waded in to argue that music is a nice-to-have by-product of the development of language. It is, in Pinker’s view, no more than an ‘auditory cheesecake’ – something we are glad of but could probably live without.
We react to a lion’s roar for good reason. The lion may be a threat. But the effect that a low cello has on us is not a matter of life and death; whether someone plays a C or a C# is not significant from an existential point of view, but it can still evoke great emotion. Like our emotional palate itself, music is immensely versatile: it can be melancholic, and foreboding, or overwhelmingly euphoric – witness Beatlemania and its successors.
Neys to ocarinas
In our day, unraveling this mystery has become a fast-growing sub-discipline of science – involving thousands of researchers around the world, armed with brain scanners, acoustics equipment, psychological tests and musical instruments from Turkish neys to Mayan ocarinas. The European Research Council has been funding some of this work, in projects suggested by scientists around the world. The ultimate answer to “why?” is still out of reach, but here we sample some of the musical themes they’re working on.
Jean-Julien Aucouturier is a researcher for France’s national research agency, CNRS, at the Institute for Research and Coordination in Acoustics/Music at Paris’ Pompidou Centre. He was trained in computer science, and has held several postdoctoral positions in cognitive neuroscience at RIKEN Brain Science Institute in Tokyo and the Université of Dijon. He is now heading the CREAM neuroscience lab in IRCAM, and uses audio signal processing technologies to understand how sound and music create emotions.
By understanding emotional responses to sound, scientists hope music could become a therapeutic tool
Ever hear a happy piano? Jean-Julien Aucouturier can digitally tweak the sound of a musical instrument – or even your own voice – to trigger emotions. From a research unit in Paris’ Pompidou Centre, he has been exploring how subtle changes to the timbre of sound can spark fear or joy, sometimes without the listener being consciously aware that their emotional buttons are being pressed.
Aucouturier, a computer scientist turned brain researcher, says advances in technology are opening doors to studying the link between music and emotions. The upshot could be musical therapies and new ways to communicate with non-verbal people including those with autism, stroke victims and coma patients. It may even equip companies with audio manipulation tools that would filter the voices of helpline staff to make them feel smilier.
“We want to crack the emotional code of music,’ he says. ‘Sound can become a clinical technology – using algorithms that sculpt sound to activate certain brain areas, just like the way pharmaceutical molecules target certain parts of the body.”
What is auto-tune?
Good news for those whose imperfect pitch prompts them to skip karaoke nights or mime the chorus of Happy Birthday: technology can make you sound like a star.
Auto-tune measures – and alters – the pitch in vocal and instrumental music recording. It works by slightly shifting the pitch to the nearest correct semitone.
This kindly covers up any bum notes, even during live performances. The technology has become a standard feature of professional recording studios, as suggested in this how-to video from a UK firm, Music Radio Creative. For his own research, however, Aucouturier developed a program tuned, not for pitch, but for emotion.
Is this the dawn of the music-aceuticals era? Access to large datasets on facial expressions, musical extracts, and emotions – along with digital tools for manipulating sound – have attracted engineers, computer scientists and data analysts to a field once dominated by musicologists and the occasional psychologist. This isn’t just shopping mall muzack; the goal is precise understanding of sound and emotion. So far, this has led researchers to categorise music as triggering “basic emotions” such as anger or fear; to identify the emotion-controlling amygdala deep inside the brain as involved in the sensation; and to observe how musical perception works across cultures – or doesn’t work at all, in people with brain injuries.
Aucouturier’s team, which leads the ERC-funded CREAM project, has found a way to look at what’s happening subconsciously when we process sound – by studying the tiny muscle twitches associated with smiling.
“We sat people at a computer and attached electrodes to their face,” he explains. “This allows us to measure activity of the muscles we use to smile.”
Next they played voices that had been manipulated to make them sound happier, sadder or more fearful by adjusting the timbre – a technique described as an auto-tune for emotion.
“We built a tool that changes your voice to make it smilier than it actually is,” Aucouturier says.
Finally, they watched the electrodes. They picked up more contractions of ‘smile’ muscles when people were played voices tweaked to sound happier. The effect was very subtle. The researchers couldn’t always see the smiles, and the subjects couldn’t always consciously tell a ‘happy’ voice from an unmanipulated one. But the electrodes didn’t lie: smiling, they were.
“Sometimes their muscles were more accurate than their conscious judgements,” he explains. “Even when they were able to tell which voice was happier, their facial muscles knew it before they did.”
Smiling comes with all kinds of physiological and psychological benefits. Even a forced smile – or a smile that forms when you hold a pencil between your teeth – can improve your mood. Using sound to fire our smile muscles, without telling a joke or a heart-warming story, could trigger positive emotions without us even realising it.
But what have smiles got to do with music?
DAVID and his electronic lyre
As part of their research, Aucouturier and his colleagues developed some software to study how voices convey emotion. They call it DAVID, partly as an acronym for Da Amazing Voice Inflection Device, but also as a tribute to Talking Heads frontman David Byrne, who was one of its first users.
The software subtly tweaks the sound properties of a spoken sentence to make it a bit more sad, happy, or scared. Often the impact is subconscious: The hearers may not detect much difference, but measurement of their facial muscles in the lab tell a different story.
Here are a few samples, can you tell the emotions conveyed?
Clip 1 - Happier
Clip 2 - Sadder
Listen to someone speaking on the phone and you can often tell whether they are smiling. The key to this is timbre: the shape of the lips alters the quality of the sound without changing the pitch or amplitude.
“It works in a similar way with a guitar,” says Aucouturier. “If the strings of two guitars are the same length they play the same notes, but a larger-bodied guitar has a different timbre to a smaller guitar, and variation in shape can also have an effect.”
Guitars, flutes, pianos – his software can be applied to any sound. “We are now testing whether people can pick up a happy signal from manipulated musical sounds. The question is do we get a smiley response to a smiley piano?”
One of the potential applications of the research is with coma patients. It can be difficult to know whether someone in an apparent vegetative state is processing information and sounds in their environment. But what if those sounds could be manipulated to make them stronger and more emotionally resonant? “If you could use sound to trigger fear, for example, you could test whether they have an emotional response,” explains Aucouturier.
People with autistic spectrum disorders could also benefit. The French team plans to work with doctors to see whether sounds with added ‘smiles’ or enhanced trustworthiness could be used to communicate with people who have difficulty maintaining eye contact or who have problems reading facial expressions. You don’t need to see a smile to feel it.
Like many technologies, some of the more sci-fi applications are borderline dystopian. The power to manipulate emotions through altered sounds is the power to manipulate people – often without their knowledge.
How do you play ‘domineering’?
Can you really tell what someone is thinking when they’re playing an instrument? French composer Eric Satie often wrote ‘thought’ instructions into his scores: “Obey…Settle down…Don’t worry…Tired…” are some of the enigmatic markings on his piano Pièces Froides of 1897.
Aucouturier’s lab decided to test that, by asking some professional musicians to improvise with an emotion in mind: domineering, insolent, disdainful, conciliatory or caring. Then others listened – some musicians, some not.
Surprisingly, it worked: more often than randomly possible, the hearers picked the right emotion. Not so surprisingly, the musician-hearers were better at it than the non-musicians. You can read the details in this 2017 paper from the journal Cognition.
In one experiment, CREAM researchers asked people to tell a story about being late for work on their first day at a new job. Participants were wearing earphones that allowed them to hear their own voice as they spoke. But, unknown to them, the sound had been manipulated to make it seem emotionally positive or negative. This influenced how people felt about the story they were telling, and even affected their word choices. Those who were hearing their own voice layered with a subtle worried timbre began to feel more stressed and view the episode in a more catastrophic light. People who heard their voices laced with happiness became more philosophical about the idea of being late for work.
“They didn’t know that they were hearing their own voice with effects that we had added in real-time,” he says. “The more they hear the happy-tuned version of their voice, the happier they felt. They even began to change the words they used depending on what they were hearing.”
Ongoing experiments on how earbuds could modify incoming sound may pave the way for devices that filter the world to make it less – or more – stressful. There are other possibilities too: the entertainment industry could offer emotionally souped-up music; workplaces could pipe in sounds that boost productivity or wellbeing; politicians could speak through a trust-laden microphone; and, when you call your mobile phone operator to complain, the customer service operator could auto-tune their voice to something warmer and more caring. They may be playing you like a fiddle.
“This is not even science fiction,” says Aucouturier. “We have some of these technologies now. We want to make socially useful tools but also raise the flag to the public about what can be done with sound technologies. It’s best to know what’s possible.”
Michael Ellison is a composer and reader in the School of Arts at the University of Bristol. He combines contemporary and traditional influences into a personal musical idiom. His first opera, Mevlâna-Say I am You (Rotterdam Operadagen and Istanbul Music Festivals, 2012) integrated Turkish traditional instruments into contemporary music—a direction his second opera, Deniz Küstü (Istanbul Music Festival, 2016, Jones/Tanbay/NOHlab) extends. Ellison has been commissioned by BBC Symphony Orchestra, Acht Brücken Festival, Radio France, Grenoble Festival, New York Youth Symphony, Siemens Foundation, Nova Chamber Music Series, amongst others. He is also co-director of Istanbul’s Hezarfen Ensemble.
Researchers in Britain and Turkey try to find a common musical language
Michael Ellison knows that globalisation is a big deal, causing cultures to collide. Result: problems, but also possibilities. Can music bridge the gap?
Practice makes perfect
What happens in the rehearsal room when East meets West.
Ellison is a soft-spoken American composer, now at the University of Bristol, leading an ERC project that morphs music from two disparate musical worlds: Turkish makam and Western classical and contemporary music. With Istanbul Technical University, he and his research team are working with Turkish instrumentalists, developing new notation systems and managing workshops to combine the two.
It can be difficult. For starters, he says, the Turkish tradition requires good improvisers: “a lot of what they do is by ear,” he says. Western classical musicians generally want it all written down. When they come together, “musicians from both sides get out of their comfort zone. It’s a slow process.”
At one workshop, the Western musicians tried to make a piece of makam music sound more ‘together’, rather than improvised – but the Turkish musicians wouldn’t buy it. Ellison relates:
“One of the Western musicians finally gave the Berlin Philharmonic as an example of the pinnacle of playing exactly together to an amazing degree.
“Our ney player, Bülent Özbek, then asked, ‘So the most interesting thing about the concert is that they all play exactly together?’”
What is makam?
“A makam (maqam in Arabic) is a series of trichords, tetrachords, and/or pentachords that make up a ‘scale’, with a particular tonic, particular dominant notes, and a melodic progression (ascending, descending, or a combination of the two). The scale of a makam is not a scale in the Western sense, as it may not repeat above and below the primary octave, and notes may shift at certain points in the progression of the makam.
“There are more than 200 makams, more than 50 of which that are relatively well known and commonly practiced today. In Turkish music a greater distinction is often made regarding the melodic direction or path of a makam than in Arabic music.”
‘Yes, they are so amazing,’ was the answer.
“To which Özbek replied, ‘Well, I wouldn't be interested in going to that concert.’”
Ellison is very nearly fluent in both types of music. Classical Turkish music is called makam, and dates to at least the 15th century and the court of Ottoman Sultan Murad II. The word denotes a place – referring to the way each of its musical modes has a starting place and final note (much as Western modes, from ancient Greece onward, have tones around which the melody moves.) The instruments are delicate and varied – from the lute-like oud to the ney, a flute made from a reed and blown from one end.
Ellison is hoping his research into East-West musical pairing will result in the birth of a new strand of contemporary music and opera. This has so far resulted in one chamber opera as well as new technologies and strategies to bring the musicians of different traditions closer together, such as a souped-up Western keyboard that can play Turkish sounds. He is studying the voice in detail, with spectrographs to analyse a singer’s tone colour and (coming soon) an ‘electroglottograph’ to measure the vibrating vocal folds. These efforts, together with videos of traditional singers, may help teach certain types of Anatolian singing. Just how do Yörük and Kurdish singers use their throats when they perform?
“The voice is the primary instrument in the music of many cultures,” says Ellison. “In Turkish music, for example, instrumentalists will listen to the great singers in the tradition to learn how to play.”
Inevitably there are also politics involved. Ellison says that when he first got to Turkey some 20 years ago he was surprised by “the amount of political information an average person tended to ascribe based on whether one was ‘Westward-leaning’ musically speaking or …’Eastern leaning’. I've had to explain to people a number of times that my interest in Ottoman or makam music doesn’t have anything to do with a political agenda; it simply had to do with the music.”
His ambition is to get more Western composers and musicians thinking Turkish; a book is in the works, for instance. But he has already written an opera combining the musical traditions. It is called Deniz Küstü: The Sea-Crossed Fisherman, based on Turkish writer and human rights activist Yashar Kemal’s 1978 novel by the same name. The 70-minute piece integrates Turkish and Western instruments, and includes contemporary choreography and video. The music varies from harsh to playful, from dissonant to dreamlike. A reviewer in Opera magazine described it: “Instruments break out of their traditional roles and mix to form novel sonorities, so that the sea music shimmers.”
There’s a second, large music theatre work in the pipeline, based on another novel by the same author, Legend of 1000 Bulls, about the end of nomadism of the Yörük tribes in Eastern Turkey.
“It’s a sad, but beautiful novel about people forced to settle in another place,” says Ellison. “There are interesting parallels with migration today.”