Music sounds. The body begins to move. Emotion flashes on the performer's face.
It seems that all three languages simultaneously tell one story. But separate the movement from the melody or hide the dancer's face—and the emotional meaning changes beyond recognition. Sometimes music amplifies the gesture, making it more convincing. Sometimes the body argues with the sound, creating productive tension. And often their combination unexpectedly dampens the experience, as if the signals interfere with each other instead of adding up.
New research shows a non-obvious result: the more expressive signals we receive, the stronger the emotion is not always. The effect depends on the emotion itself—for each feeling there is an optimal combination of perceptual channels.
Multimodal communication: emotion in the movement of sound and body
September 4, 2026 in the peer-reviewed journal Music Perception an article was published Perceived and Felt Emotion in Music and Dance: Multimodal Communication Is Emotion-Specific —a study on how perceived and felt emotions in music and dance depend on the mode of transmission.
The work was conducted by four scholars: Marco Susino from the Juilliard School in New York, Manuel Anglada-Tort from Goldsmiths College, University of London, and Suvi Saarikallio and Birgitta Burger from the University of Jyväskylä in Finland. The team combined expertise in musicology, psychology of perception, and the study of the relationship between body and emotion.
The methodology was carefully designed. The researchers used a recording of a live music-dance performance and created four different versions:
- only the performer's face—body movements were hidden by a black background;
- only gestures and movements—the face was covered;
- only music without video;
- full performance: music, facial expression, and dance simultaneously.
165 participants were randomly assigned to one of four conditions. Each watched or listened to one of the versions and rated six basic emotions on a hundred-point scale, answering two fundamentally different questions:
- How convincingly does the performer express this emotion? (perceived emotion)
- How strongly I myself feel this emotion? (felt emotion)
This distinction was key. The researchers differentiated between what we recognize and understand intellectually, and what actually arises within us as a direct experience. Such differentiation is rare in research on emotions in art.
Understanding a feeling is not the same as experiencing it
The result was unexpected for many. On average, participants rated the expressiveness and persuasiveness of the performance about seven points higherthan the strength of their own emotional experience.
Imagine this: a person sees absolutely clearly—this is fear, anger, sadness. The image is clear, the signals unambiguous. But the emotion does not capture them with the same intensity they observe. We are able to read the artist's emotional message without becoming its complete prisoners. Intellectual recognition of an emotion and its internal experience are not the same thing.
Between recognizing an emotion and experiencing it, there remains a gap. It is in this space that the work, the performer, and the viewer's personal inner world—with all its biases, memories, and states—meet. Emotion does not arise on command, even if the signal is crystal clear.
More is not always stronger: the paradox of multimodality
The most striking result of the study: the combination of music, facial expression, and movement did not consistently produce a more powerful effectthan each of these elements individually. Moreover, increasing the number of information channels sometimes weakened the experience.
The result depended critically on the specific emotion. For some feelings, a particular mode of expression proved especially important—for example, for fear, sound was key, while facial expression had minimal influence. For other emotions, combining multiple signals did not enhance the impact but neutralized it. Particularly notable: in the case of anger, the full version of the performance (music + face + dance) even reduced the intensity of the experience in viewers.
Why? The researchers offer an explanation: additional signals can make the image too complex. Music conveys one thing—for example, aggression through dissonance and volume. Facial expression adds a nuance—perhaps pain or despair. And movement offers its own interpretation—maybe defense or a display of strength. Instead of simple addition, a multiple interaction arises, where signals align, create tension, or come into contradiction. This complexity can blur the initial emotional impulse.
However, it is important not to attribute more to the work than it shows. The scientists studied participants' subjective ratings but did not directly measure their brain activity and physiological reactions. The study does not reveal what neural mechanisms underlie the integration of music, facial expression, and movement. Instead, it demonstrates a practical result: how viewers perceive and emotionally respond to multimodal stimuli can be detected and measured precisely through their subjective judgments.
Dance sounds, music moves
Music already contains movement within itself. It accelerates and freezes, rises and falls, accumulates tension and resolves it in a climax. Even without a physical dancer, we can sense in the sound approach and retreat, heaviness and lightness, ascent and fall. Rhythm is the beating heart of music.
Dance makes this invisible and immaterial dynamics visible, embodied. The body turns rhythm into a step, a musical pause into a freezing gesture, and a melodic rise into a flight of arms. Each movement is a translation from one art language to another.
But the dancer does not merely obediently illustrate the composition. He has the ability to redefine the emotional meaning embedded in the music. A soft, flowing movement can soften tense, anxious music, giving it lyricism. And a sharp, choppy gesture can uncover hidden anxiety in a seemingly calm melody, reinterpreting it upside down. Dance is not a commentary but a negotiation with music.
Thus a third work of art is born: no longer only the written score and not only the musical sound, not only the choreography—but the space of their living encounter, a dialogue that takes place right before us.
Emotion lives in the space between us
This work helps to understand one of the paradoxes of art: why the same performance evokes completely different states in different viewers. Why one viewer is swept by a wave of feeling, while another is left almost indifferent?
The reason is not randomness. We perceive not just individual elements mechanically piled into a heap. We notice relationships and connections between them: whether the smile matches the character of the melody, whether the movement corresponds to the rhythmic pulse, whether the body language supports the feeling expressed on the face. We constantly check consistency.
And then the viewer himself actively enters this composition—with his own memory, his culture and experience, with his current emotional state and anticipations. His presence is not passive. He is not a blank slate onto which a ready-made emotion is applied. He is a co-participant.
Therefore, emotion does not exist exclusively within the musical work or in the dancer's body. It is born in the living encounter of sound, bodily gesture, and the perceiving person. This is a three-part act, where each side is equally necessary.
Whom will the heart believe?
The new study does not offer a definitive winner in the long-standing debate: what is more important—music, face, or dance? On the contrary, it convincingly shows that there is no universal main channel.
For one emotion, the music itself sounds more convincing—its subtle micromodulations, its architecture. For another, movement is fundamentally more important—dynamics, energy, the rhythm of the body. A third is revealed in facial microexpressions—in the barely perceptible contraction of the eyelid muscles or the curve of the lips. And the most complex feelings require not agreement of all three channels, but a barely noticeable contradiction between them, which creates depth.
Perhaps this is why art touches us despite our skepticism and defenses. It does not transmit emotion as a ready-made object from hand to hand. It does not impose feeling. Instead, art invites us to recognize emotion, to decipher it in all its complexity—and then to decide for ourselves how deeply to allow it into our heart.
Music sounds. The body responds to its call. The face tells its own story, which may coincide with or differ from the first two.
The heart does not choose whom among the three to believe. Instead, it connects sound, movement, and gaze—and in this intricate interweaving recognizes a single, holistic feeling that is already greater than the sum of its parts.
