Musical Perception

In order to clear up my work in progress with student projects and make space for the latest post-university projects (aka after assessment!) I am cleaning up the categories, but here are the CSM-specific categories just in case of need:
(and don’t forget to scroll down if you want to read through the development of a project from start to finish!)

Projects done around CSM

Year Two at Central Saint Martins (2012-2013):
Unit 5
Visual Language: International Klein Blue
Liberty’s Department Store into Flipbooks, a building translated into a book
Yves Klein by himself, a book translated into a building
Blast/Bless: Postcards from London, Luxembourg and Paris

Unit 6 – The Bigger Picture

Unit 7
Time is of the Essence: Student Organiser
What if Music Could Be Seen: Musical Perception
Pop Up Shop: &London Notebooks

Context / Dissertation
Unit 8-9

Year Three at Central Saint Martins (2013-2014):
Unit 10
Home Town: Luxembourg(ish)
The Value of an Image
A Do Something But Not Anything Manifesto
Public/Personal Space

TSI (Typography Special Interest Group)

Final Year Digital


Events & Exhibitions

Competitions & Collaborations & Live Briefs
Barnard & Westwood
Font Aid IV

An Illustration Workshop
An Internship at the Imprimerie du Marais
Art Auction 2013 Catalogue
D&AD British Council
D&AD Monotype


exporting in progress

exporting in progress

After filming eighteen people (including me) listening to a mixed track with extracts from various songs with different styles, I have put them together into a video so that the reactions can be compared directly one next to the other.
Setting and coordinating the videos with the music was quite difficult as I relied only on the people’s facial expressions and reactions to orientate the timing.
I am surprised by how little people actually react to music, even though there were one, two songs that affected as good as everyone.

I am still waiting for the video to finish exporting and hope this version is going to be the final one.
Here is a screenshot preview from one of the earlier exported video versions.
I was sceptic about whether I should put everyone in the video or isolate a few, but I think this allows for the widest range of reactions and diversity and gives people something new to discover each time. I am going to have to decide if I am including the music as well.



During this project, I have learned what I cannot do in processing unless I am a coding master, how to book the film studio, set up/prep  and film (30-minutes-videos are a bad idea unless you want to end up with 120 gb files to transfer), how to make a music mix, how to edit a video and how to export it (with a lot of patience). So many things I had never done before. Even though my end result may look a bit rough around the edges, I think I managed to get my message through, visualising music through people’s perceptions and reactions, while showing that music is perceived differently by each individual.

As a conclusion from the final crit, the short movies showing people listening to music has a good impact already in its pure form, meaning that I don’t necessarily need to take it further as it stands good by itself.

– leave the room while the listener is being filmed
– create a short mix of different songs not more than two to three minutes long (maybe in loop?)
– make people listen to the music through earphones
– zoom in on the face (no distraction from the clothes)
– keep the same style
– no music needed to accompany the images (makes people focus more on the visual and makes them curious about what song could be played)
– a selection of people, from 4 to 8 people (to compare the different reactions)

The images of the people listening to music can stand independently as a project,  but it still seems not enough as a project to me (which might only be because I am mostly used to doing print design in opposition do digital media… we’ll see how it turns out!).

So up to the film studio to book a space for after the Easter Break!

So lets just do some more research.

Music cognition is an interdisciplinary approach to understanding the mental processes that support musical behaviors, including perception, comprehension, memory, attention, and performance. Originally arising in fields of psychoacoustics and sensation, cognitive theories of how people understand music more recently encompass neuroscience, cognitive science, music theory, music therapy, computer science, psychology, philosophy, and linguistics.


Music cognition was definitively recognized as a discipline in the early 1980s, with the creation of the Society for Music Perception and Cognition, European Society for the Cognitive Sciences of Music, and the journalMusic Perception. The field focuses on how the mind makes sense of music as it is heard. It also deals with the related question of the cognitive processes involved when musicians perform music. Like language, music is a uniquely human capacity that arguably played a central role in the origins of human cognition.[1] The ways in which music can illuminate fundamental issues in cognition have been underexamined, or even dismissed asepiphenomenal. The latter view was famously expressed by noted cognitive scientist Steven Pinker when he referred to music as “auditory cheesecake”.[2] But as cognition in music is increasingly acknowledged as fundamental to our understanding of cognition as a whole, music cognition should be able to contribute both conceptually and methodologically to cognitive science. Topics in the field include:

  • A listener’s perception of grouping structure (motivesphrases, sections, etc.)
  • Rhythm and meter (perception and production)
  • Key inference
  • Expectation (including melodic expectation)
  • Musical similarity
  • Emotional, affective, or arousal response
  • Expressive performance
  • Conceptual processing[3]

Some aspects of cognitive music theory describe how sound is perceived by a listener. While the study of human interpretations of sound is called psychoacoustics, the cognitive aspects of how listeners interpret sounds as musical events is commonly known as music cognition.




Cognitive musicology can be differentiated from the fields of music cognitionmusic psychology and cognitive neuroscience of music by a difference in methodological emphasis. Cognitive musicology uses computer modeling to study music-related knowledge representation and has roots in artificial intelligence and cognitive science. The use of computer models provides an exacting, interactive medium in which to formulate and test theories.[2]




I don’t like being stuck though (but who does?)
As a result, I tried looking at analogue options… but maybe my research isn’t really effective in long sentences.

Screen shot 2013-03-14 at 22.26.08

What I found out, though, is that it is definitely easier to convert film-based material to digital documents rather than the opposite, which sort of makes sense, but it also limits the possibilities for me to go to the roots of the images and edit straight on the film.

Tomorrow I am supposed to present my project for the “What if music could be seen?” brief for the last time. However I am still struggling trying to find a way to translate the movements of the listener into one single image, which is especially due to software restrictions (result of a lack of knowledge of Processing, OpenCV and After Effects as these aren’t the softwares I am used to), but also because it is the last week before Easter Break and there is a lot of stuff going on: helping to organise our Pop-Up Shop for the end of may, a live brief that we presented today, portfolio reviews, a report launch, an Elle/EDO event, a lecture by Seb Lester, and filming filming filming. But these aren’t excuses, so for now I am still trying to find out how I can edit the videos to make them as I want them to be, and which is not 120gb big (but that is my least problem for now).

I have also tried drawing manually over the images while the movies are playing, which was something I wanted to avoid, trying to keep the project as less personal as possible. It wasn’t convincing either, as it looks too literal, or abstract, or mostly too drawn and far away from music or perception.


So I still haven’t found one solution that could work, for now.

But during my research I came across the SMPC (Society for Music Perception and Cognition):

They are looking at perception and how music can influence our moods. It doesn’t have something to do with visuals, but with the impact of music and how it is perceived.


… maybe I should design musical typefaces.