Multisensory Ethnography
the aural in relation to the visual
" The senses are made, not given. " (Howes 2019, 17) . Whether it is sight, hearing, smell, taste or touch, as well as 'haptic visuality' or 'sixth sense', the human senses are closely linked and cannot be separated. The eyes and ears can deceive each other, the hands want to see what the eyes cannot, the gaze tries to speak ...... each sense is actively exchanging power and strength with the other senses (Murail 2017).
This ethnographic study on sight and hearing allowed me to deeply understand the interchangeability and unity between multiple senses, as well as the active interaction and exploration between the senses and the environment or between the senses and the brain (Ingold and Howes 2011; Howes 2019).
Exploring the possibility of separating the presence of the senses with the help of external devices is one of my interests. I tried to raise doubts in my participants' minds regarding the authenticity of visual and auditory experiences by showing them a film where the two were in conflict (as follows). Before each interview, I described in detail how "normal" and "lovely" this voice's owner was when he appeared in the image to make my participants completely aware of the mismatch between the image presented in the video and the voice played and to induce in their doubts about the authenticity of the vision. However, the results of this experiment not only questioned my preconceptions that vision and hearing could be separate and not credible but also pushed me to start thinking about the importance of multisensory ethnography.
I inquired about each participant's impressions of the video. I asked them if they still believed what they had seen or heard and if they had reached a new judgment about the character in the film, as opposed to the personal image I had previously described. Contrary to my assumptions, my participants did not see this as reflecting a separation between the visual and the auditory. On the contrary, they were excited because it allowed them to discover another side of the video's protagonist that they had never seen before. They argue that this does not reflect the disintegration of the visual and aural senses but rather highlights the complementary nature of the information provided by multiple senses.
This result made me question the meaning of the topic "whether what the camera captures, what the eye sees, or what the ear hears is real", which I had been exploring throughout the project. As my participants said, the authenticity of a single sensory experience is inherently one-sided and incomplete.
I realized that I had made a mistake in my methodology. I tried to break away from anthropological theories of the senses to explore new sensory anthropology, but I inadvertently fell into the abyss of "dualism". I had dropped into the Western ocular centrism of prioritizing the visual memory captured by the eye as a sense for acquiring knowledge, truth, and experience (Totaro 2002). It made me realize why I am obsessed with exploring visual reality - subconsciously, I hegemonized the sensory dominance of the visual and tried to challenge it with the auditory.
Memories are embedded in all the senses (Totaro 2002). Zhang (2017) proves that the camera can record a person's tactile sensory sensations and display them in visual form in an ethnography studying the participatory sensory experiences of filmmakers. Meanwhile, filming integrates sensory experiences into digital imagery, bringing the characters in the movie closer to the "sensory distance" between them and the real-world audience watching the movie. In other words, the audience watching the film could sense the taste, touch, and smell experience of the tea tasters in the film on a visual level.
In my ethnography, my participants and I have discovered an interesting phenomenon. We have found that when filmmakers shoot with a camera, they are not only using the senses of sight, sound, or touch, but their brains are also making up some relevant scenes on their own. Interestingly, this unconscious scene rarely arises in the everyday observation of the world with the eyes. For example, when one of my participants was filming a puppy, he became aware that his brain would pre-present images that did not exist. He would anticipate what the dog was going to do, which direction it would run in, etc. However, this participant said that if he saw the same dog in everyday life, his brain would not have any additional images.
Let's return to the original topic: is what you see and what you hear true? Besides, is what the camera captures believable? Clearly, multisensory ethnography has taught us that this question cannot be answered and does not need to be answered. The eyes can trick your brain, while the lens can change the image of the real world by zooming in and adding filters.
They are both real and unreal simultaneously. The most important thing is how your heart choose.