Available online at:
https://cc.arcabc.ca/islandora/object/cc%3Apsycjournal

J Camosun Psyc Res. (2019). Vol. 1, pp. 31-36.
Submitted April 2019; accepted 22 May 2019.

The effect of incongruent audio and visual stimuli on
memory of written words.
Author: Nu Minh An Ton*
Supervising Instructor: Grace Chan, Psyc 110 (“Experimental Psychology”)
Department of Psychology, Camosun College, 3100 Foul Bay Road, Victoria, BC, Canada V8P 5J2
*Corresponding author email: glawcc.nit@gmail.com

ABSTRACT
This study investigated the effect of incongruent audiovisual stimuli on people’s memory of
written words. Twenty adults participating in the experiment were asked to memorize written
words while discriminating the audio stimuli. In one condition, the video showed had congruent
audiovisual stimuli while in the other condition, the video had incongruent audiovisual stimuli.
The hypothesis was that people with video that had the same audio and visual would remember
more words than the people with video that had conflicted stimuli. The results paired with the ttest indicated that the number of words recalled by the participants with matched audiovisual
video is significantly higher than the number of words recalled by the participant with video that
had different audio and visual. This supported the hypothesis showing that the auditory has an
effect on word recalling. This finding can lead to future researches on improving memory by
combining the audio and visual stimuli.
1. Introduction
People have selective attention, which
allow them to focus only on certain
stimulus, and what we fail to notice, we fail
to encode it into memory (Myers & DeWall,
2015). The two-track mind help us process
many information simultaneously on
different levels of consciousness. However,
since both visual and auditory stimuli are
encoded through effortful processing,
selective attention keeps them from
happening at the same time. Moreover,
echoic memory also lasts longer than iconic
memory. As the results, many would find
concentrating and memorizing while
distracted by irrelevant sounds difficult.
Many studies have assessed the effect of
incongruent audiovisual stimuli on

perception. People would show both
auditory input discrimination impairment
and slowed visual process while performing
discrimination tasks regardless of being
instructed to only pay attention to the visual
stimuli or not (Dunifon, Rivera, &
Robinson, 2016). Participants were to
determine if the paired stimuli were the
same after presented with either the
unimodal condition or one of the crossmodal condition of which either stimuli
could be different. The ANOVA test showed
that visual discrimination was slower in the
cross-modal condition and that the
participants’ responses were slowed down
without the conflict between changes of the
two stimuli. Experiment 3 presenting an eye
tracker indicated similar results. Eye
movement pattern and EGG were also used
31

Ton - J Camosun Psyc Res. (2019). Vol. 1, pp. 31-36.

for in dept researches indicating that people
memorize better with matching sound
(Marandi & Sabzpoushan, 2014) and
respond with high accuracy faster in
discrmination tasks with congruent
audiovisual stimuli than with incongruent
audiovisual stimuli (Harrison, Witheridge,
Makin, Wueger, Pegna, & Meyer, 2015).
The current experiment investigated the
effect of mismatched audiovisual stimuli on
written word memorization. Participants
were asked to recall a list of words after
seeing a video with either incongruent
auditory or congruent one. Building around
previous researches as well as the selective
attention theory, it was hypothesized that
participants would able to recall more words
when showed the video with same audio and
written words than the video with different
audio and visual.
2. Methods
2.1 Participants
Twenty volunteers, recruited around
View Royal neighborhood, participated in
the experiment. These included 12 females
and 8 males ranging from 18 to 39 years of
ages with the average of 23.9. They were
later randomly assigned into two groups of
ten: Congruent Audiovisual Stimuli Group
and Incongruent Audiovisual Stimuli Group.
2.2 Materials
Two videos were used in this experiment.
Each video contained 20 words, each word
was showed for 2 seconds. All words were
written in 180-point black and white Time
New Roman font. There were two sets of
words. One set had the audio that fit with the
visual and the other one was mismatched
(see Appendix A for the lists of words used
in this experiment). Other material included

a laptop, a set of headphones, a stopwatch
and a consent form. The one-page consent
form providing a brief information about the
study as well as the participants’ right was
printed on a 8.5 x 11 inches white paper in
12-point Time New Roman font (see
Appendix B).
2.3 Procedure
Participants were approached and asked
if they would like to take part in an
experiment. After being briefly informed
about the study purpose and agreeing to
participate, the participants were asked to
read and sign the consent form. Participants
were then seated near window with suitable
brightness to have a good look at the screen.
The researcher then randomly gave the
participants either the video with matched
audio or the video that had the audio that are
different from the written words and asked
the participants to put on their headphone.
The instructions were to watch the video and
verbally recall the 20 words that were
written on the screen in 90 seconds while
ignoring the audio. The participants were
also told that if more than 20 words were
spoken, only the first 20 would be recorded
unless the participant explicitly fixed during
the time limit. The number of correctly
recalled words was recorded by the
researcher. Finally, the researcher debriefed
the experiment and thank the participants for
volunteering in the study.
3. Results
The level of significance set in this
experiment was .05. The mean of the
number of written words recalled for the
Congruent Audiovisual Stimuli Group was
12.1 (SD = 4.18). The mean of the number
of written words recalled for the Incongruent
Audiovisual Stimuli Group was 7.9 (SD =
32

Ton - J Camosun Psyc Res. (2019). Vol. 1, pp. 31-36.

4.

3.7). These data were analyzed using an
independent sample t-test and the results
were significant, t(18) = +2.38, p = .029,
suggesting that people remember more
written words when the audio and the words
showed on the screen are the same than
when the audio and visual are different.

Discussion
The hypothesis that incongruent auditory
would impair people’s memory of written
words was supported by this experiment.
These findings are in line with the
research conducted by Dunifon, Rivera, and
Robinson (2016) showing that people have
33

Ton - J Camosun Psyc Res. (2019). Vol. 1, pp. 31-36.

difficulty in discriminating visual stimuli
when the audio stimuli are different. The
results are also consistent with those of
Marandi and Sabzpoushan’s study (2014)
which revealed that people’s memory will
benefit from matching sound and visual.
These findings have important
application in educational setting. Students
could benefit greatly if immerge themselves
in environments with matching sound to
what they study rather than those with
distracting sound. Teachers may want to
combine lectures with writing lessons on
boards or slides to increase efficiency.
Discussions could create an audiovisual
stimulating environment which help students
focus as well as memorize lesson faster.
These findings also imply that people can’t
fully focus while doing multitasks that
require both visual and audio perception.
In the current study, participants were
asked to verbally recall the written words. It
is to be further investigated if incongruent
audio has the same effect on recalling by
writing words down. The role of decision
criterion also needs to be examined, that is if
the way the questions is shaped will affect
the number of words recalled. Other future
researches could also take another approach
with related audio stimuli and their influence
on memory.

depth. Neuropsychologia, 78, 51-62.
doi:10.1016/j.neuropsychologia.2015.09.
023
Marandi, R. Z., & Sabzpoushan, S. H.
(2014). Using Eye Movement Analysis to
Study Auditory Effects on Visual
Memory Recall. Basic Clinical and
Neuroscience, 5(1), 55-65. Retrieved
from https://www-ncbi-nlm-nihgov.libsecure.camosun.bc.ca:2443/pmc/ar
ticles/PMC4202595/
Myers, D. G., & DeWall, C. N. (2015).
Psychology (11th ed.). New York, New
York: Worth Publishers.

References
Dunifon, C. M., Rivera, S., & Robinson, C.
W. (2016). Auditory stimuli
automatically grab attention. Journal of
Experimental Psychology, 42(12), 19471958. doi:2048/10.1037/xhp0000276
Harrison, N. R., Witheridge, S., Makin, A.,
Wurger, S. M., Pegna, A. J., & Meyer, G.
F. (2015). The effects of stereo disparity
on the behavioural and
electrophysiological correlates of
perception of audio–visual motion in
34

Ton - J Camosun Psyc Res. (2019). Vol. 1, pp. 31-36.

Appendix A
Lists of words used in the experiment

List 1: Pronunciation of the words
Words
Cheese
Orange
Pain
Brother
Donut
Technology
Rejection
Dagger
Music
Anger
Doctor
Chaos
Perception
Chocolate
Panther
Hammer
Always
Line
Captain

List 2: Pronunciation of the words in List 1
Black
Iron
Heart
Moral
Home
Sun
Snake
Vision
Mischief
Winter
Surprise
Ice
Self
Tea
Wine
Thunder
Mission
Spider
Mind
Magic

35

Ton - J Camosun Psyc Res. (2019). Vol. 1, pp. 31-36.

Appendix B
Consent form used in the experiment
A Study on the Psychology of Audiovisual Memory
This study is investigating the influences that effect people’s memory. Before taking part in this
study, please read the consent form below. If you understand the statements and freely consent to
participate, please provide a signature and date at the bottom of the page.
Consent From
This study is designed to understand which factors might contribute to an individual’s memory
capacity. The study is conducted by An Ton, a student of psychology at Camosun College for the
partial fulfillment of the requirement for Psychology 110, Experimental Psychology. The study
has been approved by the instructor of the course, Grace Chan.
Participation in the study typically takes approximately 5 minutes and is strictly anonymous. All
responses will be kept completely confidential, and in no case will responses from individual
participants be identified. Rather, all data will be pooled and then analyzed.
Participation in this study is voluntary. Participants may withdraw from the study at any time.
Participants begin by watching a video showing a list of words and will be asked to recall them.
If participants have further questions about this study or their rights, they may contact the
principal investigator, An Ton (glawcc.nit@gmail.com) or the course instructor, Grace Chan at
250-370-3271 (ChanG@camosun.ca).
If you are 18 years of age or older, understand the statements above, and freely consent to
participate in the study, please sign below:
Signature: ____________________________ Date:________________

36