Research as a Learning Experience: Investigating Media Influence on Voters’ Opinions
—Bridget Farmer (Edited by Danielle Curtis)
My story begins just after the New Hampshire presidential primary in January 2007. Having recently finished a two–week internship with the Obama Campaign in Dover, New Hampshire, I was eager to continue my participation in politics. I wanted to spend the summer leading up to the Democratic Convention in August immersing myself in political thought, campaign methods, and, especially, voter reactions to media coverage. During the New Hampshire primary, I became aware of numerous news sources that are available to voters. Coverage of the election was on the internet, the radio, the television, in blogs, and in newspapers. As a people and country, we now have access to not only the words of our presidential candidates, but also their presentation style, their physical expressions and their movements (Denton and Woodward, 55–57).Therefore, I decided to research the reactions of voters to audio and video coverage of speeches by Senator Barack Obama whom, I was convinced, would be the Democratic presidential candidate even though he had not won in the New Hampshire primary. To do this project I applied for and received a Summer Undergraduate Research Fellowship (SURF) from the University of New Hampshire.
The author on campus at the University of New Hampshire.
I centered my research on a few basic questions, with the goal of investigating voters’ responses to video versus audio exposure to the same speech: Does the form of coverage (audio or video) influence the voters’ opinions of the speech? In other words, are voters more receptive to the spoken message itself or to the visual image of the speaker and his/her context, such as background, wardrobe, and mannerisms? Another question that arose during the project concerned the influence of information and opinions provided by news commentators after a speech.
While I would love to spend this article talking about my abundant success as a new researcher, that story is better suited to a work of fiction. Prior to this project, I was a research assistant with the Carsey Institute at the University of New Hampshire. My job gave me exposure to statistical analysis systems, experience creating a survey instrument, and practice with troubleshooting during a project. However, this summer research project was the first time that I would execute a project by myself (though I was under the supervision of a faculty member.) At every step of the project, I was met with a new frustration, a new puzzle, and a renewed desire to can the whole thing. In hindsight, the frustration I experienced has made me a stronger researcher and a more patient person.
The struggles I faced stemmed from my misconceptions about overall time needed for such a research project, subject recruitment and attrition, the complexities of data analysis, and working with different computer systems. (This last led to questions being left off some surveys.) Because I was constantly faced with difficulties, I learned to problem solve on the go, and to meet every challenge with energy and creativity.
Misconceptions and Methods
In my study, participants were successively sent three clips of recent (June/July 2008) Obama speeches over a six–week period, along with a survey to assess their reactions to the clip. Before listening to or viewing each clip, they were asked to fill out a questionnaire concerning their awareness of and involvement in national and local politics. In the first wave of questionnaires and surveys, all participants watched a video clip of Obama’s speech from June in which he talked about the economy. In the second and third waves, the participants were split into a video and an audio group of the same speech in order to assess differences in reactions to audio and video exposure. The second wave involved Obama’s speech on Father’s Day, the third his speech about patriotism in July. (See Appendices)
When I began my project, I thought there would be time for ten in–depth participant interviews in addition to the three waves of questionnaires, clips and surveys. It became almost immediately clear that this would not be possible for the allotted ten–week period, and so the interviews were removed from the study.
A second misconception I had about my research project was the ease of participant recruitment and retention. Participant recruitment takes a long time, I found, especially when there is no incentive or compensation for the participants other than helping the researcher. I recruited using word of mouth, mass emails, and flyers in senior centers and libraries. Though I planned for fifty participants, I started the first wave with nineteen. This number declined to seventeen for the second wave and eleven made it through the third wave. All participants lived in Hampden County, Massachusetts, the greater Springfield area: birthplace of basketball and home of Dr. Seuss. (For more information on the participants, see Table 1)
I was immensely relieved when the data collection and entry portion of my project was finished, and I began my data analysis in good spirits. Maybe I was finally getting the hang of the research process? Unfortunately, my relief was short–lived, for I soon realized my third misconception: my data analysis would take two weeks. For anyone unfamiliar with data entry and analysis, I have a suggestion: give yourself more time than you think you’ll need. In reality, my analysis took four weeks. If I had planned for that, I would have been able to devote more time to other problems that arose.
The analysis portion of this project was long, frustrating, but (of course) educational. In addition to learning about different statistical methods, I was reminded that tools, such as computers, are only helpful when one understands how to use them. I struggled to convert my data between Excel and two statistical programs, Stata and SPSS; and realize now that, if I had devoted more time to learning one system well, I would have been able to do more with my results. For this study, I used cross tabulations, which show the distribution of two or more variables simultaneously using a table, and Pearson’s Chi Square test, which measures how much evidence is provided against a null hypothesis and provides a p–value. (The null hypothesis assumes that no change or difference will occur in the sample.) A smaller p–value provides evidence against the null hypothesis, suggesting that a change did occur. In this study, variables were significant if the p–value was <.10. I learned about running cross–tabulations, chi–square tests, and a myriad of other statistical analysis tools although some may have been used incorrectly. I must caution anyone who would like to use this data that the raw version will be more helpful than my results.
Dealing with Data
When the time came to draw conclusions from my research, I found myself at a rare loss for words. Through my conclusions, I would share my knowledge with the rest of the academic and political community. If the conclusions do not add any greater knowledge to the subject, then what has been gained? There are, of course, times when a lack of conclusion is itself a finding, but I do not think that was the case here. Because my sample size was so greatly decreased, because I realized after the fact that my survey instrument was rather poorly designed, I had to get creative when discussing my results. As luck would have it, a few variables did appear to be significant based on audio or video exposure. I decided to take this gift and make the most of it, highlighting these variables and making them the focus of my conclusions.
Before analyzing my results from the surveys, I wanted to get an idea of my participants' backgrounds. When asked on the first questionnaire how informed they felt about the upcoming election, 32% were low to moderately informed, while 68% felt moderately to highly informed. When asked how involved participants were in gathering information from a list of sources, 36.8% had a low involvement level, that is, they gathered news or information from two or fewer sources; while 64.2% had high involvement, gathering information from three or more sources. The most common source of news was the online newspaper, with 73.7% reporting daily or frequent use. Interestingly, 73.7% of the sample had voted in the presidential primary elections.
Variables from the surveys, such as what the candidate highlighted, what left a lasting impression, how much the participants learned about policy, and the main message of the speech, were expected to be significant when the sample was split into video and audio groups. When I looked at the results, few of these options were, in fact, significant; and only two (highlighting what we need and highlighting what we could have) were significant across both waves.
Creative Conclusions
In the first wave, when all participants were shown a video clip of an Obama speech from early June, most participants indicated that the candidate’s words, tone, and body language were equally important. In the second wave, however, words left a lasting impression, as noted unanimously by both the audio and video groups. Interestingly though, the impression given by the words is different: the audio group felt that the speech provided information, while the video group felt the speech was meant to inspire the audience. Since both groups heard the same words, and since the first wave suggests that body language is not more important to video viewers than words or tone, there must be other factors, such as background or additional commentary, at work. Regardless of the reason, it appears that the audio group took more substance away from their clip than did the video group, who reacted more emotionally and picked up on the inspiration aspect of Obama’s speech.
In the second wave video clip, which contained Obama’s speech from Father’s Day, the candidate is speaking at the Apostolic Church of God in Chicago. His backdrop is the choir. After showing a portion of his speech, this clip showed part of an interview with Deval Patrick, the governor of Massachusetts. He, along with the host of the news show, analyzed portions of the speech and told viewers what Obama spoke about. In the audio clip, Judge Lynn Toler analyzed the same portion of the same speech. In both the audio and video groups, the audience could be influenced by the commentary as well as by Obama’s speech. This suggests that the differences in responses may not be between audio and video, but between commentators.
Though there are differences between the video and audio groups when it comes to what the candidate highlighted, it should be noted that these discrepancies exist because of an issue with the layout of the video questionnaire that caused the yes/no options for this question to be left off of the survey.
The third and final wave, dealing with Obama’s speech on patriotism, again shows differences between the audio and video groups with regard to the main message of the speech. In this case, the majority of the audio group felt that the speech was meant to inspire while the majority of the video group thought the speech was meant to attack another candidate. This surprised me because the speech was given in front of a number of American flags, and the video of the speech frequently pans the audience to show the excited faces of veterans and others in the crowd. The audio group was not exposed to this patriotic display and only had the words to influence them, yet they found the speech more inspirational than the video group. This again suggests that the difference between the two groups is not caused by visual cues present in one group and not in the other. It could also be the case that the visual cues did have an effect on the participants, but the effect was not one that would be evident when they were asked about the main point of the speech.
Because the third wave was the last, participants were asked how this study had affected their news–gathering behavior and opinions of the candidate. 76.9% said they would change their news–gathering behavior as a result of the study. When asked how their opinion of Obama had changed, 15.4% said there was no change, 46.2% said there was a little change, 30.8% experienced some change, and 7.7% noted a lot of change in their opinion of the senator.
Participants in this last wave followed some of the trends set in the second wave: significant differences existed between the groups when they were asked what the candidate highlighted in his speech, and whether he seemed to be attacking another candidate. Though there were large percentage differences in a number of the variables, these variables did not turn out to be significant between the audio and video groups.
No single variable was consistently significant across the two waves split into video and audio, but variables addressing the main message of the clip often experienced major differences. Even in the first wave, when all participants saw the same video clip, they were torn over the main message, with the majority choosing “inspiration” as the main message, and many others choosing “information.” In the second wave as well, the majority of both the video and the audio group chose inspiration as the main message, suggesting that the main message was not affected by the change in media. However, when asked about conveying information, the groups were at different ends of the spectrum; and the difference was significant. Discrepancies between the audio and video groups regarding whether the speech was meant to inspire, inform, or attack another candidate continued throughout all three waves. The lack of accord between the two groups suggests that there is some as yet undetermined influence, other than the candidate’s words, at work in conveying the main message, and the differences can be seen when a sample is split into audio and video groups.
This study does have some interesting findings: notably, that some kind of difference does exist between responses to video and audio versions of the same speech. However, it was not able to pinpoint exactly what the difference is. It would be beneficial to repeat the study with more specific questions, and perhaps with clips that are perfectly identical in spoken content, including commentary when present. In addition, the study should be repeated on a larger scale to see if other trends or differences emerge. Finally, it might be interesting to compare both Democrats’ and Republicans’ opinions about their own candidates and the other parties’ candidates to determine if differences can exist along party lines.
What Was I Thinking?!
When I put together this study, I wanted to stay involved in politics and hold on to the energy and excitement I felt after the New Hampshire primary. Though I had a vague idea of what I wanted to study and how I wanted to conduct this research project, I should have given more thought to the method of execution and goals for the project. While the “play it by ear” approach works in certain situations, a research project is not one of them.
I cannot discount the personal growth that went on during this project. I remember racking my brain after the initial wave of surveys for a way to make the study more interesting for myself. I remember the joy I felt when I realized I could redirect my project to compare audio and video (and that it would only require minor amounts of adjustment.) I muddled my way through learning the statistical programs Stata and SPSS, figuring out—sometimes too late—the proper and improper uses of chi–square tests, frequencies, and percentages in presenting results. If college is the time to develop and hone critical thinking and problem solving, then individual research projects should be required of everyone.
This project would have been impossible without funding from the Hamel Center for Undergraduate Research and my project benefactor Mr. Dana Hamel. I would also like to thank Donna Brown and Peter Akerman of the Hamel Center for supporting me in my various research endeavors. Many thanks also to my project mentor, Dr. Dante Scala, for his guidance and patience with this project, and for keeping me on track. A huge thank you is due to everyone at the Carsey Institute for making it a cheerful place to work and learn, and especially to Dr. Mil Duncan for allowing me to use the Carsey workspace. Finally, thank you to Dr. Nena Stracuzzi for endless guidance and support.
References
Denton, Robert E. and Gary C. Woodward. Political Communication in America. 3rd ed. Westport, CT: Praeger, 1998.
Table 1
| Participant Information (N=19) | |
| Age Range | 19 to 65 |
| Percentages of total: | |
| Under age 35 | 63.20% |
| Some college education | 52.60% |
| Graduate/Professional degree | 26.30% |
| Registered Democrats | 73.70% |
| Independents | 26.30% |
| Voting in first presidential election | 42.10% |
| Participant Ideology (N=17) | |
| Very liberal | 35.30% |
| Liberal-Moderate | 64.70% |
Appendices: Questionnaires and Survey Forms
Appendix A: Wave one questionnaires and clip reaction survey
Questionnaire
Part I: Personal Information
For each question, please fill–in or circle the answer that most applies to you.
- Hometown and/or Current Town: _________________
- Age: ___ years
- What is your main ethnic/racial background?
- African American
- Native American
- Asian American
- Caucasian (White)
- Hispanic American
- Other ____________ (please fill in)
- What is your highest level of education?
- Less than High School
- High School/ GED
- Some College
- Associate Degree
- Bachelor’s Degree
- Graduate or Professional Degree
- With which political party are you registered?
- Democrat
- Republican
- Independent
- Other __________(please fill in)
- Have you ever volunteered/worked with a Presidential Campaign? Y/N
If YES please give name of candidate(s) and year(s) volunteered:
Part II: Background Information
- Through what medium(s) do you receive your daily news? (circle all that apply):
- Daily Local Newspaper
- Daily National Newspaper
- Online Newspaper
- Online blogs
- Radio
- Daily/Morning News on Television
- Nightly News on Television
- Other _________________________ (please fill in)
- Do you get a newspaper delivered to your house? Y/N
If YES, please provide name of paper: ____________________ - If you get your news from the television, which station(s) do you use most frequently? (circle all that apply):
- ABC
- NBC
- CBS
- CNN
- FOX
- Other ___________________ (please fill in)
- Did you vote in a Massachusetts Presidential Primary? Y/N
If YES, in which primary did you vote? Democrat/Republican - Prior to voting in the Primary, how did you prepare? (circle all that apply):
- Read newspaper articles about the candidates
- Talked to friends/neighbors/family members about the candidates
- Visited the candidates website
- Read online blogs about the candidates
- Watched televised debates between candidates
- Listen to debates between candidates on the radio
- Talked to a representative of the candidate
- Watched the news on television to gain information about the candidates
- Other ______________________________ (please fill in)
- I did not do any of these
- Do you intend to vote for President in the November election? Y/N
- If YES how will you prepare to make your decision? (circle all the apply):
- Read newspaper articles about the candidates
- Talk to friends/neighbors/family members about the candidates
- Visit the candidates website
- Read online blogs about the candidates
- Watch televised debates between candidates
- Listen to debates between candidates on the radio
- Talk to a representative of the candidate
- Watch the news on television to gain information about the candidates
- Other ______________________________ (please fill in)
- I will not do any of these
- Do you discuss politics or political issues with friends, family, or peers? Y/N
- Do you feel that voting is important? Y/N
- Is this the first PRESIDENTIAL election in which you will be voting? Y/N
- Please describe how aware and informed you feel you are about the upcoming Presidential election and the candidates:
1 2 3 4 5 6 7 8 9 10
not at all informed very informed
If possible, please explain your answer:
Part III: Clip Reaction
Section I.
This section will ask questions about your reaction to the video clip you will watch. Please watch the clip once and then fill out the survey. It is ok if you do not remember everything.
- Is this clip familiar to you (have you seen it before)? Y/N
If YES please state approximately when/where you saw it: - What was the main message of this clip?
- To inspire listeners
- To convey information about policies
- To push a plan or policy
- To attack another candidate
- Other _____________________ (please fill in)
- Please describe what lead you to the above conclusion:
- The candidate’s tone
- The candidate’s words
- The candidate’s body language
- Other ____________________ (please fill–in)
- What left a more lasting impression on you–
- The candidate’s words
- The candidate’s movements/body language.
- What was the overall tone of the clip? Circle all that apply
- Defensive
- Attacking
- Informative
- Positive
- Negative
- Other _____________________(please fill in)
- If possible, please describe your reasoning for the answer above.
- In his speech, the candidate highlighted:
- What we need
- What we could have
- What we do not have
- Things to which we have an obligation
- Things to which we have a right
- Did you find there to be anything distracting in the clip? Y/N
If YES, was it:- The camera angle
- Camera shots not focused on the candidate
- The candidate’s movement
- The crowd
- Other ____________________ (please fill in)
- By viewing the clip, how much do you feel you learned about the candidate’s policy?
1 2 3 4 5 6 7 8 9 10
nothing quite a bit
Please expand upon your answer: - By viewing the clip, how much do you feel you learned about the candidate’s private life or family?
1 2 3 4 5 6 7 8 9 10
nothing quite a bit
Please expand upon your answer: - Based on the clip, the candidate is (circle all the apply)
- Someone you would have as a friend
- Someone you would trust
- Someone you would have over to your house
- Approachable
- Stern
- Harsh
- Easygoing
- Capable
- Other _________________________ (please fill in).
- How did the candidate convey his image/attitude/personality to you?
- Through clothing choice
- Through speech
- Through mannerisms
- Through facial expressions
- Other __________________ (please fill in).
- Please describe your reaction to the image he presented (encouraged, inspired, annoyed, wary, etc.)
- Is there anything else you would like to say/react to/share about your experience with this video and survey?
Appendix B: Wave 2 questionnaire
General Information
- Since the last survey, have you changed the medium(s) through which you receive your daily new, or started or stopped delivery of a newspaper to your house? Y/N
If YES, please describe: - Since the last survey have you (circle all that apply):
- Read newspaper articles about the candidates
- Visited the candidates website
- Read online blogs about the candidates
- Talked to a representative of the candidate
- Watched the news on television to gain information about the candidates
- Talked about the candidate with family, friends, or neighbors?
- Other ______________________________ (please fill in)
- I have not done any of these since the last survey
- Since the last survey have you become involved in one of the Presidential Campaigns? Y/N
If YES please describe that involvement: - Please describe how aware and informed you feel you are about the upcoming Presidential election and the candidates:
1 2 3 4 5 6 7 8 9 10
not at all informed very informed
If possible, please explain your answer: - Please identify yourself on the following scale:
1 2 3 4 5 6 7 8 9 10
very liberal moderate very conservative
Appendix C: Waves 2 and 3 audio reaction survey
Clip Reaction
Section I.
This section will ask questions about your reaction to the radio segment you will listen to. Please listen to the radio and then fill out the survey. It is ok if you do not remember everything.
- Is this segment familiar to you (have you heard it before)? Y/N
If YES please state approximately when/where you heard it: - What was the main message of this segment?
Not at All A Little Some A Lot A. To inspire listeners 0 1 2 3 B. To convey information about policies 0 1 2 3 C. To push a plan or policy 0 1 2 3 D. To attack another candidate 0 1 2 3 E. Other _____________________ (please fill in) 0 1 2 3 - Please describe what lead you to the above conclusion:
- The candidate’s tone
- The candidate’s words
- Other ____________________ (please fill–in)
- How much did the following leave a lasting impression on you?
Not at All A Little Some A Lot 1. The candidate’s words 0 1 2 3 2. The candidate’s tone 0 1 2 3 - What was the overall tone of the clip?
Not at All A Little Some A Lot A. Defensive 0 1 2 3 B. Attacking 0 1 2 3 C. Informative 0 1 2 3 D. Positive 0 1 2 3 E. Negative 0 1 2 3 F. Other ___________(please fill in) 0 1 2 3 - If possible, please describe your reasoning for the answer above.
- In his speech, the candidate highlighted
Not at All A Little Some A Lot A. What we need 0 1 2 3 B. What we could have 0 1 2 3 C. Things to which we have a right 0 1 2 3 D. The mistakes of others 0 1 2 3 E. How he will fix mistakes 0 1 2 3 - If present, were the following distracting?
Not at All A Little Some A Lot A. Commentary by hosts 0 1 2 3 B. Voice over by commentators 0 1 2 3 C. The crowd 0 1 2 3 D. Other ____________________ (please fill in) 0 1 2 3 - The commentary was distracting:
0 1 2 3 - By listening to the clip, how much do you feel you learned about the candidate’s policy?
1 2 3 4 5 6 7 8 9 10
nothing quite a bit
Please expand upon your answer: - By listening to the clip, how much do you feel you learned about the candidate’s private life or personality?
1 2 3 4 5 6 7 8 9 10
nothing quite a bit
Please expand upon your answer: - Based on the segment, the candidate is (circle all the apply)
Yes No A. Someone you would have as a friend 1 0 B. Someone you would trust 1 0 C. Approachable 1 0 D. Stern 1 0 E. Harsh 1 0 F. Easygoing 1 0 G. Capable 1 0 H. Other __________________ (please fill in). 1 0 - How did the candidate convey his image/attitude/personality to you?
Not at All A Little Some A Lot A. Through word choice 0 1 2 3 B. Through tone 0 1 2 3 C. Through speaking style (pauses, etc.) 0 1 2 3 D. Other ____________________ (please fill in) 0 1 2 3 - Please describe your reaction to the image he presented (encouraged, inspired, annoyed, wary, etc.)
- Has this video changed your opinion of Obama’s patriotism?
1. Not At All 2. Somewhat 3. A Lot - Is there anything else you would like to say/react to/share about your experience with this radio segment and survey?
Appendix D: Wave 3 questionnaire
General Information
- Since the last survey, have you changed the medium(s) through which you receive your daily new, or started or stopped delivery of a newspaper to your house? Y/N
If YES, please describe: - Since the last survey have you (circle all that apply):
- Read newspaper articles about the candidates
- Visited the candidates website
- Read online blogs about the candidates
- Talked to a representative of the candidate
- Watched the news on television to gain information about the candidates
- Talked about the candidate with family, friends, or neighbors?
- Other ______________________________ (please fill in)
- I have not done any of these since the last survey
- Since the last survey have you become involved in one of the Presidential Campaigns? Y/N
If YES please describe that involvement: - Before listening to this radio broadcast, are you aware of the discussion going on about Obama’s patriotism?
1. Not At All 2. Somewhat 3. A Lot - Will you change any behaviors after taking part in this study (read the newspaper, watch the news, visit political websites)? Yes/ No
- Has this study changed your opinion of Obama or his politics?
0. Not at all 1. A Little 2. Somewhat 3. A Lot
