Influence of User Comments on Perceptions of Media Bias and Third-Person Effect in Online News

As online and mobile information technologies continue to become more commonplace, Americans are increasingly turning to these sources for news. Online news sites often allow readers to post comments about news stories. Using an experiment, the authors of this study sought to understand how user comments influence individual perceptions of media bias and third person perception in online political news. Results indicate that user comments accompanying online content can influence perceptions of bias in media depictions of presidential candidates. Additionally, user comments can influence perceptions of how much online news affects the political attitudes of others (third-person perception).

As online and mobile information technologies continue to become more commonplace, Americans are increasingly turning to these sources for news (Pew Research Center, 2010). At the same time the emergence and proliferation of these new communication modalities raise questions about how the use of these forms affects users (Yzer & Southwell, 2008). Although online and mobile communication modalities are potentially more accessible, dynamic, and interactive than traditional media forms, new communication modalities also differ from traditional communication media forms in that they provide an electronic space in which interpersonal and traditional media sources converge (Walther, Carr, et al., 2010). Therefore, when a user accesses online versions of traditional media forms-such as the website for a newspaper or broadcast television station-not only might that user encounter news content created by a professional journalist but she might also encounter user comments about that content. These user comments might be astute, reasonable, and well written; they might be crass, radical, and informal; or they might be some combination thereof. How this mixing of interpersonal and traditional media information influences the processing of online content (as well as the perceptions, reactions, persuasion, or learning that results from such processing) is only beginning to be explored in the literature. The current project is focused on advancing understanding in this area by exploring how user comments influence perceptions of media bias and third-person effect (TPE) among readers of online political news.

User Comments and Online News
Online news sites often allow readers to post comments about news stories. This feature of online news provides opportunity for user engagement with a news source as well as with other readers of the site. In best cases, this user participation can result in an online conversation about the news (Glaser, 2008), while in worst cases it might result in ''cacophonous shouting matches'' among online readers (Ingram, 2010, {1). As a result, news organizations continue to wrestle with how to monitor and regulate user comments on news websites (Brisbane, 2010;Perez-Pena, 2010;Pitts, 2010;Swidey, 2010).
Beyond media production-oriented questions about how user comments affect the online environment are issues of what effects user comments have on perceptions of online content. Researchers have just begun to explore this area. For example, Walther, DeAndrea, Kim, and Anthony (2010) found that user comments affected perceptions about the effectiveness of online Public Service Announcement (PSA) videos. Also, although not studying online comments directly, Thorson, Vraga, and Ekdale (2010) found that the credibility of a news story was affected when it appeared adjacent to the opinionated commentary of a blogger. Taken together these results provide preliminary evidence that the broader online context influences perceptions of Internet content. The authors of the current study seek to continue this line of research.

Media Bias and Third-Person Perception
A significant line of research in mass communication research has focused on perceptions of bias in the media. Much of this research investigates how individual-level traits influence perceptions of media bias. For example, research about the hostile media effect has found that greater levels of partisanship results in more perceived media bias (Gunther & Schmitt, 2004;Vallone, Ross, & Lepper, 1985). Attributes of media content have also been examined in an effort to understand how these qualities influence perceptions of media bias (Lowry & Shidler, 1995;Smith, 2010). Additionally, Eveland and Shah (2003) proposed that interpersonal factors might affect perceptions of media bias, finding that more conversation with like-minded others was related to greater individual perceptions of media bias. The current study explores the convergence of media source and interpersonal factors in affecting perceptions of media bias, by examining how online user comments influence perceptions of bias in online news. While user comments are an attribute of online media content (a media source factor), they also represent a computer-mediated version of interpersonal communication. Therefore, we predict that online user comments will contribute to user perceptions of media bias in online news: Hypothesis 1: Online news stories with partisan comments will be perceived as more biased than online news stories without comments or with mixed comments about both candidates.
The TPE posits that people generally believe other people are more influenced by media content (Davison, 1983;Gunther, 1991). TPE includes a perceptual and behavioral component. Third-person perception (TPP) occurs when an individual perceives that media content affects someone else (the referent other) more than it affects the self. A recent meta-analysis of 17 years of research found a clear TPP effect across more than 100 studies (Sun, Pan, & Shen, 2008). This meta-analysis found that key moderators of TPP were desirability of the message influence (i.e., messages that are not socially desirable [e.g., pornography] are perceived to have more of an effect on others), vulnerability of others (e.g., children might be perceived to be more affected by television violence than adults), similarity of self to others (i.e., when the sociodemographics of self and others are more similar, TPP is reduced), and likelihood of the other to be an audience member of media content (e.g., adolescents might be perceived to be more affected by music videos than adults, because adolescents are more likely to be the audience for such content). In contrast to TPP, first-person perception (FPP) occurs when an individual perceives that he or she will be affected by media content more than someone else (Golan & Day, 2008). FPP is generally understood to occur with prosocial media (e.g., a public-service announcement about practicing safe sex might be perceived to have more of an effect on oneself compared to others; Duck, Terry, & Hogg, 1995).
Although not a requirement of TPP, perceptions of media bias have been found to increase the magnitude of TPP (Cohen, Mutz, Price, & Gunther, 1998;Gibbon & Durkin, 1995). For example, viewers of a television program about smoking who perceived the program to present balanced coverage of the issue reported FPP (greater effect on self), whereas viewers who perceived an anti-smoking bias in the program reported TPP (greater effect on others; Gibbon & Durkin, 1995). As discussed previously in presenting Hypothesis 1, we propose that online user comments can contribute to perceptions of media bias. If this is so, then it's also likely that user comments will increase how much an online news article is perceived to affect others' attitudes about political candidates. That is, more partisan comments will lead to greater perception of media bias and so also more TPP: Hypothesis 2: Online news stories with partisan comments will result in greater TPP than online news stories without comments or with mixed comments.

Method
In order to address these hypotheses, we conducted an experiment examining how different user comments influence perceptions of media bias and TPP among readers of online news stories.

Procedure
An online survey created with LimeSurvey software was used to randomly expose participants to experimental stimuli and to collect responses to questions both before and after viewing the stimuli. All experimental conditions included a web page with the same news story, titled ''McCain, Obama Diverge on Iraq, Energy & Student Loans.'' The story described both candidates' positions on the issues of Iraq, energy, and student loans, and provided critiques from the Obama campaign about McCain's issue positions and critiques from the McCain campaign about Obama's issue positions. Overall, the news article was written to be balanced in amount and tone of content for both candidates. The news story was designed to look like an online news article from the USA Today website and included a special ''2008 election coverage'' graphic subheading that was used by USA Today for election news during the 2008 campaign. Although the news article was the same for all experimental conditions, the user comments varied for each condition. The control condition included no comments (N ¼ 45), the Obama condition included comments that were pro-Obama/ anti-McCain (N ¼ 63), the McCain condition included comments that were pro-McCain/anti-Obama (N ¼ 71), and the mixed condition included a mixture of pro-Obama, pro-McCain, anti-Obama, and anti-McCain comments (N ¼ 48). Examples of Obama condition comments included, ''What about McCain saying we will be in Iraq for 100 years??? What a loser he doesn't want our troops out of Iraq;'' and ''Obama has principles. He'll do whatever he thinks is best. He's not run by lobbyists and fat cats.'' Examples of McCain condition comments included, ''What about Obama saying the surge didn't work??? What a loser he can't admit we are winning in Iraq;'' and ''McCain has principles. He'll do whatever he thinks is best. He's not run by lobbyists and fat cats.'' As much as possible, the comments for both partisan conditions were the same, with changes to the candidate names and, as appropriate, to the issues or positions mentioned in the comments. Comments included misspellings, slang, and incorrect grammar similar to what might be experienced in any online forum. User names were included with comments, and some names conveyed additional information about the viewpoints of the commenter (e.g., SouthParkConservative and BlueRising), while others were more generic (e.g., Mack_in_NY, true-sayer11). Overall, the Obama and McCain conditions function as partisan conditions in this experiment because the comments for those conditions present a uniformly partisan viewpoint.

Participants
Participants were undergraduate students (N ¼ 227) from the University of Oklahoma who received course or extra credit for participation. Regardless of whether they planned to vote, 65% of participants reported that they would vote for McCain (n ¼ 147), 34% reported that they would vote for Obama (n ¼ 178), and less than 1% reported they would vote for Nader (n ¼ 2

Measures
Control variables. Political ideology and presidential election involvement served as control variables. Political ideology was assessed through two questions that asked respondents to indicate their political ideology about social issues (e.g., abortion and welfare) and economic issues (e.g., taxes and jobs) with possible responses ranging from 1 (extremely liberal) to 7 (extremely conservative). These two questions were averaged for single measure of political ideology (a ¼ . Presidential election involvement was measured with eight adjective-opposite 7-point pair scales. Participants were asked ''How important to you is the 2008 presidential election?'' and then presented with pair scales that included unimportant/important, no concern/much concern, irrelevant/relevant, means nothing/means a lot, doesn't matter/matters, insignificant/significant, not interesting/interesting, and boring/exciting. Responses to all adjective pairs were averaged for a final presidential election involvement composite score, with a higher score indicating more involvement (a ¼ .93, M ¼ 6.04, SD ¼ .99).
Dependent variables. Perceptions of media bias and TPP served as dependent variables for this analysis. Several questions were asked to assess media bias. Participants were asked, ''Regarding the web page that you viewed, would you say the portrayal of the presidential candidates was strictly neutral or biased in favor of one side or the other?'' Possible responses ranged from 1 (neutral) to 7 (biased). Participants were also asked whether the news story was fair or unfair and whether the authors responsible for the news story were strictly neutral or extremely biased using a 7-point scale. These three questions were averaged to form a single measure of overall media bias (a ¼ .89, M ¼ 3.39, SD ¼ 1.56).
As part of our measure of media bias, we also assessed participant perceptions of how the candidates were depicted in the news story. Participants were asked whether the web page made Barack Obama and John McCain seem good or bad and likable or unlikable on a 7-point scale, with higher scores indicating more negative depiction. These questions were averaged for each candidate (Obama: TPP was also assessed through several questions. Participants were asked, ''Regarding the web page that you just viewed, how do you think the story affected your attitude toward Barack Obama?'' with instructions that a response of 1 meant that the respondent became more favorable in his or her opinion of Barack Obama and a response of 7 meant that the respondent became more unfavorable in his or her opinion of Barack Obama. The same question was also asked about John McCain. These questions were repeated to assess how respondents thought the news story would affect the attitudes of politically neutral students, politically neutral voters (from the respondents' home state), and politically neutral U.S. voters. To calculate TPP, we took an individual's perceived attitude effect score for the referent other and subtracted the attitude effect score for the self, using this difference as the dependent variable.

Data Analysis
Multivariate analysis of covariance (MANCOVA) was used to test for effect of experimental stimuli on dependent variables. Dependent variables in this study were perceptions of media bias and TPP. Experimental condition (control, Obama, McCain, and mixed) served as a fixed factor and political ideology and presidential election involvement were included as covariates. Although random assignment to experimental conditions generally alleviates the need to control for variables that vary within subjects, given the potential importance of political ideology and election involvement in determining how individuals react to online campaign news, we decided to conduct a more stringent test and control for these variables (Brader, 2006). Thus, our final estimates of experimental effects accounts for these potentially important individual-level traits. Significant omnibus results for experimental condition were followed by univariate tests. Multiple comparison tests with Bonferroni adjustment were then used to compare differences in estimated means for variables with significant univariate results.
Multiple comparison tests with Bonferroni adjustment were used to compare differences in estimated means for the variables with significant univariate results (see Table 1). Participants in the McCain condition perceived more media bias than those in the control condition did (though this difference was only marginally significant). Participants in the McCain condition perceived that Obama was depicted more negatively and McCain was depicted more positively than participants in any other condition did. Conversely, participants in the Obama condition perceived McCain to be depicted more negatively than participants in any other condition. Overall, these results largely support the prediction of Hypothesis 1. Furthermore, even when means weren't significantly different, the means were in the predicted direction, providing additional evidence in support of Hypothesis 1.
Hypothesis 2 predicted that TPP would be greater for the news stories with partisan comment conditions. MANCOVA results indicated a significant main effect for experimental condition, F(18, 654) ¼ 2.78, p < .001, Z 2 ¼ .07, with significant univariate results for effect on other students' attitudes about Obama compared to effect on one's own attitude, F(3, 221) ¼ 9.41, p < .001, Z 2 ¼ .11, effect on other students' attitudes about McCain compared to effect on one's own attitude, F(3, 221) ¼ 8.87,  Table 2 for multiple comparison test results for significant univariate results. These tests indicate that respondents perceived that the partisan comment conditions would have the strongest effects on other students. One might posit that the positive scores for TPP in our partisan condition results actually represents FPP, in that the positive TPP values mean respondents are reporting that they themselves are becoming more unfavorable to the candidate compared to the other students as a result of the news story in those conditions. To explore this possibility, we ran another MANCOVA with the unadjusted TPP questions as the dependent variables. These results indicated a significant main effect for experimental condition, F(24, 648) ¼ 3.13, p <.001, partial Z 2 ¼ .10, with significant univariate results for almost all of the referent other categories: students' attitudes about Obama, F(3, 221) ¼ 18.72, p < .001, partial Z 2 ¼ .20, and McCain, F(3, 221) ¼ 15.93, p < .001, partial Z 2 ¼ .18; home state voters' attitudes about Obama, F(3, 221) ¼ 5.38, p < .001, partial Z 2 ¼ .07; and U.S. voters' attitudes about Obama, F(3, 221) ¼ 11.09, p < .001, partial Z 2 ¼ .13, and McCain, F(3,221) ¼ 5.40, p < .001, partial Z 2 ¼ .07. There were no main effects for experimental condition on participant's own attitude about Obama, F(3, 221) ¼ 1.67, p ¼.18, partial Z 2 ¼ .02, or McCain, F(3, 221) ¼ 1.13, p ¼ .34, partial Z 2 ¼ .02. With regard to the control variables, political ideology had, F(8, 214) ¼ 9.17, p < .001, partial Z 2 ¼ .26, a main effect on the TPP measures.
These main effect results illustrate that the experimental condition didn't cause different levels of perceived effects on the participants' own attitudes about the candidates. Instead, the respondents' favorability toward the candidates was stable across conditions Note. Entries are estimated means (and standard errors) for difference between perceived effect of news story on referent other and perceived effect of news story on self, controlling for political ideology and presidential election involvement. Means in the same rows with the same superscripts differ at the following levels: a,b ¼ p < .01; c ¼ p < .05; d ¼ p < .10.
(see Table 3 for multiple comparison test results). Conversely, participants did perceive effects on others, and these effects were greatest for the partisan conditions, in that partisan conditions usually resulted in favorable attitude movement for the candidate supported by the comments and unfavorable attitude movement for the candidate attacked by those comments. Therefore, these results indicate classic TPP, as the respondents perceived attitudinal effects on others in the direction that would be predicted, while the respondents' own attitude stayed the same, regardless of experimental condition.

Discussion and Conclusion
This project used an experiment to examine how online user comments affected individual perceptions of media bias and TPE in online news. Overall, the results indicate Note. Entries are estimated means (and standard errors) for perceived effect of news story controlling for political ideology and presidential election involvement. Possible responses for each question ranged from 1 to 7 with higher scores indicating that the news story would make the respondent or referent other have a more unfavorable attitude about candidate. Means in the same rows with the same superscripts differ at the following levels: a,b,c, ¼ p < .01; d ¼ p < .10.
that user comments accompanying online content can influence perceptions of that content. Our results fit with other recent, preliminary findings that indicate elements of the online environment can affect how individuals perceive online content (Thorson et al., 2010;Walther, DeAndrea, Kim, & Anthony, 2010). Our main finding is important for anyone developing or studying Internet content, as it illustrates that the components of online information don't function in isolation, but might be affected by other elements of the online environment. Therefore, it's clear that all of the aspects of the online environment should be considered when examining new media effects. With regard to media bias, our manipulation of user comments didn't result in significant effect on overall report of media bias, but the manipulation did influence user perceptions of media depictions of presidential candidates. Although the news stories in our experiment were always the same, the varying tone of user comments about presidential candidates influenced participant perceptions of the bias in those depictions. These results indicate the computer-mediated interpersonal dimension of online content can influence perceptions of online news. The influence of user comments makes sense because comments are both a structural feature of online media content and they represent a computer-mediated version of interpersonal communication. In the future, researchers should continue to explore the effects of the mixing of media and interpersonal factors that occurs online. For example, researchers could compare the persuasive effects of user comments to expert or official online content. Research should also explore how offline interpersonal factors are related to online interpersonal effects. For example, research could investigate whether individuals with strong offline interpersonal networks are affected more or less by online comments than individuals with weaker interpersonal networks. This line of research could also examine how computer-mediated communication competency moderates these effects (Spitzberg, 2006).
A recent meta-analysis found that message desirability is the most important moderator of TPP (Sun et al., 2008). In our study, TPP was highest in the partisan comment conditions, so the perceived effect of online content on others was greatest when social cues (the user comments) were clear. In our control and mixed conditions, there were no significant differences between perceived effects on referent others. However, when partisan comments were introduced, participants understood which candidate was favorable or unfavorable and they perceived an attitudinal shift among others in that direction. Therefore, one of the ways in which user comments might influence the perceptions and attitudes of online audiences is by providing social cues about what content is good/bad, fair/unfair, or accurate/inaccurate. Our results indicate that user comments can be quite influential.

Limitations
Our TPP results were strongest for the other students referent group. It's likely the similarity of other students to our respondents was at play in causing our participants to report significant media influence on their attitudes. At the same time, TPP results have been found to decrease in studies where more self-other comparisons are made (Sun et al., 2008). We asked about the influence of the news story on other students first (before home state or U.S. voters), so this might have influenced our significant results for the students referent group. Rotating the order in which referent groups were presented to study participants might produce results different from those found here.
It's important to note that our analysis controlled for political ideology and presidential election involvement, two potentially important individual factors. Media bias research has often focused on the effect of partisanship on evaluations of media bias (Gunther & Schmitt, 2004;Vallone et al., 1985). Our analysis accounts for this important factor and shows that online comments can drive bias perceptions and TPP in the predicted directions even with controlling for this influence. Furthermore, one limitation of our research was that it was conducted in a conservative state where the 2008 presidential contest wasn't competitive, so controlling for political ideology and election involvement helps us account for these characteristics of our research sample. Ultimately, the strength of our results indicates this area deserves further study to more fully understand how comments influence the processing of online content.

Implications for the News Industry
Beyond contributing to the academic literature about media bias and online news effects, our results have implications for the industry that produces online news. Much of the current debate about whether online user comments should in some way be monitored or regulated by news producers is focused on the problem of uncivil, rude, or obscene user comments (Perez-Pena, 2010;Pitts, 2010). However, our results indicate that online news producers should also consider the role of user comments in influencing perceptions of the news they produce. That is, even civil comments-if they're largely providing the same viewpoint-might influence how readers of online news perceive the information presented in an online news article. So, for example, if a New York Times article about a Republican presidential candidate includes user comments that mostly offer liberal critiques of that candidate, then the depiction of that candidate in the article might be perceived by readers as being more negative. It isn't difficult to see how in the long run such perceptions could contribute to overall estimations of media bias, particularly related to an individual media source. Therefore, if managers of online news sources desire to be perceived as producing objective journalism, then news producers will need to account for the influence of user comments in public perceptions of their news. Of course, news producers have no control over what user comments will be posted on an online news story, but if online news sources have a system of moderating comments (Brisbane, 2010), then it would be possible for moderators to choose to publish or promote a range of different comments about a news story. If published, featured, or promoted user comments are diverse, then there's less likelihood that a news story would be perceived as biased toward any one individual or issue. However, knowing how best to strike the balance between user restrictions that might result from comment moderation with the news industry's general desire for greater user engagement with online content is a challenge for online news producers. Therefore, continued academic research in this area as well as insights from online news professionals are needed to guide the development of best practices in this area. Experimentation in both the academic lab and the online newsroom will help inform decisions about how best to structure the online news environment.

Further Research
In the future, researchers should continue to explore the effects of the mixing of media and interpersonal factors that occurs online. For example, researchers could compare the persuasive effects of user comments to expert or official online content. Research should also explore how offline interpersonal factors are related to online interpersonal effects. For example, research could investigate whether individuals with strong offline interpersonal networks are affected more or less by online comments than individuals with weaker interpersonal networks. This line of research could also examine how computer-mediated communication competency moderates these effects (Spitzberg, 2006).