Utterly Meaningless » Blog Archive » A Measure of Media Bias
  • A Measure of Media Bias

    Filed at 11:43 am under by Scott Somerville

    This elegant study uses think tanks and elected officials to construct an objective scale for measuring the political tilt of various media outlets.

    According to this research, the New York Times scored a 73.7 (23.7 points left of the average US voter baseline of 50.0), and Fox News Special with Brit Hume scored a 39.7 (20.3 points right). This is why the “newspaper of record” seems to be less “fair and balanced” than Fox News to me.

    The Political/Media Spectrum

    The methodology for this study was very clever. They used elected officials to construct an ideological scale, and then used think tanks to find a common referent for the media and the politicians. Thus, if Henry Hyde (R-IL) routinely cites the National Right to Life Association and the Washington Times cites NRTL just as often, one may reasonably assume they share a roughly similar perspective when it comes to abortion. The authors of the study used the Americans for Democratic Action scorecard for their numerical scale.

    Take a lot of think tanks and a lot of politicians, and you can quantify the political spectrum. Then all you have to do is go count how many times each media outlet refers to various think tanks.

    13 Responses to “A Measure of Media Bias”


    Comment by
    Scott Somerville
    July 11th, 2006
    at 3:42 pm

    Chris, I read two papers whenever I can. I was intrigued to see the Wall Stree Journal so far to the left of the scale, and would like to dig into those numbers more deeply, but it didn’t look a major design flaw to me. The Wall Street Journal is a pro-business paper that almost NEVER takes a “social conservative” position on issues like abortion or same-sex marriage, but it takes a capitalist position on matters of economic regulation. I think the Journal has more in common with Rudy Giuliani than Jerry Falwell.

    I really like the methodology these researchers used, and think it could be replicated in additional studies. These researchers used the Americans for Democratic Action scale, but you could just as easily use any other group’s “scorecard” to see whether you can replicate the results.

    It’s a thought-provoking study, which has generated some interesting discussions. I’ve looked at some bloggers’ comments on this research, plus the fisking that “Media Matters” did. The original authors of the study have responded quite effectively to these critiques. So far, I can’t see anything I would call a real methodological flaw.

    In a sense, it shouldn’t really matter. What kind of thinker really cares whether he or she is in the “mainstream” or not? I hope to have the courage of my convictions, whatever the multitudes may say. But it’s interesting to TRY to find an objective way to measure stuff like this.


    Comment by
    Daryl Cobranchi
    July 11th, 2006
    at 6:29 pm

    This study was utterly debunked months ago for flawed methodology. How many times have I cited/linked HSLDA and Mike Farris’ Smith’s posts at the Moonie Times. Does that mean that HE&OS is a right-leaning site?

    The authors did not take into account citations that were critical of the primary source.

    Better luck next time.


    Comment by
    Daryl Cobranchi
    July 11th, 2006
    at 6:32 pm

    And the WSJ is the most liberal paper in America? Have you ever read the WSJ’s editorial page? Just slightly to the right of Atila the Hun.

    And, finally, Brit Hume is far from the most rabid of the pit bulls at Faux News. Where’s Hannity show up? Bill O’Reilly?


    Comment by
    Daryl Cobranchi
    July 11th, 2006
    at 9:05 pm

    True, but a paper’s bias is most prominent on the editorial page. That’s what should be judged. Hard news (except on Faux) is rarely politicized to nearly the extent that the editorials are.

    By that standard, the WSJ and the Moonie Times are among the most conservative papers in America. And Faux News slants and spins way more than the NYT.

    This is why the “newspaper of record” seems to be less “fair and balanced” than Fox News to me.

    No, it seems more F&B to you because it reinforces your already-held opinions.


    Comment by
    Scott Somerville
    July 12th, 2006
    at 8:37 am

    Daryl, I’d appreciate a reference to the “debunking.” I was initially impressed by the research methodology, and I’ve read the authors’ response to the initial criticisms.

    I know that if anybody used this method on my own writings, they’d find me well to the left of Hillary Clinton… but that’s because Hillary is pretty close to the center and I intentionally cite the leftmost source I can find for any point I want to make. So I might well wind up on the far left along with the Wall Street Journal. That’s an interesting fact about the research methodology, but it doesn’t change the underlying argument.

    The underlying argument is that we HAVE a meaningful way to construct an objective “political spectrum” (using the ranking of politicians on the “scorecards” of various activist groups), and we have a way to measure which politicians cite which think tanks, activist groups, public interest law firms, and other public policy organizations. The basic theory of this research is that this could tell us something meaningful about the media.

    Are you arguing that:

    (a) the basic theory is sound, but needs “tweaking,”
    (b) the theory is nonsense, but there might be some other way to measure media bias, or
    (c) it is theoretically impossible to measure media bias.

    I’d genuinely like to know.


    Comment by
    Daryl Cobranchi
    July 12th, 2006
    at 11:31 am

    (b)

    Their methodology was flawed (obviously).

    Groups have been scorecarding politicians forever. It’s pretty easy as their votes are public record. There is no corresponding “vote” for papers to be judged by. Probably the nearest one could come would be to monitor the editorial page endorsements. But those are so infrequent and so binary that the data would likely end up being equally binary (GOP/Dem).


    Comment by
    Scott Somerville
    July 12th, 2006
    at 11:39 am

    So, Daryl–are you saying that media bias is not measurable?


    Comment by
    sam
    July 12th, 2006
    at 1:07 pm

    Anything is measurable, but who cares about media bias? Are they lying to us? Are they spinning the news? Are they lying and/or spinning in a greater proportion than the politicians they discuss? In dictatorships there is no media bias. The ruler tells the media agencies what the news is, and they print it. How simple is that? How can you biased when you only ever report what the ruler says to say? And Bush never lies or misleads us as we all know.


    Comment by
    Daryl Cobranchi
    July 12th, 2006
    at 2:30 pm

    Scott:
    Are you arguing that:

    (a) the basic theory is sound, but needs “tweaking,”
    (b) the theory is nonsense, but there might be some other way to measure media bias, or
    (c) it is theoretically impossible to measure media bias.

    I’d genuinely like to know.

    Daryl:
    (b)

    Scott:
    So, Daryl–are you saying that media bias is not measurable?

    I thought my answer was clear enough. Their basic theory (i.e., that you can measure “bias” by merely counting how many times one paper mentions a particular think tank) was inherently flawed and beyond “tweaking.” That, of course, does not preclude some other method from being developed. I’m a scientist; we never say never.


    Comment by
    Scott Somerville
    July 12th, 2006
    at 3:14 pm

    I see. Here’s a quick counterargument…

    Let us suppose, for the sake of the argument, that this methodology measures SOMETHING. (We can call it “sociological horizons” or “six degrees of centering” or the “X factor” or whatever you like.)

    If we were to routinely measure and report the “x factor” of various media outlets, human nature suggests that shrewd media types will change their journalistic policies to take advantage of this new measurement. Media entities on the “right” that WANT to move closer to the “center” of the scale will take pains to cite more think tanks on the “left,” and vice versa.

    I argue that any effort to manipulate cites to “appear” to be closer to the center of the “x factor” scale will be virtually indistinguishable from what journalism schools say you are supposed to do in “fair and balanced” reporting. You’re supposed to cite sources on both sides of the argument. The “x factor” scale quantifies whether you actually do that or not.

    What I’ve seen in reporting on the abortion issue has been the OPPOSITE of fair and balanced. The “mainstream media” will cite NARAL or NOW for a pro-choice position, and then they’ll dig up some crackpot to make the counterargument. That LOOKS like balanced journalism, but it’s really just a “strawman” argument–which is a fallacy, not good journalism.

    Daryl, you assure me that the method is flawed, but I’m not persuaded yet. This research methodology will reveal such “strawman journalism” every time. The most direct way to manipulate the “x factor” is to be a MORE professional journalist. Call me crazy if you must, but that still looks like pretty good research to me.


    Comment by
    Scott Somerville
    July 12th, 2006
    at 5:10 pm

    Nance, when I get my new blog (scottsomerville.com is reserved, but I haven’t gotten it up and running yet) I’ll be happy to pick up some of these issues for more discussion.

    My point here was that it would be better journalism to cite NARAL and National Right to Life for their opposing views on abortion, rather than citing NARAL and “Kill a Doctor for Christ,” and this research methodology would pick that up. Likewise, it would be better to cite the National Academy of Sciences and the Discovery Institute for their opposing point of view on Intelligent Design, rather than the National Academy and the “Six Day Creation Crusaders.” (Remember, the Discovery Institute OPPOSED the Dover School District ID policy.)


    Comment by
    NMcV
    July 12th, 2006
    at 6:29 pm

    Scott Somerville said:

    What I’ve seen in reporting on the abortion issue has been the OPPOSITE of fair and balanced. The “mainstream media” will cite NARAL or NOW for a pro-choice position, and then they’ll dig up some crackpot to make the counterargument. That LOOKS like balanced journalism, but it’s really just a “strawman” argument–which is a fallacy, not good journalism.

    Gee, Scott, I don’t know why you’re complaining about this propaganda strategy. It’s your favorite, isn’t it? Just replace “abortion” with “home-school”, and so on…

    Or is it “different” when a lawyer does it, even though the goal is the same?


    Comment by
    Daryl Cobranchi
    July 12th, 2006
    at 8:01 pm

    Nance,

    Well said. I made a similar point (though not nearly as eloquently) in the other thread.