Statistically, Female Journalists are Less Biased than their Male Peers
Introduction
We often talk about how the news is biased, but we haven’t really dissected the term “news.” One level down, we get to news organizations: CNN, Fox, NYT, WaPo, etc. Then we get to topics: what is this article about? Digging even further, we analyze the journalists themselves. At least in the context of news bias, no one has stratified news articles by the traits of the human behind them, e.g. gender.
Last summer, I developed BLUFFNet, a deep learning model which identifies biased news sentences, and in turn, biased news articles. What’s interesting is that the algorithm proved to have two applications: real world programs (the BLUFFNet Chrome Extension) and statistical analyses (these articles). After deploying BLUFFNet to analyze nearly 500,000 news articles, I was able to study news bias in depth.
If you’re interested in the statistical methodology (in summary, a t-test), scroll down. Otherwise, let’s talk about why this might be true.
Why?
It’s important to note that the articles scraped for this study were on Google News and the CNN and Fox News RSS feeds, meaning they had to be somewhat popular in the first place. What this might indicate is that highly biased articles written by women are less popular than objective ones. If this were true, that would mean my study omits a large pool of biased news articles written by women. However, most people don’t bother to look at the name of the author who wrote the article they’re reading, so the author’s gender most likely wouldn’t influence the popularity of their article.
A more plausible theory is that women are more comfortable with writing objectively than men are, reporting more of the facts than their own opinion. Or, maybe female journalists bring their biased article to their newspaper’s editor, who rejects it based on his/her own gender bias. Perhaps CNN and Fox publish fewer biased articles by women because their editors judge them for bias more harshly than they do men. This would make sense given the subtle forms of gender discrimination visible in workplaces today.
Personally, I think the truth is some combination of the previous two theories. Much like gender discrimination in 2020, the difference between female and male authorial bias isn’t outrageous, rather it exists subtly and can be difficult to notice on the surface.
Methodology
In order to examine the correlation between gender and news bias, I created an index of authors and their articles’ mean bias rating. Since the location of the author’s name on an article is highly specific to the site, I restricted my analyses to only CNN and Fox News articles. Furthermore, I restricted the articles to only those that are not Editorials or Op-Eds, giving us insight into the real issue of biased news articles: those masked as supposedly objective reporting. In order to have a reasonable estimate of authorial bias, I removed all authors from the index who had fewer than 10 ranked articles (my p-value is lower than 0.01 even when the minimum number of articles is 30, when the Central Limit Theorem kicks in). The full spreadsheet can be found here.
I then used the genderize.io API to determine the genders of the authors and ran a t-test for difference in means between the average authorial biases of men and women. The null hypothesis is that men write fewer biased news articles than women. Our p-value here is 0.0018 (n=95 journalists), statistically significant at the 0.01 level. Therefore, we can reject the null hypothesis; i.e. statistically, we can say that female journalists writing for CNN and Fox News tend to write news articles with lower bias scores than do male journalists.
Of course, the sample isn’t completely random. Here are the restrictions of this conclusion:
- CNN and Fox News journalists
- Journalists who have been publishing articles for the past 5 months
- Journalists whose articles appear in Google News and the CNN and Fox News RSS feeds
Still, these results are pretty interesting, and I want to expand the study to a much larger set of authors, ideally something like 1000. Currently, I’m working on broadening the analysis to other news sources, like the New York Times. I’ll update this article when that is complete.
