Investigating Sports Commentator Bias within a Large Corpus of American Football Broadcasts

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 6355–6361, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 6355 Investigating Sports Commentator Bias within a Large Corpus of American Football Broadcasts Jack Merullo⋆♠ Luke Yeh⋆♠ Abram Handler♠ Alvin Grissom II♣ Brendan O’Connor♠ Mohit Iyyer♠ University of Massachusetts Amherst♠ Ursinus College♣ {jmerullo,lyeh,ahandler,miyyer,brenocon}@umass.edu agrissom@ursinus.edu Abstract Sports broadcasters inject drama into play- by-play commentary by building team and player narratives through subjective analyses and anecdotes. Prior studies based on small datasets and manual coding show that such theatrics evince commentator bias in sports broadcasts. To examine this phenomenon, we assemble FOOTBALL, which contains 1,455 broadcast transcripts from American football games across six decades that are automat- ically annotated with 250K player mentions and linked with racial metadata. We identify major confounding factors for researchers ex- amining racial bias in FOOTBALL, and perform a computational analysis that supports conclu- sions from prior social science studies. 1 Introduction Sports broadcasts are major events in contempo- rary popular culture: televised American football (henceforth “football”) games regularly draw tens of millions of viewers (Palotta, 2019). Such broad- casts feature live sports commentators who weave the game’s mechanical details into a broader, more subjective narrative. Previous work suggests that this form of storytelling exhibits racial bias: non- white players are less frequently praised for good plays (Rainville and McCormick, 1977), while white players are more often credited with “in- telligence” (Bruce, 2004; Billings, 2004). How- ever, such prior scholarship forms conclusions from small datasets1 and subjective manual cod- ing of race-specific language. We revisit this prior work using large-scale computational analysis. From YouTube, we col- lect broadcast football transcripts and identify mentions of players, which we link to metadata ⋆Authors contributed equally. 1 Rainville and McCormick (1977), for example, study only 16 games. Player Race Mention text Baker Mayfield white “Mayfield the ultimate com- petitor he’s tough he’s scrappy” Jesse James white “this is a guy . . . does nothing but work brings his lunch pail” Manny Lawson nonwhite “good specs for that defensive end freakish athletic ability” B.J. Daniels nonwhite “that otherworldly athleticism he has saw it with Michael Vick” Table 1: Example mentions from FOOTBALL that high- light racial bias in commentator sentiment patterns. about each player’s race and position. Our re- sulting FOOTBALL dataset contains over 1,400 games spanning six decades, automatically an- notated with ∼250K player mentions (Table 1). Analysis of FOOTBALL identifies two confounding factors for research on racial bias: (1) the racial composition of many positions is very skewed (e.g., only ∼5% of running backs are white), and (2) many mentions of players describe only their actions on the field (not player attributes). We experiment with an additive log-linear model for teasing apart these confounds. We also confirm prior social science studies on racial bias in nam- ing patterns and sentiment. Finally, we publicly release FOOTBALL,2 the first large-scale sports commentary corpus annotated with …

Content truncated to 500 words