Sports: Study of N.B.A. Sees Racial Bias in Calling Fouls

[Guy Berger]

link

An academic study of the National Basketball Association, whose playoffs continue tonight, suggests that a racial bias found in other parts of American society has existed on the basketball court as well.

A coming paper by a University of Pennsylvania professor and a Cornell University graduate student says that, during the 13 seasons from 1991 through 2004, white referees called fouls at a greater rate against black players than against white players.

Justin Wolfers, an assistant professor of business and public policy at the Wharton School, and Joseph Price, a Cornell graduate student in economics, found a corresponding bias in which black officials called fouls more frequently against white players, though that tendency was not as strong. They went on to claim that the different rates at which fouls are called “is large enough that the probability of a team winning is noticeably affected by the racial composition of the refereeing crew assigned to the game.”

Guy

[Noj]

I've been watching the game for longer than that, and the only bias I've noticed is for whoever current the media darlings are. The star players get the star treatment. There's never been a racial bias as far as I've seen. Tell it to Chris Mihm and Sasha Vujacic, the foul magnets.

[rachel]

N.B.A. Commissioner David Stern said in a telephone interview that the league saw a draft copy of the paper last year, and was moved to do its own study this March using its own database of foul calls, which specifies which official called which foul.

The study seems, to me, to be flawed if they rely on box scores for their data. Box scores do not specify which referee makes each call. How did they come to their conclusion if they cannot determine which ref makes the call? And not to be flippant, but what about bi-racial players? Or non calls?

I agree with Noj. The bias is toward whatever player is the media darling. One just needs to have watched Michael Jordan play to know that. (especially in person--geez he always looked incredulous that he would be called for a foul...)

Edited May 3, 2007 by rachel

[PHILLYQ]

N.B.A. Commissioner David Stern said in a telephone interview that the league saw a draft copy of the paper last year, and was moved to do its own study this March using its own database of foul calls, which specifies which official called which foul.

The study seems, to me, to be flawed if they rely on box scores for their data. Box scores do not specify which referee makes each call. How did they come to their conclusion if they cannot determine which ref makes the call? And not to be flippant, but what about bi-racial players? Or non calls?

I agree with Noj. The bias is toward whatever player is the media darling. One just needs to have watched Michael Jordan play to know that. (especially in person--geez he always looked incredulous that he would be called for a foul...)

It seemed like every time Jordan drove the lane, if he didn't score a foul was called. Patrick Ewing did his bunny-hop step and never got called for that either. If Joe Blow did either it would be called the other way. I think Noj is 100% on the money.

[Guy Berger]

N.B.A. Commissioner David Stern said in a telephone interview that the league saw a draft copy of the paper last year, and was moved to do its own study this March using its own database of foul calls, which specifies which official called which foul.

The study seems, to me, to be flawed if they rely on box scores for their data. Box scores do not specify which referee makes each call. How did they come to their conclusion if they cannot determine which ref makes the call?

Presumably they used the racial breakdown of the ref team. I agree that this is not as perfect as using the racial identity of the ref, but I don't find it to be as problematic as others suggest.

And not to be flippant, but what about bi-racial players?

They explain their methodology in the NYT article.

Or non calls?

That's a good question! (Though I don't think it hurts their conclusions.)

Though the data obviously doesn't exist, in an ideal world we'd analyze how non-calls break down.

Guy

[Jim Alfredson]

Um... aren't there more black players in the NBA than white players? So therefor a white referree would by default be calling more fouls on black players than white?

[alocispepraluger102]

this is one of the most bullshit studies i have ever been aware of.

a fifteen year study?

the difference is something like .15 more per game

Edited May 3, 2007 by alocispepraluger102

[Guy Berger]

Um... aren't there more black players in the NBA than white players? So therefor a white referree would by default be calling more fouls on black players than white?

No - the conclusion is a given white player is less likely than an identical black player to get called for a foul by a white ref. From the article:

that players who were similar in all ways except skin color drew foul calls at a rate difference of up to 4 ½ percent depending on the racial composition of an N.B.A. game’s three-person referee crew.

and:

Mr. Wolfers and Mr. Price claim that these changes are enough to affect game outcomes. Their results suggested that for each additional black starter a team had, relative to its opponent, a team’s chance of winning would decline from a theoretical 50 percent to 49 percent and so on, a concept mirrored by the game evidence: the team with the greater share of playing time by black players during those 13 years won 48.6 percent of games — a difference of about two victories in an 82-game season.

Guy

Edited May 3, 2007 by Guy

[Guy Berger]

this is one of the most bullshit studies i have ever been aware of.

Why?

a fifteen year study?

the difference is something like .15 more per game

What difference?

Guy

[Dan Gould]

The study seems, to me, to be flawed if they rely on box scores for their data. Box scores do not specify which referee makes each call. How did they come to their conclusion if they cannot determine which ref makes the call?

Presumably they used the racial breakdown of the ref team. I agree that this is not as perfect as using the racial identity of the ref, but I don't find it to be as problematic as others suggest.

Why do you not find it problematic? As you well know, correlation is not causation, and without a racial categorization of the specific referee making a specific call, you don't even have correlation. What conclusion can be drawn? When there are two or more white referees, more fouls are called on black players?

Its a completely bogus study.

[J Larsen]

this is one of the most bullshit studies i have ever been aware of.

Why?

a fifteen year study?

the difference is something like .15 more per game

What difference?

Guy

I think he is referring to this:

“Across all of these specifications,” they write, “we find that black players receive around 0.12-0.20 more fouls per 48 minutes played (an increase of 2 ½-4 ½ percent) when the number of white referees officiating a game increases from zero to three.”

No matter how small the effect is, if it is statistically significant, then it is a real effect given the assumptions of the model.

[Guy Berger]

What conclusion can be drawn? When there are two or more white referees, more fouls are called on black players?

Bingo. That's the conclusion that can be drawn.

Its a completely bogus study.

Why?

BTW, the economists interviewed in the article don't question the methodology.

Guy

[J Larsen]

The study seems, to me, to be flawed if they rely on box scores for their data. Box scores do not specify which referee makes each call. How did they come to their conclusion if they cannot determine which ref makes the call?

Presumably they used the racial breakdown of the ref team. I agree that this is not as perfect as using the racial identity of the ref, but I don't find it to be as problematic as others suggest.

Why do you not find it problematic? As you well know, correlation is not causation, and without a racial categorization of the specific referee making a specific call, you don't even have correlation. What conclusion can be drawn? When there are two or more white referees, more fouls are called on black players?

Its a completely bogus study.

It would be quite an odd result if the more white refs you had on a team, the more likely the refs were to call fouls on blacks vs. whites, but it wasn't the whites making the extra calls. Not impossible, but less plausible than the result we'd all prefer to think was untrue.

That being said, I wouldn't take the results of the study at face value without getting some technical detail.

[Dan Gould]

What conclusion can be drawn? When there are two or more white referees, more fouls are called on black players?

Bingo. That's the conclusion that can be drawn.

Its a completely bogus study.

Why?

Because the only legitimate study would take the specific race of the referee making a specific call against a specific race of player. And according to the article, the NBA did that:

N.B.A. Commissioner David Stern said in a telephone interview that the league saw a draft copy of the paper last year, and was moved to do its own study this March using its own database of foul calls, which specifies which official called which foul.

“We think our cut at the data is more powerful, more robust, and demonstrates that there is no bias,” Mr. Stern said.

Given a choice between an inexact measurement that shows "bias" and an exact measurement that shows no "bias", which do you think is a better designed study?

This is a non-story. A flawed measure found bias. An unflawed measure found no bias. In short, there is no rational basis for the claim of bias.

Edited May 3, 2007 by Dan Gould

[J Larsen]

The NBA clearly wants to find no bias, and it is therefore not surprising that it finds none.

The academics are probably motivated to find a bias (investing a lot of time in a research project only to get a null result is sort of a bummer), and I'm therefore not surprised that they found one.

The question is how sensitive each result is to the assumptions of the underlying models and how reasonable their treatment of the data was. That the NBA refuses to release the data their study was run on raises an eyebrow (the proprietary personnel data argument is bullshit becuase they can code the data up in such a way that it retains its informational content but does not reveal exactly who was calling a foul on who at any given time).

Edited May 3, 2007 by J Larsen

[Noj]

I'd like to see these statistics cross-referenced with a personality profile of each player. Hot heads and obnoxious players/those who act poorly toward the refs are likely called for more fouls than calm, respectful players.

[Guy Berger]

I've been watching the game for longer than that, and the only bias I've noticed is for whoever current the media darlings are. The star players get the star treatment. There's never been a racial bias as far as I've seen. Tell it to Chris Mihm and Sasha Vujacic, the foul magnets.

Noj -- the study apparently controls for non-racial characteristics. In other words, if Mihm or Vujacic were black, they'd be called for even more fouls.

Guy

[J Larsen]

I'd like to see these statistics cross-referenced with a personality profile of each player. Hot heads and obnoxious players/those who act poorly toward the refs are likely called for more fouls than calm, respectful players.

But it seems unlikely that the more white refs you have, the more likely you are to have hot-headed obnoxious black players in the game or that the more black refs you have, the more likely you are to have calm, respectful white players in the game, which is the only way I can see your idea reversing the result.

Again, I'm not sold on either study - but it doesn't seem likely to me that this is what is driving the result.

Edited May 3, 2007 by J Larsen

[GA Russell]

It's been decades since I've watched an NBA game. I'm surprised to read that they call fouls now. Didn't used to.

Do they still ignore travelling?

[J Larsen]

Last I checked.

[Guy Berger]

What conclusion can be drawn? When there are two or more white referees, more fouls are called on black players?

Bingo. That's the conclusion that can be drawn.

Its a completely bogus study.

Why?

Because the only legitimate study would take the specific race of the referee making a specific call against a specific race of player. And according to the article, the NBA did that:

N.B.A. Commissioner David Stern said in a telephone interview that the league saw a draft copy of the paper last year, and was moved to do its own study this March using its own database of foul calls, which specifies which official called which foul.

“We think our cut at the data is more powerful, more robust, and demonstrates that there is no bias,” Mr. Stern said.

Given a choice between an inexact measurement that shows "bias" and an exact measurement that shows no "bias", which do you think is a better designed study?

The academic study has been publicly released and seems, at least on the surface, to have sound methodology. The NBA study has not -- we have no clue which (if any) econometric techniques they used.

correction: Apparently the three economists mentioned the article (Ayres, Katz, Berri) saw both sets of studies. Ayres and Berri are the only ones quoted in the article, and they seem more convinced by the academic study.

Edited May 3, 2007 by Guy

[Guy Berger]

I'd like to see these statistics cross-referenced with a personality profile of each player. Hot heads and obnoxious players/those who act poorly toward the refs are likely called for more fouls than calm, respectful players.

The study uses what are called "player effects" -- that is, a variable for each player. So unless players' personalities change from game to game, I don't think what you suggest is a problem for the study.

Guy

[Guy Berger]

The NBA clearly wants to find no bias, and it is therefore not surprising that it finds none.

The academics are probably motivated to find a bias (investing a lot of time in a research project only to get a null result is sort of a bummer), and I'm therefore not surprised that they found one.

The question is how sensitive each result is to the assumptions of the underlying models and how reasonable their treatment of the data was.

I think this a nice summary.

Guy

[J Larsen]

Hey Guy - do you know that paper that came out sometime in the last couple years that found that the results of academic studies are biased towards finding significance levels of just under 5%? They had histograms of the p values that showed a huge spike right at 4.9. It was pretty funny, and somewhat appropriate to the issue at hand.

[rachel]

Mr. Wolfers said that he and Mr. Price classified each N.B.A. player and referee as either black or not black by assessing photographs and speaking with an anonymous former referee, and then using that information to predict how an official would view the player

Ok... Jason Kidd. Black? or White? So we better ask the 'anonymous former referee'?

Seriously...