Statistical discrimination (economics)

Statistical discrimination is an economic theory of racial or gender inequality based on stereotypes. According to this theory, inequality may exist and persist between demographic groups even when economic agents (consumers, workers, employers, etc.) are rational and non-prejudiced. This type of preferential treatment is labeled "statistical" because stereotypes may be based on the discriminated group's average behavior.

The theory was pioneered by Kenneth Arrow and Edmund Phelps The theory posits that in the absence of direct information about a certain fact of ability, a decision maker would substitute group averages. For instance, labor market discrimination may exist because employers don't know with certainty workers' ability, therefore they may resort to basing employment decisions on the workers' visible features, such as group identity, as long as these features correlate with some desirable but more difficult to measure trait. The result is that atypical individuals from the disadvantaged group suffer unfair discrimination. This type of discrimination can result in a self-reinforcing vicious circle over time, as the atypical individuals from the discriminated group are discouraged from participating in the market, or improving their skills as their (average) return on investment (education etc.) is less than for the non-discriminated group.

A related form of (theorized) statistical discrimination is based on group variances, assuming equal averages. For discrimination to occur in this scenario, the decision maker needs to be risk averse; such a decision maker will prefer the group with the lower variance. Even assuming two theoretically identical group distributions (in all respects, including average and variance), a risk averse decision maker will prefer the group for which a measurement (test) exists that minimizes the error term. For example, if two groups, A and B, have theoretically identical distributions of test scores well above the average for the entire population, but group A's estimate is considered more reliable because a large amount of data may be available for group A in comparison to group B, then if two people, one from A and one from B apply for a job, using statistical discrimination, A is hired, because it is perceived that his group score is a more reliable estimate, so a risk-averse decision maker sees group B's group score as more likely to be luck. Conversely, if the two groups are below average, B is hired, because group A's negative score is believed to be a better estimate.

Statistical discrimination is often used and tolerated, for example, when older people are charged more for life insurance, or when a college diploma is required for a job (because it is believed that college graduates perform, on average, better). Some well-documented instances of statistical discrimination for involuntary group membership also do exist and are tolerated. For example, many countries allow auto insurance companies to charge men and women with identical driving records different rates (or factor in gender when deciding whether to deny coverage). The same society may not tolerate statistical discrimination when it is applied to protected groups. For example, it has been suggested that home mortgage lending discrimination against African Americans, which is illegal in the United States, may be partly caused by statistical discrimination.

Market forces are expected to penalize some forms of statistical discrimination; for example, a company capable and willing to test its job applicants on relevant metrics is expected to do better than one that relies only on group averages for employment decisions. However, this assumption does not take into account the economic cost of testing itself, which may not be feasible in some scenarios like predicting the future likelihood that an employee will quit for personal reasons.

Further reading

• Arrow, K. J. (1973), "The Theory of Discrimination", in O. Ashenfelter and A. Rees (eds.), Discrimination in Labor Markets, Princeton, NJ: Princeton University Press.
• Coate, Steven and Glenn Loury, 1993, Will affirmative-action policies eliminate negative stereotypes?,The American Economic Review, 1220--1240.
• Glenn Loury, The Anatomy of Racial Inequality, Princeton University Press. Informally illustrates the theory in the context of United States' racial differences.