Popularity as a poor proxy for utility - Mitchell & Tetlock - 2017 - Article

Introduction

The implicit prejudice construct is a construct that has moved extremely fast in the recent years from psychology journals into other academic disciplines, newspaper editorials, courtrooms, boardrooms, and popular consciousness. This term appears for the first time in the PsycINFO database less than twenty years ago. A Google search for “implicit prejudice” between the years 1800 and 1990 would return only six hits, but in the years 1991-2006 it would return over 8400 hits. In Google Scholar, you would get 46 hits for the term “implicit prejudice” in articles published between 1800 and 1990. In the years after 1990, however, it would return 3700 hits. In 1998, an article about the implicit association test (IAT) was published. This is now the most popular method for studying implicit prejudice. The IAT article has been published more than 3700 times in PsycINFO, 3400 times in the Web of Science database and 7500 times in Google Scholar.

The authors of this article state that the psychological construct “implicit prejudice” is not a good construct. They state that even though it is a very popular construct, it is misunderstood and lacking in theory and in practical use. For example, scholars do not agree about how implicit prejudice is linked to other forms of prejudice. So, it is unclear if the different measures of the construct all measure the same construct. Also, the meaning of “implicit” is challenged by some researchers. When we look at the practical use of the term implicit prejudice, the results show that implicit measures of prejudice do not predict behavior better compared to traditional explicit measures of prejudice. In this article, they discuss how the psychological research behind the construct is inadequate to support the widespread use of the implicit prejudice. They explain by this by first discussing how the term was created and how psychologists marketed the ideas. Later, they discuss their view on how the construct should be renovated. They also discuss how such popular ideas within social psychology are hard to overcome.

The implicit prejudice

There are two eras in the history of the implicit prejudice construct. The first era is the pre-IAT era. In this era, psychologists mainly developed indirect measures of prejudice with the goal of overcoming response biases. They also mainly looked at automatic processes that lead to prejudice. The second era is the post-IAT era. This is the era in which the term became popular in public discussions and even in academic ones. In these discussions, people used the term implicit prejudice for other, widespread unconscious prejudices. Then, in 1998 an article was published in Psychology Today, which was stated that they found extraordinary results. By studying the unconscious, they found that everyone uses stereotypes, all the time without that people know it. In this article, they also stated that it is more difficult to avoid the negative effects of implicit prejudice compared to the negative effects of explicit prejudice. They author of the 1998 Psychology Today article states that: “Even though our internal censor successfully retains from expressing overly biased responses, there is still danger of leakage. This leakage shows itself in our non-verbal behavior, such as our expressions, our stance, how far we are from another person and how much eye contact we make.”

The IAT article introduced the idea that automatic stereotyping and unintentional prejudice operate in automatic and subconscious ways. The IAT is said to measure widespread implicit preferences that people may have for majority groups compared to minority groups. Even people who are from a minority group themselves, might show a preference for majority groups over minority groups. These preferences would be more predictive of behavior compared to explicit measures of prejudice.

The term “implicit prejudice” became very popular after 1998, because of the IAT article. The IAT Itself also became popular in the media and on the radio. For example, Greenwald discussed implicit bias and the behavioral effects of implicit bias on the radio. He said that: “People are not actually aware that they have this bias. But, it can still affect their behavior and act on them. It can produce unintended discrimination, because it can produce discomfort in interracial interactions.” The IAT was also discussed often in opinion websites, educational websites and in popular science writing.

The authors of this article, states that even though the IAT became very popular, it is not theoretically well grounded. Also, there have been no positive impacts of IAT research. The predictive validity of the implicit prejudice measure also has not shown to be higher than the predictive validity of explicit prejudice measures. The IAT also has not lead to any solutions for discrimination.

What Is implicit prejudice?

There are two important themes repeatedly found in tests the articles of the implicit prejudice. First, reaction-time based measures of prejudice, such as the lexical decision task (Wittenbrink, 1997) and the IAT, measure implicit prejudice, are meant to avoid the influence of normative pressures, such as the social desirability. The social desirability bias is the tendency of respondents to answer survey questions in a socially desirable way. Second, psychologists believe that they can tap into subconsciousness, decisions and behaviors. The implicit prejudice construct therefore also reflects views about attitudes and about the influence of automatic psychological processes and their influence on behavior. However, these themes are highly related to each other. This makes the authors of this article question: are the processes really implicit or is it only the measure that is an implicit measure of prejudice? This distinction between the meaning of “implicit” as in the processes or as in the measurement, is a highly discussed topic. The people who view the nature as the process, disagree about the nature of these processes. For example, IAT researchers state that implicit biases sometimes operate in subconscious ways. However, sometimes individuals are aware of their biases, but can simply not control them. The authors of this article state that it is also not clear if the two most reliable measures of prejudice, being the IAT and the “Affective Misattribution Procedure” (AMP), are implicit measures.

According to Greenwald & Nosek, the distinction between explicit and implicit prejudice is not empirically testable. Also, many automatic processes of prejudice and stereotyping, for example aversive racism, are included in the description of the implicit prejudice construct. However, the theory about aversive racism is different than the theory behind the implicit prejudice.

Another argument that the authors of this article give for their idea that the implicit prejudice construct need to be criticized, is that different implicit measures of implicit prejudice, produce different patterns of results. This is also true for similar measures. The measures also do not correlate highly.

The predictive validity of implicit prejudice measures

The IAT has face validity: this means that we think we know what it means when respondents say that they like one group more than another group. However, when the implicit prejudice is a measure of intergroup prejudice, it lacks face validity. The authors of this article state that for implicit measures of prejudice, predictive validity is crucial. Because, if what is measured by the IAT or another implicit measure reliably predicts behaviour that can lead to intergroup conflicts, then this would lead to better theoretical clarity about the underlying processes.

The measures of implicit prejudice are of an indirect nature. Many measures are reaction-time-based studies. In these studies, millisecond differences in response times, are seen as evidence for a prejudicial attitude. However, if these measures do not predict any judgments or behaviors, then this concept is meaningless.

The defenders of the implicit prejudice construct, posit as an argument for the construct that it predicts discriminatory behavior. One reviewer of Blindspot wrote when discussing the IAT: “The best indicator of the test’s validity is it’s prediction of behavior”. However, the authors of this article question whether the IAT predicts discriminative behavior better than explicit measures. This does not seem to be the case. Greenwald and colleagues (2009) reported in a meta-analysis that explicit measures predicted judgments, decisions and behavior better in seven of nine criterion domains. The IAT outperformed explicit measures in one area, but according to Greenwald and his colleagues this was due to poor performance of the explicit measures. The authors of this article also conducted an updated meta-analysis, based on that of Greenwald and colleagues. They found even lower estimates of the predictive validity for the IAT! The authors also found explicit measures to be more predictive than implicit measures in studies that looked at response times. This is in contrast with the idea that implicit construct resembles subtle, automatic behaviors compared to deliberate, controlled behaviors. The authors also found a lot of differences in the results across studies. They found that the variance was much greater than the estimated effect size.This means that the IAT will be a poor predictor of whether someone will act fairly or unfairly toward a minority group member. So, the popular ideas about IAT being better predictor of discrimination compared to explicit measures, is false. This is not true.

The score interpretation problem

According to IAT results, 75% of the American population is described as implicitly “racist”. According to the authors, there are no studies that provide criteria for the IAT scores to be nonexistent, low, moderate or high. So, the feedback given by the IAT is not based on the external validation. It is only based Cohen’s test of effect sizes. However, this test has the rule of thumb that effect sizes should be considered only on the base of practical meaning and significance. So, this is a noncorrect use of the Cohen’s effect size rule of thumb. According to the authors, if the IAT creators differently at their results, they could give the participants very different feedback. This has been shown earlier: the IAT researchers changed their criteria for the extremity of their scores. This lead to that the percentage of people showing strong anti-black bias on the IAT decreased from 48% to 27%. This change was not due to societal shifts or due to studies. It was solely due to the researchers’ change in definitions.

The authors state that because of this freedom of researchers to make important conclusions, it can be used in a negative way and lead to mischief. The conclusion of the authors is that the scores of the IAT have no clear meaning. This means that when one person scores higher on the IAT compared to another person, this does not mean that this person is more likely to express bias outside the testing context

The implicit sexism puzzle

Research has found that men usually do not exhibit sexism, while women do show pro-female implicit attitudes. This finding is not in line with the common findings in IAT. In the IAT, the findings are that the historically advantaged groups, men, are favored by both the members of this group and the disadvantaged groups. Therefore, the IAT is often used in explanations of female victimization. Because of this, new findings about sexism should be taken into account, but these findings have received little attention in the literature about the implicit prejudice construct and the IAT. The IAT researchers have responded to this finding by showing that men are more often associated with math and science and with limited leadership qualities. So, an IAT is often used to asses whether men or women are more often linked to science and math, but not to asses implicit attitudes toward men and women. According to the authors of this article, this focus on implicit gender stereotypes, is problematic because of practical and theoretical reasons. First, implicit measures of these stereotypes are not predictive of discriminatory behavior. Second, only a few implicit gender stereotypes have been examined. For example, research has not looked at whether traits of good managers such as cooperativeness, fairness and integrity are more strongly associated with women than men. So, the IAT does not measure all the stereotypes that there are. Also, there is no reason to believe that these implicit gender stereotypes are true for women for all positions.

Thirdly, there also seems to be a problem between implicit attitudes and automatic semantic associations. If prejudice and discrimination depend on how feelings and beliefs interact, then an interaction would be needed at the implicit level too. The authors also state that a contextualized model is needed, in which it is clear when and where each component of implicit prejudice will be predominant and in what behavioral form.

Bias is everywhere

According to Banaji and Greenwald, implicit attitudes are just evaluative associations of varying strengths with attitude objects. It does not matter if these objects are products, places or people and it also does not matter whether the source of these associations are cultural information or personal experiences. In this view, implicit prejudice is just an evaluative knowledge about different groups. Some evaluations are more negative than others.

The subjective judgment problem

One solution that is proposed to prevent the influence of implicit bias, is to objectify judgment and decision-making processes. This means that there should be only objective measures of performance. However, the articles positing this solution do not take into account the findings that these subjective evaluation criteria are not associated with discrimination against women and minorities.

The contributions of the IAT

The only positive of the IAT, according to the authors of this article, is that humans have implicit biases, at high prevalence rates. But, the bias categories of the IAT have no external validity. The IAT scores are only a reliable predictor of future IAT scores. Also, even though there has been a lot of research conducted into the implicit prejudice and the IAT, there is still a lot of confusion about the nature of implicit prejudice. In the practical use, claims and conclusions often depend on the interpreter of the results. The authors criticize the inventors of the IAT and state that they developed a test that reliably produces statistically significant results. This makes it easier for researchers to use the IAT for their own purposes and. Also, the more researchers publish results based on the IAT, the higher the motivation for justification of the tool and its results.

Why does the implicit prejudice still persist?

The authors of this article state that the IAT is very popular among psychologists, because it makes it easy to produce statistically significant effects. They state that even though there is a lot of criticism on the IAT, it is still very popular and this is due to ideological ideas, publication bias and the lack of clear, consensual score-keeping measures within the social psychology. The publication bias is about that it is favored to publish significant experimental results. The IAT studies are therefore easier to publish and popular among psychologists. According to the authors, the research domain of implicit prejudice is a good domain for experimenting with extraordinary science. They state that the rules for ‘success’ should be set ex ante, rather than allowing test-takers to post-hoc modify any pattern of results according to their preferred theories.