Testing for universality

METHOD

PROS

CONS

Randomly select as many countries as possible.

This is the ideal option.

This is not pragmatic.

Select two maximally different cultures.

Finding no difference is strong evidence for universality.

Finding a difference will be difficult to interpret.

Testing for cultural variation

Randomly select as many countries as possible that span the entire range of a cultural value of interest.

This is the ideal option.

This is not pragmatic.

Match two cultural groups in all non-cultural variables so that the only difference is in the cultural value of interest.

This can give initial evidence for cultural variation.

This evidence will not be definitive.

METHOD

PROBLEMS

SOLUTIONS

Construct equivalence: similarity of construct across cultures.

This might lead to pronounced differences that reflect a different cultural value.

Measure different aspects of the construct and consult experts from the culture of interest.

Methodological equivalence:
equality in familiarity with stimulus material and response procedure.

The differences in interpretations of the material and differences in familiarity with the response procedure.

The solution is to adapt the material.

Linguistic equivalence: equality in language in terms of meaning.

The untranslatable words and culture-specific words.

Back-translation, a bilingual investigator or collaborator, studying bilingual participants and the avoidance of abstract items.

BIAS

PREVALENCE

SOLUTION

Moderacy bias; the tendency to select the middle-option.

This is most prevalent in East Asians.

The avoidance of scales with middle response options, using a yes or no format and standardizing scores.

Extremity bias; the tendency to select cut-off points.

This is most prevalent in Hispanics and African-Americans.

Acquiescence bias; the tendency to agree with everything.

This is most prevalent in East Asians.

Using contraindicative items, standardizing scores and specifying the context.

Reference group effect; the tendency to compare with one’s own group.

This occurs in all cultures.

Using concrete scenarios, using concrete response choices to a concrete scenario, avoid quantifiers or use behavioural or physiological measures (i.e. avoidance of self-report).

METHOD

PROS

CONS

Survey

This is useful at the beginning of a project if combined with an experimental manipulation or if a large sample of cultures is included.

This is bias-prone as it employs self-report and causal claims are not possible without an experimental manipulation.

Field experiment

This has high ecological validity.

There are extraneous variables that cannot be controlled.

Cultural neuroscience

This includes high impact studies and allows conclusions about the biological bases of cultural variation.

This is expensive, mostly uses small samples and it is not always clear what a difference means.

Cultural priming

This is usually applied to one culture to address equivalence issues and allows causal inference for the effect of a cultural mindset.

This investigates a temporary mindset whilst culture is chronic.

Cultural products

The cultural products directly reflect the most prevalent cultural ideals and messages, it captures culture in the world rather than in the mind and it is already available data.

This requires trained coders and has a limited scope. It is ideally supplemented with evidence for the effect of exposure to cultural products.