Research methods in psychology by B. Morling (third edition) – Chapter 14 summary

TO BE IMPORTANT, A STUDY MUST BE REPLICATED
Replication gives a study credibility, and it is a crucial part of the scientific process. There are several types of replication:

1. Direct replication
   Researchers repeat an original study as closely as they can to see whether the effect is the same in the newly collected data.
2. Conceptual replication
   Researchers explore the same research question, but use different procedures. In this replication, the conceptual variables are the same, but the operationalizations are not.
3. Replication-plus-extension
   Researchers replicate their original experiment and add variables to test additional questions.

The replication crisis refers to the fact that a lot of psychological studies don’t share the same results when they’re replicated. Replication studies might fail, because some original effect are contextually sensitive and when the replication context is too different, the replication is more likely to fail.

HARK-ing is hypothesising after the results are known. P-hacking is using more individuals and removing certain outliers if the results of the first experiment were not significant. The goal of this to find a p-value of under 0.05. There are three changes made to psychological research in order to increase the replication rate:

1. Open science
   Sharing one’s data and materials freely.
2. Larger sample sizes
   Most studies and replications require much larger sample sizes nowadays.
3. Preregistration
   Preregistering the study’s methods, hypothesis and statistical analyses online, in advance of data collection. This can be useful for publication in journals.

In order to increase the replication rate in journals, journals now all devote a section to replicated articles. Meta-analysis is a way of mathematically averaging the results of all the studies that have tested the same variables to see what conclusion the whole body of evidence supports. This makes use of both published and unpublished articles. The file drawer problem refers to the idea that a meta-analysis might be overestimating the true size of an effect because null effects, or even opposite effects, have not been included in the collection of the process (unpublished studies are less likely to make it into a meta-analysis).

TO BE IMPORTANT, MUST A STUDY HAVE EXTERNAL VALIDITY?
The manner in which the participants are recruited is more important than the number of participants for getting external validity. Ecological validity is the generalizability of an experiment to real-world settings.

Researchers in the theory-testing mode are usually designing correlational or experimental research to investigate support for a theory. When investigating support for a theory, the generalizability is not always necessary (e.g: if a theory is false in one sample, it should be false in all samples). Researchers in the generalization mode want to generalize the findings from the sample in a previous study to a larger population. Frequency claims are always in the generalization mode and association and causal claims are usually in theory-testing mode, but can be in generalization mode. Many lab experiments are high in experimental realism, it resembles the real world closely.

Cultural psychology focusses on how cultural contexts shape the way a person thinks, feels and behaves. They try to generalize findings to more cultures. Studies with a field setting have an advantage for generalizing to real-world settings, because the research has already been conducted in a real world setting.