EXPERIENCE
Experience is not a reliable source of information, because it has no comparison group. A comparison group in research is a group which isn’t affected by the controlled independent variable, so it is possible to really determine whether the independent variable has the effect people think it has.

E.g: Doctors used to take blood from an ill person, because they believed that it cured the illness. Some people recovered and they concluded that they recovered because they bled the patients. This is based on experience, they have experiences that some patients recovered, but they did not have a comparison group, so they had no way of knowing that the recovery was because of bleeding the patient. To make sure that it had this effect, they should have had a group with people who were ill, but were not bled, to see what would have happened.

When we are using personal experience to determine whether something works or not, we don’t have a comparison group as well. “My knee feels better with this tape”, but you don’t know how it would’ve felt if you didn’t use that tape. There is no comparison group, so it is not possible to give a conclusive answer, based on empirical evidence.

In real-world situation situations, there are several possible explanations for an outcome. In research, these alternative explanations are called confounds. Experience is confounded, because you do not know the cause of an effect, although you might think you do. When you use tape to lessen the pain in your knee, you don’t know whether the tape caused the pain to diminish. A researcher can see the situation from outside, but you can only see one condition and all you have is your experience.

Behavioural research is probabilistic. This means that it’s findings are not expected to explain all cases all the time. The conclusions of research are meant to explain a certain proportion of the cases. The two big problems with using experience as a source of information is that there is no comparison group and that experience is confounded.

INTUITION
People use their intuition to make decisions, although it is not a reliable source of information, because intuition is biased. There are ways our intuition is biased:

1. Good Story bias
   People tend to believe a good story, but this doesn’t mean that is necessarily correct.
2. Availability Heuristic
   Things that come to mind easily tend to guide our thinking. (e.g: Aeroflot is a bad airplane company, because the bad reports about Aeroflot come to mind easier than the good stories about Aeroflot) The availability heuristic occurs because sometimes things stand out more. (e.g: shark attacks stand out more than natural deaths, which causes us to believe that shark attacks are common)
3. Present/Present bias
   This bias is the name for our failure to consider appropriate comparison groups. In this case there are comparison groups available, but you fail

Construct validity refers to how well a study’s variables are measured or manipulated. There are three common types of measurement: self-report, observational and physiological. The conceptual definition, or construct, is the researcher’s definition of the variable in question on a theoretical level. The operational definition represents a researcher’s specific decision about how to measure of manipulate the conceptual variable.

SELF-REPORT
A self-report measure operationalizes a variable by reporting people’s answers to questions about themselves in a questionnaire or an interview. In research on children, self-reports may be replaced with parent reports or teacher reports.     

The problems with self-reports are the demand characteristics: a participant wants to be  a ‘good’ participant. People their self-perception is not always correct and the social desirability: people want to give a good impression about themselves.

OBSERVATIONAL MEASURES
An observational measure is sometimes also called a behavioural measure and operationalizes a variable by recording observable behaviour or physical traces of behaviour.

The problems with observational measures are:

1. Primacy effect
   The first observation sets the tone for the rest of the observations. (e.g: the first rated essay is very good, so the others that are not that good are automatically rated worse than they otherwise would have been rated)
2. Recency effect
   The last observation will be remembered best. (e.g: the last person at a job interview will be remembered the best, because that person was the last)
3. Halo effect
   A good rating on one dimension will influence the ratings on other dimensions. (e.g: if a person is friendly and that is rated first, then he will be more likely to receive higher ratings on other dimensions as well)

PHYSIOLOGICAL MEASURES
A physiological measure operationalizes a variable by recording biological data. The problem with physiological measurement is that not everything can be measured with biological data (at least not yet).

SCALES OF MEASUREMENT
All variables must have at least two levels. The levels of operational variables can be coded using different scales of measurement. 

1. Categorical variables (nominal variables).
   This are categories in which the variable fit. (e.g: sex, species)
2. Quantitative variables
   These variables are coded with meaningful numbers. (e.g: height, weight)

There are three kinds of quantitative variables.

1. Ordinal scale
   A ranking. (e.g: top 10 best-selling books) The distance between the subsequent numerals might not be equal.
2. Interval scale
   The interval between two ranked numbers means the exact same thing. The number ‘0’ doesn’t mean none. (e.g: the difference between IQ 105 and 110 is 5, so is the difference between IQ 110 and 115. The interval is the same)
3. Ratio scale
   There are equal intervals and ‘0’ truly means none. (e.g: a knowledge test with amount of questions correct)

RELIABILITY
Reliability refers to how consistent the results of a measure are.

Experiments are the only way to investigate causal issues.

EXPERIMENTAL VARIABLES
An experiment means that the researchers manipulated at least one variable and measures another. A manipulated variable is a variable that is controlled. Measured variables take the forms of records of behaviour or attitudes. The manipulated variable is the independent variable. The conditions are the different levels of the independent variable. The measured variable is the dependent variable. Control variables are variables that are also controlled. These variables are controlled by holding all other factors constant. Any variable that an experimenter holds constant on purpose is called a control variable.

WHY EXPERIMENTS SUPPORT CAUSAL CLAIMS
There are three rules for something to be causal:

1. Covariance
2. Temporal precedence
3. Internal validity

If independent variables did not vary, a study could not establish covariance, because you need a comparison group to establish covariance. It is impossible to establish internal validity if there are confounds, or alternative explanations. A design confound is an experimenter’s mistake in designing the independent variable. It is a second variable that happens to vary systematically along with the intended independent variable. Something is only a design confound if it shows systematic variability with the independent variable. It would not be a design confound if it shows unsystematic variability. If individual differences are distributed evenly in both groups, the are not a confound.

Selection effects are effects that are the result of two groups being systematically different from those in the other. This can also happen when the experimenters let participants choose in which group they want to be. The selection effects can be avoided by using random assignment, when assigning people to the conditions. Selection effects can also be avoided by using matched groups.

INDEPENDENT GROUP DESIGNS
In an independent group design both groups of participants are placed into different levels of the independent variable. This type of design is also called a between-subjects design or between-groups design. In a within-groups design or within-subjects design, there is only one group of participants and each person is pretended with all levels of the independent variable.

In the posttest-only design, also known as equivalent groups, participants are randomly assigned to independent variable groups. In a pretest/posttest design, participants are randomly assigned to at least two different groups and are tested on the key dependent variable twice, before and after exposure to the independent variable.

WITHIN-GROUPS DESIGNS
There are two basic types of within-groups design:

1. Repeated-measures design
   In this design participants are measured on the dependent variable every time they are exposed to another level of the independent variable.
2. Concurrent-measures design
   In this design participants are exposed to all the levels of an independent variable at roughly the same time.

The main advantage of a within-group design is that it ensures that participants in the two groups will be equivalent. The term

EXPERIMENTS WITH TWO INDEPENDENT VARIABLES CAN SHOW INTERACTIONS
Experiments with more than one independent variable allows researchers to look for an interaction effect. This is an effect where the effect of the original independent variable depends on the level of another independent variable. If the two lines of the independent variables cross, there is a crossover interaction, also known as “it depends”. If the lines are not parallel, there is an interaction and if the lines are parallel, there is no interaction. A spreading interaction occurs when the two lines spread out and can be labelled as an “only when..” interaction. An interaction is a difference in differences

FACTORIAL DESIGNS STUDY TWO INDEPENDENT VARIABLES
Testing for interactions is done with factorial designs. A factorial design is one in which there are two or more independent variables. In a factorial design, researchers study each possible combination of the independent variables. A participant variable is a variable whose levels are selected, but cannot be manipulated (e.g: age, the level for this variable can be selected, but not manipulated). Using factorial designs to test limits is called testing for moderators and it is a way to test the external validity of an experiment. Factorial designs can also test theories and hypotheses.

INTERPRETING FACTORIAL RESULTS: MAIN EFFECTS AND INTERACTONS
Researchers test each independent variable to look for main effects, the overall effect of one independent variable on another independent variable. Marginal means are the arithmetic means for each level of an independent variable, averaging over levels of the other independent variable. The main effect is not the most important effect, but the overall effect of one independent variable on another independent variable. The interaction itself is the most important effect. In a factorial design with two independent variables, the first to results obtained are the main effects for each independent variable. The third result is the interaction effect.

FACTORIAL VARIATIONS
In a mixed factorial design, one variable is manipulated as independent groups and the other is manipulated as within-groups (e.g: age and driving while on the phone. Age is independent groups and driving while on the phone is within-groups). When plotting a three-way factorial design and you want to check for three-way-interactions, you have to look for differences between the two states. If the lines are the same for both states in the three-way interaction, then there is a two-way interaction, but not a three-way interaction (unless the lines are parallel).

IDENTIFYING FACTORIAL DESIGNS IN YOUR READING
When looking for factorial designs in research articles it is important to look at the method part of the research description. When looking for factorial designs in regular articles it is important to look for the phrases it depends and only when.

TO BE IMPORTANT, A STUDY MUST BE REPLICATED
Replication gives a study credibility, and it is a crucial part of the scientific process. There are several types of replication:

1. Direct replication
   Researchers repeat an original study as closely as they can to see whether the effect is the same in the newly collected data.
2. Conceptual replication
   Researchers explore the same research question, but use different procedures. In this replication, the conceptual variables are the same, but the operationalizations are not.
3. Replication-plus-extension
   Researchers replicate their original experiment and add variables to test additional questions.

The replication crisis refers to the fact that a lot of psychological studies don’t share the same results when they’re replicated. Replication studies might fail, because some original effect are contextually sensitive and when the replication context is too different, the replication is more likely to fail.

HARK-ing is hypothesising after the results are known. P-hacking is using more individuals and removing certain outliers if the results of the first experiment were not significant. The goal of this to find a p-value of under 0.05. There are three changes made to psychological research in order to increase the replication rate:

1. Open science
   Sharing one’s data and materials freely.
2. Larger sample sizes
   Most studies and replications require much larger sample sizes nowadays.
3. Preregistration
   Preregistering the study’s methods, hypothesis and statistical analyses online, in advance of data collection. This can be useful for publication in journals.

In order to increase the replication rate in journals, journals now all devote a section to replicated articles. Meta-analysis is a way of mathematically averaging the results of all the studies that have tested the same variables to see what conclusion the whole body of evidence supports. This makes use of both published and unpublished articles. The file drawer problem refers to the idea that a meta-analysis might be overestimating the true size of an effect because null effects, or even opposite effects, have not been included in the collection of the process (unpublished studies are less likely to make it into a meta-analysis).

TO BE IMPORTANT, MUST A STUDY HAVE EXTERNAL VALIDITY?
The manner in which the participants are recruited is more important than the number of participants for getting external validity. Ecological validity is the generalizability of an experiment to real-world settings.

Researchers in the theory-testing mode are usually designing correlational or experimental research to investigate support for a theory. When investigating support for a theory, the generalizability is not always necessary (e.g: if a theory is false in one sample, it should be false in all samples). Researchers in the generalization mode want to generalize the findings from the sample in a previous study to a larger population. Frequency claims are always in the generalization mode and association and causal claims are usually in theory-testing mode, but can be in generalization mode. Many