BulletPointsummary for Research Methods in Psychology: Evaluating a World of Information by Morling - 3rd edition

What is the psychological way of thinking? - Bulletpoints 1

• Psychologists are scientists and psychology is based on doing research.

• Psychologists can be producers or consumers of research. Producers of research conduct research and make questionnaires, usually at universities, consumers of research read scientific articles and apply the theories in their work as therapists, advisors and teachers.

• Some psychologists are both producers and consumers.

• For therapists it’s important to follow evidence-based treatment. Those are treatments based on research.

• The theory-data circle means that scientists gather data to test, change and update their theories.

• A theory has claims about the relationship between variables. Theories lead to specific hypotheses.

• A hypothesis can be seen as a prediction. It says something about what the scientists expect to observe, if their theory is correct.

• Data can be seen as a set of observations. Data can support a theory or undermine it.

• Good theories are supported by data, are falsifiable and parsimonious (if two theories explain the data equally well, but one is simpler than the other, one must choose the simple theory).

• Scientists are empiricists and they observe the world in a systematic way. They test theories with their studies and they apply their theories on the found data.

• Scientists approach applied research (problems from daily life) and basic research (research that contributes to our general knowledge) in an empiric way.

• Scientists go further with research: when they find an effect, they set up new studies to find out why, when and for whom the effect can be found.

• Scientists tell about their studies in scientific journals and in the media.

What are sources of information in psychological research? - Bulletpoints 2

• Conclusions based on experiences or intuitions are usually not reliable. One of the reasons for that is that experiences don’t have a comparison group.

• In order to draw conclusions about certain treatments or effects, one must compare groups with each other. You have to compare the treatment/recovered group, the treatment/unrecovered group, the untreated/recovered group and the untreated/unrecovered group with each other.

• In everyday life there are many explanations for an outcome. In research these alternative explanations are called confounds.

• A confound occurs when you think that one thing caused a certain outcome, but other things changed as well and it is not sure what the actual cause was of the outcome.

• In everyday life it is hard to isolate variables. In research it is possible to control for variables and to look for one variable at a time.

• Conclusions based on intuitions are not reliable. That is because most people don’t think in a scientific way and they therefore have a distorted vision of the reality.

• The availability heuristic is a cognitive bias and it means that the things we can easily imagine, steer our thoughts. Usually those are thoughts that are very vivid or about things that happened recently.

• The present/present bias occurs if you only look at things that are present and not at things that are absent.

• Confirmatory hypothesis testing is not a scientific way of conducting research. Questions that support a hypothesis are asked, but questions that could contradict a hypothesis are not asked.

• The bias blind spot is the belief that one will not fall prey to a bias.

• Most psychologists publish their work in three different sources: scientific journals, book chapters and whole books.

• Empiricist articles rapport about the results of a study for the first time. These articles tell something about the used methods, the statistical tests that have been used and the results of a research.

• A review article is a summary about many studies on one topic. An edited book consist of a couple of chapters on the same topic and every chapter is written by another writer.

• Psychologists can tell about their research in a complete book, but they usually don’t do that.

What are the interrogation tools for consumers? - Bulletpoints 3

• A variable varies, so it has to have at least two levels.

• A measured variable is a variable for which the values are observed and noted (for abstract variables questionnaires are used). A manipulated variable is a variable a scientist can influence.

• A claim is an argument someone makes. Claims should be based on research.

• There are three different types of claims: frequency claims, association claims and causal claims.

• Frequency claims tell something about the amount of a variable, expressed in a number. These claims usually tell something about how often something occurs and they are always about one variable.

• Association claims argue that a certain level of one variable is associated with a certain level of another variable. Association claims are about at least two variables and they are measured, not manipulated.

• Causal claims argue that one variable is responsible for the other. These claims always start with an association, but they go further.

• To go from association to causality, a research should meet three criteria: covariance (correlation between two variables), temporal precedence (first the causal variable, then the outcome variable) and internal validity (there is no third variable that influences the two variables).

• In order to evaluate claims, one must look at construct validity, external validity and statistical validity.

Construct validity is about how well a study has measured or manipulated a variable and external validity is about the generalisability (how well does the sample represent the population?). statistical validity looks at how valid the statistical conclusions are.

What are the ethical guidelines for psychological research? - Bulletpoints 4

• In many ethical systems it is determined that researchers need to act according to the principles of respect, beneficence and justice.

• The principle of respect means that participants need to know what kind of questions they are going to be asked and what risks and benefits the study has. When they know this, they are able to decide to participate in the study or not.

• The principle of beneficence means that researchers need to assess beforehand the risk and advantages a study has for a participants or the population.

• The principle of justice asks for a balance between the people who participate in a research and the people who get advantages from this research.

• The guidelines of the American Psychological Association (APA) can also be used. These have five general and ten specific guidelines.

• The five general guidelines are: respect, beneficence, justice, integrity and loyalty and responsibility.

• Integrity means that a teacher needs to teach his students accurate things and that a therapist needs to stay up to date about the empirical evidence for a treatment.

• Loyalty and responsibility means psychologists are not allowed to have sexual relations with their students or clients and teachers are not allowed to have their students as clients.

• Ethical Standard 8 is the most important standard for researchers. This standard states that there should be an institutional review board that determines whether a study has been conducted ethically or not.

• It also states that in participants who have been deceived in a study, need to be debriefed.

• It is also not allowed to fabricate data (to come up with values) or to falsify data (influence results by leaving something out or by influencing participants). Researchers should also refer to the original author when using his/her ideas, otherwise plagiarism is taking place.

• When conducting research with animal participants, one must look at the three R’s: replacement (replacing animals), refinement (the animals doesn’t have to endure much stress) and reduction (using the smallest amount of animals).

What are good measures in psychology? - Bulletpoints 5

• Self-reports look at the answers people give on a questionnaire or during an interview.

• Observational measures operationalise a variable by determining the observable behaviour. Physiological measures operationalise a variable by looking at biological data, like brain activity and heartrate.

• Operational variables (nominal variables) are usually classified as categorical or quantitative, in which the levels of the categorical variables are categories. The numbers of the nominal variables have no numerical value.

• Quantitative variables can be classified on an ordinal, interval or ratio scale. An ordinal scale looks at the order of items and does not say anything about the distance between these values.

• An interval scale looks at the equal distances between levels and there is a zero-point, but this point doesn’t mean ‘nothing.’

• A ratio scale also has equal intervals and a true zero-point that does mean ‘nothing.’ Someone who really get a 0 on a test if he has not answered one question correctly.

• When variables are operationalised correctly, one can say there is good construct validity. Construct validity has two aspects: reliability refers to how consistent the results of a measure are and validity looks as the whether the variable measures what it is supposed to measure.

• Reliability can be tested in three ways: test-retest reliability, interrator reliability and internal reliability. Test-retest reliability means that the researcher finds the same scores every time he/she measures the same things.

• Interrator reliability means that the same scores are found by different rators. Internal reliability means that a participants gives a consistent pattern of responses.

• Reliability can be analysed with scatterplots and correlation coefficients. For a good reliability, the points need to be around the straight line and the correlation needs to be positive and strong (close to a value of -1 or 1).

• In order to assess the internal reliability of a scale, researchers need to look at Cronbach’s alpha. This number can be calculated with SPSS and the closer the number to 1, the more reliable the scale.

• Face validity means that a variable seems plausible: if it looks like a good measure, it has face validity. Component validity looks whether the measure contains all components of a construct.

• Criterion validity looks whether the measure is related to a concrete outcome, like a behaviour it should be associated with according to the theory. This can also be analysed with scatterplots and correlations.

• When there is validity, the measure needs to correlate strongly with other measures that look at the same construct (convergent validity) and it should correlate less strongly with measures of different constructs (discriminant validity).

How do we use surveys and observations? - Bulletpoints 6

• A survey refers to questions that are asked over the phone, on paper, during interviews, via email or on the internet. Surveys can have open-ended and forced-answer questions.

• Sometimes questions are formulated difficultly and respondents may not be able to answer the question accurately. It is best to formulate a question as simple as possible.

• Researchers should be aware of driving questions: some positive or negative words can influence the answers of respondents.

• Double-barreled questions are two questions in one. These have a bad construct validity, because you don’t know whether the respondent has answered the first question, the second question or both questions.

• The sequence of questions can also influence the answers someone gives. The best way to control for sequence effects is by making different versions of the questionnaire and changing the sequence in these versions.

• Response sets are quick responses someone can give when answering questions from a questionnaire. Sometimes people don’t think about the question and they can answer everything positive, neutral or negative.

• One form of a response set is acquiescence (agreeing with everything). One way to find out whether someone truly agrees with everything or just responds with ‘yes,’ is by reversing the statements.

• Another response set is fence sitting: people choose for the middle answer. One way to reduce this is to remove the middle option.

• A survey is suited to ask subjective questions: what a person is thinking that he is doing and the things he thinks that influence his behaviour. But if you truly want to know what people do and what influences their behaviour, you have to observe people.

• Observer bias occurs when the expectations of an observer influence his interpretations of the participant's behaviour. observatory effects occur when the observer changes the behaviour of the animals or person he is observing.

• Reactivity means that people change their behaviour in a way when someone is looking. This can be reduced by blending in as an observer or by letting the participants get used to you, so they forget you are observing them.

How to estimate the frequencies of behaviour and attitudes? - Bulletpoints 7

• External validity looks if results of a study can be generalised to a bigger population. External validity is important for frequency claims.

• A population is a whole set of people or products which a researcher is interested in. A sample is a smaller set from the population.

• In a biased sample some members of a population have a higher chance to be admitted to the sample than other members.

• Convenience sampling and self-selection result in biased samples. Convenience sampling is a sample of people who are easily accessible/available and self-selection means that a sample contains people who already wanted to participate in the study.

• In order to obtain a representative sample, researchers can use random sampling. This means that every member of the population of interest has an equal chance to be chosen for the sample.

• In cluster sampling clusters of people are randomly selected and all individuals in the selected clusters are used. Multistage sampling is similar, but two random samples are conducted: first a random sample of the clusters is conducted and then a random sample of the people within these clusters is conducted.

• In stratified random sampling a researcher selects certain demographic categories and he randomly selects people from each of these categories.

• Oversampling means that a researchers intentionally over-represents one group in his sample.

• When external validity is not important for the researcher, he can choose to use a biased sample. He can choose people in a non-random way (purposive sampling) and/or he can ask people if they can bring two other people to participate in the experiment (snowball sampling).

• Bigger samples are not always better. When you want to generalise the results to a big population (like that of the US), you can use 1000 participants.

What is bivariate correlation research? - Bulletpoints 8

• Association claims are claims that describe the relation between two measured variables. A bivariate correlation is an association about two variables.

• The data from a correlational study can be described with the help of scatterplots and the correlation coefficient r. When all participants are depicted as dots and a line is drawn through the dots, you can see whether the relationship is positive (x is high, y is high) or negative (x is high, y is low).

• The strength of the correlation can be expressed with the correlation coefficient r and this is a number between -1 and 1. A correlation of .10 or -.10 has a weak effect size, an r of .30 or -.30 has a moderate effect size and a correlation of .50 or -.50 or higher has a strong effect size.

• It is better to depict association claims with a categorical variable in bar graphs. It is also better to use the t-test instead of r for claims with a categorical variable.

• Effect size is part of statistical validity and association claims can be tested with statistical validity. The effect size looks at the strength of a relationship, and the closer the r is to 1, the stronger the relationship (in general).

• Strong effect sizes are more accurate and usually also more important than weak effect sizes. Weak effect sizes are only important when it is a matter of life and death.

• Statistical significance measures show a probabilistic estimate, p. The p states something about the chance that the association was found in a population in which the association is really zero (p smaller than .05 is significant, p equal or higher than .05 is not significant).

• Outliers usually only have an influence on small samples.

• When the whole range of a variable is not used (for example, if you only use middle income, but no high and low income), a restriction of range occurs.

• When there is a relationship between two variables, but this relationship can’t be depicted as a straight line, it could be a curvilinear relation. The relation might be positive in the beginning to a curtain point, but then turn negative (or vice versa).

• No causal inferences can be made with association claims. The three criteria for causality can’t be met with association claims, because experiment are not used in association claims.

• A moderator is a variable that changes the relationship between two variables in an association study.

What is multivariate correlational research? - Bulletpoints 9

• Research designs with more than two measured variables are called multivariate correlational designs. Examples of these are longitudinal studies and multiple regression designs.

• Longitudinal designs can establish temporal precedence by measuring the same variables for the same person on different time points.

• There are more than two variables involved in a multivariate correlational designs and the design will therefore give more than two correlations. These can be cross-sectional correlations, autocorrelations and cross-lag correlations.

• Cross-sectional correlations test whether two variables that have been measured on the same time points correlate. When you look whether the same variables correlate with each other on different time measures, you are looking at autocorrelations.

• Researchers are most interested in cross-lag correlations and those are correlations that look if earlier measures of a variable are associated with later measures of another variable.

• Longitudinal research can show that covariance is present and it can help with temporal precedence. However, this type of study can’t exclude a third variable (internal validity) and that is why it can’t meet all the criteria for causality.

• With multivariate designs researchers can try to find out if the relationship between two variables stays intact when a third variable is controlled. That third variable can be split into different subgroups.

• With multivariate regression designs you can exclude a third variable, but only if you have put this variable in your model.

• The statistical measure to look at in multiple regression is beta. Beta shows the direction and strength of the relationship between predictor and criterion variable, while the other predictor variables are controlled.

• Parsimony is the degree to which a good scientific theory can offer the most simple explanation for a phenomenon.

• When variable x directly influences variable y, but also indirectly influences y through variable z, we call variable x a mediator.

How can causal claims be evaluated with the help of experiments? - Bulletpoints 10

• In an experiment a researcher manipulates at least one variable (independent variable) and he measures another variable (dependent variable).

• Every variable that a researcher controls (so it stays constant), is called a control variable. This is done in order to be sure that there are no alternative explanations (confounds) for the results.

• Experiments comply with the three claims of causality (causality, temporal precedence and internal validity).

• A design confound is a researcher’s mistake in the development of an independent variable. It’s a second variable that varies at the same time as the independent variable of interest.

• A selection effect occurs in an experiment when the type of participant in one level of the independent variable differs systematically from the type of participant in the other level of the independent variable.

• Design confounds and selection effects aren’t good for the internal validity. Good experiments use random assignment to avoid selection effects.

• Sometimes random assignment doesn’t work and you can then use matched groups. These are groups of participants who are similar on a variable that may have an influence on the dependent variable and participants are randomly assigned to different conditions of the independent variable of interest.

• In an independent-groups design (between-group design) different groups of participants are placed into different levels of the independent variable. In a within-groups design (within-subjects design) there is only one group of participants and every person is exposed to each level of the independent variable.

• Two forms of the independent-groups design are the posttest-only design and the pretest/posttest design. In the postttest-only design participants are randomly assigned to the groups of the independent variable and they are only tested one time for the dependent variable.

• In a pretest/posttest design participants are randomly assigned into two groups and they are tested twice for the dependent variable: once before the exposure to the independent variable and once after the exposure to the independent variable.

• There are two types of within-groups designs. In the concurrent-measures design participants are exposed to all levels of the independent variable at the same time and only one behaviour or attitude is the dependent variable.

• In a repeated-measures design participants are measured more than once on the dependent variable- so after exposure to every condition of the independent variable.

• In order to prevent order effects, one can use counterbalancing. This means that researchers present the levels of the independent variables in different sequences to the participants.

What is the influence of confounding and obscure factors? - Bulletpoints 11

• Some threats to internal validity are maturation, history, regression threats, attrition, test threats and instrumental threats. Maturation is a change in behaviour that has spontaneously arisen in time and no intervention has caused it.

• Sometimes changes occur because something specific has happened between the pretest and the posttest. This is called a history threat and it doesn’t have to be a big event, but it has to be one that influences all the members or almost all the members of a population.

• Regression threats refer to regression to the mean. When an behaviour is extreme on time point 1, it will less likely be extreme on time point 2.

• Attrition is a reduction in the number of participants before the end of the study. Attrition becomes a problem when a certain type of participant doesn’t participate in the study (so, when it’s systematic).

• A test threat refers to the change in a participants because he/she makes the test more than once. An instrumental threat occurs when a measuring instrument changes in time.

• Many of these threats can be diminished by the addition of comparison groups. However, three threats can still be present: observatory threat, placebo and demand characteristics.

• An observatory bias occurs when the expectations of a researcher influence his interpretation of the results. Demand characteristics are a problem when the participants think to know what the study is about and change their behaviour accordingly.

• A placebo effect occurs when participants receive a treatment and get better, because they think they have received the real treatment (a pill instead of a sugar pill).

• The three threats can be diminished by conducting double blind studies.

• A null effect means that the independent variable doesn’t influence the dependent variable. A null effect can be found because the research wasn’t conducted properly or because there really isn’t an influence of the independent variable on the dependent variable.

• An obscuring factor might prevent researchers from seeing that an independent variable has influence on the dependent variable. The obscuring factors can take on two forms: there wasn’t enough difference between groups or there was too much variability within groups.

• When there is too little difference between groups, the variables might need to be operationalised differently. When there is too much variability within groups, within-group designs should be used.

How do you deal with experiments that have more than one independent variable? - Bulletpoints 12

• An interaction effect looks if the effect of the original independent variable depends on the level of another independent variable. There are different kind of interaction effects and you can spot them in graphs.

• When the lines of two independent variables cross each other, we speak of crossover interaction. When the lines of two independent variables are not parallel or don’t cross each other, we speak of spreading interaction.

• Researchers use factorial designs to test interactions. A factorial design is a design with two or more independent variables (called factors).

• In a factorial design one has to look at the main effects and the interaction effect(s). Effects of every independent variable, called main effects, look at the differences and interaction effects are differences of differences.

• In a between-subjects factorial design both independent variables are studies as independent groups. In a within-groups factorial design (repeated measures factorial) both independent variables are manipulated within groups.

• When researchers add a third independent variable and all independent variables have two levels, we speak of a 2 x 2 x 2 factorial design, or a three-way design. In this design, there are 2 x 2 x 2 = 8 cells or conditions.

What are quasi-experiments? - Bulletpoints 13

• In a quasi-experiment researchers don’t have full control over the conditions. Participants are not randomly assigned to the conditions.

• Matched groups can be used to check for differences between participants. Some researchers use a wait-list design, in which all participants undergo a treatment, but at different times.

• A maturation threat occurs when participants show an improvement from their pretest to their posttest, but it’s not clear whether the change is caused by the treatment or because the group improved spontaneously.

• A historical threat occurs when a historical event takes place for all participants at the same time. If that occurs, researchers can’t easily say whether the outcome is caused by the treatment or the external event.

• Comparison groups can take away these threats.

• Regression to the mean occurs when an extreme results is caused by a combination of random factors that will probably not occur again in the same combination. The extreme results will become less extreme over time.

• Attrition occurs when people don’t want to participate anymore in the research. It’s a threat to internal validity when the people who leave vary systematically from the people who stay.

• An advantage of quasi-experiments is that the study is conducted in real-life settings. There is no artificial setting and so the external validity is pretty good.

• Another advantage is that quasi-experiments help researchers conduct experiment they could not have been able to conduct in true experiments, because of ethics.

• When researchers use a small N-design, instead of collected little information from a large amount of participants, they collect a lot of information from a small amount of participants. They can even look at just one animal or one person in a single N-design.

• In a stable-baseline design researchers observe behaviour of a long baseline period before they begin with a treatment or intervention. If the behaviour during the baseline is stable, then researchers can be more confident that the treatment is effective.

• In a multiple-baseline design researchers spread the introduction of the interventions over different contexts, moments and situations.

• In a reversal design researchers observe problematic behaviour with and without a treatment, but they take the treatment away after a while (reversal period) to see whether the problematic behaviour will return. If the treatment really works, then the behaviour should become problematic again when the treatment is taken away.

Can the results of a study be applied to the real world? - Bulletpoints 14

• Replicability means that the findings, when the study is conducted again, show the same results. Replicability gives a study credibility.

• In direct replications researchers repeat the original study as close as possible in order to figure out if the original effect can be found with new data.

• In a conceptual replication researchers study the same research question, but they use different procedures. In a replication-plus-extension study the researchers replicate the original study, but they add other variables to test more questions.

• A meta-analysis is a mathematical summary of the scientific literature about a topic. In a meta-analysis there are many studies that have different sample sizes and it’s often the case that studies with higher sample sizes are weighed more heavily in the analysis.

• Meta-analyses can be biased when they don’t contain studies with null effects.

• The similarity between the context of a study and the real world is called the ecological validity. Ecological validity is an aspect of external validity.

• It depends on the goal of the study (and about whom the theory is) how important the external validity is. Some researchers don’t look at the external validity and focus more on the internal validity to help test their theories.

• Cultural psychologists have shown that many theories that have been supported in one cultural context, can’t be supported in another cultural context.

• Most studies are done with participants from the United States, Australia and Europe and these participants are called the WEIRD population: western, educated, industrialized, rich and democratic. These WEIRD people don’t represent the whole world and you can’t generalize the results from them to the entire world population.

• Many laboratory experiment are high in experimental realism. That means they create settings in which people show real emotions, motivations and behaviours.