Research methods in psychology by B. Morling (third edition) – Book summary
- 1709 reads
Construct validity refers to how well a study’s variables are measured or manipulated. There are three common types of measurement: self-report, observational and physiological. The conceptual definition, or construct, is the researcher’s definition of the variable in question on a theoretical level. The operational definition represents a researcher’s specific decision about how to measure of manipulate the conceptual variable.
SELF-REPORT
A self-report measure operationalizes a variable by reporting people’s answers to questions about themselves in a questionnaire or an interview. In research on children, self-reports may be replaced with parent reports or teacher reports.
The problems with self-reports are the demand characteristics: a participant wants to be a ‘good’ participant. People their self-perception is not always correct and the social desirability: people want to give a good impression about themselves.
OBSERVATIONAL MEASURES
An observational measure is sometimes also called a behavioural measure and operationalizes a variable by recording observable behaviour or physical traces of behaviour.
The problems with observational measures are:
PHYSIOLOGICAL MEASURES
A physiological measure operationalizes a variable by recording biological data. The problem with physiological measurement is that not everything can be measured with biological data (at least not yet).
SCALES OF MEASUREMENT
All variables must have at least two levels. The levels of operational variables can be coded using different scales of measurement.
There are three kinds of quantitative variables.
RELIABILITY
Reliability refers to how consistent the results of a measure are. Validity refers to whether the operationalization measures what it’s supposed to measure. There are several types of reliability:
The reliability is also measured in the statistics and is known as ‘p’. The closer Cronbach’s Alpha or coefficient alpha is to 1,0, the more reliable the measure is. To make sure that something is reliable researchers look for a Cronbach’s Alpha of at least 0.70 in self-reports.
VALIDITY
A measure is valid if it measures what it’s supposed to measure. There are multiple ways to check if a measurement is valid:
Reliability is necessary, but not sufficient for validity.
JoHo can really use your help! Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world
It is important to both produce and consume research. A research consumer is important, because to effectively know something or to put a theory or treatment in to use, it is imperative that the research consumer knows the evidence behind the evidence-based treatment. It is important to be able to decide how valuable and useful a research really is.
Both research producers and research consumers share an interest in psychological phenomena, such as behaviour or emotion. They also both share a commitment to the practice of empiricism: to answer psychological questions with systematic observations.
The cupboard theory is the idea that young animals (but also your dog) clings on to the caregiver because the caregiver provides food. The contact comfort theory is the idea that young animals (but also your dog) clings on to the caregiver because the caregiver provides warmth and contact comfort. These theories have been tested and followed the empirical cycle.
THE EMPIRICAL CYCLE
The empirical cycle always starts with an observation.
Induction -> Theory -> Deduction -> Prediction ->Testing -> Results -> Evaluation -> Observation - > Induction
Data are a set of observations. Depending on whether the data are consistent with hypotheses based on a theory data may either support or challenge a theory. The best theories should be supported by data from studies, should be parsimonious and falsifiable.
Basic research is used to enhance the general body of knowledge. Applied research is done with a practical problem in mind. Translational research is the dynamic bridge between basic and applied research. E.g: a basic research is about schizophrenia. Translational research is used to develop a new treatment for schizophrenia and applied research is used to see how people diagnosed with schizophrenia can fit better into today’s
.....read moreEXPERIENCE
Experience is not a reliable source of information, because it has no comparison group. A comparison group in research is a group which isn’t affected by the controlled independent variable, so it is possible to really determine whether the independent variable has the effect people think it has.
E.g: Doctors used to take blood from an ill person, because they believed that it cured the illness. Some people recovered and they concluded that they recovered because they bled the patients. This is based on experience, they have experiences that some patients recovered, but they did not have a comparison group, so they had no way of knowing that the recovery was because of bleeding the patient. To make sure that it had this effect, they should have had a group with people who were ill, but were not bled, to see what would have happened.
When we are using personal experience to determine whether something works or not, we don’t have a comparison group as well. “My knee feels better with this tape”, but you don’t know how it would’ve felt if you didn’t use that tape. There is no comparison group, so it is not possible to give a conclusive answer, based on empirical evidence.
In real-world situation situations, there are several possible explanations for an outcome. In research, these alternative explanations are called confounds. Experience is confounded, because you do not know the cause of an effect, although you might think you do. When you use tape to lessen the pain in your knee, you don’t know whether the tape caused the pain to diminish. A researcher can see the situation from outside, but you can only see one condition and all you have is your experience.
Behavioural research is probabilistic. This means that it’s findings are not expected to explain all cases all the time. The conclusions of research are meant to explain a certain proportion of the cases. The two big problems with using experience as a source of information is that there is no comparison group and that experience is confounded.
INTUITION
People use their intuition to make decisions, although it is not a reliable source of information, because intuition is biased. There are ways our intuition is biased:
There are two historical examples of studies that violated several ethical criteria.
The Belmont Report outlines three main principles for guiding ethical decision making:
The APA outlines five general principles for guiding individual aspects of ethical behaviour. Three of the give general principles are the same principles as in the Belmont Report. The other two are:
The APA lists ten specific ethical standards. These standards are similar to enforceable rules or laws.
Standard | Definition |
Institutional review board | This is a committee responsible for interpreting ethical principles and ensuring that research using human participants is |
Construct validity refers to how well a study’s variables are measured or manipulated. There are three common types of measurement: self-report, observational and physiological. The conceptual definition, or construct, is the researcher’s definition of the variable in question on a theoretical level. The operational definition represents a researcher’s specific decision about how to measure of manipulate the conceptual variable.
SELF-REPORT
A self-report measure operationalizes a variable by reporting people’s answers to questions about themselves in a questionnaire or an interview. In research on children, self-reports may be replaced with parent reports or teacher reports.
The problems with self-reports are the demand characteristics: a participant wants to be a ‘good’ participant. People their self-perception is not always correct and the social desirability: people want to give a good impression about themselves.
OBSERVATIONAL MEASURES
An observational measure is sometimes also called a behavioural measure and operationalizes a variable by recording observable behaviour or physical traces of behaviour.
The problems with observational measures are:
PHYSIOLOGICAL MEASURES
A physiological measure operationalizes a variable by recording biological data. The problem with physiological measurement is that not everything can be measured with biological data (at least not yet).
SCALES OF MEASUREMENT
All variables must have at least two levels. The levels of operational variables can be coded using different scales of measurement.
There are three kinds of quantitative variables.
RELIABILITY
Reliability refers to how consistent the results of a measure are.
INTRODUCING BIVARIATE CORRELATIONS
A bivariate correlation or bivariate association, is an association that involves exactly two variables. The nature of the association can be described with scatterplots and the correlation coefficient. Associations between categorical variables are usually presented in a bar graph.
INTERROGATING ASSOCIATION CLAIMS
The two most important validities to interrogate are construct validity and statistical validity with an association claim. The construct validity checks how well each variable was measured. The statistical validity checks how well the data supports the conclusion.
There are five questions that can be asked in order to interrogate the statistical validity:
INTERNAL VALIDITY: CAN WE MAKE A CAUSAL INFERENCE FROM AN ASSOCIATION?
The three requirements in order to establish causation are the following:
A third-variable problem can be exposed by checking the correlation with that third variable between the original two variables. If there is a third-variable present, the original association is then called a spurious association.
EXTERNAL VALIDITY: TO WHOM CAN THE ASSOCIATION BE GENERALISED
The external validity interrogation asks whether the association can generalize to other people, places and times. When the relationship between two variables changes depending on the level of another variable, that variable is called a moderator.
Experiments are the only way to investigate causal issues.
EXPERIMENTAL VARIABLES
An experiment means that the researchers manipulated at least one variable and measures another. A manipulated variable is a variable that is controlled. Measured variables take the forms of records of behaviour or attitudes. The manipulated variable is the independent variable. The conditions are the different levels of the independent variable. The measured variable is the dependent variable. Control variables are variables that are also controlled. These variables are controlled by holding all other factors constant. Any variable that an experimenter holds constant on purpose is called a control variable.
WHY EXPERIMENTS SUPPORT CAUSAL CLAIMS
There are three rules for something to be causal:
If independent variables did not vary, a study could not establish covariance, because you need a comparison group to establish covariance. It is impossible to establish internal validity if there are confounds, or alternative explanations. A design confound is an experimenter’s mistake in designing the independent variable. It is a second variable that happens to vary systematically along with the intended independent variable. Something is only a design confound if it shows systematic variability with the independent variable. It would not be a design confound if it shows unsystematic variability. If individual differences are distributed evenly in both groups, the are not a confound.
Selection effects are effects that are the result of two groups being systematically different from those in the other. This can also happen when the experimenters let participants choose in which group they want to be. The selection effects can be avoided by using random assignment, when assigning people to the conditions. Selection effects can also be avoided by using matched groups.
INDEPENDENT GROUP DESIGNS
In an independent group design both groups of participants are placed into different levels of the independent variable. This type of design is also called a between-subjects design or between-groups design. In a within-groups design or within-subjects design, there is only one group of participants and each person is pretended with all levels of the independent variable.
In the posttest-only design, also known as equivalent groups, participants are randomly assigned to independent variable groups. In a pretest/posttest design, participants are randomly assigned to at least two different groups and are tested on the key dependent variable twice, before and after exposure to the independent variable.
WITHIN-GROUPS DESIGNS
There are two basic types of within-groups design:
The main advantage of a within-group design is that it ensures that participants in the two groups will be equivalent. The term
.....read moreTHREATS TO INTERNAL VALIDITY:
There are 12 threats to internal validity. Most of these threats can be prevented with a good experiment design and only occur in the so-called ‘really bad experiment’, also known as the one-group, pre-test/post-test design. The following twelve threats to internal validity exists:
Threat | What happens? | When? | Solution |
Maturation threat | A change in behaviour occurs more or less spontaneously over time. People adapt to changed environments. | One-group, pre-test/post-test design | Using a comparison group |
History threat | A specific event has occurred between the pre-test and the post-test that affects almost every participant systematically (e.g: a change of seasons). | One-group, pre-test/post-test design | Using a comparison group |
Regression threat | If a group’s mean is unusually extreme at the pre-test, it is likely to be less extreme at the post-test, closer to the typical mean (e.g: depressed people have an extreme mean of sadness and this probably will be less extreme when it is tested again). Regression alone does not make an extreme group cross over the mean to the other extreme. | One-group, pre-test/post-test design | Using a comparison group |
Attrition threat | A reduction in participant numbers that occurs when people drop out before the end. This is only a problem if attrition is systematic. | One-group, pre-test/post-test design |
EXPERIMENTS WITH TWO INDEPENDENT VARIABLES CAN SHOW INTERACTIONS
Experiments with more than one independent variable allows researchers to look for an interaction effect. This is an effect where the effect of the original independent variable depends on the level of another independent variable. If the two lines of the independent variables cross, there is a crossover interaction, also known as “it depends”. If the lines are not parallel, there is an interaction and if the lines are parallel, there is no interaction. A spreading interaction occurs when the two lines spread out and can be labelled as an “only when..” interaction. An interaction is a difference in differences
FACTORIAL DESIGNS STUDY TWO INDEPENDENT VARIABLES
Testing for interactions is done with factorial designs. A factorial design is one in which there are two or more independent variables. In a factorial design, researchers study each possible combination of the independent variables. A participant variable is a variable whose levels are selected, but cannot be manipulated (e.g: age, the level for this variable can be selected, but not manipulated). Using factorial designs to test limits is called testing for moderators and it is a way to test the external validity of an experiment. Factorial designs can also test theories and hypotheses.
INTERPRETING FACTORIAL RESULTS: MAIN EFFECTS AND INTERACTONS
Researchers test each independent variable to look for main effects, the overall effect of one independent variable on another independent variable. Marginal means are the arithmetic means for each level of an independent variable, averaging over levels of the other independent variable. The main effect is not the most important effect, but the overall effect of one independent variable on another independent variable. The interaction itself is the most important effect. In a factorial design with two independent variables, the first to results obtained are the main effects for each independent variable. The third result is the interaction effect.
FACTORIAL VARIATIONS
In a mixed factorial design, one variable is manipulated as independent groups and the other is manipulated as within-groups (e.g: age and driving while on the phone. Age is independent groups and driving while on the phone is within-groups). When plotting a three-way factorial design and you want to check for three-way-interactions, you have to look for differences between the two states. If the lines are the same for both states in the three-way interaction, then there is a two-way interaction, but not a three-way interaction (unless the lines are parallel).
IDENTIFYING FACTORIAL DESIGNS IN YOUR READING
When looking for factorial designs in research articles it is important to look at the method part of the research description. When looking for factorial designs in regular articles it is important to look for the phrases it depends and only when.
QUASI-EXPERIMENTS
A quasi-experiment differs from a true experiment in that the researchers do not have full experimental control. In quasi-experiments, researchers might not be able to randomly assign participants to one level or the other. They are assigned by other things (e.g: teachers, political regulations or nature).
A non-equivalent control group design is a quasi-experiment in which there is a treatment group and a control group, but the participants have not been randomly assigned. A non-equivalent control group pretest/posttest design is a quasi-experiment in which participants are tested before and after the experiment, but are not randomly assigned to groups. An interrupted time-series design is a quasi-experiment that measures participants repeatedly on a dependent variable. A non-equivalent control group interrupted time-series design is a quasi-experiment in which the independent variable was studied as a repeated-measures variable and an independent groups variable.
There are several possible threats in quasi-experiments to internal validity:
Threat | Definition |
Selection effect | The participants of one level of the independent variable are systematically different from other participants at another level of the independent variable. |
Design confounds | In a design confound, some outside variable systematically varies the levels of the targeted independent variable. |
Maturation threat | An observed change has emerged more or less spontaneously over time. |
History threat | An external, historical event happens for everyone in a study at the same time as the treatment (e.g: a change of seasons). |
Regression to the mean | A measure is extreme and will thus (almost) always be less extreme and more closely to the mean on the next measurement. |
Attrition threat | Certain kinds of participants drop out systematically (e.g: only the most depressed people drop out). |
TO BE IMPORTANT, A STUDY MUST BE REPLICATED
Replication gives a study credibility, and it is a crucial part of the scientific process. There are several types of replication:
The replication crisis refers to the fact that a lot of psychological studies don’t share the same results when they’re replicated. Replication studies might fail, because some original effect are contextually sensitive and when the replication context is too different, the replication is more likely to fail.
HARK-ing is hypothesising after the results are known. P-hacking is using more individuals and removing certain outliers if the results of the first experiment were not significant. The goal of this to find a p-value of under 0.05. There are three changes made to psychological research in order to increase the replication rate:
In order to increase the replication rate in journals, journals now all devote a section to replicated articles. Meta-analysis is a way of mathematically averaging the results of all the studies that have tested the same variables to see what conclusion the whole body of evidence supports. This makes use of both published and unpublished articles. The file drawer problem refers to the idea that a meta-analysis might be overestimating the true size of an effect because null effects, or even opposite effects, have not been included in the collection of the process (unpublished studies are less likely to make it into a meta-analysis).
TO BE IMPORTANT, MUST A STUDY HAVE EXTERNAL VALIDITY?
The manner in which the participants are recruited is more important than the number of participants for getting external validity. Ecological validity is the generalizability of an experiment to real-world settings.
Researchers in the theory-testing mode are usually designing correlational or experimental research to investigate support for a theory. When investigating support for a theory, the generalizability is not always necessary (e.g: if a theory is false in one sample, it should be false in all samples). Researchers in the generalization mode want to generalize the findings from the sample in a previous study to a larger population. Frequency claims are always in the generalization mode and association and causal claims are usually in theory-testing mode, but can be in generalization mode. Many
.....read more
There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.
Do you want to share your summaries with JoHo WorldSupporter and its visitors?
Field of study
Add new contribution