Kwalitatieve Onderzoeksmethoden: Samenvattingen, uittreksels, aantekeningen en oefenvragen - RUG
- 1854 reads
The goal of qualitative research methods (QRM): to perform all stages of a qualitative research project, i.e. perform interviews in a systematic and methodologically responsible way.
Essentials of interviewing:
A good interview is a focus on the subject’s (Interviewee/IE) world
What to do?:
Create a collaborative atmosphere The more comfortable interviewees feel, the more they are prepared to open up and talk, the better your data are likely to be
Interviewer’s (IR) contribution: prepare both the task-oriented and the interaction-oriented role before the interview
Develop a genuine interest in the interviewee during the interview, listen, prompt, encourage and direct.
Interviewers first attention generally goes to content (focus on data). What you SHOULD focus on: interviewee as a way to collect those data IE mirrors the IR’s attitude: if lack of interest, so does IE.
Basic classification of interviews:
a. structured
b. unstructured
c. semi-structured
a.) Structured interview:
Advantage: consistency during multiple interviews
Disadvantage: no space for new insights on the basis of voluntary information from the interviewee
The most highly structured interview format is undoubtedly the survey questionnaire: an interview guide that pre-specifies both the content and the possible responses to each question. It involves the use of pre-formulated questions.
The whole idea of structured interviews is to ensure consistency across multiple interviews. They tend to be in market research, polling, telephone interviews and with intercept research such as in shopping centres.
b.) Unstructured interviews:
The opposite of structured interviews. There may not be a time limit. Questions are only used to jog someone’s memory.
Characteristics:
very few pre-formulated questions.
no attempt to maintain consistency across interviews
advantage: allows the interviewee to talk freely and tell you everything he or she considers important
disadvantage: if interviewee is not talkative, you may end up with too little; if the interviewee is very talkative, you may end up with too much. Also hard to compare answers by different Ies
c.) Semi-structured interview: in-depth interview (What we have to do in this course):
Advantages: takes the best of structured/unstructured approaches while minimizing the risks; it is cost-efficient and relatively easy to conduct.
Limitations:
NB: for these reasons semi-structured interviews are often combined with other forms of data, such as observations, diaries, documents.
The composition of a focus group is of vital importance: participants need to feel comfortable talking with each other about the research topic. If you want a lively conversation, you need participants who are actively interested in talking to each other about the topic.
Point of attention: style of moderating by the interviewer.
Individual interviews: more useful when the goal is to obtain depth and detail about each participant.
Focus groups: more useful when the goal is to hear from a range of participants.
Combination: for example focus groups could provide an introduction to the views of a variety of participants. Next a number of participants could be selected for in-depth interviews.
Advantage: focus groups enable a researcher to elicit opinions, attitudes and beliefs held by members of a group
Disadvantage: often time-consuming and expensive
Point of attention: interviewer moderating style
Points of attention using any kind of interview:
Lack of trust:
interviewer is complete stranger, so interviewee may decide not to open up completely. As he may thus be withholding important information, the data gathering will be incomplete
Lack of time:
time pressure may either be the cause of omissions or answers that are not entirely true for lack of fine-tuning
Level of entry / Elite bias:
the level at which the researcher enters the organization is crucial: who is the interviewer identified with and how does that affect the relationship with interviewees.
Hawthorne effect:
the interviewer is not a neutral factor in the interview and may influence the interaction: IE knows that the answers are used for academic purposes so that may affect their response (is this what you want to hear?)
Ambiguity of language:
the meaning of the interviewer’s words can be ambiguous and it is not always certain that the interviewee fully understands the questions.
Interviews can go wrong:
it is possible that the interviewer unintentionally offends the interviewee in which case the interview might be abandoned altogether.
Conclusion:
When you are aware: opportunity for intervention
When you are prepared, you know:
A good interview helps us to focus on the subject’s world. The idea is to use their language rather than impose our own.
The role of the interviewer is to listen, prompt, encourage and direct. Overall, the more comfortable interviewees are, and the more they are prepared to open up and talk, the better the disclosure is likely to be.
If your interviewee does not open up during the conversation, then the primary data you collect may be of limited value. Both the quantity and the quality of your data will be affected negatively. As a general rule, the more proficient you become at interviewing, the better your qualitative data will be
Also: how much do you know for sure after an answer has been given?
We interpret what we see and hear all the time but we need to offer our interpretation to the IE for confirmation or correction!
In order to achieve this, the interviewer is focused on two activities at the same time: to listen, prompt, encourage and direct to collect valid and relevant data + to create an atmosphere where the interviewee is willing to provide you with data.
So the two roles of the interviewer are:
Task-oriented = organize the interview in such a way that you reach your goal, i.e. elicit valid and reliable data for your research.
Criteria: interview structure, clear goal, active listening skills
Relation-oriented = conduct the interview in such a way that the interviewee is willing to provide you with valid and relevant data. Focus is on interaction, contact and trust.
Criteria: create collaborative atmosphere, expectation management = paying genuine attention to the other person’s needs
Interview guide
Interview guides provide the researchers with a script or protocol for the interview
The topics of the interview guide are based on the research question and the tentative conceptual model that underlies the research.
Benefits of this approach:
As researcher you will work on grounded theory: that is, you start from scratch; you have concepts in your mind based on theory. You will build up theory from the data.
A question-based guide outlines the expected content of the interview in terms of a series of questions the interviewer intends to ask.
A topic-based guide consists of a list of areas and issues the interviewer wants to hear about. Often they are already outlined in a format which serves as a checklist for the interviewer to make it easier to monitor which topics have already been covered.
Type of questions:
a. Open: what, where, how, who, why
Closed: only yes/no answer possible
Choice: answer is either .. or
b. Neutral: How would you characterize the atmosphere in your team?
Leading/Suggestive: Don’t you think the computer system in this organisation is out-dated?
Evaluating
By evaluating answers you checks their usability for your research focus:
Is the answer valid? Valid = what the interviewee thinks and says the same?
Is the answer complete?
Complete = checking if the answer is indeed all there is (e.g. I don’t know)
Is the answer relevant?
Relevant = a question might appear to be answered whereas in fact it is not
Is the answer clear?
Clear = are you sure you know what the IE means?
Whenever the interviewer (you) concludes that an answer does not meet any of the required criteria, or when you are in doubt, or when an interviewee has trouble finding an answer, then it is time to probe (ask further questions).
Probing does not add content to that introduced in the opening Q. It does not raise new issues.
Different types of probing:
1. Repeating or clarifying the opening question
2. Repeating or summarizing the answer
3. Focused / unfocussed probing
4. Silent probing and listening behaviour
Interviewing in a cross-cultural context requires awareness of:
What do you think + feel when you read this word?: Autumn
Season of falling leaves in the Netherlands – “no such thing as autumn: Iceland – increasingly hot: South Africa
Rain
Gloomy season
Moody, sad
versus
Bright colours
Long shadows
Moving towards Xmas
Bracing weather
Lovely season
Happy feeling
Lesson: we have a certain perspective on the world around us and we may not share IE’s experience and parameters. How to find out: probing!!
Interaction tips to consider (cross-cultural interview):
1. how do other people behave, how do they interact, their body language. As far as possible mirror and match their body language; e.g. leaning forward / back
2. Accept that social rules may be very different. Be adaptable to other people’s needs
3. Don’t assume because someone represents a particular country, that they behave accordingly
4. Keep going even when it (interview) takes longer than you had expected
Before introducing the definition and specific methods of the qualitative approach, a general discussion about research methodology is required.
In any empirical research, you will always find a chapter about “methodology”.
Methodology refers to the choices made about cases to study, data gathering techniques, data analysis methods, etc.
In your own bachelor and master thesis, you will have to include this chapter too.
Methodology comprises:
In this course, a number of qualitative methods are going to be discussed. You already know about some quantitative methods.
Essential in case of quantitative methods is that numbers is the basic material which is analysed, while in case of qualitative research, essential is text.
Recently, thanks to an increasing usage of social media, new online methods became relevant.
The traditional distinction between QUAL and QUANT is being broken down by these new opportunities, to find out what people are saying and feeling.
Quantitative: Survey, Laboratory experiment , Simulation , Mathematical modelling (based on numbers)
Using numbers, it can objectively describe, test, segment and predict (as long as the sampling is sound).
It tells you:
main behaviours
trends
satisfaction levels, attitudes, awareness …
Qualitative: Case study, Interviewing, Focus Group, Grounded theory, Action research (based on text) Online: Listening, netnography, bulletin boards
Gives detail, understanding and emotional response: Emotion is always involved in decision making – even if people aren’t aware of it, or if they rationalise it
Definitions:
Quantitative research
“Measuring things that can be counted using predetermined categories that can be treated as interval or ordinal data and subjected to statistical analysis” (Patton, 1997)
Qualitative research
“Focuses on people’s experiences and the meanings they place on events, processes and structures of their normal social setting. Such research may involve prolonged or intense contact with people and groups in their everyday situations. This provides a holistic view, through the participants’ own words and perceptions, of how they understand, account for and act within these situations” (Miles and Huberman, 1994)
understanding people and (their) context
| Quantitative research | Qualitative research |
| natural sciences | social sciences |
Pro’ s |
|
|
Con’s |
|
|
The choice between using a qualitative or quantitative approach in research is not a trivial one. When you will be in the position to choose the most appropriate approach for your bachelor or master thesis, you may face this challenge.
Depending which approach is chosen for a particular research (qualitative or quantitative), a research method or a data collection method can be used differently.
For example, in a quantitative approach, observation can be used as a preliminary work to identify key informants within a large company. Alternatively, in a qualitative approach, observation can yield the data necessary to be analysed in order to identify the behaviour of different groups of people.
In a quantitative approach, text analysis can be used to count, for instance, the frequency of all terms related to “radical innovation”, “incremental innovation”, “sustainable innovation”.
In a qualitative approach, text analysis can be used to understand what “radical innovation”, “incremental innovation”, and “sustainable innovation” means for a group of experts in product development.
In a quantitative approach, interviews consisting on closed questions can be used in a survey research about “which social media platform do you use and how often”, involving 30 randomly selected employees. In a qualitative approach, open interviews can be performed with 10 people, about why and how do they use social media applications.
Research Design:
1. Research objectives: what is the research goal/objective?
2. Research setting
Comparative study: comparing joint-ventures from US and China with respect to leadership styles
Snapshot: high employee turnover in a specific company. It is a short study
Retrospective: the biography of Thomas Edison, the challenges of liberalisation of gas market in NL
Case study: Enron as an accountancy bad practice, also example of retrospective case
Longitudinal studies
Repeated examination of a set of subjects over time
The evolution of selected indicators is of concern
Panel longitudinal studies
Explore micro-level change at individual level
Determine the type of change – QUAL goal
Measuring change is more a QUANT goal
Ex: how merging affects job satisfaction, three waves of interviews every 6 months
3. Data collection
Cross-sectional (longitudinal) studies
Explore macro-level change at context level
changing societal influences on attitudes
Ex: what shapes views in social-media usage?
Naturally occurring data
Observation, conversation data
Need: researcher’s interpretation
Generated data (primary data)
In-depth (unstructured), semi-structured interviews, focus groups
Participants give their own meanings and interpretations, interviewer can probe
Secondary data
Existing materials, collected for different purpose
Is it enough relevant and quality data for the new research?
4. sampling methods
Purposeful sampling:
Homogeneous sampling- cases having same characteristics
Heterogeneous sampling – cases which vary from each other
Extreme case or deviant sampling – cases are unusual or special
Typical case – cases characterized ‘normal’ or average to provide detailed profiling; it requires prior knowledge about overall patterns of response.
Convenience sampling and opportunistic sampling
Convenience sampling: ease of access
Opportunistic sampling: as events arise
Theoretical sampling
Type of purposive sampling
Cases are selected to base the development of new concepts
In connection with grounded theory (in GT, theory emerges from qualitative data)
After first data analysis step, additional data collection stages may occur: more interviewees, more groups
Data analysis and data collection continues, until theoretical saturation is reached
Sample size:
Not too small, not too big
Interviews: no more than 50 people
Focus groups: max 90-100 people (12 to 14 groups)
Issues determining sample size:
Heterogeneity of population
Number of selection criteria
Multiple samples within one study
Type of data collection methods
Budget and resources available
Small sample size if purposive or theoretical sampling has taken place!
Sampling frame:
Samples are generated from a population, using a sampling frame
Existing sources: ex. published lists
Generated sampling frames: ex. snowballing, flow population
5. Data analysis:
Grounded theory
Analytic induction
Text (content) analysis
Explained in Lecture 3
Difference qualitative and quantitative research:
Quantitative: ‘amount’ of certain qualitites
Qualitative: properties of objects, phenomena, people
Look back at the notes of the first lecture where the differences are shortly discussed.
In fact, there is quite an ideological discussion about these differences between different schools of thinking. You should be aware that, in essence, the difference should only be between good or bad research.
The research process of quantitative research is mainly a linear process (continuing the cycle is also possible).
In qualitative research, the research process has the tendency of being more circular. This happens because the considering of new information during the analysis process is possible.
Aimed more at theory building than theory testing
A circular model
Suppose that when we question about the (lack of) data sharing within company XYZ, we would consider in the beginning only 2 departments, then we would perform the analysis, and see that we do not have enough information.
Then we could continue to interview people from other departments as well.
Text analysis (content analysis)
In qualitative research, text is essential. There are 2 streams of thinking:
1. Content analysis involves quantification of qualitative data (e.g. words, gestures, art forms) for the purpose of statistical inferences)
2. Content analysis based on grounded-based theory (the constant comparative method – invented by Glaser and Strauss)
Hypothesis testing or Theory Development?
Quantification of qualitative data: Text analysis is used for counting entities such as words, phrases, codes within blocks of texts.
See slide 11 and 10: an illustration of words aggregated in themes. Words are counted.
Grounded theory: also used in qualitative research methods. The discovery of the Grounded Theory (1967) was done by Barony Glaser and Anselm Strauss. They wrote a book on methodology, after their success with ‘Awareness of Dying book’
It is also called ‘Constant comparative method’:
- inductive generation of hypotheses and theories from empirical data
- no preconceived theory prior to and during theory building
- aims at developing increasingly abstract and complex conceptual structure (s)
- emerge is essential
Other qualitative methods, such as case study, action research will be presented in Lecture 4
After first data analysis step, additional data collection stages may occur through additional interviewees, additional groups of people, additional decision concerning methods for data collection. See slide 14 for an example of a text analysis
Qualitative (text) analysis is based on assigning meanings to “things”!
Catching signals à combining/summarizing à pattern à possible connections à possible insights à new signals à etc.
There are two approaches of how to perform a text analysis:
1. Having at hand an existing list with predefined concepts: existence of preliminary theory: predefined themes (words, codes etc. are found to support these themes and eventually new concepts emerge).
Imagine that you know a-priori (from theory) that communication is an important factor responsible of data sharing in companies. You will read the text trying to identify all words/expressions/etc. which are related to the concept of communication.
2. Starting completely from scratch, there are no predefined concepts. There is no preliminary theory. Ground Theory: all themes/concepts emerge by coding and developing new themes (categories).
There are concepts relevant to your research objective that you simply do not have a clue about: you ‘dive’ into available text to identify new aspects.
Theoretical coding: the process of coding and categorizing qualitative data with the goal of developing a theory in an emergent manner.
Consists of 3 procedures (Strauss, 1984)
1. Open coding: coding and developing categories of related codes
2. Axial coding: relating codes (or categories) to each other, by combining inductive and deductive thinking
3. Selective coding: building a story line that connects the categories. It results a theory (framework, conceptual model, set of hypotheses)
Open coding:
Step 1: Establish segments (data fragmentation)
Step 2: Assign codes within segments
Step 3: Group and order codes
Axial coding:
relating codes (or categories) by identifying relations
a category can be included in a broader category
a category can be collapsed or combined with other categories
a category can be dropped
High level categories are candidates for high level concepts (or factors)
Selective coding:
Thematic coding: In different social worlds or groups different views can be found. It is used to compare group or cases with different perspectives
1. First, one case is analysed - open coding is used first and then selective coding is used to generate thematic domains and categories.
2. Next, developed categories are cross-checked with other cases; eventually new categories will emerge
3. The result is a thematic structure
Components of a conceptual model: It is a visual representation of a theory:
Why and when is not represented, this has to be clarified in the text.
How to test a conceptual model: predictions (propositions / hypotheses)
Article discussion: A. Ordanini, L. Miceli, and M. Piazzetti, Crowd-funding: transforming customers into investors through innovative service platforms, Journal of Service Management, Vol. 22, No.4, 2011, pp. 443-470.
Phenomena not understood:
Crowd-funding: “collective effort by people who network to pool their money together, via the Internet, in order to invest in and support efforts initiated by other people and organizations” (Ordanini, 2009)
Illustrative for…
Nestor -> Assignments -> crowdsourcing/crowd-funding materials
What is case study research? (Yin 1981, 2009)
Research technique aimed at examining a contemporary phenomenon in its natural setting
Especially relevant when boundaries between phenomenon and context are not evident
Types of case study research:
Descriptive Case Study: To describe the incidence or prevalence of a phenomenon of interest - “How have the attitudes of governmental institutions towards citizens changed over time?”
Explanatory Case Study: To trace causal linkages among actions, decisions and events over time - “Why do organizations decline?”
Exploratory Case Study: To develop pertinent hypothesis and propositions for further inquiry - “How can university education be rendered more effective?”
Five Elements of Case Study research:
Designing the case study
Preparing for data collection
Collecting data in the field
Evaluating and analyzing data
Sharing the findings / writing the report
1. Designing case studies:
1. What are the questions of the study?
Case studies focus on “how” and “why” questions
Look at previous literature to narrow down your questions
2. What are the propositions of the study (if any)?
Clear statements you want to test
Point at specific things that should be examined
Not if exploratory study
3. What is the unit of analysis?
What is the “case”?
At what level do you need information to provide answers to your questions?
Research questions should give you clear indication. If not: bad questions!
4. How can we link the data to the propositions?
What type of analysis methods can we use to obtain insights?
5. What are the criteria to interpret the findings?
No traditional statistical proof!
Try to find rival explanations for your findings
Role of theory
Case studies build on existing theory to come up with research questions
Case studies test existing theory
Case studies build new theory - Not by statistical proof - By analytical generalization
Selecting case study designs:
Single- or Multiple-Case designs?
Single case: artifacts of specific situation?
Better use multiple cases
Possibilities of direct replication
Possibilities of contrasting situations
Closed or Flexible designs?
Data collection provides new info
New info may require adjustment of setup
But: be cautious and careful!
Mixed methods designs?
Combining different methods in one single study
Quality of case study designs:
Four tests, and tactics to apply
Construct validity:
Internal validity (only explanatory studies):
External validity:
Reliability:
2. Preparing for data collection: What skills should a good investigator have?
Ask good questions
Be a good “listener”
Be adaptive and flexible
Have a good understanding of the issues that are being investigated
Be unbiased by prior knowledge and ideas; sensitive and open to new information
What is a case study protocol?: Written document to be used as a guide during the case study Keeps the researcher focused and forces anticipation of certain problems
What should be the elements of the protocol?
Overview of the project - Background info, objectives, issues, relevant readings.
Field procedures - Credentials, sites, language, source of data, procedural reminders
Case study questions - What should the researcher focus on when collecting data, together with potential sources of information for each question.
Guide for the final report
Selecting cases
Site Selection:
Geographical location
Business parameters
Market segments
Extreme ends of phenomenon of interest
Two-stage approach:
Collection and analysis of quantitative data to get about 20-30 possible candidates
Get more qualitative insights on these candidates, and then choose the case(s) for your study
Do not create mini-cases for each!
Pilot case
Selection of the pilot case:
very accessible people
Convenient location
Unusual amount of information
The most complicated case of all - Any possible issue can be found there
Goals of the pilot case:
Provides clarification of concepts
Helps in refining the research plan and questions (>< pretest)
Helps with narrowing down the research focus (hence: broader than final data collection)
Helps with solving methodological issues
3. Collecting data
Six main sources of data
Documents (see also next session)
Archives (see also next session)
Interviews
Direct observation (see also next session)
Participant-observation (see also next session)
Physical artifacts
Principles of data collection
1. Employ multiple sources of data
Single source
Problems of accuracy and trustworthiness
Triangulation
Rationale for utilizing multiple sources of empirical evidence
Advantage: Construct validity
Disadvantages: - Expensive - Time Consuming - Requirement for Diverse Skills
2. Create a case study database
Separation between gathered evidence and final report
Increases reliability
Database Content
Documents, notes, quantitative data, narratives etc.
Database should be accessible to external parties
3. Maintain a chain of evidence
To enable an external observer to follow the derivation to any evidence
Increases reliability
Trace steps
From research questions to conclusions and vice versa
4. Evaluating and analyzing data: Before ‘really’ analyzing the data: stare and glare, play with them
Put info in different arrays
Make atrices of categories & place evidence in these categories
Create displays, graphs
Tabulate the frequency of events
Provide descriptive statistics
Put info in chronological order
Analysis strategies
Four main strategies to analyze the data:
Rely on theoretical propositions:
Bases for your research
Have shaped your data collection
Develop a case description:
Alternative when the first strategy does not work
e.g. lots of data without clear set of research questions or propositions
Interesting when original objective was descriptive
Can help to identify the appropriate causal link to be investigated
Combine qualitative and quantitative data:
Not either/or, but and/and
Examine rival explanations:
Can be combined with all three previous strategies - Rival hypotheses (1) - Rival descriptive frameworks (2) - Data from other groups (3)
Types of rivals
Craft (no-real-life) rivals -Null hypothesis / Validity threats / Investigator bias
Real-life rivals - - Direct rival / Commingled rival / Implementation rival / Rival theory / Super rival / Societal rival
Threats to Validity: Effects that take place during the study - Mrs. Smith
Maturation: Physiological processes occurring within participants that can affect their behaviour
Regression to the mean: Participants with extreme scores when tested first time, tend to have less extreme scores on subsequent retesting
Selection of subjects: Bias in selecting and assigning participants to groups such that systematic differences between participants in each group exist
Selection by Maturation interaction: Treatment and No-treatment groups, similar at start, have developed differently, even without any treatment
Mortality: Subjects dropping out of the groups before the experiment is finished resulting in differences between groups unrelated to the treatment effects
Instrumentation: Changes in measurement procedures may result in differences between groups that are confused with treatment effects
Testing: After repeated testing of participants, changes in test scores may be the result of practice or knowledge of the procedure rather than treatment effects
History: Extraneous events occurring during the course of the experiment that may affect the participants’ responses on the dependent measure
Analysis techniques
Five possible techniques to analyze the data
Pattern matching
Nonequivalent dependent variables
Rival explanations as patterns
Simpler patterns
Explanation building
Build an explanation about the case
“how” or “why” something has happened
Iterative in nature
Builds on well-developed analytic insight of the researcher
Time-series analysis
Simple or multiple
Trends, structural changes
Chronologies - Putting events in chronological order and covering many types of variables: can be richer and more insightful
Logic models
Deliberate stipulation of a complex chain of events over an extended period of time
Cause-effect-cause-effect patterns
Can be at individual / organizational / … level
Cross-case synthesis
When dealing with multiple-case setup
Cases part of the same or different studies
Can you find patterns across these different cases when similar conditions?
Can you find the opposite outcomes in cases with opposite conditions?
5. Preparing the report:
Goal of Case Reporting:
Convey vicarious experience to readers about the phenomenon of interest
Render solutions to problems publicly accessible
Facilitate readers to apply the experience in real-life situations
Components of ‘Good’ Case Reporting:
Displaying sufficient evidence for claims
Communicating case boundaries
Paying special attention to conflicting propositions
Techniques for Case Reporting
Chronological Recounting
Story Telling
Reader-Centric
Characteristics of case study research: Most important characteristics
Phenomenon of interest is examined in natural setting
Data collected via multiple means
One or few entities (e.g., person, group or organization) is examined
Complexity of the unit is studied intensively
Investigator(s) should have a receptive attitude towards exploration
No experimental controls or manipulation are involved
Investigator(s) may or may not specify the set of dependent and independent variables in advance
Derived findings depend heavily on the integrative powers of investigator(s)
Changes in site selection and data collection methods could take place as investigator(s) develops and/or updates hypotheses
Useful in the investigation of “how” and “why” questions
Focus on contemporary events
Advantages / Disadvantages of case study research depend on:
Type of research question(s) that can be answered
Control over actions, decisions and events
Focus on contemporary phenomena as opposed to historical ones
Preferred research technique when:
Answering “How” and “Why” questions
Little control over actions, decisions and events
Focus on contemporary phenomena within natural settings
Action research: Interactive inquiry process that balances collaborative problem solving with data-driven analysis to understand underlying causes and facilitate future predictions.
The researcher deliberately intervenes while at the same time studying the effect of that intervention.
Action research is a cyclical process with 5 phases:
Diagnosing - Identification of problem(s) and issue(s) related to phenomenon of interest
Action Planning - Development of an action plan to specify activities for resolving identified problem(s)
Action Taking - Implementation of planned actions
Evaluating - Assessment of actions taken and their impact on identified problem(s)
Specifying Learning - Consolidation of knowledge from preceding phases to determine future course of action
Research is action research if the following elements are present (to some degree):
Purpose and value choice - Scientific enquiry and practical problem solving
Contextual focus - Research in a ‘real-world’ setting, focus on wider context
Change-based data and sense-making - Data allow for tracking the consequences of changes, and for their interpretation
Participation in the research process - Involving the people dealing with the ‘real-world’ problem at hand
Knowledge diffusion - Translating findings into scientific literature, and linking to existing literature
Advantage of Action research:
Helps to ensure practical relevance - Not too theoretical, but direct impact
Disadvantages of Action research:
Difficult to combine action with research
Tendency to overstate the importance of the intervention
Tendency to overstate the contribution to academic research
Observational research: Method of data collection by noting a phenomenon, often with instruments, in its natural setting, while recording it for scientific purposes.
Large set of different methods:
Personal observation
Mechanical observation (Scanner - Audimeter - (eye)camera)
Audit
Trace analysis
Classification of observation:
Structured versus unstructured: How prespecified is the observation?
Disguised versus undisguised: Bias?
Natural versus contrived: Own kitchen or test kitchen? (Dis)Advantages of natural?
Which are true for the Kitchen Stories example?
Roles of the observer:
Complete Observer:
Scrutinizes the phenomenon of interest without direct physical intervention or interaction - Watching other people ‘from the outside’ - People are not aware of the fact that they are being observed
Ideal objectivity: no possibility that results are affected by actions of observer
Ethical issues
Observer-as-Participant
People know that they are being observed
Observer only has a role as researcher
No direct action by researcher: only observation
Participant-as-Observer
Takes part in the action (up to a certain level), creates connection with the people observed
Actor and researcher
People still aware of research focus
Complete Participant
Observer is fully engaged and takes full part in the action
People are not always aware of (hidden) research agenda
Ethical issues
Levels of membership: Researcher may adopt different levels of engagement in community when doing participant observational research
Peripheral membership:
Active membership:
Complete membership:
Observational elements:
Space – physical place or places
Actor – people involved
Activity – set of related acts that people do
Object – physical things that are present
Act – single actions that people do
Event – a set of related activities that people carry out
Time – sequencing that takes place over time
Goal – things people are trying to accomplish
Feeling – emotions felt and expressed
See slide 16 for an example
Trace analysis: People leave traces!
E.g. tiles erosion: signs of popularity
Radio dials at car service
Shopping patterns in supermarkets
Browsing behaviour on the Internet - Tracking cookies!
Quality of observational research: 5 main criteria:
Objectivity - Do conclusions come from the data, or from biases induced by the researcher?
Reliability - Has the research process been consistent and stable over time and across methods and researchers?
Internal validity - Do the conclusions make sense and are they credible?
External validity - Are the conclusions relevant beyond the study itself?
Utilization - Can the findings be used for actions?
Advantages of Observational Research:
Ability to observe phenomenon in natural setting
Avoids possibility of self-censorship
Disadvantages of Observational Research:
Access
Labor intensive
Possible bias induced by researcher
But: triangulation is possible!
Documents: Many different types: Written materials, Pictures, Diagrams, Photographs, Videos, Television programmes, Websites/electronic documents, Software etc.
Quality of documents: Four main criteria
Authenticity - Is the object what it claims it is?
Credibility - To what extent can the author be believed?
Representativeness - To what extent is the document a good sample of a larger set of documents?
Meaning - How should the document be interpreted and understood?
Advantages of documents:
Relatively cheap
Easy to access
Make things visible and traceable
Disadvantages of documents:
Some types of documents are hard to get access to
Assessment of the authenticity, credibility, representativeness and meaning can be difficult - E.g. when no access to the original author
See slide 30/31 Huge amounts of money invested in advertising:
Good insights about returns to adspend, but what about content?
Much anecdotal evidence
Much experimental evidence on “soft” outcomes - Mainly on overlap - Some on variation
No longitudinal evidence, no evidence on “hard” outcomes
Should brands try to be consistent in their message over time?
Should brands try to be different from competitors?
Findings of Linguistic Inquiry and Word Count (LIWC2007) (Pennebaker et al. 2007):
It pays off to clearly link the product to the category, and to resemble your competitors in that sense
Not just once, but in a sustained way
When positioning the product, it is important to be clearly different than competitors
What makes the product so unique? - Look into those product characteristics that do matter to customers, and stress unique features
When positioning the brand, it is important to be consistent over time
What is the brand identity/image? - What are the personal concerns of customers the brand appeals to?
Internet and Social Media:
Enormous amounts of information are out there
People leave traces - Browsing behaviour on the Internet
People express opinions
Be aware of possibly biased information!
New opportunities › Usage of Internet focus groups/panels
Testing new product concepts
Testing design of new websites
Same panel selection criteria as in ordinary focus groups/panels apply.
Science = (from the Latin scientia, meaning "knowledge") builds and organizes knowledge in the form of testable explanations and predictions about the natural world.
Astronomy: is a natural science that is the study of celestial objects.
Astrology: is the belief system which claims that human affairs are correlated with the positions of celestial objects.
Going beyond the difference qualitative/quantitative, at a higher level, when choosing a research approach the choice is also about the nature of scientific research:
applied scientific research(design-focused)
problem-solving questions
specific for one situation
interventions
structure: regulative cycle(based on Van Strien, 1997)
Imagine the following business problem- management question: “Why customer data is not yet actively shared and used, despite the large amount of available data within company A?”
apply regulative cycle.
In the design phase we are finding solutions to solve the problem.
For instance, the solution chosen is to facilitate frequent meetings with project managers.
The management decided to organize meetings on weekly basis between all project managers of all 8 departments.
As implementation, the management offered the possibility of a (free) lunch during 45 minutes, where all incoming customer data are plenary discussed with project managers
After 6 months, this initiative is evaluated and checked whether indeed, customer data is appropriately shared and whether (new) knowledge emerged between departments.
theoretical scientific research (theory-based)
Example: “What is the influence of national cultures in managing multinational companies” (Hofstede)
Quality criteria in research:
Validity: do I measure what I want to measure?
Reliability: repeated measurement: will I find the same values?
How to assess validity & reliability in qualitative research?
translates into: How to really convince that qualitative reasearch methods are not biased or innacurate or imprecise? (Miles and Huberman, 1994, pp. 277)
Assessing validity & reliability in qualitative research (Flick, 2009):
applying traditional validity and reliability criteria, if possible
reformulating traditional validity and reliability criteria
Reliability: the process of the study is consistent, reasonably stable over time, across researchers and methods, and replicable.
Criteria used in qualitative research:
Procedural reliability (text analysis/ interviews)
Inter-rater reliability
Procedural reliability (a classic approach)
Use low-inference descriptors (Record/ document as concrete and detailed as possible, make the research process as transparent as possible (concerning methods, data and theory)
Standardise procedures in measuring instruments (interviews/ field notes/ data analysis)
Procedural reliability – Text analysis:
Reliability issues arise in coding/categorizing
Codes/categories are developed in a standardized way (themes, sub-themes, topics etc.)
Procedural reliability – Interviews:
At respondent site: understanding questions in the same way
At interviewer site:
pre-testing interviews
training interviewer(s)
Using (semi-)closed questions
Using inter-rater reliability
Using low-inference descriptors
Recording all face-to-face interviews (accurate transcription)
Presenting (long) extracts of data into the research report
Inter-rater reliability:
Same data given to a number of analysts
The analysts analyze the data according to an agreed set of codes/categories
Their reports are examined and any differences are discussed and ironed out
Validity: What do you say that you have seen, is it actually happening in reality?
What is reality? (see textbook chapter 1) and how is reality approached in your findings?
3 errors:
Type 1: To see relations/concepts when they do not exist
Type 2: To neglect relations/concepts when they exist
Type 3: To ask the wrong questions
Validity types
Content validity: Do we measure what we want to measure?
Internal validity: (credibility/authenticity): Do the finding of the study make sense in context of the study? Are they credible to the people we study and to the readers of the research?
External validity: (transferability/generalizability): Are the finding transferable to other contexts? How far can they be generalized?
Procedural validity: - In production of the data
In presentation and interpretation of the phenomenon
How do we ensure procedural validity?
In case of interviews (based on Wolcott, quoted in Flick 2009):
Do not talk yourself too much but listen
Take exact notes
Write early, while facts are still fresh
Show notes to the interviewee
Provide a complete and detailed report
Other criteria for validity:
trustworthiness/credibility - by triangulation
transferability - by generalization of results
By reporting with sufficient amount of detail and providing arguments may increase both: validity and reliability.
Triangulation: combining several qualitative methods or several qualitative methods within or not within the same case.
Types (Flick, 2009):
Data triangulation: different data sources are used
Investigator triangulation: different investigators are used to analyze data
Theory triangulation: different perspective/hypotheses are used
Methodological triangulation: - Within method: e.g. use of different subscales for measurement of an item in a questionnaire
Between method: e.g. combination of a survey method with a semi-structured interview
Triangulation: validity assessment tool: if findings obtained by all these methods correspond and draw similar conclusions, the validity of those findings and conclusions has been established.
Problem: accounts are ‘situated’ in particular contexts: Therefore methods, drawn from different theories, cannot give an ‘objective truth’.
Triangulation is not a way of obtaining ‘true’ reading but rather a strategy that adds rigor, breath, complexity, richness and depth to any inquiry.
Generalization in qualitative research: developing a theory involves the generalization of concepts and relations.
Problem: findings are often bounded to a specific context
Choice of cases (sampling) is important:
Again: Constant comparative method (grounded theory)
Generalization in qualitative research is the gradual transfer of findings from one particular case study to more general relations.
However:
Generalization is simplifying our understanding about a phenomenon
Generalization is a push towards breath, and qualitative research is often more concerned with depth
Generalization should be used when it seems to fit the research aims. Not a precondition in qualitative research.
See slide 32: Figure: Target population, operational population, sampling frame, sample
Sampling in qualitative research (See lecture 2: Research design)
Qualitative research: mostly non-probability sampling (units are not statistically representative)
Most deliberate: purposeful sampling = criterion based
Homogeneous and heterogeneous sampling, typical case sampling, extreme case or deviant sampling
Convenience sampling (ease of access) & Opportunistic sampling (as events arise)
Theoretical sampling: type of purposive sampling. Cases are selected to the base of development of new concepts.
In connection with grounded theory (in GT, theory emerges from qualitative data).
after first data analysis step, additional data collection stages may occur: more interviewees, more groups.
Data analysis and data collection continues, until theoretical saturation is reached.
Combining qualitative and quantitative research:
Sequencing: different approaches in different phases of the research.
e.g. qualitative in Problem Analysis and Conceptual Analysis (formulating hypotheses) and quantitative in Empirical Analysis (testing hypotheses)
Triangulation: for validation purposes. Combining several qualitative methods or several qualitative and quantitative methods.
Within or not within the same case
Quantitative methods (data: numbers): survey, experiment, simulation, structured interview, statistics as secondary data
Qualitative methods (data: text): action research, case study, grounded theory, narratives.
Lecture 7 was a wrap-up, no more extra information included then in the lectures before (just a summary). There was no guest lecture.
Difference between text book and lecture slides:
Sometimes terminology is slightly different but same notions:
Book (p.28-33): functions of qualitative research: contextual, explanatory, evaluative, generative.
Lecture 2 – slide 25: research goals: description, exploration, explanation
In textbook they distinguish between:
Essential knowledge: concepts, notions, theories, models, methods...
know this!
Support knowledge: explanation, examples, illustrations..
Be able to provide comparable explanations in your own words!
JoHo can really use your help! Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world
Als je probeert uit te leggen waarom de onderzoeksvraag “Hoe tevreden is men over de werktijden” is NIET kwalitatief is, kan dat alleen op basis van welk van onderstaande argumenten? Deze vraagstelling kan niet als kwalitatief worden gezien, omdat …
de mening van respondenten wordt gevraagd
de vraag refereert naar “men”
de vraag begint met het woordje “hoe”
de vraag niet begint met “waarom”
Bij het uitvoeren van zowel kwalitatief als kwantitatief onderzoek is conceptualisatie en operationalisatie erg belangrijk. Welk van onderstaande beweringen gaat over operationalisatie?
Synoniem beschrijven
Woordenboekdefinitie geven
Bepalen welke empirische observaties moeten worden gemeten
Een manier van denken over een onderwerp vaststellen
Welke van de onderstaande stellingen past NIET bij ‘positivisme?
De werkelijkheid is objectief waar te nemen
Uit wetenschappelijke theorieën kunnen hypothesen worden gedestilleerd die empirisch getoetst moeten worden
Een kernwoord is ‘understanding’ of ‘verstehen’ van een werkelijkheid
Kennis komt voort uit observeerbare fenomenen
In het boek en tijdens de cursus, zoals ook uitgelegd tijdens de hoorcolleges, wordt uitgegaan van ‘subtle realism’. Welk van ondertsaande beweringen past het beste bij dit standpunt?
Er bestaat geen externe realiteit onafhankelijk van onze overtuigingen en begrip
De sociale wereld is alleen toegankelijk middels de interpretaties van de respondent
Alleen de fysieke materie van de wereld is ‘real’
Er is geen ‘single shared social reality’
Kwalitatief onderzoek heeft volgens het boek meerdere functies die zijn gecategoriseerd in vier soorten. Bij welke functie hoort de volgende onderzoeksvraag: “Wat wordt door universiteitsdocenten verstaan onder professionele ruimte?”
Contextual
Explanatory
Evaluative
Generative
Stel dat uit voorgaand onderzoek blijkt dat universiteitsdocenten verschillend denken over wat hun professionele ruimte is, en het bestuur een focusgroep houdt om mogelijke interventies te vinden die kunnen helpen in de toekomst tot eenzelfde interpretatie te komen. Over welk type onderzoek hebben we het dan?
Contextual
Explanatory
Evaluative
Generative
Welk van onderstaande begrippen wordt in het boek gezien als een epistemologisch standpunt, in tegenstelling tot de andere drie die geschaard worden onder de ontologische standpunten?
Realism
Materialism
Interpretivism
Relativism
Het maken van de interviewleidraad (‘topic guide’) behoort tot de fase van
Het opstellen van de onderzoeksvraag
Het kiezen van de onderzoeksmethode
Het
Bij kwantitatief onderzoek gaat het om dingen meten die geteld kunnen worden en opgedeeld in categorieën. Het gaat over wat men vindt, het eest voorkomend gedrag en hoe tevreden mijn klant of werknemer is.
Bij kwalitatief onderzoek ligt de focus op de ervaringen en meningen van mensen. Hier gaat het om eigen woorden en perspectieven, dit kan men ook zien als bijvoorbeeld redenen en motieven en inzichten in complexiteit fenomeen.
Bij een kwalitatief onderzoek gaat het om 'hoe' en 'waarom'.
Verschillende filosofische perspectieven staan tegen over elkaar en zijn:
Positivisme: Hier is de werkelijkheid is objectief waar te nemen, de wetenschap is puur en alleen gericht op het testen van theorie. Dit past bij kwantitatief onderzoek.
Interpretivisme: Hier is de aanname dat werkelijkheid alleen toegankelijk kan worden gemaakt m.b.v. een sociale constructie, dit zou bijvoorbeeld taal kunnen zijn. Hier is onderzoek gericht op begrijpen van betekenis en zin. Dit past bij kwalitatief onderzoek.
'Subtle Realism': Hier is de aanname dat de sociale wereld onafhankelijk bestaat van individuele meningen, maar alleen toegankelijk is via de meningen en interpretaties van de respondent. Er is dus altijd vertekening van de werkelijkheid, maar we doen ons best.
'Pragmatisme': Bij dit beeld, gelooft men dat we de meest geschikte onderzoeksmethode kiezen. We moeten de perspectieven van mensen zo objectief en neutraal mogelijk begrijpen. Het maakt dus niet uit of je kwantitatief of kwalitatief meet, als het maar het beste bij de vraag past.
Bij een waardevol en valide onderzoek, is er sprake van objectiviteit. Dit betekent dat de informatie niet beïnvloed is door gevoelens van een individu. Dit zijn dingen die gewoon kloppen.
Objectiviteit is te vinden bij kwantitatief onderzoek. Dit komt vanuit de natuurwetenschappen en meet dus waarden waarover niet te discussiëren valt. Hierbij kan je alleen maar tot een bepaald niveau missen.
Bij kwantitatief onderzoek is dit minder het geval en gaat het vooral om de diepgang van een onderzoek. Het is dus wel meer subjectief, wat minder exacte antwoorden.....read more
The goal of qualitative research methods (QRM): to perform all stages of a qualitative research project, i.e. perform interviews in a systematic and methodologically responsible way.
Essentials of interviewing:
A good interview is a focus on the subject’s (Interviewee/IE) world
What to do?:
Create a collaborative atmosphere The more comfortable interviewees feel, the more they are prepared to open up and talk, the better your data are likely to be
Interviewer’s (IR) contribution: prepare both the task-oriented and the interaction-oriented role before the interview
Develop a genuine interest in the interviewee during the interview, listen, prompt, encourage and direct.
Interviewers first attention generally goes to content (focus on data). What you SHOULD focus on: interviewee as a way to collect those data IE mirrors the IR’s attitude: if lack of interest, so does IE.
Basic classification of interviews:
a. structured
b. unstructured
c. semi-structured
a.) Structured interview:
Advantage: consistency during multiple interviews
Disadvantage: no space for new insights on the basis of voluntary information from the interviewee
The most highly structured interview format is undoubtedly the survey questionnaire: an interview guide that pre-specifies both the content and the possible responses to each question. It involves the use of pre-formulated questions.
The whole idea of structured interviews is to ensure consistency across multiple interviews. They tend to be in market research, polling, telephone interviews and with intercept research such as in shopping centres.
b.) Unstructured interviews:
The opposite of structured interviews. There may not be a time limit. Questions are only used to jog someone’s memory.
Characteristics:
very few pre-formulated questions.
no attempt to maintain consistency across interviews
advantage: allows the interviewee to talk freely and tell you everything he or she considers important
disadvantage: if interviewee is not talkative, you may end up with too little; if the interviewee is very talkative, you may end up with too much. Also hard to compare answers by different Ies
c.) Semi-structured interview: in-depth interview (What we have to do in this course):
Advantages: takes the best of structured/unstructured approaches while minimizing the risks; it is cost-efficient and relatively easy to conduct.
Limitations:
There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.
Do you want to share your summaries with JoHo WorldSupporter and its visitors?
Field of study
Add new contribution