Quiz_ Progress Quiz Week 2
11398 Lecture 2-4
8/3/21
1
Lecture Week 2-4: Reviews and meta-analyses
Introduction to Research in the Health Sciences
1
Research designs
Descriptive
Co
elational
Quasi-experimental
Experimental
Review
Meta-analytic
2
2
Reviews
• Examine previous literature on a particular topic. Considered
secondary sources, as no new raw data is collected. Reviews
vary substantially in how systematically the literature is
collected and collated.
• Examples:
• Na
ative literature review
• Systematic literature review
3
3
8/3/21
2
Reviews:
Systematic literature review
• A systematic review answers a defined research question by using
systematic methods to collect and summarise all empirical evidence
that fits pre-specified eligibility criteria.
• Tell the reader what the research question is, describe exactly how
and from which databases you will search for studies, use specific
criteria to evaluate the studies, collect data from the studies, report
the findings, interpret the findings. Sometimes updated versions are
published later.
• Pros:
• Rigorous, replicable assessment of findings on a certain topic – the best
evidence available to answer a particular research question
• Cons:
• Tends to be very na
ow in focus, so answers one question very well, but
doesn’t necessarily give the bigger picture
4
4
https:
onlineli
ary.wiley.com/doi/full/10.1111/inm.12390
5
5
6
6
https:
onlineli
ary.wiley.com/doi/full/10.1111/inm.12390
8/3/21
3
https:
onlineli
ary.wiley.com/doi/full/10.1111/ XXXXXXXXXX
7
7
8
8
Meta-analyses
• Pull together and statistically analyse data from other studies to
generate a findings summarising a particular topic. Often reveals
patterns or subgroupings within findings in a field. Classified as a
secondary source.
• Typically conduct a systematic review and a meta-analysis – the
meta-analysis is the further statistical analysis of the findings of the
systematic review.
• Pros:
• Objective assessment of evidence
• Generates new outcomes – e.g., an effect size for that intervention
• Cons:
• Selection of studies may be biased; publication bias can affect conclusions
9
9
https:
onlineli
ary.wiley.com/doi/full/10.1111/ XXXXXXXXXX
8/3/21
4
https:
www.sciencedirect.com/science/article/pii/S XXXXXXXXXX?via%3Dihu
10
10
11
11
Research designs: Which is best?
• A theory can be explored using a wide range of designs – one
design isn’t ’better’ than another, they just provide different
ways of investigating the theory
• For example, let’s say you wanted to investigate the theory that
exposure to traumatic events leads to a higher likelihood of
addiction due to structural changes in the
ain?
• You could answer this using any of the designs we’ve discussed
so far (and many more!)…
12
12
https:
www.sciencedirect.com/science/article/pii/S XXXXXXXXXX?via%3Dihu
8/3/21
5
Descriptive: Case study
• Carefully watch and take notes of a person’s behaviour after they have
experienced a traumatic event. Interview them about their experiences.
• Report how the person’s behaviour changes over time, how they react to different
events and situations; obtain detailed information about this particular case.
Co
elational: Survey
• Hand out a questionnaire to 100 people. Ask how many traumatic events they’ve
experienced and ask how much alcohol they consume.
• Conduct analyses to work out what the relationship is between number of
traumatic events and alcohol consumption.
• Write up the findings, focusing on whether a greater number of traumatic events
in someone’s history is associated with higher alcohol consumption
13
13
Quasi-experimental: Non-randomised experiment
• Recruit two groups of people – people who’ve experienced traumatic event (treatment)
and those that haven’t (control). Assess the differences between the two groups in terms
of the amount of alcohol they consume.
• IV = exposure to traumatic events (Group A includes only people who have
experienced a traumatic event; Group B includes only people who have not
experienced a traumatic event)
• DV = alcohol consumption (Provide alcohol to both groups, measure how much
people drink)
Experiment: Randomised controlled trial
• We can’t realistically/ethically manipulate the variable we’re interested in (traumatic
events)
• Modify the research question: Higher levels of stress cause people to drink more alcohol,
so manipulate stress level and measure alcohol consumption:
• IV = exposure to stress (Group A put in a high stress environment with loud noises, hot,
ight lights; Group B put in a normal stress environment)
• DV = alcohol consumption (Provide alcohol to both groups, measure how much people
drink)
14
14
Review: Systematic review
• Define the topic of the review very specifically – so a review of only peer-reviewed
articles between 1960 and the present reporting randomised trials where the
experiment involved manipulating stress as the IV and the DV was alcohol
consumption.
• Conduct a systematic search of relevant databases using these criteria, and then
summarise the findings in a single article. Can then draw conclusions about the
cu
ent state of evidence - does the data cu
ently suggest that stress affect
alcohol consumption?
Meta-analysis
• Define the topic of the meta-analysis very specifically (as above) but also conduct
statistical analyses on the data from the studies. So for example, could conclude
that there is on average there is a significant difference of .52 standard units of
alcohol consumption between stressed and non-stressed participants, and an
effect size of .27.
15
8/3/21
6
Other ways of classifying research
•High vs low constraint
• Exploratory vs. confirmatory
• Fixed vs. flexible
16
16
11398 Lecture 2-5
8/3/21
1
Lecture Week 2-5: Operationalisation
Introduction to Research in the Health Sciences
1
Operationalising variables
• When you operationalise a variable you are defining what you
are measuring and how
• You are deciding, for the purposes of the research you are
conducting, specifically how you would manipulate or measure
that variable
• This might involve a specific question in a survey, with a particular
esponse scale. Or using equipment in a certain way under certain
circumstances. Or asking someone to perform a particular task.
• e.g., let’s say want to measure how healthy someone is, how exactly
would you do this?
• What about measuring stress? Performance? Disadvantage? Deaths?
2
2
Operationalising variables
• Often operationalisation is driven by logistical/contextual issues
as well as theoretical approach
• i.e., what can reasonably be measured? What equipment do you have?
What funding do you have? Are participants able/willing to be
measured in this way?
• How you operationalise the variable impacts what statistical
tests you can run during analysis
• e.g., age in categories (20-30; XXXXXXXXXXversus as a number (29 yrs)
• e.g., ‘experiencing pain’ (yes/no) versus as a scale (1-7)
• Huge implications for the conclusions that can be drawn
• e.g., If you have operationalised ‘health’ as BMI, does this reflect all
aspects of health?
3
3
8/3/21
2
Operationalising variables: Example
• Aggression can be operationalised in many ways, including:
• Criminal record
• Self-reported history of violent incidents (would need to define
’violent’)
• Number of violent incidents displayed when playing as a video game
characte
• Allocation of spicy sauce to another person
• Neural activation in specific regions of the
ain
• How fast they recognize aggressive words
• Aggression survey
• How much they help other people
• Internet
owser history
• Desensitisation to violent images
4
4
5
5
Operationalising variables:
Relevance-sensitivity trade-off
• What we’re looking for in a variable is a measurement that is
sensitive enough to show a difference that you’re interested in
(i.e., don’t measure something that is unlikely to show change)
• But you also need to be careful not to pick a variable that is so
specific that it isn’t relevant beyond the study to other contexts
(i.e., don’t measure something that isn’t meaningful in the
wider world)
6
6
8/3/21
3
Relevance-sensitivity trade-off: Examples
• Let’s say we want to measure the effect of alcohol consumption
on the likelihood of having an accident when driving:
• Randomise 40 people to drunk or sober conditions (IV with 2 levels)
and measure number of accidents they have (DV)
• BUT accidents are rare! In a single study probably won’t be any. So this
DV may not be sensitive enough.
• We could instead measure the number of times people look away from
the road. But does the number of times someone ‘looks away’ actually
tell us about accidents? This DV may not be relevant enough.
• How to measure anxiety?
• How to measure falls?
7
7
Tips for operationalising variables
8
Be realistic -
think about
logistics in in
your study
(where and how
will you actually
gather the data?)
Use existing
measures (e.g.,
questionnaires,
equipment)
(they're already
tried and tested!)
Try it out
(often it’s only
when you try to
use a measure that
you realise it’s no
good!)
‘Shadow’ data
collection in
similar studies
(ask other
esearchers what
they are doing, and
if you can observe!)
Use two
measures of the
same variable if
possible
(this gives you
greater confidence
and insight!)
Look across fields
(do researchers in
other areas have a
different way of
measuring that
variable you could
use?)
8
Lecture summary
• Terminology
• Research designs
• Descriptive
• Co
elational
• Quasi-experimental
• Experimental
• Review
• Meta-analytic
• Operationalisation
9
9
8/3/21
4
Next lecture
10
• Research design issues:
• Sampling
• Validity
• Reliability
• Additional design issues in surveys
• Additional design issues in experiments
10
11398 Lecture 2-2
8/3/21
1
Lecture Week 2-2: Descriptive and co
elational designs
Introduction to Research in the Health Sciences
1
Research designs
• There are
countless ways
esearch can be
conducted, and
different ways
these
approaches can
e grouped
together.
• One common
categorisation of
esearch designs
is:
Descriptive
Co
elational
Quasi-experimental
Experimental
Review
Meta-analytic
2
2
Descriptive research designs
• Don’t manipulate any variables, take advantage of the natural
flow of behavio
• Some are highly ‘flexible’ (allow the researcher to take
advantage of unexpected occu
ences and new ideas)
• Examples:
• Archival research
• Naturalistic observation
• Surveys
• Program evaluation
• Case studies
3
3
8/3/21
2
Descriptive research designs:
Case studies
• Focus on the behaviour of a single individual
• Often individuals in traumatic or extreme situations
• Pros:
• Rich source information
• Allows researchers to study rare disorders or circumstances
• Cons:
• Can’t generalise
• Don’t know for sure what causes the individual’s behaviour – the
event/disorder, some personality characteristic, or some combination
of the two
4
4
https:
onlineli
ary.wiley.com/doi/10.1111/ijun.12124
5
5
https:
www.tandfonline.com/doi/abs/10.1080/ XXXXXXXXXX XXXXXXXXXX
6
6
https:
onlineli
ary.wiley.com/doi/10.1111/ijun.12124
https:
www.tandfonline.com/doi/abs/10.1080/ XXXXXXXXXX XXXXXXXXXX
8/3/21
3
Co
elational research designs
• Look at the relationship between variables, but where the
variables are not under the researcher’s direct control (for
logistical or ethical reasons)
• Examples:
• Case-control studies
• Observational research