PowerPoint Presentation
Safe Motherhood Program in Indonesia and Maternal Mortality
Achievement and Challenges
By:
Chindy Rosari Tefa
Tracy Angelyn Wandau
The History of Safe Motherhood Program
Safe Motherhood Program: Global Tragets
Maternal Mortality in Indonesia
Key Issues in Indonesia
Indonesia’s Safe Motherhood Program
Project development objectives:
Improving demand for and utilization of quality maternal health services
Strengthening the sustainability of maternal health services at the village level
Improving quality of family planning services
Preparing adolescents to lead a healthy reproductive life
Target Populations:
Women of childbearing age
Families and newborn children
Adolescents
Achievements
Challenges
Conclusion
Thank You
Any Questions?
doi:10.1016/j.jmwh XXXXXXXXXX
I
A
g
s
t
m
t
a
s
i
o
e
e
t
c
f
o
a
t
F
m
S
e
s
p
a
e
i
A
o
C
3
©
I
Safe Motherhood Program Evaluation:
Theory and Practice
Lesley Milne, RM, RMT, MSc, Graham Scotland, MSc, Nargiz Tagiyeva-Milne, MD, MPH,
and Julia Hussein, MRCOG, MSc
Debate on the evaluation of safe motherhood programs has mainly focused on the outcome or process
measure to be used. Less attention is paid to the application of different approaches to evaluation. This article
eviews cu
ent theories of evaluation and provides examples of the extent to which these theories have been
applied in the actual practice of evaluation. Most evaluations use multiple methods and approaches, but the
ationale and intention behind these choices are often not made explicit. Factors are identified that need to
e taken into consideration when planning and conducting safe motherhood program evaluations. Safe
motherhood programs are complex interventions, requiring evaluation by different theoretical approaches
and multiple methods. Awareness of these approaches will allow health professionals to plan for evaluation
and to use evaluation findings more effectively. If cognizant of the different approaches to evalua-
tion, evaluation frameworks can be developed to improve assessment of the effectiveness of these
programs. J Midwifery Womens Health 2004;49:338–344 © 2004 by the American College of Nurse-Midwives.
keywords: safe motherhood, evaluation, interventions, program
E
T
s
a
v
m
e
a
a
w
i
d
l
I
c
c
Ø
q
t
e
o
c
o
c
e
i
a
a
p
s
NTRODUCTION
lthough the need to evaluate safe motherhood programs is
enerally agreed upon, the means by which the evaluation
hould be conducted remains more controversial. Due to
he difficulties in measuring maternal mortality, the safe
otherhood evaluation debate has largely focused on the
ype of evaluation measure to be used. Process measures
e being recommended for monitoring and evaluation of
afe motherhood programs,1–3 although measuring mortal-
ty and mo
idity outcomes is still considered necessary.4,5
By comparison, less discussion takes place on the vari-
us approaches to evaluation. These are dependent on the
valuators’ assumptions about what counts as valid knowl-
dge (see below), the methods that they are familiar with,
he overall purpose of the evaluation, and time and resource
onstraints.6–8 Approaches to evaluation have traditionally
een classified into summative (impact or outcome) and
ormative (process) evaluation.6,8 Beyond these, a variety
f additional approaches have been described.
This article reviews the theory of health care evaluation
nd compares this with approaches used in practice within
he field of safe motherhood. The objectives are threefold.
irst, this article seeks to raise awareness in the safe
otherhood community regarding evaluation practices.
econd, highlighting the complexities inherent within the
valuation process will facilitate appreciation of the
trengths and weaknesses of different evaluation ap-
oaches. Finally, knowledge of the range of evaluation
pproaches should encourage health care professionals to
ngage in planning evaluations suited to the design and
mplementation needs of programs.
ddress co
espondence to Julia Hussein, Dugald Baird Centre for Research
n Women’s Health, University of Aberdeen, Aberdeen Maternity Hospital,
wornhill Road, Aberdeen AB25 2ZL, Scotland. E-mail: XXXXXXXXXX
38
2004 by the American College of Nurse-Midwives
ssued by Elsevier Inc.
VALUATION THEORY
heoretical paradigms for evaluation comprise two main
treams: experimental and phenomenologic. Experimental
pproaches are traditionally regarded as necessary to pro-
ide rigorous evidence of effectiveness. Arguably, experi-
ental approaches cannot fully capture the complexity of
valuating the large number of interventions that constitute
program; thus, it is important to consider the use of othe
pproaches in evaluation. To do so, electronic databases
ere searched by using key words such as complex
nterventions and community-based interventions, com-
ined with terms relating to evaluation theory and study
esign. The Web sites of relevant organizations and online
i
ary catalogues were also scanned for relevant material.
dentified articles, reports, and books that were primarily
oncerned with methods for evaluating the effectiveness of
omplex health interventions were included.
For the evaluation of health care programs and policies,
vretveit describes two main paradigms, which are defined
y the different standpoints they take on the epistemologic
uestion, “what is knowledge?”7 At one end of a spectrum,
he positivist or experimental paradigm assumes that an
xternal system of cause and effect exists, which cannot be
served directly but can be infe
ed through carefully
ontrolled observation. Thus, positivists tend to argue that
nly the well-designed randomized controlled trial (RCT)
an produce valid evidence of effectiveness. At the othe
nd of the spectrum is the phenomenologic (or constructiv-
st) paradigm that takes the view that truth is always
ttached to some standpoint rather than being external to
ny one group. Thus, constructivists favor qualitative ap-
oaches designed to elicit the perceptions and opinions of
takeholders. In between these two extremes there are those
ho advocate the integration of quantitative and qualitative
Volume 49, No. 4, July/August 2004
XXXXXXXXXX/04/$30.00 • doi:10.1016/j.jmwh XXXXXXXXXX
m
o
m
e
fi
f
t
S
a
p
s
R
t
d
p
t
c
c
p
i
c
c
a
f
E
w
E
a
d
i
m
i
m
o
a
c
w
a
d
i
P
e
e
p
p
g
“
fi
m
u
d
L
D
A
G
B
N
D
A
J
a
A
J
ethods to benefit from the different insights each has to
ffer.
The experimental paradigm has been dominant in the
ovement of evidence-based practice with much emphasis
eing placed on the RCT as the “gold standard” for the
valuation of single health care interventions. Systematic
eviews of effectiveness often include only RCTs.9 In the
eld of safe motherhood, the importance of cluster RCTs
or program evaluation has been emphasized.10 However,
he supremacy of the RCT is not universally agreed upon.
ome have argued that the strict evidence-grading system
dopted by systematic reviewers can result in a wealth of
otentially useful evidence being discarded from the deci-
ion-making process.11,12 Others have pointed out that
CTs of complex interventions are not always feasible due
o ethical or logistical considerations.13 To overcome such
ifficulties, quasi-experimental approaches have been pro-
osed14 and used.15–19
However, health programs are usually complex interven-
ions consisting of multiple components, aimed at the
ommunity rather than at the individual level.20 These
haracteristics are often found within safe motherhood
ograms and raise particular challenges for evaluation. Fo
nstance, difficulty can arise in establishing how multiple
esley Milne, RM, RMT, MSc, is an IMMPACT Research Fellow at the
ugald Baird Centre for Research on Women’s Health, University of
erdeen.
aham Scotland, MSc, is an IMMPACT Research Assistant at the Dugald
aird Centre for Research on Women’s Health, University of Aberdeen.
argiz Tagiyeva-Milne, MD, MPH, is an IMMPACT Scientific Officer at the
ugald Baird Centre for Research on Women’s Health, University of
erdeen.
ulia Hussein, MRCOG, MSc, is IMMPACT Scientific Coordination Leade
t the Dugald Baird Centre for Research on Women’s Health, University of
Table 1. Characteristics of Different Approaches to Evaluation
Perspective Epistemology
Experimental
(positivist)
A system of cause and effect is assumed to
exist, which cannot be observed directly.
Causation can only be infe
ed through
controlled observations.
Phenomenologic
(constructivist)
Follows the idea that truth is always attache
some standpoint rather than being extern
any one group.
Pragmatic (action,
user-focused)
Regards as valid knowledge, that which is
considered pragmatically acceptable by
decision makers.
Comprehensive
(pluralist)
Takes the view that knowledge produced from
alternative perspectives all add important
insights to events.
Theory-based
(realistic)
Evaluations are built around explicit theories
how interventions work in specific context
Source: Pawson and Tilley, XXXXXXXXXX
aberdeen.
ournal of Midwifery & Women’s Health • www.jmwh.org
omponents interact to produce an effect. Furthermore,
omplex interventions, such as improving the quality of
ntenatal care or the upgrading of emergency obstetric
acilities, are dependent on specific contextual factors.
valuation attempts need to ask how interventions interact
ith the context so that transferable lessons can be learned.
xperimental approaches may not be particularly useful fo
nswering these questions due to the fact that they are
esigned to control for contextual differences between
ntervention and control groups by way of randomization o
atching. For example, a safe motherhood program might
nclude components of training, refe
al, and community
obilization that are implemented simultaneously with the
verall aim of reducing maternal mortality in a poor rural
ea of sub-Saharan Africa. Although a well-designed
luster RCT might be able to establish whether the program
orks as a whole in that context, it will not be able to shed
ny light on how the individual program components have
een affected by the specific environment to produce the
esired effects.
Alternatives to using purely experimental approaches are
ncreasingly described in the health evaluation literature.
awson and Tilley identify five
oad perspectives on the
valuation of complex social programs.21 In addition to the
xperimental and phenomenologic (constructivist) view-
oints already discussed, they identify pragmatic and com-
ehensive (also known as pluralist) perspectives, before
oing on to offer a new approach, which they term
ealistic” evaluation, also described by others as “theory
ased.”22 Table 1 summarizes the characteristics of these
ve different perspectives. The latter three generally use a
ix of quantitative and qualitative methods. However, the
nderlying reason for the combination of methods chosen
iffers according to the underlying perspective of the
Approach
Randomized or quasi-experimental trials with pretest, post-test,
and control group.
Qualitative techniques used to explore the meanings that
stakeholders attach to phenomena, aiming to reconcile different
meanings through a consensual process.
Qualitative and quantitative techniques used to produce the
evidence decision makers need.
Qualitative and quantitative techniques are combined to gain
greater insight into the working of an intervention and to help
define the causal pathways that might exist.
Qualitative and quantitative techniques used to test theories of
context, mechanisms, and outcomes.
d to
al to
of
s.
pproach.
339
a
p
u
a
n
i
a
s
t
m
t
c
h
E
p
t
E
A
a
h
e
d
e
a
M
t
i
m
S
d
g
e
h
a
P
T
p
t
w
a
d
a
p
i
u
m
h
a
a
c
C
e
T
t
n
h
p
o
i
a
a
t
a
u
a
h