Difference between revisions of "Impact Monitoring Guidelines"

From energypedia
***** (***** | *****)
m
***** (***** | *****)
m
Line 2: Line 2:
 
= Overview =
 
= Overview =
  
The Impact Monitoring Group of [[Energising_Development_(EnDev)|Energising Development]] has developed recommendations for sound impact assessments.
+
The Impact Monitoring Group of [[Energising Development (EnDev)|Energising Development]] has developed recommendations for sound impact assessments.
  
 
<u>We recommend to:</u>
 
<u>We recommend to:</u>
*use the recommended [[Indicator_Rural_Electrification_and_Stoves|list of indicators]]
+
*use the recommended [[Indicator Rural Electrification and Stoves|list of indicators]]
 
*conduct baseline studies before project intervention and impact studies after households, social infrastructure or SMEs received access to modern energy
 
*conduct baseline studies before project intervention and impact studies after households, social infrastructure or SMEs received access to modern energy
*use a [[Theory_Based_Approach|theory based approach]] by applying the result chain for developing the study design
+
*use a [[Theory Based Approach|theory based approach]] by applying the result chain for developing the study design
*use a [[Mixed_Methods|mixed methods approach]]: the logic of the comparative advantages of [[Catalogue_of_Methods_for_Impact_Studies|methods]]: i.e. the mix of quantitative (standardised household interviews) and qualitative methods (open / semi-structured interviews with key persons and households or focus group discussions)
+
*use a [[Impact_Evaluation_-_Mixed_Methods|mixed methods approach]]: the logic of the comparative advantages of [[Catalogue of Methods for Impact Studies|methods]]: i.e. the mix of quantitative (standardised household interviews) and qualitative methods (open / semi-structured interviews with key persons and households or focus group discussions)
*include, wherever possible, [[Control_Groups|control groups]] (households which resemble those households with access to modern energy with the exception of this access) into your baseline and impact study
+
*include, wherever possible, [[Control Groups|control groups]] (households which resemble those households with access to modern energy with the exception of this access) into your baseline and impact study
*consider [[Research_Ethics|research ethics]]
+
*consider [[Research Ethics|research ethics]]
  
 
The following tables show in more detail typical steps for baseline and impact studies, which we furthermore recommend and support.
 
The following tables show in more detail typical steps for baseline and impact studies, which we furthermore recommend and support.
Line 20: Line 20:
 
Development interventions are executed with the ultimate aim of changing the living situation in less developed countries. What is often thoroughly investigated are the inputs to a development intervention and its immediate outputs - how much money is spent, how many children are vaccinated, business trainings conducted or latrines installed. Assessing the impacts of a development intervention though - the aforementioned amelioration of living standards that can be ''attributed ''to a particular intervention - is part of a movement towards evidence-based development policy making and accountability. The central question to all impact evaluations "''What is the causal effect of a programme on the variable of interest?" ''can be applied to various policy contexts.
 
Development interventions are executed with the ultimate aim of changing the living situation in less developed countries. What is often thoroughly investigated are the inputs to a development intervention and its immediate outputs - how much money is spent, how many children are vaccinated, business trainings conducted or latrines installed. Assessing the impacts of a development intervention though - the aforementioned amelioration of living standards that can be ''attributed ''to a particular intervention - is part of a movement towards evidence-based development policy making and accountability. The central question to all impact evaluations "''What is the causal effect of a programme on the variable of interest?" ''can be applied to various policy contexts.
  
Initial but crucial steps to every evaluation is the establishment of
+
<u>Initial but crucial steps to every evaluation is the establishment of</u>
 
#questions to be answered within the scope of the given evaluation;
 
#questions to be answered within the scope of the given evaluation;
 
#the theory of change, depicting how, in theory, the intervention is envisaged to achieve results;
 
#the theory of change, depicting how, in theory, the intervention is envisaged to achieve results;
Line 28: Line 28:
  
 
Taken together, the evaluators will hereby arrive at the specific questions to be phrased by the evaluation. Ideally, this will occur in the planning phase of the development intervention, thereby linking implementation and evaluation closely.
 
Taken together, the evaluators will hereby arrive at the specific questions to be phrased by the evaluation. Ideally, this will occur in the planning phase of the development intervention, thereby linking implementation and evaluation closely.
 
<br/>
 
  
 
<br/>
 
<br/>
Line 35: Line 33:
 
= Preparation =
 
= Preparation =
  
{| style="width: 100%;" border="1" cellspacing="0" cellpadding="0"
+
{| cellspacing="0" cellpadding="0" border="1" style="width: 100%"
 
|-
 
|-
| bgcolor="#0000ff" | <span style="color: rgb(255, 255, 255);">'''Tasks'''</span>
+
| bgcolor="#0000ff" | <span style="color: rgb(255, 255, 255)">'''Tasks'''</span>
| bgcolor="#0000ff" | <span style="color: rgb(255, 255, 255);">'''Materials/ Support'''</span>
+
| bgcolor="#0000ff" | <span style="color: rgb(255, 255, 255)">'''Materials/ Support'''</span>
| bgcolor="#0000ff" | <span style="color: rgb(255, 255, 255);">'''Library'''</span>
+
| bgcolor="#0000ff" | <span style="color: rgb(255, 255, 255)">'''Library'''</span>
 
|-
 
|-
 
| <font size="2">Draft Terms of Reference for Consultants / Enumerators</font>
 
| <font size="2">Draft Terms of Reference for Consultants / Enumerators</font>
Line 70: Line 68:
  
 
|  
 
|  
*[[MDGs_and_Result_Chains|Sample Results Chain]]
+
*[[MDGs and Result Chains|Sample Results Chain]]
 
*<font size="2">[[Indicator Rural Electrification and Stoves|Impact Criteria and indicators]]</font>
 
*<font size="2">[[Indicator Rural Electrification and Stoves|Impact Criteria and indicators]]</font>
  
Line 96: Line 94:
 
| <font size="2">Sampling/ Selection target Group</font>
 
| <font size="2">Sampling/ Selection target Group</font>
 
|  
 
|  
*<font size="2">[[Guidelines_for_Sampling/_Selection_Criterias|Guidelines / Sampling & Selection Criterias]]</font>
+
*<font size="2">[[Guidelines for Sampling/ Selection Criterias|Guidelines / Sampling & Selection Criterias]]</font>
 
*Poverty criteria for selection of HH
 
*Poverty criteria for selection of HH
  
Line 114: Line 112:
  
 
|}
 
|}
 
<br/>
 
  
 
<br/>
 
<br/>
Line 121: Line 117:
 
= Implementation =
 
= Implementation =
  
{| style="width: 100%;" border="1" cellspacing="0" cellpadding="0"
+
{| cellspacing="0" cellpadding="0" border="1" style="width: 100%"
 
|-
 
|-
| bgcolor="#0000ff" | <span style="color: rgb(255, 255, 255);">'''Task'''</span>
+
| bgcolor="#0000ff" | <span style="color: rgb(255, 255, 255)">'''Task'''</span>
| bgcolor="#0000ff" | <span style="color: rgb(255, 255, 255);">'''Materials/ Support'''</span>
+
| bgcolor="#0000ff" | <span style="color: rgb(255, 255, 255)">'''Materials/ Support'''</span>
| bgcolor="#0000ff" | <span style="color: rgb(255, 255, 255);">'''Library'''</span>
+
| bgcolor="#0000ff" | <span style="color: rgb(255, 255, 255)">'''Library'''</span>
 
|-
 
|-
 
| <font size="2">Logistics</font>
 
| <font size="2">Logistics</font>
 
| <font size="2">Checklist on how to implement HH Interviews</font>
 
| <font size="2">Checklist on how to implement HH Interviews</font>
*[[Short_Household_Survey|H1: Short Household Survey]]
+
*[[Short Household Survey|H1: Short Household Survey]]
 
*[[Profound Household Survey|H2: Profound Household Survey]]
 
*[[Profound Household Survey|H2: Profound Household Survey]]
 
*[[Qualitative Household Appraisal|H3: Qualitative Household Appraisal]]
 
*[[Qualitative Household Appraisal|H3: Qualitative Household Appraisal]]
Line 153: Line 149:
 
= Data Analysis/ Reporting =
 
= Data Analysis/ Reporting =
  
{| style="width: 100%; height: 236px;" border="1" cellspacing="0" cellpadding="0"
+
{| cellspacing="0" cellpadding="0" border="1" style="width: 100%; height: 236px"
 
|-
 
|-
| bgcolor="#0000ff" | '''<span style="color: rgb(255, 255, 255);">Task</span>'''
+
| bgcolor="#0000ff" | '''<span style="color: rgb(255, 255, 255)">Task</span>'''
| bgcolor="#0000ff" | '''<span style="color: rgb(255, 255, 255);">Materials/ Support</span>'''
+
| bgcolor="#0000ff" | '''<span style="color: rgb(255, 255, 255)">Materials/ Support</span>'''
| bgcolor="#0000ff" | '''<span style="color: rgb(255, 255, 255);">Library</span>'''
+
| bgcolor="#0000ff" | '''<span style="color: rgb(255, 255, 255)">Library</span>'''
 
|-
 
|-
 
| <font size="2">Data analysis</font>
 
| <font size="2">Data analysis</font>
Line 197: Line 193:
  
 
= Further Information =
 
= Further Information =
 +
 +
*[[Portal:Impacts|Impact Portal on energypedia]]
 +
 +
  
 
= References =
 
= References =
Line 202: Line 202:
 
<references />
 
<references />
  
 +
[[Category:Monitoring]]
 
[[Category:Impacts]]
 
[[Category:Impacts]]
[[Category:Monitoring]]
 

Revision as of 13:52, 8 August 2014

Overview

The Impact Monitoring Group of Energising Development has developed recommendations for sound impact assessments.

We recommend to:

  • use the recommended list of indicators
  • conduct baseline studies before project intervention and impact studies after households, social infrastructure or SMEs received access to modern energy
  • use a theory based approach by applying the result chain for developing the study design
  • use a mixed methods approach: the logic of the comparative advantages of methods: i.e. the mix of quantitative (standardised household interviews) and qualitative methods (open / semi-structured interviews with key persons and households or focus group discussions)
  • include, wherever possible, control groups (households which resemble those households with access to modern energy with the exception of this access) into your baseline and impact study
  • consider research ethics

The following tables show in more detail typical steps for baseline and impact studies, which we furthermore recommend and support.


Introduction

Development interventions are executed with the ultimate aim of changing the living situation in less developed countries. What is often thoroughly investigated are the inputs to a development intervention and its immediate outputs - how much money is spent, how many children are vaccinated, business trainings conducted or latrines installed. Assessing the impacts of a development intervention though - the aforementioned amelioration of living standards that can be attributed to a particular intervention - is part of a movement towards evidence-based development policy making and accountability. The central question to all impact evaluations "What is the causal effect of a programme on the variable of interest?" can be applied to various policy contexts.

Initial but crucial steps to every evaluation is the establishment of

  1. questions to be answered within the scope of the given evaluation;
  2. the theory of change, depicting how, in theory, the intervention is envisaged to achieve results;
  3. the result chain;
  4. hypothesis to be tested by the evaluation;
  5. performance indicators.

Taken together, the evaluators will hereby arrive at the specific questions to be phrased by the evaluation. Ideally, this will occur in the planning phase of the development intervention, thereby linking implementation and evaluation closely.


Preparation

Tasks Materials/ Support Library
Draft Terms of Reference for Consultants / Enumerators
  • Sample Terms for national / international consultants
  • Sample Terms for enumerators
ToR Consultants:
Selection of Consultant / Enumerators
  • Consultants
Study documents and statistics on national development goals and policies, national sector goals and policies, BMZ goals and concept for the partner country
  • Checklist on relevant development goals
Developing of Results Chains/ Indicators (consider

undesired side effects being positive or negative)

Cooking:

Developing of Questionnaires, interview guidelines, PRA tools (MAPP), observation criteria and other tools (catalogue of methods)

Example Questionnaires

(Impact Studies Material)

Sampling/ Selection target Group
Training of Supervisors and Enumerators/ Pretesting
  • Training Material
  • Training Agenda


Implementation

Task Materials/ Support Library
Logistics Checklist on how to implement HH Interviews


Conducting Interviews
Data entry


Data Analysis/ Reporting

Task Materials/ Support Library
Data analysis
Data interpretation – in exchange with the project team (in a workshop)
  • Checklist for data interpretation
Report writing (+ summary writing)
  • Example for report summary
  • Example for ICS description (Annex)

Handover of raw data




Further Information


References