Impact Monitoring Guidelines

The focus of the EnDev 2 M&E Sytem has shifted from Outcome Monitoring (EnDev 1) to Outcome and Impact Monitoring (EnDev 2). Therefore the Monitoring Group is developing recommendations for a "proper" impact M&E System.

We recommend to:

• use the recommended list of indicators
• conduct baseline studies before project intervention and impact studies after households, social infrastructure or SMEs got access to a form of modern energy
• use a theory based approach by applying the result chain for developing the study design
• use a mixed methods approach: the logic of the comparative advantages of methods: that means the mix of quantitative (standarised household interviews) and qualitative methods (open/ semi-structured interviews with key persons and households or focus group discussions)
• to include control groups if possible (households which do not have acces to modern energy) into your baseline and impact study 

The following table shows typical steps for baseline and impact studies, which we would recommend and support.

 

1. Preperation

Tasks	Materials/ Support	Library
Terms for Consultants/ Enumerators	Sample Terms for national / international consultants Sample Terms for enumerators	ToR Consultants: Tor Baseline Study (Rwanda PSP Hydro) german version ToR Survey french version ToR Evaluation english version Further Documents
Selection Consultant/ Enumerators	Consultants
Study of available documents and statistics on national development goals and policies, national sector goals and policies, BMZ goals and concept for the partner country	Checklist on relevant development goals	UNDP Energy Services for the Millennium Development Goals BMZ Sustainable Energy for Development
Developing of Results Chains/ Indicators	Sample Results Chain Impact Criteria and indicators	Rural Electrifciation: Result Chain german version Result Chain engl. version Result Chain french version Cooking: Energy Impacts, Observation Fields and Indicators IA Kenya: Result Chain IA Ethiopia: Result Chain IA Kenya: Model List of Impacts, Observation Fields and Indicators
Developing of Questionnaires, interview guidelines, PRA tools (MAPP), observation criteria and other tools (summary of methods)	Questionnaires for HH (stoves), stove producers, social institutions and restaurants using ICS PRA tools for focus group discussions (MAPP) Interview guidelines for village authorities Observation criteria for stoves in use (example lists of good cooking practices and of stove conditions)	Example Questionnaires PRA tools, Interview Guidelines, Observation criteria etc. (Impact Studies Material)
Sampling/ Selection target Group	Guidelines / Sampling & Selection Criterias Poverty criteria for selection of HH	Kenya: Selection Criteria for HH, SI, PU and Producers IA  Kenya: Local Poverty criteria (example from local village authorities)
Training of Supervisors and Enumerators/ Pretesting	Training Material Training Agenda	Supervisors and Enumerators Example Training Agenda (french version)  Kenya: ICS Training of Enumerators

 

2. Implementation

Task	Materials/ Support	Library
Logistics	Checklist on how to implement HH Interviews H1: Short Household Survey H2: Profound Household Survey H3: Qualitative Household Appraisal
Conducting Interviews		Manual Conducting Interviews (english version)  Manual Conducting Interviews (french version)
Data entry

 

3. Data Analysis/ Reporting

Task	Materials/ Support	Library
Data analysis	Guiding Questions for data analysis	Kenya Stove HH: Guiding Questions for data analysis Kenya Stove Producer: Guiding Question for data analysis
Data interpretation – in exchange with the project team (in a workshop)	Checklist for data interpretation
Report writing	Example for report summary Example for ICS description (Annex)	Kenya Stove HH: Example for report summary