Profound Household Survey

Overview

Conducting a household survey in Kangemi, Kenia

Extensive and structured household questionnaire comprising information on socio-economic conditions, access to services, different wealth and income categories, health, education and gender issues. Survey covers the project region and a control region to credibly create a counterfactual situation and thereby enable causal attribution of impacts to the intervention.

-> Module H2: Profound Household survey

General Approach

Depending on the information interests, two different approaches are applicable:

• Control regions may be non-electrified (Option A) or
• already disposing of electricity access (Option B).

For stove projects, option A refers to regions without any former improved stove intervention and Option B to regions where comparable improved stoves interventions have already taken place. An appraisal of the two options can be found in Box 1. In both of the cases, surveys are conducted before and after the intervention. They are complemented by open, semi-structured key informant interviews (cf. Annex on qualitative methods). Furthermore, the wider set of socio-economic household characteristics that is covered by this module facilitates detecting unintended (positive or negative) impacts.

Purpose

Provides information that allows for deriving statements about the causal relationship between modern energy provision and development indicators using modern state-of-the-art evaluation techniques. Baseline survey delivers profound knowledge about the target region. If Option B is chosen, information from the comparison group that already uses electricity or stoves respectively can be used to assess the beneficiary’s behaviour already before project implementation same as potential difficulties and opportunities to be expected in the wake of the EnDev project.

Box1: Options for Choice of Control Region The obvious counterfactual situation for the stove and electrification projects would be Option A – a region where these interventions have not been taken place before. However, Option B provides as well for a viable counterfactual. While Option A investigates in how far the project sites outperform the control regions, Option B explores in how far the project sites catch up in relation to the control regions. The crucial point is that between the before and the after survey, no such intervention takes places. At this point, one disadvantage of Option A becomes evident: It is often not for sure that during that time period the control region actually remains non-electrified or without improved stoves inter­ventions respectively. It has already happened that unforeseen activities by other donors or partner country institutions in the course of the project period rendered the implementation of this approach impossible. On the other hand, Option A has the advantage that it can serve as an “ex-ante impact assessment”. This implies that the information gathered in the control regions helps to anticipate the beneficiary’s behaviour, potential difficulties and opportunities to be expected in the wake of the EnDev project before project implementation. These findings can then potentially be fed back into the project design.</font>

Included Tools

• Household questionnaire draft
• Data entry template
• Guidelines for open, semi-structured interviews
• Implementation notes on the questionnaire and data entry
• Material for training of enumerators and field supervisors
• Checklist for logistics
• Literature on related studies

Implementation Procedure

Substantial methodological and statistical skills as well as survey experience are required to implement this module. Therefore, an international consultant with appropriate references in this area should be included in planning and implementation of the study as well as the analysis of the gathered data.

Expected Costs:

Minimum 20,000-25,000 Euro for each of the two waves (before and after).

Integrated Indicators:

Different from Module H1, this module addresses poverty impacts that can not be easily attributed to the project. It includes questions that are less easy to determine by respondents and which exhibit a higher sensitivity to the formulation of the questions (e.g. health-related issues). This might be, for example, due to the fact that the information itself is sensitive (e.g. income) or that issues are prone to an auspices bias (the tendency for responses to be affected by the respondent’s awareness of the survey sponsor, e.g. prioritization of basic services). This is addressed by, for example, eliciting the information through asking for more subcategories so that the intention of the question is not that easy to detect and calculation work to aggregate subcategories is not left to the respondent.

All indicators mentioned in Module H1 are also integrated in this module (see the indicator table in Module H1). In addition, poverty impacts, for which the relation and attribution to the intervention is not that trivial to work out, are included. The last column of the table gives reference to the whether the indicator is applicable to stove projects (S), electrification projects (E) or both.

Observation fields	MDG relevance	Indicator	What to measure?	Applicable for
Additional income from productive activities	MDG 1	Increased income from home­business activities facilitated by improved working conditions	Average monthly net income from productive use of electricity	S & E
More food available and additional income generating activities	MDG 1	Increased agricultural activity	Annual agricultural yields	S & E
Time savings	MDG 1 + 3	Decrease of women’s workload	(Women’s) daily working hours and general time use	S & E
Improved learning conditions of school children	MDG 2	Increased time and money available for school education and learning	Enrolment and attendance rates by school type and gender Number of absence days per year (incl. absence reason)	S & E
		Improved effective level of education of school children	Percentages of children who know to read – for example – a newspaper article (by age)	S & E
Improved information and communication opportunities	MDG 2 + 3 + 6	Increased use of information and communication facilities	Usage pattern of TV, video, DVD, radio, cell phone, landline phone Purpose of use of TV, video, DVD, radio, cell phone, landline phone	E
Cleaner air in the household	MDG 3 + 4 + 5 + 6	Improved indoor air quality in households	Percentage of household members with eye infections and respiratory diseases by gender and age categories	S & E
		Decrease in medical costs for diseases related to indoor air pollution	Expenditure on treatment of respiratory and eye diseases	S & E
Improved medical infrastructure (if targeted by the intervention)	MDG 4 + 5 + 6	Increased medical service use	Percentage of households using medical services (vaccination etc.)	E
		Improved health situation	Percentage of household members with recurrent diseases by gender and age categories	E
	MDG 5	Improved maternal health situation	Maternal health indicators	E

Further Information

• Catalogue of Methods for Impact StudiesMethods
  
• Millennium Development Goals (MDGs)