Impact Evaluation - Mixed Methods

A mix of methods can be used to assess different facets of complex outcomes or impacts, yielding more breadth, depth and width in the portrait than can one method alone. One set of methods could be used to assess outcomes or impacts and another set to assess the quality and nature of intervention implementation, thus enhancing impact evaluation with information about program integrity and program experiences (Leeuw). 

A mixed methods approach is one which uses both quantitative and qualitative methods. Nearly all studies do this to some degree, so the question is rather the minimum acceptable level of application of each method, and the appropriate balance between the two. The current benchmark for valid impact estimates is that the study has a credible counterfactual, which means that it addresses the issue of selection bias where this is likely to be an issue, and other possible problems in making such estimates such as contagion (White).

Triangulation is a key concept that embodies much of the rationale behind doing mixed method research  and represents a set of principles to fortify the design, analysis and interpretation of findings in Impact Evaluation. Triangulation is about looking at things from multiple points of view, a method “to overcome the problems that stem from studies relying upon a single theory, a single method, a single set of data […] and from a single investigator” (Leeuw/Mikkelsen). 

Sources: 

Leeuw, Frans & Vaessen, Jos (2009): Impact Evaluations and Development. Nonie Guidance on Impact Evaluation. Draft Version for Discussion at the Cairo conference March-April, 2009. Nonie – Network on Impact Evaluation, p.48- 50. 

Mikkelsen, B. (2005) Methods for development work and research, Sage Publications, Thousand Oaks, p. 96.

White, H. (2008): Of Probits and Participation: The Use of Mixed Methods in Quantitative Impact Evaluation. Nonie Working Paper No. 7, p. 3-4.