LEAP II (Learning, Evaluation, and Analysis Project II)

Status: Completed

On the Learning, Evaluation & Analysis Project-II (LEAP-II), IDG provided high-quality, independent, and transparent economic analysis and MEL services to the Bureau for Development, Democracy, and Innovation (DDI) (formerly E3/EP) and Missions and Bureaus worldwide to improve programming and to better achieve evidence-based development. IDG implemented 36 activities in 30 countries across Asia, the Caribbean, MENA, Sub-Saharan Africa, with 4 international subcontractors, in addition to numerous local partners. LEAP II activities supported all stages of USAID’s Program Cycle through: performance and impact evaluations; project design; rigorous cost-benefit analysis (CBA) and cost-effectiveness analysis (CEA); organization and facilitation of learning events; Agribusiness Commercial Legal and Institutional Reform (AgCLIR) diagnostics; and capacity building and training for government officials and other stakeholders. IDG rapidly assembled and fielded teams, often in less than two weeks, in more than 30 countries worldwide for 36 activities, including 15 performance and impact evaluations and 21 economic analyses covering a range of technical areas.

Services under LEAP-II’s scope of work included:

  • Cost-benefit analysis (CBA)/cost-effectiveness analysis (CEA)
  • Macroeconomic and financial analysis
  • Single- and multi-country evaluations
  • Randomized controlled trials (RCTs)
  • Policy Analysis Matrix/Domestic Resource Cost Analysis
  • Constraints Analysis/Inclusive Growth Diagnostics
  • Other Analytical and Project Support
  • Integrating Gender Equality and Female Empowerment

Select examples of activities include feasibility studies such an analysis of customs in Timor Leste and designing a new customs project including drafting the solicitation package. Diagnostics included AgCLIR diagnostics for USAID/Sierra Leone and USAID/Guinea, and analyzing trade constraints and prioritizing trade interventions across nine countries in Southern Africa using an inclusive diagnostics approach. IDG conducted CBAs of eight USAID Feed the Future projects and provided capacity building on CBA to the Government of Kenya and Government of Haiti. Performance and impact evaluations were also key activities under LEAP-II with IDG conducting 15 evaluations oriented toward methodological rigor, utility to USAID, and cost effectiveness.

Regions

  • Asia
  • Latin America and the Caribbean
  • Middle East and North Africa
  • Sub-Saharan Africa

Country / Countries

  • India
  • Sri Lanka
  • Timor Leste
  • Haiti
  • Jordan
  • Morocco
  • Oman
  • Tunisia
  • Botswana
  • Burkina Faso
  • Egypt
  • Ghana
  • Guinea
  • Kenya
  • Lesotho
  • Liberia
  • Malawi
  • Mali
  • Mozambique
  • Namibia
  • Niger
  • Nigeria
  • Rwanda
  • Senegal
  • Sierra Leone
  • South Africa
  • Tanzania
  • Zambia
  • Zimbabwe

Integrating Gender into Cost-Benefit Analysis and Cost Effectiveness Analysis: IDG developed cutting edge thinking on integrating gender into CBA and CEAs and provided actionable recommendations to policymakers and practitioners on how to integrate gender impacts in CBAs and CEAs. Integration of gender into CBA uncovers details on how a project interacts with cultural norms and gender gaps, leading to: 1) a more accurate measure of impacts, 2) inclusion of additional impacts that could otherwise be ignored or misrepresented in the analysis, 3) alterations in the project design to maximize impact or better comply with policies around gender, and 4) highlighting knowledge gaps that can be addressed by minor changes or additions to project design and utilizing the learning opportunities that every project offers. IDG’s developed a tool for practitioners to incorporate gender into the program cycle.

Analysis of Trade Constraints in Southern Africa: IDG benchmarked nine Southern African economies using an inclusive growth diagnostic methodology to analyze constraints based on three sets of indicators: trade cost, production cost, and macroeconomic and political stability to identify the most critical constraints and prioritize trade interventions (following an approach formulated by Hausmann, Rodrik and Velasco (2005) as adapted by Hallaert (2009)). IDG produced a report identifying binding constraints and activities with the most significant returns to trade improvements.

Liberia Cross Border Trade Assessment: IDG conducted a diagnostic assessment focused on the formal and informal trade flows between Liberia and its Mano River Union (MRU) neighbors, Cote d’Ivoire, Guinea and Sierra Leone.  The study identified the key stakeholders involved in cross-border trade (CBT) and utilized a new survey of formal and informal cross-border trade (ICBT) completed under the USAID Liberia Strategic Analysis (LSA) activity. The report discussed Liberia’s cross-border trade in the context of broader complementary research on both formal and informal cross-border trade in Sub Saharan Africa, West Africa, and the MRU. Through presenting a more nuanced understanding of Liberian cross-border traders, the study sought to identify appropriate policies and regulatory responses that supports job creation and income generation for both large and small-scale traders. IDG carried out 87 KIIs with a wide range of stakeholders involved either directly or indirectly with cross border trade and conducted nine FGDs. The surveys and interviews collected information on the characteristics of the cross-border traders, identifying the products, and detailing their experiences in moving goods into and out of Liberia to the neighboring countries of Guinea and Cote d’Ivoire. The data collected from the interviews was used to shed light on the validity of assumptions regarding economic linkages, potential missing domestic revenue, and the importance of border trade for poverty alleviation and food security.

Timor-Leste Customs Assessment: IDG conducted an assessment of the customs sector in Timor-Leste to inform the design of a new project. The assessment reviewed the legal and regulatory environment, the operations and capacity of the Customs Department and related supporting institutions and entities, including the Ministry of Finance, sea and airports, importers-exporters, freight forwarders, customs brokerages, and carriers. Of particular focus were customs enforcement training and curriculum development, internal investigations and integrity training, bonded warehousing, airport screening of passengers and cargo and post-clearance auditing, assistance preparing for the Association of Southeast Asian Nations (ASEAN) “single-window,” and assistance with the design and functioning of land border posts. The assessment reviewed the current legal and regulatory framework and practices in line with requirements under the World Customs Organization (WCO) and ASEAN. The analysis identified the primary challenges and opportunities in advancing customs reform, including an assessment of the political will for reform and develop a proposed strategy for implementing reforms.

Increasing the Efficiency of Public Investment in Haiti and Kenya: IDG strengthened the countries’ public investment system through expanding government capacity and promoting transparency in making public and public-private investment decisions in multiple sectors. IDG provided training in project design, cost benefit analysis, and monitoring and evaluation to improve the analytic capacities of government officials to equip them with the technical capabilities to design and analyze public investment projects. The high-quality training program contributed to an increase in the net benefits of public investment and involved analyzing projects in conjunction with the Government of Haiti from idea, inception, design, pre-feasibility, and full feasibility. Specific training topics included an analysis of public-private partnerships and how they benefit infrastructure investment, as well as appraisals of power projects under current regulatory regimes to determine if they increased national welfare.

Cost-Benefit Analysis (CBA) of Value Chains in Sub-Saharan Africa: LEAP-II conducted four ex-post and mid-point CBA for agricultural value chains in West Africa. For each CBA, LEAP-II collects primary and secondary data and builds a model using cash inflow/outflow and economic gains of the beneficiaries to estimate the present value (PV) and internal rate of return (IRR) of incremental financial gains to the beneficiaries. Deliverables include a CBA model in Excel and a report including methodology, data sources, results, recommendations, and financial, economic and sensitivity analyses. The CBAs provide an understanding of the economic soundness of USAID interventions in the targeted value chains.

Agribusiness Commercial Legal and Institutional Reform (AgCLIR) diagnostics: IDG conducted analyses and made recommendations to improve the business-enabling environments for agriculture in Sierra Leone and Guinea. The diagnostics identified policy and regulatory constraints affecting agricultural production and investment, recommended reform activities, assessed the feasibility of implementing reforms, and prioritized constraints and associated reforms for greatest impact. The team conducted in-country interviews with public and private sector stakeholders to examine business enabling environment issues.

IDG conducted 15 performance and impact evaluations oriented toward methodological rigor, utility to USAID, and cost effectiveness, including:

IDG conducted an evaluation of Yaajeende, USAID’s first large-scale Feed the Future program focused on improving food security, agricultural development, and nutrition in four regions of Senegal. IDG implemented a mixed methods evaluation to measure the impact of USAID/Liberia’s value-chain investments in the Food and Enterprise Development project on agricultural productivity and profitability, investment and earnings, and quality of life. IDG conducted the interim assessment of USAID’s Feed the Future population-based indicators in six counties that make up USAID/Liberia’s zone of influence (ZOI). IDG conducted the first interim assessment of Feed the Future’s population-based indicators for the ZOI in Senegal. IDG conducted the mid-term evaluation of two SME development projects in Liberia. IDG conducted the final evaluation of the USAID HP-UNIDO initiative to increase jobs in the interior regions of Tunisia, especially among youth. IDG conducted the mid-term performance evaluation of USAID/India’s DO4: Innovations proven in India and increasingly adopted in other countries. IDG carried out mid-term performance evaluations of the US inter-agency PFG initiative, which seeks to leverage USG resources in support of a shared development program delivering accelerated, sustained, and broad-based economic growth. IDG evaluated the Improving Water and Sanitation Services in the Middle East and North Africa Region (IWSMR) project, analyzing if it succeeded in its goals of (1) building capacity within the Arab Countries Water Utility Association (ACWUA) to sustainably carry out its mandate; and (2) facilitating ACWUA’s self-sufficiency in operations and funds-management.

Mid-term Impact Evaluation (MIE) of the Yaajeende Agricultural Development Program in Senegal: IDG conducted an evaluation of Yaajeende, USAID’s first large-scale Feed the Future program focused on improving food security, agricultural development, and nutrition in four regions of Senegal. Using a mixed-method approach, the evaluation involved a non-experimental quantitative strategy and qualitative techniques. Qualitatively, a population-based survey was administered to 2,720 households in treatment and comparison villages across 19 sampling areas in three regions. Data from key stakeholders, program beneficiaries, and comparison groups were collected by enumerators using CAPI. Qualitative research methods included KIIs and FGDs. Findings revealed less severe reductions in nutritional status indicators in the context of worsening food security conditions in treatment areas, improvements in animal husbandry, statistically significant reductions in poverty, and increased adoption of healthy household practices due to factors exogenous to the project.

Liberia Food and Enterprise Development (FED) Impact Evaluation: IDG implemented a mixed methods evaluation to measure the impact of USAID/Liberia’s value-chain investments in the FED project on agricultural productivity and profitability, investment and earnings, and quality of life. IDG selected key variables from the 2012 FED baseline survey as well as key Feed the Future indicators to measure the impact of FED on agricultural productivity and profitability. IDG used a population-based matching design to estimate impact, operationally adjusted in mid-course. IDG developed and surveyed 1,440 households using CAPI. IDG calculated an average treatment effect (ATE) and used propensity score matching (PSM) as a robustness test. IDG also collected and analyzed qualitative data from 21 key informant interviews (KIIs) and 69 focus group discussions (FGDs) with FED beneficiaries and other project stakeholders. IDG used innovative geospatial analysis to improve monitoring activities in real time. IDG provided recommendations that would improve value chain selection and analysis for future USAID programs and additional steps to improve knowledge transfer among farmers.

Liberia Feed the Future Interim Population-Based Indicator Assessment: IDG conducted the interim assessment of USAID’s Feed the Future population-based indicators in six counties that make up USAID/Liberia’s zone of influence (ZOI). The purpose of the interim assessment was to provide stakeholders with information about the short-term progress of indicators related to poverty, income, women’s empowerment, and hunger following a baseline population-based survey conducted in 2012/13. IDG formulated the evaluation methodology, prepared the questionnaires to fit the Liberian context, and collected data from 1,861 households using computer-assisted interviewing. IDG cleaned and analyzed the data and prepared a report which highlighted a reduction in poverty and hunger in the ZOI but also displayed a high prevalence of malnutrition in children, and a higher incidence of micronutrient deficiencies in women.

Mid-term evaluation of the Feed the Future zone of influence in Senegal: IDG conducted the first interim assessment of Feed the Future’s population-based indicators for the ZOI in Senegal. The assessment was designed for use as a monitoring tool, and as such provides point estimates of the indicators with an acceptable level of statistical precision. The ZOI in Senegal was defined as 150 CRs in 10 regions. The current ZOI excluded two regions that were in the baseline frame and also included some departments and CRs that were not included in the baseline ZOI. A comparison of the baseline sample frame and current ZOI indicates that 87 communes (58 percent) of the baseline dataset are also in the current ZOI, although because of some of the baseline data was lost, the usable baseline dataset includes 63 CRs. The sample design strategy employed a two-stage cluster sampling method to select a sample of 3,830 households in order to reach a target of 3,192 households, accounting for nonresponse. Similar to the baseline, no stratification was used at either level of sampling. In total, 3,801 households completed surveys.

Mid-term Performance Evaluation of the Liberia Investing for Business Expansion (IBEX) and Sustainable Markets Initiative-Liberia (SMI-L) Projects: IDG conducted the mid-term evaluation of two SME development projects in Liberia. The evaluation team conducted interviews with the implementing partners, Government of Liberia, client SMEs, buyers, business development service providers, and banks. IDG produced an evaluation report that provided responses to the evaluation questions supported by evidence, recommendations to ensure the effectiveness of the projects, and recommendations for future private sector development work.

Tackling Youth Employment in Tunisia Final Evaluation: IDG conducted the final evaluation of the USAID HP-UNIDO initiative to increase jobs in the interior regions of Tunisia, especially among youth. This project also endeavored to keep youth gainfully employed and reduce the proliferation of extremist activities. IDG used both qualitative and quantitative data collection and analysis tools to address the evaluation questions and measure the efficacy of the program.

Performance Evaluation of USAID/India’s Development Objective (DO) 4. IDG conducted the mid-term performance evaluation of USAID/India’s DO4: Innovations proven in India and increasingly adopted in other countries. Primary data collection included key informant interviews (KIIs) and focus group discussions (FGDs) in India, Kenya, Ghana, and Malawi. The evaluation generated empirical data/information to feed into USAID/India’s CDCS development.

Mid-term Performance Evaluation of the Partnership for Growth (PFG) Initiative in Ghana and Tanzania: IDG carried out mid-term performance evaluations of the US inter-agency PFG initiative, which seeks to leverage USG resources in support of a shared development program delivering accelerated, sustained, and broad-based economic growth. PFG involves rigorous joint analysis of constraints to growth, the development of joint action plans to address these constraints, and high-level mutual accountability for implementation. The mid-term evaluations in Ghana and Tanzania included examining PFG’s effectiveness in addressing binding constraints identified in each country (energy and access to finance in Ghana, and energy and rural roads in Tanzania). IDG developed the evaluation methodology and questionnaires and conducted three weeks of data collection in each country, which included key informant interviews with a variety of stakeholders in the government and private sector, and an online survey. IDG also produced detailed case studies of the power sector in each country that addressed: public-private partnerships, policy, regulatory and institutional challenges; energy mix and power generation sources; and power usage and transmission and distribution system coverage.

Final Performance Evaluation of the Improving Water and Sanitation Services in the Middle East and North Africa Region (IWSMR): IDG evaluated the IWSMR project, analyzing if it succeeded in its goals of (1) building capacity within the Arab Countries Water Utility Association (ACWUA) to sustainably carry out its mandate; and (2) facilitating ACWUA’s self-sufficiency in operations and funds-management. The IDG team conducted 55 KIIs and two FDGs with stakeholders in six MENA countries, and implemented a self-administered online survey of training participants. The data were analyzed for before/after the intervention, relative contribution, hierarchy of changes (awareness/knowledge, attitudes/values, actions/behavior), and sustainability of changes.

Areas of Expertise

Cost-benefit Analysis (CBA)

LEAP-II conducted 12 activities involving CBA for E3, BFS, and Missions in Haiti and Kenya. IDG analyzed  interventions in value chains on USAID agriculture projects and a synthesis of CBAs of 12 value chains across 11 countries in Sub-Saharan Africa. IDG also provided capacity building and training to government officials in Haiti and Kenya on using CBA techniques to select investment projects with a goal of institutionalizing the use of rigorous criteria of economic efficiency in project evaluation, contributing to reducing inefficiencies in public spending with the long-term goal to improve transparency in the decision-making process in public spending decisions.

Enabling Environment

USAID/Sierra Leone and USAID/Guinea commissioned Agribusiness Commercial Legal and Institutional Reform (AgCLIR) diagnostics to inform and assist in prioritizing the constraints and recommending reform activities for the AgBEE in each country. IDG conducted analyses and made recommendations to improve the business-enabling environments for agriculture in Sierra Leone and Guinea.

Trade Facilitation

IDG conducted five trade analyses on LEAP II including identifying constraints to trade in Southern Africa, gathering data on informal cross-border trade in Liberia, assessing the customs sector in Timor-Leste, and analyzing inter-regional trade monitoring data in the Sahel. For each activity, IDG developed and agreed the design of the analysis with the Mission, implemented primary and secondary data collection, analyzed the data using rigorous methodologies, drafted a comprehensive report, and presented results to facilitation learning.

Impact and Performance Evaluation

IDG conducted 15 performance and impact evaluations oriented toward methodological rigor, utility to USAID, and cost effectiveness for LEAP II, including an evaluation of Yaajeende, USAID’s first large-scale Feed the Future program focused on improving food security, agricultural development, and nutrition in four regions of Senegal.

Food Security and Agriculture

IDG implemented a mixed methods evaluation to measure the impact of USAID/Liberia’s value-chain investments in the FED project on agricultural productivity and profitability, investment and earnings, and quality of life. IDG selected key variables from the 2012 FED baseline survey as well as key Feed the Future indicators to measure the impact of FED on agricultural productivity and profitability.

Gender and Inclusivity

IDG developed cutting edge thinking on integrating gender into cost-benefit analysis (CBAs) and cost-effective analysis (CEAs), and provided actionable recommendations to policymakers and practitioners on how to integrate gender impacts in CBAs and CEAs. Integration of gender into CBA uncovers details on how a project interacts with cultural norms and gender gaps.