Consultancy to conduct Mixed-Method Impact Evaluation and Learning Exercise for Somalia Crisis and Recovery (SCRP) Project

BRCiS Consortium

Consultancy to conduct Mixed-Method Impact Evaluation and Learning Exercise for the Somalia Crisis and Recovery (SCRP) Project

Terms of Reference

1. Background

Building Resilient Communities in Somalia (BRCiS) is a consortium of national and international organizations – Action Against Hunger (ACF), CESVI, Concern Worldwide (CWW), GREDO, the International Rescue Committee (IRC), KAALO, Save the Children, and Norwegian Refugee Council (NRC) as lead agency. BRCiS’ objective is to work across the humanitarian-development divide, supporting marginalized communities in disaster-prone, rural Somalia to become more resilient to shocks and stressed, including as a result of climate change. BRCiS approach is contextually adaptive, focused on the specific shocks, needs, and priorities of individual communities. BRCiS was established in 2013 and is now implementing projects funded by multiple humanitarian and development donors in more than ten regions of Somalia.

2. Program Background

Somalia Crisis Recovery Project (SCRP) is funded by the World Bank through the Federal Government of Somalia (FGS) targeting early recovery assistance for chronically vulnerable people, communities, and systems in targeted pastoral, agro-pastoral, and peri-urban livelihood zones impacted by multiple shocks, such as recurring floods, drought, locust, and pandemic diseases, in Jubaland, Hirshabelle, and Southwest State. The SCRP project contains several service delivery components that are being implemented by various agencies. BRCiS consortium implements Component 1 (C1) and Component 6a (C6a), which relate to service delivery as part of the Somalia Crisis Recovery Project.

The interventions in Component 1 aim to address the needs of the flood and drought-affected communities, and C6a addresses the immediate needs of people affected by the drought experiencing high levels of food insecurity, including those living in pastoral, agro-pastoral—riverine livelihood zones in Jubaland.

Component 1 has five outcomes: Cash for Work (CFW), Basic Health & Nutrition Services, Household Hygiene Promotion, and sanitation, Restoring and protecting agricultural crop production, and Vector-borne Diseases Control while component 6a has the following outcomes addressing the drought in Jubaland; (a) Water trucking and Cash for Work, (b) rehabilitation of small-scale water harvesting, soil improvement, Livelihood, and technical advice packages through inputs for assets, (c), Micro irrigation Kits.

The project implementation period is 24 months between Jan 23rd, 2022, and Dec 31st, 2023, and the project target locations include:

1. Jubaland: Bardere, Belethawa, luuq, Ceelwaaq, Garbaharey, Dollow, Kismayo, Jamame, Badhadhe, Afmadow.
2. Hirshabelle: Beletweyne, Buloburde, Jowhar
3. Southwest: Baidoa, Diinsoor, Qansaxdheere, Berdaale, Burhakaba.

3. Purpose of the assignment

BRCiS Consortium is seeking to engage with a consultant team to conduct a mixed method SCRP project impact evaluation and learning.

The consultants are required to do the following tasks:

• Review SCRP project documents including project proposal, progress reports, conducted assessments data and reports.
• Prepare inception report that outlines project evaluation and learning objectives, research questions, sampling and data analysis methods, deliverables and detailed feasible workplan.
• Review existing quantitative and qualitative data collection tools, suggesting any necessary changes in collaboration with BRCiS Consortium, and ensuring that the data collection tools include all required information for the project set impact and outcomes indicators.
• Train BRCiS Consortium Members on the project evaluation data collection tools.
• Support the roll out of the quantitative data collection by the Consortium members in all project target locations and provide necessary technical guidance and support during the quantitative evaluation survey data collection.
• Conduct field-based project qualitative evaluation and learning surveys across five sampled locations.
• Clean collected qualitative and quantitative surveys data, analyse it, and prepare the final project impact evaluation and learning report.
• Prepare an executive summary of the final project evaluation and learning report (10 pagers) in both English and Somali for local actors’ consumption.
• Present the project evaluation report to all project stakeholders and address any feedback received.

Below are general project evaluation and learning questions to evaluate the effectiveness, impact, and lessons learned from the project activities, leading to valuable insights for future programming and improvements in humanitarian interventions.

• What is the profile of SCRP project target areas, including demographic characteristics, socio-economic status, cultural norms, and other relevant contextual factors?
• What types of shocks and stress do target communities experience the most, how these differentially affect vulnerable groups, and what are the primary coping strategies used? How are these capacities different before and after the project implementation?
• How effective was the BRCiS members targeting process for identifying and reaching the most vulnerable households with project activities? Did the intervention reach vulnerable households?
• How did the project activities contribute to early recovery for chronically vulnerable people, communities, and systems in targeted pastoral, agro-pastoral, and peri-urban livelihood zones impacted by multiple shocks, such as recurring floods, drought, locust, and pandemic diseases?
• How has the implementation of Cash for Work activities contributed to target community’s livelihood protection through protecting & rehabilitating community assets? What were the key community infrastructures repaired and rehabilitated through cash for work, and how did they contribute to improving water access and community resilience? What are the key challenges faced in protection and rehabilitating community assets through Cash for Work, and how were they addressed? How has the provision of cash support helped vulnerable households in meeting their basic needs including food?
• How has the provision of essential health and nutrition services contributed to the overall well-being of the target communities? What is the level of GBV awareness and knowledge among the target communities, and how has the project GBV awareness creation activities impacted their understanding and response to GBV?
• To what extent have hygiene promotion and sanitation activities improved the hygiene practices and behaviors of the targeted households? How has the availability of improved sanitation facilities impacted the health and well-being of the community members?
• What is the impact of the SCRP project activities on restoring and protecting agricultural crop production? Did the project agricultural crop production activities contribute to increased production, food security and income generation for the targeted farming households? What are the lessons learned in terms of sustainable agricultural practices and crop diversification?
• How has vector-borne diseases control intervention impacted the health and productivity of livestock in the targeted communities? What were the key strategies employed in controlling vector-borne diseases, and how effective were they? Did the project activities contribute to improved knowledge and capacity among livestock owners in preventing and managing vector-borne diseases?
• What is the impact of SCRP project emergency support and drought recovery activities on the drought-affected communities’ resilience and recovery? What are the challenges faced in implementing drought anticipatory activities, and how were they addressed?
• How effective was the rehabilitation of target communities small-scale water harvesting and soil improvement infrastructures in enhancing community resilience to future droughts? Did the provision of livelihood and technical advice packages contribute to sustainable recovery and improved agricultural practices? What are the key lessons learned in promoting drought-resistant farming techniques and sustainable water management practices in riverine farming communities?
• Which project activities have the most positive and lasting impact?
• How were the project interventions integrated, layered, and sequenced across the different project components (C1 and C6a) and sectors? What factors influenced project-implementing members’ decisions and phasing? Which integrated programming activities and modalities (layering and sequencing) were most effective in drought and flooding situations?
• How did project implementing members engage with target communities and target groups in the decision-making process for the project activities to implement and the final project workplan? What mechanisms or strategies were used to ensure community participation and ownership?
• What are the challenges faced during implementation and identified best practices for promoting collaboration and adaptive management.
• To what extent have the most urgent and severe humanitarian needs been alleviated? How rapidly did these effects occur after the intervention, and how does this differ between those who received only C1 support package or C6a support package and those who received integrated C1 and C6a support packages?

4. Methodology

The SCRP project impact evaluation and learning exercise will employ both quantitative and qualitative approaches to effectively extract meaning and understanding from both numerical and non-numerical data collected from different contexts and ensure triangulation of information.

Specifically, the methods to utilize the project impact evaluation exercise include:

• Desk review: Review of SCRP project documents including project proposal, progress reports and conducted project assessment reports.
• Quantitative field data collection by BRCiS Consortium: BRCiS Consortium members will conduct required project evaluation quantitative data collection in all project target districts.
• Qualitative field data collection by the consultant in sampled four project target districts: Kismayo, Belethawa, Bardere and Beletweyne through

1. Focus group discussions (FGD) with project target communities.
2. Key informant interviews with BRCiS Consortium project staff, local authorities, community leaders, health staff and water management committees

• Direct observations of project constructed/rehabilitated infrastructure including water harvesting, canals, soil infrastructures and any other observable variant around these infrastructures in the sampled districts.

5. Deliverables

Deliverables under this assignment will include the following:

• A detailed inception plan demonstrating how the consultant plans to conduct the project evaluation and learning survey including project evaluation and learning objectives, detailed research questions, a feasible methodology and data collection tools, data analysis plan and a detailed work plan.
• Data collection tools training materials.
• Project evaluation and learning final report and ppt.
• Executive summary of the project evaluation final report (10 pagers) in both English and Somali for local actors’ consumption.
• Project evaluation surveys data sets and analysis scripts

6. Proposed Time Frame

The assignment is expected to last approximately 36 working days as shown below tables.

Deliverables

Number of consulting days allocated and Tentative Timeline

Inception Report including revised data collection tools – 6 days within Dec 2023

Consortium members training on quantitative data collection tools- 1 day within Dec 2023

Evaluation and Learning Quantitative data collection survey by BRCiS Consortium members 0 days – Dec 2023

Evaluation and Learning Qualitative data collection survey in sampled four districts (2 days per target district) 8 days within Dec 2023

Quantitative and qualitative data Cleaning and analysis 10 days within Dec 2023

Report Writing (doc and ppt format) 8 days within Jan 2024

Feedback-sharing and revision of reports 2 days within Jan 2024

Project Evaluation findings Presentation and Dissemination workshop 1 days within Jan 2024

Sum of total days anticipated 36 days

All activities to be completed before January 15th, 2024

7. Supervisor

The supervisor of the consultant is the BRCiS MEAL Manager with the support of BRCiS programme manager. More generally, the Consultants will collaborate with the Consortium’s management unit, SCRP project PIU team, and with relevant Consortium Members M&E and Project management Groups to produce and publish the commissioned deliverables.

8. Responsibilities

1. BRCiS Consortium Responsibilities

• Provide oversight during the survey.
• Approval of all deliverables including sign off for the purpose of making payments
• Mobilization of survey respondents
• Conduct Project impact evaluation and learning quantitative data collection.

2. Consultant Responsibilities

• All logistics arrangements including accommodation and transportation for all field travel.
• Completing all deliverables as highlighted in this ToR.

9. Application Procedures and Requirements

The consulting firm interested are expected to provide following documentation:

• A cover letter introducing the consultant. The cover letter should introduce the team composition and specify the role to be played by each team member.
• A technical proposal of no more than 10 pages outlining how to execute the task with a clear framework, methodology and timelines. Proposed methodology should demonstrate a clear understanding of the ToR (Terms of Reference) (impact evaluation, qualitative learning exercise, sampling, data collection and analysis strategy/methods)
• Resume of each team member
• Evidence of experience conducting similar assignments (Samples of similar work) is required.
• Proposed total budget with budget breakdown indicating consultancy fee, logistics cost and all other auxiliary costs in USD.
• License to operate: Registration certificate from the country of origin and tax compliance certificate.

Qualifications or specialized knowledge and/or experience required.

• An advanced university degree (Master’s) in Quantitative & Qualitative Social Sciences, Economics, Econometric and cost analysis, Statistics, or a related technical field(s) is required. Ph.D. is preferred.
• At least 8-10 years of experience in impact evaluation and learning studies.
• Substantial work in qualitative studies or a related field with a geographical focus on Sub-Saharan Africa, preferably on the drylands of the Horn of Africa.
• Extensive experience both in qualitative and quantitative methods demonstrated through publications.
• Extensive experience of the world bank projects is an asset.
• Intense methodological experience in both experimental and quasi-experimental evaluation methods, including specific expertise in statistical matching demonstrated through publications (ideally peer-reviewed).
• Previous experience in similar assignments in Somalia is required.

Weighing Criteria for Selection of the consultant

The following criteria will be used to weigh the applications:

• Technical proposal’s relevance and soundness (35%)
• Financial proposal (precision with the ceiling budget) (30%)
• Consultant’s verifiable of similar experience (15%)
• Consultants’ familiarity with the local (Somalia) context (10%)
• License to operate: Registration certificate from the country of origin and tax compliance certificate (10%).