World Vision
Terms of Reference
Final Evaluation
End of Project Evaluation for Community-Led Capacity Strengthening For Fragile Contexts (C4FC) Project
World Vision Somalia
SomReP Program- Somali Response Innovation Lab
Evaluation Purpose
The objective of the endline evaluation is to assess the performance of the project as well as identify best practices, lessons learned, strengths and challenges in the activity design and implementation of the project including any changes at the community level and coordination with government and other stakeholders or institutions.
The endline evaluation will collect final values for key outcome indicators in order to assess overall project performance. The endline study will gather final indicator values from routine monitoring and capacity scorecards and will utilize qualitative data collection, including Key Informant Interviews (KIIs) and Focus Group Discussions (FGDs). An external consultant will be responsible for conducting the final evaluation.
Background Information
Activity Information
Activity Name
Community-led Capacity Strengthening for Fragile Contexts (C4FC)
Implementer(s)
World Vision Sudan, World Vision South Sudan, CARE International – Somalia & Sadar Institute
Period of Performance
January 1, 2021 to August 7, 2023
Active Geographic Region
Somalia, Sudan and South Sudan
Background and Context
Since 2013 the Somali Resilience Program (SomReP) https://somrep.org has been working with communities to develop and package interventions geared to build community resilience to the cyclic shocks that face Somalia. Established in 2013, the SomReP worked to harness best practices from partner agencies and invested in research to identify intervention approaches that when combined deliver the most impact for communities residing in rural and peri-urban locations and constantly face significant losses as a result of the climatic shocks.
Over the years we have invested efforts to scale the coverage of these intervention packages however this process and journey is incomplete in the absence of partnerships with indigenous NGOs as well as in the absence of publicly available training resources to prepare in-service and pre-service humanitarian workers and their agencies to engage communities on a journey towards resilience and support them with the necessary capacities to respond to and prepare for disaster but also develop pathways to build back better form disasters.
In 2020 SomReP received funding support from USAID under BHA titled: Community- Led Capacity Strengthening For Fragile Contexts. Through this fund WV/Care (Consortium members of SomReP) worked with local NGOs in Somalia, South Sudan and Sudan to strengthen the capacity of local organizations who are on the front-lines of responding to humanitarian disasters. The goal of the project is to advance locally led development and sustainability for local organizations on the front lines of providing humanitarian assistance in Somalia, Sudan and South Sudan through organizational capacity strengthening.
The project had two tracks, the first focused on the organizational strength of local NGOs, using standardized tools. The project assessed the organizational capacity of local partners, and from this assessment, C4FC provided mentoring and worked to strengthen the organizational capacity of local partners to respond to and prepare for crises. SomReP carried this out by leveraging its consortium’s strengths and existing partnerships in all three countries and provided coordinated capacity strengthening support as well as conducting two-way learning processes with LNGOs throughout the project life cycle. The second track focused on capacitating NGOs on community driven resilience (CDR). Appreciating the vast experience that both the consortium and local NGOs have, World Vision/Care worked with local NGOs to co-create training needs assessment tools on community driven resilience as well as a training curriculum contextualized to each country and harmonized to bridge experiences and best practices ensuring that the curriculum is context appropriate. Subsequently the CDR curriculum was packaged into an online training package accessible not only for project participants, but also humanitarian workers in all three countries
Description of the Activity
The purpose of this study is to assess the relevance, effectiveness, and sustainability of the project in addition to evaluating the adoption of practice of LNGOs participating in C4FC. The Assessment will specifically assess the following objectives:
-
Assess the relevance of the project. This criterion will look at the extent to which the program interventions responded to the partners needs and priorities.
- How relevant were the activities implemented by the project in addressing the needs of the partners?
- How did LNGOs perceive the relevance and usefulness of the capacity strengthening interventions?
- How were LNGO priorities integrated into the capacity action plans?
- How did the project integrate local government priorities and local context into the curricula development process?
- What aspects of the curriculum were adapted to each context?
-
Assess the effectiveness of the project and capacity building interventions
- What were the major factors influencing the achievement of the objectives of the project?
- Were there opportunities for co-creation utilized in delivery of the interventions, which ones and how did they contribute to increased effectiveness?
- Were there any shortcomings that hindered the effectiveness delivery of the interventions?
- To what extent did programme staff feel that the CDR training equipped them with skills and tools to support communities to anticipate shocks and plan resilience interventions?
- To what extent did LNGOs experience improvements in their capacity assessment scores between baseline and end-line?
- How have LNGOs adopted the CDR approach to inform the design and implementation of resilience interventions with at risk communities?
- How have LNGOs put into practice the skills and knowledge from the institutional strengthening activities?
-
Assess the sustainability of the project. The evaluation will seek to find out the extent to which the capacity strengthening interventions provided will likely continue after external funding ends over the medium and long term. The assessment will therefore look at the following;
- Local partners ownership; how far they were consulted from the beginning in co creation of priorities and whether there are clear plans on how they will continue with the interventions after the project has ended
Evaluation Methods and Limitations
Survey Design and Methodology
The evaluation will employ a mixed method study design utilizing quantitative and qualitative methods. In addition to this, desk review methodology will be utilized to collect quantitative data from relevant project documents and monitoring data to assess the performance of project indicators. While Focus Group Discussions (FGDs) and Key Informant Interviews (KIIs) with program staff and LNGOs staff from the three countries will be used to collect data. The consultant will visit two of the countries (Somalia and South Sudan) where the project was implemented.
Data Sources
Primary data will be collected from the LNGO staff in Somalia and South Sudan through field visit while virtual interviews will be conducted in Sudan. The endline values for key outcome indicators will be generated via applied capacity assessment tools as well as the pre and post-test assessment conducted during the CBDRM and GCVCA/CAAPs trainings in the three countries. The consultant will also make use of the Learning by Doing Reports to capture key lessons learnt, challenges and to provide more depth that informs the interpretation of the data from participants.
Analysis Plan
End line data will be analyzed in order to identify and test assumptions of the program design as well as generate both intended and unintended outcomes of the project. The evaluation will review key outcome indicators listed below and explore in what ways the project has contributed to the goal of improving the self-reliance and sustainability for local organizations on the front lines of providing humanitarian assistance in Somalia, Sudan and South Sudan through organizational capacity strengthening. Analysis will be descriptive in nature rather than inferential.
Data collected about the training curricula developed and used in the C4FC project will be analyzed using Kirkpatrick’s model which assesses the capacity strengthening across for main levels of progression: 1) Reaction; The degree to which participants find the training favorable, engaging and relevant to their jobs; 2) Learning: The degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training; 3) Behavior; The degree to which participants apply what they learned during training when they are back on the job and 4) Results; The degree to which targeted program outcomes related to CDR curriculum occur and contribute to the organization’s highest-level result.[1] The project will not be able to measure Kirkpatrick Level 4 given the two year timeframe
Limitations and Mitigation Methods
The below is related to challenges in the evaluation process and the proposed mitigation measures:
Limitation: The project will not be able to measure Kirkpatrick Level 4 given the two-year timeframe.
- Mitigating Measure 1: The qualitative will be able to uncover examples of good practices of capacity Strengthening interventions from the LNGOs
Evaluation Timeline and Deliverables
Estimated End line Evaluation Schedule and Deliverables
Task
Expected Delivery Date
Responsible for Delivery
Develop inception report detailing the evaluation work plan, evaluation matrix, comments to the evaluation concept note/protocol, list of documentation, etc.
6th August 2023
Consultant
Formulation of evaluation data collection tools
August 2023
Consultant
Approval and validation of the evaluation data collection tools
August 2023
World Vision, CARE, SADAR and SomReP
Data Collection
September 2023
Consultant
Data cleaning and analysis
September 2023
Consultant
First draft of the end line report
October 2023
Consultant
Review of draft end line report
October 2023
World Vision, CARE, SADAR, LNGOs and SomReP
Incorporation of feedback received
October 2023
Consultant
Submit Final End line Report to BHA
October 2023
World Vision
Evaluation Findings Dissemination
WV and C4FC Partners will reflect on the findings of these studies and will disseminate the findings to implementing partners, respective Governments and the Donors
Evaluation Team Composition and Scope
Competencies and Skills
The evaluation team will be external with the lead having the following qualifications:
-
Technical skills in Disaster Risk Management (DRM), Capacity building, curriculum development, research, monitoring and evaluation in humanitarian emergency response contexts.
-
Able to implement the research in remote parts of Somalia, South Sudan and Sudan following the required procedures. Proof of these is to be provided by submitting, together with the application:
- An overview of relevant works and sample reports
- Contact details for 2 references
-
Have permission to work, travel and undertake primary data collection in the project target locations
-
Have proven experience in conducting participatory qualitative and quantitative evaluation studies with superior analytical skills
-
Have a good knowledge of data collection software (such as Kobo, SMAP) and statistical packages (such as SPSS)
-
Have excellent report writing skills
-
Have excellent written and spoken English, Hold a minimum educational qualification equivalent to a master’s degree in a relevant field.
Evaluation Criteria
Mandatory Requirements
- Provide a certified copy of business registration(company/organization)
- Provide a certified copy of tax registration
- Provide information on ownership structure – name of directors/owners of the company(company/organization)
- References including names and contact information from previous clients who can be contacted regarding relevant experience in developing advanced interactive learning materials (At least three)
- Successful bidder will be required to sign World Vision Supplier Code of Conduct form
Technical Evaluation
Applicants need to clearly articulate on the following, but not limited to: –
- A cover letter outlining the suitability of the consultant in undertaking the assignment and summarizing relevant experience (Not exceeding one page)
- Consulting firm profile (Not exceeding one page)
- Up to two page statement of capability (referencing the selection criteria) and the approach to be take in undertaking this assignment
- Clear work plan including time frames for the final evaluation.
- Detailed CVs of key personnel (including any formal qualifications) should be provided
Financial Evaluation
- A financial proposal with a detailed breakdown of costs (including travel) quoted in USD. Applicable tax amount must be clearly stipulated and separated from the base costs.
- Payment Terms
- Credit Period
Submission of Proposals:
Interested individuals and firms should submit their proposal accompanied with a cover letter, in English by email to [email protected]on or before 17th July 2023.
Proposals should be submitted in three distinct/separate attachments, namely Mandatory Requirements, Technical Proposal and Financial Proposal
Email title should be; – End of Project Evaluation for Community-Led Capacity Strengthening For Fragile Contexts (C4FC) Project – Somali Resilience Program.
Note: Only shortlisted candidates will be contacted
[1] Kurt, S. “Kirkpatrick Model: Four Levels of Learning Evaluation,” in Educational Technology, October 24, 2016. Retrieved from https://educationaltechnology.net/kirkpatrick-model-four-levels-learning-evaluation/
How to apply
Submission of Proposals:
Interested individuals and firms should submit their proposal accompanied with a cover letter, in English by email to [email protected]on or before 17th July 2023.
Proposals should be submitted in three distinct/separate attachments, namely Mandatory Requirements, Technical Proposal and Financial Proposal
Email title should be; – End of Project Evaluation for Community-Led Capacity Strengthening For Fragile Contexts (C4FC) Project – Somali Resilience Program.
Note: Only shortlisted candidates will be contacted
To apply for this job please visit reliefweb.int.