
World Vision
SOMALIA PROGRAMME
PROJECT ENDLINE EVALUATION TERMS OF REFERENCE
Building the Resilience of Women and Children in Protracted Crises through Increased Access to Education and Protection Services
Puntland, Somalia
Humanitarian Programme Plan – Irish Aid (2019 – 2022)
Project Number: S212183
- EVALUATION SUMMARY
Project name
Building the resilience of women and children in protracted crises through increased access to education and protection services: phase 2
Sectors
Education and Protection
Project locations
Eyl district, Godobjiran district, Dangoronyo district, Garowe district, Xarfo district and Dhahar district in Puntland
Project duration
January 2019 to December 31, 2022
Evaluation type
Endline Evaluation
Targeted peolefor this evaluation
Girls, Boys, Women, and Men
- CONTEXT
World Vision Somalia (WVS) has worked with the children of Somalia, their families and communities since 1992 through a variety of emergency and rehabilitative programming to address the emergency needs of the communities while addressing some of the underlying causes of vulnerability in those same communities. During the last 25 years, the program has grown to 15 districts spread over three main operational regions of Puntland, Somaliland and South-Central with a liaison office in Mogadishu.
The protracted crisis in Somalia continues to be characterised by recurrent extreme climatic shocks, continued insecurity and armed conflict, human rights violations and a lack of infrastructure, all of which contribute to the high level of humanitarian need in the country. Approximately 5.4 million people need humanitarian assistance, including 3 million children (OCHA’s 2018 Humanitarian Response Plan (HRP)). In 2016 and 2017, the situation was aggravated by the impact of four consecutive poor rainy seasons, leading to large-scale new displacement and increasing protection concerns. During the drought, 59% of respondents in a child protection assessment reported an increased likelihood of early marriage[1]. Drought conditions also led to increased student absenteeism from schools as families began to rely on negative coping strategies including enlisting children in the search for water, which in turn can increase children’s exposure to protection risks, including GBV.
Only approximately 30% of school-aged children are enrolled in schools while more than three million remain out-of-school. In addition, only 17% of children living in rural areas or in IDP settlements are enrolled in primary schools[2]. At the root is the inherent vulnerability of the education system which lacks adequate learning facilities, trained teachers, teaching and learning materials and toilet facilities. The Federal Ministry of Education Culture and Higher Education (MOECHE) has drafted an Education Sector Strategic Plan (ESSP) which has been adapted by the Puntland Ministry of Education and Higher Education (MoEHE) for 2017-2021, prioritising access to quality education for children.
Eyl District is one of the most vulnerable areas in the Nugaal region of Puntland. According to an assessment[3] conducted by WVS in April 2018, 57% of the surveyed households were at poor food consumption scores (FCS) and the other 22.6% were at borderline. In Eyl district, World Vision’s needs assessment shows that 68% of adults did not receive any formal education and only 42% of children are currently attending primary school. Pastoralist communities in the new proposed programme areas have also been severely disadvantaged in accessing education opportunities that can be accessed without compromising their livelihood. Just over half the school age children are in school and availability of school facilities and teachers in Dhahar district is amongst the lowest in Puntland according to the assessment[4].
The protracted crisis is further compounded by governance challenges, weakened resilience and economically vulnerable communities. Involuntary population movement caused by violence and extreme climate conditions such as drought and flooding have resulted in undignified living circumstances with severe protection implications to all affected, especially women and children. Protection concerns are aggravated by lack of access to services, low awareness of basic rights, existing gender discriminatory and harmful socio-cultural practices and marginalisation. Incidents of GBV affecting women and girls remains underreported but widespread, especially among IDPs. Key risks for children are family separation, child recruitment, arbitrary arrest, and elevated exposure to GBV including early marriage. In 2016 there were over 7,200 reported cases of GBV according to UNDP. Considering the stigma attached to GBV, the actual number of cases is likely to be higher.
- SUMMARY OF THE EXPECTED PROJECT RESULTS
Objective 1: Child Protection: Increased protection for displaced, vulnerable and conflict affected children aged 3-18.
Outcome 1.1 Community members and structures have Increased awareness and action to identify, assess, and mitigate child protection risks, coordinating with government structures.
- Proportion of children who know of the presence of services and mechanisms to receive and respond to reports of abuse, neglect, exploitation or violence against children (disaggregated by gender and age)
- Proportion of boys and girls with birth registration
- Percentage of parents/caregivers (male/female) who demonstrate improved knowledge, attitudes and practices on child protection (disaggregated by gender)
- # of child protection cases identified and referred to the appropriate child protection mechanism
Objective 2: Education: Improved Access to quality primary and life skills education for girls and boys aged 6-18 in target communities.
Outcome 2.1: Girls and boys have access to quality learning
- % increase in the number of boys and girls enrolled in school
- Proportion of boys and girls who complete a full school term
- Proportion of girls and boys in grade 3 achieving a least a minimum proficiency in reading.
Outcome 2.2: Parents and community have more positive attitudes towards the value of education and participate in education management
- % of boys and girls who state that they have used learning resources at home in the past week.
- % of parents reporting an improved knowledge, attitudes and practices in supporting their children’s learning
- % children (boys, girls) reporting an increased frequency of participation in literacy activities with household members
- % of PTAs implementing agreed actions to promote primary education for girls and boys
Objective 3: Protection: Increased prevention of and response to GBV and access to empowerment opportunities for women and girls.
Outcome 3.1 Increased awareness and action on GBV prevention, mitigation and response among women, men, girls and boys
% of community members (men and women and boys and girls) with improved knowledge, attitudes and practices towards mitigating/reporting actions on GBV
Outcome 3.2 At risk women and girls and GBV survivors have increased access to appropriate quality psycho-social support and response services
- # and % of GBV survivors referred to specialist GBV services
- % of GBV survivors who report satisfaction with quality of GBV response services.
Outcome 3.3 Increased linkages with government structures and community committees formed for sustainability
# and % of project activities or trainings carried out in conjunction with two or more local stakeholders
- EVALUATION TYPE
This is an endline external evaluation of the HPP project and the process will necessitate a complete review and analysis of the project from its start date to the end date (1st June 2019 to 31st December, 2022).
- PURPOSE OF THE EVALUATION
The endline evaluation will assess the project performance and analyse the key criteria of evaluation in emergency contexts in the project area with reference to the Development Assistance Committee of the Economic Cooperation and Development (OECD/DAC) evaluation criteria[5], i.e. relevance, effectiveness, efficiency, internal and external coherence, impact, and sustainability. In addition, the evaluation will also assess the aspect of project coverage in relation to the local context. This evaluation will provide lessons learnt, good practices, and recommendations and direction for future programming in the areas of building the resilience of women and children in protracted crises through increased access to education and protection services in Puntland in Somalia. The results of the evaluation will inform the Management Response Plan.
- OBJECTIVES OF THE EVALUATION
In view of the purpose of the evaluation highlighted above, the specific objectives of this evaluation are:
- To assess the relevance, effectiveness, efficiency, internal and external coherence, impact, sustainability, and coverage of the HPP project;
- Provide information on what worked, what did not work, and why in regard to achieving or not-achieving the objective and the anticipated outcomes of the project on the lives of children, families and communities in the targeted project area;
- To assess key lessons learnt and good practices across the project cycle stages (initiation, planning and design, implementation, and closing), and provide a set of clear actionable recommendations.
- MAIN EVALUATION QUESTIONS
The evaluation will focus on assessing 7 main evaluation question:
Relevance:
- To what extent was the HPP project designed to meet the needs of the targeted population and WV Strategy?
Effectiveness:
- To what extent were the objectives and intended outcomes of the HPP project achieved?
Impact:
- To what extent did the project make positive (intended and unintended) and negative changes, especially in the lives of the beneficiary girls, boys and women?
Efficiency:
- To what extent did World Vision Somalia make good use of its human and financial resources as well as an appropriate combination of tools and approaches to pursue the achievements of the project results?
Coherence:
- To what extent did the HPP project establish meaningful internal synergy with other WV Somalia projects as well as external relationships and linkages with other actors?
Sustainability:
- To what extent will the achievements and impacts of the HPP likely to be sustainable beyond the life of the project?
Coverage of the HPP project:
- To what extend did the HPP project cover an appropriate geographical area in relation to the need and context?
- EVALUATION SCOPE
Evaluation data will be gathered from the geographical areas where the project has been implemented in Puntland. Since this is the summative evaluation, it will be assessing the project in line with the questions indicated covering the project duration from 1st June 2019 and 31st December 2022 and investigating in detail, the results framework of the project.
- EVALUATION AUDIENCE
The evaluation findings are intended to inform multiple stakeholders that are involved directly and indirectly with the HPP project. These stakeholders include:
- Community groups and committees that were involved in the project implementation
- Local and international organisations that are operating within the project area
- Local administrative offices that were involved in the project implementation
- Ministry of Education (MoE)
- Ministry of Women Development and Family Affairs
- Ministry of Justice, Religious Affairs
- Religious leaders in the project area
- World Vision
- METHODOLOGY
WV Somalia and WV Ireland will provide the evaluation design (pre-post design), methodology and sampling plan. The selected consultant will be inducted on these design, methodology and tools that the evaluation will adopt. The evaluation will adopt KAP household survey, child/youth survey, and school observation tool. Quantitative and qualitative methods as well as in-depth review of secondary data will be used.
Table showing the estimated number of the targeted households per project village
District
Village
Estimated Households
Samples size
Garowe
Dudumaale
100
TBD
Eyl
Kabaal
150
TBD
Mareeya
300
TBD
Ceelmadoobe
45
TBD
Biyo-Cade
70
TBD
Diilin
130
TBD
Cambare
110
TBD
Aftogweyne
43
TBD
Godobj-jiraan
Suubaan
80
TBD
Wargaduud
130
TBD
Weyloliq
40
TBD
Dangorayo
Taageer
600
TBD
Yibaayil
100
TBD
Dhahar
Dhahar IDP
15
TBD
Baragaha Qol
200
TBD
Gacal guule
25
TBD
War-idaad
120
TBD
Boodacad
100
TBD
Harfo
Ceelgardi
75
TBD
Gaxandhaale
70
TBD
Mindhiciraan
30
TBD
Total
2,533
TBD
Table showing the number of the targeted young people/students (in grade 6 and above) in the schools supported by the project (samples sizes TBD)
School names
Boys
Girls
Total
Wargaduud primary school
130
119
249
Suuban primary school
137
116
253
Maraya Primary School
109
141
250
Kabaal Primary School
168
97
265
Diilin Primary School
184
70
254
Ceelmadooble Primary School
65
52
117
Biyo-addo Primary School
88
43
131
BaragahaQol Primary School
171
140
311
Cambare Primary School
101
67
168
Aftogweyne Primary School
108
85
193
Weeylolaq Primary School
73
61
134
Ceelgardi Primary School
89
72
161
Gaxandhale Primary School
35
28
63
Yibaayil Primary School
100
72
172
Taageer Primary School
148
117
265
Dhahar IDP Primary School
7
12
19
Boodacad Primary School
36
37
73
Gacalguule Primary School
30
30
60
Waridaad Primary School
71
76
147
Dudumaale Primary School
51
46
97
Maindhiciraan Primary School
29
27
56
Total
1,930
1,508
3,438
Table showing the number of teachers in schools supported by the project
Estimated #
Head teachers
TBD
Teaching staffs
TBD
- AUTHORITY AND RESPONSIBILITY
WVS will establish a team to oversee all the processes and activities relating to this evaluation. The evaluation manager will be the DME Manager – North under overall guidance of WV QA&S manager quality assurance. Process coordination will be handled in liaison with WV Ireland.
- SCOPE OF THE ASSIGNMENT
The consultant will lead on data analysis and report-writing tasks of this evaluation.
- ROLES AND RESPONSIBILITIES
- World Vision Somalia will:
- Determine the evaluation design, methodology, sample sizes, and data collection tools (including testing these tools)
- Recruit the Consultant and finalize the engagement Agreement/Contract
- Provide the necessary documents and information to the Consultant
- The Consultant will take care of the stationery and printing facilitates they needed
- Provide general guidance and coordination to the evaluation process
- Enrol, train and facilitate the enumerators
- Make payments for the consultancy services as per the arrangements provided herein.
- The Consultant will:
- Submit a technical and financial proposal (as one document). This should reflect excellent interpretation and understanding of the assignment.
- Provide to WVS a work plan for the assignment, clearly indicating the schedule of the activities to be undertaken
- Receive and analyse the primary data (quantitative and qualitative)
- Review all the relevant secondary data (desk review)
- Write and submit to WVS the draft evaluation report. The Consultant will present the preliminary findings to WVS
- Revise and submit the final copy of the evaluation report, including recommendations
- Submit to WVS the complete raw data sets. The variables names and values should be clear
- KEY DELIVERABLES
Throughout the assignment, the Consultant is expected to deliver the following key deliverables:
- Structure of evaluation report
- Presentation of the key findings and recommendations
- Final evaluation report (soft copies) submitted to WVS
- Raw data sets (both quantitative and qualitative)
- BRANDING/COPYRIGHT AND INTELLECTUAL PROPERTIES
- Copyright solely belongs to World Vision; therefore, all productions and written content will be purely World Vision property
- All products must be in line with World Vision branding guideline. Any other logo allowed, must be approved by World Vision Senior Communications Manager.
- ELIGIBLE APPLICANTS
Consultancy service company/entity/Institution or individual consultants.
- EVALUATION TIMEFRAME
The entire evaluation exercise is expected to take 14 working days. Most of the tasks will be implemented concurrently to fit within the planned time-frame.
- EVALUATION CRITERIA
The successful Consultant will be evaluated and selected based on qualifications, strengths of the technical and financial proposals, and proposed timeframe/availability.
-
- Mandatory Requirements:
-
Provide a certified copy of business registration(company/organization)
-
Provide a certified copy of tax registration
-
Provide information on ownership structure – name of directors/owners of the company (company/organization)
-
Reference letters at least 3
-
Successful bidder will be required to sign World Vision Supplier Code of Conduct form
- Technical Evaluation Criteria:
-
The consultancy team/individual consultant is expected to have minimum of a Master’s degree from a recognised University in: International Development, Education, Human Rights, Law, Gender and Development, Sociology or related discipline
-
Work experience in conducting endline evaluations in Somalia and/or other similar Fragile contexts – attach Contracts
-
Excellent analytical report writing skills, particularly evaluation reports
-
Detailed CVs of key personnel (including copies of formal qualification, be provided)
-
Clear work plan including deliverables and time frame not exceeding 14 working days
-
Consulting firm/person profile
-
Consulting firm/person Statement of capability and proposed quality assurance arrangements
-
Proven knowledge of evaluation ethical considerations and Compliance to Covid-19 protocol (including vaccination)
- Financial Evaluation:
All bidders are advised to submit the following Financial Proposal:
- Total cost to deliver on project (Quote in US dollars)
- Payment Terms
- Credit Period
-
- Other:
- Professional fluency in English, and native proficiency in Somali language.
- Strong writing and analytical skills
- APPLICATION PROCESS AND REQUIREMENTS
Qualified and interested parties are asked to submit the following;
- A Technical Proposal of not more than 15 pages. This should contain:
- Letter of interest
- The consultants’ understanding of the ToR
- The proposed study evaluation design and methodology, including sampling plan
- Proposed work plan with estimated timeline
- Capacity statement/profile
- Registration and Tax compliance certificates
- Detailed CVs of individuals or team members
- Three reference letters
- A financial proposal with amount of working days and daily rates
- SUBMISSION
All interested bidders/consultants are requested to submit their Technical Proposals and Financial Proposals in Separate documents as attachments (Bidders who will combine both technical and financial proposals shall be disqualified) via email [email protected] on or before 19th September 2022
Email Subject line: WVS End line Evaluation of HPP Project
As part of the Consultant selection process, the best three candidates may be required to make a presentation of the technical proposal to the Supply Chain and Core Project Technical team to inform the final decision on the award of the contract.
Bids received after deadline shall not be considered.
[1] Child Protection Rapid Assessment (April 2017), Save the Children.
[2] Educational Characteristics of the Somali People Volume 3, UNFPA, 2016
[3] HPP Irish Aid Education and protection project in Eyl District, Puntalnd, Somalia. 2018 to 2019, Baseline Report, WVS (July 2018)
[4] HPP Irish Aid Education and protection project in Eyl District, Puntalnd, Somalia. 2018 to 2019, Baseline Report, WVS (July 2018)
[5] OECD (2021), Applying Evaluation Criteria Thoughtfully, OECD Publishing, Paris, https://doi.org/10.1787/543e84ed-en.
How to apply
SUBMISSION
All interested bidders/consultants are requested to submit their Technical Proposals and Financial Proposals in Separate documents as attachments (Bidders who will combine both technical and financial proposals shall be disqualified) via email [email protected] on or before 19th September 2022
Email Subject line: WVS End line Evaluation of HPP Project
Bids received after deadline shall not be considered.