End-Term Evaluation of the project, | “Right to Inclusive and Safe Education (RISE) for children in Kalobeyei”

1.Introduction.

Finn Church Aid (FCA) is the second-largest Finnish development cooperation organization and an important provider of humanitarian assistance with operations in 13 countries across the globe. FCA Kenya was established in 2014 and supports three thematic areas: Right to quality education, Right to livelihood and Right to Peace.

FCA Kenya began its operation in Education in Emergencies (EiE) in Kakuma/Kalobeyei in 2019. Through funding from UNHCR, UNICEF, MFA, and BPRM, FCA supports quality primary education in seven schools in Kalobeyei Integrated Settlement. The schools have a population of approximately 21,438 learners comprises of refugee and host community learners. Kalobeyei Integrated settlement hosts approximately 44,430 refugees and asylum seekers. Children aged 0-11 constitute 47% (48% girls and 52% boys), and adolescents aged 12-17 represent 19% (54% girls and 46% boys) of the settlement population (UNHCR population statistics,2022). South Sudanese refugees make up 74% (123,593) of the total refugee population, followed by refugees from Ethiopia (12.33%) and Burundi (7%).

2.Background of the evaluation

FCA Kenya is currently implementing a one-year project funded by the Bureau of Population, Refugees and Migration (BPRM) education project namely Right to Inclusive and Safe Education in Kalobeyei (RISE) in Kalobeyei Settlement. The project is implemented across 7 primary schools that have a population of 21,438 learners (56% boys,44% girls) and 229 teachers. The goal of the project is to provide inclusive and equitable quality primary education in a safe environment for refugee and host population boys and girls in Kalobeyei Refugee Settlement.

The project addresses the specific needs of the most vulnerable children, including children with disabilities (CwDs), out-of-school children (OOSC), and overage learners’ children in Kalobeyei Settlement and surrounding host communities in Kenya. The activities aim to improve access to quality education through primary teacher education training, enhance the protection, safety, and well-being of learners in the schools, and increase access to safe and inclusive primary education. The project also focuses on the psychosocial well-being of both the teachers and learners in the schools. The project is ending in July 2022 thus the need for end term evaluation. A baseline survey was conducted in the month of November 2021 and gives an overview of the situation at the beginning of the project.

Overall project impact: The goal of RISE is to provide inclusive and equitable quality primary education in a safe environment for refugee and host population boys and girls in Kalobeyei Refugee Settlement.

Project Objectives:

1. Objective 1: Increased access to safe and inclusive primary education for refugee and host population boys and girls in Kalobeyei refugee settlement.
2. Objective 2: Improved protection, safety, and well-being for refugee and host population primary school boys and girls in Kalobeyei refugee settlement, with a focus on the most vulnerable.
3. Objective 3: Improved quality of primary education for refugee and host population boys and girls in Kalobeyei refugee settlement.

3.Rational, purpose, and priority objectives of the evaluation

The overall purpose of the end-term evaluation is to assess the continued relevance of the action and the progress made towards achieving its planned outcomes. It will also determine in an objective manner the progress of project outcome indicators to inform the program implementation team on the status of planned milestones based on observed actuals. The results will also be used by FCA for learning to continue to inform the programming of EiE and accountability to the donor.

The scope of the evaluation entails conducting a detailed analysis of the project within its implementation context and establishing the extent to which the project has contributed to the FCA Quality Education mandate. The evaluation will mainly focus on the project activities Kalobeyei settlement of Turkana west sub-county and will target project beneficiaries, implementing partners, staff, and other stakeholders,

The specific objectives of the end-term evaluation are as follows:

• To evaluate the Relevance of the project activities to the context and stakeholders.
• Measure the efficiency of the project in relation to the resources.
• Assess the overall contribution of the action to improved quality education delivered to Boys and girls in Kalobeyei Settlement.
• Document challenges, programmatic lessons learned, and key recommendations for program improvement.
• Analyse the change brought by the project implementation through comparison with the baseline report.

4.Scope of the evaluation

The end term evaluation will be done across the 7 primary schools in the Kalobeyei settlement managed by FCA. The participation of all the stakeholders of the project is essential. This includes learners, parents, BOMs, PTAs, partners, government/County officials, and project staff. The duration of the assignment will be fifteen (15) days including travel. The evaluation will begin on July 15th, 2022; however, preliminary works may begin earlier such as the development of the tools and review of the final study design. The final report incorporating feedback from Finn Church Aid will be submitted 3-5 days after the presentation of the preliminary findings.

5.Evaluation question

1. To what degree have project outcomes been achieved? Were there any unexpected outcomes?
2. How did the project perform against SPHERE standards for the construction work?
3. What has worked well, and what has not worked well?
4. To what extent has the intervention been effective and coordinated between different actors, promoted synergies, and avoided gaps, duplications, and resource conflicts?
5. Evaluate the project against the OECD DAC Criteria:
   1. Relevance: to what extent did the intervention objectives and design respond to beneficiaries’ needs and priorities and continue to do so if circumstances change?
   2. Effectiveness: to what extent did the intervention achieve or is expected to achieve its objectives, and its results, including any differential results across groups?
   3. Coherence: compatibility of the intervention with other interventions in the same location?
   4. Efficiency: to what extent the intervention delivers, or is likely to deliver, results in an economic and timely way?
   5. Impact: to what extent the intervention has generated or is expected to generate significant positive or negative, intended, or unintended, higher-level effects?
   6. Sustainability: to what extent to which the net benefits of the intervention continue, or are likely to continue?
6. Evaluate the functionality and areas of improvement of the complaints and response mechanism established by FCA.
7. Assess the awareness of the beneficiaries on the child protection referral pathways.
8. Impact of the project on the safety and well-being of children within the school environment.

6. Methodology required & available data

The evaluation should employ a mixed-method approach in data collection. It is important to have clearly defined ways of collecting both qualitative and quantitative data. The use of mobile data collection is preferred.

7. Evaluation process

a) Preparation Phase:

• Conduct a review of the baseline report, the proposal, and the logical framework of the project.
• Meet with the relevant program team to discuss the current situation of the current project areas.
• Determine the timeline for the survey and the evaluation team needed.
• Write up the protocol of end term evaluation, including background, aims, specific objectives, methodology, sample size, sampling procedures, key indicators, evaluation team, ethical issues, timeline, data collection, field supervision, quality control, data entry, data cleaning, data analysis, and reporting
• Update the baseline evaluation to be used for end term evaluation tools (questionnaires) based on the indicators presented in the log frame.
• Criteria for final evaluation should be scoped on Impact, Sustainability, Relevance / Appropriateness, Effectiveness, and Efficiency.
• Present and discuss draft protocol and tools with relevant members. Make revisions as needed.

b) Training and Pre-testing Phase

• Train the evaluation team in end term data collection
• Discuss planned activities and data collection management with the relevant project team
• Conduct the field pre-test of the evaluation tools
• Assist in revision of End Term tools after pre-testing
• Organize the data in preparation for data analysis

c) Data collection

• Conduct field supervision and monitoring during the data collection
• Provide technical assistance during the data collection period
• Help with field coordination as needed during the data collection period

d) Data processing and reporting

• Perform the data entry preferably using SPSS software
• Conduct data cleaning
• Conduct data analysis
• Prepare the End Line report including summary of the findings, conclusions, and recommendations. Include modifications of report according to recommendations of the relevant project team members.

8.Timetable

Deliverable Duration

Inception report & work plan – 3 days

Data Collection – 4 days

Draft report – 3 days

Final evaluation report – 5 days

TOTAL DURATION – 15 days

9.Deliverables & Payments Deliverables under this consultancy will include the following:

1. An inception report: The consultant will share his/her inception report that details the study design (rationale, methodology), data collection tools, and a detailed work plan within 1-3 days of engagement, to be approved by Finn Church Aid (Programs Manager M&E officer & Education Specialist).
2. Data collection tools: Develop the data collection tools and have them approved by Finn Church Aid before data collection, analysis, interpretation
3. Draft Evaluation report: The consultant will prepare a draft evaluation report with details of findings, recommendations, and lessons learned for review by Finn Church Aid
4. Final Evaluation Report: The consultant will share a final evaluation report after incorporating the comments from Finn Church Aid. This will be both in electronic and hard copy (MS Word document, PowerPoint slides). All data sets: The consultant will deliver the data sets to Finn Church Aid.

The format of the report shall include the following: Executive Summary (max 2 pages) Preliminary pages – Acknowledgment, Acronyms, and Table of contents Main text, to include: Background Information: Brief on the Project and context; Evaluation – purpose, and objectives; evaluation limitations Methodology: Design, Sampling technique, and Sample size Findings: Analysis based on evaluation objectives and interpretation Key accomplishments and Summary of project assessment Lessons learned, Best practices and opportunities for improvement Recommendations Appendices: Evaluation tools, evaluation ToR, sample framework, etc. Present the draft report to the FCA team for review before producing a second draft and Final and present to FCA and other stakeholders in the workshop to facilitate sharing of evaluation results with a view to incorporate inputs from project stakeholders in the final draft. Submit a Final Evaluation Report to FCA as stated in the Consultancy Contract.

10.Skills and qualifications required

• Post-graduate degree in Education, Social Sciences, Statistics, or related field.
• Minimum of 5 -10 years’ relevant professional experience in education programming and research.
• Strong background and experience in data collection and analysis; experience in and able/willing to use mobile technology (tablets) for data collection
• Proven experience in undertaking evaluations/research or leading evaluation/research teams, with outstanding skills in qualitative and quantitative research and data analysis using relevant software such as STATA or SPSS and ATLAS
• Technical expertise in assessing issues of education demand and supply, and the ability to draw strong and valid conclusions

11.Budget

The consultant is advised to submit an itemized budget with realistic daily rates for the assignment based on the understanding of the assignment and the current market rates. Where need, Finn Church Aid will engage the consultant at an agreeable daily rate. Finn Church Aid will cover the consultant’s air tickets in economy class to relevant field sites, accommodation on bed and breakfast plus airport transfers in the field. All other costs shall be borne directly by the consultant. Remuneration will be based on the submission of deliverables that are satisfactory to Finn Church Aid as agreed upon after discussion with FCA in the inception meeting. The payments will only be made when the deliverables have been assessed by the FCA team to be of good quality.

The following payments will be paid to the consultant using an agreed mode of payment.

• After approval of inception report: 30%
• After approval of Final Report: 70%

12.Bids assessment

Qualitative award criteria

Skills / expertise of the evaluation team

Methodology for structuring, data collection & analysis

Organization of tasks and timetable

Price factor: cheapest tender / this tender

13.Terms of contract

As the consultant, will be working on behalf of Finn Church Aid they will be required to sign and adhere to the Child Safeguarding Policy and ethical guidelines. Note that background checks will be undertaken on all applicants.

The consultant will make clear to all participating stakeholders especially children of all ages that they are under no obligation to participate in the exercise and that it is purely voluntary. All participants will be assured that there will be no negative consequences if they choose not to participate. The consultant must obtain informed consent from all participants, including children and their caregiver if a child is to be interviewed. The consultant must receive prior permission for taking and use of visual still/ moving images for specific purposes. The consultant will assure the participants the anonymity, confidentiality and will assure the visual data is protected and used for agreed purpose only.

About the documentation, the title rights, copyrights, and all other rights of whatever nature in any materials used or generated under the provisions of this consultancy will exclusively be vested with Finn Church Aid-Kenya country office.