TERMS OF REFERENCE – PROJECT FINAL EVALUATION – EDUCATION AND LIVELIHOODS SUPPORT FOR REFUGEES IN KENYA IMPACTED BY COVID – 19

1.2 About Humanity & Inclusion in the country/region

HI has been operating in Kenya since 1992, implementing projects in development and humanitarian contexts. In Dadaab and Kakuma from 2007 and 2014 respectively, implemented community-based rehabilitation, Inclusive education, and inclusive livelihoods projects.

In refugee settings and their surrounding host environs, persons with disabilities face challenges, due to numerous barriers that limit their active participation in decision-making processes and are less likely to have their humanitarian protection needs met. Most affected are children with disabilities, as well as children whose caregivers live with a disability. Further, situations of displacement and emergency have a strong psychological impact on boys and girls with and without disabilities as they are uniquely vulnerable due to their age and social status.

As of 31st May 2020, 494,874 refugees and asylum-seekers were residing in Kenya (217,514 in Dadaab and 196,345 Kakuma / Kalobeyei). 53.5% of the total refugee population are under 18 years old. Over 150,000 refugee learners were enrolled in 40 pre-primary schools (24,750 [1,534 girls]), 48 primary schools (107,756 [44,363 girls]) and 16 secondary schools (24,588 [5,617 girls]) across the country as of December 2019, 40% of whom were girls. An additional 120,000 school-aged refugee children were out of school prior to the pandemic. The net enrolment rate for primary school in Dadaab is 37.7%, 62.3% aged 6 – 13, while 44.2% are over 13 years (UNHCR EMIS, 2019) and 2.7% children with disabilities; while the population of learners in Kakuma includes 45% who are over-age; 25% girls, and 2.6% children with disabilities.

The population of refugees faces many risks as a result of multiple and complex unmet needs that cut across social, medical, and economic dimensions. They also face intersecting oppressions based on race, ethnicity, nationality, language, class, gender, sexual orientation, and disability.

2) Context of the evaluation

2.1 Presentation of the project to be evaluated

Project title

Education and Livelihoods Support for Refugees in Kenya impacted by COVID-19

Hereinafter referred to as “COVID_19 Recovery and Resilience Program (CRRP)”

Implementation dates

February 2021 to March 2023

Location/Areas of intervention

Kakuma and Dadaab

Operating Partners

Government of Kenya, UNHCR

Target Groups

Young men and women with and without disabilities, caregivers of children or household members of persons with disabilities, Children/ learners with disabilities, teachers and education actors/ stakeholders

Objectives of the project

Objective 1. Promote continuous and improved equitable access to learning opportunities for learners with disabilities through online learning and technology-mediated learning.

Objective 2. Ensure safe and dignified living conditions for young women and men with and without disabilities who own micro, small and medium enterprises (MSMEs) via continuous remote learning of needed professional skills and expanding access to finance.

Expected results and indicators

Log frame to be shared in the technical offer

Main activities implemented

Proposal to be shared with the technical offer

2.2 Justification of the evaluation

HI is seeking the services of a consultant to conduct final project evaluation for the project in the refugee hosting locations in Kakuma and Dadaab in line with HI’s planning, monitoring, and evaluation policy and the provision of the project design (proposal).

The purpose of the final project evaluation is to generate and provide mixed method quantitative and qualitative endline values for the impact, and outcome of the project as a whole whilst monitoring indicators per the overall logical framework of each project location for the COVID_19 Recovery and Resilience Program. The endline situation will enable the project to establish the standpoint for the different indicators (situation after) which will be used to compare with the situation before implementation of the project (baseline survey and during its implementation through risk assessment).

This will enable the project to assess qualitatively and quantitatively the contribution of the project to enhancing the protection of vulnerable refugees and host communities and thus ascertain the aspects that will be pivotal in attributing change as a result of the project. This will also help document best practices when designing a recovery project.

3) Objectives of the evaluation

1. 1. Overall objectives and expectations of the evaluation

The main objective of the final project evaluation is to define the achievements of result according to;

1. The objectives of the project

2. The strategies of implementation

3. The immediate effects: positives and negatives effects

That will establish the recommendations/impact/outcome and endline values of the project to improve our projects in the future.

3.2 Specific objectives

• Provide overall learning, lessons learned and good practices, through recommendations on specific areas of focus that the project should consider based on the findings of the final evaluation.
• Assess through a research methodology endline values for each of the project indicators and comparable to baseline values in intervention areas.
• Assess the effectiveness of digital learning solutions in maintaining or improving learners with disabilities learning outcomes during the COVID-19 response and recovery period in Dadaab and Kakuma.
• Assess the effectiveness/impact of the virtual clubs / (effectiveness of the provision of phones to MSMEs) formed to the MSMEs.
• Assessing the effectiveness of the training and mentorship program in improving MSMEs’ capacity to cope with COVID-19 related market disruptions through virtual platforms.
• Examine the impact of the program on MSMEs’ profitability, growth, job creation and ability to access new markets and customers.
• Assess the impact of the caregiver’s manual on children with disabilities at home learning level.
• Assess the impact of Universal Design of Learning to learners with disabilities, teachers and TVET instructors in schools.

3.3 Evaluation criteria and evaluative questions

The specific objective of the final evaluation is to provide an analysis of the quality of the project-based HI Quality Framework (annexed)

The guiding questions listed below are the basis for the evaluator. The applicant for this assignment should submit further (sub-questions as part of the application documents and the selected evaluator will, at the inception phase of the consultancy, discuss a more comprehensive TOR with HI which also covers the evaluation objectives, analytical framework, methodologies, timeline and deliverables for HI’s approval.

Criteria

Changes

(Effects, Continuity, Empowerment)

Evaluative Questions

• Did the project produce significant positive changes in the lives of the beneficiaries in a sustainable manner?
• Has the project sufficiently addressed the protection risk of the beneficiaries?
• Are the actions implemented during the project really sustainable by community structures?

Relevance

(Needs, Context, Lessons learned)

Evaluative Questions

• To what extent did the project meet the needs of the direct beneficiaries?
• Did the project use lessons learned during the project cycle to continually enhance its intervention?

Effectiveness

(Results, Adjustments, Technicality)

Evaluative Questions

• To what extent have the project activities effectively contributed to the attainment of the project’s objectives, as determined by results-based management?
• To what extent is the current solution successful in delivering its desired outcomes?

Efficiency

(Skills, Optimization, Responsiveness)

Evaluative Questions

• Have the necessary skills (staff) been developed to carry out the project?
• How has the project increased the capacity of community workers/caregivers so that they are more productive and autonomous?

Accountability to Populations

(Participation, Expression, Information)

Evaluative Questions

• To what extent has the project put in place mechanisms to involve beneficiaries and partners?
• Were the mechanisms of expression inclusive regardless of gender, age or disability?

4) Evaluation methodology and organization of the mission

4.1 Collection methodology

The consultant will use a range of methodologies that shall be participatory, inclusive, gender sensitive, child-friendly and disability friendly to the key stakeholders. Wherever possible, the data obtained should be triangulated to reach conclusions based on a thorough analysis. The data set needs to be comparable with baseline data. Methodologies need to be adapted to ensure proper calculation of each indicator.

The Final Project Evaluation will amongst others consider:

• Secondary data sources – Desk review of the project documents / external reports related to protection and disabilities – Proposal document including the Log Frame, M&E plans, project quality frameworks, baseline, etc.
• Primary data sources (Key Informant Interview, FGDs and survey) and data analysis process disaggregated by the type of disability, gender, and age parameters.
• Data collection methods: Quantitative methods including well-developed survey tools (questionnaires) to be administered to sampled beneficiaries; Qualitative methods through a thorough analysis of information gathered from observations, key informant interviews, focus group discussions, etc.
• Sampling methods – that reflects a diversity of respondents and experiences, including different types of disability, country of origin of refugees. Data collection should be conducted in the language chosen by the respondents.
• Tools will be developed in English and translated into Kiswahili and other language as needed (Juba Arabic, Somali, French). Sign language interpretations may be used when relevant. The interviewers will have to be trained in communication with people with disabilities. Beneficiary friendly approaches and tools as per the age, gender, and type of disability policy provisions.
• Comparative approach: the methodology should contain a comparative analysis between the starting period and the end, but also between boys and girls & children with disabilities and children without disabilities, various types of disabilities

All data collected should be disaggregated and analyzed by site, sex, age, and domains of functional difficulties (following the Washington Group short set of questions).

4.2 Actors involved in the evaluation

Consultant(s)

• Detailed inception report: Containing detailed work plan specifying activities and timelines agreed with HI; a detailed description of the final evaluation/endline methodology, ethical considerations, and data collections tools based on HI policies (annexed)
• Develop and provide guidelines and tools in data collection, data cleaning process, and data entry for data enumerators for review by HI team
• Training of data enumerators: Using mobile data collection tool developed by the consultant in collaboration with HI, Ethics and security related to collection of sensitive data, communication with persons with disabilities, GBV/CP issues.
• The consultant should coordinate the collection of field data, the day-to-day management of evaluation activities, data entry and analysis, and report writing while respecting the confidentiality and sensitivity of the evaluation.
• Data analysis, presentation of key research findings and draft report presentation for review and validation: quantitative and qualitative data analyzed and triangulated.
• Final Report Developed and Presented: finalization of the report (English language) in close collaboration with HI, with relevant annexes in the same language as the report. Transcript in English of the qualitative data should be shared to HI.
• PowerPoint presentation of final report
• The consultant will ensure that the endline study activities are carried out per the terms and conditions of his or her contract.

Humanity & Inclusion team

• HI staff will compile the reference documents, which include proposal, midterm assessment reports, log frame, and budget to be consulted;
• Facilitate access to information on travel and accommodation
• Provision of access to data collection software (survey CTO).
• Facilitate access to information on workshop facilities, data enumerators previously engaged with HI if needed.
• A briefing session will be held with the consultant and the different HI technical and operations personnel (Field, regional and HQ), MEAL at the beginning to agree on the overall methodology and re-emphasize key issues to be addressed.
• Provision of guidance on HI’s evaluation policies/ quality framework to the successful consultant
• Support consultant through mobilization of the respondents (beneficiaries, other stakeholders, and Government agencies;
• Pay consultancy fees for the completed work as per the agreed timelines

Beneficiaries

• Respondents/validators

Partners

• Respondents/validators

4.3 Organization of the mission

1. ARCI is to ensure compliance with the evaluation process.

Authority

Roles & Responsibilities: – Enforces the process and ensures quality of each stage of the process.

Positions : – Regional MEAL Manager/ Country Manager

Responsible

Roles & Responsibilities:- Implements the different steps of the process.

Positions :- Project Managers (Kakuma and Dadaab)

Contributors /Informed

Roles & Responsibilities:- Participate in the different stages of the process according to their expertise, on general or specific aspects

Positions:- Field Specialist, MEAL team, Regional Economic Inclusion Specialist, Regional Inclusive Education Technical Specialist, Global Specialists (Education and Livelihood), Operations Officer HQ, Technical Unit Manager, Logistics, Human Resources, Operations Managers, Project Staff

2. Steering Committee in charge of the quality of evaluation and composed of 5 people.

It has two main functions: –

1. It is the body that steers, guides and validates evaluation decisions.
2. It is the body that gives an assessment of the quality of the evaluator’s work

Composition

1. Project Managers (Kakuma & Dadaab)
2. Country Manager
3. Technical Specialist
4. Logistics Manager
5. MEAL

Overall, Roles and Responsibilities of the Steering Committee

Selecting the evaluator

The Steering Committee shall evaluate the submitted applications and validate the choice of evaluator and ensure the impartiality of the selection.

Scoping Meeting and Inception Report

The Steering Committee shall participate in the scoping meeting (methodology, expected results…) and validate the inception report for the future steps.

Presentation: evaluation findings and recommendations

The Steering Committee and the evaluator ensure that they have a common understanding of the conclusions & recommendations expressed.

The Steering Committee provides elements that allow the evaluator to refine his recommendations.

Quality of the final report

The Steering Committee shall specify the quality aspects expected from the evaluator for the final report, and provide feedback on the draft report

Final report

The Steering Committee provides feedback on the draft report and then validates the quality** of the evaluator’s final report.

Elaboration of an action plan and follow-up of the recommendations.

The Steering Committee participates and validates, with the teams, the action plan and follow-up of the recommendations.

5) Principles and values

5.1. Protection and Anti-Corruption Policy

Code of Conduct

Protection of beneficiaries from sexual exploitation, abuse and harassment

Child Protection Policy

Anti-fraud and anti-corruption policy

5.2. Ethical measures*

As part of each evaluation, HI is committed to upholding all humanitarian standards and ethical considerations. It is imperative that these measures are taken into account in the technical offer:

• Guarantee the safety of participants, partners and teams: the technical offer must specify the risk mitigation measures. Examples include:
• The collection of information should be carried out in a way that presents the least possible risk for the investigators and the interviewees (e.g. safe space, anonymous collection, etc.), be methodologically sound and be based on current experience and good practices
• Basic care and support for survivors/victims should be available locally and known by interviewers before beginning any activities that may involve disclosure of information about their experiences of violence (e.g HI and other partners toll free number to be provided)
• All members of the data collection team should be carefully selected and receive relevant specialized training (basic concepts on GBV/CP, PFA and safe referral, communication with persons with disabilities, interviewing children and/or survivor) as well as ongoing support.
• Additional safeguards will need to be put in place if children (i.e., those under 18 years of age) are interviewed.
• It will be strongly recommended to have a CP specialist on the assessment team and gender balanced team;
• Assess the interviewers’ attitudes towards sexual violence, gender, disability and the survivor-centered approach during the recruitment process of data collector.
• Ensuring a person/community-centered approach: the technical offer must propose methods adapted to the needs of the target population
  • Tools to be adapted for illiterate audiences / having sign language interpreters / child-friendly, gender sensitive and disability friendly materials, etc.).
• Obtain the free and informed consent of the participants: the technical proposal must explain how the evaluator will obtain the free and informed consent and/or assent of the participants.
  • Informed consent should be obtained from all individuals before participating in the data collection activity and HI consent forms signed
• Ensure the security of personal and sensitive data throughout the activity: the technical offer must propose measures for the protection of personal data.
• The confidentiality of those providing information about violence should be protected at all times

*These measures may be adopted during the completion of the inception report.

6) Expected deliverables and proposed schedule

6.1. Deliverables

• An inception report refining / specifying the proposed methodology for answering the evaluation questions and an action plan. This inception report will have to be validated by the Steering Committee.
• A presentation document presenting the first results, conclusions and recommendations, to be presented to the Steering Committee.
• A final report of approximately 30-40 pages maximum and the annexes: (including both quantitative and qualitative dataset)
• A summary of maximum 5 pages
• PowerPoint presentation of the findings

The final report should be integrated into the following template: https://bit.ly/41QL1Z1

The quality of the final report will be reviewed by the Steering Committee of the evaluation using this checklist: https://bit.ly/3S4SB23

6.2. End-of-Evaluation Questionnaire

An end-of-evaluation questionnaire will be given to the evaluator and must be completed by him/her.

6.3. Evaluation dates and schedule

Propose a table showing the different steps (e.g., Briefing at Nairobi office, documentary review, inception report, field mission, field presentation, drafting of the final report) and the desired periods for carrying out these stages.

Phase

Hiring Consultant

Advertise the TOR to hire a consultant 13/02/2023 – 26/02/2023

Recruitment of Consultant (interview, etc.) 27/02/2023 – 03/03/2023

Phase-1

Desk Review – Inception Report

Consultant conducts desk reviews, develops Inception Report and data collection tools, and presents to the Steering committee 06/03/2023 – 07/03/2023

Steering Committee reviews 08/03/2023- 09/03/2023

Phase-II

Field data Collection

Training of Enumerators/data collectors 10/03/2023

Make appointments with respondents

Field Data collection – 19/03/2023

Phase-III

Data Analysis, Report writing

Data Cleaning, Data Analysis 20/03/2023 -24/03/2023

Draft preliminary report with recommendations 20/03/2023 -24/03/2023

Steering Committee reviews draft report 25/03/2023 – 26/03/2023

Finalize the feedback and share the final report 27/03/2023 – 29/03/2023

Dissemination

Disseminate findings with Stakeholders through a dissemination meeting 30/03/2023

(End of evaluation questionnaire)

The evaluator is requested to fill in the HI end-of evaluation questionnaire (ref 6.2 above)

* HI activity timelines

7) Means

7.1 Expertise sought from the consultant(s)

The required background and experience of the Consultant are as follows:

• Master’s degree in social sciences, Statistics, Monitoring, and Evaluation, or other relevant fields.
• Previous experience in the design, implementation, analysis and reporting of impact studies, endline studies, final surveys, and project evaluations, in the field of education, livelihoods, child protection or disability
• Proven knowledge and skills in quantitative and qualitative analysis and excellent analytical skills
• Institutional knowledge of International Non-Governmental Organizations (preference in education, livelihoods, disability, gender, and protection).
• Rights based understanding and familiarity of disability, and related experience in working and engaging with persons with different types of disabilities,
• Excellent command of written and spoken English. Knowledge of one local language (Swahili an asset)
• Experience in using participatory and child-friendly methods
• Strong analytical and reporting skills
• Good understanding of protection systems in sub-Saharan Africa/Kenya context (particularly in humanitarian and/or refugee settings)

The specific competencies and qualifications required of the Consultant are as follows:

• Excellent knowledge of the socio-political environment of the refugee-hosting locations in Kenya.
• Proven knowledge and experience in disability and gender issues.
• Knowledge in using Washington group of questions specifically the UNICEF/Washington Module on Child Functioning.
• Practical knowledge of the use of Mobile Data Collection (MDC) in surveys.
• Proven ability to deliver quality reports/ analysis and results in line with established deadlines
• Proven experience of having participated previously in final evaluations or similar analytical work will be a plus.

7.2 Budget allocated to the evaluation

The candidate must detail in his offer: the cost per day for each evaluator; the breakdown of the time spent per evaluator and per stage of work; the ancillary costs (services and additional documents); the overall cost of the intervention including transport costs (international and local), logistics costs, and translation costs; with proposals for payment modalities, costs for reasonable accommodation. The financial proposal will indicate the all-inclusive costs of undertaking the final project evaluation.

Payment will be released in two installments:

• 50% upon signature of the contract

• 50% upon submission of the final validated report

Service cost allocation budget code: D41_072/TB03/DADA/FD41_205/4.49, D41_075/TG05/KAKU/FD41_205/4.52

Caution: please note that the last payment is conditional on the validation of the final report and not on the sending of the final report. By validation, we mean validation of the quality and under no circumstances of the appreciation of the project evaluated (based on the quality checklist attached, chapter 6).

7.3. Available resources provided to the evaluation team

(Data, documents, software (SurveyCTO)

8) Submission of applications

Application documents submitted will be evaluated in three (stages);

1. General Mandatory Evaluation Criteria

For firms or institutions

Certificate of Registration

Trade license Certificate/ Business Permit

PIN/VAT Registration Certificate

Current Tax Compliance Certificate

Personal Accident Cover

For individuals

National ID

Relevant Certification

PIN/VAT Registration Certificate

Current Tax Compliance Certificate

Personal Accident Cover

2. Technical Capacity Evaluation

a) Profile and experience of the company:

1. The firm should have proven expertise and experience in undertaking service delivery. Attach three reference letters
2. Provide a list of assignments, evaluation reports and research reports undertaken with a similar scope of work within the last 2 years along with names of clients (including contact person, email contacts, Postal address and telephone numbers – this must include the three referees provided above)

b) Professional/technical capacity/ capability

The proposed staff should have relevant experience and professional qualifications in the relevant field

c) Administrative and technical appendices

(List of contract documents, e.g. training plans, any communication media (DVDs, CD-ROMs, etc), price breakdown, methods, general purchasing conditions, etc.).

I. Methodological approach/Work schedule adopted for the service provision

II. Itinerary

III. Consultant’s CV

IV. Other

3. Financial Evaluation

At the financial stage, the competitive evaluated bidder and within budget estimates will be considered for the award of the contract.

How to apply

HOW TO APPLY

Potential service providers who meet these requirements and are available within the time period indicated should submit an application letter and curriculum vitae indicating their relevant qualifications, skills and experience, their understanding of the TOR, detailed budget as well as three recommendation letters by the current and previous institutions where similar services are/were provided.

The deadline for submission of expression of interest (technical and financial budget proposal) is 26th February 2023, 5 pm Nairobi time.

Interested and eligible bidders are requested to send soft copies of the application and company profile documents listed above to the following email address:

[email protected]

To conduct a Final Project Evaluation for a Covid-19 recovery and resilience project.

Note: This consultancy is open to both national and international applicants. Individuals, groups of individuals as well as consultancy firms having the requisite skills/experience are eligible to apply. Persons with disabilities, women and the youth and other vulnerable populations are highly encouraged to apply