PROJECT FINAL EVALUATION

1.2 About Humanity & Inclusion in the country/region

HI has been operating in Kenya since 1992, implementing projects in development and humanitarian contexts. In Dadaab and Kakuma from 2007 and 2014 respectively, implemented community-based rehabilitation, Inclusive education, and inclusive livelihoods projects.

In refugee settings and their surrounding host environs, persons with disabilities face challenges, due to numerous barriers that limit their active participation in decision-making processes and are less likely to have their humanitarian protection needs met. Most affected are children with disabilities, as well as children whose caregivers live with a disability. Further, situations of displacement and emergency have a strong psychological impact on boys and girls with and without disabilities as they are uniquely vulnerable due to their age and social status.

Children, particularly those with disabilities, are disproportionately affected by violence, facing a plethora of challenges: marginalization, lack of access to adapted services, and even direct targeting for violence. Globally, it is estimated that some 93 million children – or one in 20 of those under 15 years of age – live with a moderate or severe disability, with 13 million (0.7%) children experiencing severe difficulties. Children with disabilities as well as children whose parents and/or caregivers have a disability are not only among the most isolated, socially excluded, and marginalized of all displaced populations, but also among those most vulnerable to violence and abuse. Studies show that children with disabilities are almost four times more likely to experience violence than non-disabled children, 3.7 times more likely than non-disabled children to be victims of any sort of violence, 3.6 times more likely to be victims of physical violence, and 2.9 times more likely to be victims of sexual violence.

The population of urban refugees faces many risks as a result of multiple and complex unmet needs that cut across social, medical, and economic dimensions. They also face intersecting oppressions based on race, ethnicity, nationality, language, class, gender, sexual orientation, and disability.

2) Context of the evaluation

2.1 Presentation of the project to be evaluated

Project title

Enhancing the protection of vulnerable refugee and host communities in Kakuma, Dadaab, and Nairobi

Implementation dates

September 2020 to August 2022

Location/Areas of intervention

Kakuma, Dadaab and Nairobi

Operating Partners

UNHCR, Government of Kenya

Target Groups

Children with and without disabilities, family and community structures, child protection actors, urban refugees with disabilities

Project Budget

USD 2,000,000

Objectives of the project

Objective 1: Protection, care, and development of children with disabilities are improved through access to quality rehabilitation, psychosocial support (PSS), and inclusive services in Kakuma and Dadaab.

Objective 2: Child protection community-based mechanisms, structures, and resilience are strengthened through community-led messaging on key protection risks and related mitigation measures in Kakuma and Dadaab.

Objective 3: Capacities of humanitarian and state actors to identify and address protection risks to enhance the inclusion of children with disabilities and their families in their programming, coordination, and linkages are strengthened in Kakuma and Dadaab.

Objective 4: Functional capacities for persons with disabilities are improved through enhanced access to rehabilitation and specialized services in urban refugee settings of Nairobi.

Expected results and indicators

Log frame to be shared in the technical offer

Main activities implemented

Proposal to be shared with the technical offer

2.2 Justification of the evaluation

HI is seeking the services of a consultant to conduct final project evaluation for the project in the refugee hosting locations in Kakuma, Dadaab, and Nairobi in line with HI’s planning, monitoring, and evaluation policy and the provision of the project design (proposal).

The purpose of the final project evaluation is to generate and provide mixed method quantitative and qualitative endline values for impact, and outcome of project as a whole whilst monitoring indicator per the overall logical framework of each project location for the protection project. The endline situation will enable the project to establish the standpoint for the different indicators (situation after) which will be used to compare with the situation before implementation of the project (baseline survey and during its implementation through protection risk assessment).

This will enable the project to assess qualitatively and quantitatively the contribution of the project to enhancing the protection of vulnerable refugee and host communities and thus ascertain the aspects that will be pivotal in attributing change as a result of the project.*

3) Objectives of the evaluation

3.1 Overall objectives and expectations of the evaluation

The main objective of the final project evaluation is to collect and analyze information that will establish the recommendations/impact/outcome and endline values of the project;-

1. Provide an overall learning on the project through recommendations on specific areas of focus that the project should consider based on the findings of the final evaluation and;

2. Assess through a research methodology endline values for each of the project indicators, comparable to baseline values and protection risk assessment report, in intervention areas.

3.2 Specific objectives

o Highlight the extent of protection, care, and development provided to children with disabilities through the various services in Dadaab and Kakuma.

o Highlight the any existing protection concerns and include information on response made by the project related to concerns identified especially compared to baseline information disaggregated by age, gender and disabilities in Dadaab Kakuma, and Nairobi.

o Examine the resilience, inclusivity, and needs (met and unmet) of community-based mechanisms and structures in Dadaab Kakuma and Nairobi disaggregated by age, gender and disability per site

o Highlight the capacities of humanitarian and state actors to identify and address the protection needs of children/persons with difficulty: hearing, seeing, moving, intellectual (remembering, self-care), communicating, psychosocial… and gender inclusively.

3.3 Evaluation criteria and evaluative questions

*The specific objective of the final evaluation is to provide an analysis of the quality of the project-based HI Quality Framework* (annexed)

The guiding questions listed below are the basis for the evaluator. The applicant for this assignment should submit further (sub-questions as part of the application documents and the selected evaluator will, at the inception phase of the consultancy, discuss a more comprehensive TOR with HI which also covers the evaluation objectives, analytical framework, methodologies, timeline and deliverables for HI’s approval.

*Criteria

*Evaluative Questions

Changes (Effects, impact, mitigation)

· Did the project produce significant positive changes in the lives of the beneficiaries in a sustainable manner?

· Has the project sufficiently addressed the protection risk of the beneficiaries?

Capabilities (Autonomy, skills, learning) **

· How has the project increased the capacity of community workers/caregivers so that they are more autonomous?

· Have the necessary skills (staff) been developed to carry out the project?

Sustainability (Anticipation, continuity, resilience) **

· Are the actions implemented during the project really sustainable by community structures?

· Has the project significantly reduced the vulnerability of all beneficiaries?

Accountability (Information, compliance, availability) **

· Did the project provide accountability to all stakeholders in an impartial, equitable and accessible manner?

· Did the project fully comply with international humanitarian law and the national laws in its intervention?

Participation (Consultation, expression, understanding) **

· To what extent has the project put in place mechanisms to involve beneficiaries and partners?

· Were the mechanisms of expression inclusive regardless of gender, age or disability?

4) Evaluation methodology and organization of the mission

4.1 Collection methodology

The consultant will use a range of methodologies that shall be participatory, inclusive, gender sensitive, child-friendly and disability friendly to the key stakeholders. Wherever possible, the data obtained should be triangulated to reach conclusions based on a thorough analysis. The data set needs to be comparable with baseline data. Methodologies need to be adapted to ensure proper calculation of each indicator.

The Final Project Evaluation will amongst others consider:

· Secondary data sources – Desk review of the project documents / external reports related to protection and disabilities – Proposal document including the Log Frame, M&E plans, project quality frameworks, baseline, etc.

· *Primary data sources (Key Informant Interview, FGDs and survey) and data analysis process disaggregated by the type of disability, gender, and age parameters.***

· Data collection methods: Quantitative methods including well-developed survey tools (questionnaires) to be administered to sampled beneficiaries; Qualitative methods through a thorough analysis of information gathered from observations, key informant interviews, focus group discussions, etc.

· Sampling methods – that reflects a diversity of respondents and experiences, including different types of disability, country of origin of refugees. Data collection should be conducted in the language chosen by the respondents.

· Tools will be developed in English and translated into Kiswahili and other language as needed (Juba Arabic, Somali, French). Sign language interpretations may be used when relevant. The interviewers will have to be trained in communication with people with disabilities. Beneficiary friendly approaches and tools as per the age, gender, and type of disability policy provisions.

· *Comparative approach: the methodology should contain a comparative analysis between the starting period and the end, but also between boys and girls & children with disabilities and children without disabilities, various types of disabilities*

All data collected should be disaggregated and analyzed by site, sex, age, and type of disability (following Washington Group short set of questions).

4.2 Actors involved in the evaluation

Consultant(s) role

· Detailed inception report: Containing detailed work plan specifying activities and timelines agreed with HI; a detailed description of the final evaluation/endline methodology, ethical considerations, and data collections tools based on HI policies (annexed)

· Develop and provide guidelines in data collection, data cleaning process, and data entry for data enumerators for review by HI team

· Training of data enumerators: Using mobile data collection tool developed by the consultant in collaboration with HI, Ethics and security related to collection of sensitive data, communication with persons with disabilities, GBV/CP issues.

· The consultant should coordinate the collection of field data, the day-to-day management of evaluation activities, data entry and analysis, and report writing while respecting the confidentiality and sensitivity of the evaluation.

· Data analysis, presentation of key research findings and draft report presentation for review and validation: quantitative and qualitative data analyzed and triangulated.

· Final Report Developed and Presented: finalization of the report (English language) in close collaboration with HI, with relevant annexes in the same language as the report. Transcript in English of the qualitative data should be shared to HI.

· PowerPoint presentation of final report

· The consultant will ensure that the endline study activities are carried out per the terms and conditions of his or her contract.

Humanity & Inclusion team role

· HI staff will compile the reference documents, which include proposal, midterm assessment reports, and protection risk analysis report, log frame, and budget to be consulted;

· Facilitate access to information on travel and accommodation

· Digitize the data collection tools on Survey CTO – Mobile Data Collection platform

· Provision of data collection gadgets (Android Smartphones)

· Facilitate access to information on workshop facilities, data enumerators previously engaged with HI if needed.

· A briefing session will be held with the consultant and the different HI technical and operations personnel (Field, regional and HQ Protection / Inclusion), MEAL at the beginning to agree on the overall methodology and re-emphasize key issues to be addressed.

· Provision of guidance on HI’s evaluation policies/ quality framework to the successful consultant

· Support consultant through mobilization of the respondents (beneficiaries, other stakeholders, and Government agencies;

· Pay consultancy fees for the completed work as per the agreed timelines

Beneficiaries role

· Respondents/validators

Partners role

· Respondents/validators

4.3 Organization of the mission

1. ARCI is to ensure compliance with the evaluation process.

Authority

Roles & Responsibilities:- Enforces the process and ensures quality of each stage of the process.

Position:- Country Manager

Responsible

Roles & Responsibilities: – Implements the different steps of the process.

Position:- Project Managers (Kakuma and Dadaab)

Contributors /Informed

Roles & Responsibilities: -Participate in the different stages of the process according to their expertise, on general or specific aspects

Position:- Field Specialist, MEAL team, Regional Rehabilitation Specialist, Global Specialists (Protection and Inclusion), Operations Officer HQ, Technical Unit Manager, Logistics, Human Resources, Operations Managers, Project Staff

2. Steering Committee in charge of the quality of evaluation and composed of 5 people.

It has two main functions:-

1. It is the body that steers, guides and validates evaluation decisions.

2. It is the body that gives an assessment of the quality of the evaluator’s work

Composition

1. Project Managers (Kakuma, Dadaab & Nairobi)

2. Country Manager

3. Regional Rehabilitation Specialist

4. Logistics Manager

5. Field Specialist – Child Protection

Overall, Roles and Responsibilities of the Steering Committee

Key steps

Minimum responsibilities

Drafting of ToRs

The Steering Committee shall guide and validate the ToRs drafted by the person in charge of the evaluation

Selecting the evaluator

The Steering Committee shall validate the choice of evaluator and ensure the impartiality of the selection.

Scoping Meeting and Inception Report

The Steering Committee shall participate in the scoping meeting (methodology, expected results…) and validate the inception report for the future steps.

Presentation: evaluation findings and recommendations

The Steering Committee and the evaluator ensure that they have a common understanding of the conclusions & recommendations expressed.

The Steering Committee provides elements that allow the evaluator to refine his recommendations *.

End-of-evaluation Questionnaire

The Steering Committee shall participate in filling in the end-of-evaluation questionnaire.

Quality of the final report

The Steering Committee shall specify the quality aspects** expected from the evaluator for the final report, and provide feedback on the draft report

Midterm & final report

The Steering Committee provides feedback on the draft report and then validates the quality** of the evaluator’s final report.

Elaboration of an action plan and follow-up of the recommendations.

The Steering Committee participates and validates, with the teams, the action plan and follow-up of the recommendations.

5) Principles and values **

5.1. Protection and Anti-Corruption Policy

Code of Conduct**

Protection of beneficiaries from sexual exploitation, abuse and harassment**

Child Protection Policy**

Anti-fraud and anti-corruption policy**

5.2. Ethical measures*

As part of each evaluation, HI is committed to upholding all humanitarian standards and ethical connsiderations. It is imperative that these measures are taken into account in the technical offer:

o Guarantee the safety of participants, partners and teams: the technical offer must specify the risk mitigation measures. Examples include:

o The collection of information should be carried out in a way that presents the least possible risk for the investigators and the interviewees (e.g. safe space, anonymous collection, etc.), be methodologically sound and be based on current experience and good practices**

o Basic care and support for survivors/victims should be available locally and known by interviewers before beginning any activities that may involve disclosure of information about their experiences of violence (e.g HI and other partners toll free number to be provided)**

o All members of the data collection team should be carefully selected and receive relevant specialized training (basic concepts on GBV/CP, PFA and safe referral, communication with persons with disabilities, interviewing children and/or survivor) as well as ongoing support.**

o Additional safeguards will need to be put in place if children (i.e., those under 18 years of age) are interviewed.**

o It will be strongly recommended to have a CP specialist on the assessment team and gender balanced team;**

o Assess the interviewers’ attitudes towards sexual violence, gender, disability and the survivor-centered approach during the recruitment process of data collector.**

o Ensuring a person/community-centered approach: the technical offer must propose methods adapted to the needs of the target population

o Tools to be adapted for illiterate audiences / having sign language interpreters / child-friendly, gender sensitive and disability friendly materials, etc.).

o Obtain the free and informed consent of the participants: the technical proposal must explain how the evaluator will obtain the free and informed consent and/or assent of the participants.

o Informed consent should be obtained from all individuals before participating in the data collection activity and HI consent forms signed**

o Ensure the security of personal and sensitive data throughout the activity: the technical offer must propose measures for the protection of personal data. **

o The confidentiality of those providing information about violence should be protected at all times**

*These *measures may be adopted during the completion of the inception report.* **

6) Expected deliverables and proposed schedule

6.1. Deliverables

ü An inception report refining / specifying the proposed methodology for answering the evaluation questions and an action plan. This inception report will have to be validated by the Steering Committee.

ü A presentation document presenting the first results, conclusions and recommendations, to be presented to the Steering Committee.

ü A final report of approximately 20-30 pages maximum and the annexes: (including dataset)

ü A summary of maximum 10 pages

ü Powerpoint presentation of the findings

The final report should be integrated into the following template:

see link to TOR https://bit.ly/tor22

The quality of the final report will be reviewed by the Steering Committee of the evaluation using this checklist:

6.2. End-of-Evaluation Questionnaire

An end-of-evaluation questionnaire will be given to the evaluator and must be completed by him/her, a member of the Steering Committee and the person in charge of the evaluation.

6.3. Evaluation dates and schedule

Propose a table showing the different steps (e.g., Briefing at headquarters, documentary review, inception report, field mission, field presentation, drafting of the final report) and the desired periods for carrying out these stages.

Phase

Hiring Consultant

Advertise the TOR to hire a consultant – 15 days

Recruitment of Consultant (interview, etc.) -10 days

Phase-1

Desk Review – Inception Report

Consultant develops Inception Report and data collection tools and presents to the Steering committee -4 days

Steering Committee reviews – 2 days

Phase-II

Field data Collection

Training of Enumerators/data collectors -10 days

Make appointments with respondents

Field Data collection

Phase-III

Data Analysis, Report writing

Data Cleaning, Data Analysis – 7 days

Draft preliminary report with recommendations.

Steering Committee reviews draft report – 2 days

Finalize the feedback and share the final report – 3 days

Dissemination

Disseminate findings with Stakeholders through a dissemination meeting – 1 day*** HI activity timelines**

7) Means

7.1 Expertise sought from the consultant(s) **

The required background and experience of the Consultant are as follows:

· Master’s degree in social sciences, Statistics, Monitoring, and Evaluation, or other relevant fields.

· Previous experience in the design, implementation, analysis and reporting of impact studies, endline studies, final surveys, and project evaluations, in the field of child protection or disability

· Proven knowledge and skills in quantitative analysis and excellent analytical skills

· Institutional knowledge of International Non-Governmental Organizations (preference in disability, gender, and protection).

· Rights based Understanding and familiarity of disability, and related experience in working and engaging with persons with different types of disabilities,

· Excellent command of written and spoken English. Knowledge of one local language (Swahili an asset)

· Experience in using participatory and child-friendly methods

· Strong analytical and reporting skills

· Good understanding of protection systems in sub-Saharan Africa/Kenya context (particularly in humanitarian and/or refugee settings)

The specific competencies and qualifications required of the Consultant are as follows:

· Excellent knowledge of the socio-political environment of the refugee-hosting locations in Kenya.

· Proven knowledge and experience in disability and gender issues.

· Knowledge in using Washington group questions specifically the UNICEF/Washington Module on Child Functioning.

· Practical knowledge of the use of Mobile Data Collection (MDC) in surveys.

· Proven ability to deliver quality reports/ analysis and results in line with established deadlines

· Proven experience of having participated previously in final evaluations or similar analytical work will be a plus.

7.2 Budget allocated to the evaluation

The candidate must detail in his offer: the cost per day for each evaluator; the breakdown of the time spent per evaluator and per stage of work; the ancillary costs (services and additional documents); the overall cost of the intervention including transport costs (international and local), logistics costs, and translation costs; with proposals for payment modalities, costs for reasonable accommodation. The financial proposal will indicate the all-inclusive costs of undertaking the final project evaluation.

Payment will be released in two installments:

• 50% upon signature of the contract

• 50% upon submission of the final validated report

Service cost allocation budget code: TB01/D41_063/NAIR/FD41_199/8D.01

Caution: *please note that the last payment is conditional on the validation of the final report and not on the sending of the final report. By validation, we mean validation of the quality and under no circumstances of the appreciation of the project evaluated (based on the quality checklist attached, chapter 6).*

7.3. Available resources made available to the evaluation team **

(Data, documents, software (SurveyCTO)**

8) Submission of applications

Application documents submitted will be evaluated in three (stages);

1. General Mandatory Evaluation Criteria

For firms or institutions

Consultancy to conduct a Final Project Evaluation for a Protection Project

Certificate of Registration

Trade license Certificate/ Business Permit

PIN/VAT Registration Certificate

Current Tax Compliance Certificate

Personal Accident Cover

For individuals

*Consultancy to conduct a Final Project Evaluation for a Protection Project

National ID

Relevant Certification

PIN/VAT Registration Certificate

Current Tax Compliance Certificate

Personal Accident Cover

2. Technical Capacity Evaluation

a) Profile and experience of the company:

i. The firm should have proven expertise and experience in undertaking service delivery. Attach three reference letters**

ii. Provide a list of assignments undertaken with a similar scope of work within the last 2 years along with names of clients (including contact person, email contacts, Postal address and telephone numbers – this must include the three referees provided above)

b) Professional/technical capacity/ capability

The proposed staff should have relevant experience and professional qualifications in the relevant field

c) Administrative and technical appendices

(List of contract documents, e.g. training plans, any communication media (DVDs, CD-ROMs, etc), price breakdown, methods, general purchasing conditions, etc.).

I. Methodological approach/Work schedule adopted for the service provision

II. Itinerary

III. Consultant’s CV

IV. Other

3. Financial Evaluation

At the Financial stage, the Competitive Evaluated bidder and within budget estimates will be considered for the award of the contract.

Appendices

**

· HI’s Quality Framework, on which all evaluators must base their evaluation.

· The Disability – Gender – Age Policy, which must guide the approach and the construction of evaluation tools in the technical offer.

· Safeguarding policies (CP/PSEAH/anti-fraud)**