Ukraine Response Humanitarian Appeal – Programme Evaluation

Commencement date: June 2024

Duration: 2-3 months over the period June-September 2024

Accountable to: Lizz Harrison, Emergency Programme Manager – Ukraine Response

The deadline for Expression of Interest – 16th June 2024

Background & Context

Following Russia’s full-scale invasion of Ukraine in early 2022, over 6 million sought refuge in neighbouring European countries and beyond, and another 3.6 million were displaced within Ukraine. ukOver two years on and millions of people remain displaced and many more continue to need humanitarian assistance. In response to the full-scale invasion, CAFOD launched a new programme which has supported 157,000 people to-date with multi-sectoral humanitarian aid through partner-led projects worth £19 million. CAFOD is working with Ukrainian partners Depaul Ukraine, Caritas Ukraine and Caritas Spes-Ukraine in the country.

CAFOD launched an Ukraine Emergency Appeal in March 2022 which has so far raised more than £4.5 million. CAFOD is a member of the Disasters Emergency Committee (DEC) which has raised more than £430 million to date through the Ukraine Humanitarian Appeal, of which more than £16.5 million has been allocated for CAFOD’s response. CAFOD’s response has followed a partnership approach with no direct implementation. Humanitarian assistance has been provided to people in need across thematic areas of food, shelter, hygiene, health including psychosocial support, cash, and protection. Support has been extended to people in need in Ukraine and refugees in Romania, Slovakia, Poland, and Moldova. The following key partners have been involved:

Scope of work:

Aim: evaluation is to build on existing evaluations, learning and analysis to highlight learning on CAFOD’s Ukraine Response and provide practical recommendations which can be used to inform the review of the Ukraine Programme Strategy and future humanitarian response activities in Ukraine and beyond. The evaluation will not evaluate the humanitarian response at community level or with project participants (‘beneficiaries’) as CAFOD does not do any direct implementation and external evaluations focused on the two DEC-funded projects with partners Depaul and JRS Romania have already been conducted in-country recently. The team are seeking actionable and practical recommendations including on priorities for 2025-2027 with significantly reduced funding projected.

Evaluation period: from the start of the Ukraine Programme to date: March 2022 – June 2024.

Projects: a full list of projects will be shared with shortlisted candidates and includes a range of budget, reach and locations with the partners listed below. If you feel this project list is important when developing your evaluation proposal then please do contact us in good time (before 14th June) and request the full list.

Partners:

• Depaul International (DPI)
• Depaul Ukraine (DPU)
• Jesuit Refugee Service (JRS) Romania
• Caritas Ukraine (CUA)
• Caritas-Spes Ukraine
• Caritas Spiš, Slovakia
• Caritas Romania
• Caritas Poland
• Caritas Moldova

Deliverables:

1. Facilitation of a co-design process to identify the most effective approach to the evaluation and learning process.
2. An inception report outlining agreed-upon evaluation approach including summary of findings from desk review, planned methodology, workplan, and data collection tools.
3. Regular short meetings to update the CAFOD evaluation steering committee on the progress and emerging findings of the evaluation.
4. Facilitation of a workshop to present the preliminary findings and draft recommendations for discussion and input with the CAFOD evaluation steering committee, partners and other relevant staff. The recommendations should be reviewed together before being finalised, to ensure there is buy-in and that they are actionable and feasible.
5. An evaluation report of no more than 30 pages including findings, key learning, and practical recommendations. The report should focus on presenting the outcomes of the evaluation in a practical and accessible format to encourage adoption of findings and recommendations by stakeholders.
6. A 1-page poster summarising the evaluation purpose, methodology, key findings and recommendations. This will be translated into Ukrainian and shared with evaluation participants.

Contract Timescale:

It is anticipated that the evaluation will be conducted over a 2-3 month period over June – September 2024 (dependent on start date). The exact timing is to be agreed in consultation with the consultant, CAFOD and partners, but below is a guide:

• June: desk review, methodology development.
• July: inception report, start data collection.
• August: complete data collection, present preliminary findings and draft recommendations, final report and summary.
• September: as needed.

Evaluation approach

The aim of the evaluation is to build on existing reviews, analysis and learning from the response, with a particular focus on partnerships, capacity strengthening, localisation, resource management and other areas as outlined in the Critical Learning Questions outlined below. The aim is to learn and understand what has worked well and what can be improved in future. The following key steps are expected:

• Desk review of key project documents, including Real Time Reflection of CAFOD Ukraine Response (summer 2022), external evaluation reports for partners JRS Romania and Depaul Spring 2024), CAFOD Ukraine programme MEAL framework, reports to DEC, partnership review documents and data (Summer 2024), documentation of capacity strengthening support to partners, CAFOD Critical Learning Questions, and other relevant documentation.
• Facilitate the co-design of the evaluation approach, discussing methodology and the best way to answer CAFOD’s Critical Learning Questions, along with a suggested workplan and timeline. An inception report will then present the agreed upon approach. A CAFOD evaluation committee will be established to help guide the learning process. Consultation and discussion with CAFOD is expected at this stage to determine the most effective approach to answering the Critical Learning Questions.
• Conduct interviews, focus group discussions, workshops and other relevant data collection approaches to gather perspectives from key programme staff and volunteers (CAFOD and partners), and other relevant stakeholders. No data collection is expected with project participants (‘beneficiaries’) as they have actively participated in other recent evaluations.
• No site visits are expected due to the wide availability of data from programme monitoring and external and internal evaluations, as well as good internet connection suitable for online/remote data collection.

Critical Learning Questions

The Critical Learning Questions that have been developed by the CAFOD Ukraine Programme which would ideally be answered as part of this evaluation are as follows. These questions and the information already available to answer them can be discussed in more detail in the inception phase of the process as outlined above so that the most effective approach is used.

1. Is CAFOD effectively supporting their partners’ capacity to deliver effective, safe and dignified programming?
2. Is the support CAFOD are providing to their partners translating into better outcomes at the community level?
3. Is CAFOD providing good practice examples of quality partnership and funding within their own programme?
4. Is CAFOD providing enough financial and technical support for partners to meaningfully engage in advocacy initiatives?
5. Is CAFOD using their platform with donors, Caritas International Member Organisations (CIMOs), and other agencies to advocate for localisation?
6. Is the support that CAFOD is providing to partners, and the engagement they have in the wider policy space in Ukraine, leading to:

– Local leadership in the response?

– Equitable partnerships?

– Greater access to quality funding?

Consultant Experience:

Applicants must meet the following technical qualifications to be considered:

• Master’s degree in international development, social sciences, public administration, or a related field
• Substantial professional management or research experience in the evaluation of international development or humanitarian response programmes.
• Proven experience of leading evaluations in humanitarian relief and response contexts. Ideally be familiar with the context in Ukraine, have experience evaluating DEC-funded projects, and/or an understanding of partnership-based responses.
• A proven track record in conducting different types of evaluation and learning reviews, and knowledge of various qualitative evaluation methods and approaches.
• Strong analytical and facilitation skills.
• Excellent English report writing skills.
• Spoken Ukrainian and/or Russian is a strong advantage.
• A good understanding of the Core Humanitarian Standard (CHS) and Sphere Standards is an advantage.

Reporting:

The key stakeholders involved are:

• CAFOD Ukraine Programme staff
• CAFOD Programme Quality & Capacity Strengthening
• Partner staff and volunteers

Contact Information:

For any questions or inquiries, please contact: Lizz Harrison, Emergency Programme Manager – Ukraine Response at [email protected].

Expression of Interest:

Expressions of Interest (EoI) for the consultancy should be submitted by email to Lizz Harrison at [email protected], using subject line: EoI for Ukraine Programme Evaluation consultancy. To be considered, EoIs must include:

1. An evaluation proposal of no more than 5 pages including a summary of how you meet the technical qualifications, a proposed methodology and timeframe.
2. A CV with two referees listed that can be contacted if shortlisted.
3. A budget including the following:

o Daily fees for the evaluator(s)

o Direct costs relating to selected methodology if relevant

o Interpreter allowances (if required)

o Communication costs (as required)

o Any other foreseen costs.

The deadline for EoIs is 16th June 2024.

Applications received after the deadline cannot be considered. Shortlisted candidates will be invited to interview. Examples of past work may be requested.

Ethical and safeguarding considerations

Ethical considerations should be an important part of this evaluation. Safeguarding of evaluation participants should also be considered. Basic principles such as respect for project participants, explaining the purpose of the evaluation, and seeking informed consent from participants are essential. Evaluation participants should be made aware through clear communication of how confidential information is treated and how privacy will be protected.

Consultants will be expected to read, understand and sign the following CAFOD policies / forms:

1. Code of Behaviour
2. Safeguarding Policy
3. Confidentiality and Data Protection Agreement
4. Confidential disclosure of convictions