Review of URCS De-occupied Territories Response

1. SUMMARY

Purpose: To conduct a review of the response of the URCS with support from Movement partners in de-occupied territories and to identify the recommendations to inform humanitarian operations in these areas. This will be done by engaging an experienced consultant(s) alongside a Review Board of three Movement Partners with extensive experience in protracted displacement crises to review the outputs. The purpose of the de-occupied territories response assessment is to document the response relevance, effectiveness, complementarity and coordination – and to identify lessons for improving sectoral planning that ensures effective and continued assistance to the affected populations and their evolving needs. The review will assess the outcomes of the support in de occupied territories in terms of quality, relevance, accomplishments and timeliness, and overall accountability with reference to the response structure, system, procedure, and performance by the URCS with support from its Movement and external partners. The key recommendations of the review in terms of programmatic coverage and operational strategies, including for coordination and collaboration, which will guide the necessary adjustments ongoing response. This could include sector-considerations for relief, shelter, cash and voucher assistance, psycho-social support (PSS) as well as medium- and longer-term interventions in the de-occupied territories.

Audience: URCS, PNS, IFRC ICRC, existing and future partners

Commissioner: Illya Kletskovskyy, Deputy Director General (Programmes), NC Ukrainian Red Cross Society

Reporting to: Evaluation Management Team

Timeframe: July – September 2023

Location: Retaken territories in Kharkiv and Kherson

2. BACKGROUND

The International Armed Conflict (IAC) on the territory of Ukraine has caused tragic loss of human life, destruction of cities and civilian infrastructure, and unprecedented displacement of people inside the country and across borders. Active since 2014, the conflict has escalated in late February 2022, as hostilities spread to most parts of the country. It has led to critical concern for the protection of civilians, especially for those trapped on the Ukraine territories controlled by the Russian Armed Forces (RAF).

According to the data of the State Statistics Service of Ukraine as of 1 February 2022, the population of Ukraine was 41,130,432 people, excluding Crimea and Sevastopol. It is estimated that the conflict has affected 24 million people in Ukraine, while the number of people in need of humanitarian assistance has increased from 2.9 million before 24 February 2022 to 17.6 million in January 2023[1]. This situation has required a massive scale-up of provision of humanitarian aid across the country.

In spite of a challenging operational environment, with severely restricted humanitarian access, international organizations in Ukraine have significantly scaled up their operations, reaching over 15.8 million people through the end of 2023, including 1 million in areas under the temporary military control of the RAF. An estimated 8.3 million people, including IDPs, non-displaced conflict-affected people and returnees remain in need of Shelter and Non-Food Items support in 2023. The majority of people in need are located in the central, southern and eastern oblasts, including in areas near active hostilities and in areas under the temporary military control of the Russian Federation.

Some of the areas were retaken by Ukraine in 2022. The first of the largest liberated settlements was Balakliya in Kharkiv oblast, followed by Izyum, Shevchenkove, Kupyansk, Vovchansk, and Borova. Having pushed back Russia in the Donetsk and Luhansk regions, the Ukrainian army concentrated on combat operations in these areas and pushed toward Kherson. The process of clearing and gradually reconstructing the liberated territories then began. Ukraine, which already had experience of being occupied, but not yet of recovering long-occupied territory, faces several problems[2].

[1] Ukraine Humanitarian Needs Overview 2023 (December 2022) [EN/UK] – Ukraine | ReliefWeb

[2] What Life is Like in the De-Occupied Territories of Ukraine – UkraineWorld

3. PURPOSE AND SCOPE

3.1 Purpose

The purpose of the evaluation of the Red Cross’ humanitarian response operations in de-occupied territories in Kharkiv and Kherson is to ensure that the URCS and partners stay accountable to the communities we serve and that actionable learnings are collected on the response which help inform and improve ongoing and future humanitarian response operations and activities.

3.2 Scope

The geographical scope of the evaluation are the areas in Kherson and Kharkiv oblasts in eastern Ukraine that were de-occupied in the fall of 2022, which is assessed to include approximately 700 settlements.

The evaluation should prioritize assessing the emergency operation in de-occupied areas including how well URCS with the support of partners have managed to continue providing relevant and complementary services and assistance to the affected communities.

The evaluation should assess all modalities and interventions delivered by URCS with support from partners as part of the humanitarian response operations in Kherson and Kharkiv.

3.3 Roles and responsibilities

The URCS will be responsible for coordinating the evaluation process and will lead on the support to the consultant for data collection through coordination with the relevant stakeholders as well as providing human resources to support with interviews where relevant and agreed.

The ICRC, IFRC and Danish Red Cross will actively engage in the evaluation management and review process through EMT and Review Board and provide input on management response plan based on the results of the study. In addition, DRC facilitates the consultant recruitment and payments.

4. OBJECTIVES AND EVALUATION QUESTIONS

4.1 Objective

The objective of the evaluation is to review the previous response of the Ukrainian Red Cros in the de-occupied territories of Kharkiv and Kherson documenting the response relevance, timeliness, quality, effectiveness and impact in regard to the coordination among the humanitarian actors (government and non-government). The evaluation should conduct a step-by-step analysis, including lessons learned, of how the response unfolded and how well the procedures and systems of the URCS and partners contributed to effective coordination.

A specific priority should be given to assessing to what extent URCS and partners were able to deliver services and assistance that added value to communities in the overall response and providing concrete and actionable recommendations for how URCS and partners can ensure that future responses are improved.

The findings and recommendations should help the URCS and its partners to make immediate necessary adjustments to existing coordination procedures and systems that allow for responses to meet the needs of the affected population more effectively.

4.2 Evaluation questions

The consultant and URCS team will create specific questions linked to the above objectives, RCRC focused De-occupied territories’ response program. Recommendation of overall questions based on the criteria above are outlined below.

1. To what extent were the humanitarian operations implemented by URCS and partners in de-occupied territories Kherson and Kharkiv able to effectively meet the most urgent humanitarian needs.
2. To what extent has the assistance been able to effectively target the population in most need of the de occupied territories? And how effective was it in ensuring dignity, accessibility, protection and safety of the affected population in need?
3. How well did the humanitarian response structures and procedures of the URCS and partners perform during the operations?
4. To what extent did the interventions and activities implemented by the URCS and partners provide added value to the overall humanitarian response operation into the de-occupied territories and how can URCS and partners ensure that assistance adds value in future operations?
5. How (well) was the response coordinated among the humanitarian actors (Movement Partners, UN, government and non-government agencies) (to support the vulnerable population in the de occupied territories?
6. In relation to the earlier response, to what extent, if any, has the perception of relevance varied between geography, instrument, target group and vulnerability?
7. How well has the coordination between RCRC Movement partners worked for the planning and implementation of the response in the territories target areas?
8. What actionable recommendations can be made to improve the program/response for the future potential humanitarian response operations into de-occupied territories?

Please note that sample questions need to be adjusted based on the situation/operation, in agreement with the evaluation management team (EMT) which will be reflected and outlined in the inception report.

5. METHODOLOGY

The methodology will adhere to the IFRC Framework for Evaluations, with particular attention to the processes upholding the standards of how evaluations should be planned, managed, conducted, and utilized.

For this evaluation, a ‘Evaluation Management Teem’ will review the methodological approach of the study including the data analysis plan and associated tools (e.g. questionnaires, checklist, etc), approve the inception report and endorse the findings.

The Review Board will be responsible for guiding the evaluation process, providing strategic inputs across the whole process, from the design phase to the delivery and comment on the final report. The Review Board will have members from URCS and Partner National Society Program Lead with the experience of Ukraine Crisis Response.

Interviewees will include URCS, PNSs, IFRC and ICRC personnel (e.g. sector leads, program managers, cross-cutting sector leads, field officers, direct implementers, volunteers and community mobilizers), beneficiaries and potentially, other non-movement donors and partners (WFP, UNICEF, GIZ, Save the Children, etc.), and people who did not receive assistance.

The specific assessment methodology will be detailed in close consultation between the EMT, Commissioner and relevant key stakeholders, but will draw upon the following primary methods:

• Desk review of operation background documents, relevant organizational background and history, including URCS and National policies and SOPs, prior RCRC reports, and any relevant sources of secondary data, such as Post Distribution Monitoring (PDM), learning review report, and other surveys and report from Movement partners and participants in the operation.
• Key informant interviews as the main data collection method
• Focus group discussions as relevant and agreed with the EMT
• Field visits/observations as relevant and agreed with the EMT

The consultant is expected to work closely with Evaluation Management Team to design the exercise, for which the following should be considered:

• Sampling method is to be decided by the evaluator and reviewed by the EMT. The final sample is expected to include both URCS staff involved in the response in the de occupied territories
• Data collection methods and pace are to be decided by the evaluator, in consultation with the Evaluation Management Team but should consider the reality of difficult-to-reach areas.

6. OUTPUTS/DELIVERABLES

The assessment consultant will deliver the following outputs:

1. Inception Report: The inception report will be a scoping exercise for the assessment and will include the proposed methodologies, data collection and reporting plans with draft data collection tools such as interview guides, the allocation of roles and responsibilities within the team, a timeframe with firm dates for deliverables, and the travel and logistical arrangements for the team.
2. Debriefings / feedback to management at all levels: The team will report its preliminary findings to the URCS within one week of the field work completion.
3. Draft report: A draft report identifying key findings based on facts and will separate from the report opinions or rumours, conclusions, recommendations, and lessons for the current and future operation, will be submitted by the assessment consultant within three days after presenting the initial findings.
4. Final report: The final report will contain a short executive summary (no more than 1,000 words) and a main body of the report (no more than 10,000 words) covering the background of the study, a description of the study methods and limitations, findings, conclusions, lessons learned and clear recommendations. Recommendations should be specific and feasible. The report should also contain appropriate appendices, including a copy of the ToR, cited resources or bibliography, a list of those interviewed and any other relevant materials. Furthermore, all the raw databases including the analysis sheet must be handed to the EMT and all the outputs will be the property of the URCS. The final report will be submitted two days after receipt of the consolidated feedback from URCS.

7. SCHEDULE

The evaluation is expected to be conducted between end-June to mid-August. A detailed schedule will be developed together with the Evaluation Management Team.

8. REVIEW BOARD

A review board will be formed to review, guide and endorse the report and ensure that it upholds the key components of the IAC Ukraine crisis. The Review board will consist of maximum three members of Movement partners who has extensive experience of international armed conflict (IAC) in the response and engaged with the evaluation.

9. EVALUATION MANAGEMENT TEAM

An evaluation management team will be appointed to manage and oversee the review and ensure that it upholds the IFRC Framework for Evaluation. The management team will consist of five members i.e., Head of Quality Control and Accountability (QCA) Department of URCS, Head of PMER Unit, Head of Program Support, Two representatives from the Movement partners. Maximum five people will be part of the evaluation management team. Head of Quality Control and Accountability (QCA) Department of URCS will be the team leader of this team.

10. EVALUATION QUALITY AND ETHICAL STANDARDS

The evaluator(s) should take all reasonable steps to ensure that the assessment is designed and conducted to respect and protect the rights and welfare of people and the communities of which they are members, and to ensure that the evaluation is technically accurate, reliable, and legitimate, conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the evaluator(s) should adhere to the evaluation standards and specific, applicable process outlined in the IFRC Framework for Evaluation. The IFRC Evaluation standards are:

The IFRC Evaluation Standards are:

1. Utility: Evaluations must be useful and used.
2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost-effective manner.
3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with regard for the welfare of
4. those involved in and affected by the evaluation.
5. Impartiality & Independence: Evaluations should be impartial, providing a comprehensive and unbiased
6. assessment that considers the views of all stakeholders.
7. Transparency: Evaluation activities should reflect an attitude of openness and transparency.
8. Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.
9. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.
10. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.

It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality.

11. EVALUATOR AND QUALIFICATIONS

Evaluator(s), with knowledge of local context who will provide an independent, objective perspective as well as technical experience on evaluations, will be hired to conduct the assessment. The evaluator will be the primary author of the evaluation report. S/he will not have been involved or have a vested interest in the URCS operation or context being evaluated, and will be hired through a transparent recruitment process, based on professional experience, competence and ethics and integrity for this evaluation. The evaluator will report on progress or challenges to the evaluation management team. The evaluator should have the following characteristics:

• Demonstrable experience in leading evaluations of humanitarian programmes responding to refugee and displacement crises and preferably previous experience of conducting evaluation in Ukraine or impacted countries.
• Knowledge of activities generally conducted by humanitarian organizations in the IAC response.
• Field experience in the evaluation of humanitarian or development programmes, with prior experience of evaluating RCRC programmes desirable.
• Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner (examples of previous work).
• Previous experience in coordination, design, implementation, and monitoring and evaluation of humanitarian programmes.
• Experience in qualitative data collection and data analysis techniques, especially in emergency and recovery operations
• Knowledge and experience working with the Red Cross Red Crescent Movement preferred
• Demonstrated capacity to work both independently and as part of a multi-discipline, multi-national team
• Ability to work within tight deadlines and manage with available resources.
• Fluent in spoken and written English.
• Strong interpersonal skills.
• Relevant degrees or equivalent experience.
• Availability for the time period indicated.