Evaluation Consultant – WASH ERU Deployment – Ukraine

Background

Kakhovka dam breach on 6 June 2023 in Ukraine caused widespread devastation and human suffering. The overall impact on water supplies, sanitation and sewage systems, and health services were massive. The severe flooding downstream has displaced thousands of people and destroyed vital infrastructure including roads, electricity lines, agricultural land, health facilities and private homes. The environmental damage alone took a great toll, with potentially hazardous agricultural chemicals seeping into the water supply. Flooding in a highly industrialized zone posed the risk of additional chemical releases into water, which could severely impact people and animals for years to come.

Following the devastation, the URCS National Committee (NC) immediately activated a six-week emergency response phase to assist the affected population with evacuation, relief assistance, mine risk awareness, emergency health services including MHPSS and provision of drinking water. The overall humanitarian needs were substantial as the GoU started to strategize its emergency response and recovery plan with the support of partners in country.

As an early action and to scope out recovery plan URCS with the support of IFRC, ICRC and partners deployed a rapid assessment mission through the IFRC surge mechanism. And as URCS’s overall WASH capacity to respond to a scaled emergency situation was limited, the outcome of the rapid assessment in consultation with the movement partners suggested taking opportunity for a “No-regret” approach to deploy a tailored M15 WASH ERU, mainly focusing on water trucking, distribution and capacity building through ERU level training roll-out.

Accordingly, a tailored WASH ERU was planned and deployed for the affected regions, the context demanded consistent focus- with limited rotations to support the needs in the ground with minimum equipment for field deployment while strategizing midterm recovery support. The overall objective of the mission was to improve access to clean water in identified water deprived locations in the upstream locations affected by the crisis. The areas upstream of the destroyed dam heavily depended on almost depleted reservoir (water level dropped by more than 12m) with water intakes running dry. At this point, the WASH cluster had a draft strategy the entailed water trucking & setting up of storage/ distribution points for the next months to come; GoU has a plan to build a new pipeline, bringing water from the northwestern area to the east, which will mainly serve larger cities, but construction will likely take longer than the envisaged month of November 2023. Representatives of the Dnipro Regional Administration stated that the oblast is heavily industrialized, including agriculture and heavily contaminated due to decades of abuse, which is why surface water are not recommended to be used for being treated for drinking water with conventional filtration.

For the situation downstream, the State Emergency Situation of Ukraine (SESU) and Vodakanal Kherson stated that the water situation downstream in Kherson city was relatively stable. The dam collapse flooded downstream areas, flushing out contaminated sediments from the bottom of the reservoir to surface water sources, boreholes, hitting especially the lower-lying areas on the riverbanks. As the flooding in the areas downstream receded, effort was made by the humanitarian actors and authorities to facilitate return of displaced population and restoration of community infrastructures.

A review of the tailored ERU M15 deployment in Ukraine- post Kakhovka dam breach. The review is intended to assess the overall quality of deployment, how the partnership and organization of the complete deployment cycle worked, and the roles of partners engaged in its implementation. The review shall closely examine the detailed development identify practical ways to improve such deployment in a context like Ukraine for future emergency response. The review may also generate evidence around strategic learning area of disaster management, value for money and sustainable impact. It is expected that that the review draws two levels if recommendation.

1. Specific for Ukraine (and / or context like Ukraine), contributing to the evolving URCS WASH strategy.
2. Specific to global ERU deployment network (as may be relevant)

The review shall also keep focus on the integration efforts and impact from PGI/CEA perspective in emergency response and specific focus on context like Ukraine, and to identify practical ways to improve mainstream in the various sectoral integration and related capacity-building strings.

Scope of the review exercise

To review the overall quality of tailored M15 ERU deployment in Ukraine; capture learning and identify recommendations in a practical way, enabling IFRC and Partners to improve future deployment in a similar context and share this learning with concerned stakeholders.

For the IFRC country operation the review will focus on to draw key learnings for future deployment and recommendations to support URCS’s plan for strengthening its own WASH in emergency capacity. Improve future planning and actions in the challenging context including CEA mainstreaming/ trainings and share this learning with wider movement. This exercise will also be an opportunity to reflect on movement collaboration and working in coordination towards common objective of an effective emergency response. For URCS, to take opportunity and capture learnings from the experience to make recommendation for future WASH emergency response. It is desired that the review takes into account the evolving WASH strategy for URCS and clearly identifies certain complementary for such deployment and areas of mid to long term capacity building.

Users and uses of the review

The primary audience of this review are the parties involved in the deployment URCS, IFRC, ICRC and deploying partner national societies and other interested partners in the WASH sector in country, including the global ERU deployment network.

Final report will be disseminated internally within the above stakeholder group and the in-country delegation; A debrief on the main evaluation findings will be offered by the evaluation team. Discussions on how to appropriately disseminate findings shall take place during the process of concluding the evaluation.

URCS’s Emergency response unit plans to strengthen its own WiE (ERUs) for future deployment.

The review shall aim to focus on the following key questions:

1. What went well in terms of planning, coordination, collaboration, timeliness, including the value for money and impacts in action?
2. What did not go so well around the above subjects?
3. What are the Key recommendations for future in terms of appropriateness, efficiency, effectiveness, Programme quality and sustainability?

Appropriateness:

• Is the ERU modality and rotation fit for purpose for this type of context and response? How was the decision on areas of focus made?
• Did the deployment planning perform a comprehensive needs assessment to inform the deployment and scope in planning?
• Did the deployment identify vulnerable target group for assistance? did the planning considered the needs of vulnerable groups and communities?
• What measures did the ERU put in place to ensure the intervention meet the needs of the community in terms of overall objective to contribute to improving access to clean water in identified water deprived communities.
• How well the CEA/PGI minimum standards been applied in the Planning/implementing the interventions?

Efficiency: (including Value for money)

• How was it decided to deploy the Specific ERU? Was it a default or was it based on rational for deployment? Is the clarification and justification around decision making clear?
• Was the deployment teams and personnel selected based on required competencies to deliver.
• Were the rotations planned in a need based and consultative manner, how was the carrying forward of deployment between rotations efficiently managed?
• Was the deployment managed timely, did the deployment manage to position personnels and materials on time?
• Did the intervention achieve Value for Money? How did this compare with other actors? How did the decision to deploy the specific ERU with equipment or without equipment affect the outcome?

Effectiveness

• How are such ERUs integrated in IFRC operations? (For example: Are they able to perform independently? Organizing procurement? Organizing cash flow?), what are the limitations and how the same could be effective?
• What would be the optimal division of responsibilities between IFRC and deploying PNS? How did this affect the effectiveness of the work of the team? What could be done in future to resolve any bottlenecks?
• Did the ERU record clear impact and effectiveness of the deployment?
• How timely was the RC response compared to the other responders in relation to the needs?
• What measures did the project put in place to ensure good volunteers management, training, and quality of support to volunteers’ wellbeing?
• How effective was the collaboration with the other units of URCS? Is there any good practice and recommendations for future deployments?

Programme Quality

• How well did the project ensure Community Engagement and Accountability was achieved? (Refer to CEA guidelines)
• Was there a mechanism for community feedback and complaints and how effective was this? Where complaints received, acted upon and feedback given to the complainant?
• How did the emergency response address protection issues especially for vulnerable groups.
• Did the trainings made positive changes on the knowledge, attitude, action of team involved in response towards PGI?

Sustainability

• To what extent was environmental sustainability/ Green response considered in the design and implementation of the ERU?
• Was the environmental impact of program decisions considered (e.g. when selecting procurement options, relief supplies)? If so, what changes were made to programs to improve environmental outcomes? If not, why not? Are there any key concerns about environmental sustainability of the ERU actions?
• What information/resources concerning environmental sustainability would be useful for future ERU deployments?
• Were the team in the ground capacitated to a level to further continue with the efforts efficiently with clear plan of actions.
• Was the handover and exit strategy from ERU and transition to recovery planning in place and discussed with the URCS?

Experience

• Considerable technical knowledge and experience in WASH in Emergency context
• Good Experience of conducting evaluations, reviews and/or learning initiatives, including methodology design of remote review, data collection and analysis related to WASH in Emergency response.
• Good Knowledge on ERU deployment structure
• Experience in capacity building of local organizations and facilitating participatory learning processes.
• Demonstrable skills in producing high quality, accessible reports/outputs Fluency in written and spoken English, the team shall have at least one member with Ukrainian language skills
• Strong coordination and facilitation skills, including proven ability to design and facilitate workshops.
• Knowledge of the Red Cross Red Crescent Movement

Review deliverables

Debrief meeting to present and validate initial findings.

A validation workshop/ online meeting to present and validate initial findings with key stakeholders.

The evaluation report should:

• Include a brief project background,
• Be clear and simply written.
• include an Executive Summary
• include overview of the methods and analysis,
• lessons learned,
• specific recommendations which is usable as a free-standing document
• conclusions
• ensure analysis is always backed up with relevant data, with reference to the data source.
• ensure recommendations made are specific and include relevant details for how they might be implemented.
• contain at least the following annexes:
  • Terms of Reference,
  • list of interviews and visit,
  • List of documents reviewed, meetings attended, persons interviewed/involved in Focus Group Discussions, and
  • Data collection tools.
  • A presentation for dissemination of the findings and recommendations with slides and other resources used.

Timeframe

It is expected to be 4 weeks long evaluation, mostly with online format.

Evaluation team specification

• It is desired that the Evaluation team should have a Ukrainian speaker to be able to connect to the local team and the stakeholders involved. Between the proposed team members, the following criteria should be met:
• As organizing field visits to the locations where ERU deployment took place is challenging and time consuming, it is expected that the reviewer with innovative means work remotely to draw field information and learning.