Final Evaluation of the NABuRRS Project

TERMS OF REFERENCE

FINAL EVALUATION OF THE NABuRRS PROJECT

A Neighborhood Approach to Build Resilience and Repair Shelters (NABuRRS), A three (3) years initiative funded by USAID through the Bureau for Humanitarian Assistance (BHA)

1. BACKGROUND

During the past three (3) years, World Concern Development Organization (WCDO), in partnership with Engineering Ministries International (EMI), has implemented in the great South of Haiti (Grand-Anse, Sud, Sud-Est, Nippes) the project titled, “A Neighborhood Approach to Build Resilience and Repair Shelters (NABuRRS)”, a three (3)-year initiative funded by USAID through the /Bureau of Humanitarian Assistance (BHA). During its implementation phases, the Project has engaged diverse stakeholders including local government, civil society, partner organizations, and local community members. The overarching objective of the Project is to facilitate access to safe shelters to vulnerable populations, especially in the South, during severe weather events. More specifically, the Project has planned to achieve the following strategic objectives:

• Strategic Objective 1: Selected evacuation centers are adequately repaired and upgraded to meet minimum standards.
• Strategic Objective 2: Selected evacuation centers are upgraded to meet minimum Sphere standards for WASH.
• Strategic Objective 3: Community members, local leaders, and Government of Haiti (GoH) officials have neighborhood-level evacuation plans as a result of evacuation and preparedness training.

The underlying theory of change of the NABuRRRS’ activity

The underlying theory of change behind the NABuRRS Project identifies community resilience as the product of three factors: knowledge, agency, and assets. Community members first need to know and understand hazards, vulnerability, and local capacity. Then, they must be equipped and empowered to exercise this knowledge through training and drills, coordination, and delegation of responsibility. Finally, resources must be provided at the local level to those who have the knowledge and agency.

Therefore, WCDO and EMI along with their partners have implemented this project in three (3) phases. First, WCDO has contributed to building a knowledge baseline of hazards, vulnerabilities, and resources/assets for every commune in the four departments, ensuring that every locality is covered. This includes mapping, measuring, collating, and sharing data with all stakeholders. Second, after prioritizing the most vulnerable locations, WCDO has provided further support to 100 communities with training, simulations, empowering of local committees, and delegation of local responsibility. Finally, and simultaneously, the 100 targeted communities were equipped with evacuation centers meeting minimum standards and infrastructure improvements as needed to ensure accessibility along the evacuation routes.

1. MAIN TARGETS AND EXPECTED RESULTS
2. MAIN TARGETS

• 2,218,483 residents of four targeted departments
• 10,000 people, 2,000 households
• 100 community evacuation shelters (each repaired shelter will host 100 people)

1. EXPECTED RESULTS

• 10,000 people, 2000 households will have access to water, sanitation, and hygiene during a weather-related emergency;
• 7000 people are in settlements of project activity retaining shelter and settlement DRR knowledge two months after training
• 4,100 people participate in Disaster Risk Reduction training
• 4,000 people pass final exams and receive their certificates
• 200 shelter managers/supervisors and owners are identified and trained
• 100 evacuation drills are performed
• 100 brigades and volunteers received CERT/First Aid training
• 100 shelter supervisors/managers and owners received site-specific training
• 205 hazard risk reduction plans, strategies, policies, disaster preparedness and contingency are developed.
• 205 individuals, trained in PADR, retain skills and knowledge after two months
• 100 communities receive training, and simulations to empower them with DRR-related skills.
• 100 community evacuation centers are upgraded and equipped with minimum standards and infrastructure.
• Evacuation routes are improved to facilitate evacuation movements.
• Evacuation centers mapped and shared with communities and relevant stakeholders
• Community awareness and capacity to implement safe evacuations during severe weather events increase.
• Communities receive new or refresher training in disaster management (mitigation, preparedness, and emergency response)
• Communities receive training on CERT/First Aid
• Shelter supervisors/managers/owners receive training on shelter management to sustain readiness for severe weather events.
• Local Civil Protection Committees are formed, trained, and equipped to respond to disasters
• Search and rescue, First Aid and shelter brigades are trained and ready to be deployed
• Early warning systems are operational and community members are trained accordingly

1. OBJECTIVES OF THE FINAL EVALUATION

The overall objective of the final evaluation is to assess, review, and the Project’s performance and achievements, identify and document challenges, lessons learned, and best practices, and formulate specific recommendations to improve the design and implementation of similar future interventions. This final evaluation should inform USAID and WCDO about the design and innovative approaches to consider for future similar interventions considering past interventions, and existing strategic plans of key stakeholders involved in saving lives in Haiti. To this end, the successful applicant will conduct the following key tasks, but not limited to: (1) conducting a desk review of all project documents and reports, (2) developing robust qualitative and quantitative methods to collect data, analyze the needed information to establish the commendations/impact/outcome, and endline values for the Project. Moreover, the Consultancy’s Team will contribute to animating a lesson learned workshop with the key Project stakeholders.

The evaluation will focus mainly on the following criteria as defined by the Organization for Economic Co-operation and Development (OECD):

Evaluation criteria

Definition

Key considerations

Relevance

The extent to which the intervention objectives and design respond to beneficiaries, and partner/institution needs, policies, and priorities, and continue to do so if circumstances change

• Reviewing appropriateness between priorities and needs, objectives, interventions, and outputs as well as analyzing the relevance through the lens of the changing context of the country;
• Analyzing the quality of the design and overtime adaptation due to the changing environment and conducting the overall assessment of the interventions;
• Analyzing key roles and responsibilities of the stakeholders and formulating specific recommendations;

Coherence

The compatibility of the intervention with other interventions in a country, sector, or institution

• Reviewing the internal and external alignment of the interventions with USAID and in-country policies, DPC’s strategic plan or intervention of other actors
• Alignment of the proposed intervention aligns with the local context;

Effectiveness

The extent to which the targeted project objectives were achieved

• Reviewing interventions and outputs in relation to objectives defined in the project document
• Analyzing how the Project’s outputs, results, and goals are met;
• Reviewing the Project activities and outputs through the lens of the technical document approved by USAID/BHA;
• Analyzing the internal organizational and managerial structure of the Project in relation to the fulfillment of project objectives, the quantity, and quality of human resources employed, and the overall management of the project’s resources.

Efficiency

The extent to which the intervention has delivered, or is likely to deliver, results in an economic and timely way.

Key considerations

• Analyzing the trade-offs between planned activities, outputs/outcomes, and objectives as opposed to schedule and cost-effectiveness;
• Reviewing the Project’s day-to-day management as well as risk management strategies and decisions

The analysis should consider the overall use of resources (human, environmental, financial, and time).

Sustainability

The extent to which the net benefits of the intervention continue, or are likely to continue at the End of the NABuRRS Project

Key considerations

• Analyzing the sustainability plan for the interventions;
• Determining the lasting effects of the interventions by focusing on the financial, economic, and social benefits as well as the institutional capacity of the beneficiaries to pursue the activities at the end of the project;

Impact

The extent to which the intervention has generated or is expected to generate significant positive or negative, intended or unintended, higher-level effects.

Key considerations

• Identifying the social, environmental, and economic effects of the intervention;
• Examining the potential effects of the proposed intervention as regards saving lives, and building communities’ capacity to respond to weather-related disasters;

The analysis should consider as well the significance, the differential impact, and the unintended effects of the interventions on targeted communities.