ShelterBox Response Evaluation Consultant – Morocco

On Friday 8th September 2023, an Earthquake struck in the Ighil area of the High Atlas, Morocco, with a magnitude of 6.8. Ighil, a mountainous area with small farming villages, is about 70km (40 miles) south-west of Marrakech. According to national authorities, 2,949 people died in the earthquake and a further 5,674 were injured. 380,000 people were affected, including 320,000 estimated to live in the rural mountainous areas of the High Atlas.

ShelterBox (SB) are looking to contract a Consultant to undertake an Evaluation of the Morocco response so that the effectiveness of the response can be quantified and lessons learned for future responses.

The evaluation will provide an impartial assessment of the SB response to the earthquake in Morocco serving the dual purpose of accountability and learning. It will employ quantitative and qualitative methods to assess both project process and outcome to better understand the extent to which objectives were achieved and emergency shelter was provided in a safe and accountable way to affected communities.

Full details of the requirement, the timelines involved, and the bid format are included in the below linked Request for Proposal.

https://shelterbox.org/about/how-we-work/procurement-opportunities/

The evaluation will:

1. Generate evidence to document and celebrate project success and good practice
2. Identify the challenges faced
3. Suggest recommendations that will be relevant for future projects

Findings will support and strengthen ShelterBox project development and management and contribute to organisational learning and strategy.

Specific Objectives

The specific objective of this Summative Evaluation is to assess how well the project delivered timely and appropriate emergency shelter to earthquake affected communities.

Specific areas of inquiry will involve:

• Assess the partner identification, capacity assessment, selection, and engagement.
• Assess the design (planning, delivery, and management of the project) in accordance with organisational processes and standards.
• Identify and assess key internal and external factors that have contributed to, affected, or impeded the project and how ShelterBox/RI have managed these factors.
• Assess the extent to which accountability to affected populations was realised: How did the project engage with project participants, identify priority needs, communicate objectives, and enable feedback mechanisms?
• Draw key lessons and learning from the project and make recommendations that will help inform ShelterBox projects.

Please note, the learning from the evaluation is primarily intended to support ShelterBox organisational learning, and to support ShelterBox processes and practices so that we are better able to respond to future humanitarian crisis.

As relevant, it will be shared with Rotary International and other stakeholders to communicate findings and celebrate successes and recommendations

Principles and Approach

The following principles underpin ShelterBox approach to evaluation:

• Impartiality and objectivity – statements and judgements should be evidenced by data or evidence. Information should be triangulated, and critical thinking applied. This will provide the evaluation with credibility and therefore make it more worthwhile.
• Centrality of project participants and project partners to enhance learning and further accountability to the affected population.
• Ethical approach, respecting people’s rights and dignity.
• Do not harm: the evaluation will not cause any harm to affected communities or stakeholders.
• Utility: the evaluation will contribute to organisational and potentially sector learning.
• Timeliness: the evaluation will be carried out in a timely manner to best capture lessons and to be able to access stakeholders.
• Compliance with ShelterBox’s policies and procedures – a commitment to our code of conduct and safeguarding policy

Evaluation Framework

The evaluation will draw upon the following criteria:

Relevance

• Did the project reflect the needs and priorities of affected populations? How was this evidenced?
• Were the project activities and outputs consistent with achieving intended outcome?
• To what extent did the project logic remain sound and as intended?
• Were accountability mechanisms embedded effectively throughout the project? How robust were project safeguarding processes and mechanisms?
• To what extent was the project able to adapt and provide appropriate response to context changes and emerging local needs?

Coherence

• Were project activities coordinated with other actors? What coordination mechanisms existed and to what extent did Rotary International/ ShelterBox participate in coordination mechanisms? What were the advantages and disadvantages of doing so?
• Was the project design in line with ShelterBox’s organisational strategies and policies & national priorities for the response, laws and regulations?

Effectiveness

• To what extent has the project achieved its outcomes? Did the delivery of inputs lead to timely implementation of activities resulting in success at output/outcome level?
• What factors have contributed to achieving or not achieving intended project outcomes?
• Were the target project participants reached as expected, were differential results observed for different groups/identity characteristics?
• Were project participants satisfied with the aid distributed?
• Have the M&E system delivered robust and useful information that could be used to assess progress towards outcomes and contribute to learning?
• • Has the accountability system served to ensured participation, identification of needs, regular feedback/complaint from the community and provided a timely response?
• • Did the project implementing model prove to be the most effective model: to what extent did the partnership contribute to a) project efficacy and b) organisational strategic objectives?

Coverage

• Who were the major groups in need of humanitarian assistance? Of these groups how did ShelterBox/Rotary International select project participants?
• To what extent were project participants capacities, preferences and intentions captured in the project design?
• What, if any, were the differential impacts of this humanitarian assistance on different groups or subgroups according to identity characteristics?
• Was the assistance proportionate to needs and devoid of agendas?

Efficiency

• Was the project delivered in a timely manner? Were activities/outputs delivered on time?
• Did Value for Money (VfM) considerations/analysis contribute to decision making?
• Were alternatives approaches/models assessed?

Methodology

The firm/consultant is responsible for the development of the methodological approach which is expected to be most appropriate to achieve the aims of the evaluation.

Efforts shall be made to safeguard the inclusivity and engagement of relevant stakeholders to bring out their voices and to understand how they perceived the project.

Project data will be drawn upon from:

• • Needs Assessments
• • FCRM data
• • Endline findings
• • Project meetings
• • Project documentation e.g. sitreps/reports/decision log

The evaluation process will be compliant with data protection principles, ShelterBox’s Supplier Code of Conduct, and ‘do no harm’ principles.

The Programme Quality team will act as the point of contact and support the process regards both quality assurance and logistical support. The emergency response team will be available for Key Informant Interviews (KIIs), Focus Group Discussions (FGDs), logistical/ planning support and the validation exercise.

The Evaluation Team will:

• Review of ShelterBox strategy/ systems & processes.
• Review Project documentation/ monitoring data (as above).
• Carry out online KIIs with the project implementing team to fully understand the context, operating constraints and team perspectives upon project design and delivery.
• Conduct Endline data collection – a survey with a sample of the affected community to measure agreed outcomes and outputs stated in the project log frame.
• Triangulate quantitative findings and further interrogate/analyse through qualitative processes, such as KIIs and FGDs (with RI reps, project participants, community leaders, the coordinators from coordination structures, including the government and UN etc.).

The Consultant Requirements.

A consultancy firm/individual legally registered to operate in Morocco with formal access to visit and carry out research in the project locations.

The selected firm/ consultant should possess the following minimum qualifications as follows:

• Higher university degree in humanitarian / development studies or any related academic discipline or an affiliation with a research institution.
• 5 years evaluating humanitarian programmes, with experience in gender and protection in emergencies.
• Familiarity with the OECD/DAC evaluation framework.
• Strong background in monitoring and evaluation, with experience in the use of quantitative and participatory qualitative methods of data collection.
• Excellent analytical, communication, writing and presentation skills in English.
• Ability to analyse complex intervention.
• Conversant with the context in Morocco