Call for technical proposals – mid term evaluation

Purpose of the Evaluation.

This is a mid-term evaluation of the IA CHC 2023 – 2027 Somalia Emergency Preparedness and Response in Somalia/Somaliland. The purpose of this evaluation is to review progress against set targets and the impact derived through the programme implementation to date, as well as any learning that can be embedded into future programming for learning. This will be conducted against the OECD evaluation criteria. OECD has defined six evaluation criteria – i.e., relevance, coherence, effectiveness, efficiency, impact, and sustainability – and two principles for their use. These criteria provide a normative framework used to determine the merit or worth of an intervention (policy, strategy, programme, project or activity). They serve as the basis upon which evaluative judgements are made.

Description of the Context.

Somalia’s population continues to be battered by climate extremes, experiencing the worst drought in decades followed by the most extensive floods in generations within just a few months. Seasonal cycles of dry and wet conditions have become more extreme and frequent. The 2020-2023 drought, considered the most severe in four decades, brought the country to the brink of famine. While sustained humanitarian assistance and the Gu rainy season from March 2023 averted worse outcomes, an estimated 43,000 excess deaths occurred in 2022 alone, half of them children under the age of five. Households have struggled to recover since the end of the drought, as equally historic Deyr rains and flooding, exacerbated by a strong El Niño and a positive Indian Ocean Dipole, hit the country in October and November 2023. Deyr flooding caused significant losses to agriculture, livestock, and critical infrastructure. By the end of November 2023, almost half of the country’s districts had recorded flooding, with riverine and flash flooding greatest along the Shabelle and Juba Rivers in the south and southwest. 2.5 million people were affected, and more than 1.5 million hectares of farmland were inundated. The floods damaged or destroyed critical water, sanitation, health, education, and logistics infrastructure, threatening to reverse modest gains from infrastructure investments in previous years. Climate shocks and conflict displaced a record 2.9 million people in 2023 alone. The vast majority of people who fled their homes – 2.3 million or 75 percent – were displaced by climate shocks (1.7 million by flooding and 531,000 by drought). At the same time, the number of people newly or re-displaced due to conflict and insecurity in 2023 – 653,000 people – also stood at an all-time high. In total, more than 3.8 million people are currently displaced in Somalia, with most households displaced more than once. Civilians continue to pay the price of conflict and insecurity. Following a sharp increase in civilian casualties in 2022 due to an escalation of hostilities, this trend continued in 2023, with almost 1,300 civilian casualties recorded between January and September. Somalia also remains among the countries recording the highest number of grave violations against children worldwide. Between January and September 2023, 1,742 grave violations against 1,660 children were verified. Critical civilian infrastructure, which is protected under international humanitarian law, continues to be targeted, including 34 schools and 18 hospitals attacked in the first nine months of 2023. Ongoing conflict contributes to a volatile and difficult operating environment for humanitarians. Attacks on humanitarian workers and infrastructure, and restrictions on movements have exacerbated access and operational challenges. An estimated 580,000 people live in areas that are hard to access by the humanitarian community, the majority being women and children. Out of 74 districts, 23 are either hard or extremely hard to reach. The changing security landscape in Somalia resulting from the drawdown of ATMIS and the potential for shifting patterns of insecurity will require adaptability by humanitarians to stay and deliver and promote acceptance for humanitarian interventions. As a result of these shocks, an estimated 6.9 million people – almost two in five Somalis – remain in need of humanitarian assistance in 2024. While this 17 percent decrease compared to 2023 is mainly due to a slight reduction in food insecurity and malnutrition levels, 4.3 million people remain acutely food insecure. In a context where 54 percent of households already lived below the national poverty line before the 2020-2023 drought, more than half of all households have suffered further income reductions due to loss of employment, livestock, and other assets. The more recent destruction of farmlands and standing crops caused by the late-2023 flooding, in combination with disrupted livelihoods and damage to shelter and homes, further aggravates the situation for many households. The poorest households therefore continue to struggle to access food, income, and critical services such as water, health, education, and protection. At the same time, the record inflow of newly displaced people into mainly urban and peri-urban areas has put further pressure on already stretched WASH, health, and shelter conditions, and increased protection risks linked to exploitation and abuse. The overwhelming majority of people who were displaced due to conflict in 2023 do not intend to return any time soon. Needs for access to food, water, livelihood opportunities, sanitation, health, education, and protection services, especially in settlements and informal sites hosting internally displaced people, therefore remain significant. In turn, while most flood-driven displacement is expected to be temporary in nature, its scale highlights the unsustainable living conditions of the poorest households and displaced populations on frequently ill-suited, flood-prone lands. Going into 2024, needs for access to food, safe water, sanitation, and health specifically are expected to increase in flood-affected areas due to flood damage to crops, WASH, and health facilities, and the widespread contamination of water sources.

Description of the subject of the review/evaluation.

The programme is titled Emergency Preparedness and Response in Somalia/Somaliland.​ The programme overall goals and objectives are to provide lifesaving and sustaining multi-sectoral assistance to conflict and natural disaster affected people in Somalia/Somaliland, as well as fostering preparedness. The twin-track approach will be delivered through integrated Health & Nutrition (H&N), Multi-Purpose Cash Assistance (MPCA), Education in Emergencies (EiE), and Emergency Response Preparedness​ (ERP). It has started on 01 January 2023 and will be implemented over a total of 5 years (i.e., ending in December 2027). It is planned to reach 40,000 programme participants (12,100 men/boys and 27,900 women/girls)​ with bespoke assistance. Over the implementation period, it is intended to deliver on the below detailed outcomes:

• IDPs and vulnerable host HHs affected by natural disasters and conflict have improved access to basic needs including and non-food items.​
• Increased equitable access to quality education in a protective environment for conflict and natural disaster affected children in Somalia​.
• Improved access to quality essential primary health and nutrition services focusing on maternal, newborn and child health in Banadir and lower Shabelle.​
• Concern Somalia/Somaliland Programme and its 5 local NGO partners are better prepared to respond to natural disasters and conflicts.

The report for the first year has been submitted to donor in March 2024. Implementation is being rolled out according to plan. Burn-rate is in line with financial commitments.

Evaluation objectives and scope.

The objective of this evaluation is to provide an assessment of the Irish Aid funded programme against each of the DAC criteria. The evaluation will focus on the design, implementation, activities and results and will cover the period of Programme implementation up to now. This evaluation will also be used by the country team for learning and iterative programme improvements. OECD DAC has defined six extended evaluation criteria – i.e., relevance, coherence, coverage, effectiveness, efficiency, impact, and connectedness – and two principles – i.e., 1) the criteria should be applied thoughtfully and contextually to support high quality, useful evaluation for their use; and 2) the criteria should not be applied mechanistically, but rather depending on the purpose of the evaluation – which are meant to provide a normative framework to determine the merit or worth of an intervention (policy, strategy, programme, project or activity). They serve as the basis upon which evaluative judgements are made.

Evaluation questions.

Evaluation Criteria

Relevance/Appropriateness

EQ 1: Were the objectives and the design of the intervention relevant and appropriate to the context and to the needs of recipients?

• Was there a clear rationale for the selected intervention activities?
• Do communities/people targeted by the intervention consider that the programme design met their needs sufficiently?
• Did Concern adapt appropriately in response to any changes in the context and lessons learnt during the implementation of the intervention?

Effectiveness

EQ 2A: Is there evidence that anticipated results are being achieved at the current stage of the programme?

• What is changing as a result of the programme?

EQ 2B: Is the programme achieving the planned outputs and outcomes and are they on time?

• What evidence is there that anticipated results are being achieved at the current stage of the programme?
• To what extent can the emerging results be attributed to Irish Aid Chronic Humanitarian Crises Scheme?
• Do target communities feel that the response was effective and timely delivered when most needed?
• Does the response team coordinate effectively with external stakeholders (e.g., clusters, other NGOs and agencies, relevant government units, etc.)?

Efficiency

EQ 3: Did the intervention use the available resources in the most efficient manner to achieve its outputs?

• Did the intervention have sufficient and appropriate resources?
• Did the intervention pay competitive prices for its inputs – in other words: did it prove cost effective?
• Were there any alternatives for achieving the same results with fewer inputs?

Impact

EQ 4: Did the intervention achieve the intended impacts?

• Is there evidence of change at impact level (positive or negative)? If so, what contributed to this change? If not, why not?
• What are the wider effects of the programme on: individuals; groups (gender, age, disability); communities; institutions; or conflict dynamics? Were they positive or negative?
• Were there any unintended results? Were they positive or negative?

Coverage

EQ 5: Was the coverage of the programme extensive and inclusive?

• Did the intervention manage to leave no one behind as per Concern’s core mandate and targeting criteria?
• Was coverage of the intervention, as well as the targeting criteria, discussed in a participatory fashion with internal and external stakeholders as appropriate?
• Was the coverage inclusive of most vulnerable groups, taking into account Concern’s crosscutting themes (more details below)?

Coherence

EQ 6: Was there consistency between Concern’s activities and the relevant policies?

• Was the intervention informed by and aligned with the fundamental humanitarian principles? Did the programme adhere to the Red Cross Code of Conduct, Sphere, Core Humanitarian Standard and sector-specific standards, where appropriate?
• Was there evidence that the intervention adhered to relevant Concern policies (e.g., Approach to Emergencies, DRR, Equality, Protection, Child Safeguarding Policy, HIV and AIDS)?

Sustainability

EQ 7: Will the benefits achieved through Concern’s intervention last?

• Is the intervention building an enabling environment for sustainable development?
• Have elements for both actual and prospective sustainability been identified?
• Have risks and potential trade-offs been identified?

Connectedness

EQ 8: Are emergency, short term activities carried out in a context that takes longer-term and interconnected problems into account?

• Has the impact of relief/life-saving activities been considered system-wide so far (e.g., on local power structures, government capacity, gender equality or the environment)?
• Have strong partnerships been established?
• Has there been an emphasis on capacity building and knowledge transfers to ensure that local actors are better equipped to cope with crises once the project ends?

Cross-cutting Themes

• What were the synergies and interlinkages between this programme and other Concern interventions?
• To what extent did all aspects of operational support and programme design, implementation and monitoring consider the core humanitarian principles, the extended DAC criteria as detailed above, and the relevant Concern cross-cutting issues of the following themes? Namely: equality; protection; Disaster Risk Reduction (DRR); conflict sensitivity; the environmental impact; and partnership (more details on the latter can be found under connectedness).

Methodology

The Evaluation Team (Consultants) are expected to develop an appropriate work plan for the evaluation that is based on the context. The work plan must be as participatory as possible under the current conditions, involving different stakeholders including beneficiaries, partners, Concern staff and other stakeholders. The prescribed methodology will include the following activities:

• Detailed programme document mid-term review.
• Assessment of achievement of indicators under each outcome.
• Key Informant Interviews with Key Programme staff; partner staff; relevant local authorities; key programme participants/community leaders.
• Focus Group Discussions with programme participants (appropriately disaggregated). Where focus groups are not possible, remote data collection with programme participants should be considered (tools and approaches).
• Household surveys with a sample of beneficiaries using quantitative data aspects.

In addition to the mixed methods approach outlined above, the evaluators will also be applying Qualitative Comparative Analysis alongside the data validation exercises to increase the robustness of the evaluation process and confidence in the findings.

Expected products/outputs (including critical elements to be addressed in a mid-term evaluation report)

The outputs of this evaluation report which will contain:

• An executive summary (no more than 3 pages).
• An appraisal of how well the project has fared against each of the extended OECD-DAC criteria using the following grading scale, where:

5 = Outstanding performance

4 = Very good overall performance with few shortcomings

3 = Good overall performance but with some minor shortcomings

2 = generally acceptable performance but with some major shortcomings

1 = Barely acceptable performance with many major shortcomings

0 = Totally unacceptable performance or insufficient data to make an assessment

• An examination of the extent to which programme design, implementation and monitoring have considered the extended DAC criteria for evaluating emergency responses and the relevant Concern cross-cutting themes. Targeted achievable and time-bound recommendations for improving the remainder of the programme implementation shall be included. The report should be no longer than 25 pages (with 15 of them being focused on DAC appraisal).

Upon finalisation of the report, an in-country briefing with the Emergency and MEAL team will be held to give a preliminary overview of main findings to discuss feedback on the draft Evaluation report before the report is finalised and signed off by the Country Director or Regional Director.

Reporting lines

The Evaluators will report to the Country Emergency Coordinator and liaise with the MEAL Manager for day-to-day evaluation process. The Country Director will be responsible for overall security.

Composition, skills and experience of the Evaluation team (consultants)

The Lead Evaluator should demonstrate the following abilities:

• Experience of conducting evaluations of humanitarian responses.
• Experience in emergency responses in complex crises such as Somalia.
• Experience of qualitative data collection methods, including participatory approaches.
• Experience of working remotely with a distributed team, including providing support to colleagues at a distance.
• Ability to assess the quality of programmatic data collection systems, analyse budgets, results frameworks, M+E reports and surveys, and other documents.
• Experience of writing evaluation reports to a high standard in English.
• Links to team leaders/evaluators/enumerators in country (especially in Banadir and Lower Shabelle).
• Availability to travel for the purpose of date collection.
• Adherence with the Concern Code of Conduct (COC) and associated policies (please find here links to the English version of the COC and associated policies – please ensure careful review before applying. Overall codes and policies can be found here). More information is available below under “note on safeguarding” section.

Plan for Evaluation implementation (including timelines)

The evaluation of the Programme in Somalia will be carried out in July/August 2024 tentatively. Duration of evaluation will be approximately 4 weeks and will include:

• 0.5-week desk review of documents.
• 2 weeks in-country data collection phase – to include initial orientation with teams, all primary qualitative data collection, preliminary analysis and briefing with country teams on initial findings.
• 1 week for data analysis and to produce draft report.
• 0.5 week for presentation and incorporating feedback in Final Evaluation Report.

How to apply

Applications for this contract from qualified consultants must include the following:

• CV of evaluation team members, as well as a breakdown of the responsibilities of each individual.
• Technical proposal.
• Two samples of relevant previous work.
• Financial proposal.

Please email application documents to [email protected] by 21 June 2024 close of business.

Note on Safeguarding: Concern has an organisational Code of Conduct (CoC) with three Associated Policies; the Programme Participant Protection Policy (P4), the Child Safeguarding Policy and the Anti-Trafficking in Persons Policy. These have been developed to ensure the maximum protection of programme participants from exploitation, and to clarify the responsibilities of Concern staff, consultants, visitors to the programme and partner organisation, and the standards of behaviour expected of them. In this context, staff have a responsibility to the organisation to strive for, and maintain, the highest standards in the day-to-day conduct in their workplace in accordance with Concern’s core values and mission. Any candidate offered a consultancy opportunity with Concern Worldwide will be expected to sign the Concern Code of Conduct and Associated Policies as an appendix to their consultancy contract. By signing the Concern Code of Conduct, candidates acknowledge that they have understood the content of both the Concern Code of Conduct and the Associated Policies and agree to conduct themselves in accordance with the provisions of these policies. Additionally, Concern is committed to ensuring safeguarding and protection of vulnerable adults and children in our work. We will do everything possible to ensure that only those who are suitable to work or volunteer with vulnerable adults and children are recruited by us for such roles. Subsequently, being engaged as a consultant with Concern is subject to a range of vetting checks, which may include criminal background checking.

Note on intellectual property: The evaluation information and report will be the intellectual property of Concern. Its content can be shared with third parties only with Concern’s consent. Sensitivities to confidentiality must be respected. Names and identifiable data associated with any particular quotes or experiences should not be cited. The evaluation team, if collecting any personal data will anonymize the information to avoid any unintended harm.

Note on data protection: During this recruitment and selection process, you will provide Concern with your personal data. Concern takes its responsibilities towards this personal data very seriously and is committed to complying with all relevant data protection legislation. Concern uses this information to consider your suitability for this position and may contact you to call you for an interview. Your data may be shared internally to consider this application. Concern will store your data securely.

You have certain rights under data protection legislation. We will be happy to answer any questions from your side on the matter.