Terms of Reference for Final Evaluation of the Irish Aid Humanitarian Programme Plan (HPP) in DRC

• Purpose of the Evaluation

This is the Final Evaluation of the Irish Aid 4 year Humanitarian Programme Plan (2019-2021 (now extended to 2022)) in DRC. This evaluation will also be used by the country team for learning in order to feed into the design of other programmes and particularly into the next Irish Aid programme (from 2023).

• Description of the Context

The Democratic Republic of Congo has been affected by conflict for the past 25 years, and the level of insecurity and subsequent humanitarian impact remain severe. According to OCHA, in 2021, 86.3% of internally displaced in DRC were displaced due to armed conflict and attacks. As one of the provinces most affected by fighting, North Kivu has been in a state of chronic crisis for many years, and is home to 1.8 million of the country’s 5.6 million internally displaced people (IDPs). Moreover, the resurgence of the 23 March Movement (M23) and intensification of conflict has lead to displacement and further deterioration of the humanitarian situation in Nord Kivu (ACAPS June 2022).

A REACH assessment found that in October 2021, conflict was cited as the main reason for the disruption of livelihood activities by 71% of households interviewed. Likewise, REACH found that insecurity was the main reason for limited accessibility to land for farming and thus insufficient food levels, according to 40% of households. While significant numbers of households are affected by fighting, and experience displacement and disruption to their livelihoods, they do not always receive assistance which further exacerbates food insecurity.

The territories of Masisi, Rutshuru et Beni are some of the areas worst affected by fighting in the province. A total of 2.1m people are facing IPC levels 3 and 4 in North Kivu, according to IPC projections for September 2021-August 2022, and the territories of Masisi and Rutshuru have among the highest proportion of the population in IPC levels 3 and 4. According to WFP’s recent Emergency Food Security Assessment (EFSA), Masisi in particular is home to a significant number of displaced people living with host families.

Additionally, women in particularly are likely to face significant risks of sexual violence and inequalities around access to land and education exacerbated by the conflict. The DRC ranks 175 out of 189 countries in the UNDP Gender Inequality Index, and the figures for sexual violence are estimated to be much lower than the reality due to the limited geographical coverage of protection monitoring and services.

• Description of the subject of the evaluation.

Improved coping mechanisms and responses to new emergencies in the most vulnerable fragile communities in Eastern DRC is a programme that aims to provide cash transfers and life skills training to fragile communities that have been impacted by conflict. The communities will also be provided with protection and gender equality training. The programme includes an emergency component, which is meant to provide relief should the programme’s communities suffer a shock during the course of the programme.

The goal of this programme is to improve coping mechanisms of targeted households. These coping mechanisms and responses will be built firstly by providing beneficiaries with the means and capacity to start or invest in income generating activities. The programme will also address the issues of inequality that exist in communities and ensure that measures are taken by the communities to mitigate against various risks they are likely to encounter. Vulnerable conflict-affected households will also be supported with cash transfer or direct NFI distribution in case where there are communities who have recently been displaced and require humanitarian assistance.

This programme aims to bridge the gap between humanitarian assistance and development programmes, for communities who have recently been displaced due to conflict. The programme addresses the factors preventing communities becoming more resistant to shocks by providing cash transfers, together with training the community members on gender equality, protection and other life skills that will assist them to reduce reliance on negative coping strategies.

The programme is implement in Masisi in North Kivu.

• Evaluation objectives and scope

The objective of this evaluation is to provide an assessment of the Irish Aid funded programme against each of the extended DAC criteria. The evaluation should focus on the design, implementation, activities and results and will cover the time period from 2017 to quarter 2, 2022. This evaluation will also be used by the country team for learning in order to feed into the design of other programmes and particularly into the next Irish Aid programme (from 2023), and should therefore offer concrete and targeted recommendations for improving programming going forward.

• Evaluation questions (sub questions to be shared with the consultant once selection starts)

Relevance

– Were the objectives and the design of the intervention relevant and appropriate to the context and the needs of recipients?

Effectiveness

– Is there evidence that anticipated results are being achieved at the current stage of the programme?

– Is the programme achieving the planned outputs and outcomes and are they on time?

Efficiency

– Did the intervention use the available resources in the most efficient manner to achieve its outputs?

Impact

– Did the intervention achieve the intended impacts?

Connectedness

– How were longer-term problems given consideration during implementation of emergency activities?

Coherence

– Was there consistency between Concern’s activities and the relevant policies?

Coverage

– Is there evidence that Concern’s intervention reached the most vulnerable?

Cross-cutting

– Synergies?

– Consideration of cross-cutting themes (HIV, conflict sensitivity, DRR, protection, partnership, equality, environment)

– COVID-19 impact?

• Methodology

The external evaluator is expected to develop an appropriate work plan for the country level evaluation that is relevant to the country context. The work plan must be as participatory as possible, involving different stakeholders including beneficiaries, partners, Concern staff and other stakeholders.

The prescribed methodology will include the following activities:

• Detailed programme document review including Country Context Analysis (if applicable); HPP 2019-2022 Programme of Work; HPP Results Frameworks 2019-2022; Report of HPP 2018 Final evaluation
• Assessment of achievement of indicators under each outcome using data from Concern Digital Data Gathering (DDG) system.
• Key Informant Interviews with Key Programme staff; partner staff; relevant local authorities; key programme participants/community leaders
• Focus Group Discussions with programme participants (appropriately disaggregated). Where focus groups are not possible, remote data collection with programme participants should be considered (tools and approaches to be included in the training (see below)).

In addition to the mixed methods approach outlined above, the external evaluators will also be applying Qualitative Comparative Analysis alongside the data validation exercises to increase the robustness of the evaluation process.

• Expected products/outputs (including critical elements to be addressed in a final report)

Inception Report

After the detailed programme document review, the external evaluator will produce an inception report, which will contain :

• Objective of the assignment
• A detailed description of the methodology for the evaluation
• Data collection methods;
• Data collection tools;
• Sampling;
• Approach to quality control.
• Data analysis methods
• Justification for revising the Evaluation Questions (if relevant)
• Detailed workplan

Final Report

The outputs of this evaluation in each country will include a final country level evaluation report, which will contain:

• An executive summary (no more than 3 pages)
• Project background, with a project overview and a description of the theory of change
• Methodology, with details on:
• Logic model
• Data collection methods
• Data collection tools
• Sampling
• Quality control
• Data analysis
• Limitations of the evaluation
• Evaluation results, with an appraisal of how well the project has fared against each of the extended OECD-DAC criteria using the following grading scale, where:
• 5 – Outstanding/Exceptional Performance
• 4 – Very good overall performance (above expectation)
• 3 – Good overall performance s (as expected)
• 2 – Acceptable performance but with some shortcomings
• 1 – Barely acceptable performance with some major shortcomings
• 0 – Totally unacceptable performance or insufficient data to make an assessment
• An examination of the extent to which programme design, implementation and monitoring have considered the relevant Concern cross-cutting issues
• Targeted recommendations for improving future programming
• Annexes, including the ToRs, the evaluation matrix and full details on tools used during the evaluation

The report should be no more than 25 pages (15 pages focused on findings and conclusions against the extended OECD-DAC criteria).

Briefings and Presentations

An in-country briefing with country teams should also be held to give a preliminary overview of main findings. A virtual meeting (e.g. Zoom; MicroSoft Teams or Skype) should also be held with country teams to discuss feedback on the draft Evaluation report, before the report is finalised and signed off by the Regional Director. Where possible the global evaluators will attend these briefings remotely. A PowerPoint presentation (or similar) must be prepared for each briefing and shared with the external evaluators.

• Reporting lines

The external evaluator will report to the Programme Director and will closely collaborate with the Programme Quality and Development Coordinator and National M&E Coordinator. For field related data collection, the M&E leads for Concern will be the external evaluator’s key points of contact. Any feedback, changes or challenges will be communicated through the M&E leads to the Concern Worldwide DRC Programme Director and Programme Quality and Development Coordinator.

• Composition, skills and experience of the Evaluation team

At the minimum, the consultant/s must possess the following:

• Strong knowledge of Core Humanitarian Standards (CHS) and Extended DAC Criteria.
• Experience working in emergency context.
• Excellent knowledge of the DRC context, especially in terms of security and culture, is required.
• Experience in emergency responses in complex crises.
• Experience of qualitative data collection methods, including participatory approaches
• Ability to assess the quality of programmatic data collection systems, analyse budgets, results frameworks, M+E reports and surveys, and other documents.
• Experience of writing evaluation reports to a high standard, in English
• Experience of working as part of an agile team
• Excellent organizing, facilitating, presentation and communication skills.
• Excellent written and oral French and English is essential. Swahili is a plus.
• Excellent report writing skills.
• Plan for Evaluation implementation (including timelines)

The evaluation of the HPP in DRC will be carried out in September 2022, tentatively starting on September 26th

Duration of evaluation will be approximately 4 weeks and will include

• 0.5 week desk review of documents,
• 2 weeks data collection phase – to include initial orientation with teams, all primary qualitative data collection, preliminary analysis and briefing with country teams on initial findings
• 1 week for data analysis and to produce draft report
• 0.5 week for virtual presentation and incorporating feedback in Final Evaluation Report