Evaluation of Education Inclusion in IGAD Member States

In December 2017, the Government of the Republic of Djibouti, the Inter-Governmental Authority on Development (IGAD), in collaboration with the United Nations High Commissioner for Refugees (UNHCR), the European Union (EU), and the German Society for International Cooperation (GIZ), convened a regional ministerial meeting in Djibouti on refugees, returnees and the host community. At the meeting, IGAD ministers in charge of education resolved to take collective responsibility to ensure that every refugee, returnee and member of host communities has access to quality education in a safe learning environment within their respective countries without discrimination.

These commitments are expressed in the Djibouti Declaration on Regional Refugee Education, to which an Action Plan is annexed, which outlines the actions to be carried out in the delivery of quality education and learning outcomes for refugees, returnees, and host communities in the region. The Djibouti Declaration emphasized on five key strategic interventions:

1. Inclusion of refugees and returnees into national education systems with a multi-year costed plan of action.
2. Higher Education and Technical Vocational Education and Training/ post-secondary skills and competencies.
3. Regional Qualifications Framework
4. Accreditation and Certification of Education
5. Resource Mobilization and Partnership

The 2017 Djibouti Plan of Action stipulates the actions under this commitment as follows.

1. Determination of the status of integration of education of refugees and returnees into Education Sector Plans of Member States.
2. Mapping existing initiatives, proposals and activities to integrate the education of refugees and returnees into Education Sector Plans by 2020.
3. Supporting activities that lead to the integration of refugees and returnees into National Education systems.
4. Strengthening EMIS data collection to include the needs of refugees and returnees
5. Fully integrate into the national education systems schools for refugees, including all those residing in urban areas, camps, and settlements.

Rationale and reasons for the evaluation

Five years have passed since the initial commitments were made during the Djibouti Declaration conference. Since then, IGAD has organised annual stocktaking meetings with IGAD member states and/or the Core Partners, which has included discussions and updates on the inclusion of refugees and progress made. Some countries have made more progress than others towards their commitments and valuable lessons can be drawn that may benefit all signatories as efforts continue towards implementation of joint commitments. In addition, ahead of launching the monitoring framework which will be implemented by all Member States with standardised indicators, it is a good moment to document status quo, progress, and achievements, and determine what factors have allowed some countries to progress at a faster pace than others to support the fulfilment of the commitments made by all member states.

Objectives

This formative evaluation is commissioned for both learning and accountability purposes to contribute towards strategic reflections and:

• • Document progress and achievements at country level and critically assess and analyse drivers of and obstacles to progress, identify good practices and draw recommendations and lessons that may inform future planning and implementation in relation to the joint commitments made. – Establish what conditions and type of environment (political/financial/humanitarian) that enable countries to move up the continuum scale for education inclusion, and what role partners such as IGAD, UN, INGOs and Donors play to this end.

Against this background, the evaluation will combine features of a Process Evaluation as it sets out to establish how outcomes are generated and obstacles and drivers with an assessment of progress made, results achieved (outputs and outcomes) including of their quality, and financial, social and institutional sustainability. This will be measured by reflecting on the progress presented during the second and third IGAD high level technical conferences, and analysing each country level documentation over the last five years.

Scope of services

Scoping of the evaluation refer to the following parameters:

• Geography: The evaluation will involve all seven countries in the IGAD region, i.e. Djibouti, Ethiopia, Kenya, Somalia, South Sudan, Sudan, Uganda. It is recommended to do three in-depth country case studies to bring out recommendations and findings to compare and contrast countries that have reached different levels of inclusion of refugees into national education systems and have a lighter touch on the remaining four countries.
• Timeframe: The evaluation will focus on the period from the signing of the Djibouti Declaration until to date.
• Sector domains: The evaluation will look at both regional and national policy level instruments and include how policy has translated into action and implementation of the policy at the ground level.

Participation of main stakeholders: IGAD focal points and key technical teams from the UN and INGO partners, and the Ministry of Education will be included in the information collection stages and debriefs at country level will be anticipated with each of the seven countries to ensure adoption of recommendations and lessons learned.

The evaluation is planned to be completed within a maximum of one year, from July 2022 to June 2023. Key deliverables and timelines are indicated in the Table below:

During the case study we expect the evaluation team to work in pairs per country.

Suggested timelines:

• The inception report will be delivered by 6-7 weeks, including a scoping mission by the team leader to Djibouti for the team leader to meet with the IGAD secretariat.
• The inception report will be quality assured by the Manager of the Evaluation from UNHCR. It will be shared with the Evaluation Reference Group for comments once it has passed the UNHCR internal quality assurance check. Sharing of the inception report with the ERG is expected to take a minimum of 2 -3 weeks for review at regional and country level.
• Country Level missions for the case studies will be a minimum of two weeks and should be undertaken by at least two team members.
• The final report should be delivered within 8 weeks after the completion of the field missions

Key deliverables include:

• A desk review (approximately 20-30 pages) including a contextual analysis, preliminary findings relating to the evaluation questions and drawing conclusion on data gaps and issues to be further examined during the evaluation.
• Inception report (approximately 10-20 pages excluding annexes) confirming the scope of the evaluation, the evaluation questions, methods to be used, as well as the results framework – and summarizing findings derived from a review of existing documentation.
• End of mission country debriefs in each country visited.
• Workshops with relevant staff in Country and Regional Bureau, to validate the findings.
• Draft and Final evaluation report (50 pages), including a 5-page stand-alone executive summary.

Evaluation Team

Travel will be subject to COVID-19 restrictions in place as well as Security concerns within the IGAD countries. All travels and logistics (international and in country) will be organised by the consultancy firm. The Evaluation team should consist of a minimum of four evaluators and a research assistant, ensuring that the team has the required language and technical competencies to complete the required work, and ensuring there is at least one research assistant. The team size is considered a requirement given the scale of the evaluation is across seven countries, and the resources required to conduct the in-depth analysis of progress across four country operations. It is recommended that some of the team members be based in the region.

It is expected the team will execute the following tasks:

The Evaluation team is expected to execute the following tasks:

1. Inception report with a detailed and realistic work plan for the evaluation, an evaluation matrix and methodology to respond to the questions specified in the terms of reference (or subsequent revisions of the evaluation questions as agreed by the Evaluation Reference Group).
2. A desk review as specified above
3. Generate evaluation reports (process and outcome assessment) and strategic recommendations deliverables as shown in the table above as per agreed timelines, and in accordance with contractual requirements.
4. Facilitate engagement with the ERG and key stakeholders at the regional workshop.
5. Provide written responses to comments from the Evaluation Reference Group led by UNHCR, and update draft deliverables accordingly,
6. Maintain effective communication, including regular updates to the Evaluation Reference Group

Required Qualifications:

Lead Senior Evaluator

1. Post graduate qualification in Education, Monitoring and Evaluation, Statistics or International Development, PHD will be an added advantage.
2. Extensive knowledge of evaluating education programmes in emergencies for refugees and marginalized children.
3. At least 10 – 15 years’ experience in conducting humanitarian and development evaluations in complex environments, experience working with UN agencies will be an asset.
4. Demonstrated experience and knowledge of partnership engagement, working with governments and policy makers.
5. Extensive knowledge of and experience in participatory evaluation methods, particularly formative and forward-looking approaches, and supporting integration of learning practices
6. Extensive experience with multi-partner cooperation at field level, national, regional, and global level.
7. Excellent oral and written communication skills in English
8. Extensive knowledge of qualitative and quantitative data collection methods and analytical methods and techniques.
9. Experience in generating strategic, informative and action-oriented recommendations to governments, donors and strategic management.
10. Experience with the ethics of evidence generation; collecting data from refugee children and vulnerable groups; familiarity with ethical safeguards.
11. Demonstrated capacity to plan, implement and lead large evaluation including experience with multi-country evaluations

Team Evaluators (minimum two)

1. Post graduate qualification in Education, Monitoring and Evaluation, Statistics or International Development.
2. Extensive knowledge of evaluating education programmes in emergencies and working with refugees and marginalized children.
3. At least 5 years’ experience in conducting evaluations in complex environments, experience working with UN agencies or other INGOs will be an advantage.
4. Extensive knowledge in research, monitoring and participatory evaluation approaches.
5. Experience in coordinating evaluation activities with multiple partners – global experience preferred
6. Familiarity with UNHCR’s Commitment to the Protection of Refugees and IDPs.
7. Excellent oral and written communication skills in English, French (Djibouti) and Arabic (Sudan)
8. Extensive knowledge of qualitative and quantitative data collection methods and analytical methods and techniques.
9. Experience in working with field-based staff, community leaders and government officials.
10. Experience with the ethics of evidence generation; collecting data from refugee children and vulnerable groups; familiarity with ethical safeguards.

A Research Assistant (minimum 1)

1. Experience in quantitative and qualitative data processing, analysis and presentation.