Final Evaluation of Community Stabilization Program (CRP) – PhaseX

TITLE: FINAL EVALUATION OF CRP X Project; Community Revitalization Program

Commissioned by: IOM IRAQ COUNTRY OFFICE

The International Organization for Migration (IOM) is part of the United Nations System as the leading inter-governmental organization promoting since 1951 humane and orderly migration for the benefit of all, with 174 member states and presence in over 100 countries. IOM works on migration and development, facilitating migration, regulating migration and solutions for forced migration. IOM activities that cut across these areas include the promotion of international migration law, policy debate and guidance, protection of migrants’ rights, migration health and the gender dimension of migration.

1. EVALUATION CONTEXT
   1. Political, Environmental, and Socio-economic Context:

Iraq’s security situation and development outlook improved dramatically following the defeat of the Islamic State of Iraq and the Levant (ISIL) on Iraqi territory in 2017. Subsequent overall recovery, however, has been slow as the Government of Iraq (GoI) has confronted the task of creating opportunities for durable solutions for IDPs, returnees and the wider population through re-establishing access to basic services and address barriers to return. Since 2018, the GoI has been advancing the closure of Internally Displaced Person (IDP) camps throughout the country, resulting in a series of camp closures in 2018, late 2019 and again beginning in October 2020, which led to the eviction of hundreds of thousands of IDPs, including almost 50,000 since October 2020 alone. At the same time, an estimated 1.18 million Iraqis still remain displaced in camps, informal settlements and other out-of-camp locations. IDPs in camps and informal settlements commonly face complex challenges and barriers to the resolution of their displacement, including tribal issues and security problems in areas of origin, perceived affiliation with ISIL, as well as issues related to destroyed housing, lack of access to employment, lack of basic services and other issues.

Iraq Masterlist Report 125. January – March 2022.[1]

Setbacks encountered in 2020 related to the COVID-19 pandemic, economic challenges resulting from instability (internal, regional, and global), have created further obstacles to both IDPs return to areas of origin or local integration in their host communities as well as successful reintegration of returnees in their areas of origin.

In addition to internal displacement in conflict-affected areas of Iraq, population groups in the south of Iraq continue to face challenges linked to the increasingly limited supply of water that risk their stability and threaten to create additional internal displacement or drive rural-urban migration. In particular, environmental challenges such as water scarcity, pollution and reduced ecosystem services can affect people’s income generation possibilities and their health and physical wellbeing and serves as a factor in the decision to leave one’s place of residence[2].

Supporting the development of durable solutions policies and the implementation of durable solutions activities in Iraq is of critical importance to respond to the livelihoods, shelter, protection and social cohesion needs of IDPs, returnees, and host community members as well as support the resolution of displacement-linked vulnerabilities.

1. 1. Summary of the IOM Iraq Community Revitalization Program (CRP):

Donor

Project Title

Start date

End date

Total Budget

PRM

Community Revitalization Program X

1-Oct-21

30-Sep-22

29,100,000 USD

Since 2007, IOM has contributed to the stabilisation of Iraq by improving the socio-economic conditions of vulnerable individuals and communities. Hence, strengthening the resilience of communities and promoting social cohesion through a comprehensive approach encompassing five main outcome areas:

1. Increased Human Capital
2. Promoting Good Governance
3. Improving Access to Social Services
4. Supporting Local Economies
5. Creating Safer Communities

A cornerstone of IOM’s Transition and Recovery[3] approach is the CRP launched in 2011, following three phases of the Program for Human Security and Stabilization (PHSS) implemented from 2007 to 2011. Phase X of the CRP continues to contribute towards improving the conditions for dignified and voluntary durable solutions to progressively resolve the displacement of internally displaced persons (IDPs) through returns, local integration, and settlement in new locations, while also supporting efforts to prevent additional climate-induced displacement in Iraq. Over a 12-month period IOM developed and implemented multi-component interventions across approximately 40 communities in 12 governorates.

In locations of displacement, IOM facilitated pathways for safe, dignified and voluntary returns. IOM assessed intentions to return of the IDPs, provided information on conditions in areas of return or relocation, identified main barriers to return and provided targeted services, which included civil documentation and legal assistance, social cohesion support, financial assistance, health consultations and protection monitoring.

In communities of return, integration or relocation, and in communities where returnees are living in severe conditions, IOM provided a multi-sectoral set of activities and services through an area-based approach aimed at reducing identified barriers to returns and at increasing the likelihood of successful (re)integration of IDPs, and at the same time minimizing potential risks of tensions in communities. IOM directly engaged IDP and returnee households, communities, local authorities and other stakeholders in identifying and designing key interventions. These activities included shelter rehabilitation or reconstruction and improving access to basic services, such as water, electricity, education and health, among others. IOM also conducted interventions targeted at mitigating the risk of climate-induced displacement, in particular related to drought, water scarcity and floods. Quick impact projects (QIPs) conducted to enable access to basic service for IDPs, returnees and host communities, and therefore support conditions for sustainable return and integration.

IOM also seeks to address key needs related to employment and income generation for IDPs, returnees, host communities, and other target groups in creating conditions for successful, longer-term (re)integration and also enable communities to better absorb recent arrivals, avoid social tensions and improve the overall economic conditions in the community aiming towards durable solutions. Short-term livelihood opportunities, such as cash-for-work schemes, were complemented with individual livelihood assistance (ILA), to increase employability and income-generating capacities, as well as grants provided to small and medium businesses for job creation and economic revitalization through the Enterprise Development Fund (EDF).

In addition to addressing material conditions affecting intentions to and durability of returns, IOM provided complementary assistance based on needs of the targeted households and communities. Social cohesion issues were addressed, including through continued work on tribal engagement and reconciliation and other key activities prioritized by the communities. IOM also provided adapted legal support, protection, mental health and psychosocial support (MHPSS) and health assistance, in addition to referral to other services.

In addition to advancing efforts to support IDP movements and post-settlement (re-) integration, IOM kept directly engaging with communities, local civil society, local authorities and the Government of Iraq and Government of the Kurdistan Region of Iraq (KRI) to increase their capacities to support durable solutions and to address climate-induced displacement. Through the mechanisms gradually established to improve coordination on Durable Solutions in Iraq, notably the IOM co-led Durable Solutions Task Force (DSTF) and the Durable Solutions Technical Working Group (DSTWG).

Interventions conducted under this programme were supported by an evidence-based approach to better understand and advance durable solutions for IDPs, returnees and affected communities, and by building upon key existing research. Additional data collection, analysis and research were conducted to understand the trends on post-return dynamics, opportunities and challenges to (re)integration, in addition to targeted research on climate-induced displacement in Iraq.

CRP X involved four outcomes: (1) Improved access to pathways for durable solutions for IDPs through facilitated safe voluntary return, local integration or settlement in new locations, (2) Improved access to durable solutions in areas of return, local integration or relocation through provision of key services for formerly displaced populations and host communities, and interventions aimed at reducing risk of climate-induced displacement, (3) Government and local civil society have increased capacities to support durable solutions and address climate-induced displacement, (4) Improved understanding of migration and displacement related issues and conditions for durable solutions within the context of Iraq through research and information products, (5) Persons in need in Jeddah 1 are supported to increase resilience and improve re-integration while meeting their basic needs.

1. EVALUATION PURPOSE

The evaluation is an IOM end-cycle (final) summative evaluation to be conducted through an external firm or consultant intending to inform programme management and the donor about the effectiveness of the project. The main objective is to evaluate the program’s performance against the desired results as articulated in the project’s result framework during a period of three months from December 2022 until February 2023. Recommendations will be used at a strategic level to improve learning for future interventions.

The evaluation specific objectives aim to;

• Assess the progress of the project’s indicators against the targets, as articulated in the result-framework.
• Assess the overall project’s performance from planning, implementation and knowledge management by identifying the key strengths and areas of gaps and make the necessary recommendations for improvement.
• Document vital lessons-learned/best practices for future strategies and interventions.
• Support the use of relevant and timely contributions to organisational learning, informed decision-making processes resulting from the analysis, conclusions or recommendations as well as and accountability for results.
• Endorse IOM’s obligation on transparency and Accountability to the Affected Populations (AAP), donors and Iraqi government authorities as well as assess the effectiveness of IOM’s CFM effectiveness and the level of beneficiaries’ usage.

1. EVALUATION SCOPE

The scope should focus on all the outcomes of the CRP X as below;

1. Improved access to pathways for durable solutions for IDPs through facilitated safe voluntary return, local integration or settlement in new locations.
2. Improved access to durable solutions in areas of return, local integration or relocation through provision of key services for formerly displaced populations and host communities, and interventions aimed at reducing risk of climate-induced displacement.
3. Government and local civil society have increased capacities to support durable solutions and address climate-induced displacement.
4. Improved understanding of migration and displacement related issues and conditions for durable solutions within the context of Iraq through research and information products
5. Persons in need in Jeddah 1 are supported to increase resilience and improve re-integration while meeting their basic needs

The evaluation will exclusively be on the PRM project CRP Phase X. The primary geographical focus will be in the twelve govenorates where CRP X activities have been implemented, namely, Erbil, Sulaymaniyah (including Halabja), Dohuk, Ninewa, Diyala, Kirkuk, Salah al-Din, Anbar, Baghdad, Basra, Thi-Qar, Missan.

1. EVALUATION CRITERIA

Project’s performance should be evaluated against the evaluation criteria of relevance, coverage, effectiveness, coherence and coordination, efficiency, sustainability and impact. In total, the evaluation criteria form the basis and guidance for the evaluation suggested questions as presented below.

1. EVALUATION GUIDING QUESTIONS

Relevance

1. To what extent did the activities, outputs and outcomes remain sound and pertinent as initially intended?
2. Was the project in line with local needs and priorities?
3. To what extent was the selection of beneficiaries consistent with the project selection criteria, and did the selection criteria adequately target the populations most in need of stabilisation and revitalisation assistance?

Coverage

1. Who were the major groups in need of humanitarian assistance?
2. Of these groups, who were provided with humanitarian assistance?
3. What, if any, were the differential impacts of this humanitarian assistance on different groups or subgroups?

Effectiveness

1. What was the overall progress towards the expected results? Were there significant challenges that hindered the realisation of the planned results?
2. What were the components and delivery approaches that were most effective, and which were least effective? What needed to be done to improve the performance to achieve the expected results?
3. Did the program adequately apply ‘Do No Harm’ principles?
4. Did outputs lead to the intended outcomes/results?

Coherence

1. Were project activities and aims in line with main humanitarian aims?
2. Were project activities coordinated with other actors?

Efficiency

1. How did the following aspects contribute to the project: execution, organisation, experience of the personnel, technical expertise, administration, financial management, training, monitoring and reporting?
2. Is the project implemented according to the workplan?
3. Is the project’s progress on schedule?
4. Does the program have clear reporting lines?
5. How efficiently were the resources used to achieve the intended results in line with the implementation context?

Sustainability

1. Are structures, resources and processes in place to ensure that benefits generated by the project continue once external support ceases?
2. To what extent were relevant target groups actively involved in decision-making concerning project orientation and implementation?
3. How far was the project embedded in institutional structures that are likely to survive beyond the life of the project?

Impact

1. What are the broader effects of the project on individuals, gender, and age groups, IDPs/host-communities and institutions?
2. What are the intended/unintended positive/negative results at the macro (sector) and micro (household) levels?
3. What were the significant factors influencing the achievement of the project’s outputs and outcomes?
4. THE METHODOLOGY OF THE EVALUATION

The firm/ consultant is ultimately responsible for the development of the overall methodological approach and evaluation and is expected to propose methodologies that the firm/ consultant considers most appropriate to achieve the aims of this evaluation.

Efforts shall be exerted to safeguard the inclusivity and engagement of relevant stakeholders to bring out their voices on how they perceived the implementation of the project; notably, the returnees, IDPs, host communities, the most vulnerable conflict-affected populations, civil societies, government counterparts, community and local authorities’ leaders. Key Informants Interviews (KIIs) with representatives from the community, government authorities, and other relevant actors (if any) should be held. Similarly, interviews amd FGDs with a selected sample from the Returenees, IDPs and host-communities or beneficiaries should be conducted to assess how the project has responded according to their expectations, objectives and priorities.

The evaluation processes shall be in line with IOM Data Protection Principles[4], IOM code of conduct, Do no harm principles, UNEG norms and standards for evaluations[5]. The IOM Iraq MEAL team and the Evaluation Reference Group (ERG)[6] will provide technical support and guidance during the process. The M&E Advisor in IOM’s Regional Office in Cairo will be consulted when deemed necessary. The results of this evaluation will contribute to improving learning for future interventions.

The evaluation is expected to take place in-person and to travel to Iraq. The firm/ consultant may engage national consultants based in Iraq to support their work.

Summary of the Evaluation Methodology and Data Collection Tools

Table 01: Evaluation Methodology and Data Collection Tools

Method

Tasks

Tools required

Products

Desk study

Review project reports, M&E reports,

assessments, selection criteria, work plans and other documents.

Evaluation

rubric[7]

• Implementation strategies identification;

• Identification of progress and challenges

• Key project components implemented,

and result achieved

Key Informant Interviews

Identify and secure an interview with relevant KIs: programme staff, management, community and stakeholder’s representatives (IDPs, host-communities, governmental authorities, civil societies, and other actors). At least 50 KIIs with

relevant stakeholders

Interview Guides for the evaluation purpose, Structured questionnaire

• Qualitative data on project processes and performance about target results

• Insight on the view of problems and recommended solutions.

• Different perspectives on issues

On-site observation

Conduct field visits, observation of project activities and sites (at least one visit per activity per governorate, minimum of 60 )[8]

Observation Guides

Qualitative data/cues about project contexts/challenges as well as risks

Focus group discussion

• Al least 20 FGDs to explore stakeholder opinions and judgements towards the engagement level, processes, and project implementation.

• In-Depth information on the needs, motivations, intentions, and experiences of the group to assess how the project has responded according to their expectations

Structured questionnaire

An in-depth qualitative information

Beneficiary Interviews and surveys

Identify and secure interviews with Beneficiaries (sample for each activity should be selected based on a statistically relevant sample per the survey system, that is, a 95% confidence level and a confidence interval of 5) Around 2,000 intervews/surveys with program direct beneficiaires (half in person and half remotely).

Structured questionnaire

Quantitative information

Comparative non-beneficiary interviews and surveys

Identify and secure interviews with comparable individuals, profiled but not selected (sample for this activity should be selected based on a statistically relevant sample per the survey system, that is, a 95% confidence level and a confidence interval of 5%)

Structured questionnaire

Quantitative information

1. EVALUATION DELIVERABLES

The evaluator should produce:

1. Inception report as per the IOM template including the tools/materials/templates.
2. Sharing detailed data collection plans in time for IOM to be able to conduct quality assurance.
3. Sharing all the data collected from the different sources.
4. Briefing and debriefing meetings in addition to the routine meetings and discussions with the M&E Officer, Programme Manager/ project team, ERG, Project focal points and IOM management.
5. The final evaluation report (based on IOM template, incorporating comments and technical inputs from the reference group) with a summary of the evaluation brief (2-pager evaluation brief per IOM template), and a presentation.
6. Updates Results Framework that clearly shows the status/value of Project indicators visa-vis the targets
7. EVALUATION PROPOSED WORKPLAN

Table 02: The Evaluation proposed work plan

Activity

Responsible

Location

Start

Data Analysis

Reporting

Nov

Dec

Jan

1. Meetings: M&E team, project manager/staff/focal points, management.

IOM

Erbil/ Home-based

X

1. Desk review of project documents

Evaluation firm

Erbil/ Home-based

X

1. Inception process: Development/submission of inception report/work plan, tools for the FGD, survey and KIIs.

Evaluation firm

Erbil/ Home-based

X

1. Sharing detailed data collection plans, and updating IOM in case of changes.

Evaluation firm

Iraq – Field site.

X

1. Fieldwork – data collection

Evaluation firm

Iraq – Field site.

X

X

1. Sharing raw data

Evaluation firm

Erbil/ Home-based

X

1. Data Analysis

Evaluation firm

Home-based

X

X

1. Report writing and submission of the first draft (in IOM template)

Evaluation firm

Home-based

X

1. Incorporate feedback and submit a final report with a two-pager evaluation brief (as per IOM template)

Evaluation firm

Erbil/ Home-based

X

1. Final report presentation

Evaluation firm

Erbil/ Home-based

X

1. EVALUATION BUDGET AND DISBURSEMENT

The payment terms shall be issued per the terms and condition of the Purchase Order (PO) based on the disbursement schedule below;

• Satisfactory inception report submission – 30%
• Submission of first draft of the evaluation report together with raw data – 30%
• Satisfactory final report with relevant annexes – 40%

The final payment shall be issued not less than 30 days upon (1) the completion of the work, (2) receive of the final original invoice and (3) receive of the final evaluation report and summary of the evaluation brief following the incorporation of feedback from the IOM ERG.

1. REQUIREMENTS

An international consultancy firm with valid registrations, and it should have legal registration paperwork in Iraq, and formal access to the locations of the programme implementation.

The selected firm/ consultant should possess the following minimum qualifications as follows:

Table 03:Qualifications and Experience

Qualifications and experience

Academic skills

Master’s degree in advanced applied research/evaluation methods/ economics, business, or any related academic discipline or an affiliation with a research institution or a university, holding a PhD, or being in pursuit of a PhD in a relevant field is an advantage.

Previous Experience

• 5 of years evaluating humanitarian programmes
• At least two evaluation contracts of similar value, nature and complexity implemented over the last three years or more.
• Strong background in monitoring and evaluation techniques and ideal experience in conflict-affected countries.
• Conversant with the context in Iraq, other countries in the Middle East or MENA region.
• Experience in developing and implementing Evaluations with the UN, International NGOs or donors.
• Familiarity with the OECD/DAC and UNEG evaluation framework.
• Excellent knowledge and experience in survey design, implementation of surveys and statistical data analysis.
• Excellent analytical, communication, writing and presentation skills in English.
• Ability to analyse complex intervention.

Accountability

• Creates a respectful office environment free of harassment and retaliation and promotes the prevention of sexual exploitation and abuse (PSEA).
• Accepts and gives constructive criticism.
• Follows all relevant procedures, processes, and policies related to the organisational principles.
• Meets deadline, cost, and quality requirements for outputs.
• Monitors own work to correct errors or incorporate inputs.
• Takes responsibility for meeting commitments and for any shortcomings.

Orientation

• Identifies the immediate and peripheral programme staff of own work.
• Establishes and maintains productive working relationships with staff.
• Identifies and monitors changes in the needs of evaluation, including donors, governments and project beneficiaries.
• Keeps staff/managers informed of developments and setbacks related to the evaluation.

1. SUBMISSION OF APPLICATION/EXPRESSION OF INTEREST

The interested firm/ consultant should submit a technical proposal with a detailed evaluation methodology, indicative work plan, and the overall approach to the evaluation and an all-inclusive budget proposal no later than September 5th, 2022. The submission of proposals (technical and financial) and/or related questions should be directed via procurement email to: [email protected]

The submission should include the followings:

• Company/ Consultant profile including a history of similar projects (if applicable);
• A cover letter;
• CV and biographies of independent consultant/consulting firm and key assessment team members (if any);
• References for each evaluation team member or the firm;
• An example of a recent evaluation report.

Important Note: When evaluating the competing applicants, IOM will consider the written qualifications/capability, financial offer, the information provided by the applicants, and any other information obtained by IOM through its research.

IOM reserves the right to change the calendar of events or revise any parts of the requirements of the evaluation at any time.

[1] IOM Iraq Masterlist Report: https://iraqdtm.iom.int/images/MasterList/20224264632942_DTM_125_Report_January_March_2022.pdf.

[2] IOM, Water quantity and water quality in central and south Iraq: a preliminary assessment in the context of displacement risk, July 2020.

[3] Currently Return and Recovery Unit.

[4] IOM Data Protection Manual, https://publications.iom.int/books/iom-data-protection-manual.

[5] UNEG norms and standards for evaluation: http://www.uneval.org/document/guidance-documents.

[6] The ERG is a technical advisory group comprising of members representing different constituencies who have an interest in the evaluation outcomes. They are chosen by their relevant expertise in Monitoring and Evaluation, Research methods, Project Management, Coordination, livelihood and thematic representation.

[7] Evaluative rubric is a qualitative data assessment tool which involves articulating things that matter in the initiative being evaluated in line with the performance quality standard in project activities, their delivery strategies and resultant products or services predefined according to the evaluation purpose. See also: http://carla.umn.edu/assessment/vac/improvement/p_4.html.

[8] At least three activities per governorate.

How to apply

The Proposals must be delivered through email [[email protected]] on or before [September 5th, 2022, by 23.59 hours]. The email size not exceeding 9MB as IOM server will reject/return automatically any email exceeding this limit. No late proposal shall be accepted.

Email subject: RFP No.: 16442 / Final Evaluation of Community Stabilization Program (CRP) – Phase X