Consultancy for Real Time Evaluation of the Armenia Population Movement Operation

Summary
1.1. Purpose: The real time evaluation (RTE) will assess the ongoing IFRC response to the Armenia Population Movement to inform the continued response as well as future global emergency relief operations.
1.2. Commissioners: The Commissioner will be appointed by the Evaluation Management Team.
1.3. Audience: The RTE will be used by the Armenian Red Cross Society (ARCS), the IFRC South CaucasusCluster Delegation, the IFRC Regional Office for Europe & Central Asia (ROECA), and at the international headquarters in Geneva; it will also inform Partner National Societies working with ARCS and contributing to the Federation-wide funding of the Emergency Appeal, namely Austrian Red Cross, Italian Red Cross, Monaco Red Cross and Swiss Red Cross.
1.4. Duration of consultancy: approximately 30 working days (with approx. 10 working days in the field),
1.5. Estimated dates of consultancy: February – mid April 2024.
1.6. Location of consultancy: remotely and 10 days in Armenia for field visits and interviews with ARCS staff and volunteers, as well as with ICRC, IFRC Country Delegation and Partner National Societies (PNS), external stakeholders and affected persons (as appropriate). Primarily, the RTE will geographically focus on the areas of the capital (Yerevan) and the districts with the highest density of affected persons (at the time of writing, the regions [Marzes] of Ararat, Kotayk, and Armavir)

The situation in Armenia has escalated since September 19, 2023, with a significant influx of population from Karabakh seeking shelter due to military hostilities. The number of displaced people reached 101,848 by 20 October 2023, prompting the Government of Armenia to request international assistance through the Union Civil Protection Mechanism (UCPM). The Federation-wide Emergency Appeal for CHF 20 million, out of which CHF 15 million is through Secretariat, to address the needs of 90,000 people was issued. After the initial allotment of CHF 498,752 from the IFRC Disaster Response Emergency Fund (DREF), a second allocationincreased the DREF to CHF 1 million, subsequently transitioning to an Emergency Appeal (EA). Currently, over 6 million CHF has been raised under the IFRC Secretariat-component of the EA.

The Armenian Government has taken effective measures to streamline humanitarian efforts, establishing reception centers and closely collaborating with organizations including the ARCS to provide essential services. The response aims to address immediate and ongoing needs, particularly focusing on vulnerable groups such as women, children, older individuals, and those with disabilities. A comprehensive needs assessment had been completed with preliminary findings, emphasizing immediate priorities such as shelter, food, essential items, mental health support, and psychosocial care. In addition to addressing immediate needs, long-term support for livelihoods has been introduced to sustain daily life over an extended period.

Armenia is one of 14 countries globally piloting the IFRC New Way of Working (NWoW), emphasizing effective coordination, resource-sharing, and impact amplification. In the context of NWoW, ARCS is expected to achieve four key deliverables. 1) Shared context, situational and needs analysis: Since 2023, ARCS has been submitting unified planning that includes a shared context, situational analysis, and needs assessment. 2) Multi-Year Country Support Plan: ARCS is responsible for creating a multi-year country support plan. The Unified Plan 2IFRC – ToR RTE Armenia Population Movement Publicoutlines the humanitarian and development goals and strategies for the country. 3) Common Accountability Framework: In March 2023, ARCS developed a common accountability framework, which is currently in the testing phase. This framework likely sets standards and mechanisms for ensuring accountability in humanitarian and development activities. 4) Harmonized Resource Mobilization: ARCS plans to harmonize resource mobilization efforts. This involves information for resource mobilization during emergency operations.

Evaluation purpose & scope

This RTE is being carried out with the purpose to evaluate the response operation to date and will take stock of lessons, in order to learn and to inform the ongoing response and recovery activities, and to provide recommendations for the ARCS and IFRC systems in place. Given the pilot character of the NWoW setup, IFRCwide recommendation will be taken into account.

The RTE will not at this point focus on the impact of the operation, nor will it look into the details of the work of each sector of the operation, but will rather focus on the overall timeliness, efficiency, effectiveness, relevance and connectedness of the work done so far. The RTE team will focus on an overview of the operational areas, to see what has worked well, what has not worked well and what lessons can be learned for the future of the operation, including for the (early) recovery work.

Geographically, the RTE will focus on the areas of the capital (Yerevan) and the districts with the highest density of affected persons (at the time of writing, the regions [Marzes] of Ararat, Kotayk, and Armavir). Although other regions can be included, depending on the time available, the focus of the evaluation and the access to those areas.

The RTE is expected to take place in February-mid April2024, lasting 30 working days with 30% of the time of the evaluation team to be spent in the field (in February 2024).

Evaluation criteria and key questions

The evaluation will focus on three objectives as below:

Objective 1: To focus on the combined Movement response (ARCS, IFRC, Partner NS and ICRC), and the effectiveness and efficiency of Movement coordination and collaboration, from the onset of the operation up to the time the evaluation team arrives in Armenia:

1. To what extent is an effective management structure in place [timeliness, effectiveness and efficiency of internal management and response], providing direction, clarity and wellcommunicated decision-making, within IFRC and ARCS?

2. Are there any specific recommendations to improve coordination levels?

3. How well does the Movement coordination setup correspond to SMCC and S2.0 approach? – To what extent were roles and responsibilities of Movement partners in country fulfilled vis-à-vis the respective guidance?

4. To what extent has the response demonstrated learning from previous responses of population movement from Karabakh in 2020 and 2022?

5. Are constraints and risks regularly identified and analyzed, and plans adapted accordingly?

6. How well are the M&E and learning processes used in the response and can they be improved?

Objective 2: To evaluate the connectedness of the operational specifically with systematic longer-term approaches and initiatives to strengthen the National Society systems, capacities and structure, both at the HQ and branch level: 3 IFRC – ToR RTE Armenia Population Movement Public

1. Have the capacity strengthening initiatives / activities provided within the operation, been affective?

2. How well is the operation aligned with the longer-term NSD approaches and established structures, particularly related to Unified Planning process and New Way of Working (NWoW), integrating the EA into longer-term NSD plans?

3. How does the systems’ agility that is in place from the outset, adapt as the context / needs evolve and how they support the assessment, planning and follow up of response delivery?

4. What organizational systems strengths can we build on, and what areas of improvement need to be addressed, specifically on ARCS and IFRC side?

Objective 3: To examine the relevance and appropriateness of the assistance provided to the affected population within the given context, building on organizational systems strengths, and to evaluate areas of improvement which need to be addressed on both ARCS and IFRC side:

1. Do the programme objectives correspond with identified needs? Do the prioritized intervention choices meet the most urgent needs of the affected population?

2. Did the response adapt to changes in need, capacities and context?

3. Have protection concerns been adequately addressed in the design of the response?

4. How can the provision of humanitarian assistance be improved to better respond to the needs, vulnerabilities and capacities of girls, boys, women, men, persons with disability and marginalized groups?

5. To what extent is assistance being delivered in a way that is cost effective?

Evaluation methodology & process

The evaluation will be managed according to the draft IFRC Real Time Evaluation Guide and Procedures. The methodology will adhere to the draft IFRC Management Policy for Evaluations, with particular attention to the processes upholding the standards of how evaluations should be planned, managed, conducted, and utilized. An IFRC / ARCS Evaluation Management Team (EMT) will oversee the evaluation.

The RTE team will consist of 3-4 people: 1 external evaluator, and 1-2 internal IFRC / Partner National Society evaluators, as well as an ARCS evaluator.

The specific evaluation methodology will be detailed in close consultation between the RTE team and IFRC evaluation management team, but will draw upon the following primary methods:

1. Desktop review of operation background documents, relevant organizational background and history and any relevant sources of secondary data.

2. Field visits/observations to selected sites such as Yerevan and surrounding areas, Kotayk, Ararat and Armavir.

3. Key informant interviews including those from ARCS, IFRC and Stakeholders.

4. Focus group discussions.

Evaluation deliverables & illustrative timeline

The following will be developed and delivered in line with the draft IFRC Real Time Evaluation Guide and Procedures:

• Evaluation work plan/ inception report will be submitted prior to the RTE team being deployed to the field.

• Validation workshop in the field to take place prior to the RTE team leaving the field and verbal debriefs with key stakeholders.

• Draft RTE report will be submitted within two weeks of the consultants’ return from the field.

• Final RTE report will be submitted one week after receipt of the consolidated feedback from IFRC and meet the requirements outlined in the IFRC RTE Guide and Procedures.

All products arising from this evaluation will be owned by the Federation. The evaluators will not be allowed, without prior authorization in writing, to present any of the analytical results as his or her own work or to make use of the evaluation results for private publication purposes.

Evaluation quality & ethics

The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of the people and communities involved and to ensure that the evaluation is technically accurate and reliable, is conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the evaluation team should adhere to the evaluation standards and applicable practices outlined in the IFRC Framework for Evaluation and respect the Red Cross Red Crescent 7 Fundamental Principles. https://www.ifrc.org/who-we-are/international-red-cross-and-red-crescent-movement/fundamental-principles

Qualifications

The management group will select the members of the evaluation team, which will comprise three to four persons, including a team leader (external consultant), up to two international staff and (IFRC/PNS), and one national team member. 5 IFRC – ToR RTE Armenia Population Movement Public Team member will collectively have the following skill and experience; this set of skills also provide the basis for the selection of the external evaluation consultant:

1) Demonstrable experience in leading evaluations of humanitarian programs responding to major disasters / crises, with specific experience in RTEs preferred.

2) Knowledge of strategic and operational management of humanitarian operations and proven ability to provide strategic recommendations to key stakeholders.

3) One team member with a good knowledge on RCRC Movement coordination mechanisms (SMCC, Seville 2.0).

1. One team member with knowledge / experience in Population Movement Operations.

5) One team member with strong National Society Strengthening background.

6) Preferably at least one team member with knowledge / experience in Southern Caucasus, or in other post-soviet countries.

7) Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner.

8) Experience in qualitative data collection and data analysis techniques, especially in emergency operations.

9) Knowledge and experience working with the Red Cross Red Crescent Movement preferred 10) Demonstrated capacity to work both independently and as part of a multi-discipline, multi-national team.

11) Excellent English writing and presentation skills in English, with relevant writing samples of similar evaluation reports; Russian or Arminian language skills are an asset