EXTERNAL FINAL EVALUATION

TERMS OF REFERENCE

EXTERNAL FINAL EVALUATION

Project: ‘Resilient Civil Society and Media Respond to Ukraine War’

Background

Basic information about PIN
People in Need is a Czech non-governmental, non-profit organization active all over the world. People in Need has been helping in crisis-affected areas and promoting respect for human rights since 1992.
Throughout the 30 years of its existence, People in Need has become one of the biggest non-profit organizations in Central Europe. In addition to humanitarian aid and human rights, it now also targets education and helps people living in social exclusion.
Basic information about the project
The consortium, led by People in Need (PIN), formed together with two partner organisations The Netherlands Helsinki Committee (NHC) and Prague Civil Society Center (PCSC), implement the 24-month-long action Resilient Civil Society and Media Respond to Ukraine War funded by the European Commission (DG NEAR). With the overall aim of contributing to a resilient civil society and independent media in and from Ukraine and EaP+ region, through empowerment of civil society organisations (CSOs)and relevant actors, the project aims to respond to the evolving needs of those affected by the Ukraine war. The Action is implemented in Armenia, Azerbaijan, Georgia, Moldova, and Ukraine. The action also targets the exiled CSOs and actors in and from other EaP+ countries.
The project has a budget of EUR 9,600,000.00 and is implemented across the Eastern Partnership region with the duration of implementation between 1 June 2022 and 31 May 2024.
The Project aims at the following impact – To contribute to a resilient civil society and independent media in and from Ukraine and EaP+ region empowered to respond to the evolving needs of those affected by the Ukraine war

Objective and Scope of the Evaluation

The purpose of this evaluation is to assess the Consortium´s performance and delivery of the Action under the DG NEAR-funded project according to OECD evaluation criteria. The objective is to evaluate the project impact and help the Consortium to improve its future programming through lessons learned and good practices generated through this project.
Objective(s):

1. To evaluate the achievements of the Action in relation to the goal, objectives and results defined in project proposal and ‘Logical Framework’;
2. To evaluate impact of the project on targeted groups and communities, as well as overall effectiveness and appropriateness of the project activities, challenges in implementation of the activities, and potential for sustained outcomes; as well as to identify external and project-related factors (if any) limiting the achievements of the project;
3. To identify good practices, lessons learned, strengths, and challenges in the design of project activities, and their implementation, coordination and monitoring.

Scope and Content:
The primary purpose of the final evaluation is to assess the results of the intervention and validate the achievement of the expected outcome “Civil society and independent media have the skills, operational capacities and financial resources to safely respond to the needs of affected population”, and its outputs. Drawing insights from the whole period of project implementation, the evaluation aims to assess the impact of the complex integrated intervention, identify lessons learned, good practices, and gaps in the implementation process, external communication and visibility, and further also to evaluate the effectiveness of project management.
Stakeholders:
The project implementation team; the consortium partner organizations; targeted mid-level and experienced civil society organizations, emerging CSO from all targeted countries, activist groups and human rights defenders, displaced CSOs from Ukraine, targeted media organizations and journalists, youth groups, refugees from Ukraine, internally displaced persons (IDPs) and forcibly displaced people within Ukraine and Armenia and other final beneficiaries, that is recipients of assistance.
Geographical scope:
Specific locations of the project requiring in-country and field presence:

• Georgia (Tbilisi and surrounding, visit suggested to 5-10 CSOs and 3-5 media organizations, including interviews/focus group discussions (FGDs) or similar with final beneficiaries/recipients of assistance)
• Moldova (visit suggested to 5 CSOs and 3-5 media organizations, including interviews/FGDs or similar with final beneficiaries/recipients of assistance)

Period:
The project is in its final phase, and mostly only monitoring activities are ongoing. This evaluation should cover the whole project period.

Key evaluation questions

Proposed key evaluation questions clustered by the DAC criteria are available at the full ToR available through the link below.

Methodology

The final design of the evaluation process, including a detailed description of its methodology and the methods to be applied, will be presented by the evaluator/evaluation team in the Inception Report, along with drafts of the data collection instruments.

Data collection methods should combine desk review, interviews and focus group discussions with relevant project stakeholders including decision-makers, international partners, donors and project beneficiaries alongside with the project final beneficiaries including field visits to the target locations and on-site observations. Additionally, the evaluator/evaluation team should describe the approach on conducting online validation workshop.

For evaluation questions related to effectiveness, the evaluator/evaluation team shall consider identifying and substantiating a number of outcomes, potentially utilizing methodologies such as Outcome Harvesting.

In regard to the desk review, the following available project documents should be analyzed:

• Project proposal;
• Interim Report to Donor;
• FSTP Database;
• Capacity Building Database;
• Monitoring visits Database and reports;
• Communication reports;
• PIN Country Programmes’ and Implementing partners reports;
• Sample from FSTP recipients Interim and Final reports with annexes;
• Mid-term evaluation report;
• Documented changes and guidance on project tools (a sample of meeting minutes);
• Monitoring visit records, participant and facilitator feedback analysis; including monitoring reports.
• Other MEAL tools (OCAT, Coaches’ Questionnaire, CFRM).

The evaluation requires field research with expected physical presence in at least two of the participating countries, that is Moldova and Georgia, the others will be performed through remote research methods.

Specific Tasks

• Review of project documents;
• Review of project logic and targets;
• Drafting an Inception Report with methodology, data collection instruments and evaluation matrix;
• Meetings and interviews with project staff and relevant stakeholders;
• Review of EU strategy and civil society roadmaps for each target country;
• Provision of recommendations for future programming including lessons learnt and good practices;
• Preparation of evaluation documents (draft report, final report);
• Conducting a validation workshop with key stakeholders.

Expected Deliverables

The evaluation is expected to last 60 working days (non-consecutive days can be acceptable). The expected deliverables are:
Expected start of evaluation the 1st of April 2024.
Deliverables:

• 15th – 30th of April 2024: Desk review of project documentation
• 19th of April 2024: Inception report (draft) (overall design including evaluation matrix, methodology, data collection instruments) and Evaluation Action Plan
• 31st of April 2024: Inception report + action plan (final, after Consortium comments)
• 1st – 17th of May: Field data collection (at least in 2 countries)
• 20th – 31st of May: Online data collection (related to other project activities)
• By 25th of June 2024: Validation Workshop for Presentation of preliminary findings and developing and validating of joint recommendations (for Consortium partners)
• 12th of July 2024: First draft evaluation report
• 31st of July 2024: Incorporation of comments by Consortium and revision
• 31st of July 2024: Final evaluation report submission (with incorporated feedback)

Timeline can be adjusted following the approval of the Inception Report.

Evaluation Report

The desired structure for the Inception Report and the Evaluation Report can be found in the linked full ToR.

Duration

The evaluation is expected to start on the 1st of April 2024 and end on 31st of July 2024. The evaluation is expected to last a total of 60 working days (it is not mandatory that these days are consecutive).

Logistics

Project core staff will be introduced to the evaluator and will be responsible to provide necessary information related to the project. In-country support will be ensured for organizing meetings, interviews, focus group discussions (FDGs), etc. with relevant stakeholders.
The evaluator is responsible for all logistics arrangements, such as accommodation, local transportation during the field visit.

Qualification requirements

Qualification and eligibility criteria:
Mandatory proven proficiency in English (C1 level is required for non-native speakers) and in Ukrainian or Russian (B1 level is required for non-native speakers): the ability of at least one evaluation team member to interact and conduct interviews in either Ukrainian or Russian is mandatory formal requirement to submit the evaluation offer.
Evaluation criteria:

1. Technical aspect: Quality of an outline methodology presented by Consultant (Sub-total 60 points)
   1. Action plan with realistic deadlines. Should go in line with ToR requirements (ability to meet proposed deadlines; deadline is tentative.) (15 points)
   2. Key considerations and proposed methodology outline keeping the evaluation of EaP + civil society sector in consideration (25 points)
   3. Ensure the participatory approach in evaluation design and data collection for with sampling strategy or other alternative approach (20 points)
2. Previous experience in evaluating EC-funded projects on civil society and familiarity with EC-funded projects´ guidelines and regulations (including General and Specific Conditions) (Sub-total 20 points)
   1. Quality level of at least two samples of reports (or any alternative form of demonstration of quality of past deliverables) from previous projects relevant to this evaluation study, preferably EC-funded projects on civil society in EaP+ countries (5 points for each report x 2 = 10 points)
   2. Letters of reference from at least two previous clients (5 points for each reference x 2 = 10 points)
3. CV quality and competences of staff performing the task (years of experience in evaluations, type and scope of evaluations conducted, courses and studies completed relevant to the field of evaluation); including experience of work in the context, that is EaP+ region (Sub-total 20 points)

Total score: 100 points
First round (documents review) and second round (interview); prerequisite for the interview is to successfully pass the 1st round of evaluation; 3 candidates with the highest score will be invited for the interview.

Payment modalities

Maximum budget available for evaluation: 20,000 EUR.
Milestones:
40% After the Inception report has been submitted and approved by PIN
20% After the Workshop for presentation or preliminary findings
40% After the final version of the final evaluation report has been submitted and approved by PIN

PIN reserves the right to deduct up to 0.5% of the total contract price for each day’s delay in meeting the deliverables specified. This deduction shall be applied to the last payment of 40% of the contract price.

You can find the full version of ToR by clicking on this link.