Consultant/s – External Evaluator for the BRYCA Project (Central Asia)

ERIM is looking for a consultant/team of consultants to work on final external evaluation of the EU funded project: BRYCA – Building Resistance in Youth in Central Asia to the influence of illegal hate speech and misinformation online and on social media.

The position is Home-based, with potential for travel.

Read the full terms of reference here

About ERIM

ERIM is a French-registered non-profit, non-governmental organisation with an annual turnover of 6,8 million and funding support from multiple donors, including the European Union, United Nations, and individual governments. Our work focuses on promoting human rights, good governance and democracy. We do this through supporting and working with activists and civil society organisations directly, by seeking to improve education, strengthen independent media and reduce conflict.

Context and description of the project

Title: BRYCA – Building Resistance in Youth in Central Asia to the influence of illegal hate speech and misinformation online and on social media

Duration: 36 months (30 November 2019 – 29 November 2022)

Location: Kazakhstan, Kyrgyzstan, Tajikistan, and Uzbekistan

Partners: Media Support Center Foundation (Kyrgyzstan), MediaNet International Centre for Journalism (Kazakhstan), Modern journalism development center (Uzbekistan), and Public Organization “Gurdofarid” (Tajikistan)

Budget: 1,800,000 EUR

Donor: European Commission

BRYCA project aims to build resilience to disinformation and hate speech in Central Asia’s most vulnerable populations, by promoting improved understanding of the rights of “others” in the context of hate speech and a more critical approach to information to help ensure resistance toward fake news, propaganda and disinformation.

The project aims to achieve two outcomes:

• Outcome 1 – Vulnerable youth in remote areas of Central Asia participate in online and mobile educational games that build their critical thinking and counter disinformation and illegal hate speech.
• Outcome 2 – Access to online information and educational tools that counter disinformation and illegal hate speech is expanded, promoting critical thinking and the use of reliable independent sources across Central Asia.

Target groups:

Youth in Central Asia (18 – 35 in Uzbekistan; 12 – 35 in Kazakhstan, Kyrgyzstan, Tajikistan), particularly vulnerable youth in remote and marginalised rural and urban areas; teachers, educators and youth leaders/community workers working to improve critical thinking, combat hate speech and counter disinformation, and increase access to information in local communities; partner CSOs in Uzbekistan, Kazakhstan, Kyrgyzstan, Tajikistan, at least 35% of girls and young women.

The project is implemented across 36 months and included the following activities:

A1.1.1: Monitoring of hate speech and disinformation programs in each country.

A1.1.2: Team training in media monitoring.

A1.2.1: Development of games software and system.

A1.2.2: Team training in Game management.

A1.2.3: Ongoing implementation of games and awarding of prizes.

A1.3.1: Development and implementation of marketing campaigns for different channels.

A1.3.2: Implementation of interactive MIL-related and youth-focused activities on the game platform.

A2.1.1: Harmonisation of media literacy approach and training.

A2.1.2: Media Sabak expansion. A2.2: Media literacy capacity building for educators/youth leaders/community workers and local media literacy training for marginalised youth.

Objectives of the external evaluation

The final evaluation’s purpose is to provide an overall external assessment of the performance and the results of the project. The evaluation will focus on assessing the project intervention’s relevancy, effectiveness, efficiency, sustainability and, where possible, impact and sustainability.

The final evaluation will cover the 36 months of the project and all the activities carried out in the context of the project.

The scope of work for consultancy will include, but may not be limited to:

• Review all relevant sources of information, such as the project documents, logical framework, monthly reports, annual progress reports, QLEVER platform analytics, internal evaluation report and any other materials that the evaluator considers useful for this evidence-based assessment;
• Develop an evaluation matrix with evaluation criteria, the related evaluation questions (and, where needed, sub-questions), the data sources required to answer the questions, the data collection, and data analysis methods. The evaluation should be based on the five criteria of relevance, effectiveness, efficiency, impact, and sustainability.
• Relevance: Appropriateness of the project strategy to the actual needs and priorities of the target groups/beneficiaries taking into the account the circumstances. Is the project strategy appropriate and logical and does it work directly toward attaining the objectives?
• Efficiency: Measures the outputs – qualitative and quantitative – in relation to the inputs. How well are the means/inputs and activities converted into outputs? Are the activities implemented in line with the plans? Are they implemented and the outputs delivered in a cost-efficient manner?
• Effectiveness: Contribution of the project results to the achievement of the project objectives. To what extent are the specific objectives achieved / are likely to be achieved? What are the major factors influencing the achievement or non-achievement of the objectives?
• Impact: The positive and negative changes produced by the project, directly or indirectly, intended or unintended. Main effects resulting from the project on the local social, economic, environmental and other development indicators. What is the project’s likely contribution to the overall objective?
• Sustainability: Analyze to what extent the effects and results of the project are likely to last beyond its implementation period. What was put in place by the project to ensure the sustainability of the expected project outcomes? Collect lessons learned and good practices and express recommendations for the partners and stakeholders.

The evaluation must provide evidence-based information that is credible, reliable, and useful. The evaluator is expected to follow a participatory and consultative approach ensuring close engagement with the local organizations, experts, project consultants, the project team, BRYCA management team, final beneficiaries, and other key stakeholders;

• Prepare an evaluation report providing descriptive overviews, laying out the facts and lessons learned, providing conclusions and recommendations, and finalizing the report upon the feedback from the project team.
• Conduct a meeting with the project partners to validate the evaluation findings and recommendations.

Methodology

The consultant is requested to propose a methodological approach but the evaluation should at the minimum include desk research, semi-structured interviews, an inception report, a draft and final evaluation report as well as a ppt presentation with key findings.

• Desk research: should include a review of all the material and documentation developed within the project, the reports of local teams, beneficiaries and consultants, the tools created or adapted, including the monitoring and evaluation ones, etc. The consultant(s) should collect all quantitative data from the logframe and QLEVER platform analytics (list of activities, number of participants, number of supports, etc.).
• Key Informant Interviews/Focus group discussions: The consultant(s) should carry out interviews/group discussions with ERIM project managers and all partners as well as representatives of the target groups/beneficiaries.

Deliverables

• 1 inception meeting report, including agreed evaluation approach, outline of the report, timeline, workplan etc (12 August 2022);
• 1 draft report/first findings (before 15 November 2022) in English of up to 30 pages, including a table of contents, a brief introduction, the objectives and the methodology, and evaluation of the project based on the criteria mentioned above (relevance, coherence, efficiency, effectiveness, impact, sustainability), observations, lessons learned and conclusions as well as a list of recommendations and appendix (e.g. acronyms, list of interviewees, ToRs, methodology, schedule, etc.).
• 1 final report in English including recommendations and a presentation to the consortium (up to 30 pages) before 25th of December 2022.
• 1 presentation: The consultant(s) will finally make a 1h30 presentation to the project partners, reviewing the main conclusions and recommendations of the evaluation. This presentation will then be used by ERIM and its partners to present the results to external stakeholders.

Organisation and timeline

Overall, the evaluation requires 30 working days within the period of August – December 2022.

Provisional calendar (to be discussed at the inception meeting):

Publication of ToR 16 June 2022

Selection of the Evaluator 18-29 July 2022

Contracting the Evaluator 1-5 August 2022

Inception Meeting 8-9 August 2022

Inception Meeting Report 12 August 2022

Desk research 10-31 August 2022

Interviews and focus groups 1 Sept – 30 Oct 2022

1st Draft Report 15 November 2022

Final Report and oral presentation to the project team 25 December 2022

Budget

A maximum budget of 13,000€ (Euro) has been allocated for this evaluation activity. It is recalled that the travel and accommodation expenses for participation in the meetings and activities foreseen in the work plan will be covered by this budget.

Data protection

In carrying out the evaluation of this project, the evaluator must respect the following confidentiality rules:

• All data relating to this project are confidential.
• The information is only accessible to those whose access is authorized.
• The ERIM project managers authorize the external expert to list the data in paper or electronic media intended to carry out the activities entrusted to him. However, the ERIM project managers does not give the evaluator any right to use, disclose or publish the data or results obtained in the course of his/her activity.