Maldives PFEM Final Evaluation Consultant

Terms of reference: Final Evaluation of Protecting Freedom of Expression in the Maldives

(March 2022 – May 2024)

Terms of Reference

Final Evaluation of Protecting Freedom of Expression in the Maldives (PFEM)

Internews overview

Internews is an international non-profit organization whose mission is to empower local civil society worldwide to give people the news and information they need, the ability to connect and the means to make their voices heard. For more information about the organization please visit: www.internews.com

Project overview

As the Maldives adopts stronger systems of democratic governance, it is critical that people in the country have access to quality, impartial and trusted information to make informed decisions about their lives and hold power to account. Internews Europe (Internews) and co-applicants Maldives Journalist Association (MJA) and the International Federation of Journalists (IFJ) propose a 27-month project, “Protecting Freedom of Expression in the Maldives” (PFEM).

Overall objective:

The overall objective of PFEM is to enhance democracy and human rights in the Maldives, through promoting and protecting Freedom of Expression (FoE), with a specific focus on media freedom and development.

Specific objectives (SO):

1. Journalists and media outlets can effectively address hate speech, fake news and disinformation especially transmitted via social media; and audiences have access to quality fact-checked information.
2. Journalists are better protected against threats to digital, physical and psychosocial safety.
3. Improved professionalisation and support to journalists and media outlets to improve the quality of journalism in the Maldives.

The PFEM project is designed based on the following methodology:

Information Ecosystem Assessment: Internews’ Information Ecosystem Assessment methodology underpins activities in all, informing programming through focused and incisive analyses. The initial IEA exercise will be conducted during the inception phase and will provide a continually evolving framework which is informed through ongoing due diligence processes; project-based monitoring, evaluation and learning; and regular consultation with actors on the ground as well as the broader donor/media development community. This allows Internews to understand specific barriers to freedom of expression and information access, including for (and with a particular focus on) women and socially excluded groups, and to identify and select strategic partners and approaches that will maximise impact. To respond effectively to needs identified through the baseline IEA, Internews also maintains a sufficient level of in-house technical and global expertise, regionally specific knowledge and management response capacity to ensure the Action can adapt to evolving circumstances.

Holistic approach to capacity building: Key actors in the media and information sphere are provided with capacity building interventions drawing upon the related activities detailed in the previous section. As well as dedicated safety and security trainings and activities, safety and security of partners is mainstreamed through all programmatic activities wherever possible in order to secure their resilience towards hostile threats and other changes in their context. Over the past year and a half, Internews has implemented a flexible approach to implementing trainings and capacity building with the onset of COVID-19. The ongoing COVID-19 situation will be continuously monitored and where needed, trainings can take place online and in modular formats to allow for the highest participation and engagement rates possible. The high internet penetration rate of the Maldives will allow all participants to easily participate in online trainings.

Partnership approach: Internews is proposing a capacity building and development intervention, drawing on international expertise and best practice, which will deliver impact primarily through building the capacity of, key leading media outlets, as well as journalists, citizen journalists, social media content producers and human rights defenders: to ensure and expand access to information and continued freedom of expression. Internews’ overall approach is centred on beneficiary-led development – as such, Internews does not seek to impose changes to media partners’ mission or purpose, or indeed build new media partners from the top down. Rather, Internews aims to provide existing media outlets with the tools and expertise to maximise reach and impact within their defined mission – as reflected here.

Organisational capacity strengthening: IFJ will conduct baseline capacity assessment of MJA organisations using an Organisational Capacity Assessment tool. The OCA uses an ordinal scale to score components of organisational capacity. Scores are described and benchmarked to ensure SMART indicators are developed and utilised wherever possible. In-depth interviews with partner representatives will inform the OCA. The OCA tool acts both as a need’s assessment and analysis tool as well as a monitoring and evaluative instrument. The scores obtained inform the baseline data related to partners’ capacity and the assessment will be repeated during the close out phase of the project to produce data about changes in capacity. The results also form the basis of the co-creation of tailored capacity building plans for partners. As a result of this process, all capacity building is based on partner needs, objectives and capacities, and is driven by the partners wherever possible.

Evaluation Purpose and Objectives

The purpose of the final external evaluation is to assess the overall performance and impact of our work and inform learning. The evaluation is commissioned by the Project Manager for Maldives with the support of Internews’ MEL staff. Internews employs international best practices, Development Assistance Committee of the Economic Cooperation and Development (OECD-DAC) evaluation criteria, one of the most widely prominent and adopted criteria of aid effectiveness. The criteria will be used to evaluate various aspects of the interventions including relevance, coherence, effectiveness, efficiency, impact, and sustainability. Internews will collaborate with the selected candidate to collect information about the usefulness and trustworthiness of the content produced by the project. This is to understand whether stories produced by the project are contributing to the objective of enhancing democracy and human rights in the Maldives, through promoting and protecting Freedom of Expression. This evaluation has the following principal tasks, which must be assessed in the context of the political and environmental challenges that the project faced during implementation:

· Evaluate the project’s performance with respect to relevance, coherence, effectiveness, efficiency, impact, and sustainability, in alignment with the OECD-DAC evaluation criteria

· Derive insights and best practices from the project,

· Offering key recommendations to improve future interventions.

Evaluation Scope and Methodology

The evaluation covers the project implementation period from March 2022 to May 2024 and should encompass the entire scope of the project. Ideally, the work will be carried out in Maldives by a local national evaluator with language skills.

The evaluation should employ OECD/DAC criteria for assessing aid effectiveness, and a gender and inclusion lens should be applied throughout the evaluation process. The questions provided below can serve as a guide for the project’s evaluation and are open to discussion and agreement during the evaluation design phase.

1. Relevance:

• 1.1. Did the theory of change hold true?
• 1.2. Has the project responded to the needs of the target groups and continued to do so when circumstances have changed? Has the project responded to the different needs of the target group?
• 1.3. To what extent were lessons learned from the project challenges and successes and how were they applied to improve and adjust implementation?

2. Coherence:

• 2.1. Internal coherence: To what extent is the project in alignment with Internews policies, including its global Gender Equality and Women’s empowerment strategy “Towards a Gender Transformative Organisation” and Human-rights based approach, and with other interventions implemented by Internews in the region?
• 2.2. External coherence: To what extent is the project in alignment with national and international development priorities and policies, including the 2030 Agenda/Sustainable Development Goals (SDG), particularly SDGs 16 (peace, justice and strong institutions) and 5 (gender equality), EU Action Plan on Human Rights and Democracy and EU Gender Action Plan III 2021-2025, and compatible with other interventions implemented in Maldives?

3. Efficiency:

• 3.1. Economic efficiency: Were the resources and inputs converted to outputs in a cost-effective manner? Could the outputs have been achieved with fewer resources?
• 3.2. Operational efficiency: Were the human and financial resources used as planned and appropriately and fully utilised (or were resources misallocated, budgets underspent, overspent)? Were resources redirected as needs changed? Were risks managed? Were decisions taken which helped to enhance efficiency in response to new information? Were the logistics and procurement decisions optimal?
• 3.3. Timeliness: To what extent the results were achieved within the intended timeframe? Was the timeframe as outline in the program design realistic or appropriate? If not, was it reasonably adjusted during the project? were efforts made to overcome obstacles and mitigate delays in how the intervention was managed, as the situation evolved?

4. Effectiveness:

• 4.1. Have the right things been done to achieve the project’s objectives? Have the planned project objectives and results been achieved (or are they likely to be achieved)? If not, what is needed to close the gap and improve the potential for impact? Are there any unintended outcomes (positive or negative) as a result of the project?
• 4.2. What results have been achieved in the implementation of the project? What were the contributing factors to the achievement of the project? What were the main challenges/barriers to achieving the implementation of the project? How well did the project adapt to the significant political and security challenges during implementation?
• 4.3. Has the project achieved inclusive results? Were there differential results for different groups of rights-holders, diverse groups of women, men, and gender diverse people?

5. Impact

• 5.1. Has there been a positive or negative impact on beneficiaries? How has this impact been ascertained? What specific, real difference has the activity made to the beneficiaries?
• 5.2. Were there equal or differential impacts for different groups of rights-holders? If so, why did these differential impacts occur? Were there any gender-related differences in impacts? If and how did gender-related impacts intersect with impacts on other forms of discrimination?

6. Sustainability

• 6.1. Financial sustainability: To what extent has the project activities deployed tools and approaches that are of no or low cost to partners which will allow partners to use these tools beyond the project’s end? If so, how?
• 6.2. Institutional Sustainability: To what extent is the skills introduced to outlet staff during the life of the project will be retained? To what extent will the trainers trained by the project has enable the institutionalization of key skills in newsrooms and media outlets across the Maldives, beyond the life of the Action? Did the Action have a well-planned exit-strategy to mitigate risks of backsliding?

· 6.3. Policy level sustainability: to what extent has the Action made efforts to ensure that the Action will leave a legacy of more accurate and high-quality information that provides citizens with increased access to quality, evidence-based news reporting?

The final set of evaluation questions along with the methodology will be agreed in the inception phase.

Methodology and Data collection

The methodological approach and design of the evaluation will be collaboratively finalized and agreed upon by Internews and the evaluator. The evaluator will assume responsibility for its complete implementation. The methodological approach should be participatory, inclusive, gender-sensitive, and culturally appropriate. The evaluator is expected to present the evaluation methodology, including the proposed sampling of stakeholders to be engaged, during the first week of the evaluation period. This methodology will be defined, discussed, and agreed upon with Internews. The evaluation should encompass, though not necessarily be limited to, the following tools:

The evaluation will involve the following methods:

· Desk Review: Conduct a comprehensive review of relevant project documentation, progress reports, previous pre- and post-workshop surveys, and assessments.

· Qualitative Interviews: Conduct individual and/or group interviews with project stakeholders, either online or in person.

· Focus Group Discussions (FGDs): Facilitate FGDs with beneficiaries and stakeholders, including representatives from Internews, the donor, partners, and NGOs operating in the same domain.

· Online Surveys: Implement an online survey with stakeholders and beneficiaries if deemed relevant to gather additional insights.

These methods aim to ensure a thorough evaluation by combining document analysis, stakeholder perspectives through interviews and surveys, and in-depth discussions with key participants.

The evaluator should address logistical and time challenges, adapting to respondents’ schedules and needs where feasible. The evaluation and associated research must adhere to Internews’ policies and protocols regarding data protection and security, following ethical guidelines for participant confidentiality and privacy. Ensuring the secure transmission of data at all stages of the evaluation is paramount. A non-disclosure agreement will be integrated into the consultancy contract.

Expected Deliverables

The evaluator is required to submit deliverables as follow:

· Inception report outlining the comprehensive evaluation methodology and tools, followed by an evaluation report clearly structured to document the evaluation activities and results.

· A slide deck of preliminary findings

· Draft of evaluation report

· Final evaluation report

The evaluation report will encompass the main findings, conclusions, recommendations, and a record of lessons learned. The evaluator will first present a draft report for feedback from Internews. In considering the provided comments, the evaluator will exercise independent and impartial judgment to refine the final report. The final version should not exceed 20 pages, excluding annexes and the executive summary. The report must include:

• An executive summary highlighting the main findings, conclusions, and recommendations;
• A detailed methodology section, making explicit the limitations of the evaluation (1-2 pages);
• Key findings: An analysis of the project performance of the project in terms of relevance, coherence, effectiveness, efficiency, impact, and sustainability addressing the evaluation questions (10-15 pages);
• Conclusion and Recommendations that are specific, action-oriented, and tailored to all relevant stakeholders. (1-2 pages)

Timeframe

Internews anticipates that around 5-6 weeks will be needed to conduct this evaluation. The evaluator is expected to work a maximum of 25 working days and finish the assignment by July 1, 2024.

Activity

Est. due date / Est. LOE

Conduct desk review

1 April 2024 – 3 days

Evaluation methodology and design

15 April 2024 – 3 days

Data collection

15 May 2024 – 10 days

Draft report

15 June 2024 – 7 days

Integrate feedback and finalize report

1 July 2024 – 2 days

25 days

Evaluation Management

The evaluation will be managed jointly by the Internews Project Manager for Maldives and Regional MERL Manager who will provide support and coordinate with relevant stakeholders; provide the evaluator with administrative support and required data; connect the evaluation team with the relevant stakeholders; review the inception report, draft evaluation report and final reports to ensure quality.

Application Guidelines and Requirements

The due date for application is on 20 February 2024. Internews encourages prospective applicants to apply promptly, as applications will be reviewed on a rolling basis.

Applications must include the below documentation, and will not be considered without it:

• A 2–3-page statement of interest and draft proposal, highlighting how your methodological approach will address the scope planned for this evaluation as well as describing how your profile and experience meet the criteria.
• A short cost justification (not included in the 2-3 page limit) – including your salary history and/or daily rate for your last 3 consultancies.
• A CV and 2 references.
• Evidence of evaluation work (e.g. link to evaluation reports or PDF attachments), highlighting evaluation theories and approaches.

Essential Qualifications

• Master’s degree in social sciences, international development, evaluation, or related field.
• At least 5 years’ experience conducting evaluations.
• Experience with qualitative and quantitative M&E data collection and analysis methods.
• Fluency in spoken and written English.
• Excellent intercultural communication skills/cultural sensitivity and the ability to forge strong cross-cultural relationships demonstrated through previous experience.
• Strong facilitation, presentation, writing, and communication skills.

Desirable

• More than 5 years’ experience as a professional evaluator for international development projects.
• Experience conducting evaluations for media development projects.
• Knowledge of implementing projects in Maldives or neighbouring countries.
• Experience working in different countries across Asia.

How to apply

Link to job posting: https://phf.tbe.taleo.net/phf04/ats/careers/v2/viewRequisition?org=INTERNEWS&cws=38&rid=2195