Internews Europe
Terms of Reference
Final Evaluation of ‘ Suara Masyarakat (Community Voices) project in Malaysia
Internews overview
Internews is an international non-profit organization whose mission is to empower local civil society worldwide to give people the news and information they need, the ability to connect and the means to make their voices heard. For more information about the organization please visit: www.internews.com
Project overview
For the past 32 months, Internews in Malaysia with the support of the European has been implementing a 36-month action entitled Suara Masyarakat (Community Voices) that aims to build the capacity of journalists to produce high-quality, investigative content on reform process in Malaysia and create feedback loops between citizens, journalists and local authorities that are crucial for government accountability and transparency. Internews partners with Science of Life Studies 247 (SOLS 24/7) as co-applicant for the implementation of this action. Further, implementation has been undertaken with the support of strategically selected third parties, Sinar Project and the Centre for Independent Journalism (CIJ), who has been engaged through a sub-granting mechanism. Internews has also partnered with other NGOs and grassroots organisations such as Jaringan Kampung Orang Asli Kelantan, Faqcheck Lab and the Center for Orang Asli Concerns through microgrants. To implement this action, Internews has drawn on its robust experience supporting journalists and CSOs in the region.
Internews in Malaysia team through its local partners, has been delivering high quality investigative and data journalism on reform processes in the country and develop community information media and networks to improve governance by providing two-way communication between citizens and local authorities. This is done through workshops and trainings on investigative, data and mobile journalism which targets journalists, citizen journalists and information poor communities with a focus on women, youth, Indigenous and marginalised groups.
Overall objective: Strengthen the capacity of media as a key actor in promoting good governance, transparency, and accountability
Specific objectives (SO):
1) SO 1: Increase the capacity of Malaysian journalists to deliver high quality investigative and data journalism on reform processes in the country
2) SO 2: Develop community information media and networks to improve governance by providing two-way communication between citizens and local authorities
The Suara Masyarakat (Community Voices) project is designed based on the following methodology:
This action seeks to increase journalists’ capacity and skills through trainings and mentoring in investigative reporting and data journalism, with the goal of building the capacity of media to promote good governance, transparency and accountability. Internews’ training and mentoring programmes are built on creating an environment for independent media to safely grow and promoting local information networks for both citizen journalists and the general public. In line with this goal, Internews has designed comprehensive training and mentoring programmes that strengthen the capacity of journalists to report effectively and strengthen the
inclusion of communities and marginalised groups in democratic debate. Internews and its partners have also woven a sub-grants programme into the fabric of the programme, promoting sustainability and giving mentees real experience in reporting, publishing and sharing content. Our organisational Theory of Change frames all our work: to enable safe access to trusted information, to ensure the inclusion of systematically excluded groups, to increase quality content of local, verifiable and unbiased information; and to enhance the engagement of citizens with their information landscape. This project, designed with the co-applicant
SOLS 24/7, responds to this logic as well.
Evaluation Purpose and Objectives
The purpose of this final evaluation is to assess the overall, achievements and lessons learned of the project that will help to determine the project success and to inform future programming design and implementation in Malaysia. It should also look at signs of potential impact of project activities on beneficiaries and sustainability of results. The consultant will be responsible for producing several deliverables using a methodology and following a timeline that is mutually agreed upon with the project staff. This evaluation has the following principal tasks which must be assessed in the context of the political and environmental challenges that the project faced during implementation:
- Assess the project’s achievement towards goals and objectives and look at signs of potential impact of project activities qualitatively and quantitively, as well as sustainability of results; This includes how the project contributed to strengthening the capacity of journalists to produce high-quality, investigative content on reform process and increasing dialogues among citizens (especially women and marginalised groups) and local authorities and the assessment of the audience engagement and interaction with the content produced by the project using proxy measures.
- Analyse the underlying factors beyond Internews’ control that affected the achievement of the project results and required adaptation of the project activities including the impact of changes in the operational environment over the project lifetime and its impact on project activities;
- Assess the performance of the project in terms of relevance, coherence, effectiveness, efficiency, impact and sustainability;
- Which effects, both intended and un-intended, did the program have on beneficiaries.
- Extract learnings and good practices and provide key recommendations which could be used to enhance future interventions.
Evaluation Scope and Methodology
The evaluation covers the project implementation period from 1st January 2020 to 31st December 2022 and should encompass the entire scope of the project. Ideally the work will be carried out in country by a local national evaluator with language skills. If no suitable local candidate can be found, then this evaluation may be implemented completely remotely by the internal evaluator.
The evaluation should use the Development Assistance Committee of the Economic Cooperation and Development ( OECD/DAC) criteria (relevance, effectiveness, efficiency, impact, and sustainability), which is one of the most widely used criteria to review aid effectiveness using gender and inclusion lens. The following questions might guide the project’s evaluation but are subject to discussion and agreement during the period of evaluation design.
- Did the theory of change hold true?
- Has the project responded to the needs of the target groups and continued to do so when circumstances have changed? (Relevance)
- To what extent were lessons learned from what worked well and less well applied to improve and adjust project implementation? (Relevance)
- To what extent have other interventions (particularly policies) supported or undermined the project intervention, and vice versa? (Coherence)
- How compatible is the project with other interventions in the region or organization where it was being implemented? (Coherence)
- Have the right things been done to achieve the project’s aims? Have the planned project objectives and outcomes been achieved (or are they likely to be achieved)? If not, what is needed to close the gap and improve potential for impact? (Effectiveness)
- Were the resources and inputs converted to outputs in a timely and cost-effective manner? Did the project achieve the planned outputs? Could the outputs have been achieved with fewer resources? (Efficiency)
- What results have been achieved in the implementation of the project? What were the contributing factors to the achievement of the project? What were the main challenges/barriers to achieving the implementation of the project? How well did the project adapt to the significant political and security challenges during implementation? (Achievement)
- What are the tangible outcomes (positive and negative) as a result of this project? Has there been a positive or negative impact on beneficiaries as a whole? How has this impact been ascertained? What specific, real difference has the activity made to the beneficiaries? Have certain groups benefited more or been negatively impacted more? (Impact)
- Are the results sustainable? If so, how? (Sustainability)
- Analysis of horizontal issues such as gender (empowering women and girls), youth engagements, and HR.
- An evaluation of the EU’s visibility, impact and results.
Data collection tools
The evaluation will primarily focus on donor key indicators that will help to assess performance and success of the project using relevant survey tools. The evaluator should find ways to ease logistical and time challenges and adapt to the respondents’ schedules and needs, to the extent feasible. The evaluation and related research should abide by Internews’ policies and protocols related to data protection and security and be conducted in line with ethical protocols including participant confidentiality and privacy. The secure transmission of data is of great importance and must be ensured at all stages of the evaluation. A non-disclosure agreement will be included in the consultancy contract.
The methodological approach and design of the evaluation will be developed and agreed between Internews and the evaluator. The evaluator will then be responsible for its objective implementation. The methodological approach should be participatory, inclusive, gender-sensitive and culturally appropriate. The exact evaluation methodology (including the proposed sampling of stakeholders to be engaged) should be presented by the evaluator and defined, discussed, and agreed upon with Internews during the first week of the evaluation period. However, the evaluation should include, but not necessarily be limited to, the following tools:
- Desk review of relevant project documentation, progress reports, previous pre and post workshop surveys and assessments, etc.
- Individual and/or group interviews with project stakeholders online – (where possible);
- Online survey with stakeholders, if relevant
- Focus group discussions (FGDs) and key informant interviews (KIIs) with beneficiaries and stakeholders including Internews, donor, partners and NGOs working in the same space
Expected Deliverables
The evaluator is expected to submit an inception report describing the detailed evaluation methodology and tools and an evaluation report that documents the evaluation activities and results, following a clear structure. The evaluation report will contain the main findings, conclusions, and recommendations as well as a recording of the lessons learned. The evaluator will initially prepare a draft report, which will be submitted to Internews for comments. While considering the comments provided on the draft, the evaluator shall use their independent and impartial judgement in preparing the final report.
The final report should be no longer than 20 pages, excluding the annexes and executive summary. The report must include:
- An executive summary highlighting the main findings, conclusions, and recommendations;
- A detailed methodology section, making explicit the limitations to the evaluation;
- An analysis of the project performance of the project in terms of relevance, coherence, effectiveness, efficiency, impact, and sustainability;
- An analysis of the technical aspects and main achievement of the project;
- Analysis of the underlying factors that affected the achievement of the project results; and
- Recommendations, being specific and action-oriented, and tailored to all relevant stakeholders
Timeframe
Internews anticipates that around 5-6 weeks will be needed to conduct this evaluation. The evaluator is expected to work a maximum of 20 working days and finish the assignment by mid-November, 2022.
Supervision of the Consultant
The consultant will be under the direct supervision of Internews’ Malaysia Project Director and Regional Research and Evaluation Manager. Internews’ staff will facilitate the consultant needs for the purpose of the evaluation.
Application Guidelines and Requirements
Interested parties should apply on this page by clicking ‘Apply Now’ button or to Internews Malaysia via the following email: [email protected]
Applications should include the below documentation.
- A 2–3-page statement of interest/draft proposal, highlighting how your methodological approach will address the scope planned for this evaluation as well as describing how your profile and experience meet the criteria
- A CV and 2 references
- Evidence of evaluation work (e.g. link to evaluation reports or PDF attachments), highlighting evaluation theories and approaches
- A short cost justification (not included in the 2-3 page limit) – including your salary history and/or daily rate for your last 3 consultancies
Essential Qualifications
- Knowledge of implementing projects in Malaysia or in neighbouring countries
- Master’s degree in social sciences, international development, evaluation or related field
- At least 5 years’ experience conducting evaluations
- Experience with qualitative and quantitative M&E data collection and analysis methods
- Fluency in spoken and written English, Bahasa Malaysia
- Excellent inter-cultural communication skills/cultural sensitivity and the ability to forge strong cross-cultural relationships demonstrated through previous experience
- Strong facilitation, presentation, writing and communication skills
Desirable
- More than 5 years’ experience as professional evaluator for international development projects
- Experience conducting evaluations for media development projects
- Experience working in different country across Asia