Consultancy – READY II Program Evaluation

I. Background
When major disease outbreaks occur, non-governmental organizations (NGOs) are often on the frontlines, using their deep connections with affected communities and expertise to support outbreak readiness and response. READY, an initiative funded by USAID’s Bureau for Humanitarian Assistance and led by Save the Children and a consortium of partners, is supporting NGOs to more effectively respond to major disease outbreaks in humanitarian settings. Through investments in a robust and diverse capacity-strengthening portfolio, knowledge and best-practice sharing, and engagement with key coordination groups to identify and respond to real-time needs, READY is equipping national and international humanitarian NGOs with knowledge and skills to be ready to respond to major disease outbreaks through integrated and community-centred approaches.

The first phase of READY ran from September 2018 through March 2022. The current phase of READY launched in April 2022 and will run through the end of March 2024. As READY approaches the end of the initiative, we are seeking a consultant to lead on an evaluation of READY’s work in the second phase.
To learn more about the READY initiative visit https://www.ready-initiative.org/

II. Evaluation Purpose
The purpose of this evaluation is to identify to what extent READY’s approach, activities and events successfully supported international and national NGO readiness and capacity for outbreak response across operations, coordination and technical support.

III. Evaluation Type
READY plans to carry out an external, mixed methods performance evaluation to review how well the initiative achieved its objectives, highlight lessons learned and best practices for future programs of a similar nature. Though the methodology will be finalized with the support of the selected consultant(s), it is anticipated that the evaluation will largely rely on qualitative methodology with a focus on interviews and discussion with key stakeholders. READY encourages the use of innovative evaluation methodologies including outcome harvesting or case studies.

IV. Evaluation Questions
READY is interested in understanding the relative effectiveness of its various activities in strengthening NGOs’ ability to respond to outbreaks, within each of the project’s two objectives: 1) Capacity of humanitarian NGOs is increased for response to major disease outbreaks through innovative operational and technical capacity strengthening approaches, 2) Technical and operational systems for response to major disease outbreaks are informed, influenced, and improved by systematically capturing, disseminating, and applying learning from COVID-19 and other epidemics/pandemics.

Key themes to explore:

• Were the chosen operational and technical activities the right ones to build organizational and individual capacity to respond to major disease outbreaks, and to what extent were they successful at doing so?
• What gaps in operational and technical capacity for major disease outbreak response did READY successfully address? What potential gaps remain that are a potential area of focus for similar projects in the future?
• Did READY engage in the right global outbreak and humanitarian coordination structures at the right time? How did engagement in these mechanisms help/or not READY to achieve impact?
• How effective was READY at meeting its localization commitments? How were these efforts received and how could they be improved?

The final evaluation questions will be identified and agreed upon amongst the consultant(s) and the READY Chief of Party and M&E Advisor.

V. Evaluation Methods
The proposed methodology includes:

• Document review including review of donor reporting, after action reviews from READY activities, participant feedback, pre-and post- test scores, activity learning objectives, etc.
• Annual Engagement Survey data from March 2023 and potentially Q1 2024.
• Online survey targeting READY email subscribers (a global network of over 2,000 individuals who have subscribed to READY updates and/or engaged in READY trainings, events, and activities), stakeholders, and partners
  o Note: Given the nature of the survey and READY’s target populations, there is limited capacity to develop a scientific sample. This will be a non-probability sample using self-selection sampling.
• Qualitative interviews (key informant interviews and/or focus group discussions) with READY staff, consortium partners, advisory group members, external stakeholders who have engaged with READY’s work (webinars, capacity building trainings, technical guidance development and operationalization, etc.), and coordination bodies (Health Cluster, RCCE Collective Service, etc.)
• READY encourages proposals from applicants using alternative approaches including outcome harvesting, case studies, etc.

The Consultant will work with the READY M&E Advisor and the READY Chief of Party to develop an evaluation plan and methodology at the start of the consultancy. READY can provide support and access to online survey platforms such as Survey Monkey, Google Forms or Kobo Toolbox.

VI. Evaluation Findings Dissemination Plan
READY will share findings from the evaluation with key stakeholders, and relevant partners. It will also upload the final report to the Development Experience Clearinghouse and READY’s website. A full dissemination plan will be developed with the READY team and BHA and will include a final report and presentation of findings to key stakeholders.

VII. Expected Deliverables & Timeline
Recruiting and onboarding the consultant will take place in final quarter of 2023. Data collection for the evaluation will take place in the final quarter of the program (January-February 2024). The evaluation report will need to be finalized and approved by the end of the project on March 31, 2024.
The Consultant’s deliverables and tentative timeline (subject to the commencement date of the evaluation) are outlined below. Key deliverables are in bold. The Consultant and the READY M&E Advisor will agree on final milestones and deadlines at the inception phase.

Deliverables and Tentative Timeline – Milestones

Early Dec 2023 – The Consultant is contracted, onboarded and commences work

Early Jan 2024 –

The Consultant will submit a detailed workplan and timeline including:

• evaluation objectives, scope, and key evaluation questions
• description of the methodology, including design, data collection methods, sampling strategy, data sources, and evaluation matrix against the key evaluation questions
• data analysis and reporting plan
• caveats and limitations of evaluation
• risks and mitigation plan
• ethical considerations including details on consent
• stakeholder communication and engagement plan
• key deliverables, responsibilities, and timelines
• resource requirements
• data collection tools

Once the report is finalized and accepted, the Consultant must submit a request for any change in strategy or approach to the M&E Advisor and Chief of Party.

Late Jan/Early Feb 2024 – The Consultant will lead on data collection and analysis

Mid Feb 2024

The Consultant will share the Draft Report including the following elements:
• Executive summary
• Background description of the program and context relevant to the evaluation

• Scope and focus of the evaluation
• Overview of the evaluation methodology and data collection methods
• Findings aligned to each of the key evaluation questions
• Specific caveats or methodological limitations of the evaluation
• Conclusions outlining implications of the findings or learnings
• Recommendations
• Annexes (Project logframe, evaluation ToR, Inception Report, Evaluation schedule, List of people involved)

Note: A consolidated set of feedback from key stakeholders will be provided by Save the Children within 2 weeks of the submission of the draft report.

Early March 2024 – The Consultant will develop and share the Final Report incorporating feedback from consultation on the Draft Evaluation Report

Mid March 2024 – The Consultant will present findings of evaluation to management team and donor.

VIII. Consultant Profile

The evaluation will require an external Consultant with:
– Master’s Degree, preferably in public health or a related field;
– Proven experience in program evaluation;
– Strong experience in qualitative methods, including conducting interviews or group discussions
– Excellent English language oral and written communication skills (required);
– Knowledge and experience in humanitarian response and outbreaks (required);
– Familiarity with humanitarian and global health coordination mechanisms (required);
– Experience with USAID/BHA Monitoring and Evaluation practices (preferred);
– Comfort working in teams as well as acting independently in the implementation of specific tasks, multitasking and prioritizing, working under pressure, and meeting deadlines;
– Willingness to work across multiple time zones as needed.

IX. Submission of Proposal

• Total proposal must be less than 5 pages, excluding annexes/attachments
• Proposal must include the following:
  • Description of the individual(s) and/or firm and their relevant experience against Section VIII (including CV)
  • Summary of proposed approach in line with the above methodology and timeline (300-500 words)
  • Budget (including proposed payment schedule)
  • Sample of similar/ relevant work (1-3 pages, can be included as an annex)