Family Planning Project MEL Consultant – Qualitative realist evaluation and implementation learning

Period of Engagement

We anticipate the consultancy will take approximately 45 days over the period of five months starting mid-December 2022 through end of May 2023.

Location: Remote

Project Description/Background

The Connect Project (2019-2024), aims to generate evidence on scalable postpartum family planning (PPFP) approaches for first-time mothers (FTMs) that leverage and strengthen health systems, inclusive of government health systems and community-level health efforts, in Bangladesh and Tanzania. In Bangladesh, Connect also aims to improve coverage, timing, and use of quality postnatal care (PNC) by FTMs.

Connect’s learning agenda aims to identify whether “enhancements” to local health systems are effective in increasing FTMs’ service use at scale; factors that facilitate/impede implementation, scale-up, and institutionalization; and unintended consequences. Country-specific enhancements provide opportunities for learning about the feasibility, effectiveness, and sustainability of these enhancements in improving FTM’s PPFP and PNC service use at the facility-and community-levels.

To better understand how Connect impacts FTM health outcomes, Connect aims to evaluate how systems for health, and various contextual and management factors, interact with program enhancements, and to identify whether and how existing systems can feasibly institutionalize the program enhancements. Three concurrent learning streams underpin these learning efforts:

1. A qualitative evaluation employs a realist evaluation approach, incorporating a system lens, to understand the mechanisms by which approaches contribute to outcomes, and how they are scaled and institutionalized in the respective country health systems. The evaluation aims to identify transferable scalable approaches and best practices for improving PPFP and PNC use among FTMs in other contexts. In each country, a research firm will conduct the evaluation including data collection, analysis and presentation of findings in both countries. The qualitative study will be conducted over multiple stages in both countries. The initial stage will take place during Q1 and Q2 2023 and will include key-informant interviews (KIIs) with government officials, and a document review of relevant government policies, plans and procedures.
2. Implementation learning efforts, conducted by country-based project staff, will include brief remote surveys, and pause-and-reflect sessions with frontline providers and their supervisors.
3. A quantitative impact evaluation, led by the George Washington University, will evaluate the impact of the enhancements on FTMs’ use of PPFP and PNC, as well as intermediate outcomes (i.e., couple communication, self-efficacy, knowledge).

Description of Services

Save the Children US is seeking a consultant for five months, between December 12, 2022, and May 19th, 2023. This role will provide partial coverage for a maternity leave and will: 1. lead Connect’s qualitative evaluation activities in Bangladesh and Tanzania ; 2. Lead implementation learning efforts in Bangladesh and Tanzania; and 3. triangulate findings from the qualitative and implementation learning efforts with Connect’s quantitative impact evaluation (led by George Washington University [GWU]).

Details of the services to be covered by the consultant include:

1. Leading qualitative evaluation for Connect (50%)

The protocol will be finalized and submitted to the relevant ethical review boards by the time the consultant is on-boarded. Their role will focus on supervising the day-to-day technical needs of the qualitative evaluation implementation during the consulting period. The tasks required include:

• Supervising data collection, transcription and translation, analysis and presentation of findings by country-based research firms in liaison with country-based project teams;
• Address field-work challenges as they arise, and document challenges and resolutions;
• Work closely with in-country research firms on codebook development and adjustments to the analysis strategy;
• Review transcripts, memos and analysis briefs developed by in-country research firms;
• Contribute to, and review analysis reports by research firms; and
• Propose adaptations to the qualitative evaluation approach for future rounds of data collection and/or overall improvement of the design.

2. Provide technical assistance to design, implementation, and documentation of implementation learning activities (40%)

The consultant will provide support and technical assistance for implementation learning activities in both countries, closely working with country-based project staff. The methodology, tools, and procedural plans for these activities, and reporting matrices are expected to be developed by time the consultant is onboard, but suggestions for improvement from the consultant will be welcome. These activities include one round of brief quantitative “pulse polls” and pause-and-reflect sessions with frontline providers and their supervisors, which focus on the acceptability and feasibility of Connect’s enhancements. The required tasks include:

• Provide guidance to the country-based project staff through field challenges,
• Follow up and review of reporting of results by country-based project staff,
• Update tools and analysis approach as needed,
• Provide technical assistance to country teams for descriptive analysis of pulse poll and pause-and-reflect data to produce brief practical programmatic recommendations.

3. Triangulate findings across data sources, including the impact evaluation baseline (10%)

The consultant will attend weekly project team meetings, coordinate with the GWU team and country-based project staff to address challenges, and review quantitative research tools and preliminary findings to inform adaptation of qualitative evaluation methods and to allow triangulation across data sources.

At the culmination of the consultancy, the consultant will triangulate findings across learning sources, and will develop an internal interim report of challenges with the qualitative research and implementation learning efforts, and recommended adjustments and next steps for each country. This report will incorporate lessons from implementation learning activities and impact evaluation described below, and will document challenges with the qualitative evaluation and any shifts from the protocol.

Deliverables and Timeline

1.) Orientation: Handover, orientation, and review of documents

Deliverable – Work plan

Timeline – December 2022

# of Days – 2

2.) Support to qualitative realist evaluation: Finalize revisions to data collection tools, codebooks and analysis strategy if needed (collaborating with research firms)

Deliverable – Updated codebooks and summary document of analysis strategy for each country, finalized data collection tools

Timeline – January 2023

# of Days – 6

3.) Support to qualitative realist evaluation: Provide remote supervision to data collection activities, carry out debriefing sessions, review analysis procedures, and ensure compliance with protocol by research firms AND document any deviation or updates to evaluation protocol, and their rationale

Deliverable – At least one call weekly with research firms and country staff; Internal report of changes to protocol and tools

Timeline – January-February 2023

# of Days – 10

4.) Implementation learning: Provide technical assistance to country-based project staff in conducting implementation learning activities (pause-and-reflect exercises, pulse polls), and to analyze data

Deliverable – Brief written analyses of the implementation learning findings and recommendations to be incorporated into the interim report for each country (see below)

Timeline – February-March 2023

# of Days – 16

5.) Provide support for impact evaluation: Coordinate with GWU as needed for triangulation of findings, contributing with regular donor reporting, including preparing presentations or other deliverables to the donor as related to the evaluation process. Participate in regular coordination meetings.

Deliverable – Substantive feedback to donor reports and presentation of quantitative findings.

Timeline – Throughout consultancy

# of Days – 5

6.) Reporting of lessons learned: Development of the interim report of findings across qualitative evaluation, implementation learning, and baseline impact evaluation.

Deliverable – Internal report of interim evaluation findings

Timeline – May 2023

# of Days – 6

Management of Consultancy

The consultant’s main point of contact and supervisor will be Melanie Yahner, Connect Project Director and Senior Advisor for Adolescent Sexual and Reproductive Health, in the Department of Global Health at Save the Children US (SCUS). Other key working relationships will include: SCUS Connect project team, Save the Children International country office (CO) project teams in Tanzania and Bangladesh, research firms in both countries carrying out the qualitative evaluation, and the GWU team carrying out the quantitative impact evaluation.

Qualifications

• Minimum of a Master’s degree in Public Health, Health Sciences, Social Sciences, Anthropology, or relevant field, and at least 10 years of relevant experience.
• Familiarity with FP, PPFP, PNC and/or adolescent health.
• Proven proficiency and experience in qualitative research methods, including study and tool design, overseeing data collection, coding, analysis, and interpretation
• Experience with evaluation of complex health programs and systems, including demonstrated experience with leading at least one qualitative evaluation using realist evaluation approaches
• Proven experience with implementation learning, such as collaboration, learning, and adapting (CLA) approaches
• Professional proficiency in MS Office suite and at least one qualitative data analysis software (e.g., Atlas.ti, NVivo)
• Demonstrated experience overseeing research partners, including universities and local research partners, to implement mixed methods evaluations
• Demonstrated experience with developing technical products (such as briefs and technical reports)
• Professional proficiency in spoken and written English
• Demonstrated ability to communicate and collaborate effectively with individuals and teams at all levels, both internally and externally
• Passion and commitment to meeting agreed deadlines
• Ability to produce high quality work under tight time frames
• Excellent presentation and communication skills