Consultant – Year Two Implementing Partner Survey + Baseline – Remote

Background:

Mercy Corps is a leading global organization powered by the belief that a better world is possible. In disaster, in hardship, in more than 40 countries around the world, we partner to put bold solutions into action — helping people triumph over adversity and build stronger communities from within.

GAYA is an Associate Award under the IDEAL project, funded by USAID’s Bureau for Humanitarian Assistance, with Save the Children as the prime. The Gender and Youth Activity (GAYA*) Associate Award works to improve the quality and impact of emergency and non-emergency food security and resilience activities by addressing the barriers and challenges implementing partners face when integrating gender and youth within their work. Understanding and meeting the unique and intersectional needs of women and youth across their life-span and addressing the root causes of inequalities are essential to achieve broad and equitable impacts in food and nutrition security.

GAYA uses a human-centered design approach to understand the critical capacity, knowledge, and practice gaps of BHA nutrition and food security program implementers; provide practical and grounded support to address those gaps; and put the interests, voices, intersectional identities, and unique experiences of women, men, girls, and boys at the center of our work.

GAYA has two intermediate results:

• IR1: BHA IP staff demonstrate improved application of gender and youth resources and research from peer learning activities
• IR2: BHA IP staff demonstrate improved application of gender and youth analyses

Mercy Corps started work on this five-year award in late September 2021. From June 2022 – November 2022, Mercy Corps contracted a firm to conduct formative research to better understand implementing partners’ knowledge, attitudes, and practices regarding gender and youth integration into their activities. This research included a quantitative survey that received over 600 responses from implementers working in over 30 countries. To triangulate and better understand these findings, GAYA then conducted complementary qualitative research. This formative research helped GAYA gain better insights into the contexts and challenges implementers face to meaningfully integrate gender and youth in their work.

Since that formative research was conducted, GAYA submitted a revised Monitoring & Evaluation (M&E) Plan to USAID/BHA. The most recent version of the M&E Plan is attached as Annex 1. As part of the M&E plan, GAYA plans to conduct a second annual implementing partner (IP) survey in July 2023, with an initial report submitted to BHA in August, and the final report submitted to USAID/BHA in September 2023.

This IP survey will build on and refine (shorten) the tool used in the YR1 IP survey, and will incorporate additional attitude measurement metrics currently being developed by GAYA. Additionally, this YR2 IP survey will endeavor to measure four of GAYA’s indicators from the approved M&E plan. In measuring these four baseline indicators, the selected consultant will work with GAYA to fully define their measurement methods and calculation:

• Percent of respondents newly applying GAYA-promoted behaviors, following participation in GAYA activities.
• Percent of respondents participating in GAYA activities, reporting improved application of gender and youth resources and research.
• Percent of respondents participating in GAYA gender and youth analysis/assessment capacity strengthening, reporting improved application of gender and youth analyses.
• Percent of GAYA activity participants with a favorable attitude towards using gender and youth analysis/assessment findings or M&E data to inform or adapt IP program design.

In addition to a well-written, copy-edited, 508-compliant final report, every effort should be made by the consultant to avoid extractive or decolonial data collection – in essence, to shift power from GAYA and the measurement consultant to IPs themselves. This may include innovative methods to “share back” data with IPs, as was experimented with in the YR1 research.

Purpose / Project Description:

The primary purpose of this year’s annual IP survey is to capture baseline values for four of GAYA’s indicators, and to provide updated values against YR1 IP survey measurements for gender and youth integration knowledge, attitudes, and practices. The results of the survey will be used to inform GAYA’s intervention adaptation (through a human-centered design approach) and to set targets for the four indicators for the remaining years of this five-year program.

The secondary purpose of this year’s IP survey is to continue to refine and improve GAYA’s measurement tools for IP knowledge, attitudes, and practices, and to share back IP data with the IPs themselves.

Consultant Objectives:

Objectives will be further refined during the inception period for this SOW in partnership with this consultant. The proposed objectives of this study are as follows:

Objective 1a: To capture baseline values for four of GAYA’s indicators

1. Document the current values for the following indicators:
   1. Percent of respondents newly applying GAYA-promoted behaviors, following participation in GAYA activities.
   2. Percent of respondents participating in GAYA activities reporting improved application of gender and youth resources and research.
   3. Percent of respondents participating in GAYA gender and youth analysis/ assessment capacity strengthening reporting improved application of gender and youth analyses.
   4. Percent of GAYA activity participants with a favorable attitude towards using gender and youth analysis/assessment findings or monitoring, evaluation, and learning (MEL) data to inform or adapt IP program design.

Objective 1b: To provide comparison between IP knowledge, attitudes, and practices between the YR1 and YR2 IP survey

1. Document the current state of BHA IPs’ self-reported or self-reported on their team’s behalf knowledge, attitudes, and practices (KAP) as well as their needs and gaps, related to gender and youth integration in BHA programs, as compared to YR1 values.

Objective 2: To elaborate on the attitude measurement scale developed in the YR1 survey

1. Document the current state of BHA IPs’ self-reported or self-reported on their team’s behalf attitudes, using the expanded list of key gender and youth integration attitudes, developed by GAYA

Consultant Activities:

The consultant will support GAYA with the design of the survey, including the instrument itself, and will be expected to input into operational elements of this survey work including considerations around format, length, accessibility, and data output structure, flow, and linkages with the GAYA M&E system. This consultancy will be entirely remote.

• Phase 1: Quantitative data collection design (estimated at 15 person days)
  • Kick-off meeting with GAYA team
  • 5-7 initial interviews with IPs to understand how they prefer to receive data back, and what data they want to receive back. These interviews will inform how the YR1 survey tool is refined and condensed.
  • Review background documents provided, including list of attitudes to measure provided by GAYA, facilitate conversations with GAYA to finalize the sample frame/size and sample methodology, and the quantitative survey study protocols (including considerations on study length, accessibility, data output structure, data flow, linkages with M&E system)
  • Deliver inception report – inclusive of final sample design, sample size, study designs and an early draft Data Analysis Plan (DAP), and modified Performance Indicator Reference Sheets (PIRS) for four baseline indicators
  • After acceptance of the inception report – submit a full DAP, including dummy tables, placeholders for charts/graphs/images, description of how data triangulation/synthesis will be conducted, etc.
  • Deliverables: inception report, initial data analysis plan, final quantitative survey protocol, final DAP
• Phase 2: Survey questionnaire design and coding (estimated at 15 person days)
  • Facilitate conversations with GAYA team to revise YR1 survey questions and responses.
  • Add new survey questions to measure four baseline indicators and additional attitude metrics (GAYA will provide the list of attitudes, the consultant will adapt the list into new survey questions).
  • Code survey in Ona
  • Respond to and incorporate GAYA and BHA feedback on questionnaire prior to survey launch
  • Translate the survey tool into selected languages
  • Deliverables: Drafts of survey questionnaire in excel and Ona
• Phase 3: Survey real-time analysis (estimated at 10 person days)
  • Monitor survey results and perform real-time troubleshooting of survey issues
  • Perform analysis of results in real-time to feed drafts of analysis report
  • Provide recommendations on ways to improve the survey for future iterations
  • Deliverables: updated survey questionnaire based on recommendations for improvement, final survey questionnaire, datasets
• Phase 4: Reporting (estimated at 20 person days)
  • Draft initial analysis report, which will contain recommendations for the revised annual IP survey tool
  • Respond to and incorporate GAYA and BHA feedback on analysis report
  • Draft initial data sharing reports for individual IP organizations (up to 7)
  • Respond to and incorporate GAYA feedback on data sharing reports
  • Participate in data-sharing event (remotely), organized by GAYA
  • Deliverables: drafts of analysis report – including the final well-written, copy-edited, 508-compliant, formatted version, drafts of data sharing reports for IP organizations, final improved survey tool, recommended revisions to PIRS.

Consultant Deliverables:

The consultant will charge GAYA for the days used, not the estimated level of effort. Should the work require more time than the estimate, the consultant will agree on priorities, with the GAYA contact person and confirm changes to the consultancy work plan in writing one month ahead of the contract expiration date.

• Inception report and quantitative protocols
• Survey, which must include:
  • Revised survey protocol – including length, accessibility, data output structure, data flow, linkages with M&E system
  • Draft #1 of survey questionnaire in excel, based on consultations with GAYA team
  • Draft #2 of survey questionnaire based on GAYA and BHA feedback, coded in web-based survey tool (likely Ona)
  • Draft #3 of coded survey questionnaire in web-based survey tool based on GAYA piloting
  • Translated versions of survey questionnaire
  • Rolling analysis of survey results and real-time troubleshooting of survey issues
• Draft #1 analysis report
• Final analysis report, revised based on feedback from GAYA and BHA
• Draft #1 data sharing reports (up to 7)
• Final data sharing reports, revised based on feedback from GAYA

Total = 60 working days over 5 months. This is an estimated LOE, consultants should propose the number of working days/schedule they feel necessary for this work, and justify shifts in the LOE.

The core GAYA team is available to:

• Provide input into the quantitative data collection tools, including the list of attitudes to measure, and participate in meetings with the consultant to discuss the aim of the tools and provide feedback on the drafts prepared by the consultant
• Develop the communications protocol for sharing the survey with respondents – including networking with BHA and BHA RFSA and emergency programs to establish a list of survey recipients
• Consult with BHA on the tools, design, and communications protocols to obtain BHA’s feedback and consensus
• Circulate the survey to respondents based on the above-mentioned communications protocol
• Review and provide feedback on analysis report prepared by the consultant
• Liaise with BHA (the consultant should not engage directly with the donor)
• Organize and facilitate a data sharing event with IPs and/or BHA
• Share report with BHA along with program adaptations

Key Considerations

• The consultant will be responsible for translating and verifying translations for the survey tool. The YR1 IP survey was translated into the following languages, however the tool should be fully validated in these languages before launch: French, Arabic, Spanish, Oromo, Amharic, Bengali. Bidders should assume translation/translation verification in the same languages, though actual languages may shift. Notably, completion of the YR1 survey in Spanish, Oromo, Amharic, and Bengali was quite limited – thus GAYA welcomes suggestions from applicants and the selected consultant on whether to pivot to new languages or remove some of the language options.
• The survey must be short enough that respondents complete it, while being sufficiently detailed to measure baseline for four indicators, knowledge, attitude, and practice changes from the YR1 IP survey, and add in new attitude measurement questions.
• Data should be collected in a usable format, based on the purposes mentioned above.
• GAYA understands “implementing partners” or IPs to include the following: headquarters staff, regional hub staff, country staff, field implementation staff, subawardee or partner staff, and local organization staff, inclusive of entry, mid, and senior level professionals, across regions and geographies, and across both BHA emergency and RFSA programming. IPs also include technical experts, program directors/managers/chiefs of party, gender experts, youth experts, and field officers. Additional categories of IPs may be identified by the consultant, or through conversations with the consultant, GAYA, and BHA.
• Previously, data sharing reports with IPs were prepared in Tableau. GAYA’s preference is that the selected consultant prepare YR2 data sharing reports in PowerBI rather than Tableau.
• Reports produced must at minimum be copy-edited, formatted in GAYA branding, and 508 compliant. GAYA can provide guidance on GAYA branding and 508-compliance, but it is expected that the consultant will budget and perform these tasks.
• GAYA may support in writing contextualized recommendations, where relevant.
• GAYA will be doing qualitative interviews with implementing partners in April and May to better understand the relevant attitudes which enable/prevent gender and youth integration. GAYA will provide the output of these consultations to the consultant during the final week of May, after which the consultant will adapt this list into new survey questions.
• It is possible that the YR2 IP knowledge, attitudes, and practices survey should be separated out from the survey which measures the four baseline indicator values. Suggestions on whether to separate these two surveys should be delivered by the consultant as part of the inception report.

Study Methods

The study design will include an annual participant-based, quantitative survey distributed via the Ona tool. The unit of analysis is IP staff.

Disaggregation categories should include:

• Type of IP: Headquarters staff, regional hub staff, field implementation staff, local organization staff and donor staff
• Gender
• Age/work experience level (entry, mid, senior)
• Language used in survey or FGD
• Region/country/sub-region of program implementation
• Type of IP program: emergency/RFSA
• Conflict/non-conflict/natural catastrophe
• Role – technical expert, PM/COP, gender expert, youth expert, other
• Number of GAYA activities in which they participate
• Duration and attention while participating in GAYA activities

Additional details of the study design will be determined by the selected consultant in consultation with GAYA.

Timeframe / Schedule:

The anticipated length of this consultancy is from approximately April 11 to July 29, 2023. GAYA welcomes recommendations from the consultant as to deliverables that can be submitted earlier than the below deadlines, or areas of the schedule that can be streamlined.

The number of days and consultants required will be suggested by the bidder and assessed by Mercy Corps based on previous experiences. The consultant will charge Mercy Corps for the days used, not the estimated level of effort. Should the work require more time than the estimate, the consultant will agree on priorities, with the GAYA contact person and confirm changes to the consultancy work plan in writing.

• Phase 1: Quantitative data collection design (May 2023)
  • Review background documents
  • Kickoff meeting
  • 5-7 initial interviews with IPs to understand how they want to receive data back, and what data they want to receive back.
  • Support GAYA to revise survey protocol including considerations around format, length, accessibility, and data output structure, flow, and linkages with M&E system
  • Prepare inception report, initial data analysis plan, and revised PIRS
  • GAYA consult with BHA on the survey SOW, tools, and communications protocol
  • GAYA develop the communications protocol for sharing the survey with respondents, based on sampling methodology developed by consultant
  • Prepare final DAP and incorporate BHA and GAYA feedback into inception report
• Phase 2: Questionnaire design and coding (May and June 2023)
  • Draft survey questionnaire for GAYA team input and review
  • GAYA review survey questionnaire and provide feedback to the consultant
  • Update survey questionnaire based on GAYA feedback and code in Ona
  • GAYA pilot survey questionnaire in Ona
  • GAYA deliver final BHA and GAYA feedback to consultant
  • Update coded survey questionnaire with final BHA and GAYA feedback
  • Obtain translation services and review translations for accuracy
  • Update coded survey questionnaire with translations
• Phase 3: Survey real-time analysis (July 2023)
  • Survey launch
  • Analyze survey results (rolling) and troubleshoot survey in real-time
  • Provide final survey questionnaire and recommendations for improvement in next annual survey
• Phase 4: Reporting (August and September 2023)
• Prepare draft analysis report with recommendations for YR3 annual survey
• Review draft #1 of the report and provide feedback
• Draft initial data sharing reports for individual IP organizations (up to 7)
• Review initial data sharing reports and provide feedback
• Participate in data validation/data sharing event
• Finalize all reports based on feedback – due August 31, 2023

Report Details and Content

• Page limit (20 plus attachments)
• Cover Page
• List of Acronyms
• Table of Contents which identifies page numbers for the major content areas of the report.
• Executive Summary (2 to 3 pages) a clear and concise stand-alone document that gives readers the essential content of the report in 2 or 3 pages, previewing the main points in order to enable readers to build a mental framework for organizing and understanding detailed information within the report. The Executive Summary should include: major lessons learned; maximum of two paragraphs describing the program, summary of recommendations.
• Methodology: sampling method including strengths and weaknesses of method used, inclusion of stakeholders and staff, rough schedule of activities, description of any statistical analysis undertaken. This section should also address constraints and limitations of the study process and rigor. The methodology section should also include a detailed description of data collection techniques used throughout the survey.
• Results: presentation of significant results organized around the key study questions (above) and sub themes. A table with disaggregated results by indicator should also be provided.
• Synthesis, Recommendations and Lessons Learned: This is space for the consultant to think about the data and results, and make concrete recommendations for the program’s design and generally comment on data and results. Everything presented in this section must be directly linked back to the information presented in the Results section of the report. Ideally, items discussed here will not be completely new to the reader, but rather will refer to previous discussions.
• Annexes: data collection instruments in English and translations; SOW; disaggregated data sets in electronic format with comparisons to YR1 IP survey, where applicable.

GAYA Team Composition and Participation

The consultant will report to Technical Officer, Research and Learning. GAYA Program Director, Gender Advisor, Youth Advisor, Program Officer, and Knowledge Management/Communications will be providing technical support oversight on all project deliverables. The consultant will also work with Senior Technical Lead – MEL.

Required Experience & Skills:

• At least 8 years of experience conducting research for international development or learning projects
• Master’s degree or equivalent experience in social sciences, research, or related field
• Strong methodological experience in quantitative research
• Prior experience with coding in Ona or similar technology
• Experience conducting research studies for learning, gender, youth, food security, and/or nutrition programs
• Detail oriented to ensure quality and consistency
• Ability to remotely manage and coordinate with evaluation stakeholders and implementers
• Offers a strong understanding of how power, gender, and intersectional dynamics can influence data collection results
• Strong work ethic and the ability to work well independently and as part of a team.
• Written and oral fluency in English required

Additional Requirements (preferred)

• Familiarity with human-centered design principles and application of HCD principles to survey design
• Familiarity with designing for social behavior change, and social behavior change principles

Diversity, Equity & Inclusion
Achieving our mission begins with how we build our team and work together. Through our commitment to enriching our organization with people of different origins, beliefs, backgrounds, and ways of thinking, we are better able to leverage the collective power of our teams and solve the world’s most complex challenges. We strive for a culture of trust and respect, where everyone contributes their perspectives and authentic selves, reaches their potential as individuals and teams, and collaborates to do the best work of their lives.

We recognize that diversity and inclusion is a journey, and we are committed to learning, listening and evolving to become more diverse, equitable and inclusive than we are today.

Equal Employment Opportunity
We are committed to providing an environment of respect and psychological safety where equal employment opportunities are available to all. We do not engage in or tolerate discrimination on the basis of race, color, gender identity, gender expression, religion, age, sexual orientation, national or ethnic origin, disability (including HIV/AIDS status), marital status, military veteran status or any other protected group in the locations where we work.

Safeguarding & Ethics
Mercy Corps team members are expected to support all efforts toward accountability, specifically to our stakeholders and to international standards guiding international relief and development work, while actively engaging communities as equal partners in the design, monitoring and evaluation of our field projects. Team members are expected to conduct themselves in a professional manner and respect local laws, customs and MC’s policies, procedures, and values at all times and in all in-country venues.