RFP for India Programme Evaluation

Who We Are

Girl Effect is an international non-profit that builds media that girls want, trust, and need. From chatbots to chat shows and TV dramas to tech, our content helps adolescent girls in Africa and Asia make choices and changes in their lives.

Our team in India has expertise in content, digital technology, and behaviour change. Over 4 years we have demonstrated impact at scale, reaching 21 million girls and initiating conversations with over 250000 girls via our digital content and products. We work closely with our audiences to build products they want, trust and need, ensuring these are embedded in culture and rapidly innovating with changes in the digital landscape.

We create safe spaces for girls, sharing facts and answering questions about health, nutrition, education, and relationships, empowering girls with the skills to negotiate and redefine what they are told is possible “for a girl.”

Our reach is 50 million and counting and we’re using technology to reach girls at scale so every girl can choose to be in control of her body, her health, her learning, and her livelihood.

Because when a girl unlocks her power to make different choices that change her life, it inspires others to do so too. She starts a ripple effect that impacts her family, community, and country.

That’s the Girl Effect.

Background

Girl Effect (GE) first partnered with Gavi, the Vaccine Alliance, between 2016 and 2021 on routine childhood immunization and human papillomavirus (HPV) vaccine demand generation across Ethiopia, Tanzania, Malawi and Rwanda. The collaboration continued with a new phase that started in 2022 and will run until 2026: the Gavi 5.0 programme. India is part of the latest phase.

Since 2006, the Human Papilloma Virus (HPV) Vaccine has been available in many countries, and many adolescent girls and young women have been vaccinated against HPV in higher and upper-income countries. In India, which has among the highest incidence of the virus, the vaccine has been privately available since 2008.

In 2022, India’s National Technical Advisory Group on Immunization (NTAGI) recommended introducing the Human Papilloma Virus (HPV) Vaccination to all eligible girls between the ages of 9-14 years under the Universal Immunization Programme. In the 2024 Union budget, the government announced its aim to encourage vaccination for girls aged 9 to 14 to prevent cervical cancer.

At the time of HPV vaccination launch in India, Girl Effect, with support from GAVI, will roll out a communication strategy on HPV. The overall strategy will include the use of digital media to reach the wider public, particularly parents of adolescent girls, to build awareness and acceptance of the HPV vaccine. Additionally, it will include on-ground advocacy and training in hotspot locations.

At the nascent age of 9-14 years, parents of adolescent girls are the primary decision-makers around matters related to health and immunization. These decisions are also influenced by other key actors in the community, such as teachers, health system actors, and counselors amongst others. The strategy, therefore, aims to include a digital component and the use of social media to reach the wider public, with a strong engagement and outreach to parents to build awareness and acceptance of the vaccine.

Girl Effect India aims to build a multi-channel intervention focused on the Hindi belt. It targets adolescent girls between 9-14 years and their parents from lower income backgrounds, school teachers, health workers, and other key influencers to deliver HPV content to marginalized communities. Key solutions being designed by Girl Effect are:

Through a robust social and behaviour change communication (SBCC) framework, the products designed by Girl Effect seek to influence key behavioural drivers such as knowledge, attitudes, perceived social support, self-identity and social identity, among others. In addition to this, Girl Effect understands the media behaviour and caters to developing products for an underserved media audience who have been exposed to digital media only recently.

Scope of the Campaign

1. Online Campaign Component

High production video content on social media and 1-1 conversation content on menu based chatbot to reach the digitally connected audience, i.e. parents and older adolescents 18-24 years old.

To ensure that we are reaching the right audience on digital platforms, we employ a sophisticated targeting approach leveraging the demographic and interest-based targeting that respective social media platforms allow – this is embedded in formative research on the media consumption and behaviour of our target audience [i.e. lower socio economic strata (SEC C&D) with digital access]. This will be deployed in all 11 hindi speaking states (list mentioned below in target audience and location section).

1. Offline Campaign Component

Champion app (a technology-based solution that can be accessed offline) to deliver HPV content to digitally inactive as well as marginalized communities of adolescent girls (9 – 14 years old).

For on-ground activations to reach school as well as out of school girls, we will leverage organisations with onground networks that are also engaged with the in the HPV campaign – This will include training health workers, community leaders, teachers and peer influencers (older adolescents) on the application. This will be deployed in select hotspot locations.

Campaign Audience

• Primary Audience: Parents of girls aged 9-14 years old, i.e. men and women aged 31-55, across 11 states in the Hindi belt
  • Rationale: Parents of girls 9-14 years old are the key decision-makers for girls’ health choices at this age
  • Tentative Campaign Channel: Facebook, Youtube, + 1 other platforms like Moj or Takatak
• Secondary Audience: Teachers and health workers facilitating conversations with younger adolescents 9-14 years old across 11 states in the Hindi belt
  • Rationale: Girl Effect, being experienced in speaking to adolescent girls intends to build relevance and readiness among girls for uptake of vaccination.
  • Tentative Campaign Channel: Champion Application delivered by on-ground partners in schools and hard-to-reach communities via ‘Health Champions’ such as teachers, health workers, and community leaders who will facilitate discussions on the HPV vaccine with younger adolescent girls between 9 and 14 years.

Campaign Locations

11 Hindi-speaking states and union territories:

• Himachal Pradesh, Uttarakhand, Uttar Pradesh, Delhi, Haryana, Rajasthan, Madhya Pradesh, Chhattisgarh, Bihar, Jharkhand, Chandigarh
• Within each state, the hotspot locations /districts for offline components of the campaign will be identified as and when the campaign launch is finalized.
• Potentially we may scale the reach of the campaign to other cities beyond the Hindi belt, with high digital penetration and consumption of the Hinglish language.

Recruitment Criteria for Evaluation

Audience*

• Parents of girls 9-14 years old, likely between 30-55 years of age
• Girls between 9-14 years of age in school and out of schools
• Socio-economic class C & D classified under SEC/NCCS.

*more criteria are under discussion: e.g. number of hours of digital usage of the respondent.

Geographic locations

• 4 states from the list of 11 Hindi-speaking states which evaluates the impact of the overall campaign (with online and offline components).
• The hotspot locations for on-ground intervention are to be further discussed, therefore these 4 states for evaluation will be finalized later.

Representative sampling for both audiences will be required in all 4 locations.

The Scope of Evaluation

Currently, there are no formalized plans for an HPV campaign or national rollout of the vaccine. The launch plan is expected to shape up later in the year, and we anticipate the vaccine may be rolled out in phases, eg. Phase 1 launched in 7 states, Phase 2 next 10 states, and so on and so forth.

Within these phases, Girl Effect would target the campaign primarily towards the Hindi-speaking geographies (from the above-mentioned list) as and when HPV is launched in the respective states. We anticipate the campaign may not run simultaneously across all geographies, and different states may receive campaign communication at different times.

Keeping the above assumptions in mind, Girl Effect is looking to commission a research agency with experience in quantitative research and immunization work, to run an evaluation of the programme. We would want to understand from you:

• What is your recommended measurement approach to evaluate the overall campaign, capturing the impact of online and offline campaign components?
• The evaluation approach must be robust such that it gives conclusive results (extrapolatable) for the total population in the 4 target locations.
• While quantitative methodology is the primary approach, we’d want to incorporate a mixed method, capturing qualitative insights to support trends/results.
• Please note, finalizing the measurement plan will be an iterative approach, which Girl Effect and the chosen research agency will co-create, based on further updates on the national launch plan for HPV vaccines.

We would want to conduct a quantitative evaluation of the levels of knowledge, awareness and attitude of the target audience (primary audience: parents) towards vaccination and health-seeking behaviour and be able to isolate the impact of the GE intervention. At the onset of the project, we’d also like to understand the penetration levels of various digital media platforms for this target audience.

Girl Effect anticipates it to be a Baseline/Endline evaluation however, we are also open to reviewing other quasi experimental methodologies/approaches recommended by you.

Proposals must include methodology design details, representative sample size and sample calculations for conclusive impact results. We are looking for robust quantitative (and mixed methods) methodologies that can validate the impact of the programme on the total population.

The agency must be based or have substantive operations in India, have past expertise in immunization/vaccination work and must be conducive to working with teams in the UK & Sub-Saharan Africa.

Please include a proposed methodology design and budget for running evaluation for the following scenario:

Robust evaluation of 4 priority states

• Assume states with a larger population for this scenario, to understand the maximum scale and budget required for this research design
  • Do you have additional information based on past experience of the socio-ecological model of the states, vaccination uptake, presence of vaccine related initiatives or any other factors to suggest which 4 states can be prioritized? (this is not a mandatory question, but would appreciate it if you have point of view)
• Any other less robust/light touch measurement approaches you recommend for other states that can complement the programme’s evaluation and impact narrative.
• Girl Effect can potentially also support in facilitating online recruitment from the audience targeted in the online intervention – this would need to be further discussed.
• Proposed sample size with participant sampling calculations and approach
• Associated budget for this scale of research

Campaign Objectives

• To measure Girl Effect’s programme impact on the key barriers and enablers to immunisation among parents of girls 9-14 years old.
• To measure Girl Effect’s programme impact on the uptake of HPV vaccine for eligible girls

Evaluation Outcomes*

• Increased knowledge of parents: Parents of girls (9-14) have improved knowledge about the benefits, safety and availability of the HPV vaccination
• Increased intention of parents to get their daughters vaccinated: Parents want their child to get HPV vaccine when it rolls out in their state
• Increased knowledge and positive attitude of girls (9-14) towards HPV vaccination

*these outcomes are being validated currently with formative research and are likely to be modified further.

Tasks

The successful agency will be responsible for the following:

Design:

• Design the evaluation to meet Girl Effect’s requirements. We anticipate this will likely be a baseline and endline action/contributional research study, but we are open to other recommendations that deliver on objectives.
• Agreeing on the final design with Girl Effect’s evidence and insight team
• Design sample specification to be representative of target audiences
• Process and acquire required ethical approvals and obtain approvals at Regional, district, and ward-level public entities.
• Translate (and back translate) and contextualizing the data collection tools into local languages

Setup

• Recruit a suitable team of quantitative and qualitative researchers and field enumerators.
• Recruit a suitable team of female quantitative and qualitative researchers and field enumerators for gender-matched data collection with young women and girls.
• Facilitate training for all researchers, including a pre-testing exercise under real-field conditions
• Complete survey testing and scripting
• A post-pilot debrief and working sessions to refine and adapt the data collection following the pre-testing
• Sample and recruit communities and participants according to the provided purposive sampling criteria
• Obtain informed consent from all participants as well as consent from parents/caregivers of participants and assent from minors.
• Conduct quantitative and qualitative data collection in the selected districts, following strict quality assurance procedures and ensuring high standards of data management. All field worker management, logistics, etc, shall be the agency’s responsibility.

Fieldwork

• Capture and store all survey data using digital devices.
• Compile and submit field summaries with a breakdown per locality on agreed-upon frequency.
• Ensure all work is conducted according to the Girl Effect Girl Safeguarding Policy and an agreed safeguarding agreement.
• Facilitate all logistics to the field.
• Survey tools will need to include questions in accordance with global guidance on immunization, based on BeSD guidance and other resources.

Analysis and reporting

• Provision of raw and clean data sets
• Conduct analysis for both qualitative (if applicable) and quantitative data.
• Deliver draft reports for Girl Effect’s feedback and input
• Deliver final reports for Girl Effect and Gavi, along with video-call debriefs
• Additional, final report delivered to ethical approval boards as required

The agency will coordinate closely with Girl Effect to deliver work that delivers on all Girl Effect and Gavi reporting requirements. The Girl Effect Teams will conduct quality assurance during the fieldwork implementation. And review and agree on all major stages of the design and delivery.

Deliverables

• Evaluation Design (with GE’s support and input)
• Participant sampling
• Study Inception report
• Survey tool
• Consent and Assent forms
• Translations of survey tools into relevant languages and formats
• Risk assessment
• Apply and process for ethical approval certificates
• Training plan
• Fieldwork plan
• Raw data files for each stage (baseline and endline)
• Clean data files for each stage (baseline and endline)
• Data tables as per GE specification, including sig testing for each stage (baseline & endline)
• Baseline report
• Endline report
• Ethical approvals final report

Management

The agency will have a primary point of contact with our India Evidence and Insights Team. The language of the materials and reports shall be English. The contract supervisor shall approve all deliverables submitted by the agency before any payment is made.

Tentative Timelines – dependent on further updates on the HPV launch

• Agency procurement completed – June 2024
• Ethical approval submitted and achieved – July – August 2024
• Baseline (assumed methodology) fieldwork – August – September 2024
• Baseline (assumed methodology) reporting – October 2024
• Content running from December 2024 to October 2025 (assumed campaign dates)
• Endline fieldwork – December 2025
• All final reporting – March 2025

Who You Are

Skills Experience and Expertise:

• Extensive experience in designing and conducting quantitative and qualitative research
• Have past expertise in immunization/vaccination work and must be conducive to working with teams in the UK & Sub-Saharan Africa.
• Previous experience working in India and proficiency in conducting research in both Hindi and English languages.
• Capacity to deliver all areas expected of this research, from design through recruiting and training researchers to leading the data collection process in target field locations, ensuring quality assurance that aligns with research ethics principles, and delivering high-quality reporting.
• Experience designing and conducting quantitative studies in India, especially with adolescent girls, preferably in multiple regions.
• Experience conducting sensitive research with hard-to-reach, most vulnerable populations
• Experience in interviewing minors and adolescents and safeguarding policies and procedures
• Excellent and demonstrated understanding of Child Protection/ Safeguarding and ethical issues in research
• Excellent fieldwork supervision and data quality control strategies
• Ability to conduct a risk assessment and mitigation plans for research
• A fieldwork team of interviewers
• A fieldwork team of female interviewers for girls and young women interviews
• Proven record of effectively managing relationships with research partners
• Excellent interpersonal skills and demonstration of effectively managing relationships with research partners
• Excellent interpersonal skills and demonstration of effectively managing relationships with research partners
• Exceptional communication and organization skills
• Ability to respond to comments and questions in a timely, appropriate manner
• Experience in delivering outputs on time and budget
• Ability to write clearly and concisely in English
• The reputation of the responding agencies and any previous experience in similar surveys/research will also be considered.

Proposal Submission

Interested agencies are asked to submit the following to support their candidature (Max 10 pages):

Technical Proposal:

• Your understanding of the brief and why you feel you are well-placed to advise us on this
• Credentials – Please showcase any previous work that you consider relevant to what we are trying to achieve, how we approach our work, and the audience we cater to. Describe the qualifications, experience, and capabilities of the firm or consultant in providing the requested services (CV/Profile).
• An initial proposal for the design/Methodology
• Work Plan with clear timelines
• Reference – Provide at least three references for similar contracts with a description of the service provided, the value of the contract, and the contract periods of performance

In their technical proposal, the bidder must demonstrate an understanding of the requirements described in the RFP and demonstrate how the bidder will meet the requirements of the evaluation criteria. The technical proposal must be at most ten pages.

Financial Proposal:

• A breakdown of the financial proposal in INR indicating the daily rate for each of the proposed experts, time input, and all applicable reimbursable expenses
• All applicable taxes should be quoted separately.

Technical and Financial proposals will need to be submitted as separate documents. Financial bids will not be opened until the technical evaluation and only for those proposals deemed qualified and responsive. The proposals shall be prepared in English.

GE is not liable for any cost incurred during the award/contract preparation, submission, or negotiation of the award/contract. All submitted documentation and/or materials shall become and remain the property of GE.

The proposal’s VALIDITY shall be 90 days from the bid closure date.

Evaluation Criteria

The criteria against which proposals will be evaluated are listed below.

Technical Evaluation:

• A well-written capability statement clearly outlines your experience in delivering the Scope of Work and how you meet the ‘Who You Are’ requirements above – 20%
• An initial proposal for project design and methodology (to be finessed with the GE team upon contracting) – 20%
• Ability to achieve project goals/deliverables, i.e., does the proposal have strong feasibility in moving forward with the critical deliverables on schedule? -10%
• Demonstrate geographic experience in India (With a focus on the target locations)- 10%
• Clear and Concise Profile/CV demonstrating relevant expertise – 10%
• Evidence of a minimum of three contactable references – 10%

Financial Evaluation:

• Value for money/proposed budget breakdown – 20%

Procurement Timeframe

• Terms of reference published: 14th May
• Deadline for Questions/Clarifications: 22nd May
• Deadline for responses from GE: 29th May
• Proposal submission: 5th June 2024
• Supplier selection, contracting, and briefing: Mid-June 2024
• Project commencement: July 2024

Ethics

• The successful agency must adhere to Girl Effect protocols and safeguarding measures during all stages of research. This will ensure all girls’ participation will be conducted safely and securely.
• Ethical approval must be submitted to the relevant ethical board to ensure the research takes place with the proper permissions.
• Consent must be obtained from girls with consent forms that clearly state that at any time, research participants are free to decide to leave the research should they feel a reason to do so.
• Care also should be taken by the successful agency to maintain the confidentiality of the information provided by the respondents during the community. Informed voluntary consent should be taken before starting from the community, and parent/caregiver consent should be obtained for a minor. If an individual refuses to participate, they should not be compelled to participate or demoralized by any means. These cases will be treated as no-response under this survey. The research team must all undergo safeguarding training per Girl Effect’s safeguarding procedures and sign the Girl Effect safeguarding policy.
• All data must be stored in password-protected electronic files and shared with Girl Effect via a secure file transfer protocol (FTP)

Tax

Girl Effect Services India is obliged by the Indian tax authorities to ensure all taxes are charged where applicable. Applicants are advised to ensure that they have a clear understanding of their tax position regarding provisions of Indian tax legislation when developing their proposals.

Copyright

All materials/documents arising from this consultancy work shall remain the property of Girl Effect.

Disclaimer

GE reserves the right to determine the structure of the process, number of short-listed participants, the right to withdraw from the proposal process, the right to change this timetable at any time without notice and reserves the right to withdraw this tender at any time, without prior notice and without liability to compensate and/or reimburse any party. GE shall inform ONLY successful applicant(s). The process of negotiation and signing of the contract with the successful applicant(s) will follow.

Please note: We will evaluate only proposals submitted following the application process outlined in the RFP and using our specified email address ([email protected]).

Safeguarding

You may be required to undertake safeguarding checks. Shortlisted consultants will be assessed on our organizational values at the interview stage. The successful consultant will be expected to adhere to our safeguarding policy. We encourage you to read and understand our safeguarding policy, the executive summary of which can be found here. We have zero tolerance for violence against children, beneficiaries, and staff.

Equal Opportunities

Girl Effect Services is committed to equal opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, marital status, disability, gender, gender identity, or expression. We are proud to be an equal-opportunity workplace.

We are committed to building an increasingly representative organization that works extensively with the communities we serve. To this end, due regard will be paid to procuring consultancy service organisations and individuals with diverse professional, academic and cultural backgrounds.

How to apply

Please submit proposals, as described above, to Girl Effect’s procurement team ([email protected]) by the 5th of June 2024. Please mark your email with the subject line, ‘‘Proposal – India Programme Evaluation.’

Questions/Clarifications

If you have any questions about this RFP, please email [email protected]by 22nd May 2024. All questions will be answered and shared through an FAQ.