Final Evaluation – “Toward a Gender-Equitable Society” project

Search is seeking a consultant who can conduct a Final Evaluation to understand and investigate the extent to which the project has created an enabling environment in which social norms around VAWG and SGBV in Lebanon are shifted.

1. Context

The Project**

Toward a Gender Equitable Society is a project that is being implemented by Resource Center for Gender Equality (ABAAD) in partnership with Search, with the aim of reducing Sexual and Gender Based Violence (SGBV) including intimite partner violence and child marriage, and improving the prevention and response mechanisms in close partnership with existing national and governmental frameworks to ensure sustainability of the national system for SGBV prevention and response.

The project aims to do so by increasing the access of women, girls, men and boys to quality SGBV prevention and response services and enhancing the capacity of media professionals to produce gender-sensitive media products in Lebanon.

TOC: If media is used to break gender stereotypes AND survivors and those at risk of GBV have access to services, THEN an enabling environment will be created in which social norms around violence against women and girls will be shifted.

Under the first aspect of the project outcome – increased access to GBV prevention and response services – ABAAD aims to ensure and improve access to specialist services in two ways: first, through its Midway Houses, emergency shelters for women survivors of SGBV and their dependents; and second, through its Women and Girl Safe Spaces (WGSS), centers that provide prevention and response services for women survivors of SGBV (shelter, legal, psychotherapy, psychosocial, case management, and medical care) and those at risk. In addition, ABAAD aims to engage community members, both men and women, in dialogues on SGBV through its Mobile Unit, which will visit communities located farther from other direct SGBV services.

Simultaneously, under the second aspect of the project outcome – enhancing the capacity of TV/film media to produce gender and GBV-sensitive media products – Search aims to engage media professionals and students on topics related to gender-sensitivity in the media and the ethical representation of situations and storylines that include scenes of SGBV and breaking gender stereotypes. Through training workshops and partnerships with media companies, Search’s activities will include a media for social change workshop to student filmmakers and producers, and mentoring and coaching sessions which will lead to the production of gender-sensitive short films and TV shows, as well as a new generation of students and young professionals with increased knowledge of and capacity in implementing ethical guidelines and principles related to portraying gender and SGBV in the media.

Target groups

Search target groups: 16 film students, media professionals from the production companies and the wider audience from the campaign

2. Goal and Objectives of Study

• Search is seeking a consultant who can conduct a Final Evaluation to understand and investigate the extent to which the project has created an enabling environment in which social norms around VAWG and SGBV in Lebanon are shifted. The study will focus only on Search’s contribution to the project which is the second aspect described above.

The Final Evaluation should enable Search to inform future programme design and will assess the overall success and effectiveness of the implemented project. In specific terms, The Final Evaluation has the following objectives:

1. Assess the intended (according to the project’s logical framework) and the unintended outcomes (positive or negative) and determine Search contribution to identified changes.
2. Update the project’s logframe and indicators with endline values;
3. Assess whether the project produced the effects as envisioned in the Theory of Change in addition to assessing the project’s relevance, efficiency, effectiveness and impact, and sustainability as per the Organization for Economic Co-operation and Development – Development Assistance Committee (OECD DAC) evaluation criteria in addition to the overall project’s adaptability;
4. Assess whether the project is aligned with Search Levant Strategy objective 4 “To enable young people to independently develop multi-dimensional identities” and assess its indicators; and
5. Capture good practices, compile lessons learned throughout the lifespan of the project, and draw specific recommendations to inform future programme design in a similar context and on leveraging media to promote pluralism, diversity, and multi-dimensional identities.

3. Key Questions of the Study

• The Final Evaluation should support the team to answer the key questions below. The study is expected to prioritize collecting feedback from direct and indirect beneficiaries including film makers, producers, and community members.

Relevance:

• To what extent did the intervention’s objectives and design respond to promote an enabling environment in Lebanon which allows the shifting of social norms?
• To what extent did the intervention’s objectives and design of Search’s contribution respond to promoting access to safe and quality prevention and response services to SGBV and VAWG?
• To what extent has ‘do no harm’, gender sensitivity and barriers been taken into consideration in the design and implementation and what were the effects?
• Have experiences from previous interventions been used successfully to improve the overall quality of the project?
• To what extent did the intervention’s objectives and design contribute to Search Levant Strategy?

Effectiveness and Impact:

• To what extent has intervention been effective in meeting the essence of the Theory of Change and achieving its results and planned outcomes?
• What unintended outcomes (positive and negative) emerged from project implementation?
• What factors (positive and negative) have had the greatest influence on the achievement of results?
• To what extent has the intervention been able to contribute to longer-term effects (impact)?
• Were the criteria of selection (media professionals and production houses) appropriate to support the effective achievement of the intervention’s objectives?

Efficiency and Value for Money:

• To what extent have resources been allocated and utilised in an efficient manner to achieve value-for-money?
• How efficient was the delivery of the intervention in terms of expenditure and implementation of activities?

Sustainability:

• To what extent are the benefits of the program likely to sustain in the long term? What are the main factors behind this?

Cross Cutting Issues:

• How and to what degree did the project adapt in response to monitoring data and contextual factors and/or changes?
• What efforts have the project made to adapt to these changes, if any?
• To what extent have these adaptations been effective and adequate?

Lessons Learned:

• What lessons can be learned or good practices can be identified from the implementation of the project?

4. Geographic Locations

• The Final Evaluation is expected to provide quantitative and qualitative information, gender-age-nationality disaggregated data and high-level analysis across Lebanon.

5. Methodology and Data Collection Tools

• The Final Evaluation will utilise a mixed-method approach consisting of both qualitative and quantitative primary and secondary data collections and sources. The Final Evaluation will be conducted to triangulate data from multiple sources and stakeholders in order to infer reliable findings. In addition to the proposed conventional approach the consultancy team will be taken to carry out the evaluation through using the evaluation questions (section 3) included in the ToR, the consultancy team are expected to integrate and apply ‘Outcome Harvesting’ techniques to identify the most significant results and the project’s contributions to these changes. The level of integration should be defined clearly in the proposed methodological approach in their technical proposal.

Firstly, the consultant(s) will review key project documents including the proposal, baseline, MEL framework, log frame, and primary available monitoring and evaluation data that was formerly collated by project staff including activity surveys and assessments, and set meetings with key personnel to better understand the project, its goal, and activities. The consultant is expected to review the above background documentation as part of the desk review phase of the study among other relevant and available literature.

Secondly, the consultant will develop quantitative and qualitative data collection tools and collect data through surveys, interviews, focus group discussions (FGDs), social media and possibly case studies. In addition to other participatory methods of Outcome Harvesting e.g. validation/substantiation workshops. The evaluation should use a solid and representative sampling approach and participatory methods with project beneficiaries and other stakeholders. The consultant(s) shall propose the team set up for data collection).

The consultant shall determine the appropriate sample size and approach in consultation with Search’s MEAL Manager and Project Manager and referring to the M&E plan. The sampling strategy will take into consideration carried out activities, target population and the project areas (95% Confidence Level and 5% Margin of error to the degree possible) and other contextual consideration to accessibility. S/he will meet with the project participants, Search staff and other relevant stakeholders.

The methodology and data collection tools shall be validated by Search’s MEAL Manager.

The Consultant(s) is also required to develop a detailed evaluation matrix with judgement criteria that captures the study’s objectives, evaluation questions of which will guide the evaluation process into reliable judgments and analysis.

Data collection, analysis, and results should represent inclusiveness—the methodology should include a wide range of viewpoints, specifically gender and age sensitivity when applicable.

Deliverables

Search expects the following deliverables from the external consultant(s) as they correspond to the timeline and budget:

• An inception report detailing the methodology, data collection tools and timeline;
• Any necessary training of data collectors or set-up of systems for data collection;
• All original data will be submitted to Search;
• A draft endline report in English for review by Search staff and partners to be presented and discussed with Search’s MEAL and project team; and
• A final endline report in English (40 pages max in length, excluding appendices) consistent with Search branding and standards for evaluation. The report:
• Uses the Search reporting template unless otherwise agreed in the contract;
• Provides a clear connection between the conflict or context assessment and the intended results, articulate the project’s ToC, and include other relevant project specifics;
• Fully explains the objectives and research questions of the study, limitations and methods chosen for analysis;
• Findings respect Search’s evaluation standards, are structured around the main objectives of the study, and are presented in relation to the intended target groups. The findings should speak to the link between the project, it’s Theory of Change and its contribution to our strategy. It should explain adaptations that occurred during the project and their impact on results;
• Recommendations should have a clear audience and be specific, accessible, and actionable.
• Indicator table showing all indicators;
• Appendices should include detailed research instruments, list of interviewees, terms of references and evaluator(s) brief biography.
• A one to two page document with key project lessons that Search can use in its business development efforts; and
• A PowerPoint presentation and the actual presentation of results to Search and the donor.

6. Logistical Support

• The firm/consultant(s) will be responsible for organizing their own logistics for data collection (vehicles, fuel, drivers, and communication), and this must be budgeted into the financial proposal. Search can provide support in arranging logistics as will be agreed upon with the consultant(s) based on the technical proposal.

In addition, Search will share the following elements with the external consultant(s): Background materials including the project proposal and logframe.

7. Ethical Considerations

• The consultancy should take consideration of the rights of participants in line with the ethical codes and guidelines of research. The participants should be asked to provide their consent and made aware that their identities would not be revealed. Cultural rights will be respected; minors below 18 years of age would not be assessed while collected data would be protected for confidentiality all through the data management process.

The firm/consultants are required to respect the following ethical principles:

• Comprehensive and systematic inquiry: Consultant should make the most of the existing information and full range of stakeholders available at the time of the review. Consultant should conduct systematic, data-based inquiries. He or she should communicate his or her methods and approaches accurately and in sufficient detail to allow others to understand, interpret and critique his or her work. He or she should make clear the limitations of the review and its results.
• Competence: Consultant should possess the abilities and skills and experience appropriate to undertake the tasks proposed and should practice within the limits of his or her professional training and competence.
• Honesty and integrity: Consultant should be transparent with the contractor/constituent about: any conflict of interest, any change made in the negotiated project plan and the reasons why those changes were made, any risk that certain procedures or activities produce misleading review information.
• Respect for people: Consultant should respect the security, dignity and self-worth of the respondents, program participants. Consultant has the responsibility to be sensitive to and respect differences amongst participants in culture, religion, gender, disability, age and ethnicity.
• Safeguarding: Remain alert and responsive to any child safeguarding risks, acquire relevant knowledge and skills which will enable promoting strong safeguarding practices, understanding the child safeguarding policy and procedures.

The assignment should take into account precautionary measures to avoid Covid-related complications during data collection. These measures should ensure social distancing and remote interactions where possible. Therefore, applicants are kindly required to integrate a detailed multiple scenario strategy in their proposed methodologies and work plans that would ensure efficacy in the delivery of the assignment and allow for flexibility to switch between different modalities (i.e., to virtual data collection and vice-versa) whenever there is a need to do so.

8. Data Quality Assurance and Management

• All deliverables will be reviewed and approved by the country office and global Institutional Learning Team prior to acceptance of the final product. The consultant(s) are also required to develop a clear and rigorous quality control plan during the inception phase of which will be implemented and deployed throughout all phases of the evaluation process to ensure credibility and validity of findings and usefulness of recommendations.

9. Timeline

• The evaluation timeline is expected to be kicked off first of May 2022 and concluded at the end of June 2022 by submitting the final deliverables requested above.

10. Budget

• The total budget available for this study is USD 17,000**.** A detailed budget should be provided, including daily rates for personnel, and costs related to data collection (per total number of people sampled, sites for collection, etc.), analysis, and production of deliverables. The selection committee will give preference to the applicant(s) with a relatively lower budget and higher quality technical proposal.

11. Requirements of Consultant

• Education:

Lead consultants must have at least a Master’s degree (PhD preferable) in research methods and/or evaluations, development, gender and media expert, conflict or peacebuilding, M&E or any related social sciences.

Skills and experience:

The following skills and experience are expected by Search for our evaluator for this project:

• Excellent proficiency in English and Arabic – written (including professional English reporting) and spoken;
• More than five years of experience in programme/project evaluation, including collecting data in interviews, surveys and focus groups and analysis; Social media for change analysis;
• Experience in conflict analysis and working with the gender and protection sectors;
• Experience working with international organizations;
• Experience conducting quantitative surveys and analysis;
• Monitoring and Evaluation methods and data collection skills; and
• Familiarity and experience with contextual challenges in the geographic location(s) where the study will take place.

12. Selection Criteria

• Consultant proposals will be selected for:
• Relevance of proposed methodology to the goal, objectives, and research questions for the project.
• Quality of proposed methods, conflict sensitivity approaches, and quality control measures.
• Qualifications of the candidate(s).
• Proposed budget in relation to proposed methodology, deliverables and team.
• Timeline for proposed activities.