Endline Evaluation Study Partners for Justice Project

1. Context

Search for Common Ground

Search for Common Ground (Search) is an international conflict transformation non-governmental organization that aims to transform the way individuals, groups, governments, and companies deal with conflict, away from adversarial approaches and towards collaborative solutions. Headquartered in Washington DC, USA, and Brussels, Belgium, with 52 field offices in over 30 countries, Search designs and implements multifaceted, culturally appropriate, and conflict-sensitive programmes using a diverse range of tools, including media and training, to promote dialogue, increase knowledge and determine a positive shift in behaviors.

The Project

Despite the various efforts made in Lebanon to achieve gender equality and end violence against women and girls (VAWG), the situation remains troubling. In 2021, Lebanon ranked 132th out of 156 countries in the world on the World Economic Forum’s global gender gap index and ranked 16h among the 19 MENA countries (World Economic Forum (2021), Global Gender Gap Report Insight Report). Patriarchal social structures and practices limit women and girls’ access to economic resources and to political and leadership positions, while reinforcing sexual and gender-based violence in public policy. While Lebanon has a developed civil society and empowered women activists their actual inclusion in the governance and political processes is still lacking due to the patriarchal nature of the system.

Theory of Change: IF men and boys are provided with a safe platform where they can deconstruct and change their own attitudes around the socially built norms of masculinities that push men and boys to play the role of “protector” and “controller” of women and girls in the public and private space, AND they understand and embrace the role men and boys can play to contribute to shifting communities’ attitudes toward the permissibility of violence against women and girls, THEN they will become women’s and girls’ allies, model positive behaviors, ultimately contributing to changing negative social norms around violence against women and girls in Lebanon.

The Partners for Justice project’s overall objective is to reduce violence against VAWG in Lebanon. Search and ABAAD-Resource Center for Gender Equality (ABAAD) have built on their expertise and experiences to raise awareness on gender equality and equitable masculinities and shape attitudes of men, women, boys and girls around VAWG, and to create opportunities for men and boys to become allies in ending VAWG. This has been supported by the following specific objectives:

• SO.1 Engage men and boys to positively work with women and girls to become allies and champions of gender equality and prevention of violence against women and girls;
• ER 1.1: Youth organizations are more aware of issues and assumptions related to the social constructions of masculinities and violence against women and girls;
• ER 1.2: Youth organizations show increased capacity to reflect on masculine identities and gender equality in their communities.
• SO.2: Foster a change in communities’ perceptions and attitudes in target areas around violence against women and girls and all of the assumptions related to it.
• ER 2.1: Media stakeholders and youth organizations are better able to address negative social dynamics that justify violence against women and girls and/or perpetuate victim-blaming; and
• ER 2.2: Target communities and audiences, especially male members, change attitudes towards the permissibility or justification of violence against women and girls.

Under its first objective, the project has focused on working with youth organizations and their representatives by providing them with knowledge and building their capacities on gender and masculinities, as well as on participatory youth-led research ethics, methodologies, and approaches. With these activities being done, the youth representing the organizations in the project have demonstrated an increase in their skills on exploring and analyzing narratives of masculinities in the contextual settings where the youth organizations are working, and contributing to providing men and boys with a safe space to reflect on their own attitudes and behaviors. Of note, selected representatives from youth organizations already carried out research in the target geographies to identify local-level barriers to positive and equitable masculinities and gender equality as well as to identify characteristics of alternative masculinities the project seeks to encourage. The findings of this research were published in December 2021 (Participatory, Youth-Led Gender and Masculinities Research, SFCG, December 2021. https://www.sfcg.org/wp-content/uploads/2021/12/Participatory-Youth-Led-Gender-and-Masulinities-Research.pdf). The first objective also included a training of trainers on Program Ra- a training consisting of modules around gender, masculinities, and sexual and reproductive health and rights, which the participants then delivered to community members.

Under the project’s second objective, Search and ABAAD selected media professionals to participate in a workshop on media for social change, along with the youth representatives. Afterwards, the youth-led research on gender and masculinities (completed under SO1), the series of working sessions with youth and media professionals, and the knowledge sharing opportunities collectively fed into the creation and design of a national awareness media campaign that is relevant to Lebanon rural and urban contexts. Initiatives aiming at raising awareness around gender and masculinities were also implemented and led by the youth participants across Lebanon. The campaign aimed to 1) enhance national and regional knowledge and understanding on masculinities and VAWG through a youth-led approach; 2) influence policy and decision-making processes by targeting key influencers; 3) create a momentum for local and international organizations to support youth in their cutting-edge research findings and research-informed campaign and 4) inform and support future work on engaging men and boys in preventing VAWG drawn from empirical research and contextualized definitions and understandings of masculinities. The National Media Campaign was launched in November 2021 across social media platforms.

Target groups
The primary target groups are 30 youth representatives selected from 16 CSOs targeting the Lebanese population in five geographic areas: Beirut, Mount Lebanon, Bekaa, South, and North. The secondary target group are 10 media professionals who have participated in the media for social change working sessions and helped develop a national awareness media campaign. The tertiary target group is the wider audience reached by the national campaign (through TV broadcast, initiatives, billboards etc) including men (male local leaders, religious leaders, policy makers, educators, social workers, male family members), boys, women, and girls from a variety of backgrounds, who were encouraged to challenge their own perceptions on masculinities and VAWG, and were provided with safe spaces to constructively discuss these topics.**2. Goal and Objectives of Study**

Search is seeking a consultancy team/firm who can conduct an Endline Evaluation to understand the extent to which the project has contributed to reducing VAWG in Lebanon. This study will also aim to assess the project’s progress towards its specific objectives.

The Endline Study should enable Search to inform future program design and will assess the overall success and effectiveness of the implemented project. The evaluation will also identify the added value, or comparative advantage of the intervention by understanding the degree to which the intervention has made a difference, positively or negatively, beyond the sheer of expected results and planned objectives. The evaluation therefore will apply Outcome Harvesting activities, which will be intensified into the final report of the Endline Evaluation In specific terms, the Endline Evaluation has the following objectives:

• Assess the overall impact of the project based on evidence-based outcomes whether intended (according to the project’s logical framework) or unintended outcomes (positive or negative) and determine Search’s contribution to identified changes;
• Update the Project’s log frame and indicators with End-line values;
• Assess whether the project produced the effects as envisioned in the Theory of Change in addition to assessing the project’s Relevance, Efficiency, Effectiveness and Impact, Coherence, and Sustainability as per the Organization for Economic Co-operation and Development – Development Assistance Committee (OECD DAC) evaluation criteria, in addition to the overall project’s Adaptability;
• Assess whether the project is aligned with objective 4 of Search’s Greater Levant Strategy “To enable young people to independently develop multi-dimensional identities” and assess its indicators; and
• Capture good practices, compile lessons learned (i.e. ending September 2022), and draw specific recommendations to inform future programme design in a similar context.

This study is expected to give specific focus to assessing the progress, effectiveness and relevance of the activities done with the youth organizations’ representatives and media professionals. It also focuses on assessing the impact of the national media campaign in target communities. The activities conducted are capacity building training on conducting gender equality and positive and equitable masculinity training, gender and masculinities research, training on conducting Sexual and Reproductive Health and Rights training, implementing gender-sensitive initiatives across Lebanon, and implementing a national awareness and advocacy media campaign, among others.**3. Key Questions of the Study**

Evaluation Questions

The analysis will be focused on the following evaluation questions organized by criteria. The study is expected to prioritize the collection of feedback from direct and indirect right holders from the respective youth and media, and community members.**Relevance**:

• EQ1: To what extent has the intervention’s objectives and design responded to reducing violence against women and girls in Lebanon so far?
• EQ2: How effectively were barriers specific to this project taken into account in the design and implementation of the project?
• EQ3: In light of the recent developments that occurred in Lebanon, are the objectives of the intervention still valid?
• Are the activities of the intervention still consistent with the overall goal of the project and fulfilling its specific objectives?
• EQ4: Have experiences from previous interventions been used successfully to improve the overall quality of the project?

• EQ5: To what extent did the intervention’s objectives and design aligned with and contribute to Search’s Levant Strategy?**Effectiveness and Impact:**

• EQ6: To what extent has the intervention been effective in meeting the essence of the Theory of Change and achieving its intended results so far?

• Extent to which the intervention has achieved its planned outcomes; and

• Extent to which the intervention outputs (activities) contributed to the achievement of the planned outcomes.

• EQ7: What unintended outcomes (positive or negative) emerged from project implementation?

• EQ8: What factors (positive and negative) have had the greatest influence on the achievement/progress towards the desired change/objectives?

• To what extent has the intervention been able to contribute to longer-term effects (impact) so far?

• EQ9: Was the level of partnership(s) appropriate to support the effective achievement of the intervention’s objectives?**Efficiency and Value for Money:**

• EQ10: To what extent have resources been allocated and utilized thus far in an efficient manner to achieve value for money?

• How efficient was the delivery of the intervention in terms of expenditure and implementation of activities?**Sustainability**:

• EQ11: To what extent has the project advanced in building mechanisms to sustain the benefits produced thus far? What are the main factors behind this?

• What are efforts required to improve and strengthen sustainability?

• EQ12: To what extent did the project stakeholders (youths and youth organizations) show efficiency and ownership towards the project?**Coherence**:

• EQ13: To what extent has the project been coherent with other major interventions in the VAWG field in Lebanon? To what extent the intervention is adding value while avoiding duplication (complementarity)?

• EQ14: Existence of effective coordination and communication mechanisms established by Search and the extent to which partner organizations have been effectively engaged in them?**Cross-Cutting Issues:**

• EQ15: To what extent have gender issues, persons with disabilities, Do No Harm, and conflict sensitivity been taken into consideration in design and implementation and what are the effects?**Adaptability**:

• EQ16: How and to what degree did the project adapt in response to reflection and learning outputs, monitoring data, and contextual factors and/or changes?

• What efforts have the project made to adapt to these changes, if any?

• To what extent have these adaptations been effective and adequate?

The evaluation questions and methodology for this assignment may need to be further elaborated by the experts in the inception report, particularly with planning of the use of OH (see Methodology section). The expert(s)/consultant(s) may suggest additional questions. The final version of the evaluation questions will be agreed at the end of the inception phase.**4. Geographic Locations**

The Endline Evaluation study will provide quantitative and qualitative information, gender-age and area disaggregated data, and analysis in Lebanon in the following areas: Beirut, Mount Lebanon, North, South, and Bekaa.**5. Methodology and Data Collection Tools**

The Endline Evaluation study will utilise a mixed-method approach consisting of both qualitative and quantitative primary and secondary data collections. The study should triangulate data from multiple sources and stakeholders in order to infer reliable findings. In addition to the proposed conventional approach the consultancy team will be taken to carry out the evaluation through using the evaluation questions (section 3) included in the ToR. The endline is expected to integrate and apply ‘Outcome Harvesting’ techniques to identify the most significant results and the project’s contributions to these changes.

Desk review: the consultant(s) will review key project documents including the proposal, the baseline report, the Youth-Led Research, MEL framework, Reflection and Learning outputs, log frame, and set meetings with key personnel to better understand the project, its goal, and activities. The consultant is expected to review the above background documentation as part of the desk review phase of the study, in addition to the literature review about the Gender and VAWG context in Lebanon. Also, the consultant will utilise primary available M&E data that was formerly collated by project staff including activity surveys and assessments, baseline data, project quarterly reports, etc.

Data collection tools and approach: the consultant will develop quantitative and qualitative data collection tools and collect data through surveys, interviews, focus group discussions (FGDs), and possibly Case Studies using a solid and representative sampling approach. To ensure a better understanding of the key issues that will be addressed in the project, the data will be disaggregated by age, nationality and sex, as well as by geographical location.

Sampling strategy and approach: The consultant shall determine the appropriate sample size and approach in consultation with Search MEAL Manager and referring to the M&E plan. The sampling strategy will take into consideration the activities carried out, target population and the project areas (95% Confidence Level and 5% Margin of error). The consultant(s) will meet with the project participants, partners, Search staff and relevant stakeholders.

Outcome harvesting approach: In addition to the proposed conventional approach of the evaluation, the evaluation is expected to integrate and apply, where feasible steps 2-5 of the six-steps of ‘Outcome Harvesting’(https://www.betterevaluation.org/en/plan/approach/outcome_harvesting ) to answer the questions of what actual achieved results are as opposed to those are planned, what has changed, when and how, identifying the most significant results, and assessing project’s contributions to these changes. Step 6 will be limited to and where feasible, the presentation of actionable recommendations in the evaluation report. The level of integration should be defined clearly in the proposed methodological approach of the consultant(s) technical proposal.

Evaluation matrix: The consultant(s) is also required to develop a detailed evaluation matrix with judgment criteria that captures the study’s objectives and evaluation questions of which will guide the evaluation process into reliable judgments and outcomes.

Search’s MEAL Manager will validate and approve the final methodology and tools before data collection begins, as well as provide feedback on the report.

Data collection, analysis, and the results should represent inclusiveness—the methodology should include a wide range of viewpoints, specifically gender and age sensitivity when applicable.**6. Deliverables**

Search expects the following deliverables from the external consultant(s) as they correspond to the timeline and budget:

• An inception report detailing the methodology, data collection tools, analytical framework, approach and timeline;
• Developed or used training materials for data collectors or set-up of systems for data collection;
• All original data set should be submitted to Search;
• Outcome Harvesting Workshops;
• A draft endline report in English for review by Search staff and partners;
• A final report in English (40 pages max in length, excluding appendices and the list of outcomes and contribution statement) consistent with Search branding and standards for evaluation. The report will:
• Uses the Search reporting template unless otherwise agreed in the contract;
• Provides a clear connection between the conflict or context assessment and the intended results, articulate the project’s ToC, and include other relevant project specifics;
• Fully explains the objectives and research questions of the study, limitations, and methods chosen for analysis;
• Findings respect Search’s evaluation standards, are structured around the main objectives of the study, and are presented in relation to the intended target groups. The findings should speak to the link between the project, it’s TOC and its contribution to our strategy. It should explain adaptations that occurred during the project and their impact on results;
• Recommendations should have a clear audience and be specific, accessible, and actionable;
• Indicator table showing all indicators;
• Appendices should include detailed research instruments, a list of interviewees, terms of references and evaluator(s) brief biography and list of outcomes and contribution statement
• A PowerPoint presentation that will be used by the consultancy team to present the findings to Search and the donor after the final report is approved;
• A one to two hours online presentation of research findings to the Search and donor team; and
• A two-pager summary highlighting key lessons learnt in this project that will inform Search and the Gender and VAWG programming in the future.
  7. Logistical Support

Regardless of the modality chosen, the firm/consultant(s) should be fully responsible for the data collection, including working with data collectors to be recruited and trained by them in case needed. In case of in-person data collection, the firm/consultant(s) will be responsible for organizing their own logistics for data collection (vehicles, fuel, and drivers). All of this must be budgeted into the financial proposal.

In addition, Search and partners will share the following elements with the external consultant(s): background materials including the project proposal and logframe, etc.**8. Ethical Considerations**

The consultancy should take into consideration the rights of participants in line with the ethical codes and guidelines of research. The participant should be asked to provide their consent and made aware that their identities would not be revealed. Cultural rights will be respected.

The firm/consultants are required to respect the following ethical principles:

• Comprehensive and systematic inquiry: The consultant should make the most of the existing information and full range of stakeholders available at the time of the review. Consultant should conduct systematic, data-based inquiries. He or she should communicate his or her methods and approaches accurately and in sufficient detail to allow others to understand, interpret and critique his or her work. He or she should make clear the limitations of the review and its results;
• Competence: Consultant should possess the abilities and skills and experience appropriate to undertake the tasks proposed and should practice within the limits of his or her professional training and competence;
• Honesty and integrity: Consultant should be transparent with the contractor/constituent about: any conflict of interest, any change made in the negotiated project plan, and the reasons why those changes were made, any risk that certain procedures or activities produce misleading review information; and
• Respect for people: Consultant should respect the security, dignity, and self-worth of the respondents, program participants. The consultant has the responsibility to be sensitive to and respect differences amongst participants in culture, religion, gender, disability, age, and ethnicity.
• Safeguarding: Remain alert and responsive to any child safeguarding risks, acquire relevant knowledge and skills which will enable promoting strong safeguarding practices, understanding the child safeguarding policy and procedures.

The assignment should take into account precautionary measures to avoid Covid-related complications during data collection. These measures should ensure social distancing and remote interactions where possible. Therefore, applicants are kindly required to integrate a detailed multiple scenario strategy in their proposed methodologies and work plans that would ensure efficacy in the delivery of the assignment and allow for flexibility to switch between different modalities (i.e., to virtual data collection and vice-versa) whenever there is a need to do so.**9. Data Quality Assurance and Management**

All deliverables will be reviewed and approved by the country office and global Institutional Learning Team prior to acceptance of the final product. The consultant(s) are also required to develop a clear and rigorous quality control plan during the inception phase of which will be implemented and deployed throughout all phases of the evaluation process to ensure reliability and validity of findings.

To ensure effective data management, all the qualitative collected data (through surveys, FGDs, KIIs) will be transcribed with the respect to the data collection ethics. All the quantitative surveys will be cleaned and stored in a secured database. Access to the data will be allowed to the staff members who work on the final evaluation and will be limited to those who have the password.**10. Timeline**

The evaluation timeline is expected to be kicked off in July 2022 and concluded by the end of September 2022, by submitting the final deliverables requested above.**11. Budget**

The total budget available for this study is 25,000 USD. A detailed budget should be provided, including daily rates for personnel, and costs related to data collection (per total number of people sampled, sites for collection, etc.), analysis, and production of deliverables. The selection committee will give preference to the applicant(s) with a relatively lower budget and higher quality technical proposal.**12. Requirements of Consultant(s)**

Education:

• Lead consultant(s) must have at least a Master’s degree (PhD preferable) in research methods and/or evaluations, development, conflict or peacebuilding, M&E or any related social sciences. Skills and experience:
  The following skills and experience are expected by Search for our evaluator for this project:

• Excellent proficiency in English and Arabic – written (including professional English reporting) and spoken;

• More than five years of experience in programme/project evaluation, including collecting data in interviews, surveys and focus groups and analysis;

• Experience in conflict analysis, working with justice, governmental and gender related topics;

• Strong understanding of concepts related to gender and masculinities;

• Experience in doing evaluations for peacebuilding and democracy, human rights and governance programmes in MENA region;

• Experience working with international organizations;

• Experience conducting quantitative surveys and analysis;

• Monitoring and Evaluation methods and data collection skills; and

• Familiarity and experience with contextual challenges in the geographic location(s) where the study will take place.