Consultants to conduct the End-term Evaluation for the Women and Girls Rebuilding

1. Background

Oxfam is a global movement of people working together to end the injustice of poverty. Together we save, protect, and rebuild lives. And we won’t stop until every person on the planet can defeat poverty or any type of inequality or discrimination. We are an international confederation of 20 organizations (affiliates) working together with partners and local communities in the areas of humanitarian, development, and campaigning, in more than 90 countries. All our work is led by three core values: Empowerment, Accountability, and Inclusiveness.

Iraq’s humanitarian context remains fragile. The country has been affected by multiple crises, including decades of cyclical violence, displacement, a contracting economy, the activities of armed groups, poor provision of public services and challenges to governance. In early 2020, Iraq was affected by twin shocks to its economy, with the COVID-19 pandemic and the drop in oil prices, which have contributed to growing socioeconomic vulnerabilities. Overall, the social, ethnic, and sectarian tensions will continue to contribute to the country’s insecurity. Furthermore, the current political landscape of Iraq is unstable and evolving following the early parliamentary elections that were successfully held in October 2021, with the level of turning out to vote estimated at 40%.

Despite these constraints, the humanitarian context in Iraq has gradually transitioned into a new phase allowing humanitarian actors to plan and implement recovery and development interventions addressing community reconciliation, resilient livelihoods, as well as participation in governance structures and decision-making. The situation of many women and girls requires particular attention. The crimes committed by ISIL have left deep psychological scars on many women and their families. Displaced women continue to suffer sexual and gender-based violence in both in-camp and out-of-camps settings even while the military offensive by ISIL is over. Despite some progress in the legislative protection of women’s equal rights, most of the women in Iraq are affected by long-term social, economic and political exclusion exacerbated by deeply entrenched patriarchal values held by a large proportion of the population.

1. Women and Girls Rebuilding Iraq

The intervention (hereafter “project”) entitled Women and Girls Rebuilding Iraq implemented by OXFAM in Iraq contributes to shaping policy, decision-making mechanisms and peacebuilding processes at multiple levels with the specific objective to ensure women and girls play a significant role in the new rehabilitation and development agenda. The initiative is working with women and girls in the Kirkuk and Diyala governorates, both of which have endured a protracted conflict. Part of the project also takes place at the national level involving public policy engagement and advocacy on women’s and girls’ rights. The project is implemented in cooperation with Women’s Rights Organizations (WROs) promoting transformative leadership in formal and informal governance structures. The initiative seeks to engage 9,000 vulnerable women and girls in Iraq. In addition, 40 men and boys from local communities are to be trained to act as role models for progressive masculinities. They will act as change agents in shifting attitudes and practices that hinder women’s equal rights.

Two intermediate outcomes have been envisioned for the project:

• Increased active participation of women and girls in public decision-making processes as transformative leaders.
• The improved social and political environment for the effective participation of women and girls in government structures.

Four immediate outcomes include the following:

• Increased access to leadership roles for women and girls in governance structures and better inclusion of women’s and girls’ needs and interests
• Increased support by community members, community leaders, and other opinion influencers (media, academia, practitioners, etc.) on women’s and girls’ participation in governance structures and decision-making processes
• Increased capacities of WROs to engage with decisions and policymakers and design gender-responsive policies integrating women’s and girls’ needs and interests
• Increased skills of WROs to design and implement gender-responsive programming and advocacy to support women’s and girls’ voices

The project works at 3 levels:

• Micro: individual capacity building and community engagement (involving both women and men);
• Meso: working with interlocutors, WROs and women’s rights defenders to improve the well-being of women and girls;
• Macro: work with opinion formers and policymakers for increased active participation of women and girls in governance structures.

1. Project status as of October 2022

The inception period took place in the first six months of the project. The COVID-19 pandemic affected the initial implementation in Year 1 requiring the team to request a 12-month No Cost Extension (NCE). As per NCE, the end of the project is expected in June 2022 with a 3-month closeout period ending in September 2022. The final evaluation will take place during the closing period. In terms of the activities undertaken in Year 1 (March 2019 – March 2020), the Project Implementation Plan was developed and validated by the donor in January 2020. Several studies were carried out in Year 1 including a study on barriers for women and girls to participate in governance and decision-making; a study on social norms and masculinities; a gender analysis; an outline of recommendations based on the Northern Iraq Livelihoods Assessment report; and adaptation of the Gender Action Learning Systems (GALS) methodology. These outputs contributed to rolling out awareness-raising sessions for women and girls at the micro-level along with community dialogues, awareness-raising sessions for men and boys, and GALS sessions. At the meso level, the project supported the capacity building of two implementing partners focusing on the partners’ capacity to engage in gender justice programming. Activities at the macro-level related to policy-related outputs largely on campaigns around international women’s days, the Anti Domestic Violence Law, CEDAW reporting and other campaigns run by Oxfam’s partnering WROs. The second phase of Women and Girls Rebuilding Iraq implemented after the parliamentary elections scheduled for October 2021. During this period, the project rolled out a Leadership Programme for women rights defenders to train them on requisite skills in policy and advocacy to enable them to conduct gender transformative campaigns with a more unified messaging and goal. Additionally, the project was allowed to implement 60 more MRM sessions and distribute grants to an additional 90 women after meeting the initial base target of 100 women. The project will finalise the implementation of activities by the end of December 2022 and a close-out period that ends in March 2023.

1. Objectives of the end-term Evaluation

The main objective of the End-term evaluation is to systematically review the overall achievement of the project, with a strong bias to the impact created towards the project’s outcomes, with detailed analysis of and provide recommendations on the overall design of the project, modifications, and specific actions that might be taken into consideration in future gender justice programming.

The final evaluation will also test the TOC of the project and the assumptions that were defined in the project design and how they influenced the change tracks. It will further provide an analysis of the project activities and how they contributed to achieving the project outcomes and outputs, and an analysis of challenges and opportunities that might have happened during the project duration. It will also capture the unintended results and unseen factors of change.

The evaluation shall conform to OECD/DAC’s Quality Standards for Development Evaluation. It will further identify and assess key internal and external factors that have contributed, affected, or impeded the achievements of the project, and how Oxfam and the partners have managed these factors. The objective of this evaluation is to establish the relevance, effectiveness, efficiency, impact, and sustainability of implemented actions.

Specifically, the consultant will:

1. Assess the project progress against the intermediate and immediate outcomes as outlined in the project logic model.
2. Review the indicators and update the Endline qualitative and quantitative values in the Performance Measurement Framework (PMF).
3. Apply Development Assistance Committee (DAC) methodology as the evaluation criteria: relevance, effectiveness, efficiency, sustainability, coherence and impact to assess the project’s progress and recommend measures to improve the project’s implementation processes and its partnerships to meet project goals.
4. Assess the efforts of stakeholders, and interlocutors in support of the implementation of the project and how this category contributes to or against the success of the project.
5. Examine the extent to which the project has impacted the intended People We Work With () and the three levels leading to structural changes in micro, meso and macro levels.
6. To identify and assess key internal and external factors that have contributed to, affected, or impeded the achievements of the project, and how Oxfam and the partners have managed these factors.
7. Assess the likelihood of sustainability of the project outcomes and provide recommendations to strengthen the project’s sustainability.
8. Identify best practices, lessons learnt and strategic recommendations that are emerging from the project.
9. Scope of Work

The end-term evaluation will require the following key steps:

1. Review project documentation and engage with the project staff and partners’ representatives to familiarize yourself with the intervention.
2. Based on the understanding of the terms of reference and the project, prepare tools to collect quantitative and qualitative data in line with the objectives of the end-term evaluation.
3. Develop an evaluation matrix with respondents for each tool, while providing sufficient space for direct project participants, in addition to the project team, partners and other key stakeholders.
4. Based on the data collected, analyze to what extent the project activities have contributed to the achievement of the project’s immediate and intermediate outcomes and outputs.
5. Ensure appropriate context, gender and conflict analyses are part of the end-term evaluation process.
6. Identify recommendations that need to be implemented at the micro, meso and macro levels of the project that meet the project outcomes and contributes to its ultimate goal and future gender justice programming.
7. Are there any exceptional achievements of the project substantiated with evidence, such, as but not limited to, case studies and best practices? Detail this on a page and share it as an annexe.
8. Methodology

The end-term evaluation intends to apply a combination of qualitative and quantitative data methodology with a purposeful bias towards collecting qualitative data. In addition, due to the challenges associated with the global COVID-19 pandemic, the selected consultant(s) will be required to conduct in-person data collection combined with a remote approach.

The following methods are recommended to conduct the End-term evaluation:

1. Desk review: this will entail the review of the project documentation such as progress reports, baseline reports and research studies carried out as part of the project. A strong analysis is recommended based on these materials primarily as well as other materials from related programmes in Iraq
2. Primary data collection: this can include surveys, case studies, key informant interviews, and focus group discussions with selected individuals and groups within Iraq. The data involving these methods should be collected in person, particularly when considering direct project participants.
3. Secondary data collection: this will entail NGO reports and similar documents with useful information that can contribute to the understanding of the context.
4. 1. Key Questions and dimensions of the end-term evaluation (To be revised for finality with the consultant)

The key questions the End-term evaluation aims to respond to are framed by the DAC criteria. These include the following:

RELEVANCE:

• Has the initiative responded or is responding well to the needs and interests of women and girls as identified in the problem analysis?
  • Has the project involve girls at all?If yes, how? with what expected results, and with which specific approach? If no, why?
• Did the programme design miss anything, either at the planning or implementation stage that can be addressed in future programming for women’s rights, in particular women’s political empowerment and participation in public life, peace and security?
• Did the programme design include actions to contribute to the sustainability of the project? How were these effectively and successfully implemented otherwise?
• Did the project goal resonate with the aspirations of Iraq women’s rights defenders and women’s rights organizations involved in the project? Has the project engaged with them meaningfully?
• What was the nature of the participation of the beneficiaries, in their diversity, and the partners in project design?
• What changes took place in the implementation of the project that differed from the original design? What impact have these changes had on the project?
• To what extent did the key contextual changes, threats and opportunities that arose during implementation been influencing and informing project implementation?
• How and to what extent were the monitoring findings used to inform decision-making and the improvement of project implementation by OXFAM and partners?
• Did working in partnership with the two NGOs propel the project and how? What could have been done better?

COHERENCE:

• To what extent did the initiative contribute so far to the progress at micro, meso and macro levels as understood by this project?
• To what extend did the project design and implementation course remain flexible and adaptive to changes in context, keeping to the sought impact while integrating new insights?
• To what extent did the partners’ approaches aligned with the project objectives and OXFAM’s commitment to gender equality and women’s empowerment?Have the steps undertaken by the project to contribute to the objectives of the Women, Peace and Security agenda (e.g. as illustrated by the Iraq National Action Plans but not only) been adequate?

EFFECTIVENESS:

• Were the project activities effective in contributing to their related immediate outcomes? What gaps can be identified? Did these activities lead to transformative change?

• How effective was the series of methodology applied to change social and gender norms hindering women’s participation in governance and decision-making at the micro,meso and macro level? Consider the methodologies used in the project. Annexed. In particular:

  1. How effective was the GALS methodology in enabling / leveraging not only attitudes but also behaviours supportive of women’s participation (economic/public…)?
  2. How effective was the Male Role Models methodology in enabling / leveraging not only attitudes but also behaviours supportive of women’s participation (economic/public…)?
  3. How effective was the mix of methodologies in enabling / leveraging women’s confidence, aspirations, and self-efficacy?

• How effective has the project advocacy been at the governorate and national levels?

• Were there any significant differences between the implementation sites in Kirkuk and Diyala, including the consideration of the implementation by two project partners?

• To what extent did the project contribute to partners’ capacity building as identified in capacity building plans?

• What unintended consequences (if any), whether positive or negative, did the intervention have on women, men, boys and girls so far? Any location that was adversely affected by the unintended changes?

• What system and mechanism are in place (at OXFAM and in partner organisations) to ensure accountability to the beneficiaries and how well did they work throughout the project?

• What factors contributed to achieving or not achieving the intended outcomes and set objectives of the project? What influenced these factors if any?

EFFICIENCY:

• How cost-effective was the intervention? What cost-effective alternatives could have been used?
• Is the progress in achieving project targets in line with the project implementation plan?
• Were the resources of the project (human, financial, material) invested the project timeframe and the intended changes identified in the project’s logical framework?
• Were there any missed opportunities and gaps in the cost-effectiveness of the implementation that can be addressed in for future gender programming?

SUSTAINABILITY:

• Considering the work already done with the partners, has there been any noticeable change in the ways how the partners are working? Will this be sustainable and how?
• Are modalities of engaging community volunteers appropriate in ensuring that they will remain key resource persons beyond the life of the project? What are the signs of community appropriation? What if any hindered community acceptance?
• How well are the project’s outputs linked to more long-term focused objectives?
• How durable or translating into action have changes in attitudes and behaviours among women and men participating in various methodologies been, a few months after activities?

IMPACT:

• What changes in the lives of the women and girls in Iraq can be identified so far in line with the project’s ultimate outcome?
• To what extent changes have happened at women’s personal level, relational level, in women’s relation to their environment (the community, the social and political institutions, the public….)
• To what extent did the project contribute to transformative change? i.e. changing negative patterns and obstacles to women’s participation deeply, structurally (whether formally or informally) At which levels?
• How has the relationship with the government supported the sustainability plans
• Are there any unintended changes (positive and negative), which were not foreseen by the project design?

Questions can be finetuned or expanded, informed by Oxfam’s “’How To’ Guide to Measuring Women’s Empowerment: Sharing Experience from Oxfam’s Impact Evaluations” available here:

https://policy-practice.oxfam.org/resources/a-how-to-guide-to-measuring-womens-empowerment-sharing-experience-from-oxfams-i-620271/

1. 1. Supervision and management of the data collection process

The MEAL Officers and the project teams in Diyala and Kirkuk will provide support and ensure coordination on data collection at the field level, supported by the Country MEAL Coordinator and the Gender Justice Programme Manager based in Erbil.

1. 1. Safeguarding, data collection and management ethics and management of data collection process

The following safeguarding and ethical standards are to be guaranteed at every stage of the evaluation:

Informed consent: Respondents will be fully informed of the purpose of collecting the data, what data is to be collected, how the information/findings will be used, any potential adverse impacts of participation and who will have access to the findings.

Voluntary participation: Participants who will participate in data collection will not be forced/pressured to participate but will be made aware their participation is voluntary and are free to participate or withdraw at any point of data collection without fear of retaliation.

The anonymity of respondents: The project will maintain anonymity and will not disclose the full identity of respondents in data collection and management. In cases where quotes are to be used in the report, the respondents’ permission will be sought first.

Confidentiality: The project will ensure that the information provided by respondents is kept secure and will not be made available to unauthorised individuals.

Do no harm: The team will ensure that there is no negative impact of the evaluation process on the respondents, the target population, and the enumerators in terms of any physical, psychological or political consequences (stress, pain, anxiety, diminishing self-esteem or an invasion of privacy). The data collection process will make sure enumerators do not make any promises, whether material or in any other form, to respondents or any other individuals.

1. 1. Data quality checks

Regular data quality checks will be conducted by the consultant(s) with support from the MEAL team and corrective actions undertaken by the consultant(s). Throughout the data collection in the field, regular follow-up will be conducted, including daily de-brief sessions.

All raw data will be submitted to Oxfam apart from the comprehensively analysed data that will inform the report

1. Key Deliverables
2. Inception report with detailed methodology, data collection tools and respondents:

The consultant will prepare the endline qualitative and quantitative data collection tools at the onset of the End-term evaluation process. The OXFAM Country MEAL Coordinator and Gender Justice Programme Manager will review a draft English version of the tools, and inception report and provide consolidated feedback to the consultant(s). The consultant(s) will be responsible to translate the final tools into Arabic.

1. An updated Performance Measurement Framework (PMF):

The updated version will include progress on targets as per the project’s qualitative and quantitative indicators. This will have logical explanations for variances and possible actions.

1. A preliminary report in English:

The report will provide draft findings responding to the key questions identified in these terms of reference and including draft recommendations for corrective measures that can be applied in the upcoming phase of the project. OXFAM will provide consolidated feedback on the preliminary report. A template will be agreed upon between OXFAM and the selected consultant(s).

1. A final report in English:

The report will detail the methodology applied, analysis of findings and practical recommendations following the template agreed upon for the preliminary report. The final report should not exceed 30 pages (excluding annexes), include a brief executive summary, and be accompanied by a maximum of 10 slides with the summary (PowerPoint presentation). Additionally, Oxfam will expect the consultant to present a summary of this finding in the closing ceremony or a specified related event.

All deliverables will be considered final upon receiving a written acceptance note from OXFAM’s Gender Justice Programme Manager.

1. Dissemination / publication / audience

Primary Audience

OXFAM in Iraq programme team, OXFAM Quebec, Global Affairs Canada, Project Steering Committee

Secondary Audience

Partners co-implementing the project

Beneficiaries

The Oxfam International’s Policy on Program Evaluation requires Confederation members to act on the commitment to transparency by making public the Executive Summary and a Management Response to all final evaluations. The Policy is available at this link: http://www.oxfam.org/sites/www.oxfam.org/files/oxfam-program-evaluation-policy-dec10.pdf

Although free to discuss with the authorities on anything relevant to the assignment, under the terms of reference, the consultant is not authorized to make any commitments on behalf of Oxfam. All data collected as part of this consultancy belongs to Oxfam and public dissemination of the data and evaluation products can only be done with the written consent of the Oxfam.

1. Supervision and Management

The assignment will be supervised by the Gender Justice Programme Manager and directly managed by the Gender Project Manager responsible for the management of Women and Girls Rebuilding Iraq with technical guidance from the Country MEAL Coordinator. The Gender Justice team will ensure that the partners and other stakeholders in the project have adequate space in providing their feedback on the processes and the content of the End-term evaluation.

1. Logistics and Constraints

Considering the limitations of movements due to the COVID-19 pandemic, international consultants may not be able to travel to Iraq. Access to the targeted communities is also likely to be limited. Therefore, it is recommended to form a team including Iraq-based counterparts and adopt a flexible work approach. HOWEVER, unless there is no justification, the consultant is expected to be in the country (Iraq) in person to take lead on the evaluation

1. Timeframe

It is estimated that the assignment would require 25 working days, preferably undertaken from November to Dec 2022. The consultant(s) will present a work plan of activities leading to the completion of the assignment accompanied by a detailed financial proposal (budget) in USD, including the costs for the translation of tools used in the assignment. The table in Annex 2 is an indicated template that can be used by the consultant(s) to submit their technical offer.

The total expected level of effort: 25 working days

Note: Following is the suggested number of days. Actual days will be agreed upon with the selected consultant(s)/firm as per the work plan to be delivered during the time frame.

• Start-up meeting (1 day);
• Review essential documents of the project, including but not limited to the original project proposal, interim or ongoing internal reports, and evaluations and lessons learned exercises undertaken thus far and review the key questions suggested and if necessary, propose adjustment (3 days) – work to be done from consultant’s/firm’s home location, all documents will be shared by Oxfam via email);
• Develop and submit the draft inception report with a detailed evaluation plan (to be endorsed by Oxfam) (2 days);
• Developing and translation of evaluation tools (tools must be developed in English and translated into Arabic to be administered in the field. (2 days);
• Primary data collection (quantitative and qualitative) (7 days in Iraq);
• Data analysis and preparation of draft evaluation report in English to be presented to Oxfam (no more than 30 pages excluding annexes, including executive summary not exceeding 3 pages) (7 days);
• Meeting (online) to share and validate the findings from the evaluation (1 day). Selected Oxfam and partner staff will participate in this meeting. The donor will be invited to participate in a feedback meeting if available;
• Finalize the final evaluation report according to changes/modifications agreed upon between Oxfam in Iraq and the consultant/firm and feedback received. Feedback from Oxfam staff and partners will be received within one (1) week after submission of the draft report. (2 days). Oxfam will then prepare a management response to be annexed to the evaluation report.

1. Required Knowledge, Expertise and Experience for the consultant

The following key technical requirements will be taken into consideration when assessing consultants’ technical competencies:

• University degree in gender studies, political science, sociology or other relevant fields.
• Experience in conducting external evaluations, reviews and similar assignments assessing large-scale humanitarian and development interventions.
• Expertise in applying a mixed-method approach in evaluation processes, involving data collection with the participation of diverse audiences.
• High proficiency in understanding gender transformative, women’s empowerment frameworks and rights-based approaches, and how these are relevant in evaluating civil society interventions for gender equality and women’s rights.
• Familiarity with constraints and challenges encountered by civil society actors working on gender transformative interventions in a conflict-prone and fragile context.
• Familiarity with the situation with women’s rights and issues affecting women’s participation in decision-making in Iraq.
• Full English proficiency by the lead consultant.
• Spoken Arabic in the team skill mix.

Annexe 1: Work plan template

Key activities

November 2022 December 2022

Week… Week…

Week… Week…

Week… Week…

Annexe 2: Budget template

No. of days Unit Cost in USD Total in USD Comments

Fees

Per diem

Travel

International

National

Accommodation

Enumerators

Interpreter

Translation costs

Communication

Other (specify)

TOTAL (including TAX)

Bid Evaluation Criteria

Part Criteria Description Score

A Consultant’s qualifications and experience based on TOR Academic qualifications 10%

Specialization and proven experience in the field related to the assignment15%

Qualifications, skills, and experience of the team to be engaged for the assignment 10%

B Understanding of TOR

Meeting technical requirements (quality of the proposal; level of understanding of the assignment as per TOR 15%

Adequacy of the proposed approach and methodology to undertake the task based on the TOR 20%

Feasibility of Workplan (detailed work plan required with key milestones) 10%

C Financial proposal

Clarity of cost breakdown 5%

Realistic and justifiable budget 15%

Annexe 3: Bid evaluation criteria

Annex 4: Recommended outline of the Final Evaluation (maximum 30 pages)- To be refined further.

1. Title page and opening pages

• Name of project being evaluated /affiliate identification code; Timeframe of evaluation and date of the report; Locations; Names and/or organizations of evaluators; Oxfam as the organization