Call for Tender – External Endline Evaluation Syria

Terms of Reference (TOR) for an External Endline Evaluation

1. Introduction

Project title

Multisector Humanitarian assistance for conflict-affected local populations and internally displaced persons in Northeast Syria through access to safe water, food assistance and NFI.

Implementing Organisation

arche noVa

Project Outcome

The risks of water-borne diseases and negative coping strategies are reduced for conflict affected populations.

Project Duration /Evaluation Period

01.08.2021 – 31.12.2023

Project Donor

The German Federal Foreign Office (GFFO)

Time period for Evaluation

01.01.2024 – 28.02.2024

1. Description of project activities

The above-mentioned project is being implemented in three locations in Northeast Syria (NES): Al Hassake city, Ar-Raqqa governorate, and Deir ez Zor governorate (Al-Kasra District). The sectors of the interventions are NFI (Non-Food Items), FSL (Food Security and Livelihoods) and WASH (Water, Sanitation and Hygiene) and targeted Internally Displaced Persons (IDP) and Host Communities (HC).

Please note: at the time of the evaluation process, the project might still be implementing some last activities until its completion.

The activities include:

1. Water trucking in communal and household tanks
2. Distribution of water tanks, children winter clothes, food parcels, hygiene kits (including hygiene promotion sessions), Dignity kits and core Hygiene kits.

The total planned beneficiaries are around 125,000 individuals and 26,300 households. The beneficiaries are IDPs and host communities, residing in and out of Camps.

Notes**:**

• The current security situation might impede the access to the project sites.
• arche noVa will ensure coordination between the evaluation team and the field team to carry out the evaluation process appropriately and is available to provide any project information and data.

1. Evaluation purpose

The aim is to evaluate the overall impact of the project according to its intended goals by comparing the Outcome and Output indicators against the achievements. Furthermore, the purpose is to identify to what extent the used methodology in the activity implementation has contributed to the overall Project Goal and what impact the project had on the target groups. This includes a critical reflection on relevance, coherence, effectiveness, efficiency, impact and sustainability. The evaluation should provide useful information in identifying further needs and applicable lessons learnt according to organization capacities and taking into account donor regulations to inform future programming.

1. Scope of evaluation

The evaluation shall cover the project activities according to the Logframe over the whole project period.

Project Goal: The risks of water-borne diseases and negative coping strategies are reduced for conflict affected populations

Project Outcome: Access to safe water, food assistance and non-food items is improved for 26,300 households.

Project Outputs:

1. Food insecure households have received monthly food baskets.
2. Conflict affected households have increased access to safe water and sanitation along with increased hygiene knowledge including risk of contracting COVID-19.
3. Children of conflict affected households have increased access to winterization items.

Here the evaluation questions more in detail:

Project Goal: The risks of water-borne diseases and negative coping strategies are reduced for conflict affected populations

• To what extent has the project contributed to a reduction in the risks of water-borne diseases among the targeted population?
• How were negative coping strategies identified, and to what degree were they mitigated by the project?

Project Outcome: Access to safe water, food assistance, and non-food items is improved for 26,300 households.

Indicator 1: % of households reported that their basic needs are met:

• Are there specific needs that were addressed particularly well or are there needs that should have been prioritized for more relevancy of activities?
• In what ways did the project enhance access to food assistance and non-food items, and how were these improvements quantified?

% of households with improved access to safe water:

• How has improved access to safe water influenced the overall health and well-being of the households?
• Are there challenges or barriers that hinder further improvement in this aspect?
• What are the perspective needs for households targeted for water deliveries and are there more sustainable options – considering financial considerations and the socio-environmental and political challenges of water supply in Hassakeh?

Project Outputs:

1. Food Insecure Households Receiving Monthly Food Baskets:

Number of vulnerable households receiving food parcels:

• What criteria were used to identify and prioritize food-insecure households and were they the most useful ones? How can selection criteria be improved in the future?
• Were there any challenges or successes in the distribution process, and how were they addressed?
• Beyond the quantity of food parcels distributed, can you assess the nutritional impact on the recipients? Are there indications of improved food security and well-being among the targeted households?
• Which alternative delivery modalities are possible in the targeted areas to improve food insecurity? How should the modalities be set-up considering a transition from in-kind support? What are the budgetary and personnel implications for the organization?

2. Increased Access to Safe Water and Sanitation for Conflict-Affected Households:

Number of vulnerable households receiving hygiene kits and hygiene promotion:

• What were the impacts of the provision of hygiene items?
• How has the distribution of hygiene kits contributed to improved hygiene practices within households?
• Are there gaps in knowledge or behaviors that need further attention?
• Which alternative delivery modalities are possible in the targeted areas to improve access to hygiene items? How should the modalities be set-up considering a transition from in-kind support? What are the budgetary and personnel implications for the organization?

% of surveyed female beneficiaries reporting their needs have been met:

• In what ways did the project address the specific needs of female beneficiaries?
• Are there areas where their needs were not adequately met, and how could these be addressed in future projects?

Number of households with daily access to safe water:

• How has daily access to safe water positively impacted the health and well-being of the households?
• Are there challenges in sustaining this access over the long term?
• How quickly and effectively was access to safe water improved for households, and were there any barriers or constraints in ensuring rapid improvements?
• How accurate is the estimated number of households benefitting from water provision through communal water tanks? Define a method how the beneficiaries for this activity can best be calculated.
• What are the differences between household water delivery and delivery to communal tanks considering the effectiveness, efficiency and impact?

% of targeted beneficiaries with knowledge about at least 3 critical times of handwashing with soap:

• How effective were the hygiene promotion activities in increasing awareness and changing behaviors?
• Are there specific areas where further education is needed and that should be included in hygiene sessions in future projects?

Number of female beneficiaries receiving dignity kits and hygiene promotion:

• What impact have dignity kits had on the well-being and dignity of female beneficiaries?
• Are there additional needs that should be considered in future projects?

Number of vulnerable households receiving water filters and spare parts:

• How have water filters contributed to improving the quality of water for households?
• Are there challenges in maintaining and replenishing consumables for water filters?
• What is the long-term use of water filters in the context of the provided water schemes, with special considerations for IDP households?

3. Children of Conflict-Affected Households have Access to Winterization Items:

Number of children of vulnerable households received a winter clothing kit

• How did the winterization items contribute to improving the well-being of children in conflict-affected households?
• Beyond the provision of winter clothing, how has this contributed to the overall well-being and protection of children in vulnerable households?

Additional evaluation questions:

Do-No-Harm:

• How were potential negative impacts on the environment, local communities, and other stakeholders mitigated throughout the project lifecycle?
• Were there any unintended consequences or harm caused by project activities, and how were these addressed to ensure a positive and ethical impact?

Cross-Cutting Evaluation:

• How did the various project outputs synergize to achieve the overall goal of reducing water-borne diseases and negative coping strategies?
• Were there unexpected positive outcomes or challenges that emerged as a result of the interconnected project activities?

Community Engagement and Participation:

• How actively were the targeted communities involved in the planning and implementation of the project, especially concerning the distribution of food baskets, hygiene kits and dignity kits, access to safe water, and winterization items?
• In what ways did community input influence the project design, and how did the project adapt based on community feedback?

Sustainability and Long-Term Impact:

• What measures were taken to ensure the sustainability of the project outcomes, particularly in terms of continued access to safe water?

Monitoring and Adaptive Management:

• How was project progress monitored in real-time, and were there instances where adjustments to the project strategy were necessary? How were these adjustments implemented?
• Were there any specific lessons learned during the project implementation that could inform future projects?

Collaboration and Coordination:

• How effectively did the project collaborate with other humanitarian actors, local authorities, and non-governmental organizations to avoid duplication of efforts and maximize impact?
• Were there any challenges in coordinating efforts with other organizations, and how were these challenges addressed?

Beneficiary selection, Gender and Vulnerable Groups:

• To what extent did the project consider the unique needs and vulnerabilities of different genders and people with disabilities in the distribution of resources, especially in terms of food assistance and winterization items?
• Were there specific strategies in place to address the diverse needs of the affected population?
• Beneficiary selection criteria: analyse selection criteria and process and recommend improved/alternative and feasible ways of targeting with available resources.
• Registration questionnaire and PDM (post-distribution monitoring) survey: propose updated questionnaires in order to get most reliable results and ensure appropriateness to the context
• Verification process: Propose an improved verification mechanism within the resources

“Greening” strategy

• Identify biggest originator for carbon emissions and solid waste caused by the project
• Propose easy to implement and low-cost options to reduce carbon footprint and solid waste, considering current interventions and modalities.

1. Users of the evaluation

User of this evaluation will be

1. Primary User: arche noVa
   1. Senior Management and Project Implementation team in the field
   2. Program Management at HQ level
2. Secondary User: Donor
3. Evaluation methods

The evaluation team designs an adequate process and methodological approach for the evaluation. The evaluation team should propose proper sampling where relevant to cover the components of the evaluation tasks. In general, participatory methods that include the project’s target group, a mix of quantitative and qualitative methods, triangulation of findings and a gender sensitive/special needs approach are required. A critical assessment and discussion of the project’s strengths and challenges is welcome. Arche noVa is ready to learn from challenges/mistakes. Recommendations should be included in the evaluation report. The study design should follow international standards concerning accuracy, reliability of data and codes of conduct. The evaluation shall include onsite visits in an appropriate way.

All documents and data collected are to be treated confidentially and are to be used solely for the purposes of this evaluation. In principle, interviews are to be anonymized. Participants must not be quoted in the reports without their consent.

1. Deliveries

The evaluation team will be expected to submit the following documents:

1. Inception report: This should include amongst others, the data collection and analysis methodologies, the selection of enumerators, tools proposed, and the respective timetable to achieve the objectives. It should include descriptions of checklists, questionnaires and reporting formats.
2. Draft evaluation report: This should include
   1. the executive summary
   2. description of methodology (including estimated number of on-site visits and interviews)
   3. limitations, challenges faced
   4. findings,
   5. recommendations and
   6. Conclusions.

The evaluation team should support the analysis of project’s achievements with relevant data and state how this has been sourced. It should also include the terms of reference, data collection tools and references.

1. Presentation/workshop: Before the final report is issued, the evaluation team should participate in a presentation/workshop to arche noVa. The aim of this presentation/workshop is to receive feedback/clarifications/explanations on findings and preliminary conclusions and recommendations. The presentation/workshop will focus also on lessons learnt and good practices for future programming.
2. Final report: The final report is to be submitted after feedback has been received and integrated into the draft report. Further, the evaluation team should provide all of the data collection in the form of Excel sheet.
3. Resources

Following resources will be provided:

1. Project proposal
2. Logframe
3. MEAL reports
4. Project reports (internal and external reports)
5. Organisational documents
6. Others
7. Adaptation of OECD/DAC criteria for the deliveries
   1. Relevance:

For the evaluation tasks stated above, the team is advised to adapt the OECD/DAC criteria of relevance, effectiveness, efficiency, impact and sustainability. The evaluation tasks will focus on the impact of project components and can be developed around the following key questions:

• To what extent is the project suited to the priorities and needs of the target groups?
• To what extent is the project suited to the context?
• To what extent has the project been successful in adapting to the changing context in Northeast Syria?
• To what extent have the gender, special needs and vulnerability considerations been mainstreamed into activities?

1. Effectiveness/Coverage:

• To what extent has the project achieved its stated project objective and outcomes?
• To what extent could the project reach the most vulnerable people, particularly woman, children, PWD or minorities?

1. Efficiency:

• To what extent has the project been implemented in the most efficient way (time, materials, resources)?

1. Impact and sustainability:

• What were the positive and negative changes produced by the project, directly or indirectly, intended or unintended?
• What were the main challenges during the implementation of the project?
• What positive or negative impact had the project activities to the environment?
• What positive or negative impact had the project activities to and peaceful and inclusive coexistence?

1. Timetable and Workplan

The evaluator is requested to fill in the table (Annex 1), listing suggested dates for each task of the evaluation (the content can be adjusted according to the plannings). Furthermore, a detailed workplan breakdown is to be provided.

1. Technical proposal

The applicant should present a technical proposal, reflecting the evaluation tasks, including (at least):

1. Methodology:

• Type of study design
• Sampling technique
• Data collection technique
• Methods of data compilation, analyzing and interpreting

1. Understanding of the TOR
2. Organizational experience and capacity to fulfil the tasks
3. Description and breakdown of the planned evaluation team members (including CVs of main consultants and field coordinators/team leaders)
4. Filled in timetable (see above)
5. Detailed workplan
6. Profile of the evaluation team

The profile of the desired team members should include:

• Main consultants to be fluent in English, field team to be fluent in Arabic (field coordinator/team leader to have basic English skills)
• Relevant experience in conducting evaluations of similar projects in Northeast Syria
• Familiarity with the Humanitarian context in Northeast Syria (especially in project regions)

How to apply

Quotations must be sent by email in PDF format not later than the 15/12/2023, 12.00 German time, to the following email address:
[email protected] with reference: Tender AN Project External Endline Evaluation NES

If you are interested in applying, you can request all tender documents at the following email address: [email protected]