Caritas Austria
Terms of Reference – Final Evaluation COPE
Project Title
COPE – COVID-19, Primary and Emergency Health Care in Jordan and Lebanon
Project country/locations:
Jordan: Hashmi/Amman, Mafraq, Jarash:
Lebanon: All regions (Akkar, Baalbeck-Hermel, Beirut, Bekaa, Mount Lebanon, North Lebanon, Nabatiyeh, and South Lebanon)
Project duration
01.07.2021 to 31.12.2022
Lebanon implementation period: 01.07.2021 to 31.12.2021
Jordan implementation period: 01.01.2021 to 31.12.2022
Anticipated budget
4.700€
Deadline for submission
20th September 2022
- INTRODUCTION AND BACKGROUND
This year marks the 12th year of the Syrian conflict which has heavily affected both Jordan and Lebanon due to the high influx of refugees. Further, the COVID-19 pandemic has challenged the national health systems, and specifically Lebanon is in the midst of an economic crises whilst still overcoming the devastating destruction of the Beirut explosion. The national health systems in Jordan and Lebanon were overwhelmed and needed support in order to provide access to urgently needed health care services for their respective population. Capacity of most hospitals exceeded. The increase of access for vulnerable patients to life-saving Covid-19 medical care and follow up medical care for those suffering from Long COVID symptoms became crucial. Especially refugees were among the most vulnerable groups. In total, the COPE project targeted 21.800 individuals, 70% refugees and 30% vulnerable people from host-communities, and migrants with a focus on Pregnant and Lactating Women (PLW), elderly, People with Disability (PWD), COVID-19 infected and Non Communicable Disease (NCD) patients who meet set vulnerability criteria. Gender responsive medical and social services under this project should contribute to alleviate health related and social effects caused by Covid19 pandemic and other crises in Lebanon and Jordan. The project was implemented by Caritas Lebanon and Caritas Jordan both in urban as well as remote areas. The interventions responded to health related challenges by following a tailored approach to the countries’ respective context. Activities of this project aimed to support patients and their families who have limited access to health care services due to the frail health systems both in Jordan and Lebanon through the following interventions:
- Provision of highly needed oxygen for COVID-19 patients in Lebanon, as well as access to life-saving medical care since the capacities in public hospitals are overstretched
- Follow up on patients with Long COVID symptoms delivered through professional staff
- Provision of Primary Health Care (PHC) services to relieve overcrowded public hospitals
- Provision of Psychosocial Support (PSS) to patients affected by multiple crises
- Conducting internal and external referrals to additional services such as Secondary Health Care (SHC) Services (following a case management approach)
- Awareness raising on identified topics as prevention of COVID-19, Sexual and Gender Based Violence (SGBV) and information for Pregnant and Lactation Women and Non-Communicable Disease will be provided.
Overall Objective:
Contribution to alleviate the health related and social effects caused by Covid-19 pandemic and other crisis in Lebanon and Jordan
Specific Objective:
Strengthening local health care services weakened by Covid-19 and protracted crisis in Lebanon and Jordan through the provision of gender responsive medical support & social services
Output 1: Immediate medical support and outreach to COVID-19 affected persons requiring oxygen support in their homes (LEB)
Output 2.: Long COVID (LC) patients are supported (LEB)
Output 3.: Access to primary health care services are provided or supported to refugees and vulnerable host community patients (JOR)
Output 4: Refugee and vulnerable host community patients referred to secondary healthcare services or internally referred to other Caritas services (LEB, JOR)
Output 5: Refugee and vulnerable host community patients are reached with health awareness campaign (on NCD risks, maternal and neo-natal best practices, Covid-19, SGBV, CD/NCD, food hygiene etc.)
Output 6: Refugees and vulnerable host community affected by the pandemic benefit from cash grants
With the current project “COPE (COVID-19, Primary and Emergency Health Care in Jordan and Lebanon)”, Caritas put the focus on medical and psychosocial needs emerging through the Covid-19 crisis.
- APPROACH AND METHODOLOGY
The evaluation will be undertaken in line with the OECD/DAC Quality Standards for Evaluation (2010).
It is up to the evaluation team to design a final methodology and detailed evaluation questions (based on the guiding questions below) during the inception phase, so the team feels well placed to respond to the guiding questions outlined below in the most appropriate and credible manner.
As the implementation period in Lebanon ended on 31st of December 2021, this part of the evaluation is to be done via remote interviews.
The evaluation/review consists of several phases:
- Contract and Kick-off meeting: Contract is signed and a discussion of the assignment takes place. First documents, including available data, are provided to the evaluation team.
- Desk Study: The evaluation/review team studies all necessary project/programme documents; re-construct and analyse the intervention logic/programme theory and theory of change and its assumptions. Existing data needs to be analysed and interpreted.
- Inception-Phase: In the inception report the evaluators will describe the design of the evaluation and will elaborate on how data will be obtained and analysed. The use of a data collection planning worksheet or a similar tool is required. First interviews take place.
- Data triangulation and quality control are very important and need to be discussed in the inception report.
- The field trip will only take place upon official approval of the inception report by the contractor.
- Field-phase: Data needs to be gathered, analysed and interpreted. It is expected that the evaluation will include quantitative and qualitative data disaggregated by sex.
- Presentation: Presentation of key findings (feedback workshop) at the end of the field trip.
- Final Draft Report: Submission and presentation of final draft report, inclusion of comments from partners and contractor.
- Final Report: Submission of final report, see reporting requirements under point 4).
- All data collected needs to disaggregated by sex.
For the different phases it is expected that data and information will be obtained through different methods such as: analysis of documents, structured interviews, semi-structured interviews face-to face or by phone, group discussions, others.
All data collected needs to be disaggregated by gender. It is expected that the evaluation/review team will present concrete recommendations which are addressed to the specific stakeholders. It is currently estimated that xx number of people need to be interviewed in xx.
- EVALUATION QUESTIONS
A special focus should be given to the following areas of the Health Programme:
- Access to healthcare: To what extent has the programme contributed increased equitable access, uptake and quality of healthcare considering availability, accessibility, timeliness, and affordability? To what extent are the primary health care services offered at CJ clinics adequate and in support of the shift towards primary health care in line with commitments to health care reform and UHC?
- NCD Management Course: To what extent has the NCD Management Course contributed to sustainable changes in the lives of those who have been selected to participate? Which elements of the course has shown most effect/success, in what way and why? What areas could be developed further?
- Health Promotion: Are there documented positive results and examples from the health promotion component? Which strategies have shown most effect/success, in what way and why? Has the health promotion component had any effect on the other components in the programme?
Criteria
Guiding evaluation questions
Appropriateness
1. To what extent was the Caritas response tailored to respond to the needs of vulnerable Syrian refugees and host communities?
2. To what extent are the objectives of the programme consistent with country needs?
3. To what extent was the Caritas response to the Syria displacement crisis in Lebanon and Jordan aligned with government plans and priorities, including SDG 3 commitments, and donor policies?
4. Was the Caritas response in line with relevant humanitarian standards such as SPHERE and CHS?
5. Were appropriate systems of downward accountability (participation, information sharing, and feedback) put in place and used by project participants? Did the feedback shape project design and implementation?
Effectiveness
6. To what extent were the objectives of the programme achieved?
7. What are the (concrete) contributions of interventions of the programme for achieving the objectives?
8. What factors were crucial for the achievement or failure to the project objectives?
9. To what extent were the target groups reached?
Efficiency
10. How cost-effective was the Caritas response in Lebanon and Jordan?
Coverage
11. Did the Caritas response reach the most vulnerable Syrian refugees and vulnerable host community in targeted areas based on needs regardless of age, ethnicity, religion, and gender? Was the targeting appropriate and transparent?
12. To what extent was the Caritas response geographically adequate to cover the needs of the beneficiaries?
Coherence (coordination)
13. How has the project been coordinated with the activities and priorities of other agencies and organizations (including local and national government)?
Impact
14. What are the positive and negative, primary and secondary long-term effects produced by the programme, directly or indirectly, intended or unintended?
15. To what extent has the project interventions contributed to changes in the lives of the Syrian refugees and host communities in the targeted areas?
Sustainability
16. To what extent are the target groups capable and prepared to receive the positive effects of the programme intervention without support in the long term?
17. To what extent has the local capacity of Caritas been strengthened at organisational and sectoral levels? What are strengths and weaknesses of Caritas Jordan’s and Caritas Lebanon’s capacity and what are areas for improvement / growth to better deliver future programming?
- DELIVERABLES
The consultants will submit the following reports:
- an inception report (10-15 pages without annexes), clearly outlining the details of the evaluation methodology and approach based on preliminary findings from the document review. The report should include a tentative outline, a detailed work plan with key milestones as well as an evaluation matrix with key and specific evaluation questions, their expected data collection source and method of data collection. Interview guides and/or questionnaires should also be annexed to the draft inception report. The report should be 10-15 pages long excluding annexes. The report should be shared with CAUT for consultation and approval before finalisation.
- a final draft evaluation report (about 25-30 pages without annexes), including a draft executive summary and the results-assessment form (part of the reporting requirement)
- and the final evaluation report (25-30 pages without annexes), the final executive summary and the results-assessment form (part of the reporting requirement)
- A dissemination meeting (proposed online or in Amman), whereby key findings, lessons learned, and recommendations are shared with partners including a PowerPoint presentation of key findings and recommendations.
- A minimum of 5 case stories of best practices and/or most significant change stories of maximum 1 page each.
All reports need to be written in English.
The executive summary should summarize key findings and recommendations (three to five pages) and needs to be submitted as part of the final draft report.
The findings and recommendations of the draft final report and final report have to be
structured according to the evaluation questions. An outline of the report’s structure needs to be agreed upon during the inception phase.
- RESPONSIBILITIES AND CONDITIONS
Confidentiality of information: All documents provided to the evaluator(s) and all data collected during field research must be treated as confidential and used solely for the purpose of this evaluation. Respondents shall not be identified in reports without their explicit written permission. Photo, video, and other research data that can be traced back to individual research participants shall be anonymized unless agreed otherwise, in writing, with the person concerned.
Ownership of data, findings, products: CAUT retains full ownership of all data, findings, and products produced as part of this assignment.
Support by CAUT and partners: All relevant documents including programme proposal, reports, etc. will be provided for the consultant(s). Field research can be supported and facilitated by Caritas Jordan/Lebanon based upon the methodology and approach defined and depending on Covid-19 related restrictions. Relevant contacts of other stakeholders will be shared upon the consultant(s)’ request.
Safeguarding and protection: For the time of the assignment, the evaluator commits to safeguarding and protection policies and procedures that are shared and reiterated during the preparatory phase.
Distribution Policy: Intended users of deliverables are CAUT, CJ, CL, back donors of CAUT, other health partners of CJ (e.g. Caritas Denmark, Caritas Germany).
All deliverables generated through this evaluation will be subject to approval by CAUT before public dissemination.
- TIMELINE
Procurement Phase
August/ early September
- Publication of ToR
- Submission of proposals by applicants
- Assessment of proposals and selection by Caritas
- Contracting (early September)
Preparatory Phase
October
- Kick-off meeting
- Desk review of existing project documents and data
Inception Phase
October
- Submission of inception report
Feedback Inception Phase
Early October
- 1 week after inception report has been shared with CAUT
- Inclusion of comments in the inception report by CAUT, CL, CJ
Evaluation Phase: Lebanon
October
- Research and data collection – Remote evaluation
Evaluation Phase: Jordan
November/December
- Field research and data collection – evaluation
Report Writing and Feedback Phase
December
- Preparation and submission of draft report
- Feedback by Caritas
- Incorporation of feedback
Presentation of final draft report
December
- Presentation of the final draft report; meeting including CAUT, CJ, CL
Submission of final evaluation report
December
- Submission of final report (hard copy and electronic copy) to contractor
The consultant is requested to include a suggested timeline in their technical offer.
How to apply
- APPLICATION PROCESS
Interested applicants should submit their application via e-mail to [email protected] with the subject line ‘Final Evaluation COPE”. Deadline for submission of applications is 20th September 2022 8:00am CEST. Consultants who do not submit their quotation by this deadline will not be considered.
The application should include as a minimum:
- CVs of the consultants
- Cover letter which clearly summarizes relevant experience
- Technical proposal including:
- Methodology and tools
- Detailed work plan incl. timeline
- Examples of relevant previous work
- Financial proposal (in EUR) including a clear breakdown of costs including fees and other expenses such as travel, accommodation and transportation
- The Evaluation Team
Key Qualifications in the team should be:
- Relevant academic degree (master level)
- A minimum of three years’ experience and expertise
- Team leader has conducted at least three evaluations in the last five years ideally in the relevant field
- Team member has participated in at least three evaluations ideally in the relevant field
- Knowledge of Jordan and Lebanon with focus on topics such as health care
- Working experience in country Jordan and/or Lebanon
- Experience in project cycle management
- Experience preparing and analysing a theory of change
- Experience and expertise in evaluating cross-cutting issues
- Experience in social science methods
- Excellent oral and written English and Arabic skills
- Sound MS Office and IT skills