External Evaluation of JRS South Africa’s Health Project

1. Introduction

Jesuit Refugee Service (JRS) is an international organization with a mission to accompany, serve and advocate on behalf of refugees and other forcibly displaced persons, that they may heal, learn, and determine their own future. It is non-profit catholic church-based organization with over 40 years of global experience in various humanitarian emergency and relief operations worldwide and is currently reaching out to over 1 million displaced people in 58 countries. JRS South Africa works with urban-based asylum seekers and refugees living in Gauteng province, providing education, livelihoods, advocacy, pastoral care, and health activities.

2. Context and rationale

According to the UN Refugee Agency (UNHCR), South Africa is home to a large refugee and asylum seeker population, totalling 250,2501 individuals. Including individuals from countries such as Burundi, the Democratic Republic of Congo (DRC), Rwanda, South Sudan, Somalia, and Zimbabwe. South Africa has been a preferred destination for many migrants from other parts of Africa due to its advanced economic status and progressive laws and policies regarding migrants and refugees. Within South Africa, Gauteng province stands out as a major destination for refugees and asylum seekers, housing a significant portion of this population. Despite Gauteng’s status as the economic center of the country, refugees and asylum seekers in this province often encounter challenges in accessing essential healthcare services. While the law guarantees them the right to healthcare, refugees, and asylum seekers often struggle to translate this right into reality. These challenges stem from various factors, including limited healthcare infrastructure, financial constraints, language and cultural barriers, legal and administrative hurdles, and stigma and discrimination. The lengthy process to register and obtain refugee permits often delays access to healthcare benefits. JRS has learned that, even with proper documentation, navigating the public healthcare system can be challenging. Financial strain and unfamiliarity with the health system often leads to frustration and missed appointments. As a result, many refugees and asylum seekers in Gauteng struggle to obtain the medications and healthcare they need to address their health conditions effectively.

3.Project Goal and Objective

The Health Project’s goal is to improve vulnerable refugees’ access to healthcare services and community-based strategies for health promotion and disease prevention in Gauteng province, South Africa.
The specific objectives are:
•To provide healthcare, psychosocial support services, and social assistance to chronically and terminally ill refugees, asylum seekers and South Africans in Johannesburg and Pretoria.
•To strengthen the capacities of community-based caregivers through the provision of psychosocial support and training workshops.
•To strengthen community-based strategies for health promotion and disease prevention in non-communicable diseases through health advocacy, health roadshows, and health ambassadors.

4. Purpose of the Evaluation

The purpose of the evaluation is to assess the progress made towards achievement of the specific project objectives over the project period, 1st September 2019 to 30th April 2024. This includes an assessment of the project outcomes and impact on the lives of key beneficiaries who were directly involved in project activities in Johannesburg and Pretoria, in Gauteng province. The project beneficiaries are as follows:
• Chronically or terminally ill asylum seekers, refugees, and indigent South African nationals, aged 1 to 90 years.
• Caregivers of chronically and terminally ill beneficiaries
• Youth Ambassadors
• Participants of various training workshops, including asylum seekers, refugees, nursing staff in designated public facilities, and representatives of non-government organisations.
The evaluation assessment will also gauge the level of community and other stakeholder participation and ownership of the implementation process; identify the lessons learned; and make recommendations on how JRS South Africa can improve the coordination and management of future projects of this nature.

5. Specific Objectives of the Evaluation

The evaluation will assess the project according to its Relevance, Effectiveness, Efficiency, Impact, and Sustainability. In line with the identified themes, the below-listed key guiding questions should be considered in the evaluation. These are:
5.1 Relevance of the Project Design
Assess the appropriateness and relevance of the project design and activities towards the expected outcomes and outputs; and assess what adjustments have been made if any. The following are the key guiding evaluations questions for the evaluator in this section:
•How well is the project responding to the needs of target beneficiaries?
•How did the project seek the views of its beneficiaries and stakeholders?
•How relevant are activities towards expected outputs, outcomes, and impact?
•To what extent are the objectives, planned activities and planned outputs consistent with the intended outcomes? Are there differences between the time when the programme/project was designed and now?
•To what extent are the objectives of the project/programme still valid?
•To what extent has the project increased the skills and knowledge of Community Health Workers (CHWs) and caregivers?
5.2 Efficiency in Planning and Implementation
Assess whether project resources were used economically in relation to the planned activities, outputs, and outcomes. The key evaluation questions include the following:
•To what extent have the project costs been justified by the benefits?
•What percentage of planned activities have been implemented?
•Is the financial spend/use in line with the project plan and budget?
•How were risks in project implementation managed?
•Were project objectives achieved on time?
5.3 Effectiveness of the Project
Assess the major achievements of the project in relation to its stated objectives, expected outcomes, and outputs. The key evaluation questions include the following:
•To what extent is the project progressing towards achievement of the long-term goal?
•What were the major factors influencing the achievement or non-achievement of the objectives?
•To what extent has the project contributed to increasing asylum seekers and refugees’ access to public healthcare services?
•What is the current amount of social assistance received by beneficiaries for various needs such as rent, transportation, food, and medical expenses? What would be the most appropriate amounts required to effectively meet these needs?
•To what extent has the project contributed to increasing awareness of refugees’ health rights among local authorities and public health service providers?
•To what extent is the project accomplishing planned activities?
•To what extent is the project working with appropriate local stakeholders to enhance the effectiveness of its interventions?
•How effective is the project’s targeting strategy in identifying appropriate beneficiaries?
•To what extent is the organization providing quality services?
•What are the key drivers and barriers affecting the delivery of results for the project?
5.4 Outcomes and Impact
Assess the extent to which the project had a wider overall effect on improving vulnerable refugees’ access to healthcare services and community-based strategies for health promotion and disease prevention in targeted communities in Johannesburg and Pretoria. The key evaluation questions include the following:
•What changes, if any, with respect to asylum seekers and refugees’ access to public health services occurred because of the project interventions?
•What changes, if any, has the project brought about for beneficiaries? How different would their situation have been without the involvement of the health project?
•How has the project activities contributed to social cohesion in the targeted communities?
•What are the unintended impacts of the project and how have these affected beneficiaries?
•Is there potential for replicating similar initiatives in other areas and are there any
pre-conditions to enable that replication?
5.5 Sustainability
Assess the lifespan of the project interventions in the long-term beyond JRS South Africa’s involvement. The key evaluation questions include the following:
•To what extent will the positive impacts or changes of the project continue?
•Do the target beneficiaries have the capacity to still maintain the improved practices introduced by the project?
•To what extent will the benefits of the project persist beyond donor support?
•To what extent will the project impact on cross-cutting issues: gender and protection?

6. Approach and Methodology

The Evaluator is expected to use a variety of data collection methods. The key methods will include, but are not limited to:
•Document Review: This will include the approved project proposals, budgets, quarterly project progress reports, work plans, and any other documents deemed critical for the evaluation.
•Interviews: The Evaluator will conduct interviews with key informants (groups or individuals) including project staff, partner non-government organisations, health professionals and/or nursing staff in targeted public health facilities, and key government departments
•Focus Group Discussions with past and current project beneficiaries, partner organisations, government departments; and other social groups and key stakeholders.

7. Timing and Deliverables

The evaluation will commence on 18th June 2024 and be completed with the submission of the Final Evaluation Report on 15th August 2024.
The Evaluator is expected to deliver the following:
•An Inception Report, detailing a description of the methodology to answer the evaluation questions, proposed data collection and analysis tools and methods, sampling strategy and ethical considerations, and a detailed schedule for the tasks to be undertaken.
•A Draft Evaluation Report to be discussed within JRS and among project partners to provide feedback and comments.
•A Final Evaluation Report.
•Submission of hard and electronic copies of materials, data collected/analysed and other evaluation documents.

8. Key qualifications and expertise of the evaluators

JRS is looking for a consultant or a team of two with at least 5 years of consulting experience for similar projects/programs. The Consultant/s will possess the following:
•A post-graduate degree in Migration and Refugee Protection, Development Studies, International Development/Relations, Human Rights, or equivalent is required.
•Experience in the NGO sector with excellent analytical and report writing skills is necessary. The consultant(s) should have worked with IDPs, migrants, asylum
seekers, and refugees and possess a solid understanding of national and
international policy frameworks on migration and displacement.
•Knowledge and experience of working in South Africa is highly desirable.
•Language proficiency in English and one local language is required, with French being desirable.
Submission of Applications
In responding to this Terms of Reference, the Consultant’s application should include the following.
•CVs of all evaluators to participate in this mid-term project evaluation, showing education and expertise.
•Technical proposal detailing the methodology and approaches to be used for data collection and inclusion of crosscutting issues such as gender, engagements with youth, women and people living with disabilities, protection, among others.
•Detailed work schedule
•Financial proposal which shows, complete cost estimate that includes both, the fee as well as any secondary costs to be incurred, such as transport, and other related costs to be incurred in the scope of the evaluation