Fred Hollows Foundation
Introduction
The Fred Hollows Foundation (The Foundation) is a secular non-profit public health organisation based in Australia, which was founded in 1992 by eminent eye surgeon Professor Fred Hollows. The Foundation focuses on strengthening eye health systems and the treatment and prevention of avoidable blindness caused by Cataract, Trachoma, Diabetic Retinopathy, and Refractive Error. It operates in more than 20 countries across Australia, The Pacific, South and South-East Asia, and Africa. The Foundation was named The Australian Charity of the Year 2013 at the inaugural Australian Charity Awards.
The project is being implemented with the partners being Ruharo Mission Hospital Eye Centre (REC) and Mbarara University Referral Hospital Eye Centre (MURHEC) located in Mbarara city and CBM as the main implementing partner. The project area includes sub regions of Southwest, West and South Uganda, targeting Regional Referral Hospitals, District Hospitals, and sub district Health Units. Two recent studies from Uganda on Cataract Surgical Rate (CSR) showed that this region had the lowest CSR of 174/million compared to the national average of 230/million. This is an extremely low CSR rate when compared to other African countries. Consequently, data from the Uganda National Health Information Management Systems (HMIS) showed that this region had the highest prevalence of operable cataract. In this report, the proportion of patients with operable cataract was 84.7% in Bunyoro and 64.6% in Tooro region. The gender dimension also presents varying statistics on how male and female access eye health services with access rates for female being relatively lower.
The project has two parts which have distinct focus areas and evaluation requirements.
- Equitable Access to Effective Cataract Services (focusing on outcomes for the general population and improving service quality)
- Achieving Equity Through Integration of Health Services for Women (focusing on improving outcomes for women)
Project background
The project is working in partnership with government and mission-based hospitals and health centers in West, and South-Western Uganda to increase quality and equitable access to eye care services and other health services, with parts particularly focused on reaching the disadvantaged sub-populations of older persons and women. A summary of each of the components of the project that is to be evaluated are provided below.
Part 1) Equitable Access to Effective Cataract Services:
To increase community engagement in eye health services, communities will be educated on eye health issues through primary health workers, and eye patients will be screened and mobilized through existing health facilities. Primary health workers will be trained in basic eye screening and referral, and PEEK technology will be introduced to improve the accuracy of screening and referral. This approach will be scaled up throughout the project implementation period to cover all districts in the project area.
Increased eye health services will be achieved through infrastructure development, resourcing, andequipping cataract service-delivery sites to deliver quality cataract and other eye health services, training of eye-health workers, and strengthening district health systems to plan, budget and monitor eye health services. The project will also strive to improve the productivity and quality of cataract surgery through the training of surgical teams in theatre management, training of optometrists and ophthalmic nurses on biometry, high volume cataract surgery, standardization of quality measurement, patient follow-up procedures, expansion of the medical consumables store, eye centre admission ward and the operating theatre at Ruharo Mission Hospital eye Centre and provision of essential equipment including (biometry machines) and consumables including a wide ranges of IOLs. ensuring refraction and provision of eyeglasses/ prescriptions of patients operated for cataract. The project will improve the referral system and achieve efficiency gains in the patient flow to strengthen the health system.
The community will benefit from improved awareness of eye health and treatment options and increased access to eye care services. A total of 300,000 people will be reached with community education/ sensitisation sessions on eye health of which 191,000 are expected to have their eyes screened. The project plans to conduct cataract surgery for 14,700 people, distribute spectacles to 7,346 people and provide other sight saving interventions to approximately 88,151 people.
As required under ANCP funding the project is required to meet the ‘Significant’ DAC Gender Mark and ‘Significant’ ANCP Disability Marker. Therefore, the project has a focus on activities that engaged marginalised people, women, people with disability, indigenous people and people living in remote hard to reach areas. The project will pay particular attention to reducing access barriers for marginalised people.
Outcomes and outputs
Outcome 1: Communities are aware of cataract and other eye health services and actively seek services.
- Output 1.1: Community eye health education and promotion of eye health services is delivered in communities including pathways to access eye care services.
- Output 1.2: Peek technology solutions rolled out for screening and referral.
Outcome 2: Equitable access to eye health services for women, men and other under-served populations in target areas.
- Output 2.1: Awareness is increased among eye health workers on gender and disability related barriers to accessing eye health.
- Output 2.2: Eye health awareness raising activities specifically consider and address barriers to access services for under-served populations especially women and persons with disability.
- Output 2.3: Cataract surgical outreach services are delivered to under-served areas.
Outcome 3: Quality cataract and other eye health services available and delivered by implementing partner hospitals.
- Output 3.1: Implementing district hospitals have adequate capacity to screen, refer and deliver quality cataract and other eye health services.
- Output 3.2: HREH trained to deliver quality and comprehensive eye health services.
- Output 3.3: Cataract Surgical outcomes improved and monitored.
Outcome 4: Accurate eye health information is available to inform decision.
- Output 4.1: Rapid Assessment of Avoidable Blindness conducted in target area.
- Output 4.2: CSOM systems are established and outcome data available for all cataract surgeries.
- Output 4.3: Eye health data is integrated in HMIS and routinely analysed and reported to inform program decision.
- Output 4.4. Advocacy work will be done to strengthen HMIS and increase eye health indicators at national level.
Outcome 5: Local governments in target area are effectively managing and resourcing eye health in respective districts.
- Output 5.1: Technical support provided to district health offices to plan, manage, and coordinate eye health services.
- Output 5.2. Advocacy work will be carried out with district health offices for an incremental budget allocation for eye care.
Part 2) Achieving Equity Through Integration of Health Services for Women
In Uganda, the Health Sector Strategic Plan has focused on ensuring universal access to a minimum healthcare package of services. While utilization of health services has improved over time, inequities in the use of health services have persisted. For instance, the Uganda National Household Survey Data shows that more women than men reported receiving no care and up to 50% of those reporting having received no health care are in the lowest and second lowest income quintile of the population.
Women in Uganda face considerable barriers to access health services. Evidence from various studies shows that distance to health facilities as well as the economic opportunity costs associated with travel to the health facility[1] are some of the major barriers to accessing health services. This project seeks to address these barriers by integrating eye health into maternal and other health services such that women accessing eye health services at Ruharo Mission are encouraged to seek other health services and vice versa. The rationale for this project is to improve health equity for women through integration of health services and thereby improving health outcomes for women through coordinated care and promoting preventive health care.
Lack of information on availability of services and transportation to the service points have been reportedamong the commonest factors limiting access to eye care, maternal and other health services. The project will seek to demonstrate the effectiveness of integrating eye health with maternal and other health services to increase access to services for women at a single point. This would be an effective approach to not only reducing the cost of transport and time spent accessing health services but an opportunity to access health services that may otherwise not been perceived as important due to lack of awareness.
The project has a two-pronged approach, 1) to integrate maternal and other health services for women accessing eye care services under the current FHF funded project, 2) integrate screening for eye health conditions among women attending the maternal child health clinic and other health services at Ruharo mission hospital. The project focuses on integration and delivery for the following continuum of services:
- Eyecare screening and service provision
- Screening for high blood pressure, care, and referral
- Diabetes mellitus screening, care, and referral
- Cervical cancer screening
- Breast cancer screening
- Eye screening for children attending child health days
- Family planning
- Counselling (including gender-based violence, and other health conditions)
- HIV counselling and testing
- Sexually Transmitted Infections testing and treatment.
A detailed evaluation plan for this project is currently being developed independently and is slated for completion by November 2023. The detailed evaluation plan will inform evaluation activities for this project.
Outcomes and outputs
Outcome 1: Integration and referral between services improves health service uptake.
- Output 1.1: Health education and promotion delivered to women attending Ruharo mission hospital including pathways to access the health services.
- Output 1.2: HMIS at Ruharo improved to capture data on service delivery.
- Output 1.3: Referral pathway rolled out.
- Output 1.4: Integrated health services delivered at both static and through outreach.
Outcome 2: Early detection and treatment of health conditions
- Output 2.1: Health workers trained to triage and refer as appropriate.
- Output 2.2: Patients treated or referred as appropriate.
- Output 2.3: Patients at eye clinic camps screened for other health conditions.
Outcome 3: Improved patient/ client satisfaction with the health system
Output 3.1: Patient exit interviews conducted to assess satisfaction with the services.
Project partners
Christian Blind Mission (CBM)
Mbarara University of Science and Technology
Mbarara University Referral Hospital Eye Centre
Ruharo Mission Hospital
Purpose of the evaluation
The purpose of this evaluation is to assess whether the project, including the two parts, achieved its intended objectives and how lessons gained from the project can be integrated and used for ongoing improvement. This means that there are two overall purposes of this evaluation:
- Accountability – proving what has been achieved.
- Assess to what extent the two parts of the project have achieved their objectives and key outputs, outcomes, and impact.
- Verify the monitoring data that has been collected by the in-country program teams throughout implementation.
- Improvement – understanding learnings to inform decisions about future work.
- Capture lessons from each of the two parts regarding project implementation to help improve current and future programming in the country and support the development of future projects.
- Assess the potential scalability of the project focused on older populations in the country.
- Identify if the gender project can be scaled up and replicated across different contexts (part 2)
- Identify the factors that will enable scaling-up and replication of the gender project (part 2)
Evaluation Methodology
For Part 1, the consultant is expected to conduct a mid-term evaluation, reviewing the evaluation questions, and developing the necessary tools to collect data. For Part 2, the consultant is expected to complete the final evaluation and will be provided with a package of pre-designed methodology and tools that will only require minimal review and local contextualisation (if necessary).
The primary intended users of this evaluation will be:
- Eye health Implementing organizations including CBM.
- Ministry of Health in Uganda.
- International eye health sector NGOs, the Australian Department of Foreign Affairs and Trade
- FHF Uganda team
- FHF Strategic Initiatives Team
- FHF Senior Equity and Inclusion Advisor
The evaluation will also be of interest to a range of secondary users, including:
- FHF divisions (Program Development Team, country program teams, Country Support Network and Development Effectiveness Team)
- Donors and potential partners
Evaluation scope and key questions
This evaluation covers the activities of two sub-projects and will be structured around the following evaluation criteria: Effectiveness; Efficiency; Relevance; Impact; Coherence; Equity and Sustainability.
Additionally, the evaluation should capture lessons learned through implementation, including an identification of enablers and barriers for implementation.
An outline of the tentative Key Evaluation Questions and sub-questions for Parts 1 and 2 of the projects is provided in Annex 1. The consultant will have the possibility to propose expanding the set of questions to suit the evaluation, with any changes to be reviewed and approved by the FHF Evaluation Working Group prior to finalization.
Approach
FHF’s Country Program Manager for Uganda will recruit a research consultant/institution to conduct the evaluation. The selected evaluation consultant will be required to prepare a detailed research methodology in collaboration with FHF Uganda and other relevant departments.
For Part 1, The consultant will be required to propose the right methodology for the evaluation but at the minimum, the chosen methodology should be broad enough to accommodate both quantitative and qualitative methods.
For part 2, a detailed evaluation plan will have already been developed and will be provided to the preferred provider upon contract execution. The provider will be given the opportunity to adapt the plan in collaboration with the Fred Hollows Foundation team.
Information to answer the evaluation questions in the 2 parts will be gathered through the following processes:
- Review of project documents, reports, and data, including data verification with project staff.
- Focus group discussion with project partners and beneficiaries.
- Key informant interviews with implementing partner, eye health service providers and other stakeholders.
- Online surveys
- Review of Health Information Management System
Note: dependent on the agreed final methodology presented by the selected consultant, the evaluation will require approval by the relevant Ethics Committee prior to commencement of data collection. This will be assessed based on the presented methodology, with the ethics approval application for the 2 project parts combined to be managed by the consultant with support from the Fred Hollows Foundation.
Deliverables
The following deliverables are expected from the consultant in completing the assignment:
- Evaluation/Inception Plan and methodology, including timelines and project schedule (except for Part 3).
- Completed data collection tools in line with evaluation questions.
- Prepare and obtain ethics review to conduct the evaluation.
- Presentation and validation of preliminary findings including outputs from data collection (e.g. survey results in spreadsheet format)
- Draft and final evaluation reports (one set for each of the parts) including also a summary report, and summarized slide decks for dissemination of findings.
- Presentation of evaluation summaries (methodology, findings, recommendations, and learning) to broader Foundation staff. The content and timing of the presentation will be negotiated with the consultant.
- The Foundation shall require all raw data collected/generated during the evaluation provided in an agreed format. A backup of raw data shall be provided in a format accessible to The Foundation.
The Foundation will provide the following:
- Focal person to support in undertaking the assignment.
- Provide documents required for desk review.
- A detailed evaluation plan for Part 2.
- Give feedback on the evaluation plan and methodology.
- Provide feedback on draft data collection tools, reports, and other deliverables.
- Support coordination between the consultant and partners during the entire evaluation activities.
- ANCP Evaluation template which the required DFAT funding acknowledgement
Schedule
The evaluation is expected to start in November 2023 and be completed by February 2024. This will include an overarching submission of ethics approval, desk reviews, fieldwork, interviews, writing 2 distinct reports for the 2 parts and final presentations to FHF and stakeholders. Due to a likely delay in the ethics review process, the consultant will be expected to compile and submit the application for ethics review in November, with approval and field work to commence in January 2024.
Evaluation team & qualifications
This evaluation will be contracted to an independent evaluator or team who will work closely with The Foundation staff in the design and implementation of the evaluation. The Foundation seeks to engage the services of an independent individual or registered organization, that have the following experiences and expertise in project/program evaluation.
At least a master’s degree in medicine, Public Health, Ophthalmology, or related health sciences.
- Demonstrable experience in evaluation of public health programs including health systems assessments.
- Extensive knowledge and practical experience in conducing evaluations
- Experience in the development and use of quantitative and qualitative data collection tools and evaluation methods
- Experience in preparing ethics application and obtaining ethics approval from a research institute for qualitative and/or mixed-method research.
- Demonstrable experience writing evaluation reports.
- Strong knowledge and experience working with the Ugandan health care systems and overall understanding of Ugandan health sector policies.
- Strong experience in using equity framework(s) to evaluate progress against equity objectives.
- Strong analytical skills
- Excellent spoken and written communication skills in English.
- Computer literacy, including expert use of Microsoft Office package
- Experience in eye health and/or gender equity will be an advantage.
Confidentiality
The consultant (s) agree to not divulge confidential information to any person for any reason during or after completion of this contract with The Foundation. Upon completion or termination of this contract, the consultant (s) undertake to return to The Foundation any materials, files or property in their possession that relate to the work of The Foundation. The consultant will be responsible for safety, security and administration of primary and secondary data collected from FHF or otherwise.
Intellectual Property
All intellectual property and/or copyright material produced by the evaluator/s whilst under contract to The Foundation remain the property of The Foundation and will not be shared with third parties without the express permission of The Foundation. The evaluator/s are required to surrender any copyright material created during the term of the contract to The Foundation upon completion or termination of the contract.
Safeguarding People
The Fred Hollows Foundation is committed to ensuring that its activities are implemented in a safe and productive environment which prevents harm and avoids negative impacts on the health and safety of all people, particularly children, vulnerable people, and disadvantaged groups. The Foundation has a zero-tolerance approach to sexual exploitation, abuse, and harassment of any kind. All personnel including contractors/consultants are expected to uphold and promote high standards of professional conduct in line with The Foundation’s Safeguarding People Policy including Code of Conduct. Contractors/consultants will be expected to sign and adhere to The Foundation’s Safeguarding Code of Conduct and provide any background checks as required.
INSURANCE
Any consultants involved in this evaluation will be required to have in place insurance arrangements appropriate to provision of the requirements in this Terms of Reference including travel insurance.
Ethical and other Considerations
The evaluator and evaluation team are expected to maintain high professional and ethical standards and comply with The Foundation’s Research and Evaluation Policy. The Foundation is committed to ensuring a safe environment and culture for all people, including children, with whom we come in contact during our work. All members of the evaluation team will be required to comply with The Foundation’s Safeguarding People Policy and sign the Safeguarding Code of Condu
ANNEXES
Annex 1 – Key evaluation questions
The project evaluation will be focused on each of the two project components discussed which includes access to cataract services, and gender and health parts.
Part 1: Equitable Access to Effective Cataract Services
This is a mid-term evaluation of Part 1 ‘Increasing Equitable Access to Effective Cataract Services in West, South and South -Western Uganda 2021-2025. The following indicative evaluation questions will be used to assess the status of the project. However, the consultant will have latitude to expand the set of questions to suit the evaluation.
Evaluation Criteria
Key evaluation question(s)
Relevance
How appropriate was the project approach in addressing eye health needs in the target districts?
Effectiveness
To what extent did the project contribute towards increased uptake of eye health services?
To what extent has community awareness been supported and tailored towards the needs? Which existing gaps should the project address? To what extent have community systems and pathways been established to enhance access to eye care services?
To what extent have the PEEK technology enhanced screening and referral services? What are the existing gaps that the project should address?
To what extent has the project delivered training of health workers at different cadres to effectively deliver quality eye services? What are the existing gaps?
To what extent have the targets for cataract surgeries been met? To what extent has the project achieved good cataract surgery outcomes? What are the existing gaps?
To what extent are post operative follow up conducted? What improvements are needed?
To what extent have CSOM systems been established to manage outcome data for all cataract surgeries?
To what extent have Rapid Assessment of Avoidable Blindness (RAAB) informing eye health service delivery?
To what extent are eye health data collected, analysed, and used for decision making? What improvements are needed?
To what extent did the technical support provided to district health offices improve eye health service delivery? What improvements are needed?
Efficiency
To what extent have The Foundation, implementing partner and other eye health partners worked together to ensure the project objectives are realized?
Sustainability
To what extent are the project results in eye health service delivery going to be sustained at project locations beyond the project period? What interventions are required to increase the sustainability of the project results?
Impact
To what extent did the project contribute to an increase in the cataract surgical rate in the target districts?
Equity
To what extent has the project enhanced awareness among health workers on gender, disability, and equity?
To what extent have the project achieved the DAC Gender Markers as per ANCP requirements (Refer ANCP guidelines on DAC gender Marker). What needs to be enhanced to promote gender equity?
To what extent has the project met the DFAT ANCP disability marker for level one on disability mainstreaming and how can the project improve to Twin Track marker (Refer to DFAT ANCP disability inclusion guideline? What needs to be enhanced to promote disability inclusion?
To what extent have the underserved areas been targeted and reached for eye health service delivery? What needs to be enhanced to improve access to services in the underserved areas?
Part 2) Achieving Equity Through Integration of Health Services for Women
This is an endline evaluation of Part 2 of the project that covers a subset of activities implemented from July – Dec 2023. The following indicative evaluation questions will be used:
Evaluation criteria
Key evaluation question(s)
Effectiveness
KEQ 1: To what extent did the integration of health services increase women’s use of those services?
SQ 1.1: To what extent did the integration of eye health and other health services increase the use of eye health services and vice versa?
SQ 1.2: What were the key barriers to access addressed by the pilot? What barriers remained?
SQ 1.3: Did the integration of health services improve early detection and treatment?
SQ 1.4: Did the provision of integrated services improve patient experience?
Sustainability
KEQ 2: To what extent are the integrated services made through the pilot project likely to be carried forward by the hospital?
SQ 2.1: What were the barriers to the hospital integrating its services without the Foundation’s involvement?
SQ 2.2: What are the implications of the increased use of health services for the hospital?
SQ 2.3: What are (if any) the hospital’s resourcing requirements to meet increases in demand and continue delivering integrated services? Can the hospital meet these needs?
SQ 2.4: Are the pilot project’s activities aligned to the priorities of the hospital?
Scalability
KEQ 3: Can the pilot be scaled to enable greater reach within Uganda? What are the drivers of scalability?
SQ 3.1: What are the key factors that drive scalability of the pilot?
SQ 3.2: What are the barriers and enablers to delivering the Uganda pilot model at scale?
Effectiveness
KEQ 4: To what extent can the pilot be replicated in a context other than Uganda or with other clinical services? What are the drivers that enable or are barriers to
replication?
SQ 4.1: Can the model be applied to different health issues?
SQ 4.2: Is the model applicable in different contexts?
SQ 4.3: What are the features of the pilot that are inherently unique to the Ugandan and/or clinical service context? Is this feature critical to the delivery of the pilot?
Efficiency
KEQ 5: In what ways did the pilot project facilitate greater collaboration between
different clinicians to deliver integrated health services?
SQ 5.1: In what ways did the pilot project increase collaboration between different clinicians?
SQ 5.2: Did the pilot involve stakeholders at the right level of collaboration to enable the delivery of outcomes?
SQ 5.3: What were the key drivers of attitudinal and behavioural change for greater collaboration?
[1] Hooft A, Nabukalu D, Mwanga-Amumpaire J, Gardiner MA, Sundararajan R. Factors Motivating Traditional Healer versus Biomedical Facility Use for Treatment of Pediatric Febrile Illness: Results from a Qualitative Study in Southwestern Uganda.
How to apply
This evaluation is being commissioned by The Fred Hollows Foundation Uganda. The Uganda program team will work closely with selected consultant (s). Individual or institutions interested should submit a cover letter, resume, technical and financial proposals to The Foundation’s Kenya regional office by 21st November 2023. Please reference “Consultancy to Conduct Project Evaluation for the Increasing Equitable Access to Effective Cataract Services in West, South and Southwestern Uganda” and send to the following address: [email protected]. Only submissions with complete documents as stated in the TOR and those meeting minimum requirements will be considered. Qualified consultants may be subjected to a background check on child protection as a condition for engagement.