RESULTS BASED FINANCING (RBF) FOR INCREASES IN (EFFECTIVE) CATARACT SURGICAL COVERAGE & EQUITY AND FINANCIAL LEVERAGING NHIF: IMPACT EVALUATION BASELI

IMPACT EVALUATION BASELINE ASSESSMENT

Background

The Fred Hollows Foundation team in Kenya (FHFK) has been partnering with the Government of Kenya since 2004 to improve and protect eye health for Kenya’s population. During this time FHFK has contributed to a tripling in cataract surgical rates it supported counties and enabled eye health policy and systems nationally. FHFK support to date has comprised direct subsidisation of an agreed volume of eye surgeries at its partner facilitates coupled with direct provision of equipment and supplies. As the Government of Kenya prioritises Universal Health Coverage, including expansion of the National Hospitals Insurance Fund (NHIF), FHFK is shifting toward leveraging these important policies and financing mechanisms to secure more sustainable financing for cataract and other eye health services.

In 2017 alone over 1 million years lived with disability in Kenya were due to cataract, representing a potential economic welfare loss of over $400 million US dollars to the country. Cataract surgical coverage in Kenya remains low and large socioeconomic and geographical inequities exist. Despite some delays with payment and limited involvement from health care providers in setting NHIF reimbursement rates for services delivered, hospitals in Kenya are known to value the additional revenue that reimbursements from NIHF represent. For cataract surgery, whilst somewhat unclear and variable insurance payments have been reported, common reimbursements to public sector facilities range from around 20,000 – 50,000 Kenyan Shillings (KES). These are significantly higher than out of pocket charge rates (generally around KES 7000 – 10,000) and likely higher than provider costs of delivery, meaning that cataract reibursement revenue can represent a net gain to facilities / counties[1].

Rolling out subsidised national health insurance coverage for poor and vulnerable groups in Kenya has progressed more slowly than in many countries. A World Bank supported pilot of a Health Insurance Subsidy for the Poor (HISP) have not scaled up as rapidly as planned and, government initiated free coverage for vulnerable groups such as for people with disability or older Kenyans has been effectively halted in a number of instances where NHIF had no funding to sustain such coverage. Increased financial protection and health service utilisation (including for eye care) associated with health insurance coverage therefore remains inequitable (Barasa et al 2018). NHIF is facing challenges in the sustainability of the fund with new higher health care costs associated with COVID-19 in addition to ongoing issues of high drop out rates and variable budget transfers to support fully subsidised coverage for vulneralbe groups.

Universal Health Coverage, however, remains a high priority for both national and county governments as one of the Big 4 development plan priorities of the Kenyan president. A number of counties are leading the way, allocating portions of their own budgets to cover populations with health insurance (NHIF or own county schemes). However, competing priorities for limited and strained health care budgets have meant that implementation of some of these commitments has been variable.

The Sustainable Models of Eye Health Financing (SMEHF) project in Kenya

The “Sustainable Models of Eye Health Financing” (SMEHF) project in Kenya, running from 2020 to 2022, is being piloted in five counties – Homa Bay, Kisumu, Kitui, Makueni and Meru. The project is taking a whole of systems view to support transition to greater sustained government financing for cataract services in Kenya. It seeks to demonstrate a system at county level where greater government subsidisation of NHIF coverage for poor and vulnerable older Kenyans and performance of cataract surgeries under such coverage, can return direct revenue benefits from NHIF reimbursements to providers (on top of social and economic benefits from restored sight). Hence it aims to demonstrate that investing in county ability to provide greater numbers of cataract surgeries and at a quality to attract greater take up of these at public facilities (versus private) can return benefits to county governments (via reimbursement revenue), to NHIF (from having longer term subsidised memberships and from reduced reimbursements versus at private facilities) and to patients and their families (including lower out of pocket costs) all supporting progress toward UHC goals.

This pilot demonstration project includes

1. a cost-revenue study of cataract surgery across 10 hospitals (including county hospitals in the five project sites)
2. initial package of support to basic equipment and supplies provided by FHFK to enable provision of cataract surgeries at the five county hospitals
3. efforts to raise community awareness of the availability of surgery to restore sight for those with cataract and benefits under NHIF
4. support to outreach eye screening jointly attended by NHIF, office of social services (for identification of poor and vulnerable) and eye health teams to reach greater numbers of older vulnerable Kenyans
5. the piloting of results-based financing approaches that support supply side improvements to deliver cataract surgery commensurate with increases in county subsidised NHIF coverage for poor older Kenyans and / or cataract surgeries performed under NHIF(versus out-of-pocket payment).

Complementary support to both increased demand for health services along with strengthened supply to meet this demand are long known to be essential in progressing universal health coverage.

FHFK’s SMEFH project therefore has three linked key objectives:

1. To increase NHIF coverage for Kenyans, particularly for poor and vulnerable, needing or at higher risk of needing cataract surgery
2. To strengthen the supply (availability and quality) of cataract surgical services particularly in public sector county level hospitals
3. To increase cataract surgical rates under NHIF, particularly for poor and vulnerable older Kenyans

The SMEHF project’s inception phase ran from September 2019 – March 2020 and some data collection started in October 2020; but PBF contracts were delayed by the pandemic, hence official commenced was in November 2021. Project commenced providing the initial package of basic equipment to participating county hospitals to undertake quality cataract surgeries and commencing demand generation through supporting eye care teams from county hospitals to undertake outreach eye screening in selected communities including identification and referral of those requiring cataract surgery and raising awareness regarding NHIF and its cataract surgical inclusions. In addition, significant working relationships have been built between FHF, county health departments, county social development and NHIF offices supporting shared commitment to the SMEHF project goals.

INCORPORATING A RESULTS BASED FINANCING PILOT

A results-based financing pilot was introduced into the SMEHF project at roughly its halfway implementation point late in 2021 to explore whether this can further SMEHF goals to sustainably increase NHIF coverage, and cataract surgical coverage under it, particularly for poor and vulnerable Kenyans. SMEHF supported counties were allocated to one of two alternative RBF mechanisms:

Mechanism 1: Performance Based Contract with Counties (PBCC): Which incentivises county government full subsidisation of NHIF coverage (for a minimum of two years) for poor and vulnerable older Kenyans. For each registration of this type FHFK allocates KSh 4000 on a quarterly basis toward procurement of county hospital defined supply side needs (equipment, supplies, training etc) for enhanced cataract surgical provision. This approach is implemented in Homabay and Makueni Counties.

Mechanism 2: Hospital Performance Based Fund (HPBF): Which incentivises public county hospital provision of NHIF covered cataract surgeries. For each NHIF fully covered patient (no additional out of pocket cost) cataract surgery provided, FHFK allocates KSh 4000 on a quarterly basis toward procurement of county hospital defined supply side needs (equipment, supplies, training etc) for enhanced cataract surgical provision. This approach is implemented in the counties of Kisumu, Kitui and Meru.

THE IMPACT EVALUATION OF RBF UNDER SMEHF

The Foundation is seeking the skills of a suitably qualified consultant to undertake the comparative analysis of the above funding mechanisms. This section of the Terms of Reference describes the objectives, methods, activities, and outputs expected of the professionals applying for this consultancy.

PURPOSE and use OF THE IMPACT EVALUATION

The overarching purpose of the impact evaluation is to explore the extent and types of contribution results-based financing approaches makes to increases in volume, equity, quality, financial protection, and potential sustainability (post donor financing) in cataract surgery in Kenya versus traditional forms of input-based support as well as the health systems and contextual that influence the impact the RBF approaches have had.

The impact evaluation will provide evidence to inform if and how The Fred Hollows Foundation (and others) might support results based financing and other forms of strategic purchasing and systems and financing leveraging approaches to better support sustainable growth in UHC inclusive of eye health into the future in Kenya, but also in other country programs.

The evaluation will also be used by Government of Kenya partners to consider the potential for including performance-based payment incentives in financing mechanisms for priority health interventions going forward.

KEY OBJECTIVES

Key objectives of the impact evaluation are to

1. Describe the relative cost and contribution of RBF broadly versus a traditional input-based support model (usual care) to sustainable and equitable increases in effective coverage of cataract surgery whilst promoting financial protection (both together representing cataract surgery in UHC) drawing out intended and unintended consequences. §
2. Determine the relative changes in each of the key outcomes arising from the PBCC versus HPBF mechanism of RBF piloted under the SMEHF drawing out reasons for this.
3. Capture the contribution of the RBF versus usual care and in combination with other forms of support provided under SMEHF by comparing trends in same counties pre-post RB interaction together with cross sectional comparisons with non SMEHF FHFK supported counties.

§ Unintended consequences include benefits like use of NHIF for other conditions by the principal member and the dependants or drawbacks which may include discrimination of those without insurance by the providers. Information on unintended consequences will be gathered through provider and beneficiary interviews.

These objectives are informed by the original project implementation plan (relevant section in Appendix 1).

KEY EVALUATION QUESTIONS

Key question to be addressed:

1. Which of the two SMEHF mechanisms resulted in the greatest increases in cataract surgical rates under NHIF and why?
2. How does the quality of surgery outcomes compare between the two funding mechanisms?

Approach to the impact evaluation

Robust evaluation is essential to determining the effectiveness of the RBF pilot approach in supporting sustainable increases in cataract surgical rates particularly for poor and vulnerable Kenyans whilst lowering out of pocket costs.

Type of study

FHFK seeks to commission an impact evaluation of the SMEHF project that includes three comparisons of effectiveness;

1. An effectiveness comparison between the two different results-based financing mechanisms as above
2. An effectiveness and cost effectiveness comparison between the newer SMEHF systems type approach and the traditional approach building in measures of sustainability and equity

COMPARISONS AVAILABLE

Given that budget and timing of the SMEHF project does not allow a fully randomised or step wedge design and roll out, the impact evaluation will likely (subject to final discussions with the selected consultants) employ a cross sectional design using counties as the unit of comparison[2].

In another five counties the traditional input-based funding approach of subsidy payment to surgeries coupled with unconditional direct provision of equipment, training and funding of outreach and other complementary activities remains under a separate support project. These counties include three newer counties with similar starting timing as the SMEHF project, two of which (Kilifi and Tharaka) will act as comparator counties to determine the relative costs and outcomes of the RBF approaches with this traditional input-based funding (further details are provided under the methodology section below).

The introduction of the RBF later in the project provides a useful opportunity to also explore the relative contribution of the RBF versus the other forms of support under SMEHF provided solely in the first year of implementation.

Implementation

Comparisons should be made both between SMEHF counties as with different financing mechanisms and between key SMEHF counties, one undergoing each financing mechanism pilot, and comparison counties using a traditional direct support approach. Where possible, counties should be matched on key characteristics likely to influence the outcomes. Where this is not possible statistical regression / econometric techniques should be employed to, as far as possible, account for these potential influences.

The term results based “allocation” is used rather than “payment” as monetary transfers will not be made. The KSh 4000 per indicator output achieved (plus up to 15% based on a quality multiplier[3]) will be set aside by FHFK each quarter to strengthen county hospital eye service provision. County hospitals and health departments will develop a prioritised plan of supply side needs to improve cataract surgical provision that can include equipment, supplies and professional development. These will be referred to quarterly jointly by the county hospital, health department and FHFK. Procurement will ultimately be made by FHFK. This approach is aimed to avoid perverse incentives that can occur with monetary transfers, leverage benefits of lower cost and time responsive procurement available to FHFK whilst protecting a high level of facility autonomy in decision making crucial for results-based financing. Such an allocation but external procurement approach has been used in previous results-based financing pilots in Kenya[4].

Where the cost revenue study experience of county hospitals and governments under the SMEHF project illustrates the benefits of investing in cataract surgery provision under NHIF it is hoped that in the medium term (3 – 5 years of implementation) the FHFK supply side support under the RBF would no longer be necessary in SMEHF counties. However, a robust evaluation of the contribution of the RBF and other aspects of the SMEHF project is essential in providing lessons for consideration of scale up both by the Government of Kenya and The Fred Hollows Foundation in Kenya and beyond.

METHODOLOGICAL APPROACH

Findings from a number of previous performance-based financing pilots in the health sectors of low- and middle-income countries were inconclusive in determining the contribution of performance payments given concurrent general increases in resources to health or other reforms that could have equally contributed to improved outcomes and / or before after designs that cannot take such changes into account.

This RBF pilot has been specifically designed and timed with three forms of comparison being available

1. comparison between the Performance Based Contract with the County (PBCC) and Hospital Performance Based Financing (HPBF) for results-based financing to determine any differences in terms of impacts on volume, equity, quality, and financial protection
2. comparison between PBCC results-based financing and the traditional approach to determine any differences in terms of impacts on volume, equity, quality, and financial protection
3. comparison between HPBF results-based financing and the traditional approach to determine any differences in terms of impacts on volume, equity, quality, and financial protection

There are important differences between the PBCC and HPBF mechanisms that could contribute to such differences. The PBCC incentivises NHIF coverage only and does not pre-determine whether and where the person covered and identified with cataract goes to receive their surgery. This may mean that volume of surgeries may be lower (if less people take up the surgery) or the proportion undertaken in public facilities increases less than in private (which will shower as lower growth in volume at public facilities and potentially higher out of pocket costs given the higher costs often faced at private facilities even when covered by NHIF (ref).

BASELINE ANALYSIS REQUESTED

It is therefore important to determine in both SMEHF models and in comparison.

Resource allocation: Draw the flows of income and expenditure under each scheme and put a rough estimate of KES on each flow: that is,

• The estimated resources directly used by FHFK on infrastructure for cataract surgeries (including equipment, supplies and professional development) in the beginning for set up
• The estimated resources directly used by government hospitals on infrastructure for cataract surgeries (including equipment, supplies and professional development) in the beginning for set up
• FHFK incentive allocation for the RBF indicators attained by the partners in each approach
• Subsidy in KES from government for NHIF registration for each patient with cataract
• Out of pocket expenditure by patients if any on cataract surgery in each approach
• Key informant’s Perceived financial barriers
• Spending by hospitals into equipment and staffing to get the surgery running in each approach

Baseline indicators:

• • Number of cataract surgeries performed under NHIF reimbursements vs number of cataract surgeries conducted through out of pocket in each approach,
    – Total amount (KES) generated from NHIF reimbursement vs amounts generated through out of pocket for cataract surgeries done in each approach
    – Number and rate of cataract surgeries conducted for the poor and other vulnerable (women, older people, rural residents) through NHIF in each approach
    – Percentage of cataract surgeries considered of good quality outcomes in each approach (examples: patient discharged successfully, no need for re-surgery within 28 days, no postoperative infections, and/or evidence of vision restored).
    – Reporting rates into the Kenya Health Information System2 (KHIS2) for number of reported cataract surgeries for the different approaches

For the comparisons on subsequent years, the following data also needs to be extracted

• Draw the flows of income – expenditure under each scheme and place a rough estimates of $ on each flow: that is,
  • Subsidy $ from FHFK and central government in the beginning for setting up (equipment, etc.) – investment
  • Subsidy $ from FHFK and central government for each registration
  • $ out of pocket expenses from patients if any
  • Spending by hospitals into equipment and staffing to keep the surgery running. There might be differences in how hospital in each county set up their operation – but this is related to economies of scale.

The consultant will design relevant data extraction forms and obtain the data for each year the period of implementation from 2020 (baseline before new funding mechanism implemented) and up to the most recent year of data available at time of data collection.

Outcomes of interest: If possible and meaningful (enough sample size), the outcomes listed below should be broken down by each hospital for each county. Note that the question here is about what each hospital did, under each scheme. We want to look for economies of scale.

key OUTCOME measures for the impact evaluation

Primary outcomes (at 1, 2, 3 years as appropriate)

Aligned with the objectives of SMEHF the primary outcome measures for the SMEHF should include:

1. changes in the cataract surgical rate in study counties (overall)
2. changes in the cataract surgical rate under NHIF
3. changes in the cataract surgical rate among those poor and other vulnerable such as women, older people, rural residents (through GoK social protection mechanisms / eligible)

The following intermediate and secondary outcomes should also be addressed:

Intermediate outcomes (at 1, 2, 3 years as appropriate)

1. changes in the NHIF registrations under government subsidy
2. changes in informants’ perceived financial barriers and out of pocket costs associated with cataract surgery
3. quality – increased internal reporting and use of CSOM data and essential eye health data

Secondary outcomes (at 1, 2, 3 years as appropriate)

1. changes in cataract surgeries performed in private versus public practice (under NHIF)
2. quality – reductions in proportion of surgeries with poor outcomes over time (infections, need for re-surgery, not restoring vision)

Other evaluation activities

Most of the information for this consultancy should be extracted from project documentation and service databases/records. In-depth interviews with key program informants -if required- can be conducted to supplement quantitative information with qualitative details to meet the requirements of Output 3.6 “lessons learned” and Output 5.1 “advocacy and communications plan” (See appendix 1).- The consultant is expected to design the data extraction forms and key informant interview instruments and obtain input and endorsement from the Foundation staff before submission for ethics approval. Deliverables: A detailed report making data-driven inferences on which of the two funding mechanisms generates the greatest increases in cataract surgical rates under NHIF is expected. In addition, a descriptive analysis of the comparative quality of surgery outcomes would be anticipated if feasible. Or a practical recommendation for how this could be measured in the future. The consultant is expected to participate in the dissemination of results to intended audiences in Kenya.

Timeline

This consultancy will be a maximum of 24 weeks duration and activities are broadly specified as illustrated below.

Activity

• Finalise detailed project proposal with data collection instruments designed
• Teleconference(s) with FHF for approval/progress
• Submission to ethics & secure approval
• Contact data custodians and negotiate access
• Data retrieval and cleaning/processing
• Data analysis
• Draft report delivered for FHF comment
• Revisions/amendments to report and finalisation
• Dissemination meetings with relevant stakeholders
• Manuscript development and publication

Evaluation team & qualifications

This evaluation will be contracted to an independent evaluator, team or organization who will work closely with The Foundation staff in the design and implementation of the baseline impact evaluation. The Foundation seeks to engage the services of an independent individual or registered organization, that have following experiences and expertise in project/program evaluation.

At least a master’s degree in Health Economics or equivalent, Public Policy, Statistics, Development studies, Public Health, Demography or related social sciences. PhD in these fields is desirable.

• Demonstrable experience in evaluation of public health programs including health systems assessments and impact evaluations.
• Experience in development and use of quantitative and qualitative data collection tools and evaluation methods
• Strong knowledge and experience working with the Kenyan health care systems and overall understanding of Kenyan health sector policies
• Strong analytical skills
• Excellent spoken and written communication skills in English

The following additional skills and experience are highly desirable:

• Knowledge and experience in eye health and a sound understanding of programs and strategies relating to eye health in Kenya

Management and logistics

This evaluation is being commissioned by The Fred Hollows Foundation Kenya. The Kenya program team will work closely with selected consultant (s).

Individual or institutions interested should submit a cover letter, resume, technical and financial proposals to The Foundation office in Kenya by 31st October 2022. Please reference “Consultancy for Impact Evaluation – Sustainable Model for Eye Health Project” and send to the following address: [email protected]. Only submissions with complete documents as stated in the TOR and those meeting minimum requirements will be considered. Qualified consultants and institutions may be subjected to a background check on child protection as a condition for engagement.

Confidentiality

The evaluator/s agree to not divulge confidential information to any person for any reason during or after completion of this contract with The Foundation. Upon completion or termination of this contract, the evaluator/s undertake to return to The Foundation any materials, files or property in their possession that relate to the work of The Foundation. The consultant will be responsible for safety, security and administration of primary and secondary data collected from FHF or otherwise.

Intellectual Property

All intellectual property and/or copyright material produced by the evaluator/s whilst under contract to The Foundation remain the property of The Foundation and will not be shared with third parties without the express permission of The Foundation. The evaluator/s are required to surrender any copyright material (electronic and paper-based) created during the term of the contract to The Foundation upon completion or termination of the contract. An opportunity exists for the consultant to attain co-authorship right of manuscripts deriving from this work in conjunction with the Foundation’s country team, but these will require written approval from the Foundation office in Australia. The consultant is not authorised to report on the methods or findings at venues or to audiences beyond the scope of dissemination plan for this project.

Safeguarding People

The Fred Hollows Foundation is committed to ensuring that its activities are implemented in a safe and productive environment which prevents harm and avoids negative impacts on the health and safety of all people, particularly children, vulnerable people, and disadvantaged groups. The Foundation has a zero-tolerance approach to sexual exploitation, abuse, and harassment of any kind. All personnel including contractors/consultants are expected to uphold and promote high standards of professional conduct in line with The Foundation’s Safeguarding People Policy including Code of Conduct. Contractors/consultants will be expected to sign and adhere to The Foundation’s Safeguarding Code of Conduct and provide any background checks as required.

INSURANCE

Any consultants involved in this evaluation will be required to have in place insurance arrangements appropriate to provision of the requirements in this Terms of Reference including travel insurance.

Ethical and other Considerations

The evaluator and evaluation team are expected to maintain high professional and ethical standards and comply with The Foundation’s Research and Evaluation Policy. The Foundation is committed to ensuring a safe environment and culture for all people, including children, with whom we come in contact during our work. All members of the evaluation team will be required to comply with The Foundation’s Safeguarding People Policy and sign the Safeguarding Code of Conduct.

Appendices in next 2 pages

APPENDIX 1. Activities originally planned and anticipated outputs and outcomes

GOAL (IMPACT): To reduce avoidable blindness in Kenya

END OF PROJECT OUTCOME: Sustainable increase in Cataract Surgical Rate through integrating and leveraging existing national and county level financing systems

Goal / Outcome / Output:

Activities:

OUTCOME 3: Performance based contracting, payment-financing mechanisms are functioning at county and provider level, and lessons are being learnt and applied from these new approaches / pilots

Output 3.1: Finalization of financing mechanism and development of business plans

3.1.1 Undertake scoping in the five Counties

3.1.2 Undertake facility assessment in five counties (infrastructure, equipment, commodities, Human resource, service delivery, Health information)

3.1.3 – Undertake and use results from the cost revenue study to set payment levels per output 3.1.4 Develop performance based support packages for both PBCC and the PBCH The value of these support packages should be based on good unit costing per output (cost to county of supporting NHIF registration for poor and vulnerable household or cost of conducting a cataract surgery under NHIF in hospital) which will be informed by the cost revenue study Each support package will be of clearly distinct higher value as performance increases according to four levels

1. Base level of support
2. Initial level
3. Progress level
4. High performance level

Output 3.2: Information briefs developed for engaging with counties on the 3 mechanisms (Mechanisms 1, 2, 3)

3.2.1 Information briefs developed for engaging with counties on the 3 mechanisms (Mechanisms 1, 2, 3)

Output 3.3: CHMTs and HMTs have a good understanding of and buy into their respective financing mechanism / contractual approach

3.3.1 FHF share scoping report with the counties

3.3.2 Facilitate Dialogue meetings on the financing mechanisms and contractual approaches

3.3.3. Launch of the FHF/Counties health financing for cataract

Output 3.4: Agreements and frameworks are established for the performance financing / contractual approach

3.4.1 Documentation of the consultative processes up to and including agreements

Output 3.5: Review for counties where graduation or shift between models is planned and regarding eye health equipment, commodities and HR

3.5.1 Joint annual review meeting and action planning with Homabay

3.5.2 Bi annual review meeting and action planning in (Meru, Kitui, Makueni, Homabay and Kisumu)

Output 3.6: A “How to/lessons learned” in gaining coverage for priority services (in our case, cataract) under NHIF – a case for scale up.

3.6.1 Facilitate documentation of processes, challenges, achievements, lessons learnt in the models of financing

OUTCOME 5: Increased leadership in evidence informed planning and budgeting for eye health capital investment, human resources, equipment and commodities

Output 5.1: A UHC policy is dis