RfP: Evaluation of the MSF-OCB’s Kenema Project, Sierra Leone

Summary of key information:

• Start date: March 2024

• Duration: Final report to be submitted latest August 2024

• Requirements:

  1. A technical proposal
  2. A financial proposal
  3. CV(s)
  4. A previous work sample

• Application deadline: 2359hrs CET on March 31, 2024

• Send your application to: [email protected] marked KENEM

  –> Download the Terms of Reference here. <–

Note: We value quality over quantity. Providing only the requested and necessary documentation should prove your interest, capacity, and competency in the best possible manner. This evaluation will require a site visit to the project, which will be planned during the initiation phase, through discussions with the project, the consultation group, and the SEU.

______________________

BACKGROUND

Kenema District lies in the Southe-East of Sierra Leone and has an estimated population of 840,000 individuals in 2022[1], scattered in an area of 6,053 km2. Kenema city is the capital of the district and is the second largest city in Sierra Leone with a population of 255,110 as of 2021.[2] The district is inhabited by different ethnic groups: the majority is Mende, other groups are Temne, Fullah, Limba and Kissi. The Ministry of Health and Sanitation (MoHS) is the major health service provider of primary care, covering community health programs and secondary care which includes district and referral hospitals. In 2010, the Government of Sierra Leone adopted the Free Health Care Initiative (FHCI) which aims to ensure that health care for under-fives and maternal care should be free of charge, an initiative that is dependent on external global funding.

Despite massive progress over the past 23 years, in 2020 Sierra Leone still ranked 18th in the list of countries with the highest maternal mortality globally. In 2021, 443 maternal deaths per 100,000 live births and an under-five mortality rate of 104.7 deaths/1000 live births were estimated.[3] For both women and children there are several factors influencing access to healthcare including health health-seeking behavior, financial barriers, lack of health staff at primary and secondary level, lack of equipment etc.[4] Many of these were exacerbated by the Ebola outbreak that hit the country in 2014. Lassa fever is a considerable public health problem in the region. In Kenema district, the first outbreak was declared in 1996 and Lassa fever has since been considered as endemic. The Kenema Governmental Hospital (KGH) is the only referral hospital for Lassa fever patients in Sierra Leone and beyond. Following the Ebola outbreak Sierra Leone launched a 5-year health sector recovery plan and although more than five billion dollars were pledged by the international community, donors have failed to deliver. Out of fear of corruption, donors work mainly through implementing partners (UN agencies and NGO’s), with only a small amount of funding (allocated for monitoring purposes) going directly to the government. The government of Sierra Leone is currently covering 16% of total health expenditure, the rest is financed by donors (13%), and private household contributions (71%).[5] This model of financing results in a kind of privatization of health service delivery, meaning the state is not the main service provider but an administrator of health service delivery by a mix of public and private providers.

Sierra Leone has been suffering from health worker shortages and this has worsened since the Ebola disease outbreak in 2014. The MoHS has made significant efforts to rebuild the health sector and strengthen the health workforce. Nevertheless, additional challenges are leading to continued shortages in skilled health care forces. Among them an unequal distribution of human resources skewed to urban areas and partially volunteering health care staff that is not on payroll.[6]

Project History and Strategic Orientation

Under the main objective to reduce under-five and maternal morbidity and mortality in Kenema district, MSF MSF-OCB started operations in 2017/18 by initiating two projects in Gorama Mende & and Wandor chiefdoms and in Nongowa chiefdom, supporting 3 three Peripheral Health Care Units (PHU) (Largo, Nebako, Hangha). In 2018, MSF decided to additionally build its stand-alone secondary health care hospital, MSF Mother and Child Hospital (MSF MCH), a 168-bed paediatric and maternity hospital in Hangha village. By the end of 2019 the primary health care activities in Nongowa were merged with the MCH hospital project. The support in Gorama Mende & and Wandor chiefdoms was reduced and ended in 2022. The support for the PHU in Hangha has continued partially up to the moment.

MSF’s current Kenema project consists of several components centring the MCH. Considering the need to ensure quality of care and the difficulty of recruiting health personnel, the project was designed to start-up activities in phases. The first phaseincluded the establishment of Emergency Room, Intensive Care Unit (ICU), Intensive Therapeutic Feeding Centre (ITFC), Inpatient Paediatric Department (IPD) and ancillary departments (including laboratory, radiology, pharmacy, and others) that were opened in March 2019. In 2020/21, the second phase kicked off with an increase of capacity in the paediatric IPD, ITFC and ICU departments and the opening of a Lassa Fever Isolation Unit. In April 2021, a third phase initiated withthe roll out of an integrated Community Case Management (iCCM in 43 villages) and the support of 6 PHUs, in partially new chiefdoms, to provide community based medical care and with the objective to ensure continuity and quality of care throughout all health care levels. Activities in up to 90 villages include malnutrition screening and treatment, immunization (EPI), Sexual and Reproductive Healthcare (SRH) services covering ante- and postnatal care (ANC, PNC), safe delivery, Safe Abortion Care (SAC), Family Planning (FP), care to victims of Sexual and Gender-Based Violence (SGBV) as well as the support of an existing but weak referral system. This phase further included the construction of the maternity department (CEmONC services), that was opened in May 2022. In 2023, Mobile Clinic activities were initiated as part of the community-based component of the project. The community outreach strategy is currently being revised.

Training and capacity building of the staff is a central objective in this project, mainly provided by the MSF Academy. This MSF governed learning entity organizes and implements training for locally hired MSF staff working in various roles and areas of the project. It started with scholarships to Ghana for 62 Sierra Leone Health staff, followed by the roll out of several curriculums, the main being the basic clinical nursing care (BCNC) training.

In 2023, Kenema project was budgeted with 6.6 Mio EUR, and more than 840 people staff were employed. In the years 2019-2022, MCH has registered monthly between 700-1,000 admission. In 2023, between 90-190 monthly CEmONC admissions were recorded, 635 consultations at mobile clinics and 116 live saving referrals from supported villages. Since 2019, 153 health care workers completed a program of the MSF Academy, and 198 staff are currently enrolled.

The whole project was designed for at least 10 years (2019 – -2029). With the opening of the maternity services in 2022 onwards, the project should not increase its activities but focus on consolidation and stability through continuous capacity building of the staff, strengthening of the management team and maintaining quality of care. The interventions differ from most other projects both in its size, it’s important technical and medical ambitions and in its long-term defined exit strategy of handing over the structure and its activities, either partly or completely, to the MoHS by 2027-2028. This requires a close, continuous, and complementary partnership with MoHS, and particularly with the Kenema Government Hospital (KGH). MSF has a Service Level Agreement (SLA) on national level. In 2023, a Memorandum of Understanding (MoU) was agreed on district level to ensure a clear framework of partnership.

Embedded in the exit plans and based on MSF budget constraints, it has been decided in late 2023 to integrate CEmONC services from MCH hospital into MoH structures (KGH), while other services and components of the project will continue. This integration process is currently being designed and aims at a scale up of support of CEmONC services in MoH structures, including IPC, logistic rehabilitation, medical supply, referral system, human resources, and capacity building. A detailed plan of the integration process is developed in joint discussions with the MoHS, the District Health Management Team (DHMT) of Kenema and the Direction of KGH, to ensure visions are aligned and sustainable.

Purpose AND INTENDED USE

PURPOSE. This mid-term evaluation aims at assessing the success of the Kenema project so far in reaching its set objectives. It should provide a description of the evolution of the project, identify lessons learned and provide recommendations for potential adaptations and/or reorientations of the project’s components.

INTENDED USE. The evaluation findings as well as the evaluation process will be useful primarily for MSF MSF-OCB management in Sierra Leone to inform upcoming operational decisions and the general direction of the project. It will additionally feed into the upcoming design process of the exit phase of the project. The results may also be used more broadly within MSF MSF-OCB as a learning opportunity and to inform projects in other similar complex contexts.

EVALUATION QUESTIONS

1. To what extent is the project relevant in the operational context?
   1. To what extent is the project responding to the needs of the targeted populations?
   2. Has the project been in accordance with the priorities of MSF MSF-OCB?
   3. How does the project align with the priorities of the relevant local authorities?
2. To what extent was the project implemented appropriately to its operational setting?
   1. Was the organizational, strategic set up (fe. human resources, functional/hierarchical matrix, project components) and the deployed resources flexible enough to respond to changes in the project?
   2. To what extent were limitations and barriers of access to health services known and appropriately considered in the design and/or at a later stage of the project?
   3. Which opportunities can increase the project’s appropriateness?
3. To what extent has the project been effective in reaching its objective?
   1. What are the set objectives and expected results in the historical evolution of the project?
   2. What are enablers and barriers (expected or unexpected) that were influencing the achievement of the set objectives?
   3. What opportunities can be identified to make the project more effective?
4. To what extent was the project efficient in reaching the set objectives?
   1. What kind of resources have been invested to achieve the results assessed?
   2. Were resources used timely and efficiently in the context of changes in the project?
   3. How could resources have been used more economically and timelier to achieve results?
5. To what extent does the project influence larger contributions (impact)?
   1. What do target beneficiaries and stakeholders perceive as wider contributions of the project?
   2. Which unintended consequences (positive or negative) can be identified?
6. To what extent is the project coherent with its operational context?
   1. Which kind of external and internal interlinkages have been established?
   2. To what extent was internal coherence maintained?
   3. Which barriers hindered the establishment and/or maintenance of external interlinkages, especially with the MoHS?
   4. How can internal and external coherence be improved?

The six main questions should be in the center while conducting the evaluation. MSF is open to consider adaptations to the secondary questions in case the evaluator(s) discover the need for it in the inception phase.

EXPECTED DELIVERABLES

1. Inception Report

Based on conducting initial document review and preliminary interviews, the inception report should include a detailed evaluation proposal, including methodology and analysis.

1. Draft Evaluation Report

The report should answer the evaluation questions addressing the set objectives and intended use of the evaluation. It should include analysis, findings, and conclusions and, where applicable, lessons learned and recommendations.

1. Working Session(s)

As part of the analysis and report writing process, the evaluator(s) will present (preliminary) findings, collect attendances´ feedback and validation and will facilitate discussion on lessons learned with the commissioner and consultation group members.

1. Final Evaluation Report

The final report should consider comments and feedback received during the working session. An additional short version of the final report can be requested.

1. Dissemination

To be defined in a separate dissemination plan, can include presentations, learning sessions, meaning-making exercises, or other communication materials with partners, or other affected stakeholders (communities, patients, or others).

We expect the evaluator(s) to be flexible in considering additional deliveries that might be necessary to successfully proceed in the evaluation process. Each deliverable is reviewed by the SEU and approved by the Evaluation Commissioner.

TOOLS AND METHODOLOGY PROPOSED

In addition to the initial evaluation proposal submitted as part of the application, a detailed evaluation protocol will be prepared by the reviewers during the initial phase, following access to the documentation and initial discussions with the evaluation consultation group. The initial report will include a detailed explanation of the proposed methods and their rationale based on validated theories. It will be reviewed and validated as part of the creation phase in coordination with the SEU. Valid and robust participatory approaches that can increase the process learning outcomes are welcomed by the SEU.

RECOMMENDED SOURCES FOR SECONDARY DATA

• Routinely collected medical data (raw and aggregated from MSF, MoHS).
• MSF and OCB strategic and project documents (project proposals, logical frameworks, annual reports, project visit and end-of-mission reports, evaluation reports etc).
• National, regional, and global strategies, documentation, and guidelines.
• External literature, research, and documentation.

PRACTICAL IMPLEMENTATION OF THE EVALUATION

Number of evaluators: TBD

Timing of the evaluation: March – August 2024

Dates for the data collection at project level: It will take place after the approval of the inception report, exact date TBD.

The SEU engages a Consultation Group (CG) in this evaluation process with the goal of increasing understanding, buy-in, process learning, and the quality and utility of the evaluation. The CG is headed by a commissioner and contributed to the finalization of this ToR.

PROFILE/REQUIREMENTS FOR EVALUATOR(S)

• Requirements:

• Proven evaluation competencies

• University level degree in public health, health service management, or related areas

• Expertise in project and/or service management in a medical area of the Kenema project (specifically Maternal and Child Health or pediatrics)

• Fluency in English (spoken and written)

• Ability to conceptualize complex matters and report concisely

• High communication and interpersonal skills (ie. engaging stakeholders with diverse interests)

• Assets:

• Experience and/or understanding of MSF

• Expertise in complex project or program evaluation

• Professional experience in the Western Africa region, specifically Sierra Leone

• Expertise in Health System Programming

• Expertise in specific medical competences related to the project’s focus

• Experience with participatory approaches

• Knowledge of local languages (ie. Mende, Krio)

  ________

[1] Kenema – Sierra Leone – Area Database – Global Data Lab

[2] Kenema (District, Sierra Leone) – Population Statistics, Charts, Map and Location (citypopulation.de)

[3] WHO, UNICEF, UNFPA, World Bank Group, and UNDESA/Population Division. Trends in Maternal Mortality 2000 to 2020. Geneva, World Health Organization, 2023

[4] Saez, A.M. 2013. Accessibility Strategy for the Health Care System of the District of

Koinadugu, Sierra Leone. Ayuntamiento Zaragoza;

MSF OCB 2015. Health Seeking Behavior and Perception on Public Health Facilities in Kenema District.

MSF OCBA 2016. Delivery is a secret: Health seeking behaviour and sexual reproductive health in Koinadugu district, Sierra Leone.

[5] MoHS 2021. SIERRA LEONE RMNCAH STRATEGY 2017 – 2021, Sierra LEONE NATIONAL Reproductive, maternal, newborn, child and adolescent health Strategy 2017 – 2021 (who.int)

[6] MSF Kenema Project Document ARO 2024.