Call for Proposal: Evaluation of MSF SIDA Project – Readvertisement (English)

Médecins Sans Frontières (MSF)/Doctors Without Borders is an international medical humanitarian organization determined to bring quality medical care to people in crises around the world, when and where they need regardless of religion, ethnical background, or political view. Our fundamental principles are neutrality, impartiality, independence, medical ethics, bearing witness and accountability.**

The Stockholm Evaluation Unit (SEU), based in Sweden, is one of three MSF units tasked to manage and guide evaluations of MSF’s operational projects. For more information see: evaluation.msf.org.

Promoting a culture of evaluation is a strategic priority in order to be accountable, seek for continuous improvements and achieve organizational learning. MSF does not evaluate only because of external requirements, for example donors related ones. These Terms of Reference should be seen as a starting point for the evaluation process. The evaluator(s) are welcome to challenge them and suggest for example different or additional perspectives, as they see fit during the inception phase. The evaluation process should rely on solid methodology to achieve credible results and should also ensure to put values and use in the forefront. The evaluation must involve and include different actors and counterparts in an adequate manner during the whole process.

The Stockholm Evaluation Unit (SEU) is calling for proposals for an evaluation of an MSF HIV/AIDS project in Kinshasa, DRC.

Please note that this evaluation is looking for a partly French-speaking evaluation team. Please read the ToRs thoroughly before sending in your application.

Evaluation period: May – October 2022

Terms of Reference: more detailed information can be found at http://evaluation.msf.org/vacancy/call-for-proposal-evaluation-of-msf-sida-project-kinshasa-drc-readvertisement-english.

The project and the Evaluand

Médecins Sans Frontières Operational Center of Brussels (OCB) has been operating in the Democratic Republic of Congo (DRC) since 1977 and is since 2002 running an HIV/AIDS project in Kinshasa (Sida project), at different levels of the health pyramid.

The Sida project aims to achieve quality of care, innovation, and advocacy for patients. It is “catalyst for change” in the sense that it aims to influence national politics[1]. MSF activities provide care at all levels of the cascade and raising awareness among healthcare providers and patients (in waiting rooms, in PoDis – distribution points for ARVs – and via patient associations) is also crucial.

The project provides care for advanced HIV at the Kabinda Hospital Center (CHK, 100% MSF structure), with over 1800 patients in 2021. The CHK was initially an Outpatient Treatment Centre, which later included hospitalizations and then also became a referral hospital for patients with advanced HIV, as well as a training and excellence centre for MSF and its partners.

The project also supports, through a decentralization component, activities for non-advanced HIV/AIDS in different structures. The decentralization component includes among others support through mentoring as well as simplified technical platform. The decentralization of HIV/AIDS care to primary and secondary levels was initiated by the project in 2005 and has undergone many changes, in terms of strategy, number, type and location of the structures supported, as well as the type of support offered by MSF. The objective of the decentralization component is to improve access and quality of care within the city’s health facilities, with screening, treatment, monitoring of adherence, suspicions of treatment failures, as well as viral load.

It is this decentralization component that is the subject of this evaluation, for the period 2017-2022.

In February 2022, the project supports via the decentralization component:

• two partner hospitals managed by the Ministry of Health (MSPP): Luyindu Hospital Centre (end of MSF support scheduled for the end of April 2022) and Saint Joseph Hospital Centre.

• six health centres (St Clement, St Ambrose, St Anne, St Joseph (Out-patient Department), Mokali and Tshimungu).

• activities at community level.

Throughout the years, the project has supported several structures through the decentralization component, and has disengaged from some of them[2], handing over to other actors. The evaluation will include some of these structures supported in the past.

An evaluation of the decentralization component has already been conducted and finalized in 2020. It focused on the evolution of the strategy from 2005 to 2017, as well as the results achieved. It also suggested avenues for reflection on the strategy implemented from 2017 onwards. The round table organized by the project in 2017 and the mid-term review of September 2020 confirmed the decentralization strategy and defined the main lines until 2022, with a particular focus on mentoring in the supported structures, as well as implementation of a so-called integrated zonal approach (MSF supports healthcare provision at all levels of the health pyramid within a health zone).

Purpose and Expected Use

The purpose of this evaluation is to evaluate the decentralization component of the project from 2017 to 2022.

A round table will be organized in early 2023 to define the future of the project. This evaluation will inform the preparation of this round table and the decisions to be taken for the future of the project, with regard to decentralization. What are the lessons to take into account when continuing with decentralization activities and defining the strategy from 2023 onwards?

The evaluation will be used primarily by the Sida project, but it can also inform other or future HIV/AIDS projects, in DRC or elsewhere, as well as be useful for partners in DRC and other actors in the field of HIV/AIDS.

The perspectives and values of different stakeholders (e.g., MSF, MSPP, partners, patients, and communities) will be given particular attention in this evaluation.

Evaluation Questions

QE 0: Description of the decentralisation component, from 2017 to early 2022. This decentralisation component is the subject of all subsequent evaluation questions.

QE 1: Were the objectives relevant given the needs observed and expressed, the context, and MSF’s priorities?

QE 2: Was the strategy, design and implementation coherent given the context and existing resources? How could the approach have been more coherent?

QE 3: To what extent have past experiences (and in particular the previous evaluation) been taken into account in the definition and implementation of the decentralisation strategy 2017-2022?

QE 4: Were the different actors and counterparts sufficiently taken into consideration?

EQ 5: Has the decentralization component achieved its expected results?

§ To what extent are the outputs in line with the quality standards and expected results?

§ What were the reasons (enabling or hampering factors, expected or unexpected challenges) behind achievement – or not – of expected results?

§ How could decentralisation have been more effective?

QE 6: Which resources were needed, were they available, could they have been mobilised in a more efficient or sustainable way?

QE 7: What is the impact of decentralisation component?

§ To what extent has the decentralisation component achieved its general and specific objectives?

§ To what extent has decentralisation led to or is likely to achieve sustainable results? This question will cover among others the prospect of handover of activities and takeover by other actors.

§ What are the effects of decentralisation as perceived by patients and counterparts?

§ What are the expected or unexpected negative and positive consequences of decentralization?

Expected Results

1. Start-up report in French (May 2022)

According to SEU standards, after a preliminary review of documents and initial interviews with key stakeholders, not least consultation group members for the evaluation. It will include a detailed evaluation proposal, including methodology and timeline. 2. Restitution/debriefing with the project and stakeholders at the end of field data collection (summer 2022).

3. First version of the final evaluation report (Sept 2022)
According to SEU standards, it will answer evaluation questions and include conclusions, lessons learned and possible recommendations (where appropriate, and ideally generated collaboratively, see point 4. Working session). The report will also include to what extent the previous evaluation has been taken into consideration.

4. Working session With the participation of the commissionaire and consultation group for the evaluation. This session is an integral part of the final report writing process: the evaluator(s) will present the findings, in order to gather initial feedback from participants, as well as start a discussion on it, lessons learned and recommendations (where appropriate, and ideally generated collaboratively).

5. Final report in French (SEU will organize the translation into English)

After sending the feedback received during the working session, as well as the written comments of the Evaluation Manager (SEU) and the consultation group, finalization of the report.

6. Webinar presenting the evaluation, followed by questions and discussion (audience MSF OCB all staff, and beyond if relevant). Other dissemination activities may be suggested during the evaluation, including by the evaluation team. See suggestions in table below.

Tools and Methodology

In addition to the proposal submitted during the selection process (see evaluator profile section below), a detailed evaluation protocol should be prepared by the evaluators in the initial phase of the evaluation and presented in the start-up report. It will include an explanation of the proposed methodology as well as a justification based on validated theory(s). This protocol will be reviewed and approved at the end of the initial phase, under the coordination of the SEU.

DOCUMENTATION and recommended sources

§ Project documents (eg 2017 and 2020 round table conclusions, narratives, logframe, monthly and annual reports, medical database, organizational charts, presentations, visit reports, decision-making documents)

§ Direct contact with person(s) responsible for the collection and management of medical and other data at project and/or mission level.

§ MSF sources (strategic orientations, operational priorities, protocols and policies, universal medical indicators from MSF-OCB Medical department, operational research, 2019 evaluation report etc.)

§ National reports, guidelines, national policies

§ External literature, similar experiences, actors, standards etc.

Implementation of the Evaluation

Number of evaluators: to be proposed**

Evaluation Timeline: May-October 2022

Field visit for data collection: June/July 2022, to be confirmed.

PROFILE of the Evaluator(s)

§ Requirements:

o Demonstrated expertise in evaluation.

o Experience in managing HIV/AIDS programs in middle and low-income settings.

o Experience with HIV/AIDS decentralization programmes.

o Excellent command of French and English. The evaluation report will be produced in French, most data will be available in French.

§ Merits:

o HIV/AIDS clinical experience and training.

o Knowledge/experience of HIV/AIDS projects in middle and low-income urban contexts.

o Public health training.

o Experience and/or training in health systems, particularly in capacity building.

o Knowledge of the context, experience in DRC.