Rapid Mid-Term Evaluation of the Nadoum Programme

To view the full Terms of Reference, please use this link: Consultancy: Rapid Mid-Term Evaluation of the Nadoum Programme

——

Background

Programme: Nadoum; Together for change; Sustaining HIV services in MENA

Donor: The Global Fund to Fight AIDS, Tuberculosis, and Malaria

Applicant: MENA H Coalition

Principal recipient: Frontline AIDS

Period: 2022-2024

Grant amount: $7.5 million

Focus countries: Egypt, Jordan, Lebanon, Morocco, and Tunisia

Starting in 2019, the MENA H Coalition began the implementation of a three-year Global Fund grant entitled Sustainability of HIV and Key Population Services in the MENA Region. The overall goal is to improve the provision of HIV services to people living with HIV (PLHIV) and key populations in the MENA region through, amongst other strategies, improving the regional and national level ownership and sustainability of these programmes. In 2020, the Global Fund invited the MENA H Coalition to submit a Validation of Continuation funding request which would extend the period for the achievement of this goal and strategic objectives through to the end of 2024.The continuation of the grant over the 2022-204 period is guided by the following six strategic objectives (SO):

SO1: Increase resource mobilization, technical and management capacity for KP organizations;

SO2: Develop, document, support and promote use of innovative and sustainable community led service delivery models;

SO3: Advocate for increased domestic financing for quality HIV prevention, treatment and care services for KPs though public health systems, community systems or through social contracting;

SO4: Reduce structural barriers, including stigma and discrimination against KPs and improve their access to health services and retention in care;

SO5: Document human-rights violations against key populations and provide ways of responding to them; and

SO6: Support processes and policy reforms towards the scale-up of comprehensive prevention programs for people who use drugs.

The grant proposes to make this contribution through achieving six outcome-level results by 2024:

• Increased technical and operational capacity for KP CSOs and networks at national and regional levels to advocate for and to secure their health and rights;
• Innovative approaches to service delivery that incorporate partnerships, integration, technology and community-led approaches;
• Diversified funding base and funding modalities for KP services (including through increased domestic investment and social contracting) to ensure their sustainability;
• Increased social protection for KPs across MENA to reduce or remove structural barriers to access, uptake and retention in HIV and other services;
• Improved promotion and protection of human rights for KPs; and
• Stronger regional and national level programme to promote HIV prevention and treatment for people who use drugs (whose HIV-related vulnerability remains a main driver the HIV epidemic across MENA).

Evaluation approach

Formative evaluation with the aim of providing feedback to the Nadoum partnership to make necessary adjustments for program improvement.

Time period being evaluated: January 2022 – July 2023

Mixed approach:

• Participatory stakeholder-focused: gaining an understanding of program’s performance by eliciting the perspectives of key program stakeholders and beneficiaries, drawing on their direct experiences of the program or their interpretation of the program based on available data (degree to which the program contributed to or achieved change and types of changes experienced through their direct involvement with the program)
• Social Justice: producing and using evaluation findings to reduce identified inequalities. Findings to be employed in advocacy, social policy development, and to inform strategies for social change. i.e. What (mechanisms) work (outcome) for whom and under which circumstances (context). Focus on the identification of those groups the program benefitted and those it did not.
• Systems/Developmental: the grant operates in complicated and complex environments. Relationships and dynamics are non-linear. There is heavy reliance on multiple components implemented by multiple partners with no linear causality able to be established between the program’s methods and its effects. Predictability of outcomes is not easily established. Developmental evaluation approach is recommended responding to the realities posed by evaluating complicated and complex programs such as Nadoum. This approach supports the conceptualization and adaptation and development of the program. The objective of using this approach is to guide program development.

Purpose of the evaluation

The main objective of this evaluation is to assess the continued relevance of the programme, the progress made towards achieving planned objectives, while providing pragmatic recommendations that can be integrated as part of the grants upcoming replanning and reprogramming process.

Specific objectives and evaluation questions

The rapid midterm evaluation will focus on validity and efficiency of the grant to determine if Nadoum addresses the needs identified and how well the project is being implemented to meet those needs. Frontline AIDS proposes addressing the objective of the MTE through the following evaluation questions:

Validity of design

EQ1: Does the project design address the gaps identified in the initial project proposal? What needs to be modified (if any)?

EQ2: Were the underlying assumptions about how change occurs clearly identified? If not, what is missing?

Efficiency

EQ3: What are key examples of significant changes achieved by implementers? Are the achievements likely to be sustained? Why and Why not? [1]

EQ 3.1 What are the approaches that have been efficient/inefficient in progressing towards change? What alternative approaches can be used?

EQ4: Is the project making sufficient[2] progress towards planned objectives (as per ToC)?

EQ 5: What are the major external factors (positive and negative) that influenced the grant? How have Nadoum partners been responsive to these changes?

The evaluation questions will be further refined with the consultant/consultancy team.

How the evaluation will be used

The rapid midterm evaluation will primarily be used to strategically direct the grant’s upcoming replanning and reprogramming process. It is important for the programme to take stock of its achievement to date, to understand fully the new constraints it faces both operational and strategic, and be able to plan accordingly as it goes into its final year of implementation. Its main audience will be Frontline AIDS (PR) and Nadoum implementing partners (SRs and SSRs).

Scope of work

Indicators and evaluation questions should be evaluated according to the following criteria:

• Evidence base: availability of evidence[3] to substantiate claims by evaluation respondents.
• Contribution: of the programme to the observed outcomes
• Significance: of the observed outcomes to the overall goal of the programme

Methodology

The methodology will be finalised in consultation with Frontline AIDS and is likely to include (but is not limited to):

• Desk review of key documents such as: partner workplans, quarterly reports, grant annual report.
• Qualitative methods: including interviews with key stakeholders (to be identified with Frontline AIDS) and a theory-based approach in order to address the causal linked between interventions and its specific effects (change), and whether these are logical and comprehensive.
• Outcome Harvest (addressing EQ3): As an organisation we are using Outcome Harvesting to help better describe the outcomes that we have contributed to. Where possible, the evaluation should include some outcome harvesting as part of the methodology. This type of data collection can both complement or illustrate quantitative indicators. Outcomes harvested might be in addition to the data that is collected in the results framework, or it might provide a more in-depth and contextualised way of describing the changes reported through quantitative indicators. However, it is designed as part of the evaluation, the final report should include a set of outcomes written up using the template in annex 2.

Outputs

• Verbal presentation to stakeholders to discuss and validate initial findings.
• Draft and Final evaluation reports (no more than 20 pages excluding annexes). Annexes should include the terms of reference, a list of people and organisations interviewed; a list of documentation and materials reviewed; a timeline of the evaluation process and data collection instruments used.
• Executive summary of the final evaluation (of no more than 5 pages): this may be published on the Frontline AIDS website so should be readable as a standalone document.
• PowerPoint visual presentation of final evaluation findings (no more than 10 slides)

Management and governance of the evaluation

This evaluation will be directly managed by the lead consultant who will assume overall responsibility for the deliverables. Senior Advisor: M&E and Program will act as the first point of contact for the team of consultants and will be responsible for overseeing the implementation of the evaluation.

Frontline AIDS will support all stages of the evaluation process including: providing relevant documentation, assisting in the organisation of data collection (providing contact details, ensuring availability of interviewees and relevant data), providing feedback on drafts of all agreed outputs, including the methodology.

The evaluation will be guided by a steering committee. They will:

• Review and refine objectives, tools and methodologies in consultation with the consultant team.
• Provide feedback on draft inputs presented by the team of consultants.
• Sign off final deliverables.
• Ensure a management response to the evaluation is written and recommended actions are assigned to named individuals to implement.

It is envisaged that this work will take approximately 20 days.

Profile of consultancy team

The successful bidder will have one or more members who meet the following criteria:

Essential:

• Substantial experience in conducting evaluations of HIV and/or human rights programmes.
• Experience in undertaking assessments using qualitative methodologies.
• Sensitivity and understanding of needs of marginalized and vulnerable communities.
• Experience working in an international development context.
• Understanding of participatory evaluation approaches
• Ability to systematically analyse and present complex data and information.
• Excellent communication and facilitation skills
• Excellent written and spoken English and Arabic
• French comprehension skills
• Ability and commitment to deliver the expected results within the agreed period of time.

Desirable:

• Based in, or from, the countries where the programme is operating.
• Has knowledge of the HIV sector in the MENA

[1] One specific workstream will be identified in consultation with implementing partners. This workstream will be selected primarily based on budget allocation.

[2] This will be further defined with the consultancy team

[3] Drawing on the BOND principles for assessing the quality of evidence (October 2013), good quality evidence for Frontline AIDS means:

1. Voice and Inclusion: the perspectives of people living in poverty, including the most marginalised, are included in the evidence, and a clear picture is provided of who is affected and how.
2. Appropriateness: the evidence is generated through methods that are justifiable given the nature of the purpose of the enquiry.
3. Triangulation: the evidence has been generated using a mix of methods, data sources, and perspectives
4. Contribution: the evidence explores how change happens, the contribution of the intervention and factors outside the intervention in explaining change.
5. Transparency: the evidence discloses the details of the data sources and methods used, the results achieved, and any limitations in the data or conclusions.

How to apply

We are looking for a consultant team comprising of those who are independent of Frontline AIDS and Global Fund, i.e. not an employee of either the Global Fund or Frontline AIDS, but it could be someone with previous experience of either organisation.

If you are interested, please send a technical and financial proposal outlining your approach, qualifications, previous similar work examples and indicative budget to Hoda Mansour at [email protected] by July 25th, 2023.