EVALUATION OF ARTSENHULP (Medical Supplies) PROJECT

PRESENTATION OF AH PROJECT
Artsenhulp (AH) Ukraine was founded in March 2022, with a mission statement of doctor-to-doctor support with provision of medical supplies, to address one of the needs that arose following the conflict between Russia and Ukraine. At the start, needs were identified via contact groups (doctors to doctors). Initially the foundation gathered medical supplies from hospitals and shipped them to Ukraine. AH Ukraine foundation started procuring medical supplies from global supply centers and delivered them to the destination.

The AH Ukraine foundation was dissolved end of last year and taken over by SV in 2023. The main purpose of SV’s Artsenhulp is to provide direct support to health facilities, medical staff, NGOs, and physicians in Ukraine with medication and medical equipment. As other international actors also support health care facilities in Ukraine, SV’s Artsenhulp has its focus on identifying acute medical gaps in the supply lines and on providing a fast response by supplying medication and medical equipment until other actors can step in.

The Medical coordinator based in Ukraine is focal person for identifying needs and assessing and validation of requests for approval. The logistics team based in the Netherlands and in Ukraine is responsible for processing orders, procurement, transportation, custom clearance, storage in warehouse and delivery to the final destinations. The steering committee for the AH project advises AH project team on implementation of the project, strategic issues and regularly inform the private donors of the Artsenhulp project in close coordination with members of the Stichting Vluchteling management team.

EVALUATION OBJECTIVES

The purpose of the evaluation:
The purpose of the evaluation is to learn from the AH project, assess the impact of the medical supply project, and define recommendations on how to improve the response.
General objective:
Evaluate the effectiveness and relevance of AH project, and review the approaches (strategy, project design, implementation, monitoring) and organizational set-ups that enabled SV to achieve its objectives.

The evaluation will address the following points: design, strategy, efficiency, effectiveness and coordination of the response and will answer the following questions (see the Annex Evaluation questions-AH project for sub questions):

• What is the performance (Timeliness and efficiency) of the supply chain of goods?
• To what extent is the strategy relevant to SV’s supply chain?
• How effective and relevant was the response (delivery of medical supplies)?
• How was the response coordinated with other actors?
• What is the satisfaction of our client partners (Health facilities) about the project?
• What have been the unintended consequences of the response?
• How did SV navigate the dilemma faced on compromise of humanitarian principles (neutrality, impartiality)?

EVALUATION METHODOLOGY
The evaluator, local and/or international consultant, is expected to employ a methodology for unbiased results, designed to contribute to the general learning objective by semi-structured evaluation questions for the project, to aggregate the findings.

Data collection methods could include:
1.1.
Desk-based literature review: meta-analysis and summary of reports addressing humanitarian context, needs, missed opportunities, etc., analysis of project proposals and workplans, baseline and existing Monitoring, Evaluation, Accountability and Learning (MEAL) data, performance indicators and budgets and other broader regional and contextual background material.
1.2.
Secondary data analysis: analysis of data already collected by the project staff/supported health facilities.
1.3.
Semi-structured interviews: key informant interviews with staff, local stakeholders, direct beneficiary groups, and the affected population (if feasible even if the project does not have a direct link with the affected population). Since the response has changed significantly since the start and initial beneficiaries have moved away, including different groups will allow a reconstruction of the first months and the reflections of future developments.
1.4.
If desired, a small survey around central evaluation questions, to solicit a wider range of perspectives and uncover/test cause-effect correlations.
1.5.
(Participatory) Focus group discussions: facilitated small group meetings on specific issues and topics to solicit comments and responses around key issues arising from interviews and secondary data analysis. Preferably using more participatory methods.
1.6.
Direct observation of project interventions and relationships between different stakeholders for 2023 project (Health facilities which received medical supplies).
1.7.
Meetings and consultation with a wide range of other key stakeholders such as partner and SV staff but also external stakeholders who were of priority in the implementation of a coordinated intervention. An outcome harvesting and/or after-action review session with partner staff is encouraged.
Analysis methods are to be proposed by a consultant, we can give suggestions.

STAKEHOLDER INVOLVEMENT
The principal intended users of the evaluation are SV, the (client) partners (refer to health facilities, association responsible for custom clearance), as well as SHO as the project funding source. The partners are expected to share relevant findings with the local stakeholders.
The partners and SV staff will provide the evaluator with all the necessary background documentation and data available from the projects to conduct desk-based assessments in preparation for the fieldwork phase. Partner staff will accommodate the evaluation team’s access to project sites for data collection and setting up meetings with relevant stakeholders. The evaluation team will coordinate with SV MEAL advisor and Ukraine Head of Mission and should discuss any technical and methodological matters that arise.
The evaluation team is expected to hold a post-fieldwork meeting with SV and partner staff and other stakeholders to present and discuss preliminary findings. Draft reports are expected to be shared with partners and/or stakeholders, and SV staff to incorporate feedback and ensure factual correctness before final versions are signed off prior to completion of the final evaluation report. The final version of the evaluation report will be provided to both SV and shared with the partners.

EXPECTED DELIVERABLES
The evaluation team is expected to deliver the following key deliverables:

• Inception Report (of a maximum of 5 pages) produced by the lead consultant within two weeks of contract being signed detailing the evaluation team’s intended methodologies (for data collection and analysis), activity workplan, team responsibilities and overall analysis of the desk-based review and secondary data from partners.
• Project Evaluation Report with detailed findings and recommendations specific to this project:
• Summary of Evaluation Report for wider distribution (including in-country sharing with local stakeholders.)
• The Project evaluation report must be written in English and not exceed 25 pages (excluding annexes). The report must be written in Word and submitted in one stamped paper and one electronic copy. The draft report will need to be presented to SV for final input and fact checking before the final version is submitted.
• The written report should be based on (but not limited to) the following framework:
  1. An executive summary (2 pages maximum)
  2. A methodology section outlining the inception report content and planning process with Ukraine team, make-up of the team (e.g., sectoral expertise, local knowledge, gender balance) and its appropriateness and independency for the evaluation and any potential biases.
  3. A constraints, limitation (including reflexivity) and challenges of the evaluation section
  4. A context analysis of the overall humanitarian response in Ukraine and how the projects are situated within it.
  5. A findings section in line with the set evaluation questions discussed in detail
  6. A recommendations and lessons learned section following the analysis of the findings and the discussion with key stakeholders of the preliminary findings. It will be structured with specific subsections for each evaluation question, ensuring that findings are tailored to each context/question providing general conclusion for each and any observed differences are explained.
  7. List of Annexes, including to the minimum: the present ToR, the inception report, sample of the data collection tools used, a list of sources of information used/documents reviewed, any relevant graphic material that could not be placed in the report.

QUALIFICATIONS
The selection will follow a competitive bidding process, which will be based on a request for questions (inclusive both of a technical and financial offer) published online on relevant website(s). Following a pre-screening of the received offers, those meeting the minimum requirements will be ranked on both the technical and financial offers, also following a panel interview of the shortlisted consultants/firms. SV will make every effort to ensure there is no conflict of interest based on the consultants selected to guarantee independence and objectivity.

Specific skills and experience sought:

• Significant demonstrable experience in the evaluation of humanitarian projects, including field-based data collection, analysis and reporting and experience of conducting desk-based evaluations emergency response projects within a conflict environment.
• Extensive knowledge of humanitarian contexts, preferably with knowledge specifically in Ukraine and/or lived experience, and health programming.
• Relevant academic qualifications and training in the related fields.
• Significant experience with qualitative research methodologies, particularly involving semi-structured interviews, focus groups and qualitative analysis.
• Excellent communications and presentation skills.
• Ability to write clear and sensible reports (examples of previous work shall be submitted).
• Fluency in English and the national language is essential.
• Understanding of national humanitarian mechanisms and context is preferable.
• Ability to manage the available time and resources and to work on tight deadlines.
• Independence from SV, project stakeholders or other parties involved.

No data can be used by the consultant concerning this study without the written permission of the partners. The consultant acting as service provider will make sure to present itself as such for all discussions held within the framework of the consultancy.
The consultant immediately agrees to respect all specific security instructions of the partners and most recent Government directives as well as to operate in line with the partners’ Prevention of sexual exploitation and abuse (PSEA) and child protection policies.

TIMEFRAME
The evaluation is expected to take around 21 days and will be mainly based in Ukraine.
We intend to advertise the vacancy for a period of three weeks, during which we will conduct ongoing screening. Should suitable candidates be identified prior to the scheduled closing date, we reserve the right to conclude the recruitment process ahead of the stipulated timeframe.