Project evaluation consultancy

Catholic Commission for Justice and Peace (CCJP) and the Scottish Catholic International Aid Fund (SCIAF) wish to engage an independent consultant to evaluate a joint project implemented in Zomba and Machinga districts in Malawi with support from the European Union (EU).

II. Brief Overview of the Project:

The project is a EU funded 2-year project that aims to protect and promote the political, social, economic and cultural rights for Persons with Albinism in Malawi. The project concentrates on the violations, abuse and denial of human rights of persons with Albinism. The main components of the project have been: i) challenging stigma and discrimination against PWA; ii) increasing access of persons with albinism to appropriate healthcare and legal support; iii) supporting government in its efforts to protect and promote the rights of persons with albinism; building the confidence and capacity of persons with albinism to claim their rights and access services. This consultancy is to evaluate the two years (November 2019 to January 2022) achievements of this project that has been implemented in 6 villages of Zomba and Machinga Districts.

The implementation of the programme took a participatory approach in the spirit of the United Nations Convention on the Rights of Persons with Disabilities (CRPD), which Malawi ratified in 2009. Strong relations have been promoted between: Community Rights Groups (24 volunteers mostly with Albinism); village/area development committees; local/district/national authorities; District Social Welfare Offices; CSOs/ and non-state institutions such as Association of Persons with Albinism (APAM); Federation of Disability Organisations in Malawi (FEDOMA); Malawi Council for the Handicapped (MACOHA); Malawi Human Rights Commission (MHRC); District Disability Forums at district/national level; relevant Ministries, Departments and Agencies (MDAs) and relevant Parliamentary committees.

In addition, the project had a special focus on monitoring accessibility, availability, affordability and quality of sunscreen lotion in public health facilities like health centres and district hospitals. Additionally, the intervention also focused on providing psychosocial support and legal advice and support to PWAs and their families at the community level.

III. Impact and Outcome Statements:

Impact: Persons with albinism in two districts of Malawi are better able to access their rights to healthcare & justice, resulting in more secure & dignified lives

Outcome 1: Reduced sociocultural stigma and discrimination and increased social integration of persons with albinism within their families and communities.

Outcome 2: Increased access for persons with albinism and their families to appropriate healthcare and legal support in cases of discrimination and rights violations.

Outcome 3: The legal and policy framework and Government services more effectively challenge discrimination and meet the needs of persons with albinism.

Overall project budget: 295,965 Euros

IV. SCIAF Evaluation Approach and Methods:

These Evaluation Terms of Reference (TOR) have been developed by CCJP and SCIAF, in line with the SCIAF evaluation guidelines, EU Guidance and issues specific to this grant.[1]

SCIAF’s approach to evaluations aims to combine the two purposes of accountability and learning. These terms of reference do not intend to define the methodology in detail. However, SCIAF expects the methodology to incorporate certain principles and processes, set out below.

The Evaluation should:

1. Ensure that all key stakeholder groups affected by the project (both directly and indirectly) are consulted and their views considered in the Evaluation. The consultant should consider in the design if it is possible to involve partner staff or beneficiaries in the process beyond simply being consulted.
2. Consult stakeholders work in both project locations
3. Ensure it is in line with the BOND Evidence Principles[2] of: voice and inclusion; appropriateness; triangulation; contribution and transparency.
4. Include where possible a feedback session with main project stakeholders (24 GRG; PWA& families; community representatives and representatives from main targeted institutions at local level (health facilities; district council; district police judiciary; teachers) District Welfare district office; main CSO/ institutions targeted visited to discuss and clarify key findings and observations.
5. Include a debrief of key findings with the Evaluation Committee (CCJP and SCIAF) and relevant project staff.
6. Inclusion of SCIAF standard indicator in the evaluation and Summary and Scoring as a separate report. (Details to be shared with successful candidate.)

V. Evaluation Objectives:

As mentioned, the project aimed at: I) challenging discrimination against PWA ii) increasing access of persons with albinism to appropriate healthcare and legal support iii) supporting government in its efforts to protect and promote the rights of PWA and building the confidence and capacity of PWA to claim their rights and access services. For this reason, we would like the consultant to:

1. Evaluate the extent to which the project achieved its intended outcomes as set out in the project logframe. This should enable assessment of: change of attitudes and practice within targeted communities/community authorities and targeted institutions towards inclusion; effective and action-based coordination amongst stakeholders; effective support from health facilities to deliver and monitor sunscreen lotion supply; effective psychosocial and legal support delivered; appropriate networking mechanisms with district and national authorities in protecting and promoting rights of PWAs; active participation of PWAs in community development and primary justice; improved responsiveness, accountability and transparency of duty of safeguarding the rights of PWAs at all levels..
2. Evaluate the extent to which women and girls living with Albinism were able to access and benefit from the project’s different activities and identify the obstacles that prevented them from participating and benefitting whether they were internal or external to the project. This should include both project participants and women with albinism that could not participate if they can be identified.
3. Consider the relevance, efficiency, effectiveness, impact, and sustainability of the project, in line with OECD/DAC definitions and guidance.
4. Evaluate the project logic / theory of change. Assess the strengths and weaknesses of the project and strategies chosen to achieve project objectives; and provide information on what worked well, what did not, and why.
5. Give a view on the accuracy of project reports against indicators, based on evaluation findings as appropriate’
6. Assess whether activities chosen were the best option to implement given the objectives of the project and the scarce resources.

We appreciate that the scope of the evaluation is wide and that it may not be able to consider all aspects in equal or great depth. The consultant should indicate how they plan to manage this challenge in their application.

VI. Required Outputs

The consultant will be expected to provide to CCJP and SCIAF with:

1. A detailed Inception Report setting out the detailed methodology including sample sizes and sampling methods, and timeframe for the evaluation to meet all aspects of this TOR, in line with the proposal received as part of the DPA procurement process. The Inception Report will need to be approved by CCJP / SCIAF before the fieldwork begins. Sufficient time should be included in the schedule to allow for review and possible change to the proposed methodology if required;
2. A full draft report, no later than the 8th of August 2022 for debriefing meeting. The draft report should include a specific recommendations section.
3. A final report will be due by the 12th of August 2022. The consultant will be responsible for writing the final report and submitting it to CCJP National Coordinator, Evaluation Committee, and SCIAF Programme Coordinator.

VII. Roles of Participants

1. Consultant/Team Leader: The evaluation will be led by an external consultant known as the Team Leader. The specific responsibilities of the Team Leader are outlined below:
   1. Review all relevant documents and draft and confirm with the Evaluation Committee the objectives of the evaluation, process and methods to be used, expected outcomes, and the format for the presentation of the results.
   2. Provide leadership to the Evaluation Team in developing appropriate interviewing techniques, ensuring data collectors are adequately trained and supervised, preparing for village feedback sessions, and presenting preliminary findings and lessons learned to key stakeholders mentioned in section IV of the present document.
   3. Coordinate the activities of the Evaluation Team and keep the Evaluation Committee and SCIAF informed of progress on a weekly basis through online meetings.
   4. Use a participatory approach that encourages discussion of the findings, and ownership of the conclusions and recommendations by all primary stakeholders.
   5. Upon completion of the final draft, discuss the results with the Evaluation Committee.
   6. Complete the final evaluation report, following the outline agreed upon with the Evaluation Committee.
   7. Include in recommendations inputs to future programming design: beneficiaries identify what is important to them for the development of their communities and families.
2. Evaluation Team: The Evaluation Team will consist of the lead consultant and any other members of their consultant team (to be defined by the consultant).
3. ***Evaluation Committee:***The Evaluation Committee will consist of the CCJP National Coordinator, CCJP Project coordinator, and CCJP assistants (2), SCIAF Programme Officer, Programme Manager and MEAL Manager. The Evaluation Committee members will be available to the Team Leader for consultation on the general direction of the evaluation and to address any issues/concerns arising during the exercise.

VIII. Evaluation methodology: To be provided by the consultant in his or her Proposal/Inception Report.

**IX. Proposed Schedule:**The following is intended as a guide to the times estimated for each component of the work. The Consultant will be required to prepare and submit an Inception Report including the detailed methodology and work plan at the beginning of the evaluation. The evaluation is scheduled to begin from 27th of June 2021 and Finish on the 10th August 2022.

Timeframe

27th June: Meet with the Evaluation Team and Evaluation Committee, study relevant programme documents, and work on the design evaluation process and methods, select sample villages, develop questionnaires with team, and finalise and submit work plan. CCJP to inform relevant stakeholders about consultant visit and support in the organization of meetings

30th June: Present to Evaluation Committee Evaluation process, methods and design

4th July: Finalized Evaluation inception report and approved by Evaluation committee

24 July: Submit the first draft evaluation report to DPA Evaluation Committee, and SCIAF

26 July : Debriefing meeting to present key findings and recommendations to Evaluation Committee, CCJP teams and SCIAF

1 August: Submit 2nd draft evaluation report

4 August: Evaluation Committee, SCIAF and CCJP send back the written final comments on the evaluation report to the Consultant for finalisation of the report

10 August: Finalise and submit a final report to CCJP National Coordinator, Evaluation Committee and SCIAF

X. Outline of the evaluation report:

Acknowledgement

Executive Summary

I. Introduction

II. Objectives

III. Methodology and limitation

IV. Detailed Findings and analysis in both provinces

V. Conclusions, lessons learned and recommendations

Annexes

The consultant will be responsible for writing the final report and submitting it to CCJP National Coordinator, Evaluation Committee, and SCIAF Programme Officer and Manager, within agreed dates mentioned above. The Evaluation report, which will be written in English language, should be clear and concise, easy to understand, concise and useful for the organisations and beneficiaries. The main body of the report should usually not exceed 30 pages (A4 pages) and will include an executive summary and recommendations. Technical details should be confined to appendices that should also include a list of informants and the evaluation’s team’s work schedule. Background information should only be included when it is directly relevant to the report’s analysis and conclusions. Recommendations should be recorded separately and should also include details as to how they might be implemented. Each recommendation should be accompanied by an indication of the lead actor responsible for taking it forward.

As a separate document, the SCIAF Evaluation Summary and Scoring Sheet should be completed and submitted with the draft report. One soft copy of the final report will be submitted to CCJP and SCIAF by the evaluator.

1. Consultant Evaluation Criteria

A consultant with relevant and demonstrable experience in the following areas will be selected:

1. Qualification in International Development/Social Science; Disability Law and Policy or similar fields.
2. Extensive experience in programme design and evaluation (min 5 years)
3. Practical experience in participatory approaches and M&E
4. Practical knowledge, skills and experience in project cycle management (PCM), logical framework approach (LFA), outcome-impact-orientation (OIO), management of education projects/ programmes
5. Experience with Human rights/advocacy projects directed to protect and include persons with Albinism or persons with Disabilities in similar contexts in Africa.
6. Familiarity, knowledge and sensitivity about the current socio-political environment of Malawi
7. Familiarity with the prevailing human rights situation of persons with disabilities in Malawi including the legal, policy and institutional framework governing disability programming.
8. Fluency in English (spoken and written)

Along with the need to meet the above experience criteria, the consultant will be selected according to the following criteria:

1. Consultant expertise/CV (40%)
2. Technical Proposal (40%)
3. Financial Proposal (20%)
4. Budget:

Total Budget for the final evaluation is €7,117

1. Requirements

The consultant will act in accordance with:

• EU contractual agreement
• Safeguarding policy (SCIAF and CCJP policies)
• Security Policy (SCIAF)
• Code of conduct (SCIAF)