Evaluation consultancy – livelihoods project

The British Red Cross are seeking a consultant (individual/organisation) to conduct a Final Evaluation of a livelihoods project implemented 2021-2022 in Lebanon in cooperation with the Lebanese Red Cross (LRC) and the International Committee of the Red Cross (ICRC).

The findings from the evaluation are expected to analyse the success of the project to date, and inform operational decision-making for similar projects in the future.

Full information on the evaluation and project can be found below – see section ‘Evaluation Details’.

Timeframe: October-December 2022.

Location: The evaluation will involve direct activities in Lebanon, including a workshop in Beirut and direct data collection in project implementation areas (Arsal, Lebanon). Other elements of the evaluation can be completed remotely.

Consultant Profile:

• Demonstrated experience of conducting similar evaluations or related work.
• Proven track record of conducting quantitative and qualitative research and data analysis.
• Experience in participatory approaches.
• Experience in humanitarian programming and project cycle management – particularly related to livelihoods programming.
• Understanding of Accountability to Affected People.
• Knowledge and experience working with the Red Cross Red Crescent Movement preferred.
• Excellent written and spoken English and Arabic skills.

EVALUATION DETAILS

About the livelihoods project:

The project to be evaluated intended to implement a set of livelihood activities based on labour market assessment results. The project’s overall objective was ‘Vulnerable Lebanese residents, Syrian refugees and people of other nationalities are able to cover their essential needs and unavoidable expenditures as defined by their biology, environment and cultural standards in a sustainable manner’. This objective was achieved through three outcomes –

• Outcome 1: Syrian refugees, resident Lebanese and other vulnerable groups, affected by conflict and not assisted by other organisations, in Arsal and LRC priority areas have a sustainable income. This outcome aims to capitalise on the previous BRC-LRC-ICRC programme in Arsal and further improve the population’s capacity to generate income to cover their essential needs. This will be achieved through working with a range of actors to address market gaps, bottlenecks and challenges; supporting existing micro, small and medium enterprises to grow, especially those involved in local production; and supporting increased absorption of labour.
• Outcome 2: Staff and volunteers from the Lebanese Red Cross have the capacity to implement a pilot livelihoods intervention. Within the framework of LRC’s livelihoods strategy, BRC, ICRC and LRC will work together to share learning on livelihoods programmes (particularly from interventions under outcome 1) to build LRC’s capacity to undertake a livelihoods pilot project in Arsal.
• Outcome 3: LRC, ICRC and BRC have a strong partnership model, capitalising on complementary capacities, encouraging effective knowledge and skill sharing and communication and coordination, with the value add on humanitarian outcomes clearly demonstrated for replication in other Movement contexts. Activities under this outcome are focused at the partnership level, to document lessons and communicate this partnership approach to others.

Evaluation Scope:

This final evaluation has the following aims:

1. Assess the effectiveness, efficiency, impact, relevance, and appropriateness of livelihood activities delivered in Arsal in the period under review.
2. Evaluate the success of capacity-building work done in support of LRC Livelihood Focal Point and LRC Volunteers.
3. Review the extent to which the project achieved its target outputs and outcomes, and identify the extent to which the following elements were effective and appropriate for achieving the intended outcomes: a) project design and methodology; b) project management and partnership model
4. Assess the extent to which the project’s achievements are sustainable, taking into consideration future ambitions of the LRC-BRC-ICRC partnership.
5. Collate lessons learned across the project, and make recommendations for improvement in future similar programmes.

Evaluation Outputs:

1. Initial draft Inception Report with evaluation methodology, plan and tools.
2. Finalised Inception Report that has incorporated edits and an approved and finalised methodology and tools.
3. Draft Evaluation Report of findings and recommendations for dissemination. The Evaluation Report should: (a) be jargon free, clear and simply written; (b) not exceed 30 pages; (c) include an Executive Summary, brief project background, outline of the methodology used (including any limitations), findings and recommendations by evaluation/review criteria and question; (d) ensure analysis is always backed up with relevant data, with reference to the data source (e) ensure recommendations made are specific and include relevant details for how they might be implemented; (f) contain at least the following annexes – (i) Terms of Reference, (ii) Itinerary for field visit, (iii) List of documents reviewed, meetings attended, persons interviewed/involved in Focus Group Discussions, (iv) Data collection tools.
4. Design and delivery of a Learning Workshop to present and review findings with key stakeholders.
5. Finalised Evaluation Report that has incorporated edits, complete with annexes mentioned above.

Evaluation Criteria and Questions:

The evaluation criteria and questions should assess the projects’ progress against the project design (to the extent that is possible), identify the role that BRC has had during inception, planning and implementation, and identify recommendations for future projects and partnerships. The evaluation should also review the relevance and sustainability of procured goods and services and how they were used by LRC.

The evaluation should allow for BRC, ICRC, LRC and other relevant stakeholders to discuss and consolidate their learning, successes, and challenges to date and focus on recommendations and plans for future work in these areas.

Recommended criteria and questions are outlined below –

Relevance and Appropriateness:

• To what extent and how has the project shown adaptability in relation to (i) the changing context of needs during socio-economic crisis in Lebanon, and (ii) shifting external circumstances?
• What measures did the project put in place to ensure relevance and appropriateness of the assistance delivered? (e.g., targeting, identifying/monitoring exclusion)
• How relevant were selected livelihood activities from the perspective of the beneficiaries disaggregated by age, gender and disability? To what extent did the partnership model adopted for the implementation of the partnership respond to the needs of beneficiaries and of the three institutions?
• To what extent was the project approach and model aligned with BRC/ICRC/LRC strategy and humanitarian cluster response?

Efficiency (including Value for Money):

• What major factors contributed to or hindered management efficiency and achievement of project outputs? (project outputs should not only include assistance delivered)
• Did the project achieve Value for Money?
• Was the clarification and justification around decision making clear?
• Were the inputs considered for the project necessary for the achievement of the objectives identified? And were they available when required?

Effectiveness:

• To what extent was the project successful in achieving its three major outcomes detailed above: (i) Syrian refugees, resident Lebanese, and other vulnerable groups, have a sustainable income; (ii) LRC staff and volunteers have the capacity to implement a pilot livelihoods intervention; (iii) LRC, ICRC and BRC have a strong partnership model.
• How did the capacity building activities delivered as part of the partnership address existing gaps and/or contribute to develop new competencies in the LRC team?
• Did the partnership model adopted in the implementation of the project contribute to the sharing of complementary knowledge and skills?
• Which elements of the model adopted constituted added value that can be replicated in other contexts or projects?
• To what extent did the project outputs and activities contribute to the achievement or non-achievement of project outcomes?
• What were the major internal and external factors contributing to the achievement or non-achievement of project outcomes?
• What good practices, success/failure stories, and/or other lessons learned can be identified, and how can they be shared to positively influence future programming?

Impact:

• What other positive and negative intended and unintended impacts/potential impacts have been generated at individual, household, community and institutional level?
• What is the perceived impact of the partnership for the three institutions?

Sustainability and connectedness:

• Was there an exit strategy in place and was it implemented? To what extent did the partnership under review (2021/2022) contribute to the development of a new phase?
• To what extent are the outcomes and results of the project sustainable beyond the life of the project?
• What were the challenges to project sustainability and what adjustments were needed?
• The review should be guided by/adapted from the OECD/DAC Quality Standards for Development Evaluation. In addition, the review will adhere to the BRC evaluation policy and quality standards (which relate to utility, accuracy and credibility, feasibility, accountability and transparency, ethics and legality).

Approach and Methodology:

The methodology for this evaluation should be mixed. Both quantitative and qualitative methods should be included. The following suggestions need to be further developed and finalised with the evaluator in the Inception Report –

1. A desk review of key documents including project proposal, progress reports, logical framework and indicators.
2. Key informant interviews with current (and past, as appropriate) ICRC, LRC, and BRC staff and volunteers, and other relevant stakeholders (e.g., Movement actors with expert perspectives on programming in the Lebanon context).
3. Focus Group Discussions with staff and volunteers.
4. Surveys with project participants, staff and volunteers.
5. Field visits to project implementation areas.
6. Participatory project and partner review workshops to share major findings with key staff and volunteers focusing on what has happened, what has changed and what should be done in the future.