Final Evaluation of Multi-hazard Project Consultancy.

TERMS OF REFERENCE

Consultancy Service For Final Evaluation of project “Creating an Advanced Transformation of Civil Society Environment for Resilience Building in Multi- hazard Settings” (Multi-hazard Project)

1. INTRODUCTION

Each year, natural and human-induced disasters adversely affect hundreds of millions of people. Since the last decade, 2.3 billion people have been directly affected by disasters. In 2011 alone, almost 200 million people were affected, including 100 million children1. Overall, children constitute 50-60 per cent of those affected by disasters2. Currently 1.5 billion children live in conflict affected or fragile states3. These children experience heightened exposure to abuse, neglect and exploitation, and child protection is commonly identified as a top priority amongst children in conflict and disaster affected communities.

During emergencies and fragile contexts, girls may be especially vulnerable4. Child protection violations, particularly gender-based violence not only have immediate and life-threatening consequences for girls and boys, but also impact on the resilience of communities and their ability to cope with emergencies. The girls and boys, who are among marginalized groups, such as children and adolescents with disabilities, children and adolescents from ethnic or religious minorities, children and adolescents without parental care, and children and adolescents from the poorest segments of society, are even more vulnerable when living in communities under multi-risk circumstances affecting them.

As a Child-Centered Organization, Plan International Asia-Pacific (Plan APAC) is regionally implementing with support from different donors a Safe Schools and Resilience Program in Asia to support Children in the most at-risk communities have access to safer education through duty-bearers minimizing the impact of disasters on their right to the quality education. The program has been running in thirteen countries and benefits around 1,950 schools and 264,855 children both in urban and rural settings.

From July 2019, with the support from Sida, Plan APAC started a new project in two typical high-multi-risk areas for its implementation, in two countries of Asia region: Myanmar (conflict country) and Timor-Leste (post-conflict), where communities are among the most disaster-prone and high gender-based violence risk exist, especially for girls, with conflict and post-conflict settings. This project (FAD ARO100110), entitled ‘Creating an Advanced Transformation of Civil Society Environment for Resilience Building in Multi-hazard

Settings’, was financially supported by Sida through Plan International Sweden (SNO) under the framework agreement with Sida’s unit for support through civil society (Sida Civsam)5.

Through the collaboration with regional and country level CSO partners, the project has aimed to achieve its goal in forming a dynamic, and inclusive civil society, vertically and horizontally linked from sub-national to national and regional arena to promote the children’s rights and resilience against natural and human- induced disasters, including girl equality in both normal and at-risk communities.

The specific outcomes of this Multi-hazard project are:

• Civil society organisations, including child and youth led networks, have strengthened their capacity and collectively take actions holding duty bearers accountable for implementing their policy and practice commitments on disaster risk reduction initiatives.
• Children and youth, including child and youth led networks, have increased their participation in, and influence on, community/national decision-making processes on issues that affect their right to safety.
• Duty bearers, including government agencies and community leaders, have actions on their responsibility, and strengthened, resourced and implemented disaster risk reduction initiatives for making children safe.

1. OBJECTIVES OF EVALUATION

The overall aim of the evaluation is to assess result and impact of the above-mentioned Multi-hazard project against the project goal and outcomes in targeted countries and at regional level.

In particular, the evaluation aims to:

1. 1. Assess the project result areas in relation to the following evaluation criteria:
      • Effectiveness – this criterion focuses on the assessment of achievement of project objectives at the outcome level.

(Questions to be considered for study: How did the project achieve the outcomes? How relevant was the original project design for the priorities of the target group?)

• • • Relevance – this criterion focuses on the relevance of the project design to the priorities and challenges for children and youth in the specific country context.

(Questions to be considered: What is the stakeholders’ view of the relevance of project activities, in particular related to strengthening of civil society?

• • • Efficiency – this criterion focuses on the assessment of implementation of the project from the perspective of cost efficiency and beneficiary reach.

(Questions to be consider: To what extent was the project implemented in a cost-efficient way?)

• • • Sustainability – this criterion focuses on the sustainability of the achievements of the project when the projects end.

Questions to be considered for study: How has the project ensured sustainability of the achievements made (e.g. use of and strengthening existing systems, and contributing to institutionalization)? How do partners promote internal learning and communication of the achievements to external audience?

• Cross-cutting issues to what extend the project activities have been designed and implemented to be considerate of child-centeredness, inclusion, ethnicity, gender, disability and accountability.

Questions to be considered: How consistent is the project with the levels of Plan standards and accountability (Working with children and communities, Gender equality and inclusion of marginalized groups, engaging with civil society, Influencing government)?

• Coherence: The extent to which other interventions (particularly policies) support or undermine the intervention, and vice versa. Includes internal coherence and external coherence. Questions to be considered: internal: whether there is consistency of the project interventions with the relevant international norms and standards to which institution/government adheres? And external: whether or not the project interventions have complementarity, harmonisation and co-ordination with others, and the extent to which the intervention is adding value while avoiding duplication of effort?
• Impact: to what extent to which the project interventions has generated or is expected to generate significant positive or negative, intended or unintended, higher-level effects?
  1. Assess the added value and relevance of Plan International for strengthening civil society to become drivers of change able to hold duty bearers accountable for commitments made for advancing the rights of girls and boys;
  2. Identify the project’s key challenges during implementation, and lessons learnt
  3. Generate concrete conclusions and recommendations that can feed into decision making processes regarding up-scaling Safe school programming in the region and targeted countries and beyond.

The evaluation is expected to present evidence, recommendations and learnings which can be utilized by Plan International, and implementing partners in the development of potential future projects. The findings of the evaluation are also important for improving programme quality, identifying collaborative approaches with the civil society, and develop working methods together with children and youth at the community level.

1. SCOPE AND METHODOLOGY

Scope of the Evaluation:

The evaluation should assess the results of the Multi-hazard project against the outcomes set out in the original project documents. Given the challenges with Covid-19 with travel difficulties and security risks in targeted countries, in particular Myanmar, the evaluation will conduct remote assessment, including interviews with key informants via virtual meetings with colleagues and partners in Myanmar and Timor- Leste – two targeted countries within the project.

The evaluation should collect feedbacks from CSO sub-granted partners, and the assessment will explore the projects’ related information for its full project period as mentioned in the background.

Methodology:

• The recruited consultant team will be expected to propose a detailed methodology (based on the information given in this ToR) suggesting adjustments to the methodology recommended below and to the available tools, including online survey tools, as necessary to meet the objectives of the evaluation as detailed above.
• It is expected that the methodology will include both qualitative and relevant quantitative data collection methods and will consider both primary and secondary data sources.
• The consultants will be expected to review project documentation, conduct virtual meetings for interviews with key stakeholders including project management and field staff, local partner organizations and government representatives where possible, as well as representatives of beneficiaries in the project areas of targeted countries.
• It’s encouraged that virtual interviews enable a participative and child-friendly method. Other methods including use of focused group discussions (FGDs), observations, documentation of the most significant change stories/case studies are also highly recommended.
• For communication and interviews with key informants, Plan International’s staff in each related country will be appointed to support the consultants in coordinating for meetings and invite informants.
• For related costs associated with virtual interviews, e.g. Internet connection, the consultants will receive a reasonable lumpsum, based on proposal and in consultation with local Plan staff, and must be agreed as part of contract terms, as part of consultancy fee, which will be used by consultants to support their data collection process, and also to cover for the key informants’ connection to join the meetings where needed. The consultants should settle this directly to the key informants themselves to ensure the connection during interviews, where required.
• The total number of working days for evaluation will be the sum of evaluators’ working days of the consultant team.
• The consultants will be selected by Plan International Asia based on the acceptance of the consultant on the above outputs and timeframe. The recruitment will be proceeded based on Plan International Asia’s Administrative procedure.

1. DELIVERABLES

The Consultants will be expected to provide the following deliverables for project evaluations:

• Proposal with detailed methodology, proposed tools, workplan, and costs/fee
• Inception report upon award of the contract
• Zero draft evaluation report for internal review
• Debriefing on the final findings (can be arranged via presentation over virtual meetings)
• At least one most significant change story/Case study
• Final evaluation report addressing all objectives, evaluation questions and including clear recommendations. The final evaluation report is the sole deliverable expected from the Consultants. The report shall be:
  • Produced in English language and should be simple in expression.
  • Maximum of 50 pages. Annexes to be attached separately
  • The report format and text should be on A4 paper size in Arial font size 11.
  • The consultants will deliver an electronic version of the evaluation report in Word version by the agreed deadline. The same copy in PDF version should be attached in the final submission.
  • The report should follow Plan International’s branding guidelines and use of logos (to be provided after signing the contract)
• As a reference, the report should be structured as follows (difference to this outline can be

agreed between consultants and Plan International focal staff of this evaluation:

• • Cover page – Acknowledgement – Table of contents – List of tables

1. • List of figures
   • Executive summary comprehensive enough to highlight all the key aspects of the evaluation. The summary should present a highlight of the methodology, key findings, summary of conclusions and recommendations.
   • Introduction section
   • Literature review
   • Methodology that clearly outlines the sampling procedures and the sample size, how the data was collected, analyzed and presented with clear justification and limitations if any.
   • Findings addressing all the evaluation questions and the unexpected outcomes.
   • Conclusions presenting summary of the findings based on the evaluation questions.
   • Lessons learned and recommendations which should be clearly related to the conclusions.
   • Case story highlighting best practices.
   • Report annexes that include the tools used in the evaluation and the list of respondents.
2. SCHEDULE

The assignment is estimated to consist of not more than 40 working days (based on the sum of working days of each member in the consultant team). The total number of working days includes days home-based for the documentation review and report writing.

The main evaluation works will be expected within 20 August – 10 September. The final report/deliverables should be delivered no later than 15 September 2022.

Precise schedule will be agreed between the consultant team and Plan International upon consultant submitting the draft work plan in the proposal.

The timing below is approximate and for reference, all work and final payment has to be completed before 15 September 2022.

Activities / Outputs / Responsible persons /Time

– Inception discussion with Regional Program Coordinator for review of project final evaluation proposal, including budget, and review of key project documents and existing data collection tools. / Contract signature upon agreement on TOR and final evaluation proposal / Consultant team and Plan focal staff (Regional Specialist) / 2.5 working days

– Production of detailed draft methodology, work plan, staffing, and data collection tools. / Draft Evaluation Inception report submitted for approval to Plan / Consultant team / 2.5 days

– Conduct consultation and data collection (expecting 5 days via online meetings, recommended for no longer than 1,5 hoursper call) / Qualitative and quantitative data collection completed. / Consultant team, Plan focal staff at regional and country level / 7 days

– Analysis and drafting of evaluation report / Raw quantitative and qualitative data files consolidated; first draft report submitted for review to plan /Consultant team / 5 days

– Review and feedback from Plan International (including online calls if needed) / Comments on the draft report /Plan colleagues / 2 days

– Finalization of the evaluation report based on feedbacks from Project Staff and Plan International / Final report submitted /Consultant team /2 days, all works and payment completed by 15 Sep 2022

1. CHILD PROTECTION AND ETHICS

• Individual consultant will be asked to sign a child protection policy as this is critical for the work that Plan International do;
• Selection process and awarding of contract will be done following Plan International Asia Pacific’s HR and Admin procedure
• The consultant will be checked for anti-terrorism before the awarding of contract.

1. COMPENSATION OF CONSULTANCY

• The total professional fee for the Consultant depends on the qualification. The Consultant will be paid in tranches upon the satisfactory completion and approval of indicated outputs:
  • 30% upon signing of contract
  • 70% upon the satisfactory completion and approval of final report version.
• The consultant is responsible to pay their own tax, unless the country of domicile requires that tax and other government liabilities be deducted at source.
• The agreed cost will be paid to the recruited consultants after the satisfactory delivery of work according to this TOR, upon receipt of consultant’s invoice in accordance with Plan International’s payment policies, and the service agreement between the Consultant and the Plan International Asia Pacific’s maximum paid working days should not be over 20 days for the entire evaluation works.
• For support to the organization of remote assessment (e.g. local transportation and internet connection for informants for group discussion in targeted sites, translation), the consultant team will be provided a lumpsum of budget to cover for those eligible costs (specific amount will be based on proposal, and in consultation with Plan’s local colleagues), and receipts of the eligible expenditure are required as part of liquidation to Plan by consultant team.

1. SKILLS AND QUALIFICATIONS

The consultant team is expected to meet the following desired criteria:

• Demonstrated experience of conducting evaluations of similar focus and scope;
• Demonstrated experience of conducting evaluations in development contexts; demonstrated experience in targeted countries, and/or having local team members based in targeted countries of the project is desired.
• Extensive experience of developing and applying methodological approaches including quantitative and qualitative elements, with online tools.
• Extensive and demonstrated experience of child centered, participatory evaluation methods for data collection and analysis in programme evaluation
• Excellent understanding of the role of civil society for development in general and for children’s rights in particular;
• Excellent understanding of the thematic priorities of Plan International, such as child protection, gender equality and inclusion.
• A commitment to promoting and supporting child rights including gender equality and inclusion
• Fluent spoken and written communication skills in English; Strong interpersonal and analytical skills.
• Strong commitment to deliver high-quality on tight time frames and meet deadlines within agreed budgets;
• Previous experience of taking on mission for Plan International or other civil society organizations.
• Experience in facilitating online meetings and conducting assessment using online survey tools.
• Experienced and having effective interaction with local and national organizations and government institutions.

1. CONFIDENTIALITY

All materials and data furnished by or on behalf of Plan International Asia Regional Office in connection with these Terms of reference, materials created or produced shall be treated as confidential information. The Consultant shall not, during the continuance of these Terms of Reference or at any time thereafter, disclose to any person any confidential information.

1. RIGHTS TO COPYRIGHT

The copyright of the content and materials produced for this assignment shall be the absolute property of Plan International and ownership shall automatically vest in Plan International. The consultant irrevocably and unconditionally waives in favour of Plan International any moral rights that may exist in any of the works or materials, provided that Plan International uses all reasonable endeavours to ensure that the consultant is credited in all appropriate uses of a photograph provided as part of the service, provided that Plan International shall not be in breach of contract if credit(s) are accidentally omitted.