Final Evaluation of Renewable Energy Access for Off-grid Communities and Households (REACH) Project

Terms of Reference (ToR)

Final Evaluation of Renewable Energy Access for Off-grid Communities and Households (REACH) Project

1. Background

People in Need (PIN) is an international non-governmental organization with headquarters in Prague, Czech Republic. In its 30 years of history, PIN has worked in over 40 countries in relief, development and human rights and democracy support projects.

PIN is an active member of Alliance 2015, a strategic network of seven European non-governmental organizations engaged in humanitarian aid and development projects and the START Network which is made up of more than 50 humanitarian agencies across five continents. PIN began its operations in the Philippines back in 2013 as part of the relief and rehabilitation efforts for Super Typhoon Yolanda (international name: Haiyan). Since then, PIN has expanded to the areas of agricultural livelihoods, youth empowerment, peacebuilding, health, renewable energy and disaster resilience. Over the years, we have been based in various marginalized areas in the Philippines such as Northern Mindanao, the Bangsamoro Autonomous Region of Muslim Mindanao (BARMM), Eastern Samar and Northern Samar.

PIN Philippines seeks services of a qualified evaluation expert/s (referred to as Evaluator in this ToR) who will lead and conduct final evaluation of the project as described below. The assignment will be carried out in close cooperation with the Monitoring & Evaluation (M&E) Staff and/or another designated staff.

About the project

In line with the Department of Energy’s goal to attain 100% electrification of the Philippines, the Renewable Energy Access for off-grid Communities and Households (REACH) Project was created to reduce the vulnerability of rural poor Filipino communities towards adverse impacts of climate change through enhanced use of renewable energy (RE) sources. This initiative will energize off-grid and underserved households, community-based businesses and community-based organizations, train local RE technicians, and equip vulnerable barangays with disaster risk reduction systems across ten municipalities in Northern Samar. Through innovative, scalable, and sustainable RE technologies and systems, the project hopes to enhance social welfare, disaster resilience and economic growth in these remote areas.

The REACH Project (2018-2022) is funded by the European Union (EU) under the Access to Sustainable Energy Programme (ASEP) and implemented by People in Need (PIN) Philippines in partnership with Entrepreneurs du Monde (EdM) and Malteser International (MI).

Overall Objective: To contribute to reducing the vulnerability of rural poor Filipino communities towards adverse impacts of climate change through enhanced use of renewable energy sources

Specific Objective: To enhance social welfare, disaster-resilience and economic growth of vulnerable off-grid and under electrified communities in remote areas of Northern Samar through innovative, scalable, and sustainable RE technologies and systems.

Outcomes:

1. Rural poor off-grid and/or under electrified households (HHs) increased their demand and access to renewable energy through targeted awareness campaign and customized financing mechanisms
2. Rural poor off-grid and/or under electrified HHs and businesses gain improved access to income-generating activities (IGAs) and sustainable livelihood opportunities through the testing, financing, validation and roll-out of innovative RE technologies and systems
3. Rural poor off-grid and/or under electrified HHs and communities benefit from enhanced social welfare services and living conditions through the use of RE technology
4. Increased capacities of stakeholders at local and national level to manage, maintain, promote, and disseminate RE technology solutions

Key Activities:

• Formative research on barrier analysis and customer surveys to identify affordable, reliable, efficient, and disaster-resilient RE options in target barangays
• Awareness campaigns on use of RE sources among HHs and communities
• Development of financing mechanisms and capacity building of NORSAMELCO
• Procurement, provision, installation, customer service, and maintenance of RE systems
• Market assessment and analysis of RE-based productive uses in target barangays
• Capacity-building of CBOs and CBBs on business modelling, installation, management, and maintenance of productive RE solutions and systems
• Provision of sub-grants for RE solutions to CBOs and CBBs
• Implementation of local RE-powered livelihoods projects
• Rural poor off-grid and under-electrified HHs and communities benefit from enhanced social welfare services and living conditions through the use of RE technology
• Energy needs assessment of social services in target barangays, inclusive CBDRR capacity building and selection of disaster-resilient RE options for improved electrification in remote areas
• Supply, installation, and maintenance of disaster-resilient RE devices and systems and Barangay DRR kits
• Tailor-made capacity-building of local technicians on use and maintenance of RE technology
• Advocacy campaign, business for a and events to promote innovative RE technology uses and stimulate investment from government and private sector sources
• Dissemination of best practices and lessons learned on off-grid electrification through experience sharing and exposure visits

2. Objective and Scope of the Evaluation

The evaluation will aim to provide assessment of achievement of project objectives, make judgment on how intended or unintended, positive or negative changes came about; levels of attribution and participation of the target groups. The evaluation will provide lessons and recommendations that will assist implementing partners and other stakeholders to enhance the impact of their future interventions. The Development Assistance Criteria (DAC) of Relevance, Effectiveness, Efficiency, Impact, Coherence and Sustainability will be used. Cost-effectiveness analysis will be part of the Final Evaluation.

Specific Objectives:

Evaluate to what extent the project has met its goals based on the objectives in the project proposal/logical framework

Assess the added value of the project to the peacebuilding intervention and positive impact to communities

Identify and assess key lessons learned, challenges and draw recommendations for future programming

Geographical scope – 10 Municipalities (LGUs) of Northern Samar (Mapanas, Mondragon, Palapag, Pambujan, Biri, Lao-ang, Catubig, Lope de Vega, Las Navas and Silvino Lubos

Period under review – 18 December 2018 to 18 December 2022.

3. Key evaluation questions

The Evaluator will be responsible for developing the evaluation design and methodology in order to meet the evaluation’s goal and objectives. Details about the approach and the data collection methods to undertake this task should be included in the technical proposal. The evaluation is expected to apply qualitative methods of research – semi structure interviews, focus group discussions, desk research of related secondary data, including project documentation. The methodology is subject to discussion with PIN.

The work to be carried out by the Evaluator will be based on the following:

Relevance:

1. Was the program designed in a way to meet the needs of the target groups, specifically of community members and CBOs/CBBs, including in the changing context of COVID-19 pandemic and political developments?
2. How relevant is the project to the context?
3. To what extent did the project activities contribute in reaching the outcomes?
4. Were the design of activities and modalities of support appropriate considering hard to reach areas?

Effectiveness:

1. To which extent the project contributed to the ASEP goals?
2. To what extent have the intended objectives been achieved? Especially:
3. To what extent were each of the projects’ objectives achieved?
4. How properly was the project executed in terms of quality and timeliness?
5. What were the major factors influencing the achievement or non-achievement of the objectives?

Efficiency (sound management and value for money):

1. To what extent were the financial and human resources used economically and efficiently?
2. Were the management mechanisms proving to be appropriate to achieve planned outputs and contribute to outcomes?
3. How were the project resources used to achieve results of the desired quantity and quality?
4. What was the capacity of project management team to respond to challenges and needs?
5. Was the management, coordination, and monitoring of the project efficient and appropriate?

Sustainability

1. To what extent were the net benefits of the intervention continue, or are likely to continue?
2. What are some of the major factors that may influence sustainability of the project?
3. What can be observed as the most significant changes due to the project?
4. What could be done to enhance the sustainability of the project results?
5. To what extent the support reduced vulnerable households’ dependence on kerosene, oil and other unsustainable energy sources?

Coherence

1. How well does the intervention fit?
2. How compatible is the intervention with other interventions in the Philippines, and in Renewable Energy sector?

Impact

1. What has been the impact of the project on local networks and community groups? Were there different impacts for different sub-groups?
2. Are there changes in perception or behaviour of community members or clients of CBBs/CBO projects (such as willingness to pay and subscribe for access to energy from a RE source, increase in awareness of advantages of using renewable energy products, etc.)?
3. Are there any other unintended, positive, or negative effects or impacts of the project?

Cross-cutting Issues:

Community Feedback and Response Mechanism (CFRM) / Accountability Mechanism

1. To what extent the project was able to reach the most vulnerable groups of rural communities?
2. How familiar are communities regarding project’s accountability mechanism?

Gender-equality and Social Inclusion (GESI)

1. To what extent the project implemented gender equity, social inclusion, and human rights principles and what measures are required if the principles are not adequately and effectively implemented?

PIN will require specific and data-supported answers to each of the agreed evaluation questions or strong justification for why the data could not be obtained. Justification of data unavailability must be communicated by the evaluator without any delay as soon as it becomes apparent and approved by PIN. Failure to do so can result in decreasing the evaluator’s remuneration.

4. Methodology

Desk review

• Review of project’s logical framework and description of action, including its amendments/revisions
• Review of the following documents:
• Baseline assessment
• Formative research
• Project reports
• Partners’ reports
• ROM report
• Other available reports as reflected in the MoV archive

Field Data Collection

• Key informant interviews (KII) with project teams, partners, and target groups
• Focus Groups Discussions with beneficiaries
• Most Significant Change, Outcome Harvesting, or similar participatory qualitative approaches
• Surveys

Tools:

• Semi-structured interviews (For KIIs)
• FGD Guide

The data collection process is in the responsibility of the evaluator; however, PIN will oversee and follow up on the data collection process, and the project partners will help facilitate communication with the project stakeholders and beneficiaries. Data collection will take place in the Philippines; the modality of data collection will also depend on the health situation in relation to COVID-19, and the security situation in some municipalities in Northern Samar.

When dealing with personal data, People in Need follows Guidelines on Processing Personal Data and The Principles of Work with Personal Data. PIN’s MEAL manual contains guidance, specifically related to data collected for the monitoring and evaluation purposes (relevant extract of the document will be provided to the Evaluator contracted for the service upon request). In line with these documents, the evaluation team will make clear to all participating stakeholders that they are under no obligation to participate in the evaluation data collection activities. All participants will be assured that there will be no negative consequences if they choose not to participate. The monitoring team will inform respondents about acquiring their data from PIN and obtain informed consent from the participants. In case any special category data (e.g. about health, ethnicity, religion etc.) are collected that was not collected by PIN during beneficiary registration and/or no written consent was obtained, the evaluation team shall obtain signed consent. The evaluation team will ensure prior permission is received for taking and use of visual images for specific purposes. The evaluation team will ensure the visual data is protected and used for agreed purposes only. In particular, the Evaluator will employ robust data security measures (related to storing and sharing datasets containing personal information) to further ensure participants’ confidentiality and anonymity.

5. Specific Tasks

• Desk review of project documents
• Development of methodology to be used for conducting evaluation, including data collection tools
• Data collection, including interviews with project staff, partners, and other relevant stakeholders
• Data analysis and presentation of preliminary findings
• Draft Report, including presentation of main findings
• Preparation of final evaluation report (maximum of 25 pages excluding annexes and executive summary)

6. Expected Deliverables

• Inception Report detailing the methodology, workplan, and data collection tools, etc. in consultation with PIN
• Draft evaluation report
• Final evaluation report

The evaluator is also expected to submit cleaned data (final versions of KII/FGD transcripts used in the analysis, survey data, if applicable, etc.)

7. Evaluation Report

The Final Evaluation report should include the following:

EXECUTIVE SUMMARY: Summary of the evaluation, with particular emphasis on the main findings, conclusions, lessons learned and recommendations.

INTRODUCTION: Description of the evaluated intervention, its logic, history, organisation and stakeholders. Presentation of the evaluation’s purpose and questions.

METHODOLOGY: Description of the sampling strategy and methods used for data collection; description of the limitations.

FINDINGS: Factual evidence relevant to the questions asked by the evaluation and interpretations of such evidence (answered evaluation questions).

CONCLUSIONS: Assessments of intervention results and performance against given evaluation criteria and standards of performance. For example:

• Problems and needs (Relevance)
• Synergies and interlinkages (Coherence)
• Achievement of purpose (Effectiveness)
• Sound management and value for money (Efficiency)
• Achievement of wider effects (Impact)
• Likely continuation of achieved results (Sustainability)
• Mainstreaming
• Capacity building and partnerships
• M&E processes

LESSONS LEARNED: General conclusions with a potential for wider application and use.

RECOMMENDATIONS: Specific and actionable proposals regarding improvements of the project or management addressed to the client of the evaluation or other intended users.

ANNEXES: Terms of Reference, evaluation tools, Logframe, references, etc.

The evaluation report Executive Summary should not exceed the limit of 3 pages, and the remaining parts of the report should not exceed 25 pages (excluding annexes).

8. Duration

The evaluation duration is maximum six weeks, starting from the second week of November 2022.

9. Logistics

All logistical arrangement is the responsibility of the evaluator and should be included in the budget.

10. Qualification requirements

The applications will be evaluated according to the following scoring criteria and weighs:

• Evaluator’s relevant experience and expertise (CV, work samples) – 40 %
• Evaluation plan and methodology – 25 %
• Quality Assurance and Safeguarding/risk management plan – 15 %
• Price in EUR – 20 %

11. Payment modalities

Maximum budget available for evaluation is EUR 10,000.

Payments will be made in instalments as follows:

• 30 % – Milestone: Following the signing of the Contract on Evaluation Services
• 20 % – Milestone: After the inception report has been submitted and approved by PIN
• 20 % – Milestone: After the 1st draft of the final evaluation report has been submitted and approved by PIN
• 30 % – Milestone: After the final version of the final evaluation report has been submitted and approved by PIN

PIN reserves the right to deduct up to 0.5% of the total contract price for each day’s delay in meeting the deliverables specified. This deduction shall be applied to the last payment of 50% of the contract price.

How to apply

Applications should be submitted to PIN by courier service to reach the PIN office address: Unit 309 The Sally Apartment, 32 Mapagbigay St., Brgy. Pinyahan, Quezon City 1100 Philippines or send to PIN email address: [email protected] cc: [email protected] before closing date and time: 13 October 2022, 17:00hrs Philippine time.

The application package should contain:

– Technical Proposal not exceeding 2 pages

– Quality assurance and safeguarding/risk management plan for this evaluation not exceeding 0.5 page

– Timeline draft with milestones

– at least 1 sample (max.2) of past work related to the focus of this evaluation not exceeding 10 pages combined

– Budget with daily consultancy rates, accommodation costs, translation, transcription, and other evaluation related costs

Additionally:

– For organizations

• Registration documents (SEC, CDA, others)
• CVs of evaluation team, clearly stating their roles and responsibilities for this project evaluation
• Evaluation report (sample of an evaluation conducted, need not be the full report)
• 2 verifiable references

– For individuals

• CV
• Government-issued ID
• Evaluation report (sample of an evaluation conducted, need not be the full report)
• 2 verifiable references

The documents mentioned above shall demonstrate the following desired competencies of the evaluators:

• Relevant subject matter knowledge and proven experience as an evaluation firm in relevant topics;
• Previous experience of conducting evaluation in a similar context with local community knowledge;
• Previous experience in a variety of data collection methodologies (quantitative and qualitative, mixed method);
• Previous experience in inclusive approach and gender sensitivity;
• Fluency in local language/s;
• Proven experience of work with INGOs;
• Capacity to provide high quality materials and reports in English on time.