Re-Advertisement – Consultant for the Evaluation of the APVEL Project

Terms of Reference

Consultant for the Evaluation of the

APVEL Project in Bobonaro

1. Description of the Project

WVTL is currently implementing the Adolescent girls and boys Protected from (gender-based, domestic, at school, at the community level) Violence and Equipped for Life (APVEL) project. The APVEL project operates in 6 sub-villages in Bobonaro municipalities from July 2021 until August 2022.

1. Evaluation Description

2.1 Objective

This evaluation aims to determine the achievement of the desired project objectives and results and provide recommendations and lessons learned for future redesigns. The evaluation will be guided by a questionnaire-based household survey and address key evaluation questions established in the evaluation Terms of Reference section below through focus group discussion and key informant interviews.

Goal

Adolescent girls and boys are Protected from Violence and Equipped with life skills

Learning questions:

• To what extent have the project goal and outcomes been achieved? How has the project impacted beneficiaries?
• What are the perceived benefits of the project from the perspectives of project beneficiaries? What could have been improved? What more could have been done to achieve greater impact?

Logframe Indicators

• Percentage of participants (community including children) report that the incidence of GBV (including harmful traditional practices) has been reduced
• The proportion of adolescents who report having experienced some form of violence in the past 12 months, by any perpetrator, by sex

Outcome 1

Establish GBV free environment for adolescents in schools

Learning questions:

• To what extent is there a GBV environment for adolescents in the school? Is there scope for improvement?
• What core services/structures for preventing GBV and violence against children will continue after the project finishes?

Logframe indicators:

• % of girls absent from school due to period
• % of schools that meet the criteria on gender-responsive school guideline
• % of students (girls and boys) who feel safe in/around their school
• % of girls who have a positive attitude on menstruation
• % of girls who have proper MHM /CSE knowledge
• Number of newly-built toilets with MHM requirements
• % of girls with improved MHM practice during the menstruation period

Outcome 2 Community promotes nonviolent behaviors and zero tolerance for GBV

Learning questions:

• What behaviors and practices have been adopted by teachers, parents, healthcare workers, and service providers for creating a protective environment for children as a result of project activities?
• To what extent are children, including children with disabilities, confidently participating in families, community, and schools?

Logframe indicators:

• % of students (girls and boys) who feel safe in/around their community
• % of students (girls and boys) who witness any form of violence on girls by boys in/around their school for the last 12 months
• % of students (girls and boys) who witness any form of violence on girls by teachers in/around their school for the last 12 months
• % of responded stakeholders (boys, teachers, and caregivers: male and female) who have a good understanding of gender equality
• Number of campaigns conducted focusing on GBV issues
• Number of people who are reached with advocacy campaigns on GBV
• Number of health workers trained on inclusive adolescent-centered health services
• Number of health centers with GBV reporting/response system

Qualitative data

Qualitative data will be collected through focus groups discussions and key informant interviews using a semi-structured interview guide

Focus Group Discussion (FGD)

• Representative parents and IMPACT Club members will be invited to participate in FGDs to explore qualitative aspects of learning questions. Care will be taken to ensure a range of members are invited and encouraged to attend, and FGDs will be held separately and concurrently for male and female participants.
• Teachers, local leaders, and health care workers an FGD will be held to identify potential case studies, barriers, and enabling factors.
• Document and data review, project monitoring reports, ITT and any other secondary monitoring data to be reviewed by the selected external consultant to inform the development of the evaluation measurement tools.

1. End line Study Team

The evaluation will be led collaboratively by an external consultant, WVTL Evidence and Learning Manager, and the Baseline and Evaluation officer. Oversight management will be provided by the WVTL PQ Manager and Gender and Child Protection Technical Specialist. The field data collection team will be led by the Baseline and Evaluation Officer. Field staff will include hired enumerators, hired drivers, and 1 supervisor (project staff). Teams will be assisted by local leaders and Teachers. The survey team will be trained by the WV M&E team and Gender specialist, and then undertake the pilot testing and finalization of the tools, and complete data collection using tablets for data entry. Data analysis and report writing will be the responsibility of the External consultant with the support from the Baseline and Evaluation Officer.

1. Survey Sample/Coverage

The respondents will be sampled from all 6 sub-villages. Within these six sub-villages, a selection of 4 (out of 6) will be selected for inclusion in the end-line sample. This aldeia will be selected randomly with probability proportional to size, with some tweaking to ensure aldeia in the sample will have a sufficient number of adolescents and caregivers of children aged 12-19 years. The sample of adolescents and caregivers will be random; however, at least 50% will be direct project beneficiaries.

1. Roles and Responsibilities of Consultant/Evaluation Lead

The consultant is expected to lead the evaluation process while WVTL will manage the in-country data collection, Given COVID-19 travel restrictions remote management is the likely approach. Specifically, the consultant is expected to:

• Develop the evaluation inception report,
• Finalize the evaluation methodology and sample size, data collection tools including syntax to enable the use of tablets in data collection
• Provide training of the evaluation team or training materials to support remote training
• Provide technical support and supervision for the data collection
• Analyze the evaluation data, and write up of the evaluation report complete with analyzed data as part of annexes and recommendations.

1. Deliverables and Language

All deliverable documents need to be in English. Documents which the consultant is expected to submit:

• Evaluation Inception Report, including data collection Instruments
• Final Evaluation Report

The report should include (but is not limited to) the following:

• • Executive summary (max. 2-3 pages) – Evaluation aims, objectives, and scope – methodology – Findings (related to the objectives and structured considering DAC criteria and questions), comparison of baseline data with evaluation data. – Conclusions – Recommendations – Lessons learned – Annexes (including ToR, maps, questionnaires, list of interviewees, and bibliography and the database (raw data files with variable names and labels) as an attached file.
• 10-slide PowerPoint covering methodology, evaluation findings, and recommendations

1. Qualifications of the evaluation consultant

• Demonstrated experience in the area of gender and child protection, adolescent’s empowerment including disability inclusion.
• Demonstrated oral and written communication skills
• Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations, and prepare well-written reports in a timely manner
• Demonstrated experience in digital data collection, including tablet syntax coding
• Knowledge and prior experience of Timor-Leste
• Fluent in English