Request for Proposals- EXTRAS Project Endline Evaluation – Jordan

CONSULTANT TERMS OF REFERENCE:

‘EXTRA-CURRICULAR TRAINING, REHABILITATION AND SUPPORT (EXTRAS)’ ENDLINE STUDY

2. RIGHT TO PLAY

Established in 2000, Right To Play is an international non-governmental organization committed to improving the lives of children and youth affected by conflict, disease, and poverty. Headquartered in Toronto, Canada, Right To Play is active in 15 countries across Africa, Asia, and the Middle East, and has fundraising offices in six European and north American countries. Pioneering its innovative child-centered, play-based learning methodology, Right To Play works across the five key outcome areas of quality education, peaceful communities, health and well-being, gender equality, and child protection.

Right To Play has been active in the education and sports for development sectors in Jordan since 2006, training teachers in UNRWA and Ministry of Education schools as well as coaches in local communities to provide engaging interactive sessions that utilize games and sports to promote psychosocial wellbeing, education and gender equality. Vulnerable Jordanian, Palestinian, and Syrian children who take part play days and tournaments organized by Right To Play are more engaged in school and have vastly improved life skills. In turn, the youth with whom Right To Play works achieve higher grades, experience greater levels of cooperation and cohesion, and are empowered to overcome trauma, adversity, and cultural stigmas to fulfil their dreams.

All RTP projects are gender-responsive, placing particular emphasis on female empowerment. They are also usually supported by a sustainable rehabilitation component whereby school classrooms and play spaces are renovated, football pitches are constructed within underserved communities, or students lead maintenance initiatives within their schools. In 2019, RTP reached a total of 102,848 Jordanian, Palestinian, and Syrian children and youth in schools across Amman, Irbid, Zarqa, Mafraq, and Aqaba. In addition, the organization currently has 2,904 active teachers and coaches, having trained 766 this year.

2. Extra-curricular Training, Rehabilitation and Support (EXTRAS)

Approximately 1.2 million school-aged children have been affected by the closure of all educational facilities in Jordan since March 2020 in response to the COVID-19 pandemic.[1] Huge educational losses abound, while the risk of teacher attrition increases, threatening a decline in educational quality and increased student dropout rates.[2] Although the Ministry of Education has launched a distance learning strategy involving lessons aired on television and its online ‘Darsak’ platform for both UNRWA and MoE students, the challenging socio-economic circumstances faced by the most vulnerable UNRWA students prohibit access to technology and good internet, significantly hampering learning. According to an UNRWA rapid assessment undertaken in response to the crisis, 25% of students reported no access to Darsak or televised lessons; 24.6% had no internet, a further 42.4% poor quality internet, and only 60.6% of households possessed a TV. Many parents, furthermore, reported that it was too difficult to continue learning at home, or that that they could not motivate their children.[3]

These challenges come on top of an already overburdened education system. UNRWA schools in Jordan’s Palestinian refugee camps have been detrimentally affected by funding cuts, and suffer from crumbling infrastructure and severe overcrowding, often accommodating 40 to 45 students per class and running double shift systems. With students having been out-of-school and often unengaged for over a year, the importance of improving the school environment and building motivation to learn in order to bridge the learning gap effected by the COVID-19 crisis is especially pertinent.

The EXTRAS project, funded by Latter Day Saints (LDS), aimed to increase students’ motivation to learn, helping them to bridge the learning gap affected by school closures in Amman and Zarqa. Six UNRWA schools (four in Amman and two in Zarqa) have been selected to benefit from the project. The project will train a total of 21 English and mathematics teachers, as well as teachers of Grades 1 to 3, reaching a minimum of 944 students.

3. PROJECT GOAL AND OUTCOMES

IMPACT

Students are more motivated to learn, helping them to bridge the learning gap effected by school closures

OUTCOMES

1. Students have access to improved learning environments.
2. Students receive quality education and support.

Target Geographies

1. Amman and Zarqa, Jordan

Target Beneficiaries

• At least 944 vulnerable children
• 21 teachers

Full Project Timeline

15 months (September 2021 – December 2022)

Endline Study Timeline

Approximately six weeks, beginning at end of November 2022 by Mid of January 2023.

Under the goal we have the following two indicators:

% of students achieving an improvement of at least 30% in literacy and numeracy (disaggregated by gender, grade, school, and governorate).

% of students demonstrating increased motivation to learn.

And the project has two closely linked outcomes, as explained below.

Outcome 1: Schools provide safe, positive, and well-equipped learning environments for students

• % of teachers who demonstrate an increase in the effective application of PBL and knowledge of minimum standards for a positive learning environment according to RTP standards (disaggregated by gender, school, and governorate)
• % of students with access to improved physical learning environments in terms of indoor and outdoor safety, hygiene, and accessibility.

To achieve Outcome 1, Right To Play will take a three-pronged approach to cultivating environments conducive to student learning:

• Rehabilitating schools: Classrooms and washrooms were rehabilitated to meet positive learning environment principles and hygiene standards, while playgrounds were rehabilitated to provide safe, sun-shaded spaces for students.
• Training teachers in RTP’s play-based learning methodology: Pedagogical studies prove that creative, interactive, and applied learning is more effective for promoting information retention and transfer, as well as socio-emotional wellbeing and attitudes of inclusion, tolerance, and acceptance. RTP’s ‘Continuum of Teacher Training’ methodology guides Jordanian educators in attaining new knowledge, shifting their attitudes, and gaining the skills to facilitate children’s educational development. Combining the ‘CoTT’ and its literacy and numeracy training approaches, the teacher trainings will enable teachers to design and conduct effective play-based sessions tailored specifically towards literacy and numeracy learning objectives.

Outcome 2: Students receive quality education and support while building their life skills and motivation to learn

• % of students regularly participating in play-based and/or interactive learning activities in school (disaggregated by gender, age group, school, and governorate)
• % of parents that show positive attitudes towards supporting their children’s learning needs (disaggregated by gender, school, and governorate)

To achieve Outcome 2, RTP will:

• Support schools to identify critical learning needs: Learning needs were identified through needs assessments.
• Facilitate the delivery of tailored catch-up classes: Trained English and math teachers in each school delivered catch-up classes in their respective subjects informed by the needs assessments.
• Provide play-based learning extracurricular English and math activities: “Reading through Play’ sessions were conducted for students, while school spelling bees and book clubs established to build students’ literacy skills while cultivating enthusiasm for the English language. Concurrently, play-based math sessions were be conducted and students participated in sports-based ‘mathlete’ tournaments to enhance numeracy skills. Finally, teachers were trained in cricket-based learning (CBL) by Right To Play trainers. CBL aims to enhance life skills and support children and young people’s learning.

4. PURPOSE OF THE CONSULTANCY

The objective of this consultancy is to evaluate end of project values for all project indicators outlined in the performance management framework and measured during the baseline assessment and generate learning for future projects The consultant will measure the values obtained against those identified during the baseline assessment, taking into account data obtained through regular monitoring, to provide a comprehensive analysis indicating the degree to which the project has achieved its expected outcomes. An overall assessment of the project’s success will be provided, alongside recommendations for future programming based on the evaluation and, if necessary, an evaluation of the reasons why expected outcomes were not achieved.

• Perform a comprehensive review of existing project materials, monitoring and evaluation activities undergone, and measurement tools, including, in particular, the baseline study and performance management framework.
• Design and plan a final evaluation in collaboration with Right To Play staff. The consultant should maintain continuity with the baseline data collection tools, sampling, and methodology if possible; a review of these tools and methodology will nevertheless be necessary and alterations should be suggested if needed.

Submit an inception report prior to the start of the study, providing a detailed evaluation plan that gives:

• An outline of the study approach/methodology, stressing continuity with the baseline study if possible (if not, thorough reasons for alterations must be discussed Right To Play prior to submission of the inception report)
• The timeline of activities
• Sampling requirements and framework, stressing continuity with the baseline study if possible (if not, thorough reasons for alterations must be discussed Right To Play prior to submission of the inception report)
• The methodology of collecting, triangulating, and summarizing qualitative and quantitative primary and secondary data, stressing continuity with the baseline study if possible (if needed, data quality protocols should be developed to guide data collection/entry for RTP field staff, including spot checking protocols). If continuity is not possible, thorough reasons for alterations must be discussed with Right To Play prior to submission of the inception report.
• Fieldwork logistics and schedule
• Fieldwork manual including data collectors’ training guide.
• Provide training for data collectors and submit the following:
• Data collectors’ training agenda
• Data collectors’ training report
• Provide a comprehensive final evaluation report (both a draft and final version) that:
• Establishes end-of-project values for all outcome and output indicators
• Evaluates the degree to which project outcomes were achieved by comparing end-of-project indicator values with baseline values, as well as analyzing any other relevant qualitative or quantitative data obtained during regular project monitoring
• Provides recommendations for future programming based on the findings of the final evaluation and, if necessary, identifies areas for improvements based on a thorough analysis of any reasons why outcomes may not have been achieved
• Provides a summary of the project and the findings of the baseline assessment to serve as an introduction to the final evaluation

5. SCOPE OF WORK:

1. Prepare an evaluation plan outlining the approach/methodology to be employed in executing the assignment and a detailed plan for the baseline with:

• Timeline/schedule of activities
• Sampling requirements and framework (determine sample size for intervention and comparison areas)
• Detailed methodology of how to collect, triangulate and summarize the primary and secondary data of both quantitative and qualitative (if needed, develop data quality protocols to guide data collection/entry for Right To Play field staff, including spot checking protocols)

1. Revise and identify the final baseline indicators and assumptions from the project log frame and/or any other relevant M&E planning tools (proposed revised version)
2. Take a lead role in developing/revising and piloting various data collection tools (both qualitative and quantitative) to be used for the endline as per the agreed methodology, including pre-testing or piloting of tools for linguistic and cultural appropriateness
3. Actively participate in regular meetings with Right To Play staff from Jordan and Regional Office, consulting on research plan/methodology/timeframe, discussing results and findings and agreed recommended follow-up actions
4. Depending on the consultant’s evaluation plan, collaborate and coordinate with RTP field staff and travel to project locations to manage data collection process, providing support, supervision and monitoring of data collection and storage. This includes ensuring the credibility of field data collected by interviewers
5. Compile a comprehensive first draft endline report based on the field research findings according to format agreement upon with Right To Play staff
6. Prepare and submit final endline report to Right To Play staff in Jordan, including:

• PowerPoint presentation with summary findings for formal presentation to key stakeholders
• Conduct de-briefing meeting at the Right To Play Jordan office
• Copies of original and cleaned data sets including any field notes, as well as annexes of processed results tables (with all levels of disaggregation, including geographical areas breakdown) are to be submitted to Right To Play Jordan office with the final report

6. ENDLINE METHODOLOGY

The final evaluation will utilize all tools used during the baseline study (surveys, focus group discussions, and key informant interviews) to collect both qualitative and quantitative data, ensuring in-depth analysis of the status of project outcomes. A review of the baseline study as well as relevant project and organizational documents must be undertaken prior commencing the evaluation. Specific tools must also be reviewed, with any recommendations for alterations discussed with Right To Play prior to data collection. All data collection practices must be in line with child safeguarding principles.

The final evaluation will be conducted in the communities where the project is being implemented. Right To Play will support the consultant in making logistical arrangements for data collection and meetings with key stakeholders, and the consultant will be expected to travel to project locations chosen on the basis of sample size.

Right To Play will review the consultant’s submitted inception report prior to data collection and will provide guidance for the final evaluation. All raw data must be submitted to Right To Play prior to submission of the final report.

7. ROLE DURATION

The endline study should begin in Mid December 2023 with design, data collection, analysis and reporting lasting approximately six weeks. We expect data collection to begin by December2022/ January 2023, with the draft report submitted by Mid January 2023.

8. DELIVERABLES

Deliverables are illustrated in the table below; the specifics and timeframe should be negotiated with Right To Play upon confirmation of the consultancy.

DELIVERABLE

DETAILS

TIMEFRAME

1. Right To Play consultations

• An initial kick-off meeting will be held with RTP to finalize plans for the study.

20/12/2022

2. Submission of Inception Report with evaluation and fieldwork plan

• Evaluation plan:

  • Outline of approach/methodology, stressing continuity with the baseline study if possible
  • Timeline of activities
  • Sampling requirements and framework, stressing continuity with the baseline study if possible
  • Methodology of collecting, triangulating, and summarizing qualitative and quantitative primary and secondary data, stressing continuity with the baseline study if possible (if needed, data quality protocols should be developed to guide data collection/entry for RTP field staff, including spot checking protocols)

• Fieldwork plan:

  • Fieldwork logistics and schedule (ie. in Excel)
  • Fieldwork manual including data collectors’ training guide

21/12/2022

3. Training of data collectors and submission of report

• Data collectors’ training agenda
• Data collectors’ training report

23/12

4. Data collection

• Ensure that data pertaining to all outcome and output indicators is collected.

27/12/2022 – 30/12/2022

5. Data submission

• Cleaned, raw data in two formats: Excel and a stats software format (e.g. SPSS or STATA) if possible.

4/1/2023

6. Validation Workshop

• Consultation with key stakeholders for revision and validation of findings.

4/1/2023

7. First Draft Final Evaluation Report

• Revised data analysis framework document
• Excel spreadsheets, or SPSS/Stata scripts with clear, easy to understand notes
• Ensure that all values for outcome and output indicators are provided (filled in on the PMF).
• Draft final evaluation report.

11/1/2023

8. Finalized Final Evaluation Report

• PowerPoint presentation with summary findings for formal presentation to key stakeholders.
• Copies of original and cleaned data sets including any field notes, as well as annexes of processed results tables (with all levels of disaggregation, including geographical areas breakdown).
• Final report

22/1/2023

9. De-briefing meeting with RTP

• N/A

TBC

9. QUALIFICATIONS

• A consultant/consultancy firm/group with at least 3-5 years of experience in the research and/or evaluation field, including experience in quantitative and qualitative data collection, analyzing quantitative and qualitative data, and reporting.
• Extensive experience managing and designing evaluation studies in diverse contexts (e.g. Middle East context, with children and young people, in urban settings, etc.).
• Extensive experience creating measurement frameworks, refining indicators, and creating measurement tools for education-focused programs.
• Experience in education.
• Experience in Positive Youth Development (PYD), life skills measurement and behavioral change measurement, and youth psycho-social wellbeing.
• Experience using participatory methodologies and following OECD-DAC principles for evaluation and measurement.
• A relevant degree in social sciences, international development, statistical sciences, or another related field.
• Excellent skills in, and experience with, data analysis using statistical computing tools (Excel, SPSS, STATA, NVIVO).
• Proficiency in Arabic and in English.
• Experience conducting online studies is desirable, though not required.

(Noting we likely have budget constraints).

How to apply

10. PROPOSAL APPLICATION SUBMISSION

Interested organizations should submit proposals with the following documents:

• Cover letter including the consultant’s/firm’s suitability for the assignment and current contact information.
• CV including contact details of two references. If the lead consultant is planning to recruit enumerators in Jordan to support the data collection, data entry, and dataset development, please state this in your application and if possible, provide details (cost, expertise, expected role of each consultant, etc.).
• Include one or two samples of previous evaluation work.
• A detailed response to the ToRs, including: a technical proposal addressing the purpose and objectives of the assignment; plans for in-person data collection as well as COVID-19 contingency plans in the event of new restrictions, and key sampling criteria.
• Financial proposal with a detailed budget breakdowns for each methodology based on expected daily rates and an initial work plan.
• Initial drafts of the proposed workplans for each methodology in the form of a Gantt chart.

Both the financial and technical proposals must be submitted to Logistics Coordinator Fakhri Al Syouri at [email protected] **no later than the 18th of November 2022.**Please also copy Heba Al Qaisi, Global Monitoring, Evaluation, and Learning Officer, at [email protected]

All offers should be valid until at least 28th February 2023. If you are applying as an international organisation, please ensure that you include all travel costs in your financial offer. All enumerator and training costs must also be included.

The selected candidate must adhere to both Right To Play and governmental standards regarding health and safety in light of the COVID-19 crisis, including all required social distancing measures.

Right To Play is a child-centered organization. Our recruitment and selection procedures reflect our commitment to the safety and protection of children in our programs.

Note: Final candidates will be contacted by RTP and only the ones who make it to the final phase (Interview phase) of the recruitment process will be notified whether they have been selected or not.