Terre des hommes
TERMS OF REFERENCE FOR EXTERNAL EVALUATION
Project: REVIS – REsponding to VIolence against children in Schools and adjacent settings
Terre des hommes is seeking a qualified evaluator or evaluator team to conduct a final project evaluation on the REVIS project (2021-2022).
- Presentation of Terre des hommes
Terre des hommes (Tdh) is the leading Swiss organization for children’s aid. Through our health, protection and emergency relief programmes, we provide assistance to over four million deprived children and their families in more than 40 countries each year through innovative and sustainable solutions.
In Europe, Tdh helps children who are affected by migration, who come in contact with the law as a victim, witness or offender, and who need general protection including from the risk of abuse, trafficking or exploitation. Tdh’s child protection teams work to develop and strengthen national child protection systems and build professionals capacities who work for children. Tdh mobilises communities and educate families to create safe and nurturing environments for children and youth. We also empower children and increase their resilience through self-led art initiatives and mental health and psycho-social support. Tdh concentrates its work in Albania, Kosovo, Moldova, Romania, Ukraine, Hungary, and Greece, while together with local partners, it also reaches children in 14 other European countries. In 2021, more than 68,820 children and their communities benefitted from Tdhs support through 81 projects implemented in Europe. For more background about Tdh globally and in Europe, visit our global website and our European website.
2. Context and the project
Tdh Hungary, together with four civil society partner organizations, is implementing an EU-funded project entitled: “REVIS: Responding to peer violence among children in schools and adjacent settings”. The project takes place in four countries: Bulgaria (by SAPI), Croatia (by Brave Phone), Romania (by Tdh Romania) and Serbia (by the Center for Youth Integration), with project coordination work out of Hungary (by Tdh Hungary).
The overall goals of the project are to:
- to prevent, combat and respond to peer violence among children through transforming social and gender norms and behaviours (leading to violence);
- and to create space for children to challenge social and gender norms that lead to peer violence; and to become social influencers to develop their own solutions to the issue.
The specific expected results of the project are as follows:
1) Increased awareness and capacity of relevant professionals (pedagogical staff, social workers, youth clubs’ staff etc.) & children in challenging harmful social norms and behaviours and promoting positive ones
2) Increased capacity of children in identifying opportunities for intervention and challenging bystander attitude in case of peer violence
3) Children act as social influencers on preventing, combating and responding to peer violence through self-developed solutions
4) Increased access to methodologies and best practices on how to effectively address harmful social norms and to empower children in developing their own solutions in preventing, combating and responding to peer violence
5) Increased access of parents to guidance information on how to detect, deal with and respond to episodes of peer violence and how harmful social norms and behaviours lead to peer violence.
The project works with 40 schools/educational settings (10/country) and trained a total of 8 trainers (2/country) and 80 facilitators (20 teachers/country) in the target schools via a blended training curriculum developed on shifting norms around violence in schools. The trained facilitators conduct participatory activities with more than 1.600 children and young people (400/country) and awareness raising activities for other school professionals to help prevent, combat and respond to peer violence in and around schools and empower children in addressing the issue through their own solutions. Additional details about the project and its outputs are available on the dedicated project website here: REVIS | ChildHub -Child Protection Hub
Project timeframe: 15 January 2021 –14 January 2023.
Of note:the REVIS partnership has requested a 2-month no-cost extension (NCE) from the donor and is currently waiting for the donor’s reply/approval. If approved, the new project end date will be 14 March 2023, and in that case the timeline of the final evaluation is expected to be adjusted accordingly.
Monitoring and Evaluation: The project’s Monitoring and Evaluation plan includes output- and outcome-level indicators tracked at regular intervals and/or at the end of the project (as relevant). There is a quantitative baseline (survey), qualitative mid-term (FGDs) and quantitative and qualitative endline assessment (survey+FGDs) to measure changes in knowledge, attitude, practices and behaviours relating to peer violence among children and professionals involved in the REVIS activities in Bulgaria, Croatia, Romania and Serbia. So far, the baseline and mid-term assessments have been completed. At the mid-term of the project an internal mid-term review was also carried out looking at project progress and performance using OECD DAC and CHS criteria with a formative purpose to conclude internal learning and inform onward steering of the project. Regular monitoring is ongoing on the achievement of deliverables, based on the monitoring and evaluation plan.
3. Objective of the evaluation
The final evaluation is a mandatory deliverable for the donor European Commission. In addition, as per Tdh’s Minimum Requirements for M&E, every project that is multi-country and multi-year must undergo a final external evaluation to ensure a robust and neutral external assessment and judgement on project performance.
The objectives of the evaluation are three-fold:
- Accountability and credibility: to the donor, to the project team and the corresponding organizations, and ultimately to the beneficiaries (professionals and children). It will also ensure transparency and openness about the performance and quality of the project.
- Organizational learning and knowledge: to conclude all important lessons learnt from the way the project was set up, implemented and managed; and how it has succeeded or, potentially, not succeeded to achieve all its intended (or unintended) results.
- Informing operational decisions: Resulting from the lessons learnt, the evaluation will help to reveal what the project team and the corresponding organizations can take further and/or may need to change in future programming.
4. Scope of the evaluation
The final (external) evaluation will cover the entire REVIS project, including all partner organizations. Geographically, it will cover the four local implementation countries: Bulgaria, Croatia, Romania, Serbia and project coordination work out of Hungary.
The evaluation will be conducted during the final two and a half (2.5) months of the project; thus it will focus on the project activities carried out and results achieved up to the time of the start of data collection. (As mentioned earlier, if the no-cost extension is approved by the donor, the new project end date will be 14 March 2023, and in that case the timeline of the final evaluation is expected to be adjusted accordingly.)
Limitations:
- As the timeframe for the conduct of the evaluation is quite tight, and project partners are still focusing on the final project activities, their involvement in data collection (for consultation) must be well planned to avoid conflicting schedules.
- While some data collection may happen in person – in particular in case of children and young people – via travel to the four local implementation countries (Bulgaria, Croatia, Romania, Serbia), some other data (e.g. from professionals) may be collected online only. Given that online interviews may have different dynamics than in-person ones, data collection methodologies will need to consider this and be designed accordingly, working in any format.
5. Intended users of the evaluation
The results of the final evaluation will be used by the:
- REVIS project team: to reflect on and learn from how the project has performed and how it can inform the follow-on future initiatives.
- The broader Child Protection programme team within Tdh: to learn from the findings, build institutional knowledge, and assess how findings can generally inform future programming in this field.
- Senior management of all implementing organizations as well as Tdh HQ: to assess how project learning can inform future programmatic portfolio decisions and strategy-making, and what effective and corrective actions might need to be taken going forward in similar or continuation projects.
- the donor EC DG Justice and Consumers: to assess how its support and funds have contributed to delivering project objectives and positive changes for children at risk of or victims of peer violence.
6. Evaluation criteria and questions
The evaluation should respond to the following questions (combining selected OECD DAC and Core Humanitarian Standard (CHS) evaluation criteria):
Effectiveness:
At the level of teachers/facilitators:
- How effectively could the project team motivate, engage, and empower target teachers/facilitators, particularly at the first period of the project?
- How effectively has the community of practice among target teachers/facilitators worked in terms of peer mentoring between the 2 pilot and 8 follow-on schools in each country AND in terms of knowledge cascading between teachers involved and not involved in the REVIS project?
At the level of children:
- Has the participatory action research element of the project contributed to empowering children and building their agency to find solutions to their own problems and become influencers among their peers?
- Has the prototype design element of the project contributed to empowering children and building their agency to find solutions to their own problems and become influencers among their peers?
- Has the child-led social media campaign element of the project contributed to empowering children and building their agency to find solutions to their own problems and become influencers among their peers?
Relevance:
- How relevant were the participatory action research and the prototype design element of the project to address social and gender norms leading to or contributing to peer violence?
Child participation method:
- To what extent have project activities been fully child-led (instead of adult-led)? Are there any lessons learnt to conclude from this for the future?
Diversity and inclusion:
- Were project activities (activities with children, trainings, assessments) and project outputs (info guide for parents, curriculum etc.) sensitive to diversity and inclusion considerations (e.g. age, gender, disability, socio-economic background and other diversities)?
Sustainability/replicability:
- How sustainable are the project results once the project and donor funding ends?
- To what extent could the project concept/model be replicable in other classes, schools/educational settings, or even other age groups based on how effective it has been so far? What do we know from this project’s experience?
7. Methodology
Interested applicants are invited to propose their methodology for the conduct of the evaluation. Ideally, a mixed-method approach will be followed balancing suitable quantitative and qualitative methods to draw on different sources and triangulate information. Existing M&E data and collected quantitative and qualitative information (from the quantitative baseline, qualitative mid-term, and quantitative and qualitative endline assessments, monitoring visits, monthly monitoring calls, mid-term project review, feedback forms after the training of trainers, facilitators and sessions with children) will also be available for the evaluator.
The REVIS project team members should be closely involved in the data collection (for consultation). They typically include one regional or local coordinator per organization. Tdh and partners will be able to facilitate contact with the school professionals and children who will also be involved for data collection. Some of the school professionals have a basic level of English, while children typically only speak their local language.
Generally, a participatory approach should be followed throughout the evaluation. Due to the limitations described above, any methodology proposed should also be feasible in an online format.
The evaluation methodology should keep in mind the following underlying principles:
- equality and non-discrimination of any individual involved in the data collection and consultation process;
- participation and inclusion of different diversity groups to the extent possible, while ensuring that participation and inclusion does not cause any harm or distress to the individual;
- transparency and accountability throughout the evaluation process towards all organizations and individuals involved;
- empowerment of marginalised children whenever children are involved at any point of the process e.g. communication of evaluation findings;
- confidentiality must be preserved and names and personally identifiable information should not be collected or, if they are necessary, they should be duly coded and encrypted;
- informed consent and assent: should be sought from all participants, including providing them with information on the objective of the evaluation and how data will be used as well as ensuring the opportunity to ask and confirm questions before agreeing to participate;
- child safety: whenever children are involved, the evaluator(s) and anyone affiliated with the evaluation (assistants, translators) must sign the Tdh Child Safeguarding Policy and Code of Conduct and adhere to its principles.
8. Deliverables
The evaluator will be required to submit the following deliverables (all in electronic version):
- Inception report – in English that includes the following: results of the initial desk review, the proposed methodology and data collection tools, an evaluation/analysis plan (in Tdh’s template to be shared after contracting), and overall work plan for the assignment.
- Draft final report – in English, with the preliminary results of the analysis for the project team’s review and feedback. Tdh will share its template after contracting. The final report shall include an executive summary, narrative report, summary table with main conclusions and recommendations and lessons learned. The report shall also contain all usual annexes such as terms of reference of the evaluation, data collection tools, consent forms and protocols, tables, graphics, list of desk review references, survey data, interview, focus group etc. transcripts, and a PPT presentation of the findings (see below).
- Presentation of the draft report and preliminary findings – in English in a dedicated meeting, attended by the evaluator and the project team.
- Final report – in English, with full and final analysis results and incorporation of the feedback received on the draft report. The final report will be prepared in Tdh’s template (to be shared after contracting).
The evaluator is also expected to be available for regular and brief check-in calls, as necessary, throughout the assignment to share updates on the progress.
9. Chronogram
The foreseen timeframe and number of working days for the assignment are indicated in the table. The final schedule and number of days will be mutually agreed between Tdh and the selected evaluator, also taking into consideration the proposal in the Technical Offer.
The applicants are invited to develop their detailed work plan for the phase of the assignment period and can break these steps down to further sub-steps to illustrate the process they would plan to follow.
Steps/phases & deliverables
Timeframe/deadline
(tentative)
Starting date of contract/assignment
2 Nov 2022
(tentative)
Desk review of available documents
5 – 7 days
Development of data collection methods and tools
Inception report
Data collection phase
If online only: 8 – 10 days
If includes field travel: 16-20 days
Data analysis and drafting of the final report
5 days
Presentation of the draft report and preliminary findings
1 day
Finalization of the final report
3 days
Closing date of contract, final administration and payment (upon approval of the report by Tdh)
By 14 Jan 2023
10. Roles and responsibilities
Tdh will designate an Evaluation Manager (the regional project coordinator) who will be the main focal point vis-à-vis the evaluator. The key roles and responsibilities of the Tdh Evaluation Manager will include:
- Managing the administrative tasks of the process (from advertising to contract closure) and internal liaison within the project team.
- Sharing all key project documents for the desk review and facilitating access (together with local coordinators) to key individuals (professionals and children) selected for data collection.
- Consolidating and sharing team feedback on the deliverables (reports). Communication will mainly take place via e-mail, and online chat and conferencing platforms (MsTeams, Zoom etc.) or in-person in the countries in case travel is proposed/takes place (with Covid-measures fully considered).
- Acting as a focal point for the evaluator in case of any queries and sharing updates if any. Communication will generally take place in English (no separate translation services will be provided by Tdh and the broader project team).
From the side of Tdh, the evaluation process will also be overseen by a Steering Committee (as per Tdh’s Minimum Requirements for Monitoring and Evaluation), composed of the Tdh Evaluation Manager, the Tdh Regional Q&A Advisor, the Tdh Romania M&E Officer, and the Tdh Deputy Head of Regional Office. Committee meetings will be held internally as per need, where committee members will follow the progress of the assignment. They will also provide review, feedback and approval of the deliverables (all reports).
Roles and responsibilities of the evaluator:
- To deliver the assignment as per the signed contract, Terms of Reference and the Technical and Financial Offer and relevant annexes, including Tdh policies where applicable (e.g. those related to Data Protection and Child Safeguarding). To submit all required deliverables as per the agreed deadlines.
- To ensure that all the underlying principles (described in the methodology section) are respected throughout the evaluation.
- To ensure that the Evaluation Manager is regularly informed in case of any questions and issues (particularly, delays and blockages) that may emerge during the assignment and may require mutual discussion/planning.
- To be available for quick updates, whenever requested, on the progress of the assignment.
- To generally ensure the confidentiality of the process as well as the information and documents received from the Evaluation Manager and the people involved at any stage in the evaluation.
- To immediately inform the Evaluation Manager in case of any risk of not being able to comply with any of the above points.
11. Profile of the evaluator (or evaluator team): qualifications and experience
Tdh will accept applications from both individuals, teams, academic entities, NGOs and companies as long as they are officially registered with a tax number (i.e. are able to issue an invoice) and have evaluation, research, impact study or similar and/or consultancy as their object of activity, which can be proven by legal documentation. In case the evaluation is conducted by a team, the team should ideally be diverse and gender-balanced.
Required experience and skills:
- At least 3 quality evaluations/impact studies prepared for 3 different projects. Examples can be annexed unless confidential. Previous experience in conducting evaluations of projects in the area of child protection, school violence, peer violence, gender and social norms and similar thematic is a significant asset.
- Proven skills in using quantitative and quality data collection and analysis methods.
- Strong understanding of diversity and inclusion incl. age, gender and diversity sensitivity aspects.
- Strong writing skills and ability to convey technical and complex information in a structured, logical, clear and concise manner for different audiences.
- Full written and oral proficiency in English for production of quality reports. Knowledge of the local language in the implementing countries is an asset, given that certain project beneficiaries do not interact easily in English.
- Strong computer proficiency to prepare quality written reports with clean data visualization and presentation of findings.
- Experience in the countries of implementation (i.e. Bulgaria, Croatia, Romania, Serbia) is an asset.
- Master’s degree (or higher) in social sciences, economics, public policy, law, research and evaluation, management or other related field(s) from an accredited university.
- Official registration as an individual consultant or as an organization, company, think-thank, academia entity etc. with social research (or similar) being under the registered scope of activities.
- Capacity to issue invoices with tax registration in Europe (UK and Switzerland included).
The evaluator(s) must not have been involved in the design, implementation or monitoring of the REVIS project at any stage.
12. Budget
The proposed budget for the evaluation will be provided by the consultant in the Financial Offer. Upon the review of the Financial and Technical Offers, Tdh may choose to negotiate the final price with the preferred evaluator in light of the Financial Offer and Tdh’s available budget (incl. VAT – 27% in Hungary – and any other applicable taxes). Tdh may also go with an offer different from this (incl. much lower) depending on the combination of the quality and competitiveness of the Technical and Financial offers.
As noted earlier, for this assignment, the evaluator may wish to propose field travel and data collection. All related travel and logistical costs, just as translation costs should be clearly specified in the financial offer, separated from the daily fee and other non-travel and non-translation related costs.
How to apply
13. Application procedures
Questions relating to the assignment, the project or the application process can be submitted to the Evaluation Manager, Marta Bene at [email protected]. Answers that may concern all applicants will be shared with everyone as deemed necessary.
Interested applicants shall submit their application via e-mail to the following e-mail address: [email protected]. The e-mail should have the subject-line: REVIS final evaluation – Applicant’s name. The deadline for submission is: 26 October 2022 (23:59 CEST).
The application package shall include:
- Technical Offer including approach, workplan with timeline – max. 10 pages;
- Financial Offer – with a detailed line item budget, indicating costs in EUR. The indicated prices should be net amounts, VAT is payable by Tdh.
- Up-to-date CV of the evaluator(s) – max. 3 pages/CV;
- Example of previous work carried out (if any, unless confidential or if it can be anonymized);
- Contacts of 3 professional references;
- Confirmation of legal registration (e.g. tax number, EU VAT number) of the applicant for providing evaluation services and of the ability to issue an invoice.
- Declaration of impartiality (in the applicant’s format).
Applications – incl. all annexes – must be in English. Other languages will not be accepted.
Applications must be submitted as a single application package. Documents sent separately will not be accepted. Tdh will confirm the receipt of each application within 2 working days.
The Technical and Financial Offer must be signed on the first and last page and stamped (if an official stamp is available). Both offers shall contain the name and contacts of the evaluator (or the representative if it is a team, company or similar).
Applications will be assessed based on the following criteria:
Assessment criteria
Max. points
Technical Offer – based on: 1) the understanding of the nature and objectives of the assignment; 2) understanding of the ToR; 3) data collection methodology proposed; 4) specific schedule, plan and approach proposed for the data collection.
40 points
Financial Offer – based on: 1) price-value ratio and demonstration of maximum value for money; 2) completeness of information; 3) consistency with the technical offer and approach.
40 points
Profile and experience of the applicant – based on: 1) submitted CV(s); 2) submitted previous work.
20 points
TOTAL
100 points
The applications will be evaluated by an Evaluation Committee consisting of the Tdh Evaluation Manager, the Tdh Regional Quality and Accountability Advisor and the Tdh Romania Monitoring and Evaluation Officer. The application scoring the highest will be selected for contracting. Prior to contracting, Tdh may conduct a short interview or coordination call with the pre-selected applicant to clarify certain details, experience, motivation or the budget.
All applicants will be informed of the outcome of the selection process by e-mail after the contract is concluded with the selected applicant.
Tdh will provide a detailed briefing on its various policies and principles related to ethical considerations to the selected consultant. All Tdh consultants and contractors (just as staff members, interns and volunteers) are required to sign Tdh’s Child Safeguarding Policy and Code of Conduct and adhere to its principles upon entering into contract.
14. Reference documents and annexes
The selected evaluator will be provided with the following documents upon contract start:
- original project proposal and main annexes
- mid-term project report submitted to the donor
- project follow-up tool (project management Excel tool used for progress tracking)
- national and regional baseline assessment reports
- national mid-term assessment reports
- internal mid-term project review – summary report
- minutes of project coordination calls
- events’ reports when available
Please find the full Terms of Reference in pdf format here:
Terms of Reference for External Evaluation of the REVIS Project