Consultancy Services for Digital EORE Tools Evaluation

1. Who are we?

The Danish Refugee Council (DRC) is a leading international humanitarian displacement organization, supporting refugees and internally displaced persons during displacement, in exile, upon return, or when settling and integrating into a new place. DRC provides protection and life-saving humanitarian assistance, supports displaced persons in becoming self-reliant and included in hosting societies – and works with communities, civil society, and responsible authorities to promote the protection of rights and peaceful coexistence. The work of Humanitarian Disarmament and Peacebuilding (HDP) seeks to address the causes of insecurity through both a weapons-related and people-related approach. HDP works with conflict management, security governance, weapons and ammunition management, and Mine Action (MA). DRC currently has 9,000 staff and 7,500 volunteers with programs in more than 40 countries worldwide.

DRC has been present in Ukraine since the late 1990s with a continuous presence since 2014. Since the escalation of the conflict in 2022, DRC is undergoing a considerable scale-up, opening offices in Lviv, Kyiv, Chernihiv, Sumy, Dnipro, Kharkiv, and Mykolaiv. In Ukraine, DRC currently implements Shelter, Protection (including Legal Assistance), Non-Food Items, Cash, Economic Recovery, as well as HDP sectors of work. DRC is the only organization in Ukraine that implements all four admissible pillars of MA – clearance (including Non-Technical Survey), Explosive Ordnance Risk Education (EORE), Victim Assistance, Advocacy, as well as the cross-cutting activity of capacity building of national MA duty-bearers.

1. Purpose of the consultancy

DRC based in Ukraine seeks proposals from a consultant to conduct an evaluation of the effectiveness of its digital EORE tools when compared to traditional EORE direct sessions. DRC digital EORE tools in Ukraine include 1.) three EORE online courses for children, adolescents, and adults; 2.) EORE Chatbot on Viber and Telegram; 3.) EORE coloring book with Augmented Reality application for children; and 4.) digital EORE mass media campaign.

The following six objectives are applicable to this comparative evaluation:

• Evaluation objective 1 – Assessing knowledge and awareness: Measure the extent to which the target audience has gained knowledge and awareness about Explosive Ordnance (EO) risks and safe behavior;
• Evaluation objective 2 – Evaluating attitude and perception change: Evaluate whether the digital EORE tools have influenced the target audience’s attitudes, beliefs, and perceptions towards EO risks and their potential consequences;
• Evaluation objective 3 – Analysing behavior change: Determine whether the digital EORE tools have effectively motivated the target audience to adopt safe behaviors when encountering EO, such as avoiding, reporting, or seeking professional assistance.
• Evaluation objective 4 – Assessing social norms: Explore the impact of the digital EORE tools on social norms within the target audience’s community or social networks. Evaluate whether the tools have contributed to the establishment of safe practices as social norms and whether they have reduced the acceptance or tolerance of EO risks;
• Evaluation objective 5 – Evaluating risk perception and risk reduction strategies: Assess the target audience’s perception of risk associated with EO and their understanding and application of risk reduction strategies;
• Evaluation objective 6 – Investigating emotional connection: Explore the emotional impact of the digital EORE tools on the target audience. Evaluate whether the tools have effectively conveyed the potential dangers of EO and fostered a sense of urgency or concern among the audience.

NB: DRC matter specialists will provide technical training, guidance, and approval on all products developed to ensure technical validity, relevance, and quality.

1. Background

The armed conflict that restarted in 2022 in Ukraine led to a sharp increase in the presence of EO, affecting 15 out of 25 Oblasts. 176 different types of EO, including Anti-Personnel Mines and cluster submunitions, have already been identified. As per ACLED data, 37,777 conflict events have been recorded since February 2022, often in populated areas. By March 2023, the UN has reported 21,793 civilian casualties because of the escalation (8,173 killed and 13,620 injured), with adult men representing most victims. At this stage, there is a lack of clarity on the exact scope, scale, and location of explosive contamination; HNO 2023 reports 25% of all of Ukraine being exposed to armed conflict. According to the 2022 EORE Rapid Needs Assessment on average 49% of the respondents in affected Oblasts have personally seen an EO. The Mine Action Sub-Cluster (MASC) evaluated that the ubiquity of EO currently puts 14.6 million people in peril.

DRC implements interventions that tackle the dangers and consequences of EO and other explosive weapons with, inter alia, activities such as EORE. The provision of EORE is a quintessential response to the prevention of EO accidents – both for people’s safety, humanitarian assistance, and the resumption of development. During recent years, both in Ukraine and globally, the usage of digital EORE has been on the rise as well. However, very little research has been conducted on evaluating the effectiveness of digital EORE initiatives compared to traditional EORE tools used by Mine Action operators in terms of promoting knowledge, changing attitudes, influencing behavior related to EO risks, and so on.

1. The objective of the consultancy

The consultant will be required to:

• Conduct an evaluation and produce an evaluation report, compliant with DRC Design Guide. As a minimum, this report will include an Executive Summary of the main findings (maximum 2 pages); background information on the context; description of the evaluation aim and objectives; a detailed description of methodology and data collection methods (including their limitations); findings of the evaluation; analysis of findings through SBC theory; conclusions and recommendations; lessons learned on using digital EORE tools in Ukraine; and annexed questionnaires, interview questions, and other methodological tools;
• Collect, clean, analyze, and hand over the entire raw evaluation data plus the cleaned database of the evaluation;
• Develop and present the evaluation via a PowerPoint presentation, including infographics of evaluation results, compliant with provided Editorial Manual.

NB: all deliverables are to be provided in English, following professional proofreading services. DRC Design Guide and DRC Editorial Manual will be provided to the successful applicant.

1. Scope of work and methodology

DRC is looking for a consultancy to conduct an evaluation in select conflict-affected Oblasts of Ukraine to:

• Develop a mixed-method survey grounded in Social and Behaviour Change (SBC) theory;
• Design the research methodology;
• Evaluate the differences between digital EORE tools and traditional EORE sessions on topics of increase of knowledge and awareness; attitude and perception change; behavior change; social norms; risk perception and risk reduction strategies; emotional connection;
• Provide recommendations for the improvement of DRC EORE digital tools.

The evaluation is expected to focus on Kyiv, Chernihiv, and Zhytomyr Oblasts – currently deemed safe for operations by DRC Safety Team in Ukraine (ceteris paribus). Ideally, the evaluation will differentiate between settlements never occupied and those previously occupied settlements, as well as between rural and urban areas. For sufficient quality of evaluation results, DRC recommends a minimum of one Focus Group Discussion (FGD) (with eight to 12 people) per activity per Oblast as well as at least two KIIs per activity per Oblast on top of quantitative pre- and post-testing conducted on all receiving digital EORE education. For comparative results, DRC recommends a control group to also be considered which receives standard EORE direct sessions as opposed to the groups that receive digital EORE education. A gender and age balance should be sought, as well as the inclusion of persons with disabilities to the extent possible.

The consultant will be required to prepare a detailed methodology—adapted to children, teenagers, and adults—as well as a work plan indicating how the objectives of the evaluation will be achieved, and the support required from DRC.

NB: Depending on access and security situation, the methodology might be shifted from mixed to more qualitative. The modifications shall be agreed with the technical supervisors at DRC.

1. Deliverables

The Consultant will submit the deliverables stated in the Annex C Terms of Reference

1. Duration, timeline, and payment

The total expected duration to complete the assignment will be no more than 31 working days. The consultant shall be prepared to complete the assignment no later than 31 August 2023.

1. The proposed composition of the team

• Consultant

1. Eligibility, qualification, and experience required

• Qualification:

  • At least a master’s degree in international development or related social sciences.

• Experience:

  • A minimum of 5 years of research and/or academic work with experience in conducting (applied) research in difficult contexts;
  • Previous experience in conducting similar evaluations.

• Skills and knowledge:

  • A solid understanding of Ukrainian or similar context.

• Language requirements:

  • Mandatory: Written and spoken fluency in English.
  • Preferred: Working knowledge of Ukrainian and/or Russian.

1. Technical supervision

The selected consultant will work under the supervision of:

• DRC HDP Programme Manager, Nick Vovk, [email protected].
• DRC EORE Specialist, Olena Siver, [email protected].

1. Location and support

DRC Ukraine offices in Kyiv and Chernihiv. DRC will allocate a table and a chair in every office for the consultant – the use of conference rooms will also be possible with advance notice. The Consultant will provide her/his own computer and mobile telephone.

1. Travel

Since the evaluation will be conducted in three northern Oblasts, extensive traveling will not be required. A tourist visa for Ukraine will last for three months; a longer (work) visa cannot be provided for the consultant.

While the consultant will have to budget for and book the international flight (with Rzeszow, Poland, being the closest international airport), DRC Ukraine will support all in-country travel and accommodations.

1. Submission process

Proposals should include:

• Curriculum vitae detailing previous experience with evaluations within humanitarian assistance projects/contexts, including copies of academic certifications;
• Samples of previous work accomplished;
• Technical proposal including the survey design and methodology, data collection and analysis, activities (e.g., development of tools), as well as confirmed timeline considering contextual limitations;
• Financial proposal for the survey. All costs related to the survey without exceptions should be figured into the financial plan of the proposal, including (but not limited to) consultancy fees, travel, accommodation, interpreter (if needed), data entry, logistics, etc. Please, note that no per diem will be paid to the engaged personnel on top of the overall fee;
• Three reference letters of which two should be from a previous surveying experience

For information regarding the bid submission process and tender evaluation criteria, please check RFP Invitation Letter.