Developmental Evaluation of UNICEF’s New Internal Communication and Staff Engagement Collaboration Structure

CONSULTANCY TITLE

Developmental Evaluation of UNICEF’s New Internal Communication and Staff Engagement Collaboration Structure

RECRUITING OFFICE

Evaluation Office

LOCATION

Home-based

TRAVEL

The consultant (one) will work mainly remotely; however, their presence in UNICEF headquarters in New York is expected to be required for at least five days every two months for interactions with stakeholders (such as consultations or workshops)

LANGUAGE(S) REQUIRED

English is required, and knowledge of other UN languages is an asset.

DURATION OF CONTRACT

100 days (estimated) over a period of 11.5 months, starting in December 2022

Introduction

In the UNICEF Strategic Plan (SP) for 2022-20251, UNICEF recognises Internal Communication and Staff Engagement (ICSE) as a vital organizational enabler in executing its SP and achieving the organizational goals and targets laid out in it. This represents a strategic shift in the relevance and criticality of UNICEF’s ICSE function, with a new, globally networked, organization-wide ICSE collaboration structure piloted from September 2022 to December 2023 to deliver on the SP.

In operationalising this new SP enabler, UNICEF seeks to conduct a dynamic, learning-oriented, developmental evaluation (DE). These Terms of Reference (ToR) describe the main purpose, primary users, objectives, tentative learning matrix, proposed methodological approach, associated risks, and required qualifications for an individual consultant to undertake this evaluative exercise. The evaluation will be conducted between December 2022 and December 2023, covering the pilot phase of implementing the new ICSE networked collaboration structure to deliver on the SP.

Background and Rationale

In 2021, UNICEF senior management recommended centralising the ICSE function in the Division of Communication (DOC, now known as Global Communication and Advocacy, GCA) and requested relevant divisions and the Global Staff Association to strengthen the ICSE function, using available resources. This proposal built on the findings of the 2019 Independent Task Force (ITF) report on Workplace Gender- Discrimination, Sexual Harassment, Harassment and Abuse of Authority2, which highlighted that UNICEF’s default practice of internal communications, rather than serving as a pillar for healthy organizational culture, was risk-averse, top-down, and one-way. In response, UNICEF engaged in a change management process on internal communication that sought to strengthen UNICEF’s internal communication so that it can support change management and knowledge sharing and ultimately increase staff satisfaction and motivation.

The ICSE SP enabler vision and first-ever Global Internal Communication and Staff Engagement Strategy for 2020-2030 outline UNICEF’s goals and approach to strategic internal communications.

These are:

1. ICSE GOAL 1: Improvement of Values-Based Behaviours – increase staff awareness and practice of UNICEF’s Core Values to help ensure all behaviours of staff are fully aligned with them;
2. ICSE GOAL 2: Acceleration of Organizational Results – increase staff awareness and engagement on how to accelerate the achievement of UNICEF’s global targets in all SP Goal Areas; and
3. ICSE GOAL 3: Synergized Digital Internal Communication Ecosystem – improve ICSE, engagement, and collaboration among all staff globally through user-friendly and integrated digital ICSE sites, channels and tools.

In the Strategic Plan 2022-2025, ICSE became, for the first time, a vital organizational enabler known as “Strategic internal communication and staff engagement”. This enabler aims to foster inclusive, transparent, constructive, and authentic exchanges between staff at all levels of the organization for enhanced awareness, agile collaboration and connectedness – advancing UNICEF’s programmatic priorities and co-creating a values- based organizational culture”.3 The inclusion of ICSE in the Strategic Plan represents a remarkable shift in UNICEF to raise the relevance and criticality of the ICSE function to allow for greater awareness, transparency, collaboration and connectivity across the organization.

To deliver on these goals, a new way of collaborating across the organization has been proposed, using existing communication resources (human and financial), for implementation through a pilot approach so that lessons can be learned and incorporated into a full-roll out of ICSE as a Strategic Plan enabler. The pilot phase will take place from September 2022 to December 2023.

The recent Evaluability Assessment and Formative Evaluation of UNICEF Positioning to Achieve the Goals of the Strategic Plan 2022-2025 (under publication) noted that the efforts to improve ICSE as part of the new Strategic Plan are ongoing, and UNICEF has been working to build a bottom-up, consultative approach to internal communications in line with the 2020-2030 strategy. Despite this, the perception of ICSE as being top- down persists. It was also noted that UNICEF staff would benefit from shorter, sharper communications and more visual or multi-media content due to the large volumes of information being disseminated.

Evaluation rationale: As part of the roll-out of ICSE as a Strategic Plan enabler and to build on the early findings of the Evaluability Assessment and Formative Evaluation of the Strategic Plan, UNICEF seeks to conduct a credible, impartial, and independent evaluation led by its Evaluation Office (EO), that takes stock of the evolution of ICSE in UNICEF since the ITF report (2019), reviews progress in the implementation of the pilot, identifies good practices and lessons learned and provides recommendations for course correction in the pilot phase as well as for the full roll-out beyond 2023. Based on evidence-based insights, the evaluation will use a dynamic, developmental approach (DE) to support agile management and adaptation.

Purpose, Objectives and Scope

This evaluation aims to review the implementation of UNICEF’s new globally networked, organization-wide ICSE collaboration structure as part of the SP enabler to allow for course correction and generate learning to inform a full roll-out in 2024. Using a dynamic developmental evaluation (DE) approach, the evaluation will produce evidence to understand to what extent the new ICSE strategy is implemented, how the newly created ICSE pilot is performing, and why so, to support regular reflection, learning and revision of communication activities.

The evaluation will use a DE approach to identify early-on effective (and ineffective) aspects of the new ICSE strategy, collaboration structure and existing organizational frameworks, barriers, and bottlenecks to strategic ICSE across UNICEF. The DE will also be able to take stock and document the evolution of ICSE in UNICEF since the ITF report and assess the relevance and coherence of the new ICSE strategy and structure to meet the needs of the organization and strengthen internal communication staff engagement. A developmental evaluation, with its approach to assessing new initiatives by working in partnerships with decision-makers, will allow identification of how to implement the ICSE goals in practice across the organization over the coming months, at the global but especially at the regional and country level, and with other essential organizational functions where synergies should be realised (for instance, knowledge management and digital transformation). Although a relatively new approach in evaluation, DE has been used in the past to facilitate innovation and promote change in UNICEF through regular interaction and reflections on what is working, what is not working, and why.

The primary users of this evaluation include the Global ICSE Chief, the Team in the Global Communication and Advocacy division (GCA) directly involved in the implementation of the new vision and strategy for ICSE, the Global Programme and Budget Review (PBR) members who endorsed the pilot collaboration structure, all key individuals and stakeholders identified as key contributors in the collaboration structure, as well as the Executive Director. Other users include Regional Directors, Regional Management Teams, and country offices involved in the pilot roll-out.

The objectives of the evaluation will include the following:

• Assess the uptake and the implementation process of the new ICSE pilot to identify patterns that positively influence awareness, engagement, and connectedness among staff;
• Identify, early on, the barriers and bottlenecks in the implementation of the new ICSE pilot, as well as the enablers and facilitators to achieving the ICSE goals over the coming months;
• Identify good practices and lessons learned that can inform the way forward for a full roll-out, including looking at sustainability beyond the SP in terms of resources; and
• Review internal M&E mechanisms and tools to inform their roll-out and scale-up beyond 2023.

The learning framework will include, but not be limited to, the following areas of inquiry:

• Awareness: Staff awareness of content, tools, resources, and initiatives on UNICEF’s core values (ICSE Goal 1) and how to accelerate the achievement of UNICEF’s global targets and goals in the SP (ICSE Goal 2). This can be measured through regular surveys and digital metrics (intelligence data on the use of ICSE materials, etc.), and recommendations will help refine the current ICSE materials and activities to effectively communicate throughout the organization.
• Engagement: Staff engagement with ICSE materials and activities highlights how staff interact with information on UNICEF’s intranet (ICON) and other channels and portals and the impact of ICSE work. Evaluation methods will include digital metrics such as opens, click-throughs, comments, and attendance rates and qualitative in-depth interviews to collect feedback on critical events and understand their impact. Recommendations will provide solutions to improve further staff interaction with information and engagement with the organization.
• Connectedness: Staff are aware of, engaging and committed to connecting with other staff and advocating for ICSE efforts to strengthen UNICEF’s core values and accelerate the achievement of UNICEF’s Strategic Plan. The evaluation will thus try to measure staff contribution in promoting activities or acting as champions because of ICSE’s materials and activities. Recommendations will increase staff interaction and commitment to contribute to the realisation of the SP through better internal communication and more meaningful engagement.

As the roll-out of the new ICSE pilot unfolds over 2022-2023, the evaluation will gather data about its implementation to inform continuous learning and improvement and enhance the capacity of the Global ICSE Chief and Team and UNICEF’s Executive Leadership, to take corrective action. The evaluation will also generate new knowledge about ICSE processes and interactions in UNICEF that will occur during the implementation of the pilot between September 2022 and December 2023.

This evaluation will include multiple deep-dive case studies that will provide additional information on implementing the ICSE pilot across the organization and nurture the DE learning agenda.

Evaluation Approach and Methodology

The evaluation will require applying an adaptive and dynamic approach, using developmental evaluation (DE) and a methodology informed by the learning needs of UNICEF. This approach is expected to generate opportunities for promoting real-time learning and innovation. Below are some of the steps that will be followed in building the methodology of the DE. They include:

1. Orientation: In the inception phase of this evaluation, the Evaluation Office (EO) will work with the Global ICSE Chief and Team to consolidate its understanding of the new ICSE pilot and identify opportunities and challenges in the roll-out process. This will also be an opportunity to map out key stakeholders involved in the process and maintain engagement.
2. Developing a learning framework: A learning framework will be an essential component of this evaluation. Working in close collaboration with the ICSE Chief and Team, the Evaluation Office will map out potential areas for learning about the uptake and implementation of the new strategy, vision, structure, and tools, identify data and information that will be required to make decisions and articulate feedback mechanisms during the process.
3. Observing: The evaluation will carefully observe the unfolding of the ICSE strategy, new structure, materials, and activities to help identify leverage points, assess UNICEF efforts, and stay true to the core intent and principles of the evaluation. This may include: (a) key roll-out moments; (b) uptake;

(c) implementation barriers/bottlenecks and enablers/facilitators; and (d) other opportunities and challenges. This will also include an assessment of the new ICSE structure within UNICEF and its optimal positioning regarding efficiency and effectiveness.

1. Sense-making: Sense-making is mainly about making sense of the data collected through verification and triangulation. The evaluation will help UNICEF identify patterns, integrate new information about the pilot roll-out, and consider the implications of proposed solutions.
2. Intervening: The evaluation will actively help shape the work of the Global ICSE Chief and Team and the roll-out of the pilot by: (a) asking questions; (b) facilitating discussion; (c) sourcing or providing information; (d) modelling solutions to address barriers and bottlenecks; and (e) making propositions to strengthen the ICSE strategy and structure beyond 2023. The use of data visualisation throughout will be highly encouraged to convey critical messages.

Other elements that will be part of the evaluation approach include:

• Having a reference group who will be contacted regularly to sound out reactions and feedback.
• Asking the UNICEF network of staff to answer one or two questions periodically to test out new materials and activities.
• Monitoring and analysing comments on the internal discussion fora to track recurring themes.
• Creating a rolling programme of focus groups around the organization to gather opinions and views.
• Setting up a simple pulse survey to track awareness and engagement with a standard set of questions.
• Using internal social media channels to gather a live flow of unstructured data.

The evaluation process and approach are expected to follow the principles outlined in the UNICEF guidance and tools for evaluation4; the Norms and Standards for Evaluation; the Ethical Guidelines for Evaluation; and the Guidance on Integrating Human Rights and Gender Equality in Evaluations, published by the United Nations Evaluation Group (UNEG). They should also follow the UNICEF Procedure on Ethical Standards in Research, Evaluation, Data Collection and Analysis and may also find the UNICEF discussion paper on Ethical Considerations When Using Social Media for Evidence Generation of interest. The process will include the following mechanisms:

• Respecting gender and human rights principles throughout the evaluation process, including the protection of confidentiality, the protection of rights; the protection of dignity and welfare of people; and ensuring informed consent.
• Data validation will take place at all levels with participants’ consent
• Maximizing the degree of participation of stakeholders in the Evaluation itself wherever feasible and a commitment to using participatory approaches in carrying out consultations.
• Maximising data disaggregation by gender, disability, equity, and human rights-relevant factors will be necessary for ICSE’s success.
• Ensuring that outputs use disability-inclusive, gender-sensitive and human-rights language.
• Ensuring privacy protocols and compliance with all legal data management rules and considerations.
• Applying the principle of ‘do no harm’ into practice during the duration of the exercise.

Good professional practices not covered therein are also to be followed. Any sensitive issues or concerns should be raised with the Evaluation Specialist managing the DE as soon as they are identified.

The selected applicant will have access to critical internal data and perspectives. These must be held with the utmost confidentiality. Likewise, the willingness of internal and external stakeholders to speak to these issues critically will depend on the provision of absolute confidentiality. The selected applicant must sign the non- disclosure agreement (NDA), abide by UNICEF’s security protocols, and protect sensitive data.

Timeframe and Key Deliverables

The evaluation has a timeline of 11.5 months, from December 2022 to December 2023. Given the nature of the exercise, which has a strong focus on agile learning and adaptation, deliverables are expected to evolve throughout the implementation of the DE. They will be adjusted and negotiated on a rolling basis with the Evaluation Office (EO) in coordination with the Global ICSE Chief and Team.

Initially expected deliverables include the following:

• Completion of two orientation workshops for staff in headquarters (HQ), regional offices (RO) and country offices (CO).
• Submission of an after-action report regarding learnings, key findings, and results of orientation workshops, including opportunities and challenges in implementing the ICSE strategy and structure.
• Submission of a work plan and regular monthly reports.
• Weekly meetings with ICSE and the Evaluation Specialist in the EO managing the DE.
• Regular learning document following critical moments in implementing the ICSE pilot.
• Final evaluation report, to be prepared according to UNICEF evaluation reporting standards and guidance.

Management and Governance Arrangements

The DE will be conducted by engaging a consultant, or embedded evaluator (EE), to undertake the DE, and provide overall quality assurance and backstopping support as needed.

The EE will be responsible for the design and implementation of the DE and for ensuring its quality. The EE will be responsible for the following:

• Facilitate multiple orientation workshops for ICSE and other relevant stakeholders, which will help to collaboratively conceptualise and develop the DE design and approach (learning framework, methodology, work plans, reporting, etc.);
• Develop a living work plan for the evaluation, including an agreed-upon set of deliverables (which may change over time, depending on the needs of the programme);
• Provide quality assurance of all deliverables, including regular intermediate report logs that track details of the DE and document all events that occur, i.e., planned and unplanned and associated outcomes, and the final report of the evaluation;
• Facilitate regular meetings with senior management and key staff within UNICEF to collect data;
• Analyse the data collected and present digestible analyses (through compelling narrative and data visualisations) regularly to programme staff and partners through the data-review process and other participatory meetings;
• Track and communicate high-quality information to ICSE staff, senior management, and other relevant stakeholders for course correction; and
• Collaborate with an Evaluation Specialist in the Evaluation Office to regularly assess the performance of the DE, including the documentation of lessons learned about DE implementation.

Qualifications

The EE will report to an Evaluation Specialist in the Evaluation Office while working in coordination with the ICSE Chief and team. The EE will be provided with office space within UNICEF premises for meetings. The EE should have the following experience:

• Extensive evaluation experience (at least eight years) in international development with an excellent understanding of evaluation principles and methodologies, including capacity in an array of qualitative and quantitative evaluation methods and UNEG Norms and Standards;
• Experience with evaluating complex systems;
• Familiarity with DE, including skills such as respect for local stakeholders and partners, ethical research (maintains confidentiality and anonymity when required), flexibility, energy, humility, willingness to learn on the go, and ability to resolve conflicts;
• University degree (Master’s or higher) in international development, public policy or similar, with a robust M&E focus;
• Extensive technical knowledge, skills and expertise in evaluation design, concepts, and approaches and evaluating complexity, in particular;
• Strong analytical skills to support both qualitative and quantitative research;
• Facilitation skills, mainly design and execution of stakeholder consultations;
• Excellent oral and written communication and report writing skills in English;
• Keen attention to detail, especially related to documenting data and associated processes;
• Active listening, proactive learning, and time management skills;
• Leadership and strategic thinking skills;
• Expertise in organizational theory, development, and communications;
• Knowledge of the UN’s human rights, gender equality and equity agendas; and
• Readiness to be in a learning role.

Vacancies | UNICEF Careers