Monitoring, Evaluation, Accountability, Research and Learning (MEARL) Consultant/Firm

Project: Estidama++ Fund – Extension of Coverage and Formalization

Duty Station: Amman, Jordan

Duration: 01/09/2022 – 31/12/2022 (based in Amman, Jordan)

Reporting to: Sandra Silva, Social Protection Technical Specialist and Estidama++ Manager

I. Project and Assignment Background

On March 11, 2020, the World Health Organization (WHO) declared COVID-19 a pandemic. In a short time, this public health crisis has transformed into an important economic and labour market shock, leaving many workers in both developed and developing countries at risk. The impact of the crisis on employment is large, expected to be prolonged and diversified. In Jordan, this crisis builds on pre-existing decent work deficits, in particular those related to gaps in social protection coverage. A recent study on opportunities to extend social security coverage in Jordan has shown that over half of Jordan’s workforce is in the informal economy, and many lacked accesses to income support even before the COVID-19 crisis. Around half of all workers in Jordan were not registered with the Social Security Corporation (SSC) at the time of the crisis.

Estidama++ Fund – Extension of Coverage and Formalization (former EUESF) aims to turn the COVID-19 crisis into an opportunity and extend social security coverage to vulnerable informal workers, including refugees, while providing short-term income support to incentivize registration with SSC and compliance with contributions in the medium/long-term. Phase one of the project has started in June 2022 and will have a total duration of 18 months, the facility will focus on certain key priority sectors and typologies of workers. In Phase II (with prospective start up in mid-2023) and other subsequent phases of the project, it is expected that the overall scope can be extended to additional sectors and types of workers. The project is implemented in partnership with Social Security Corporation, with funding from The Kingdoms of Netherlands and Norway.

One of the key outputs of the project is to support the Social Security Corporation (SSC) in the administration of Estidama++, including on assessing the readiness and robustness of their internal monitoring systems so that they can function as primary monitoring data source, as well as on building/strengthening SSC’s capacity to comply with the project’s reporting requirements. Both in-house human and systems capacity is very limited and SSC has explicitly requested support to strengthen these areas in a manner that can be easily applicable and scaled up further once the project is finished.

To prepare for a prospective Phase II and the eventual development of a permanent subsidizing model, it will be critical to exhaustively document the lessons learned over the process of implementation, including experimentation of this extension of coverage model and its features, as well as conduct any relevant research to inform policy, strategic and programmatic decisions. Hence, the need to generate evidence and learning for the overall agenda of extension of coverage.

A comprehensive MEARL framework and plan will need to be developed to ensure learning is documented, necessary research is conducted, thorough monitoring of processes throughout implementation, rigorous evaluation of results in order to allow for further improvement and more sustainable outcomes.

II. Purpose

The main purpose of this consultancy is to assess the readiness and robustness of SSC’s internal monitoring systems, develop a comprehensive Monitoring, Evaluation, Accountability, Research, and Learning (MEARL) Framework and Plan, and build the M&E capacity of SSC.

Additionally, and as per the guidance of the Project Manager and the Regional M&E Officer, the MEARL Consultant/Firm will also be responsible for carrying out the mandatory project’s evaluability assessment.

By doing so, both the ILO project team and SSC will better equipped to ensure that donors’ funding is being well used and for the agreed purpose, while fostering trust and reliability amongst stakeholders, including donors, development partners, government, social partners, civil society, and beneficiaries overall.. This way, the project team will be able to effectively monitor implementation, track progress, identify learning opportunities and do the necessary adaptive corrections, and measure results at the end of the project.

III. Work assignment, Scope of Work, and Methodology

The assignment will focus on (1) developing a comprehensive Monitoring, Evaluation, Accountability, Research and Learning (MEARL) Framework and Plan for Phase I of Estidama++ Fund – Extension of Coverage and Formalization based on the project’s existing logical framework, which will inform the project’s evaluability assessment, (2) assessing and building the capacity of SSC and ILO relevant staff, (3) providing the necessary support to strengthen reporting capacity, (4) defining the scope, indicators and quality assurance processes of third-party monitoring, and (5) draft the ToR of the impact evaluation. This work will be performed under the guidance of the Social Protection Technical Specialist and Project Manager, and in close communication with the project core team and ROAS relevant specialists.

The consultant/firm will perform the following activities:

• MEARL Framework & Plan

The consultant/firm will develop a comprehensive MEARL framework and plan, including action and formative research to address knowledge gaps, test critical assumptions, validate the project’s theory of change, assess activity value and impact, as well as any MEARL interventions needed to improve SSC extension of coverage strategy and the adoption of sustainable measures throughout the project that can be useful and valid to absorb by SSC after the project end. This MEARL framework and plan shall be used (and updated as necessary) throughout the project life cycle. Once developed and approved, the consultant/firm will train the necessary staff in ILO and SSC to undertake the planned activities and follow up on the plan accordingly.

This exercise will include a desk review for all project documents including the initially appraised project document and the revised summary, the project’s logical framework and respective expected results, indicators, targets, means of verification, theory of change and other relevant documents. This includes a clear (re)definition of all the indicators with necessary disaggregation. Given that the project has multiple donors to whom the ILO needs to report to during the project implementation, the development of a comprehensive and innovative MEARL framework will require using evidence-based theories and methodologies.

The MEARL framework should include the recommended strategic approach, which in turn should reflect the project’s context, learning and operational needs, and donors’ requirements. The consultant/firm will validate the project results chain, risks, assumptions, indicators, baselines, targets, and means of verification and discuss any required/ recommended change with the project manager while developing the M&E framework & plan. The M&E plan is to also describe who collects the data, how, and when. Considering the reporting schedule, the consultant/firm will indicate the overall timeline in the M&E plan to retrospectively specify by when each monitoring & reporting activity is to be conducted/completed. The consultant/firm will specify required data collection tools that are to be standardized among implementing partners to the extent possible. The consultant/firm will also describe how the quality of data is assured. For visibility and accountability, the consultant/firm shall also think of how to effectively disseminate the collated data and findings with stakeholders, such as the Regional Office for Arab States (ROAS), donors, development partners and others, with data visualization. In addition, the consultant/firm shall develop a document with clear guidelines for the use of project staff and SSC on how to document the best practices and lessons learned.

In summary, the suggested outline of the MEARL framework and plan should include at least the following sections: (1) introduction and background, (2) purpose and intended audience, (3) desk review of existing systems and resources, (2) strategic approach, (2) updated theory of change with risks and assumptions, ideally represented by a clear diagram, (3) updated logical framework, clear indicators, their description, disaggregation, baseline, targets, and means of verification, (4) data collection methods and tools, (5) roles and responsibilities, (6) MEARL detailed plan (including activities and sub-activities and suggested timeline), (7) capacity building plan, (8) research agenda, and (9) quality assurance mechanisms. The MEARL plan should provide a very clear timeline of key monitoring, evaluation, and research tasks that will be conducted in order to collect evidence, as well as a comprehensive description of how the evidence will be used to support learning and adaptive management.

Based on the ILO evaluation policy, the project is required to undergo an evaluability assessment done within one year of start-up, ideally in the first six months. Under the supervision of the ILO Regional Monitoring and Evaluation Team, the consultant/firm will combine this mandatory evaluability review with the development of the MEARL Framework and Plan described above.

• Capacity assessment and building

As part of the process of developing the MEARL Framework & Plan, the consultant/firm will assess the monitoring capacity of the implementing partner (SSC). This includes but is not limited to assessing the staff capacity and the readiness of their systems and mechanisms to capture the required data. Besides an initial desk review and interviews with relevant ILO staff engaging directly with SSC, the methodology adopted by the consultant/firm to conduct this assessment should also include visits to SSC and interviews with their relevant staff. Based on that assessment and the identified capacity gaps, the consultant/firm will prepare and conduct capacity building initiatives designed to cater the specific needs. Besides the tailored capacity building sessions, the consultant/firm will also convene a workshop for the ILO, SSC and other relevant stakeholders to launch the MEARL framework and plan, address the importance of MEARL activities throughout the life cycle of the project and build the understanding of SSC on how to report on the indicators, identify means of verification, suggest corrective measures, and document best practices and successful stories that can later be reported using the narrative and financial reports. This is expected to help with developing a common understanding with the implementing partner regarding expected results, agreed targets, indicators and data collection for reporting. Throughout the consultancy duration, the consultant/firm is expected to assure the quality of data collected and provided by the implementing partner, including by having regular M&E meetings with the relevant staff and occasional field visits. At all stages, the consultant/firm will actively involve the project staff in the relevant activities, including the national officer recruited for that dedicated purpose. The consultant/firm will build the capacity of that specific staff, including on all aspects related to the MEARL framework and plan, relevant tools, guidelines and best practices concerning the results-based management approach, theory of change, logical framework, data collection, reporting skills, and others. By the end of the consultancy period, the national officer, part of the core team of the project, is expected to be well trained and capacitated to autonomously perform the necessary tasks related to monitoring, evaluation and learning across the project cycle.

• Implementing partner quarterly progress reporting and annual reporting to donors

The consultant/firm will be responsible for training SSC in drafting the quarterly progress reports that SSC has to regularly submit to the ILO, and for coaching the ILO relevant staff in drafting the annual narrative report to the donors, as per the ILO standards and donor requirements, using information and data collected and aggregated during the respective period. Draft reports will go through a series of reviews before being approved. The consultant/firm will be responsible for coordinating the various reviews and clean the final products in consultation with the project manager and team. The report templates will be provided, and all necessary data is available and can be easily extracted from SSC management information systems. The overall goal is that, after the duration of the consultancy, SSC is capable of drafting the quarterly progress reports as autonomous as possible and with very minimal support from the ILO, while the ILO relevant staff are also capable of independently drafting the donors’ annual narrative report.

• Third-party monitoring

The project will use both internal and third-party monitoring (TPM) to ensure that all project interventions are systematically and periodically monitored, analysed, and reported. TPM will serve the purpose of objective tracking, independent program oversight and monitoring functions. This will ensure that appropriate project milestones are accomplished, to allow the project team to take appropriate action if any obstacles to expected progress appear.

The scope and operationalization of the TPM will be outlined by the consultant/firm when developing the MEARL framework and plan. The most relevant stakeholders involved in the project’s initial design, including social partners and civil society, shall be included in the TPM process. Different data collection tools shall be employed to account for the mixed nature of the indicators used to measure the project’s implementation and results progress. The collection tools shall include but not be limited to surveys, interviews, focus groups, observations, and case studies. Data should be always disaggregated by gender, age, nationality and disability as relevant. In consultation with the project manager and the project team, TPM shall include a well-balanced mix of monitoring tools and approaches for each component, output, and activities. The consultant/firm will be responsible for developing the indicators and tools, and for establishing the necessary procedures for quality control of the analysis done within the scope of TPM. The consultant/firm will also be responsible for developing the detailed ToR concerning the TPM exercise (mostly related to data collection) that will be commissioned to a local organization.

• Impact evaluation

In order to ensure accountability and lesson-learning, the MEARL framework and plan should include an impact evaluation exercise, which should be understood as the main mechanism to draw lessons from implementation of Estidama++, including successful and unsuccessful design features, efficiency and efficacy of delivery systems, and factors and perceptions influencing the decision-making process regarding social security contributory systems. Such an evaluation will assess the impacts on participants that can be attributed to direct participation in the project and should help the team to determine effectiveness of the adopted approach and shed light on any corrective measures and improvements needed.

Under the supervision of the project manager, the consultant/firm will draft the ToR of the impact evaluation to be commissioned to an external evaluator/firm, as well as regular follow up and quality assurance of the evaluation at each stage during the duration of the consultancy. The ToR should include but not be limited to the following sections: (1) Background and context, (2) Purpose and target audience, (3) Objective and scope, (4) Evaluation questions and tasks, (5) Approach and Methodology, (6) Timing and deliverables, (7) Evaluation team composition and required competencies, (8) Management arrangements, (9) Budget and Payment, (10) Proposal Submission.

As part of the impact evaluation exercise, the consultant/firm will design and oversee a workers’/ beneficiaries’ survey. The objective of this survey is to assess to what extent the project has been addressing the needs of beneficiaries and collect relevant feedback to better inform any corrective measures, including any necessary tweaks in the technical and/or operational design to trigger change on behavior related to registration and compliance with social security commitments. This task will include the development of the Terms of Reference to commission the implementation of the survey to a specialized firm/NGO/think-tank or university. Throughout planning, implementation and reporting of the survey, the consultant/firm will manage the work of the contracted firm and will be responsible for the respective quality assurance.

IV. Deliverables and timeline

The consultancy will begin on the 1st of September of 2022 and should be completed by 30th of November of 2022 for a maximum of 60 working days. The consultant/firm is expected to be based in Amman, Jordan, for the whole duration of the consultancy (approx. three months) or at least have one representative in-country.

Deliverables – Deadlines (indicative)

1. Kick off meeting with the ILO team and SSC – 1 September
2. Impact evaluation ToR – 30 September
3. Capacity assessment workshop and visits to SSC – 3 October
4. Final draft of Q1 quarterly progress report – 13 October
5. SSC and ILO capacity assessment report – 20 October
6. Third-party monitoring ToR, indicators and tools – 10 November
7. MEARL Framework & Plan final draft (including evaluability assessment report) – 17 November
8. Launch of the MEARL Framework and Plan and capacity building workshop on results-based reporting – 21 & 22 November
9. Follow-up MEARL capacity building sessions (tailored to specific needs of specific staff) – 30 November

V. Payment Terms

Payments will be made against submission of deliverables, subject to the approval of the project manager as per the ILO quality requirements, in accordance with the following breakdown:

• 20% of the total fee following submission of deliverables 1 and 2
• 20% of the total fee following submission of deliverables 3, 4 and 5
• 20% of the total fee following submission of deliverable 6
• 20% of the total fee following submission of deliverable 7
• 20% of the total fee following submission of deliverables 8 and 9

VI. Qualifications and Selection Criteria

Education

• Advanced degree in one of the following disciplines: M&E, or technical topic such as development and social studies, social protection, public policy, behavioural and social sciences, information management, or any other relevant applied science with M&E background.

Experience

• 5 or more years of experience in designing and implementing MEARL frameworks and plans, as well as in monitoring, reporting and overseeing MEARL tasks in a development context.
• Experience of work on M&E systems for social protection, social security, and/or informality is a must and highly valued.
• Experience in managing quantitative and qualitative primary data collection.
• Experience in applied research and monitoring and evaluation research approaches, preferably in the field of social protection and behaviour change initiatives.
• Experience in writing monitoring and evaluation reports, including donor reports.
• Experience working with a UN agency or other international development organization preferable.

Skills and knowledge

• Demonstrated knowledge in practical application of Results Based Management.
• Extensive knowledge of qualitative and quantitative evaluation methods.
• Excellent written and spoken English and presentational capacities. Fluency in Arabic language is an advantage.
• Excellent inter-personal skills and communication skills
• Basic understanding on norms and standards for evaluation.
• Proficiency in MS Excel is required. Familiarity with statistical software (e.g. STATA and R) and data visualization programmes (e.g. Power Bi and Tableau) is an asset.
• Knowledge of the development context in Jordan and/or Middle East would be desirable.