Director, Monitoring, Evaluation, and Economic Analysis

Position Objective:

The Millennium Challenge Account (MCA) in Timor-Leste, collaborating with the Government of Timor-Leste and the Millennium Challenge Corporation (MCC) of the Government of the United States of America, is responsible for implementing the $420 million Timor-Leste Compact, which, together with an additional contribution of $64 million from the Government of Timor-Leste, represents a total $484 million investment in Timor-Leste. The Compact aims to address human capital development as a binding constraint to economic growth in Timor-Leste through two projects: the Water, Sanitation, and Drainage (WSD) Project and the Teaching and Leading the Next generation of Timorese (TALENT) Project.

The Director, Monitoring, Evaluation, & Economic Analysis (MEE), leads the combined Monitoring & Evaluation (M&E) and Economic Analysis (EA) aspects of the program. The goal of the MEE unit is to produce timely, relevant, and reliable data and assessments related to the progress and quality of project implementation and the achievement of targeted results by the Timor Leste program. The MEE Director, is responsible for leading the monitoring and reporting of progress toward program targets, supporting the independent evaluations of the program, updating the economic analysis activities of the program, and managing the program M&E budget and MEE staff. This role requires working closely with MCA staff. In addition to MCA staff, the MCC counterparts to this position with whom the MEE Director will liaise are the M&E Lead(s), Results Reporting Analysts, and Economist(s).

Duties and Responsibilities:

• Building on the Program M&E Framework described in the Compact agreement, lead the development and periodic updates of the Timor Leste M&E Plan, in accordance with MCC policies and guidelines and in coordination with MCC and relevant stakeholders.
• Lead the implementation of the monitoring component of the Timor Leste M&E Plan to provide timely and relevant input to project management and external reporting, including:
  • Define the performance indicators to be monitored along with their baselines and targets in collaboration with project teams; ensure that monitoring indicators are relevant to project management needs and that risks and assumptions included in the M&E Plan are appropriately tracked.
  • Identify critical data gaps or data quality issues related to the M&E Plan indicators and design and implement a plan to resolve these issues and build capacity with the party that produces the data (e.g., government entities or project implementers).
  • Work with implementers and Implementing Entities to ensure M&E requirements are appropriately incorporated in their contracts and Implementing Entity Agreements, respectively, and that there is a shared understanding of relevant indicators and other requirements.
  • Produce the Indicator Tracking Table (ITT). This should include reviewing all ITT data on a quarterly basis, checking inconsistent values with project leads and reporting entities, and submitting supporting documentation for all data that is reported.
  • Conduct a Data Quality Review of monitoring data reported by MCA-Timor Leste as well as likely sources of future monitoring data (i.e., data sources identified in the M&E Plan) and implement a plan to address any identified weaknesses.
  • Design and implement field assessments of progress towards the achievement of outputs and intermediate outcomes, especially in situations where concerns have been raised regarding data quality or validity.
  • Disseminate information from the approved ITT to relevant stakeholders in Timor Leste and to MCC.
• Support the design and implementation of the evaluation component of the Timor Leste M&E Plan to promote accountability and learning, including:
  • Provide data and documentation required to inform the design and implementation of evaluations to MCC and independent evaluators.
  • Monitor adherence to the project design and implementation plans and report any deviations to MCC and the independent evaluator.
  • Review evaluation reports, survey instruments, and other materials produced by each of the evaluators hired by MCC to conduct independent evaluations.
  • Manage local stakeholder reviews of evaluation reports and materials.
  • Contract survey firms to undertake baseline and interim data collection for independent evaluations and supervise their work in coordination with the independent evaluator. This work includes adhering to guidelines for protecting human subjects as outlined in MCC’s TREDD (Transparent, Reproducible and Ethical Data and Documentation) Guidelines.
  • Organize meetings/events for the dissemination of evaluation findings for each evaluation report.
• Collaborate with the MCC Economist to ensure that the Cost-Benefit Analyses and Beneficiary Analyses, including estimates of the Economic Rate of Return (ERR), are modified and updated in accordance with MCC Guidelines as improved information becomes available (updating key parameters and variables upon the receipt of new or higher quality data.)
• Maintain close collaboration and integration between MEE and project teams to ensure that MEE’s data and analysis is accurate, up-to-date, and supports evidence-based project design and management.
• Maintain regular communication with MCA-Timor Leste leadership, MCC M&E and Economic Analysis staff to provide updates on the status of MEE activities and to communicate risks related to the progress of implementation or the achievement of results.
• Manage the program M&E budget and provide updates to MCC on commitments and disbursements on a quarterly basis.
• Manage the procurements and contracts funded by the program M&E budget and report progress to MCC on a quarterly basis.
• Help foster a relationship with the post-program counterpart designated to support the completion and dissemination of independent evaluations of the Timor Leste program.

Requirements (Education, Experience, Technical Competencies):

• Advanced degree (Masters or higher) in Economics, Statistics, or a related subject.
• Ten (10) or more years’ experience, seven (7) of which is in managing teams that conducted quantitative and qualitative analyses of programs, including a minimum of five (5) years of experience in Monitoring & Evaluation.
• Previous managerial experience with demonstrated ability to balance a variety of tasks and demands and to meet deadlines in a responsible and flexible manner. Must demonstrate history of delivering high quality projects on time and within budget.
• Demonstrated experience designing and implementing monitoring systems, including collecting, analyzing, and reporting performance data.
• Previous supervisory experience managing at least 2 subordinate staff.
• Experience applying economic concepts to real-world problems, using data and analysis to reach conclusions and provide recommendations. Experience conducting this type of work in Timor Leste is desired but not required. Demonstrated competency in cost-benefit analysis of projects is desired but not required.
• Demonstrated experience designing and implementing program evaluations; experience implementing rigorous, quantitative evaluations involving household or business surveys preferred.
• Strong and demonstrable skills in contract management.
• Willingness to undertake regular field visits and interact with the different stakeholders. Occasional international travel may also be required.
• Strong analytical and report writing skills.
• Proven ability to work with national government counterparts.
• Experience working with international donor organizations preferred.
• Good written and oral communication skills, ability to work with wide range of stakeholders.
• Written and spoken fluency in Tetum and English, Portuguese a plus.
• Demonstrated experience in one or more of the following statistical analysis software and databases, such as STATA, SPSS, SAS, CSPRO, Access, Excel and strong proficiency with MS Word and PowerPoint.
• Excellent organizational skills related to data and document management.
• Experience working in Education and Water and Sanitation sectors preferred.
• Demonstrated similar work experience in Timor-Leste is an advantage.
• Familiarity with data collection requirements for research involving human subjects and/or at-risk populations is an advantage.