End-Line Evaluation for Accelerating Health Agriculture and Nutrition (AHAN)

Terms of Reference

End-Line Evaluation Terms of Reference

Accelerating Healthy Agriculture and Nutrition (AHAN)

World Vision in Lao PDR (WVL) is leading a consortium of partners to implement the Accelerating Health Agriculture and Nutrition (AHAN) Project, funded under pillar three of the European Union’s (EU) Partnership for Improved Nutrition in Lao PDR, and by the Australian Government through the Department of Foreign Affairs and Trade (DFAT) from 2018 to 2021.

The project aims to create supportive conditions for enhanced household (HH) nutrition through five sub-objectives on agriculture, community/clinical health, WASH, gender and multi-sectoral coordination.

World Vision, together with Agronomes et Vétérinaires Sans Frontières (AVSF), Green Community Development Association (GCDA) and the Burnet Institute (BI), target 12 districts across three southern provinces, supporting 10 villages per district:

· Savannakhet: Phine, Sepone, Thapanthong, Phalanxay , Atsaphone and Xonobuly; **

· Saravane: Saravane, Ta Oi, Toomlarn and Lao Ngarm; and**

· Attapeu: Xaysetha and Samakkhixay.

The EU funds project implementation across all three provinces, while the DFAT funding is for Saravane Province only.

Below are the objectives of the project:

Overall Objective: To improve food security and create supportive conditions for enhanced HH nutrition

Specific Objectives:

Outcome 1: Improved access to and availability of sufficient and/or diverse food in year round.

Outcome 2: Improved dietary and care practices among Women of Reproductive Age (WRA) (15-45 years) and Children Under Five (CU5)

Outcome 3: Reduced incidence of selected Water, Sanitation and Hygiene (WASH) related diseases/illnesses linked to undernutrition

Outcome 4: Improved gender equitable relations at the HH level, particularly in decision-making and distribution of workload.

Outcome 5: Strengthened multi-sector coordination and support for nutrition.**

1. End of Project Evaluation Objectives and Principles

The principal objectives of the endline evaluation are:

1. Verify the extent to which the project has achieved the stated outputs, outcomes and targets, including among disaggregated stakeholder groups (gender and disability, young mothers and children)**

2. Document best practices, key lessons learned and recommendations to inform future project designs and funding opportunities of related projects **

3. Identify weaknesses in the project design, implementation or operating environment that constrained optimal project effectiveness**

4. Assess the effectiveness of project consortium/partnerships and recommend ways to improve project implementation in a related manner**

5. Provide project partners, including government and the communities, with a platform to evaluate the journey of project, and utilize the evidence to generate recommendations for future opportunities, and partnership with the Lao Government through the Ministry of Health. **

2. Evaluation scope

Geographic

· Savannakhet: *Phine, Sepone, Thapanthong, Phalanxay, Atsaphone and Xonabuly; **

· Saravane: Saravane, Ta Oi, Toomlarn and Lao Ngarm; and*

· Attapeu: Xaysetha and Samakkhixay.*

Demographic/Strata

· Head of households, mothers and caregivers of children under 5 and under 2 years of age

· Rice millers and operators

· Savings for Nutrition (S4N) members

· Beneficiaries of rice mills

· Home Garden beneficiaries

· Farmers Groups

· Mothers Groups

· Village water and sanitation management committee (VWSMC)

· People with disabilities

· Health Workers and Village Health Volunteers

· Lao Women Union Volunteers, TBAs, Health Promoters etc.

The evaluation will leverage The Organisation for Economic Co-operation and Development (OECD), Development Assistance Committee (DAC) Method with a focus on: *1. Effectiveness, 2. Impact, 3. Relevance, 4. Sustainability, 5. Efficiency, 6. Coherence, 7. Cross-cutting themes – including, Disability Inclusion and Equality; climate change and environment; protection; and peace-building.*** For this assessment, the groups to be consulted in response to the questions are, but not limited to WRA and CU5, Private Sector, Government partners, Senior Operation Managers, Program Director, CSO etc., with suggestions inserted in parenthesis for key selected questions).**

3. Methodology

The study will use a mix of qualitative and quantitative methods. Key informant interviews (KII) will be utilised with project implementation partners such as AHAN consortium partners (AVFS, GCDA, BI), government authorities at ministry and sub-national levels, and in the private sector at the key levels – national, provincial, and district. The KIIs are required to collect information from a wide range of partners who have engaged routinely with the project. Focus group discussions (FGD) will be conducted with project direct beneficiaries and stakeholders who engage with the project collectively. Project beneficiaries disaggregate by sex (female/male headed household, young mothers), poverty status, and people with disability. The tool will be reviewed by WVA and the project team prior to data collection. The consultant will hold the responsibility to select participants for FGDs and KIIs with the support of project staff to coordinate the participants.

The tentative sampling and data collection methods are suggested below:

Focus Group Discussion

FGDs will be conducted with different groups from different angles of project beneficiaries, classified by geography, gender, disability, Female headed household, and Local Agriculture Agent. The total number of FGDs are suggested in table below.

Key Informants Interview

Key Information Interviews (KIIs) will be conducted to elicit specific information about indicators of interest. The following list represents the Minimum number of respondents who will be considered for an interview. The actual number Will depend on the methodology that the consultant will propose and on the resources that are engaged.

HHs Survey and anthropometric measurement

HHs data will be collected using a quantitative survey technique, and obtaining anthropometric measurements of children under two years and under five years in comparison with baseline values. There is a preference to collect household survey data using digital means such as Kobo, to reduce errors and transcribing time. Training of enumerators, and piloting of the household survey should be accounted for when planning timeframes. The project plans to collect anthropometric measurements of children around March-June, 2022 to coincide with the same period baseline anthropometric measurements were taken.

The consultant will set out final study parameters in an end of project evaluation plan for approval before commencing any primary data collection. However, participatory methods are encouraged and can be proposed in line with this ToR, alongside the minimum requirements below:

Province

District

# Villages

HH Survey

Total HHs

FGDs

KIIs

VWSMC

PWD

Farmers Group

S4N Group

Rice Mill

Partners incl. DPO

WVI-L staff

M

W

M

W

Savannakhet

Atsaphone

5

20

100

















Savannakhet

Phalanxay

5

20

100

















Savannakhet

Phine

5

20

100

















Savannakhet

Sepone

5

20

100

















Savannakhet

Thapangthong

5

20

100

















Saravane

Lao ngarm

5

20

100



















Saravane

Saravan

5

20

100

















Saravane

Ta oi

5

20

100

















Saravane

Toomlarn

5

20

100

















Attapeu

Samakkhixay

6

20

120

















Attapeu

Xaysetha

6

20

120

















Total

1,240

12

12

12

12

12

12

12

1

12

97

DESK REVIEW AND SECONDARY DATA

• Consultant will be provided with a reasonable amount of information and project documents. This includes but not limited to Project Design Documents, M&E system/framework, project reports. The M&E Manager shall coordinate this and liaise with various Technical Advisors to share relevant and needed information

• The consortium of partners shall also provide project information and reports as source of secondary data. Partners include: AVSF, GCDA, BI, and LWU.

• The government partners and line ministries can also be a point of information on the area, our project progress etc. Some of these can also be found on government internet portals and websites**

4. Endline evaluation requirements

The Lead Consultant will lead on verifying the methodology, and the design of survey tools and analysis framework, allowing sufficient time for tools development translation, pre-testing and training. Lead Consultant (Team Leader) will recruit research assistants and the team will be required to travel to all survey locations, in collaboration with WVI-L. Data collection will be mobile-based for HHs survey and paper-based for qualitative information. Survey size for WVA is usually guided by a confidence interval of 95% and 5% margin of error.

Note that mobile data collection is preferred in most instances, and that previous experience setting up online data collection is usually included among consultant criteria for selection.

Note that consultation with children requires additional ethical checks, and that previous experience consulting with children should be included among consultant criteria for selection, depending on the volume of engagement with children and the need for child-friendly methodology.

The consultant will submit an evaluation plan including methodology and tools for approval by the project team as a first deliverable.

5. Proposed Evaluation Deliverables and Milestone Dates

The following proposed dates will be subject to negotiation and revision, with the chosen candidate.

Products

Due date

Proposed # days

Inception workshop with AHAN Team (WVI-L, AVSF, GCDA, BI) to review the research objectives, design, data collection plan, tools, and plan for analysis and reporting; days include preparation**

3rd week June

2

DELIVERABLE 1: End of project evaluation design/plan, including data collection tools (survey questionnaire, guide questions for FGD and KII), data collection guideline**

4th week June

10

1. Training of anthropometric data collectors and supervisors

2. Training of enumerators, focus group leaders and WVI-L staff; data collection (Subject to adjustment with lead evaluator): All data records whether in soft or hard copy e.g. transcripts, databases, spreadsheets, photographs, audio data, are to be made available to WVI-L in-country at the end of data collection; privacy and confidentiality of data must be considered (eg. anonymity of individual records). Explore opportunity to employ PWD as enumerators. **

2^nd Week in March (2days)

1st-2nd week July

7

Field work to collect data in 3 provinces (Anthro)

Field work to collect data in 3 provinces

3^rd week in March (2days)

3rd-4^th week July

16

Data cleaning

4^th week of July

5

DELIVERABLE 2: Analysis and PPT presentation of preliminary findings and key recommendation at a workshop with AHAN Team and DPO; feedback considered**

1^st week August

7

DELIVERABLE 3: PPT Presentation of draft findings and key recommendations, at a stakeholder workshop including donors, partners, government and community representatives; feedback considered**

2^nd week August

1

• 1st draft Consolidated EU and ANCP Endline Project Evaluation and ANCP (Savarane only) Report (2 separate reports)

• 2^nd draft: Revised reports based on technical review and inputs

• A standalone, ‘user-friendly’ abridged version and PPT for stakeholder audiences; this will be available to a public audience and potentially translated to local language

3^rd week August

8

DELIVERABLE 4: final version of 2 End of Project Evaluation Reports, incorporating second round of feedback; including WVI-L management response; including executive summary that can stand alone as a self-contained summary report**

End of August**

1

Total billable days

57

6. Criteria for appointing Lead Consultants/Firm

Essential:

This Endline evaluation should be undertaken by a senior consultant team who will be working closely with the AHAN program M&E, Operations team and AHAN Director. The Team Leader will have a range of skills and experience in order to deliver an insightful baseline assessment for the AHAN program. They should be experienced in gender sensitive data collection techniques and have the ability to lead the data collection team, including application on agreed data collection software. It is expected that the consultant team will have the following qualifications and experience:

• Extensive and demonstrated experience in nutrition programming or in MNCH programming
• Extensive and demonstrated experience in doing similar studies using quantitative methods, integrating statistical analysis and interpretation of results.
• Experience designing and using digital data collection methods, for example, ODK, Kobo, etc.
• Consultant Team Leader holds an MPH, MSc or MA (preferably PhD) in nutrition or public health
• Demonstrated ability to lead and coordinate a multi-disciplinary team in M&E activities
• Excellent English communications skills – both interpersonal and written. Example reports will be requested from shortlisted applicants
• Previous experience in doing studies in Laos will be an advantage

Preferred:

· Demonstrated experience in (contextual to study; e.g. research with children; gender-sensitive research; environmental safeguarding; social inclusion and disability)

· Demonstrated experience in mobile / online data collection methods (note, if your office has limited previous experience in working with mobile data, and household survey is intended, move this criterion upward to essential)

· Consider cultural or security implications

7. Travel Expectations of Lead Evaluator/Firm and Security Context of Location

The lead consultant/firm is expected to be available to spend approximately 31 days in the country of the project for data collection preparation, collection, cleaning and debrief in the month of March-June, 2022 (subject to refinement at contract stage).

The evaluation budget will cover the costs of travel visas, transit to and from their designated home point to the project locations, and accommodation, for the duration of the data collection period. The consultant will be responsible for his or her own insurances, vaccinations, health, and security preparedness, and must be prepared to sign all relevant policies relating to in-country engagement with World Vision, including child protection policies.

World Vision International Laos will cover the two workshops – the Inception and first draft result presentation and any cost related to meeting between consultant and the project team. Furthermore, World Vision International Laos will make the following resources available to the consultant for the review:

• Contacts of potential trained enumerators, facilitators and translators

• Contacts of services providers including transport and accommodation providers

WVL can also share office for meeting room at the Vientiane National Office and Field Office for initial preparation meetings, singing agreement contract.

The Lead Consultant will be responsible for his or her own insurances, vaccinations, health, and security preparedness

With regard to the current security context of the project country and locations, it is recommended to exercise normal safety precautions in Laos. For more information visit the following site, also highlighted below: https://www.smartraveller.gov.au/destinations/asia/laos