Mid-Term Evaluation (MTE) for the EU-funded Project in Tanzania. Project Title – Passport to Coffee Export (PACE)

Project Consortium
Tanzania Coffee Research Institute (TACRI): Co-applicant, a government research body with a mandate to initiate, implement, promote and carry out research on coffee production, processing, quality, farming systems and husbandry of other crops associated with coffee.
Envirocare (co-applicant): is a non-profit organization working in Tanzania, primarily focused on conserving the environment and improving livelihoods in local communities with gender, human rights and community based organisation integration through a participatory approach.
Solidaridad Tanzania Country Office: an affiliate in this action and will carry out Country level implementation roles for the PACE Project.
Solidaridad Europe: an ‘affiliate’ of this action was established in 1969 in the Netherlands. Developed Max Havelaar label for sustainable supply chains in 1988, and have garnered over 50 years’ experience on sustainable supply chains throughout the globe
Project Summary
Passport to Coffee Export (‘PACE’) is a three-year project funded by the European Union (EU). The project is targeting approximately 22,500 coffee smallholder farmers in Mbeya, Ruvuma and Songwe regions in Tanzania.
Project Objectives
PACE’s overall objective is to promote a sustainable and inclusive sector development through a robust export oriented coffee value chain by 2023.
Project Outcomes

1. To improve productivity and enhance food and nutrition security of smallholder coffee farmers in Tanzania

2. To increase incomes for coffee farmers in Tanzania through job creation.

3. To improve export oriented coffee sector sales and incomes in Tanzania

4. To enhance sectoral value addition and marketing
   Project Target Groups and Final Beneficiaries
   By 2023, PACE Project is targeting to reach approximately 22,500 coffee smallholder farmers in Mbeya, Songwe and Ruvuma regions as narrated below;
   ▪ 22,500 smallholder coffee farmers with 52% being youth aged (18-35 years) and women.
   ▪ 50 care groups within 25 community based groups in the 75 villages (To support work on malnutrition especially in children under 5 years’ levels)
   ▪ 200 factory workers from the 25 cooperatives, and 4 community processing and value addition centers (Will be supported to acquire technical skills and other best practices in order to enhance quality, reduce post-harvest waste and encourage resource efficiency and recyclicity.)
   ▪ 200 extension staff from relevant Ministries, Local Government Authority (LGA’s), TaCRI and Envirocare (Will be trained to deliver innovative agricultural practices to farmers.)
   Project Intervention Logic
   The proposed intervention logic underpinning this action is based on the understanding that:
   (a) IF the capacity of small-scale coffee farmers to adopt sustainable nutrition sensitive climate-smart strategies is strengthened;
   (b) IF the coffee cooperatives/groups and/or associations’ capacity to operate professionally is strengthened;
   (c) IF gender inclusive strategies are employed to enhance the meaningful and gainful participation of youth and women in the coffee sub-sector is strengthened; and
   (d) IF the private sector is developed to create access to inclusive financial and market services; – THEN, the action will improve the productivity, commercialization and competitiveness of the coffee value chain and furthermore improve the small-scale farmers’ incomes, food and nutrition security.
   Project Assumptions
   Our assumption is that;
   i. Coffee farmers and cooperatives currently lack the capacity and technical knowledge to implement climate smart sustainable production practices and do not operate efficiently and in a transparent manner respectively;
   ii. Farmer capacity building and sharing best practices will lead to the adoption of recommended production practices that will make the coffee sub-sector sustainable and competitive, leading to improved productivity, improved incomes, food and nutrition security and decent jobs;
   iii. Women and youth gainful involvement will make the coffee supply chain vibrant and profitable;
   iv. Engaging with coffee buyers and consumers that have defined a sustainable sourcing strategy, through a service and demand-oriented approach, will ensure interest is garnered and orders are placed for Tanzanian coffee;
   v. Communicating to consumers and buyers with the Tanzania coffee storyline and messaging will draw attention and influence the sourcing habits;
   vi. Financial inclusion will be improved through strengthened relationship between coffee farmers and financial institutions, impact investors;
   vii. Access to genuine and cost effective farm inputs will be through identification and engagement of reputable service providers to participate on the PACE project.
   Purpose of the Evaluation The objectives of the MTE are to:

   1. Assess performance towards the program’s outcome and output indicators and identify possible gaps towards achieving full results (accountability).
   2. To capture lessons learned and provide information on the project approaches identifying what has worked and what has not worked for learning and as a guide for future planning and potential upscaling (learning)
      Scope of Work and Key Tasks The MTE will cover the implementation period from the start of the program. This MTE is expected to assess the effectiveness of the implementation strategy of the project in a consortium setup. This will include the implementation modalities and participation by the other consortium partners. The evaluation will include a review of the project design and assumptions made at the beginning of the program development process. It will assess whether results are on track, the implementation strategy has been optimum, and recommends areas for improvement and learning. The evaluation will also assess whether project resources are efficiently utilized to produce planned results and whether results can be realized within the planned
      period. Lastly, the sustainability of the project will be looked at to ensure a lasting impact.
      Indicative Evaluation questions: Relevance of the program strategy (design and focus of the program)
   • To what extent are the program objectives meeting the needs of small-scale coffee farmers and their families?
   • Does the program address the key bottlenecks in export oriented coffee for the farmers?
   • Are the underlying assumptions and context of the program still correct? Effectiveness (project progress)
   • To what extent has progress been made towards the planned outputs and outcomes, as defined in the program results framework? What recommendations for improvement can be made?
   • Were there any unexpected positive or negative side effects and how were these dealt with by the program management?
   • Are the chosen program interventions and activities effective in delivering the desired outcomes?
   • To what extent was the program implementation effective (Management arrangements, work planning, finance and co-finance, project-level monitoring and evaluation systems, Stakeholder Engagement, reporting, communications)
   • What factors have contributed to or hindered the achievement of intended outputs and outcomes?
   • To what extent have partners in the consortium played their expected roles toward the achievement of the program goal?
   • Has the consortium partnership been appropriate and effective?

Outcomes and Impact:

• How does the program contribute towards lasting change to the lives of beneficiaries with regard to:
  • To improve productivity and enhance food and nutrition security of smallholder coffee farmers in Tanzania
  • To increase incomes for coffee farmers in Tanzania through job creation.
  • To improve export oriented coffee sector sales and incomes in Tanzania
  • To enhance sectoral value addition and marketing
• Has progress so far led to, or could in the future catalyse beneficial development effects (income generation, gender equality, and women’s empowerment, improved governance, etc.) that should be included in the project results framework and monitored on an annual basis for the remainder of the project? Efficiency:
  • Are the inputs identified realistic, appropriate and adequate for the achievement of the results?
  • Do the actual or expected results (outputs and outcomes) justify the costs incurred? Are resources effectively utilized?
  • What are the factors contributing to implementation efficiency? Sustainability: – Are there any remaining barriers to achieving the project objectives that require review?
• How can successful aspects of the project be further built to expand the benefits?
• What is the likelihood of continuation and sustainability of program outcomes and benefits after completion?
• What are the key factors that require attention to improve prospects of sustainability of outcomes?
• What are the important challenges the program should overcome? And to what extent are these adequately addressed?
• What are the key lessons learned based on the experiences of project implementation?
• NB: The Consultants may choose to categorize the questions differently, as long as these questions are addressed.
  Methodology This evaluation requires a mixed-method approach that allows for methodological triangulation to increase the validity and credibility of data. Participatory methods shall be used to collect both qualitative and quantitative data. The key methods shall include the following but are not limited to:
  • A household survey using a survey questionnaire (a representative sample of the total beneficiary population can be used)
  • Focus Group Discussion (FGDs) with lead farmers and AMCOSs/farmer groups board members
  • Literature review of existing documents and review of context and poverty trends including the project proposal and other documents, annual and quarterly reports, monitoring, and evaluation reports
  • Field observation of the targeted project areas
  • Key informant interviews (KII) with AMCOSs, farmer groups board members, Lead farmers, Extension officers, DAICOs, Project partners (TACRI, Envirocare), and project management team reflection and feedback sessions with the project management team. The final methodology to be adopted shall be discussed and agreed upon with consultants/firm during the inception meeting.
    The MTE team will first conduct a document review of project documents (i.e. Initiation Plan, Project Document, Project Inception Report, Baselines, Tracking Tools, Project Steering Committee meeting minutes, Financial and Administration guidelines used by Project Team, etc.) provided by the Project Team. Outcome and output indicators from the Logical Framework and reported upon in the baseline and annual reports will be used to the extent possible. In case these indicators are not sufficiently reported upon, data collection will need to be complemented.
    The consultants will participate in an inception workshop to clarify their understanding of the objectives and methods of the MTE, producing the MTE inception report thereafter. The MTE mission will then consist of interviews and site visits to selected areas in the Project areas.
    Expected Outputs & Deliverables
    Solidaridad and the consultant will work together to identify the study team. However, Solidaridad recommends that consultants identify one Team Leader (TL) to coordinate/conduct the comprehensive study and liaise with Solidaridad.
    To collect information from the field, an adequate number of Field Enumerators (FEs) will be deployed in the facilitation of Solidaridad’s PMEL Officers. Supervision of the fieldwork and the quality (reliability and validity) of the data/information collected from the field is the primary responsibility of the TL. The TL will work closely with Solidaridad PMEL Officers and Project Team. In each step and process, consultation with the Program Manager and PMEL Officer is vital.
    The consultancy/consultant team will be primarily responsible to:

1. An expression of interest (EOI) for the study, outlining the proposed methodology and process of data collection and analysis that specifies:
   a. Proposed methodology for implementation of the study, including sample sizes.
   b. A detailed timeline for the study
2. Inception report based on the work plan and EOI:
   a. Detailed methodology and evaluation matrix
   b. Draft data collection tools for all indicators to be measured in the study, standard indicators definitions agreed upon by the project should be used to develop data collection tools.
3. Presentation of methodology to Solidaridad team in an inception meeting and incorporate suggestions provided.
4. Training of enumerators (Solidaridad PMs and PMEL Officers will be part of it)
5. Draft report of the study for the feedback and comments of Solidaridad and partners involved.
6. Presentation of the main findings of the study at field level validation involving project teams, local authorities, and other stakeholders as agreed with the project team.
7. A final report in English following the Guidelines under MTE Annex 1.
8. Datasets: The consultant shall submit all datasets, cleaned & raw, transcription or audio files during focus group discussions, KI, scripts for data analysis, and any form of data captured during the implementation of this assignment
9. The evaluator shall also submit a pager infographic summary of the finalized report.
   Coordination and cooperation with Solidaridad
   The Consultant undertaking the study will work under the coordination of a PMEL Officer, who will support the mobilization of respondents and offer technical backstopping. The Project team will coordinate consortium partners in ensuring that the sampled survey respondents (beneficiaries and Key informants) are sensitized and mobilized for the exercise while availing a workable itinerary and technical backstopping.
   Profile of the Evaluators
   The MTE team will ideally consist of independent consultants that will conduct the MTE – one team leader (with experience and exposure to projects and evaluations in Tanzania) and local team experts.
   The team must have strong combined expertise in the sectors of agricultural economics, extension and education, gender, and Natural Resource Management that can provide adequate experience to conduct the assessment and that meets the following requirements:
10. Advanced degree (masters) in the area of agriculture, economics, agri-business development, social sciences, or other related disciplines; 2. At least five years of proven experience in leading and conducting similar exercises in Africa 3. Demonstrated experience in both quantitative and qualitative methods of data collection and analysis; including survey design, implementation of surveys and statistical data analysis, and the use of participatory appraisal techniques in data collection and analysis. 4. Ability to interpret and analyse complex qualitative and quantitative data and to present findings and recommendations in a clear and concise way; 5. Strong expertise in value chains (coffee, other food crops, horticulture) 6. Knowledge and sensitivity to the political and social context of Tanzania context is an asset;
11. Earlier experience with the work of Solidaridad is an asset.
12. Application and Timeframe
    Each application package should include the following:
13. An application letter addressing the selection criteria including how the firm’s/group’s previous experience matches the consultancy objectives as well as the interest for the consultancy opportunity. It should also indicate the availability of the assessment team. The letter should be no longer than two pages.
14. A brief proposal for the study with a proposed methodology and work plan (not more than 8 pages). The proposal should have a budget with breakdowns of different costs involved, to the finer detail. A budget with aggregated figures will not be accepted.
15. A sample of recently written reports for a similar assignment preferably in the coffee sector.
16. Updated CVs for all consultants proposed to conduct the assignment including relevant work experience and qualifications.
    The overall time frame of the assessment will be 30 working days, which will include inception meetings, finalizing survey methodology, training of enumerators, data collection exercise, processing of data analysis, report writing, and presenting the findings of the survey. The team undertaking the study will be fully responsible to administer the study including management of data collection.
    Details will be worked out jointly with the consultant team and project staff. However, the work is expected to commence in March 2023 for an estimated 30 days period.

MTE Annex 1.
Guidelines on Suggested Contents for the Midterm Evaluation (MTE) Report
i. Basic Report Information (for the opening page or title page)

• Title of project
• Project Number
• MTE period and date of MTE report
• Region and countries included in the project
• Implementing Partner and other project partners
• MTE team members
• Acknowledgements

ii. Table of Contents
iii. Acronyms and Abbreviations

1. Executive Summary (3-5 pages)
   • Project Information Table
   • Project Description (brief)
   • Project Progress Summary (between 200-500 words)
   • Concise summary of conclusions
   • Recommendation Summary Table
2. Introduction (2-3 pages)
   • Purpose of the MTE and objectives
   • Scope & Methodology: principles of design and execution of the MTE, MTE approach and data collection methods, limitations to the MTE
   • Structure of the MTE report
3. Project Description and Background Context (3-5 pages)
   • Development context: environmental, socio-economic, institutional, and policy factors relevant to the project objective and scope
   • Problems that the project sought to address: threats and barriers targeted
   • Project Description and Strategy: objective, outcomes and expected results, description of field sites (if any)
   • Project Implementation Arrangements: short description of the Project Board, key implementing partner arrangements, etc.
   • Project timing and milestones
   • Main stakeholders: summary list
4. Findings (12-14 pages)
   4.1 Project Strategy
   4.2 Progress towards Results
   4.3 Project Implementation and Adaptive Management
   4.4 Sustainability
5. Conclusions and Recommendations (4-6 pages)
   5.1Conclusions
   • Comprehensive and balanced statements (that are evidence-based and connected to the MTE’s findings) which highlight the strengths, weaknesses and results of the project
   5.2 Recommendations
   • Corrective actions for the design, implementation, monitoring and evaluation of the project
   • Actions to follow up or reinforce initial benefits from the project
   • Proposals for future directions underlining main objectives
   Annexes to be included in the final report:
   ● MTE ToR
   ● MTE evaluation matrix (evaluation criteria with key questions, indicators, sources of data, and methodology)
   ● Questionnaire or Interview Guide used for data collection
   ● MTE mission itinerary
   ● List of persons interviewed
   ● List of documents reviewed
   ● Data set