(Offre en anglais) Mercy Corps lance un appel à consultation pour l’évaluation de la performance du programme en Tunsie Retour vers les opportunités

Project/Consultancy Title: Tunisia Program Performance Assessment

Project Location(s): Tunisia

Background:

Mercy Corps has been active in Tunisia in July 2011, working to support youth employability and entrepreneurship opportunities, increasing civic engagement, and improving the capacities of local authorities, civil society organizations, and private sector organizations to engage in positive governance. Our goal is to empower and engage citizens, support the development of an inclusive and enabling environment, foster resilience, and support social cohesion.

Purpose:

The purpose of this evaluation is to explore past programming and generate insights and evidence on impact, sustainability and lessons learned, both to benefit Mercy Corps and contribute to the broader debate over effective development interventions in Tunisia and similar context. Mercy Corps has been present in Tunisia since 2011 with a country strategy focused on supporting youth employability and the development of local governance/civic engagement. Programs have been developed adopting an array of program approaches, including financial education, access to financial services, soft skills training, Entrepreneurship Clubs, vocational training and mentoring services. While these program approaches are intrinsic to Mercy Corps Tunisia’s history and current operations, there is insufficient hard evidence to demonstrate the impact these approaches have had on achieving their intended outcomes and additionally, whether these outcomes have been sustainable in the face of shocks and stresses by strengthening the resilience of each target group. Without this hard impact evidence, it is both challenging for Mercy Corps to advocate to donors for support and Mercy Corps staff do not know if one approach may be more effective than another when designing future programming.

This performance evaluation will enable:

• Mercy Corps to advocate to donors with proof of our impact and success;
• Mercy Corps to influence donor learning and the broader community of practice on “what works and why” in Tunisia; and
• Mercy Corps staff to have the necessary evidence for future evidence-based program design.

Consultant Activities:

The performance evaluation will measure the impact of the following Mercy Corps Tunisia program approaches:

● Employability: The evaluation will examine Mercy Corps’ approach to increasing employability and reducing unemployment among youth. Specifically, the evaluation will look at the Tunisia Works! Program (2012-2015) and the various program strategies it adopted to increase employability – including financial education, access to financial services, soft skills training and Entrepreneurship Clubs, vocational training and mentoring services – and evaluate the effectiveness and sustainability of each approach and assess whether each approach contributes to building individual resilience to unemployment. The evaluation will look at Mercy Corps’ approach and impact to extrapolate lessons learned to inform the donor community about how best to reduce unemployment among Tunisian youth.

● Local Governance & Civic Engagement: The evaluation will examine Mercy Corps’ approach to improving local governance and civic engagement. Specifically, the evaluation will look at program strategies under the ADEL Program (2015 – 2018) and Tunisia Leads! Program (2012-2015), which sought to establish partnerships between – and increase the capacity of – civil society organizations (CSOs) and the private sector to advocate for beneficial policies for economic development. The evaluation will also look at whether and how economic policies changed as a result of the advocacy efforts.

As a result of this evaluation, Mercy Corps will be able to determine which program strategies have achieved a sustainable impact, warranting prioritization in future program design, and which approaches should be de-prioritized or abandoned. The evaluation will
also allow Mercy Corps to examine the extent to which this programming achieved positive results that could be scalable and help inform donor approaches to youth employability and governance gaps.

Background & Relevant Documents

Since 2011, Mercy Corps Tunisia has worked across offices in Gafsa, Medenine, Jendouba, and Tunis with a focus on enhancing youth employability and entrepreneurship opportunities, increasing civic engagement, and improving the capacities of local authorities, civil society organizations, and private sector organizations to engage in positive governance. Our strategy over the next three years is to build on this work in order to continue to empower and engage citizens, support the development of an inclusive and enabling environment, foster resilience, and support social cohesion.

In the years since the 2011 Revolution, Tunisia has made some political progress as it transitions to an open and democratic system of governance. The pace of this progress, however, has continued to be constrained by pressing socio-economic challenges, including a slow decentralisation process, frequent government re-shuffling leading to delays on local elections and reforms, and domestic and external security issues. Economy recovery has been slow, and there is a growing social dissatisfaction with the lack of employment opportunities. There are also big regional differences, with underdeveloped infrastructure and higher rates of unemployment in the central and southern regions of the country.

Mercy Corps Tunisia’s Goal is to support the development of a Tunisia where people are empowered, included and engaged with equitable access to opportunities by focusing on three key objectives –

● Objective 1: An enabling environment supports the local population to realise their development potential
● Objective 2: Local governance processes are operational, participatory and effectively provide minimum services
● Objective 3: Youth are empowered and engaged to participate in their personal and local development

Mercy Corps’ programs, Tunisia Works, Tunisia LEADs and ADEL were designed to address regional development through improved local coordination and planning, and increasing access to decent employment opportunities for youth that contribute to the development of the region.

Tunisia Works: The goal of Tunisia Works! was to increase the capabilities of youth to become more employable and create a favourable environment for entrepreneurship initiatives by stimulating private, public and civil society actors to provide appropriate services for marginalised youth in the targeted areas (Jendouba and Medenine). It aimed to achieve this by improving the skills, confidence and capacity of youth students/ graduates from tertiary and vocational training to integrate into the formal labour market (through formal employment or self-employment). The projects intended outputs were to:

• Strengthen the employability of 2,500 Tunisian youth (18 – 35 years) through increasing the capacity of youth to analyse market needs and identify high potential jobs, working with the private and public sector “business advisory committees” to match training with market needs and apprenticeship schemes, supporting job clubs in vocational schools or universities; and
• Strengthen Tunisia’s private sector by supporting at least 75 entrepreneurs to develop business plans and to access financial and non-financial services, and by increasing the capacity of 3 business support providers and 6 microfinance institutions to target youth entrepreneurs

The program was implemented with funds from the FCO/DFID between September 1, 2012 and September 30, 2015 and is now closed. It was closely aligned with the AFD-funded Medenine Works! project and additional smaller projects funded by SwissContact and the British Council. The existing final evaluation draws on the results of all of these programs combined. Following completion of the program, the Club Entreprendre component was expected to be adopted by the Tunisian Agency for Vocational Training (ATFP) and integrated into their training centers. However this transfer of ownership did not take place due to funding and coordination challenges, and therefore the Club Entreprendre approach has continued to be built into Mercy Corps’ continued programming with no clear sustainability plan.

Tunisia Leads: The goal of Tunisia Leads! was to establish effective and productive partnerships between economic development Civil Society Organisation (CSO) partners and the private sector in southern Tunisia, as an advocacy platform for integrated regional development. The projects’ intended objectives were to:

• Enhance capacity of PADIL as lead training partner to create, deliver and sustain service provision for economic development and cross-sector partnering;
• Strengthen organizational, technical capacity of 15 CSOs to understand opportunities for civil society to lead local economic development and forge cross-sector partnerships with the private and public sectors; and
• Support public advocacy for regional economic development through CSO networking and partnership with local private sector entities.

Existing Program Information Sources:

• Program Proposal
• Final Report

ADEL: The goal of ADEL was to support the establishment and the engagement of associated actors in multi-stakeholder consultation, dialogue and improved citizen engagement to improve the relevance and the effectiveness of activities focused on local economic development in the governorates of Gafsa, Sidi Bouzid and Kasserine. Specifically the project aimed to establish a community of local, state and private actors able to provide relevant support services for employability and entrepreneurship that respond to the economic development issues of the target governorates.

The projects intended outputs were to:

• Establish a “shared vision” of the economic development priorities of the governorates of Gafsa, Sidi Bouzid and Kasserine, developed following activities of consultation, research and information sharing, and multi-stakeholder dialogue.
• Strengthen the capacity of local community, state and private actors to provide collaborative services to support employability and entrepreneurship.
• Establish an association of actors to oversee the implementation of the commitments made by economic development actors, and conduct coordinated advocacy campaigns to facilitate economic development in the region.

Evaluation Design

Evaluation Questions

• To what extent were the programs successful in achieving their intended goals/outcomes

1. Have they been sustainable (or what prevented sustainability)?
2. What were the cumulative achievements of the programs?

•  What (if any) were the longer-term impacts of the program in the years following program close out? For example:

1. What percentage of beneficiaries who received vocational and/or life skills training or mentorship were able to find and keep a job?
2. Did the skills, resources and/or strategies that beneficiaries gained through these programs appear to help them cope with unanticipated or anticipated shocks and stresses, and to what extent?
3. Is unemployment lower among former Mercy Corps beneficiaries than among the general population?
4. How many entrepreneurial beneficiaries were successful at creating a business? Is the business still successful?
5. What evidence is there that the employment market system has been strengthened?
6. Which approaches played an important role in helping people and the systems they depend on for employability and representative governance cope, adapt and transform in the face of high-frequency and/or high-impact shocks and stresses?
7. How effective have CSOs and the private sector been at advocating for beneficial policies for economic development? What policies have been adopted, implemented, or strengthened based on CSO or private sector advocacy? To what extent can that be attributed to Mercy Corps beneficiaries?
8. What have CSOs been able to accomplish thanks to the capacity building training they received?
9. Did Mercy Corps’ programs have any unintended effects (both positive or negative)?
10. Did Mercy Corps’ programs miss opportunities to build resilience?

• What programmatic approaches were most effective and should be expanded in future programming? Which ones were least effective and should be replaced or redesigned? Were any gaps identified during the course of the evaluation?

Evaluation Methodology

The following data collection and analysis methods will be used to address the evaluation questions:

Facilitated workshops – A workshop/s will be held with remaining Mercy Corps program staff who worked on the above past programs, alongside other relevant program staff, to reflect on program implementation, challenges and successes. Evaluation planning will also take place during the workshop to visit select sites, identify former beneficiaries and stakeholders to interview, organizing logistics, and refining evaluation questions and data collection instruments.

Semi-structured stakeholder and beneficiary interviews/focus groups – Evaluators will use questionnaires to guide interviews and/or focus groups with stakeholders including past program beneficiaries, mentors, vocational training centers, company partners, donor representatives (assistance coordinator, project officer, etc…).

Quantitative Survey – A survey will be undertaken on a representative sample of past beneficiaries to augment the qualitative data gathered in the interviews and focus groups to help determine the longer-term impact of the programs.
All data will be disaggregated by sex and age. While the evaluation team will be responsible for analyzing the collected data and explaining the findings in the evaluation report, they will also provide Mercy Corps Tunisia with access to the raw data.

Consultant Deliverables:

The evaluation will be led by an external Lead Evaluator who will be supported by Mercy Corps Tunisia program and M&E staff. The Lead Evaluator must have strong French language skills, as well as skills and experience designing and conducting focus groups, key informant interviews, and surveys for performance evaluations. While Mercy Corps Tunisia staff will help the Lead Evaluator by, for example, providing access to key program documents and existing M&E data, and facilitating access to beneficiaries, the Lead Evaluator will be responsible for designing and conducting the evaluation, and writing the evaluation report.

Report Structure & Content

The performance evaluation report will be written in English and should be around 10 pages not including annexes. It will follow the formatting and style of the attached “Mercy Corps Long Report Template.” It will be organized as follows:

• Cover Page
• Table of Contents
• Executive Summary: This section should be a clear and concise stand-alone document that gives readers the essential contents of the evaluation report, including a summary of major findings. It should be “external facing” with the potential to be used as an advocacy document for donors to inform their development interventions in Tunisia. It should not exceed two pages.
• Results: This section should provide a clear answer to the evaluation questions based on gathered data, and should include charts and tables.
• Synthesis, Recommendations and Lessons Learned: This is space for the evaluation team to think about the data and results, and make concrete recommendations for current or future program improvements, pull out lessons learned, and generally comment on data and results. In addition to providing actionable recommendations for Mercy Corps programming in Tunisia, this section should also extrapolate best practices and recommendations to donors for how to structure and implement effective development interventions in Tunisia. Everything presented in this section should be directly linked back to the information presented in the Results section of the report.
• Annexes: These should include the Methodology (which should be sufficiently detailed to help the reader judge the accuracy of the report and its findings); Limitations (which should address constraints and limitations of the methodology, and the implications of these limitations for the findings, including whether and why any of the evaluation findings are inconclusive); a complete file of data collection instruments in French with English translations; any data sets (these can be provided in electronic format), and any other documentation deemed appropriate.

In addition to the report, two 2-page documents will be developed (one for each program approach) that will serve as “program approach factsheets” for external audiences. The factsheets will explain the approach, highlight evidence supporting the approach, and provide an illustrative “success story” giving readers a concrete example of the approach’s impact, as well as lessons learned.

Timeframe / Schedule:

It is expected that the evaluation will be conducted over a period of 1.5 months.

The Consultant will report to:

Director of Programs