Project Effectiveness Review
AMAL: Supporting Women’s Transformative Leadership in Middle East in MENA
As part of Oxfam GB’s Global Performance Framework, the Project Effectiveness Review will rigorously assess the impact of this project on the people it is intending to support. Impact will be evaluated particularly against Oxfam’s global indicator for women’s empowerment. Data will be captured through the administration of questionnaires, both to project’s participants and to comparable non-participant.
The consultant will train, supervise and manage a team of enumerators to carry out the data collection. The data collection will consists in a standard quantitative questionnaires and a micro-narratvie questionnaire with self-signification component (SenseMaker). The consultant will work especially to ensure that the data collected meets Oxfam’s high quality standards. The data analysis will be carried out by Oxfam’s impact evaluation team. The Oxfam team will be responsible for the overall methodology, but will work in close collaboration with the consultant (as well as Oxfam’s programme staff and the partner organizations) to ensure adequate sampling, identifying the comparison group, and create the questionnaire.
Scope of Work / Key Tasks
- Review project documents and discuss the project with Oxfam and partner staff, to understand the details of the project objectives and its implementation.
- Work with Oxfam staff to identify indicators and design a quantitative and SenseMaker questionnaire suitable for evaluating the project’s impact on women’s empowerment.
- Co-facilitate together with Oxfam staff a one-day workshop with a sample of project participants in order to define the characteristics of women’s empowerment in Tunisia.
- Recruit a team of experienced enumerators for the data collection. Enumerators need to be all women.
- Lead a three-day training workshop for enumerators. This will include two days working through the paper version questionnaires and the use of the mobile devices/tablets and one day of field testing. (including arranging all logistics, such as finding venue, printing, stationary, etc)
- Organise all of the logistical arrangements necessary to carry out the field work, including transport and accommodation for enumerators, arrangements with local authorities, and contacting and planning in advance the interview with the selected women.
- Help identify a valid comparison group of individuals that were similar to the project participants before it started, but who did not participate in the project.
- Prepare a list of sampled intervention and comparison women (500 approximately) and a list of replacement women to ensure sufficient sample size, in collaboration with Oxfam and project partners.
- Manage the survey work in the field, ensuring that the enumerators interview the correct respondents that surveys are carried out to a high standard, and that targets for numbers of respondents to be interviewed are met.
- Arrange the translation of both questionnaires (quantitative and SenseMaker) into the local language.
- Alongside the individual quantitative interviews, introduce and conduct the SenseMaker exercise.
- Translate in English the SenseMaker stories, and perform data entry.
- Conduct a socialisation exercise presenting summary statistics of the estimates collected.
- Consult Oxfam’s Impact Evaluation Adviser on any problems that affect the selection of respondents for the survey or the number of respondents to be interviewed.
- Write a brief report on the conduct of the field work, including details of any problems which were encountered and how they were resolved, and any feedback which may be useful for future surveys.
- Please note that the consultant is NOT expected to conduct data analysis
Oxfam has a strong preference that the survey is conducted using mobile devices or tablets. It is therefore expected that the consultant will:
- Work with Oxfam’s Impact Evaluation Adviser to build the mobile survey and upload it onto the mobile/tablet devices.
- Manage mobile/tablet devices ensuring each device is signed in and out at the beginning and end of each data collection day and that they are safely stored each night. (Supervisors will take responsibility to charge devices and to upload the data collected every evening.)
- Ensure that the data files are uploaded and transmitted to the server every night.
In case data collection is done with traditional paper version the consultant is also expected to:
- Manage the data-entry process, ensuring that each questionnaire is entered twice by two different data entry clerk. Oxfam’s advisor will produce a report identifying the discrepancies between first and second entries. The consultant is expected to perform the reconciliation for these entries. Please note that the data-entry interface will be provided by Oxfam.
- Ensure that the data files are submitted online following the guidelines provided, ensuring adequate encryption and using an adequate file sharing system (i.e. Box) on completion of the data-entry work.
- Successfully completed (high quality) questionnaires of 500 interviews (230 beneficiary women and 270 comparison women). Digital data needs to be successfully uploaded on the online serves. (If needed, the physical questionnaires have to be delivered to Oxfam in Tunisia office.)
- A brief report on the conduct of the field work, including description of the sampling strategy for both beneficiary and comparison Individuals, details about any problems which were encountered and how they were resolved, and any feedback which may be useful for future surveys.
- A brief report reporting the findings from the workshop.
- Training materials enumerator/supervisor training
- A finalised individual questionnaire (English and local language)
- A finalised detailed field work planning, indicating 1) the number of individuals surveyed on each day, 2) the precise location of each of these households, 3) the survey team responsible for each of these surveys.
- A finalised budget clearly indicating the allocation of the budget to the various field work components.
- Receipts of all payments made in the context of this review (see below)
- Flexbility to move in different regions of Tunisia to monitor the interviews
- The assignment is expected to take approximately 25-30 working days to complete which includes days for training of enumerators and field work.
- Preparation for the survey is planned to take place from Wednesday 23th November.
- Training of the enumerators is planned to take place from Wednesday 29st November
- Data collection should take place between Friday 2 December and Friday 23 December.
Line Manager : Simone Lombardini – Oxfam’s Impact Evaluation Adviser
Proposals are invited from organisations and individual consultants who meet the profile described above. The proposal should outline the consultant’s profile (including their CV/resume), proposed management arrangements, and examples of previous work.
Proposals must also include a clear itemised budget. Any proposals that do not include an itemised budget will not be considered. Please use Oxfam template.
The budget submitted by the consultant should include:
- Consultant’s fees and subsistence for approximately 25-30 days of work.
- Fees for a team of enumerators to carry out a questionnaire of 500 individuals, of approximately 60 minutes’ duration each.
- Use of a training venue and of any equipment required.
- Transport and subsistence costs for the teams of enumerators.
- Translation of questionnaire into local language, where appropriate.
- Printing of training materials.
- Any other costs anticipated in completing this assignment.
- Contact details for the consultant.
The gross amount of 30% will be paid up-front (to cover all logistical costs + a share of the consultancy/survey team fees), whereas the remaining amount of 70% will be paid upon successful completion of the assignment (i.e. once all deliverables have been submitted to and approved by Oxfam). This budget must be allocated to the various data collection inputs according to a budget which will be prepared by the consultant in close collaboration with and approved by the Oxfam team.
Receipts of all payments made in the context of this review need to be kept by the consultant and submitted to Oxfam for verification of whether expenses are made according to the agreed budget.