Tool for Agroecology Performance Evaluation

Livestock and agroecology

Livestock are distributed throughout the globe and provide a multitude of products and services, including meat, milk, eggs, fiber, hides and skins, natural fertilizers, fuel, transportation, and drought power. They are kept by over half of rural households and are crucial to livelihoods, nutrition, and food security. When managed sustainably, they can contribute to vital ecosystem functions, such as nutrient cycling, soil carbon sequestration, and the conservation of agricultural landscapes. Furthermore, they can enhance livelihoods and incomes.

The contents of Steps 0 to 3 are given below, which outline the TAPE approach along with the main phases to follow.

Step 0

Preparation and contextualization

Discovering the questionnaire: TEST version 

If you want to have a preview of the questionnaire and do a TEST without starting a formal survey follow the link to have access to the Kobo digital version (https://ee-kobo.fao.org/x/0lc8wiMW).

You can fill in the form for step 1 and step 2 and have access to the automated results of step 1 (step 2 results are not available on the Test version).

Preparing a formal survey

Before starting a survey at large scale please check if you have already exchanged with the TAPE team and shared your objective through this form  https://ee-kobo.fao.org/x/Eicyb5k0 to gather the necessary guidance.

Identify study objectives

1. If you want to make some adaptations to the questionnaire or if you are ready to start surveys ask FAO (Tape@fao.org) for a username and password to:

● access the questionnaire, 

● submit the survey data,

● access to the automated results visualization,

● access your dataset,

● have your data considered in the fao global data set.

2. Review the project or initiative you want to assess and specify the objectives of using TAPE: project design or one-time assessment/ capacity development/ monitoring, evaluation/ impact assessment

3. Identify key research questions

4. Finalize your methodology to help answer the study objectives, e.g. quantitative methods (surveys), qualitative methods (FGDs, KIIs), or mixed methods 

5. Define your budget and your team and initiate the scoping for possible support from organizations and individuals (consultants, enumerators, research coordinators)

Engage stakeholders & assess context

1. Involve relevant stakeholders for a comprehensive understanding of the agroecological context (social, economic, environmental) and the objective of the study

2. Initiate the diagnostic phase to understand the context and key factors influencing the agroecological transition

Study groups, farm pre-typology & sampling

1. Identify pre-defined study groups, if any (beneficiaries vs. non-beneficiaries, by geographical location, etc.) and farm pre-typologies (e.g., small rice paddy farms, members of farm association or not, etc.)

2. Determine the sample size and define a sampling strategy (e.g. multi-stage stratified random sampling) and conduct sample selection

Customize the TAPE questionnaire and access to Kobo

1. Adapt the questionnaire to the project and context: a new version Adaptation of step 1 criteria and thresholds and select relevant performance indicators in step 2 (one by dimension minimum)

2. Translate into local language (as needed).

3. Ask for admin access in the FAO TAPE KoboToolboxto to incorporate the approved changes. Create an account  and login https://kobo.fao.org/accounts/login/using the information provided (username and password).

4. Download Kobo Collect App (Android devices) and ask for username and password for each enumerator to organize the surveys.

5. Complete the standard FAO metadata form (link to be provided) with all methodological details and sign the data-sharing agreement to enable inclusion of your data on the global platform.

6. Test the questionnaire for any possible technical issues.

Enumerator training & pilot survey

1. Prepare supporting documentation and digital tools (tablets/Android phones).

2. Train enumerators, including carrying out a collective dry run of the questionnaire.

3. Conduct a pilot survey with trained enumerators and organize a final debriefing and Q&A session

4. Set-up a communication channel for daily feedback between the research coordinator(s) and enumerators during the data collection process.

Download

Step 1 and 2

Surveys and data management

Conduct the survey & ensure Quality Control

1. Implement the survey ensuring a well-informed prior consent of contributors (enumerators, respondents, FAO) on the agreement in the use of the TAPE tool (included in the questionnaire) and provide first feedback on the results.

2. Monitor the data collection on a regular basis to ensure data quality and address emerging challenges (at local level and FAO level if needed)

3. Provide preliminary data collection feedback to the local implementation leader or FAO as necessary

Organize, process, analyze and visualize the data

1. Conduct preliminary data analysis and interpretation with data cleaning and harmonization from the Kobo form file followed by rigorous analysis.

2. Data processing & analysis, perform indicator calculations, data visualization and comprehensive statistical analysis, including complex data modelling, if necessary. 

An automated visualization is available in real time on a dedicated link (Tabeau) for step 1 and 7 core indicators of step 2. 

To go further, R and Stata codes (to be updated)  for indicator calculation, data validation, data analysis and visualization are available on Github (GitHub - MAUS-team/tape_calculator) 

Report preparation

1. Prepare a draft report summarizing the background and objectives of the study, Step 0 contextualization, methods, and initial results.

Step 3

Participatory analysis and policy dialogue

Participatory discussion of the results and report finalization

1. Participatory analysis: Facilitate workshops with farmers representatives’ stakeholders for validating and interpreting initial results

2. Engage with stakeholders in a participatory manner to identify barriers and drivers of agroecological transitions

3. Finalize the technical report, share with stakeholders and decide concrete actions to improve farmers livelihood, project interventions efficiency or

Policy discussion and actionable recommendations

1. Develop communication material like policy briefs, 2-pages report and short presentation for policymaking stakeholders 

2. Identify actionable recommendations based on the evidence generated and the stakeholder’s feedback

3. Dissemination of data and results at local, national, sub-regional, regional and possibly global level to inform evidence-based policymaking

Choose a project method

All BidCarbon Standard Scheme projects are required to choose a project method and collaborate with TAPE to execute their project. TAPE is a comprehensive tool specifically designed to evaluate the multidimensional performance of agroecological systems across various sustainability dimensions.