SELF ASSESSMENT INFORMATION SYSTEM FOR SUSTAINBLE SOY PRODUCTION LESSONS LEARNT IN PARANA

Pieter Sijbrandij, Violaine Laurens, Stefan Lanz and Rui Valena Solidaridad /Gebana / October 2012
Summary of the most relevant lessons learn on the development, application, report generation and sharing of the Self-Assessment guide for soy family farmers, as tested and implemented in the southwest of Paran State, Brazil in 2011/12 by the joint project of Solidaridad/Gebana/Coopafi.

Development Application Data processing Reporting Sharing Conclusion

On development:
The development of the guide takes more time and effort than planned. It requires a diverse set of skills which implies in the involvement of several people. The design of the methodology had to cope with some basic requirements: application in maximum two hours, self-applicable for secondary school education level, cost effective through standardizing, facilitating mass processing and covering regional complexity. So the people involved should know about soy in general and mixed farming systems in the south of Brazil in specific, be able to communicate in the language of local family farmers, understand about building questionnaires and formulating the right questions, have insight in the possibility for data processing, understand the economics of mixed family farming and its drivers and option, have detailed knowledge of (shifting) environmental legislation, have knowledge of health and safety regulations, know the set of consolidated labor laws, be familiar with the RTRS and other certification systems, their principals, criteria indicators and guidance. Bringing this set of knowledge and abilities into a room creates a vibrant community, not always easy to manage but a cross-learning encounter for all involved. Translating subjects into
Lesson Learnt / October 2012 Self Assessment Information System

questions proved more difficult than imagined. Splitting up as issue in four different development levels provoked a lot of discussion. Reaching an agreement on how to address sensitive issues took several rounds. As different aspects had to be re-done a couple of times and in smaller groups, the first version was seen as a great achievement to be tested in the field.

On application:
The four Gebana/Coopafi field extensionists trained did a very good job in reaching out to the local farmers. True, a good part of them was already involved in the project and had visits planned and issues to be checked an discussed anyway. But it still is a heck of a task to work with 378 farmers and register carefully the information. Due to the education level of about half of the participating farmers, the application was a close interaction process in which the field extensionist worked patiently with the producer and sometimes his/her family members though the self-assessment guide. Though planned for taking maximum two hours, the guide proved an good instrument for discussing several matters. Not registered, but frequently mentioned, a single application led to hours of discussing and sometimes even visiting parts of the property. During the application also some methodological errors became apparent. Of these errors especially two types had direct implications in the processing of data. The first, hard lesson, was that aiming for natural speech in formulating the question, some questions had been phrased in the negative. Now, having only the option yes/no as an answer the respondents became confused whether they should respond: NO I Dont (double negative, like is usual in Portuguese) , or YES I dont (affirmative that I am not into it). This phrasing a question led to the exclusion of some questions and to the exclusion of one issue (child labor). The second methodological error was that in re-doing and revising the set of questions for each issue and attempting to clean it up, in some cases the questions left would, independent of the answers chosen, not enable to differentiate between all the levels. That is, in some cases no identification could be mad whether the producer would score level 1 or level 2. In two cases this resulted that the respondent was scored in the higher level. So, of the application we learnt that it is more difficult to ask a question the right way, but that when done with respect and patience, the self-assessment is a great instrument to engage the farmer into reviewing his economic, social and environmental situation, the starting point for looking forward. Another important learning is that for low level education farmers there are basically two options: assisted application like the we tested this time, or a more audio-visual way for which other material should be developed (with nice videos and an application via iPad). As to the higher educated (that is secondary school complete or higher) the self-assessment can be completed individually on paper or, in the near future, on-line.
Lesson Learnt / October 2012 Self Assessment Information System

On data processing:
With 378 questionnaires and around 200 data per questionnaire the amount of data to be processed was tremendous. No need to say that some computer-power and brains had to be allocated. Luckily this intelligence was available within project partner Gebana who used some wiz-kids of its units in Switzerland and the Philippines to They set up easy data input screens and wrote the formulae for converting the raw data in individual scores, linking the individual scores to pre-written information and aggregated scores for the group analysis. A daunting task which was with limited means very well executed. Data processing was hampered by some late incoming inputs, the methodological error mentioned above and the not always clearly described expected output. Although not perfect, the results were very satisfying and with a couple of months delayed the group report and individual reports were produced. The process of translating of the information into data processing led to a description of the basic methodology in English with as precise definitions as possible.

On reporting:
The consolidation of the group report has taken considerably longer than expected. While the data processing led to aggregated numbers, the interpretation of these numbers required quite some discussion. The discussion was basically spurred by the hard data produced by the system and the lack of recognition of some of these data by those who know the reality. It was this discussion that led to identifying some of the weaknesses of the original design. Only once understood why the data were wrong, the related issue or question was kicked out. Although this created some gaps in information and answers (like on child labour), it made the designer team understand better the complexity of their proposal. With this hurtful kicking out of some data, the tough test was to verify whether the results of the self-assessment guide matched other existing data. This has been verified in three ways: one by verifying with the smallholders representative whether the group data coincided with their view of the local situation. The second was using the internal control system of participating Gebana suppliers to check the validity of individual reports. The third has been contrasting the output with the external audit as realized by Control Union in August of 2012.

Lesson Learnt / October 2012 Self Assessment Information System

Although this checking has not been done with statistical analysis, the conclusions of the team were satisfactory. This implied that the methodology has likely validity and that the assisting to the application did not lead to significant distortion of the information.

On sharing:
The aggregated results have been presented to over 200 farmers during the Feira de Melado event. Coopafi farmer representatives, local government, regional university and state rural extension agency all were present. In talks afterward it was confirmed that the data pointed out fed well into the local agenda setting. As the sharing event has been during the month prior to local elections and in a relatively small community, the project partners agreed no public debate was to be held at that moment and feedback will be sought after the elections have taken place. Individual reports are handed out only to the applicant him/herself and explanation is given when requested. As to date the feed-back on the individual report is limited. The couple of farmers the team has talked to mentioned they recognized this was they talked about, agreed with the gaps indicated but did not commit directly to closing this gaps.

End conclusion:
The methodology is promising but not yet at a level for public sharing. Its development and its application is more labor, time and resources intensive than originally projected. The project partners have decided to apply for an extension of the project and in this extension would like to work on an improved version so this can be shared with a broader public.

Lesson Learnt / October 2012 Self Assessment Information System