Ensuring rigour from a single researcher study.

H.K. Crawford ¹, H. Jenkins², R. B. Murray-Prior².

1 Current address: Agriculture Western Australia, Bunbury

2 Curtin University of Technology

 

Abstract

Qualitative research has often been accused of lacking rigour. However, developing explicit frameworks and procedures can negate this accusation. A study of how farmers learn to incorporate agrosilvopastoral farming systems into their farm management practices is being conducted by interviewing 15 farmers over three years. To ensure and maintain high levels of rigour frameworks were developed prior to commencing interviews. Regular meetings were held with supervisory staff to ensure that the framework was valid and to adjust for any problems encountered along the way. Each interview was analysed before returning to the farm for the next. The qualitative data collected about the phenomenon of farmer learning was analysed using NUD.IST using an approach suggested by Strauss, comparing the data to literature as well as within their context. This was then checked with the farmers. The researcher and the farmer build their understanding of farmer learning in conjunction. Clear documentation of the research process will show the rigour in this project. Understanding of the learning process will be the result of many conversations with the participants; it is up to the reader to decide whether they agree with that interpretation.

Introduction

International, national and regional economic pressures mean that agriculture in Australia has entered a period of rapid change (Sorensen 1991; NSW Standing Committee on State Development 1993; Bamberry, Dunn and Lamont 1997). The awareness of the need for sustainable farming systems, increased biodiversity and environmental protection are increasing the pressure for change to agricultural production systems (Prinsley 1991; Davidson and Davidson 1992; Vanclay and Lawrence 1995; Bamberry, Dunn and Lamont 1997). Furthermore, farmers are under greater pressure from climatic variations as the effects of El Nino weather patterns increase the frequency of drought in some areas (ABARE 1995). It is also likely that climatic variability and uncertainty will continue to change (Pittock 1996; Nicholls 1996). In order to cope with the increasing level of change, farmers need to learn about a diverse range of topics and how to manage increasingly complex and more difficult and variable environments or ‘resource bundles’. As we do not know how they learn, or what type of learning is taking place, a study focusing on finding out how farmers learn, what their learning processes are and the links and relationships with decision making is being carried out in Western Australia. It also aims to develop a ‘model’ of farmer learning.

The study is significant as legislative and structural changes within government agencies that support agriculture, conservation and natural resource management are likely . However, farmers may be able to adopt more sustainable systems without the need for government regulation. Agricultural research and extension is also affected by this change. Top-down extension methods are giving way to more holistic and participatory models. These will need new learning environments that foster the interaction between researchers and farmers. Extension agents must become more aware of how farmers learn to enable them to redefine their roles and to help facilitate the most appropriate learning environment.

This paper outlines the methodologies considered appropriate for studying farmer learning, and details the research process used. It concludes by proposing that although we are using elements of accepted qualitative research methodologies the rigour of this study is guaranteed by the explicit reporting of how it was carried out . We also suggest that irrespective of what qualitative methodology we are using, we are simply asking the farmers questions, analysing and interpreting what they tell us and reporting our conclusions about what this means. This is supported by .

Available Methodologies

A qualitative methodology was recognised as the best way of obtaining the in-depth data necessary to achieve the objectives of this study. The methodologies assessed for their applicability were: Phenomenology - as learning is a phenomenon, Ethnography - as farmers are a ‘culture’ and Grounded Theory - as the empirical evidence supplied by the farmers to be used to develop a model of farmer learning.

As Phenomenology is education based, descriptive in nature, and uses the participant’s experiences to describe what is being studied it is appropriate. Ethnography is also appropriate as it is used to study cultures and is descriptive in nature. Grounded theory, particularly that espoused by , links the empirical evidence to theory and previous research.

has suggested that each qualitative study is liable to be unique and, therefore, is likely to require a unique methodology. We accept this assertion that the methodology will evolve and accept that before the study is complete it may involve elements of other qualitative methodologies. Initially it was accepted that the model of farmer learning would be built out of the empirical evidence obtained from the farmers; this is Grounded Theory. There is also an element of observation involved as the research is carried out on the farmer’s properties and what they do and often how they do it is seen during the research process; we see this as having its roots in Ethnography. Also as much of the writing will involve detailed descriptions of the phenomenon of learning on individual farms as told by the farmers; this is Phenomenology.

Methods

Although surveys, historical research and case studies were all applicable, case studies were considered the most appropriate at answering the question ‘How do farmers learn’ ; ; . As earlier studies (Crawford, 1996; Crawford & Leybourne, 1997) had suggested that farmers learn about how to make changes on their farms over time, a variation of the multiple-case case-study method was selected as the design for the research. A longitudinal study was considered the most appropriate variation to use. This variation has meant that each of the individual cases will be studied concurrently rather than cyclically as suggested by and . As farmers who were making changes to their farming systems were considered to be learning they were thought to be an appropriate group to study. Cases were defined as: "A farm family who owns or manages a property or properties and who in the last five years has introduced agrosilvopastoral systems into their farming system."

Research Method

Key informants from Agriculture Western Australia, The Department of Conservation and Land Management and Landcare Officers provided names of farmers who fitted the selection criteria. Farmers were selected by phoning those on the lists supplied by the key informants. The first 20 to agree to be a part of the study were selected. This number was chosen to allow for natural attrition during the course of the three-year study. The selection process was not so much concerned with obtaining a ‘representative’ sample such as one would obtain for a survey as with obtaining a group of farmers from a wide variety of farming regions who would be able to tell us about how they learned.

A rapport building session was held with farmers before beginning the research as suggested by . This involved visiting each farmer to tell them about the research and eliciting their co-operation for the three years of the study. Following this short visit, usually 1 to 1½ hours duration, 17 farmers commenced the study. One farmer declined to take part, one farmer did not fit the criteria and the third was unavailable on a number of occasions.

A first interview collected demographic information about the farm and the farming enterprises, as well as information about the farmer, their goals and their farming philosophy. This enabled the researcher to fully contextualise what the farmers were telling him about as well as increase the overall understanding of the farming system. The researcher had a ‘working knowledge’ of the farming systems employed by these farmers but it was important, I in order to fully understand the environment in which the phenomenon of learning took place, to have considerable detail about each farming system.

In three cases a farm tour could not be carried out and in two others full interviews were not able to take place, as the farmers were involved with more urgent farm work. This was not considered a problem as the interviews and the farm walks could take place at a later date. The interviews and associated farm visits took between three and five hours. As this was essentially an interview to collect background demographic information and details of changes that had been made very few questions were asked about learning processes as it was felt that increasing the time on each farm may jeopardise the participation rate later in the study. Where time permitted some information was obtained about the changes that had been made to their agrosilvopastoral system and the reasons for making the changes. This information was collected as the reasons for making changes were thought to be triggers which would stimulate a learning episode.

Prior to the second interview the transcripts of the interviews were typed and a draft description of each farm was prepared. Farmers were sent a copy of their farm description to check for errors.

Although the second interview, three to four months after the first, was a continuation of the first, it was the first to focus specifically on the phenomenon of learning. However, one of the problems associated with changes to farming systems is that they sometimes occur very slowly. Therefore, to make sure farmers were able to find changes to discuss this interview focused on changes made first to the agrosilvopastoral system, second, the farm as a whole, third, the production systems and lastly changes they had planned for the future. A checklist of questions was used to guide the interview through these four areas. To understand more about their learning processes farmers were asked to talk about the information they needed, how, and where they gathered that information and how they processed that information to make the changes. Catch up farm walks and details missed from the first interviews were clarified and the farm description was checked and corrected.

Between the second and third interviews the case descriptions were refined and returned to the farmers for final checking, and the first two interviews were analysed using the NUD.IST program. A base set of codes was formulated using information from the literature and the interviews. It is here we follow the Straussarian view of Grounded Theory and created cursory links to the literature. However, to avoid being unduly influenced by existing learning theory, and to capture the true essence of the farmer’s learning phenomenon, we started with only four over-aching codes, information source, information collection method, motivation or triggers for learning, and information processing.

The text of each interview was broken down into sentences as the most appropriate unit of analysis and where appropriate allocated to an existing code or a new code was created if there was no appropriate code. Phrases used by farmers were often used as code descriptors so the models built for each farmer would reflect how the saw their learning. Breaking the conversations into sentences proved to be the most difficult task. The reason for this is we do not naturally converse in a grammatically ‘correct’ manner. This makes punctuating conversations problematic. Sentence length therefore varies from a few words to many lines of text that often contain a number of different linked ideas. Despite the difficulty of determining where one sentence ends and another begins NUD.IST allows researchers to allocate data bits (in this case sentences) to a number of different nodes. This greatly increases the researcher’s ability to link ideas within conversations.

As one of the most common learning episodes seemed to take place at field days attended by farmers, the third interview focused on that subject. During this interview farmers were first asked what they understood by learning. This was done so any changes in farmer understanding of learning during the course of the study might be measured. To obtain information about how farmers learned from these field days they were then asked to reflect on the latest one they had visited and to ‘talk’ through the day. The guiding questions for the interviews were based on the assumption that farmers would have visited one of the recent machinery field days. This, however, proved incorrect, as many farmers had not done so. Therefore, they needed to be slightly modified when farmers had attended field day events other than machinery days. These included farm field walks, focus farm days, better business courses and trial field days.

Interview three was then analysed in the same way as the previous two. The three interviews were then ‘combined’ using NUD.IST and each farmer’s data summarised using a framework based on the processes of learning identified from the data. These summaries were then combined to form a more generic idea of farmers’ learning processes. During this process it was concluded that some of the data seemed to be superficial and that more ‘depth’ was needed to fully understand how farmers learned. Part of the problem was due to the difficulty that some participants had in reflecting on and talking about what they were doing. It was hoped that each interview would build on earlier conversations. However, it appeared that farmers were finding it difficult to articulate what they were learning and how they were linking this with the decisions needed to make changes to their farming systems. Most farmers found it very easy to talk about what they had done during a field day. For example, what stands they had visited, what they had seen, whether it was relevant to their farming system, but found it more difficult to talk about what value they had seen in a piece of technology and how and why it might contribute to changes to their farming system.

To overcome this problem, and to get farmers to talk about different aspects of their learning behaviour, individually and as a family, it was decided that each farm family would be asked to complete a Myers Briggs personality Type Inventory. This was done to allow a conversation to stimulate a conversation based on the results. The conversation asked about whether the members of the farm family recognised themselves in the questionnaire’s results and whether the results reflected the way that information collection, decision-making and learning took place on the farm and within the family. The results of the questionnaire may be interesting for further analysis in another study.

Farmers were also asked to comment on what they saw as the greatest challenge to their farming system and how they were going to handle that as well as what they felt was unsettling them the most at the time of the interview. These questions were asked in order to obtain information about what might trigger the next learning episode which would be followed up during later interviews.

At the time of writing these interviews are being transcribed and made ready for analysis. It is proposed that one final interview will be held to complete the cycle and follow any changes that have been made between the fourth and fifth interview. It is hoped that a meeting of all participants will be held within 18 months to bring them up to date with preliminary findings and to elicit their opinions as to the accuracy of the interpretation of their learning processes.

Comments to date

It is probably not prudent to comment on any of the ‘results’ of the study at this juncture. However, it is appropriate to comment on how the method used has been adapted during each stage. Changes are made to interviews as different questioning techniques are tried and discarded. Farmers were never interviewed in the same order so questions missed during a previous interview can be covered later. In some cases interviews have been delayed when farmers are unable to find time to be interviewed. Although there was an increased effort required recalling field day visits it did not cause any major problems. The only ‘problem’ that has been found is that the males were not able to ‘talk’ through their field trip days as easily, or as well as were the female participants.

The original methodologies are still being followed as the case descriptions provide the descriptive context of the phenomenon, and the ethnographic portions still apply to the cultural aspects of farming. Data analysis allows the empirical grounding of the data in the conversations as the farmer and researcher create the meaning of farmer learning on each farm.

It might be possible to call the emerging methodology Agrophenomenology or something else to reflect the diversity of the situation we are studying. However, following the lead taken by , who suggested that many qualitative researchers have postured behind unnecessary terminology, we prefer to say that we are presenting a description of how farmers see themselves as learners, how they learn and deriving a model of this learning, with their assistance. We are also of the firm belief that irrespective of what methodology finally emerges from this study, our explicit reporting of what we have done creates the ability to repeat the study, and this is what gives the study rigour.

References

ABARE. (1995). Australian commodities: forecasts and issues. 2 [2]. Australian Bureau of Agriculture and Resource Economics. Canberra.

Bamberry, G., Dunn, T. & Lamont, A. (1997). A pilot study of the relationship between farmer education and good farm management. RIRDC Research Paper No 97/30. Rural Industries Research and Development Corporation. Canberra.

Brule J F Blount A (1989). Knowledge acquisition. New York, McGraw-Hill Publishing Company.

Crawford H.K (1996). How seasonal dairy farmers in the lower North Island of New Zealand achieve high per cow production - A participatory case-study. Palmerston North, Massey University.

Crawford K. Leybourne M. (1997). A pilot study of productive farm systems which address rising water tables and salinity in Western Australia. R G C Mineral Sands Limited State Landcare Conference: Who Dares Wins - Act Now, Geraldton, Western Australia, Soil and Land Conservation Council.

Crotty M. (1997). Phenomenology as a way of life. unpublished.

Davidson R. Davidson S. (1992). Bushland on farms: Do you have a choice?. Canberra, Australian Government Publishing Service.

Eisenhardt K M (1989). Building theories from case study research. Academy of Management Review 14(4): 532 - 550.

Leybourne M.L (1997). Natural resource conservation in Western Australia: Legislative and structural change. Landcare changing Australia, Adelaide, Primary Industries South Australia.

Leybourne M.L. Crawford H.K. Arnott A. Benson R. (1999). Ensuring rigour from a multi-site, multi-discipline, multi-researcher study. Issues of rigour in qualitative research, Melbourne, Association of Qualitative Research. These proceedings

New South Wales Standing Committee on State Development. (1993). Regional business Development in New South Wales: Trends, policies and issues, Discussion Paper No. 4. Parliament of New South Wales. Sydney.

Nicholls N. (1996). Climate variability - Climate change. The impact of weather and climate on Agriculture, The proceedings of the second Australian conference on agricultural meteorology, Brisbane, University of Queensland Printer.

Pittock A.B. (1996) Adapting agriculture to climate change: A challenge for the 21st century. The impact of weather and climate on Agriculture, The proceedings of the second Australian conference on agricultural meteorology, Brisbane, University of Queensland Printer.

Prinsley R.T. (1991). Australian agroforestry: Setting the scene for future research. Canberra, Rural Industries Research and Development Corporation.

Ragin C.C. Becker H.S (1992). What is a case? Exploring the foundations of social inquiry. Cambridge,, Cambridge University Press.

Strauss A .Corbin J. (1994). Grounded theory methodology: an overview. Handbook of qualitative research. N. K. Denzin. Y. S. Lincoln. Sage, Thousand Oaks, California: 273 - 285.

Sorensen, A. (1991). Misleading impressions: The recession and the farm sector and its impact on rural Australia. Policy. 7 [3]. pp. 19 - 24.

Vanclay F. Lawrence G. (1995) The environmental imperative: eco-social concerns for Australian agriculture. Rockhampton. Queensland Australia, Central Queensland University Press.

Williams P. Gardner J. Gray D. Kemp E. Kuiper D. (1997). Rapport building in New Zealand farm management consultancy. Farming in the information age, University of southern Queensland, Toowoomba, Queensland, Australian Farm Management Society Ltd.

Wolcott H.F. (1992). Posturing in qualitative inquiry. The handbook of qualitative research in education. eds. M D LeCompte; W L Millroy; & J Preissle. San Diego, Academic Press Inc: 4 - 52.

Wright A (1963). The development of unploughable hill country. Palmerston North, Massey University.

Yin R K (1994). Case study research: design and methods. Newbury Park, California, Sage Publications.