Positioning qualitative and quantitative methods: Applications to crime in tourist settings
Mervyn S. Jackson
&
Diane Mead Niblo
Department of Psychology and Intellectual
Disability Studies
RMIT University
PO Box 71
Bundoora 3083
Abstract
In a current Psych-Lit survey for the years 1993 - 1998, 578 papers were identified and classified as using the qualitative approach as the principal method. Approximately half of these papers (249) were theory only with 55 (or 9.5%) of the articles focussing on the debate regarding the relationship between qualitative and quantitative approaches. The radical position within this debate highlighted the different assumptions and conceptual frameworks of each approach and concluded that researchers should not mix methods within each of the paradigms. At the other extreme, researchers focussed on how to use different methods to answer practical complex research questions and de-emphasized epistemological differences. A third strategy was to select either qualitative or quantitative research as the principal method and then use the other approach as a complementary method before or as a follow-up to this principal method (Morgan, 1998). This research paper will evaluate each of these positions and then proposes revisiting a more traditional model to describe the relationship between paradigms (Wallace, 1971). A (new) time-series alternating paradigm model will be outlined and illustrated using continuing research in the relatively unexplored area of crime in tourist settings.
In any discussion regarding positioning or combining qualitative and quantitative research approaches, an appreciation of the fundamental differences between the paradigms is necessary (Guba & Lincoln, 1994; Morgan, 1998; Reichardt & Rallis, 1994). The basic philosophical assumptions of the qualitative approach include humans as: having free will, rule-using psycholinguistic (cognitive) beings, playing an active role in their own experiences and their behaviour being only understood from their own unique cultural / developmental / linguistic framework. In contrast, researchers from the quantitative perspective assume human behaviour is determined by (discoverable) causes, humans are rule-governed, passive beings whose behaviour can be reduced to universal laws of nature.
The qualitative researcher takes a cooperative approach toward informants and holds the basic optimistic assumption that people will grant access to "scientific" researchers and be as truthful as possible. The qualitative researcher-informant relationship includes the following characteristics: empathetic; trusting; equalitarian; intense; and long-term. There is also a need for disengagement at the end of the research. In contrast, quantitative researchers have a negative, pessimistic outlook toward subjects, believing people either do not know the truth or choose to cover-up the truth to present the best impression. Thus quantitative researchers use covert methods to discover the truth and often include truth testing items. The researcher-subject relationship includes: power; control; structure; evaluation; and involves only short-term, detached, formal, standardised contact throughout the study.
In terms of selection and design, both paradigms hold conflicting views. Qualitative researchers use non-probability sampling (that is, theory or data driven) and they focus on selecting a small number of key informants who provide a representative picture of knowledge about the research focus. In contrast, quantitative researchers use probability sampling to collect relevant, objective information from a large sample. Selection is validated by sampling theory and yields a sample of people who represent relevant features (age, gender, ethnicity) of the total population. Interestingly, an increase in a qualitative sample may decrease validity as the study begins to include informants who are not central to the research question. In quantitative research, increase in sample numbers increases the representativeness of the sample in regard to the population and allows more powerful statistical tests to be used. Further, such an increase will also increase both the reliability and generality of the findings.
The design of qualitative research is flexible and evolves through direct observation / data collection. Hypotheses, procedures, and measurements all arise as a consequence of preceding data collection and the research question is one of the products of data collection. In contrast, quantitative research is based on well-informed literature reviews, research design arises from such reviews with hypotheses, procedures and measurements all decided upon before any data collection begins. Quantitative research design is a detailed plan of operation, with very predictable research outcomes.
Qualitative data analysis is on-going, uses analytical induction processes, and relies on constant comparison of themes and concepts until models and theories evolve. Quantitative data analysis, in contrast, occurs at the end of data collection, uses statistics to deduce significance, with statistical significance driving theory confirmation or rejection. While the methods and measurements vary, qualitative researchers typically use fieldwork, participant observation, intensive interviews, case studies and life histories. In contrast, quantitative methods are experiments (or at least quasi-experimental) and involve inventories and standardized questionnaires.
The major disadvantages of each approach seem to be the strengths or the advantages of the other. Qualitative research has difficulty studying large populations, is time-consuming with masses of observational data that are difficult to conceptualise. Procedures and measures are not standardised and therefore there are issues with reliability and replicability. However, qualitative research has strong validity as it investigates key informants on cognitions relevant to the area of research interest. Quantitative research is invasive, obstructive and creates an artificial setting, it has difficulty trying to control both extraneous and error variables and as a consequence quantitative research has problems with the many types of validity. However, with standardised measuring instruments and sound sampling procedures, quantitative research is perceived as highly reliable with high generality to the target population.
Such analyses have led some researchers (eg., House, 1994) to conclude that neither of the paradigms is fully adequate. The qualitative / interpretivist tradition correctly highlights that behaviour can be only understood from the informant’s interpretation of social reality (within their cultural context), but neglects external (to the person) causes and conditions and intra-personal conflicts / contradictions. On the other hand, quantitative / positivist approach traditionally emphasises causal links but errs in the belief that complete understanding (via scientific deductions) can occur without extratheoretical or extraobservational judgements within the particular context. Both Toulmin (1982) and House (1994) argue that complete research must take account of both the cognitions of informants (processes) and the causes of social practice (outcomes). House (1994) asked the question "Why the paradigm war"?" and placed the debate within an historical context and found the debate began by early quantitative (behavioural) research making an uncompromising claim that they had developed a methodology that was "value-free", "politics-free", and "trouble-free". House (1994) argues that researchers opposing these extravagant claims were forced into the opposite extreme (they could not be quantitative, could not be objective; and had to admit the possibility of personal and professional biases). While the adherents to both paradigms continue to focus on extremes, no compromise or integration appears possible (House, 1994). Further, highlighting the limitations of the other tradition increases stereotypic views and allows each paradigm to underestimate the flaws of their own tradition. Reichardt and Rallis (1994) suggest that as first steps toward re-integration, researchers should ignore the limitations of the other tradition, identify and work toward minimising the flaws of their own tradition and accept that while the qualitative and quantitative traditions differ, both have a common research purpose: to understand and improve the human condition. In conclusion, there are substantial fundamental differences between qualitative and quantitative research paradigms.
A number of researchers (Barbour, 1996; Creswell, 1994; Gantley, 1994; Guba & Lincoln, 1994; Leininger, 1994; Morgan, 1998; Smith & Heshusius, 1986) have indicated that the two approaches are not compatiable and that they should not be combined due to the fundamental conflict between the two paradigms in regard to the nature of knowledge, the relationship between inquirer and the object of inquiry, and the appropriate means of generating knowledge. In contrast, other researchers ( Datta, 1994; Greene, Caracelli & Graham, 1989; Hedrick, 1994; Reichardt & Rallis, 1994; Yin, 1994) postulated that it is possible to integrate both approaches. These researchers present valid points on the following issues: many of the supposed incompatibilities are illusory (eg., nature of reality); many specific assumptions can be widened and become common to both approaches (ie., world is complex and stratified); the complexity of the many different factors influencing behaviour require such a multi-paradigm approach; both approaches emphasise scientific rigour and critical analysis; both approaches have been used for years and both have contributed to the understanding of human behaviour; and most current researchers consider themselves equally balanced between the approaches. Finally, each paradigm agrees that good studies include testing all evidence, testing all competing hypotheses, and achieving results that have significant implications.
Hedrick (1994) concludes that many issues of difference are really about practicalities, not paradigms (eg., impossible to do large scale cost-effective national evaluations using the qualitative approach). In fact, Morgan (1998) suggested that many authors who advocated operating within one paradigm (eg., Creswell, 1994; Gibloe-Ford, Campbell, & Berman, 1995; Guba & Lincoln, 1994; Smith & Heshusius, 1986) concede the possibility of combining qualitative and quantitative methods but are critical of such research for failing to acknowledge the existence of, and address the fundamental differences between, the research paradigms. Those who are primarily interested in the technical aspects of combining theoretically different methods have concluded that this can be achieved without violating basic paradigmatic assumptions (Morgan, 1998; Reichardt & Rallis, 1994; Riggen, 1997). A number of researchers (Brannen, 1992; Brewer & Hunter, 1989; Bryman, 1984, 1988; Cook & Reichardt, 1979) perceive this as a technical challenge which will be ultimately resolved (Morgan, 1998).
In summary, the major approaches of combining qualitative and quantitative methods agree that it is possible, and most proponents of combining methods suggest that after addressing fundamental paradigmatic differences the barriers to combining approaches are essentially technical / practical. The next step is to determine the major motivations for combining qualitative and quantitative approaches.
There appear to be three major reasons / motivations for combining qualitative and quantitative methods: the iterative nature of research; convergence / confirmation; and complementary needs. The iterative nature of research (Barbour, 1996) positions the research process as an unclear sequence of procedures that interact within the conceptual and empirical world with inductive and deductive reasoning occurring simultaneously. That is, the two paradigms reflect two aspects of research that already have combined in the minds of researchers as they develop strategies to answer socially relevant questions within the socio-cultural context. Thus, researchers who report all phases of their research inquiry will already have combined inductive and deductive reasoning and therefore should use both qualitative and quantitative methods to completely explore the emerging issue.
The convergence / confirmation motivation was first mooted by early education researchers (eg., Campbell, 1988; cited in Anastasi & Urbina, 1997) who suggested that combining different methods (eg., the multi-trait, multi-method, research strategy) allowed the researchers to cross-validate results. That is, such researchers perceived that the ultimate goal of research was to achieve convergence and confirmation of results across different methods. Researchers were expected to conduct at least two different research studies to achieve the same results, thereby demonstrating the outcomes were not an artefact (or lack of validity) associated with a particular method. This expectation lost influence for two reasons: the failure of most research to achieve the same result using different methods (Chesla, 1992); and the problem of cost-effectiveness, where applied researchers do not have the time, effort, skills and resources to research until they have achieved the same result twice (Morgan, 1998). While these concerns are valid, such attempts at convergence / confirmation should not be rejected or ignored. A reason for using multiple methods of different paradigms would be to explore the extent to which different methods elicited different data from the same respondents. This inherent contradiction would enable researchers to make more subtle and sophisticated analyses of these differences in an attempt to explain them (Barbour, 1996).
The complementary needs motivation (Greene et al, 1989) is based on using the strengths of one method to enhance the performance of another method. For example, while qualitative research has good validity, it has been critically questioned by quantitative researchers for problems of reliability and generality. In contrast, quantitative research has always been perceived as able to demonstrate reliability and generality of findings, but has been criticised by qualitative researchers on the validity (or interpretation) of outcomes. Carefully combining both qualitative and quantitative approaches, a mixed method research paradigm can theoretically achieve the strengths of both paradigms (that is, the whole is more than the two parts combined). The ultimate aim of this is to develop a valid (qualitative) study with demonstrated reliability and generality of (quantitative) findings.
Morgan’s (1998) approach acknowledged the importance of the two different paradigms and recognised the importance of exploring differences in the pursuit of philosophical understanding. Morgan (1998) also acknowledged combining qualitative and quantitative methods without a clear understanding of these differences would indeed be a futile and risky business. However, Morgan (1998) contended that once a researcher had developed a clear framework for a research project, then the selection of methods (even mixed methods) was largely a technical task.
Morgan’s (1998) priority-sequence model attempts to address the complementary needs aspect of the qualitative - quantitative debate. Morgan (1998) indicated that for many research projects, a single well-selected principal method (qualitative or quantitative) will be entirely sufficent. Where the project would benefit from multiple research methods, Morgan (1998) suggested the use of a priority-sequence decision process. Therefore, the first research-design decision is to determine priority. That is, the researcher must decide which approach (qualitative or quantitative) will be the principal data collection method. This is decided upon by selecting the data collection method which has the strengths that are most important to the project’s goals. Then, the researcher selects a contrasting complementary method which offers a set of strengths that can add to the research design’s overall ability to meet the project’s goals. The second research design decision is to determine the order or sequence of the two methods. That is, the researcher must decide how to connect the different methods to maximise their contributions to the success of the overall project. The sequence decision depends on the goals of the research project. If the goal is to improve the effectiveness of the data collection of the principal method, then the complementary method should come first. If the goal is to maximise the value of the collected data, then the complementary method should follow the main data collection. This two step decision-making process leads to four major research designs:
The major contentions about the priority-sequence strategy for combining qualitative and quantitative methods is that half of the combinations raise the political issues about the relative standing of the two approaches (eg., qualitative research followed up by quantitative research indicates that qualitative results must be treated as tentative and need to be confirmed by quantitative research). The major limitation of the priority-sequence strategy is the middle option; that is, giving qualitative and quantitative data equal priority and collecting all data simultaneously. Such a project would support both complementary needs and convergence / confirmation. Morgan (1998) rejects this as too ambitious (in terms of time and effort needed) and too risky (having to explain different or contradictory findings). There are three other limitations. Firstly, it is linear (and not iterative) and therefore does not include the reality that research develops using both inductive and deductive processes simultaneously. Secondly, it has only minimal convergence / confirmation in that only one of the four possible strategies has an overlap of data collection (eg., a qualitative study following a quantitative study to explain unexpected outcomes), while the other three possible combinations use the paradigms separately (eg., using qualitative research in a preliminary study to ascertain the sample for a large scale quantitative study). Thirdly, the priority-sequence strategy is time-limited and once the decision is made, the research process is inflexible.
Wallace (1971) proposed a scheme where qualitative and quantitative research are completely integrated. Wallace’s model involved two sets of overlapping processes: theory development and doing research; and inductive and deductive reasoning. These processes are related in a cyclical stage. Wallace (1971) proposed researchers should begin researching a new (emerging) area of concern with non-structured (qualitative) observations which would involve inducing generalisations and / or estimating population parameters. From these empirical generalisations (and through qualitative data reduction and concept formation techniques), a theory about the emerging phenomenon would be established. At this point, the quantitative approach would be initiated where the researcher would deduce consequences and make predictions from the theory. Finally, hypotheses are formulated, measuring instruments designed, samples randomly selected and quantitative research completed. Wallace’s original model emphasised convergence / confirmation, but minimised both complementary needs and the iterative process (the model being static and time-limited).
Jackson’s (1999) evolutionary research model highlights the strengths of Morgan’s (1998) complementary needs model and Wallace’s (1971) static cycle model, but adds, in a complex manner, the notion of an iterative process. The evolutionary model assumes the following: research is on-going and there is no end, theoretically, to the research process; that at any one point in time, there is a need to think both inductively and deductively about the research project (that is, an iterative process); that the research question (process versus outcome) determines which research approach should be implemented; at the end of a phase of the research, either qualitative or quantitative approaches (or both) may be needed to continue the research process; that qualitative and quantitative research should show convergence of findings (qualitative processes explaining quantitative outcomes); combining qualitative and quantitative research should lead to complementary, methodological solutions (qualitative validity combining with quantitative reliability and generality).
The Jackson evolutionary model uses a time-series strategy and its use and parameters can be explored using a series of studies in the emerging research area of tourism and crime. Jackson and others (Jackson, White & Schmierer, 1994, 1996; Jackson & Schmierer, 1996, 1997; Jackson & Niblo, 1997) began to research the relatively unexplored area of psychology of tourism. While there was one book (Pearce, 1982) and a few research studies (eg., Pearce & Catalbiano, 1983) in the area, these contributions relied on applying general psychological principles to tourism (eg., Pearce, 1982) or adapting traditional psychological theories to predict / explain industry generated survey-type data (the 1983 Pearce & Catalbiano study applying Maslow’s theory to tourists). However, at the time, there had not been any in-depth, tourist-specific research into how individual’s perceive their tourist experiences. Thus, without a substantial body of empirical literature, unknown research parameters, and yet-to-be-formulated research questions (Jackson et al, 1994) completed and reported a series of studies. The first study was a qualitative study of a set of key individuals. From these interviews / observations and using analytical inductive techniques, Jackson et al (1994) concluded that tourists appeared to bias their interpretations regarding tourist experiences and they perceived they were disempowered by the tourist experience. This led to the deductive hypothesis that individuals (in general) may systematically bias their understanding of tourist experiences. Jackson et al (1996) next completed a large quantitative survey (N = 464) using a stratified sample and analyzed the results within Heider’s (1958) attribution framework. The results indicated that tourists were crediting themselves for positive tourist experiences (eg., ability to plan, effort to complete trips) but attributed negative tourist experiences to others (other tourists, hosts, tourist industry). Through deductively reasoning of quantitative data, Jackson et al (1996) concluded that this self-enhancing self-protective bias meant that tourists perceived they had less control and were unlikely to change behaviours to avoid negative (problem) experiences.
It is at this point in the research process that the evolutionary model of research differs from both Morgan (1998) and Wallace (1971). These approaches infer research is finite and time-limited. However, it is at this point where researchers use both inductive and deductive cognitive processes to position future research. As a consequence of this iterative process, two streams of research occurred. A qualitative research study was used to explore the process which tourists used to interpret tourist experiences and the linkage between these cognitions and type of experiences. A quantitative research study (via deductive thought) arose from applied general psychology research that attribution bias is limited to individualistic cultures (eg., most Westernised cultures) and therefore does not apply to collective cultures (eg., non-Westernised cultures). Jackson, Inbakaran & Shilvaskar (1999 in press) have completed a quantitative comparative research (using India as the collective culture) which partially supports the attribution theory. Qualitative research at this time (Jackson et al, 1994) determined the major negative experiences were health- and crime-related. These issues which were explored using the analytical inductive reasoning led to the concept of a tourist (with less perceived control) being away from the known, safe home environment. At this point, previous mainstream criminology research highlighted an environmental theory of victimisation. A quantitative paradigm was thus created with the (deductive) hypothesis that as tourists move further away from their home (environment), the more likely they are to become victims of crime. Jackson and Schmierer (1996) collected survey data on tourist crime victimisation, time spent on activities (work, home, leisure, tourism) and the outcome of any criminal victimisation. Jackson and Schmierer (1996) reported two findings. First, tourists are more crime prone (compared to their home environment). Second, tourists reported that the further thye were from home, the less likely they were to report crime to the police. Again, at this point, both inductive and deductive cognitive reasoning occurs. Future qualitative research will investigate why tourists do not report crimes while future quantitative research evaluates whether crime is associated with the psychophysical distance from home, or whether tourist environments are inherently more dangerous (Jackson & Schmierer, 1997).
References
Anastasi, A., and Urbina, S. (1997). Psychological testing. (7th Ed.). New Jersey: Prentice Hall.
Barbour, R. (1996). Mixing qualitative methods: Quality assurance or qualitative quagmire? . Qualitative Health Research, 8, 352 - 361.
Brannen, J. (1992). Combining qualitative and quantitative methods: An overview. In J. Brannen (Ed.). Mixing methods: Qualitative and quantitative research. Brookfield: Avebury.
Brewer, J., and Hunter, A. (1989). Multimethod research: A synthesis of styles. Newbury Park: Sage.
Bryman, A. (1984). The debate about qualitative and quantitative methods: A question of method or epistemology? British Journal of Sociology, 35, 75 - 92.
Bryman, A. (1988). Quantity and quality in social research. Boston: Unwin Hyman.
Chesla, C. (1992). When qualitative and quantitative findings do not converge. Western Journal of Nursing, 14, 681 - 685.
Cook, T., and Reichardt, C. (1979). Qualitative and quantitative methods in evaluation research. Beverly Hills: Sage.
Creswell, J. (1994). Research design: Qualitative and quantitative approaches. Thousand Oaks: Sage.
Datta, L. (1994). Paradigm wars: A basis for peaceful coexistence and beyond. In C. Reichardt and S. Rallis (Eds.). The qualitative quantitative debate: New perspectives. San Francisco: Jossey-Bass.
Gantley, M. (1994). The qualitative and the quantitative: An anthropological perspective on research methods. Critical Public Health, 5, 27 - 34.
Gibloe-Ford, M., Campbell, J., and Berman, H. (1995). Stories and numbers: Coexistence without compromise. Advances in Nursing Science, 18, 14 - 26.
Greene, J., Caracelli, V., and Graham, W. (1989). Toward a conceptual framework for mixed-method evaluation designs. Education Evaluation and Policy Analysis, 11, 255 - 274.
Guba, E., and Lincoln, Y. (1994). Competing paradigms in qualitative methods. In N. Denzin and Y. Lincoln (Eds.). Handbook of Qualitative Research. Thousand Oaks: Sage.
Hedrick, T. (1994). The qualitative-quantitative debate: Possibilites for integration. In C. Reichardt and S. Rallis (Eds.). The qualitative quantitative debate: New perspectives. San Francisco: Jossey-Bass.
Heider, F. (1958). The psychology of interpersonal relations. New York: John Wiley & Sons.
House, E. (1994). Integrating the quantitative and qualitative. In C. Reichardt and S. Rallis (Eds.). The qualitative quantitative debate: New perspectives. San Francisco: Jossey-Bass.
Jackson, M. (1999). Integrating qualitative and quantitative approaches: An evolutionary model. Seminar presented to the Department of Hospitality, Tourism and Leisure, RMIT University.
Jackson, M., Inbakaran, R., and Shilvaskar, S. (1999 in press). Individual versus collective cultures: The use of an attributional framework for tourism experiences.
Jackson, M., & Mead Niblo, D. (1997). Crime and the tourist: A research agenda for industrial and organisational psychologists. Interface, 21, 11 - 15.
Jackson, M., & Schmierer, C. (1996). Tourism and crime: More crime but less reporting. In G. Prosser (Editor) Tourism and Hospitality Research: Australian and International Perspectives. Canberra: Bureau of Tourism Research.
Jackson, M., & Schmierer, C. (1997). Behaviour analysis of the urban environment: Linking city design to crime against tourists. Paper presented at the 20th national conference of the Austalian Association for Cognitive and Behaviour Therapy, July 1997, Brisbane.
Jackson, M., White, G., & Schmierer, C. (1994). Crime and the tourist: Tourist perceived causes and implications for prevention.In R. Homel and N. Nash Proceedings of the Second Annual Crime Prevention Conference. Griffith University, Brisbane.
Jackson, M., White, G., & Schmierer, C. (1994). ' The return from Xanadu': A qualitative and quantitative analysis of tourist experiences generated from a large Melbourne sample. In T. Veal & B. Weiler (Eds.). Leisure in Australia. Sydney: ANZALS.
Jackson, M., White, G., & Schmierer, C. (1996) Tourism experiences within an attributional framework. Annals of Tourism Research, 23, 798 - 810.
Leininger, M. (1994). Evaluation criteria and critique of qualitative research methods. In J. Morse (Ed.). Critical issues in qualitative research methods. Beverly Hills: Sage.
Morgan, D. (1998). Practical strategies for combining qualitative and quantitative methods: Applications to health research. Qualitative Health Research, 8, 362 - 376.
Pearce, P. (1982). The social psychology of tourist behaviour. Oxford: Pergamon.
Pearce, P., and Caltabiano, M. (1983). Inferring travel motivation from travellers’ experiences. Journal of Travel Research, 22, 16 - 20.
Reichardt, C., and Rallis, S. (1994). Qualitative and quantitative inquiries are not incompatiable: A call for a new partnership . In C. Reichardt and S. Rallis (Eds.). The qualitative quantitative debate: New perspectives. San Francisco: Jossey-Bass..
Riggin, J. (1997). Advances in mixed methods evaluation: A synthesis and comment. In J. Greene and V. Caracelli (Eds.). Advances in mixed methods evaluation: The challenges and benefits of integrating diverse paradigms. San Francisco: Jossey-Bass.
Smith, J., and Heshusius, L. (1986). Closing down the conversation: The end of the qualitative-quantitative debate among educational inquirers. Educational Researcher, 15, 4 - 12.
Toulmin, S. (1982). The construal of reality: Criticism in modern and post-modern science. Critical Inquiry, 9, 93 - 111.
Wallace, W. (1971). The Logic of Science in Sociology. Chicago: Aldine-Atherton.
Yin, R. (1994). Evaluation: A singular craft. In C. Reichardt and S. Rallis (Eds.). The qualitative quantitative debate: New perspectives. San Francisco: Jossey-Bass.