Direct and cross-examination of qualitative rigour
Missy Morton
160 Quinns Road, Christchurch, New Zealand
Paper presented at, Issues of Rigour in Qualitative Research, Conference of the Association for Qualitative Research, Melbourne, Australia, 8-10 July, 1999.
In this paper I describe a qualitative study that came to examine legal and scientific discourses of validity and reliability, including the expert status of qualitative research and qualitative researchers. I use feminist and poststructural analyses to examine the discourses used to support and to challenge claims about facilitated communication (fc), a non-speech communication strategy for people who have been labelled 'intellectually handicapped'. From its earliest use, questions were raised about the authenticity of the communication, as well as the means of its authentication. Most of the early published research on the phenomenon of fc was qualitative research. When allegations of abuse were purportedly made using fc, the Child Protective and legal gaze was turned on individuals using facilitated communication, people who teach and/or research the use of fc, as well as the phenomenon of fc itself. I propose to examine the complex and competing discourses that frame our understandings of what it means when the legal system begins to address the question of the validity of facilitated communication.
Introduction
My interest in my thesis, and in this presentation, is the discursive production of science in the legal forum. In my thesis, I look at how the qualitative investigation and reporting of a particular phenomenon was received by professionals who traditionally used quantitative approaches, in particular experimental research, in their own studies. The subsequent debates about the utility of research approaches and any subsequent reporting of findings were played out in various professional and research journals. Eventually these debates also made their way into family and criminal courts in the US. It began to seem as if, at every turn, the only way to answer important questions was through a science of experiment and control, through which reliable predictions could be made, a one right answer found. If large scale experiments were not required, then at least an approach which mimicked experimental, controlled analysis on an individual level was required. I don't wish to argue here that there is never a place for investigation through experiment, what some have described as a natural science approach. Rather, I'm interested in tracing how such an approach comes to be seen as the approach.
Background
Ten years ago Douglas Biklen, a Professor of Special Education at Syracuse University, was in Melbourne. He was engaged in a qualitative project – participant observations and interviews with a number of people with disabilities, their families, and professionals who worked with them. He subsequently published his research in the Harvard Educational Review. He described how these individuals with disabilities were communicating using a technique called facilitated communication, and how this facilitated communication was described by the people with and without disabilities. He described earlier controversies surrounding the approach; how people wondered whether or not people with disabilities such as autism could really be communicating. He concluded that we might have some re-thinking to do, about the ways we understood disability and communication, the meanings of labels such as 'autism' and 'retardation' (Biklen, 1990).
This was controversial stuff and responses were fast. A response to Biklen's original article was published in a later issue of the same journal (Cummins & Prior, 1992). The criticisms were typical of others (e.g. Schopler, 1991). Biklen's (1990) and others' reports of facilitated communication (e.g. Biklen, Morton, Gold, Berrigan & Swaminathan, 1992; Crossley & McDonald, 1980) flew in the face of fifty years of received wisdom on autism; reports of literacy and intelligence seen in people with autism simply do not fit the facts. Further, qualitative research is not capable of making a contribution to our understanding in this area:
[i]t has no utility for testing ideas. In this context Biklen has used an inappropriate method to investigate the issues under discussion.…nor does the ethnographic approach that has been used shed light on the operational characteristics of this phenomenon [of facilitated communication] (Cummins & Prior, 1992, p.240).
This debate has been repeated in later professional journals in the field of disability. In the Special Topic portion of an issue devoted to facilitated communication, the editors of the Journal for the Association for Persons with Severe Handicaps, Halle, Chadsey-Rusch, & Riechle (1994) set out what they considered to be the boundaries of the discourse: "The primary controversy surrounding facilitated communication is whether or not it is a valid and reliable method of communication for people with disabilities." Biklen & Duchan (1994, p. 189) challenged Halle et al’s delimiting of the discourse of facilitated communication.
We have argued that the controversy is not just about whether or not particular individuals are authoring their own messages, nor is it about whether the method is successful or not. It is not about what percentage of people can be proven competent, or about the percentage who have achieved or will achieve independent typing. It is not about whether or not "developmental dyspraxia" or "movement disorders" can account for the successes of those using facilitated communication. It may be about ethics, but only to the extent that ethics are related to the more fundamental issues at the heart of the debate.
Rather, "at the heart of the debate," are questions about the nature of science and the nature of mental retardation. The authors of two of the articles that followed in the Invited Commentary never-the-less remained firmly committed to a positivist view of science as an endeavour that produces incremental knowledge. They did not address directly the issue of the nature of mental retardation, but assumed that it is real and will be solved by the application of appropriately derived techniques and strategies (Kaiser, 1994; Whitehurst & Crone, 1994). While Horner (1994) observed that many would-be facilitators would rely on their own experiences to determine authorship, he also resisted framing the debate as "just another debate about how to do research." He described how facilitators should regularly test themselves, either formally or informally (using what has been described as the message-passing approach) as a means of scrutinising their own influence. Horner concluded that "[t]he results from these personal experiences will hold the real future of facilitated communication" (p.186). Thus the debate about facilitated communication has become a forum for contesting epistemologies, which many of the authors wish to push to one side as, at best, an interesting but not terribly relevant debate.
Contesting epistemologies
Biklen's (1990) study drew on a tradition of questioning the assumptions dear to the traditions of many professionals in the disability field. Bogdan & Kugelmass (1984) have previously challenged researchers in the area of disability to look at the assumptions they bring to their research. These assumptions are:
(1) disability is a condition that individuals have;
(2) disabled/typical is a useful and objective distinction;
(3) special education is a rationally conceived and coordinated system of services that help children labelled disabled;
(4) progress in the field is made by improving diagnosis, intervention and technology (p. 173).
These assumptions have their origins in what some have described as an ideology of segregated control, grounded in the science of control. Various authors trace the rise of this science through eugenics, and through the universities, legitimating the authority of science and the rise of the professionals. In their recent articles, Kliewer & Drake (1998), Radford (1994) and Sullivan (1996) all describe the relationships between the academy and the asylum as essential to the creation and legitimation of a "dogma of mental deficiency" (Radford, 1994, p.10). Within these segregated settings were developed the specialised knowledges and techniques foundational to the practices and claims of authority made by the disability professionals. There has been a traditional reliance on segregated and incarcerated populations for the development of techniques, the design and execution of experiments, the publication and dissemination of results and the concomitant professional status and authority based on 'science'.
Critiques such as those above are a long way from winning the day with respect to practice, research and publication in the disability field. The medical discourse of disability, disability as deficit inherent within the individual, is still the dominant discourse.
Despite the academic debates and civil rights movements of disabled people and their allies, the medical model has not been dislodged to any significant degree. Equally, the role of educational psychologists remains dominant in the assessment and educational career of children with ‘special educational needs’ (p. 33).
I turn now to the playing out of these debates in the legal context.
Science in the Court
The practice of the law and the practice of science are both constructed and re/presented as the objective pursuit of the discovery of existing facts – as an examination of those facts, the evidence. In general, the courts deal with new ideas and new approaches by inviting the input of consultants and experts.Judges make decisions about when experts are needed to assist in the explanation or interpretation of facts, who can be an expert and about what, and which expert interpretations will be given preference. When prosecuting attorneys sought to introduce statements made by children using facilitated communication, purporting to be disclosures of abuse, defense attorneys sought to preclude those statements by invoking the Frye standard.
The original hearing, Frye v US, was decided in 1923. The Supreme Court was asked to decide whether or not evidence obtained by polygraph (lie detector) should be admissible. The Supreme Court ruled that such evidence was not admissible. The standard that was applied concerns the general acceptance of a scientific technique or theory within the relevant field or scientific community. Since the 1923 decision, Frye hearings have been held on the admissibility of expert scientific testimony as related to DNA testing, hypnosis as an aid to recall for victims or witnesses, and the use of validation interviews to determine the presence or absence of child sexual abuse syndrome or rape trauma syndrome as corroborating evidence. A Frye hearing is generally requested by the defense attorney to preclude admission of evidence.
Two family court judges in New York state made the first rulings in the US where allegations were made by individuals using facilitated communication (Matter of Jenny S, Matter of M.Z.). In each case, a young woman had told her facilitator that she was being abused by a family member. (In each case the facilitator was, by law, a mandated reporter – mandated to report on suspicion of abuse.) The allegations were investigated by Department of Social Services and law enforcement agencies. The reports were indicated and petitions were filed. The results of these investigations were never able to be put before the court for the judge to rule on, so they do not appear in either judge's decision. At issue in each hearing was whether or not any person's statement, if made using facilitated communication, should be admissible in court. By choosing to rule that Frye applied, the judges neither heard nor ruled on the merits of the evidence obtained through the investigations that were conducted; nor did they rule on the specific abilities of the two young people in question to communicate by facilitated communication.
The expert evidence proffered by disability professionals, on the appropriate means of conducting research in this field, and to the contributions of such research to the understanding of disability went directly to the issue of acceptance demanded by Frye. The expert witnesses cited newly emerging "controlled" studies that "provided incontrovertible proof" that the people with disability were not really communicating – that facilitated communication is solely the communication of the facilitator (albeit unconsciously so - see Wheeler, Jacobson, Paglieri and Schwartz, 1993; also Moore, Donovan, Hudson, Dykstra and Lawrence, 1993).
When the facilitator could not see what the person with disability was asked to describe, communication usually broke down. When the facilitator could not hear what the person with disability was asked to type, communication invariably broke down. Such results clearly indicated that there was not general acceptance in the relevant scientific community as to the acceptability of facilitated communication. In both cases, the judges determined only that the standard of Frye had not been reached. Under such a decision, the ability of an individual to validate his or her own communication is not considered.
In contrast to the Frye hearings, in each of the three cases described below, the question that needed to be answered was whether or not the specific individual before the court or grand jury was able to use facilitation to testify.
In the Matter of P. (anonymous), Luz
The New York State Supreme Court Appellate Division: Second Department ruled on March 29, 1993 in the Matter of P. (anonymous), Luz that the Family Court of Orange County had "legally erred" in requiring a Frye hearing. Rather, the justices wrote:
The test for the court in cases such as these is a pragmatic one. Can the interpreter, or in this case the facilitator, effectively communicate with the witness and reliably convey the witness's answers to the court? A determination of these questions does not require expert testimony. To the contrary, the proffered facilitated communication lends itself to empirical rather than scientific proof. Thus, the test proposed by the County Attorney, whereby the court could question Luz outside the presence of the facilitator and then hear her responses through facilitated communication, should adequately establish whether this is a reliable and adequate means of communication by Luz.
The justices distinguished this case from the Matter of Jenny S. which sought to introduce out of court statements made through facilitated communication:
Moreover, even as to out of court statements, Matter of Jenny S. appears to be of questionable value. The better position, in our view, is set forth by our colleagues in the Appellate Division, Third Department, in Matter of Marshall R. (73 AD2d 988). In that case the Appellate Division reversed the dismissal of a child abuse and neglect petition because the Family Court ruled that an out of court statement made by a six-year-old deaf child could not be testified to by an interpreter for the deaf. In language that is equally applicable to this case, the court observed:
"The best interests of the child are far more important than some technical objection, which on this record, appears to have little substance. The testimony of the interpreter should have been admitted, and then it would become the duty of the court to weigh and evaluate such testimony in the light of the circumstances under which it was given."
The justices ordered that the case be remitted to family court, where the judge would determine whether or not Luz' facilitator was reliably transmitting Luz' testimony. "If the court is not convinced that the facilitator is reliable then that facilitator may not serve as the interpreter. However such a finding should not foreclose Luz from testifying if a reliable facilitator can be found elsewhere." A number of tests were devised; Luz had to indicate the names of pictures that her facilitator could not see. She was able to do this, and the prosecutor wished then to call her to give testimony. The defense attorney objected and requested that the facilitator not hear the questions that were put to Luz, so the facilitator wore headphones. Luz was still able to communicate. The defense attorney again objected and requested that the facilitator not look when questions were asked, nor look where Luz was pointing to indicate her answers. At this point Luz could no longer communicate. The prosecutor requested time for Luz to be able to practice trying to communicate under these new conditions. The request was denied, the case was dismissed.
The People of the State of New York v. Dean Webb
Judge Eugene Nicandri was petitioned by the defense to dismiss the testimony before Grand Jury of a young boy using facilitation. During the young boy's testimony, the facilitator wore ear phones playing white noise and was unable to hear the questions asked of the boy. As Judge Nicandri noted, this case was distinguishable from both the Ulster County (Matter of Jenny S.) and Onondaga County (Matter of M.Z. et al.) because in this case the "... Grand Jurors actually saw the witness testify, saw the facilitator, saw the headphones, and saw and heard the machine produce sounds indicating responses to questions asked of the witness."
The parallel was drawn between this case and that of a person with severe cerebral palsy: "In allowing a speech therapist to interpret for a witness suffering from a severe case of cerebral palsy, one court recently observed that,
just because a procedure is unusual does not mean that it should not take place in a courtroom. The Courts today should make every effort to open their door to all who seek to come through them. We can no longer take the attitude that if it has not been done in the past, it should not be done in the future. The age-old stereotyping of people with physical or mental disabilities or a combination of both should be dispensed with as soon as possible. The Courts have come out of the dark ages with respect to the deaf and hearing impaired, and we should likewise do so with respect to other physical and mental disabilities..."
In July of 1993, this case was one of twelve cases in St. Lawrence County that were dismissed because the District Attorney had exceeded the 180 day limit allowed between charging an individual with a crime and presenting a case to the Grand Jury (the other cases did not involve facilitated communication).
The State of Kansas v. Marc Warden
On March 30, 1993, Marc Warden was convicted by a jury of sexually abusing a 12 year old boy, "JK" (Randall, 1993). "JK" used facilitated communication to testify. During a preliminary hearing on admissibility, the judge ruled that Frye did not apply and that the jury would determine the credibility of the youngster and his testimony. Warden appealed the conviction on the grounds that Frye should have applied, and that there had been no assessment of "JK's" ability to communicate.
The Supreme Court reviewing the case found that
We are not persuaded that statements produced through FACILITATED COMMUNICATION are scientific evidence subject to the Frye test. FACILITATED COMMUNICATION is just what the name implies: a method of COMMUNICATION. Unlike the tests revealing that JK is autistic and mentally retarded, which require scientific interpretation of JK's skills and behaviors, FACILITATED statements require no scientific interpretation (Kansas v. Marc R. Warden, #70,377).
In this decision, the judges distinguish between assessing a communication strategy and standardised tests of an individual's intelligence and diagnostic category. On one hand the judges challenge the claims of the scientific community to stand in judgement; on the other they reify this right of the same group of professionals to label, define, constitute, the very same group of individuals. Having determined that Frye did not apply, the judges were nevertheless somewhat critical of the trial judge's decision to not adopt any of the protocols suggested in Luz, or even by Biklen and colleagues (Borthwick, Morton, Crossley & Biklen, 1992). The judges noted, however, that the jury had been able to see JK and his facilitator, and to assess the weight to be given to his testimony. They also noted that Warden had confessed twice, once to police, once to a friend, although these confessions were retracted at trial. The conviction was upheld.
Establishing credibility; re-establishing science?
In the Matter of P. Luz the County Attorney proposed that the facilitator not be present while questions were asked. The Appellate Justices determined that if one facilitator was not seen as reliable using this process, than another facilitator might be found. The Family Court judge should determine the credibility of the individual and her testimony. In People v. Dean Webb, the facilitator wore headphones playing white noise as a boy used facilitation to testify before the Grand Jury. In State of Kansas v. Marc Warden, the facilitator looked away as the youngster pointed to yes and no on a perspex board. The young man gave his testimony via closed circuit television. In the last two cases, the judge, jury or grand jury determined whether or not the individual using facilitation, indeed the facilitation itself, was credible.
A number of law review articles have been published following the decisions described above (Candelora, 1995; Dwyer, 1996; Luxton, 1994; Maurer, 1995; Phipps & Ells, 1995). They are critical of Frye as an appropriate standard, and instead argue from a discourse of rights, citing, for example, the Americans with Disabilities Act. Like the appellate decision in Luz, these reviewers point to the right of individuals to have someone act as an interpreter. There is a need to balance this right with the rights of the accused to ask "is this individual really communicating with facilitated communication?" Unlike Frye, which asks if the technique has reached a standard of acceptance so that we might predict whether or not people can communicate this way, taking an individual, case-by-case approach means that the court need only be concerned with the validity of a particular individual's communication. On the face of it such an approach would appear to be less restrictive. As can be seen from Luz, however, a judge may simply move to attempt to replicate the same "controlled experiment" approaches on an individual basis. Candelora (1995) describes this individual approach as "qualitative testing" suggesting there is still no little confusion about distinctions that might be made between quantitative and qualitative research methods. As Dwyer (1996) points out, no such requirements were made of Luz's parents and their Spanish interpreter. Maurer (1995) notes that it is ironic that those individuals who are arguably the most vulnerable, children with disabilities, should have a higher burden of proving their credibility and competence as witnesses than do the adults who might be accused.
However "individualised" the alternatives to Frye advocated in the Luz decision and by the reviewers above, they have chosen approaches that derive explicitly from experimental studies. Why does this seem to go unquestioned? I ask myself how I might react if I were accused of abuse – would I settle for investigative approaches premised on knowledge as co-constructed, partial, historically situated?
The sociological question …[is] how the hegemony of a discourse is established as it insinuates itself into the institutional arrangements of the social order. It is from these institutional bases that the hegemony of a discourse realizes itself as a practical system of power and a system of social control. (Gillian Fulcher, 1989, p.25; citing Beng Huat Chua, 1981, p.22. Emphasis in original.)
What intrigues me here is that it seems so reasonable to so many that, because of the urgency of legal proceedings, particularly allegations of abuse, we must have sound means of establishing the authenticity of a person's communication. I have no quibble with this. That our means of doing so should be fundamentally positivist, should be open to further interrogation.
References
Biklen, D. (1990). Communication unbound: Autism and praxis. Harvard Educational Review, 60 , 291 - 314.
Biklen, D. & Duchan, J. (1994). "I am intelligent": The social construction of mental retardation. Journal of the Association for Persons with Severe Handicaps, 19, 173-184.
Biklen, D., Morton, M.W., Gold, D., Berrigan, C. & Swaminathan, S. (1992). Facilitated Communication: Implications for Individuals with Autism. Topics in Language Disorders; 12, 1-28.
Bogdan, R. and Kugelmass, J. (1984). Case studies of mainstreaming: A symbolic interactionist approach to special schooling. In L. Barton and S. Tomlinson (Eds), Special education and social interests. London: Croom Helm.
Borthwick, C., Morton, M., Crossley, R., and Biklen, D. (1992). Severe Communication Impairment, Facilitated Communication and Disclosures of Abuse. Syracuse, NY: Facilitated Communication Institute. Reprinted in the TASH Newsletter, 19(5), 6 - 7.
Corbett, J. (1996). Bad-mouthing: The language of special needs. London: The Falmer Press.
Crossley, R. and McDonald, A. (1980). Annie's coming out. Melbourne, Australia: Penguin Books.
Cummins, R.A. & Prior, M.P. (1992). Autism and assisted communication: A response to Biklen. Harvard Educational Review, 62, 228-241.
Dwyer, Joan (1996). Access to justice for people with severe communication impairment. Australian Journal of Administrative Law, 3(2), 73-120.
Frye v United States, 293 F 1013
Fulcher, G. (1989). Disabling policies? A comparative approach to education policy and disability. Lewes: The Falmer Press.
Halle, J.W., Chadsey-Rusch, J. & Reichle, J. (1994). Editorial introduction to the special topic on facilitated communication. Journal of the Association for Persons with Severe Handicaps, 19, 149-150.
Horner, R. (1994). Invited commentary: Facilitated communication: keeping it simple. Journal of the Association for Persons with Severe Handicaps, 19, 185-186.
Kaiser, A.P. (1994). Invited commentary: The controversy surrounding facilitated communication: Some alternative meanings. Journal of the Association for Persons with Severe Handicaps, 19, 187-190.
Kansas v. Marc R. Warden, #70,377
Kliewer, C. & Drake, S. (1998). Disability, eugenics and the current ideology of segregation: A modern moral tale. Disability & Society, 13(1), 95-111.
Luxton, M.S. (1994). Facilitated communication for people with autism in the courts: Balancing the need for reliable evidence with the requirements of the constitution. Hamline Law Review, 18(2), 201-230.
Matter of P. (anonymous), Luz, N.Y.S. AD2d 92-07565
Matter of M.Z., 590 N.Y.S. 2d 390 (NY Family Court, Onondaga County, Sept. 16, 1992, Buck, J.)
Maurer, N.M. (1995). Facilitated communication: Can children with autism have a voice in court? Maryland Journal of Contemporary Legal Issues, 6(2), 233- 282.
Moore, S., Donovan, B., Hudson, A., Dykstra, J. and Lawrence, J. (1993). Brief report: Evaluation of eight case studies of facilitated communication. Journal of Autism and Developmental Disorders, 23, 531 - 539.
People of the State of New York v Dean Webb, (NY County Court, County of St. Lawrence, #92-193, March 26, 1993, Nicandri, J.)
Phipps, C.A. and Ells, M.L. (1995). Facilitated communication: Novel scientific evidence or novel communication? Nebraska Law Review, 74(4), 601-657.
Radford, J.P. (1994). Intellectual disability and the heritage of modernity. In M.H. Rioux & Bach, M. (Eds). Disability is not measles: New research paradigms in disability. Toronto: L'Institut Roeher Institute.
Randall, G. (1993a). Disabled boy gives landmark testimony. The Wichita Eagle, March 30, 1993, pp. 1A, 5A.
Randall, G. (1993b). Live-in aide convicted of sexual abuse. The Wichita Eagle, March 31, 1993, pp.1A, 8A.
Schopler, E. (1991). Informal review of Crossley's facilitated communication. Journal of Autism and Developmental Disorders, 21, 561-563.
State of Kansas v. Marc Warden, #92-CR1198, 18th Judicial District, Sedgwick County, Kansas.
Sullivan, M. (1996). Regulating the anomalous body in Aotearoa/New Zealand. New Zealand Journal of Disability Studies, 1(1), 9-28.
Wheeler, D.L., Jacobson, J.W., Paglieri, R.A. and Schwartz, A.A. (1993). An experimental assessment of facilitated communication. Mental Retardation, 31, 49 - 60.
Whitehurst, G.J., & Crone, D.A. (1994). Invited commentary: Social constructivism, positivism and facilitated communication. Journal of the Association for Persons with Severe Handicaps, 19, 191-195.