Grappling with Criteria in Constructivist Clinical Inquiry

Sue Price (Lecturer) and Alison Stewart (Senior Lecturer)
Department of Nursing and Midwifery,
Otago Polytechnic, Private Bag 1910, Dunedin, New Zealand.

Fax: 03 474 1957. Email suep@tekotago.ac.nz or ajwillems@xtra.co.nz

 

Introduction

How to attest to the quality and trustworthiness of qualitative research has been a perennial concern since the wide spread emergence of approaches to research that don’t adhere to scientific positivist traditions. Recent debates in health and education literature, continue to highlight many of the tensions that exist for qualitative researchers as they grapple with the thorny issues of how best to deal with these concerns (see for example:. We acknowledge that there have been many attempts from a variety of disciplines to address issues of quality and trustworthiness in qualitative research. These have been aptly described in a recent nursing paper by Emden and Sandelowski . This presentation, however, explores how member checking was used in two health research studies, using constructivist inquiry informed by Lincoln and Guba’s Naturalistic Inquiry and Guba and Lincoln in their later book Fourth Generation Evaluation . Reference to many of the other approaches to rigour outlined by Emden and Sandelowski (1998) will not be made here.

We take as a starting point in this presentation points that we are all familiar with:

And so how to establish credibility - member checking as one strategy

Member checking is a key element of credibility, and Guba and Lincoln (1989: 239) describe it thus:

The process of testing hypotheses, data, preliminary categories, and interpretation with members of the stakeholding groups from whom the original constructions were collected. This is the single most crucial technique for establishing credibility.

Member checking, as proposed in Naturalistic Inquiry , is a continuous process of checking throughout the inquiry process, e.g. that interview transcripts reflect what individuals wanted to convey and then to ‘test’ data, categories and conclusions with participants. The purpose being as Lincoln and Guba (1985) state:

If the investigator is to be able to purport that his or her reconstructions are recognizable to audience members as adequate representations of their own (and multiple) realities, it is essential that they be given the opportunity to react to them (p.314).

Whilst member checking is often an informal process undertaken as the research proceeds, Lincoln and Guba (1985) proposed a formal checking towards the end, with representatives of different groups in the research commenting and debating on the inquiry report. Even if there is not total agreement on the content, what should be agreed is that the "reconstructions are ‘fair’" (p.315).

Hence, this process of member checking forms part of the hermeneutic-dialectic circle as we both used it in our research of discharge planning and grandparent bereavement. So, participants edited transcripts to reflect the ideas and constructions they wanted to put forward, and by going round the circle several times for subsequent interviews and/or comment on categories and final constructions. We both set out on the path of constructivist inquiry adhering to criteria such as credibility and using strategies such as member checking, which at the time appeared to fit well with our intentions. However, as we both experienced, utilising member checking was not without issue.

As outlined above, member checking is used for a methodological purpose. We believe, in line with other researchers such as Sandelowski that there is another purpose, which links back to the idea of ‘fair’, presented by Lincoln and Guba (1985) above. That is, the moral imperative within the research, which is based on the premise of cooperative inquiry where no construction is valued higher than another. Which means using member checking as a process which reflects working in partnership with participants (stakeholders) to respect the gift which people have given in taking part. It is also about using member checking to avoid any unexpected surprises for participants about the content of the inquiry report. For example, in the final report participants may not be able to recognise their story or contribution, a concern also noted by Sandelowski (1993: 5) when she stated "members will inevitably look for themselves and their own reality in researchers’ accounts of their lives".

Given the often sensitive nature of health research, the ethical purpose of member checking was particularly important to us, because we had asked people to reflect and re-visit painful (or potentially painful) experiences relating to their health and lives. Yet balanced with the ethical purpose was the need to consider and reflect on the methodological implications and the guiding "rules" and "criteria" from which we were operating.

What happened when we tried member checking?

I was interested in exploring issues and concerns surrounding discharge planning and the transition experience from the acute hospital setting to home environments, utilising the perspective of 6 different stakeholders. Three interviews were undertaken with women who had experienced recent hospitalisation in the acute care setting. Three interviews were undertaken with nurses; one nurse from an acute hospital setting, one

who worked as a district nurse, and one who worked as a practice nurse. Data from these interviews were to form the combined negotiated construction of discharge planning - that is - form the hermeneutic-dialectic circle.

Initially in this study member checking involved returning all transcripts to participants for comments and editing. However, as I began to work with the data I was challenged by another tension. In Guba and Lincoln’s methodology the constructed claims concerns and issues of all stakeholders are shared until an agreed reconstruction emerges. It is at this point that an unease arose for me. The nature of the information shared by the three women who had been in hospital was quite different in focus to that of the three nurses. The women’s stories were embedded in their illness experience, and often influenced by past illnesses and many life experience situations. Their discharge experience could not be separated from this, and their stories were much more about their transition through hospitalisation as a personal experience.

Helen’s situation exemplifies this. She was hospitalised acutely for four days with an undiagnosed illness. She was taken to hospital by ambulance with a sharp pain in the side of her chest. The underlying cause of this illness remained uncertain, but the possible diagnosis included recurrence of a previous cancer. When she was discharged, Helen had been in bed for four days, but thought she would manage fine. In previous hospitalisations she had anaesthetics, and so believed it was these that had made her feel awful when she went home. She assumed she would feel much better this time round, however, even getting to the lift on the way out of the hospital was an ordeal. Helen’s expectations of herself were based on her previous experiences with illness and hospitalisation, and she had no expectation of nurses taking an active role in supporting her transition home, and no expectation of support services at home.

In contrast to the three women’s personal experiences, the three nurses involved in the study, could very clearly articulate issues and concerns surrounding discharge planning from their professional position with knowledge of processes of the health care environment. They commented on many of the contextual and structural problems which impacted on their practice, including issues of professionalism and accountability.

My response to the tension was to keep the three women’s stories as whole constructions of their experiences, and therefore separate from the negotiated construction of the nurses in the study. I re-constructed stories from the transcripts that I interpreted as representing their experience. I then returned these to the women for further member checking and comment on how fairly it represented their experience. For me, this aspect of member checking remained crucial for respecting their story and to give these women the opportunity to check if the story was ‘fair’. All three women commented positively on their re-constructed stories and two also said how much easier they found it to read, compared to the unedited transcripts from the interviews.

I gained permission from the women for these stories to then be shared with the nurses involved in the study. As such, these stories fed in to the hermeneutic circle as constructions informing the study via the nurses, but the women were not part of the negotiation process of the combined nurses construction. Comments back from the nurses included how powerful the stories had been and how illuminating they were about the three women’s experiences.

What became clear to me was the tension between the methodological purpose of member checking (to confirm the re- construction of multiple realities) and the ethical question of "What level of involvement is appropriate for different participants to have in the process of member checking?" In responding to that tension I relied on my own knowledge gained from years of practice as a nurse, from an intimacy with the data, and from an understanding of the local context as a basis for that decision. From the content of their stories, I could clearly see it would have been difficult for the women involved to comment on some aspects of nurse’s practice of which they had little knowledge. Whilst I strongly believe that discharge planning should be a collaborative process - involving the person being discharged and other appropriate people as necessary in a partnership of planning care needs - asking people to comment on a service and aspect of nursing practice for which they may have gained only limited, or no understanding from their experience, felt uncomfortable, and indeed unethical. I trusted this discomfort. I realise I am at risk here of being charged with abandoning a key tenet of a Guba and Lincoln’s (1989) constructivist methodology, that of negotiating where possible a consensus from differing realities and agendas - or put another way - I could be accused of disenfranchising a group of stakeholders. However, I have not chosen to do this to avoid the potential conflict from different agendas - that possibility remained with the potentially divergent realities of the three nurses in the study - rather, I believe the essence of what the women shared with me was different from that of the nurses, and would not sit comfortably within a construction of nursing practice. If I had attempted to do so, their voice may have been lost - and potentially disenfranchised, whereas I believe their voice is much more powerful in its whole. Instead, I responded to the data from intuitive personal experience, and familiarity with the practice area, and thus retained their stories as complete entities. It is this ‘knowing’ that guided my decision to deal with the information that I was given in different ways. Interestingly, this is a point noted by Appleton and King , who suggest when utilising constructivist methodologies in nursing inquiries, the researchers "personal and intuitive knowledge of the field can inform the inquiry process".

This tension, and the decisions I made in response to it, has also made me question how appropriate it would be in any inquiry based on interactive and intersubjective methods, such as the one described here, for a researcher without knowledge of the field or practice area to undertake such an approach to research. Without the practice experience and contextual knowledge there is a risk of not making decisions about the emergent design which respond to nuances that are embedded in the data.

This study used constructivist inquiry to explore grandparent bereavement in the family context, where a grandchild had died suddenly and unexpectedly. Taking part in the research were three groups (who could be called stakeholders) of participants; 16 grandparents, 6 parents and 3 health workers. Of the 23 parents and grandparents, 17 were from the same families, with 2 complete families where all living parents and grandparents took part.

Member checking in this study involved a continuing process of returning to individual participants for discussion. This included:

Member checking also applied to stories. Early in the study I made the decision that I wanted each participant to have a clear voice and position in the research, as well as being part of the whole process of trying to reach a joint construction. I offered participants the opportunity to include their own ‘story’ (construction of grandparent bereavement) in the research report which I constructed using only their actual words from the series of interviews and letters. Participants then checked the story – to edit, totally re-write, accept or reject it. The final draft of everyone’s stories I then put out for member checking as an acceptable construction of their experiences to be seen in the context of other people’s, and particularly other family members’, stories.

I started out with the view of member checking as one strategy in the pursuit of credibility. What I became aware of, and interested in, was the effect of member checking for participants and the ethical issues involved.

The therapeutic aspects of member checking

Having a chance to see the context of their experience in a joint construction and to read others’ draft stories was for many people affirming of their own experience.

Elisabeth wrote having seen the first draft of her story and the ideas of a developing joint construction,

I knew I was benefiting from sharing my experiences with you and also my grief was moving along between letters and questions. It was very comforting to find that other parents make the same comments and have similar feelings.

Elisabeth 7:1

The flow-on effect of member checking in families

The consequences of talking to one or more members in a family did mean that the research affected the whole family, as Boss wrote, "perhaps the family wants the system disturbed, which is why they elected to invite an outsider in". Member checking if analytic categories and ideas across participants and of the final stories had effects within families. Sarah commented, when she read the book of stories of other participants, that it was a special chance to read how her parents had felt when Matthew had died and to gain insight into their feelings. A grandmother commented after reading drafts of analytic ideas, that it made her suddenly realise how important it was to parents that grandparents acknowledged both dead and living grandchildren when asked, "How many grandchildren do you have?". It was something that she planned to do in the future.

The upsetting aspects of member checking

I felt that a chance to read each other’s stories before the study was completed was an important aspect of the partnership on which the research was based. In addition it was also member checking for agreement as to the final written form. What actually happened? Twelve of the participants spontaneously commented that it had been really positive, helpful or confirming to read other people’s stories. However, one participant was upset about their story when seeing it in the context of other stories. After contrasting it with other stories the participant felt it was inadequate and that it was not a "good" story. I had not anticipated that reaction; from my perspective all the stories are outstanding and are unique. I talked with the participant about the uniqueness of their story, and of the opportunity they had to expand, re-write or even withdraw the story. We agreed to leave it a few weeks and I would go to visit and we would re-edit or re-interview if the participant wanted. In the end whilst the distress cannot be "undone", on reflection and discussion the participants decided to leave the story unedited as their story.

Member checking affecting the decision to use pseudonyms or real names

Seeing the first draft of their own story and the plans for writing up the research provoked several participants to re-viewing the use of pseudonyms. They formally wrote requesting the use of their real names so that the story was a memorial to, and acknowledgment of, their grandchild/child. I decided that this option, after being approved by the ethics committee with a further consent form, clearly had to be "checked" with everyone else, to see if they wanted to use their real names as part of the public construction of their experiences. By the end of the research the 23 participants chose real names.

Member checking and participants’ right to a voice

At the time we wrote the abstract for this paper, there were several items of information that two participants had decided they were not happy to include on public view in the final version of their stories and as part of the illustration of the joint construction. At the time I felt that several of these points were of importance and relevance to the study focus. At that time I felt it was an important decision to uphold. I was also aware that by encouraging participants to edit their stories and polish their individual constructions, which linked to the developing joint construction, ready for public view meant that I have accorded participants the ‘rights’ which writers expect; to re-fine, re-work and re-view their material before it is published. I believed that without participants’ willingness to share their pain with me, this research would not exist. Having asked them to revisit that pain I have no grounds to cause them any further pain, by presentaing material or aspects of their experiences, which they are not willing, with hindsight, to plan on public view. By upholding this positing I have placed the ethical use of member checking above the methodological use. Does such a position compromise the trustworthiness of the study? Or does it uphold participants’ ownership of their realities, stories and ideas? I have reached a position where I believe that it does not threaten the trustworthiness of a study which is based on partnership. In contrast, limiting participants’ opportunities to member check would have avoided this situation but would have meant that the study was untrustworthy by claiming in theory to be cooperative research but not carrying this out in practice. The interesting sequel to this issue is that after several months of reflection, discussion with me and their families, the two participants decided to include the information which they originally wanted to be outside the final research report.

 

Is member checking always useful? Does member checking pose a paradoxical threat to credibility?

As we have described we have both been challenged to reconsider member checking from both a methodological and ethical standpoint. The criteria proposed by Lincoln and Guba provided us with a useful starting point on our research journeys, yet we have both shifted to a more questioning position of their usefulness . Clearly the need to respect the gift and offer people opportunities for comment and reflection remains paramount. As Alison has identified there have been many unanticipated benefits to this. Yet likewise the tensions of member checking cannot be ignored.

Margarete Sandelowski (1993:4) wrote

Yet its [member checking] potential to enhance qualitative work belies the deeply theoretical and ethical difficulties involved in this technique that may serve paradoxically to undermine the trustworthiness of a project. Indeed, what is often lost in the discourse of member checking is the recognition that both researchers and members are stakeholders … there are different stories to tell and different agendas to promote.

For the participant representation of their views can be of primary importance whilst for the researcher the representation of multiple realities necessarily becomes the focus. Sandelowski does not provide answers to this dilemma of whose voice can be heard, but urges that we find creative ways to present narratives which enhance our work rather than deaden it with rigid rules.

Schwandt also makes this point when suggesting the flexibility of constructivism:

Unlike scientific methodologies, constructivist methodologies are guided by the ideal of fidelity to subject matter (versus primacy of method). That is, they claim no special status for a particular way of investigation, and rather than impose a general set of methodological principles on all forms of experience, the constructivist will adapt both design and method of investigation to the nature of the phenomena at hand.

So how does this occur in practice? Is this what we have both done? One answer might be that with cooperative inquiry a consensus might be reached, and in our research this did generally happen. But it would not happen with all research. And we had experiences, similar to that which Margarete Sandelowski wrote about, where there were different stories. The joint construction we presented, and the stories which individuals looked for in the research report saying along the lines of, "Well yes, that [the construction] looks fine, but that was not EXACTLY what happened to me."

But if there are these differences does it make the research ‘untrustworthy’? Isn’t the response to this to present these differences between stories, and note the participant(s)’ comment for the reader of the research, so then both participants and researcher have the same "rights" and opportunities to polish, re-work, re-view and re-visit the final, public version of their stories and construction(s). Where participants have reservations that the joint construction is not ‘exactly’ what happened because it is an abstraction away from their own concrete experience (construction) then this is noted.

One way out of this debate is to stand back and say "Should we worry about criteria at all?"

Why have criteria when working from a relativist position of reality? Steinar Kvale points out that knowledge does not re-present an objective world as the world is seen in multiple realities. Hence "knowing [is] not as having an essence but as a right to believe". In which case shouldn’t anything, in terms of research, go, as long as we [researcher and participants] believe in it? And that belief might be different, more global than checking the specifics as we have done in the process of member checking. It might be belief in the outcome or the overall research.

The adherence to the need for criteria can mean we become focused on what Thomas Schwandt (1996) has written about "criteriology", without thinking about the context of each individual study. A middle perspective is probably that presented by John Smith (1990: 178) who wrote:

Whereas initially these lists [axiomatic, action criteria etc in Guba and Lincoln (1989); Lincoln and Guba (1985)] were often thought of as the first steps on the road to a more definitive criteria, recently they have been recognised for what there [sic] actually are – an open-ended, always evolving, enumeration of possibilities that can be constantly modified through practice.

Which leads back to what we wrote in the abstract: The reasons were in response to local contextual conditions, as Guba and Lincoln (1989, p.242) suggest that such methodological shifts are "hallmarks of a maturing – and successful – inquiry".

We have presented you with our constructions of grappling through this interesting but challenging aspect of research and put these ideas out for debate.

 

References

Smith, J.K. (1990) Alternative research paradigms and the problem of criteria. In E. Guba (Ed.). (1990). The Paradigm Dialog (pp. 167-187). Newbury Park, California: Sage Publications, Inc.