Chapter 12

Community Research Methods:

Post-Positivist and Social Constructivist Paradigms

Whereas the previous chapter examined the goals, processes, and paradigms of community research, the focus of this chapter is on the methods of community research. Community research methods are the concrete tools that reflect more abstract philosophical assumptions that underlie different research paradigms. We use the research paradigms that we outlined in the previous chapter as the organizing framework for the methods presented in this chapter and the next chapter. In this chapter, we consider methods that are used to understand oppression, liberation, and well-being (analytical or basic research) and methods to overcome oppression and promote liberation and well-being (activist or interventionist research) for research operating from the post-positivist and social constructivist paradigms. See Table 12.1 for an overview of the different research methods. We begin with a presentation of the more traditional, quantitative research methods that have been used in community psychology and related disciplines.

POST-POSITIVIST RESEARCH METHODS

Analytic Research:

Indicators, Epidemiology, and Survey Research

Indicator approaches, epidemiological research, and survey research involve examining existing data or collecting new data on large samples of people in order to make generalizations to the population from which the samples are drawn. These approaches use quantitative methods to assess community needs and resources.

Putting Problems on the Map: Needs and Resources Assessment

Needs and resources assessment is concerned with the following questions: What kinds of problems/needs are there in the community? What are the resources, capacities, and strengths of the community that can be mobilized to address the problems/needs? What kinds of interventions are needed to address the problems and meet the needs in the community? The primary function of needs assessment and resources assessment is to provide data for planning an intervention (McKillip, 1998; Posavac, & Carey, 1997; Witkin, & Altschuld, 1995). Needs and resources assessment can show how widespread or prevalent a problem is and what some of the factors are that are related to the prevalence of the problem. The two primary quantitative methods used in needs and resources assessment are indicator approaches and surveys.

Indicator approaches include social indicators and indicators of service utilization that likely reflect problems that a community might be experiencing. Typically indicator data have already been collected and the researcher's job is to access them and put them together in a way that is meaningful and useful. A variety of social indicators (e. g., age, income) are typically gathered by government bodies or planning agencies through a census (information on an entire population is gathered) or a survey (a sample of the population). Indicators of service utilization are those that show the number of people who use different health, social, or community services. The utility of indicator approaches for needs assessment is that they can help to pinpoint communities that might be selected for an intervention. Communities with a relatively high percentage of low-income families (a social indicator) and relatively high rates of police calls and families involved with mental health and child protection services (service utilization indicators) demonstrate a greater need for intervention than communities with lower rates on these indicators. However, in selecting a community for intervention, care must be taken not to further stigmatize these communities. That is why it is also important to examine the assets and strengths of communities (Kretzmann & McKnight, 1993).

An illustration of the usefulness of this approach to needs and resources assessment can be found in the Better Beginnings, Better Futures project described in Chapter 22, the province of Ontario. The provincial government developed a policy research demonstration project to promote the development of young children and prevent problems in living and invited communities across the province to become a Better Beginnings, Better Futures' site. More than 50 communities applied, but only eight were selected. In their applications, communities provided indicator information on the needs of the community, along with information about community strengths and resources, both of which were considered in the process of site selection.

Epidemiological and survey methods involve collecting new information about a problem or need. In public health, epidemiology is used to document the incidence (the number of new cases of a disease) and prevalence (the number of existing cases of a disease) in a population, as well as factors that put people at risk for or protect them from developing the disease. Community psychology has been quite interested in the application of epidemiological approaches to the study of mental health problems (see Box 12.1 for an example). Moreover, Reinharz (1992) has argued that feminist survey research can be used to "put a problem on the map by showing that it is more widespread that previously thought" (p. 79). For example, date rape, child sexual abuse, sexual harassment, and violence against women, problems that were previously unnamed, have been shown to occur at alarmingly high rates through feminist survey research (Reinharz, 1992).

Making Inferences about Populations

Epidemiology and survey research are closely tied with prevention and health promotion, which have a population-level analysis. Prevention and health promotion are the action steps that follow from epidemiological and survey research. The concepts of population, sample, generalizability, and sampling are important in this type of research (Fowler, 1984). Population refers to everyone in a defined geographic or social space; whereas a sample is a sub-set of people in the population. Samples are studied in epidemiological and survey research so that inferences can be made about the population from which they were drawn. Therefore, researchers strive to ensure that the samples that they study are representative of the larger population, so that findings obtained from a sample can be generalized to the population. Sample representativeness can be obtained if some method of random sampling is used to draw the sample from the population. In random sampling, every member of the population has an equal chance of being selected. Also, the larger the random sample, the more likely it is to reflect the population as a whole.

In the study of people from different cultures or ethnic backgrounds, post-positivist researchers take an "etic" or outsider approach. The etic approach assumes that there is a single reality and that research can determine how participants from diverse backgrounds differ with respect to psychological constructs.

Measurement

Epidemiological and survey research utilize quantitative methods. Various measurement scales, administered as self-report questionnaires or through interview schedules, are used to tap theoretical constructs that are presumed to correspond to some external reality. Two key concepts related to psychological measurement are reliability and validity. First of all, a measurement tool must reliable. That is, it must produce consistent or repeatable findings. Several different methods have been developed to test reliability, including the test-retest method, the split-half approach, measuring the internal consistency of the items in the measurement scale, and inter-rater reliability. Measurement scales must also be valid; that is, they must measure what they intend to measure. Typically scales are validated through demonstrations that scores on the scale in question show significant correlations with other scales designed to measure the same or a similar construct. This is called criterion validity. The concept of construct validity includes both criterion validity (or what has been called convergent validity) and divergent validity (Cronbach & Meehl, 1955). Divergent validity is required to demonstrate that the scale is not significantly correlated with measures of presumably unrelated constructs.

An example of a measurement scale that is of interest to community psychologists is a questionnaire that taps the empowerment of mental health consumer/survivors. This scale was developed by Judi Chamberlin, who is a mental health survivor, and colleagues in collaboration with consumer/survivors from several different self-organizations in the U. S. They began by defining empowerment and its components, including learning about and expressing anger, having decision-making power, and learning to think critically (Chamberlin, 1997). Next, they generated items that reflect these different components. After pilot testing and refinement, the authors had created a 28-item scale (Rogers, Chamberlin, Ellison, & Crean, 1997; see sample items of this scale in Box 12.2). Moreover, the reliability and construct validity of this scale have been established (Corrigan, Faber, Rashid, & Leary, 1999; Rogers et al., 1997).

Testing Theory and Examining Relationships

Epidemiology and survey research go beyond determining how prevalent a problem is in a population to testing theoretically-derived hypotheses about factors that are related to the prevalence of the problem. In particular, this type of research can examine the role that risk and protective factors, that were defined and discussed in Chapter 4, play in contributing to the problem. Low-income and unemployment, for example, are major risk factors for health and mental health problems (see Chapters 1 and 15), while social capital is an important protective factor for health and mental health (see Chapter 7). Moreover, longitudinal survey research can shed light on the causal role that risk and protective factors have in the development of psychosocial problems.

Challenges and Limitations of Epidemiological and Survey Research

It is very challenging to mount these types of large-scale epidemiological and survey studies. Researchers need a great deal of technical expertise in surveys, sampling, and statistical analyses and ample budgets to conduct this type of research. Because of the level of expertise and funding required, it is difficult for community groups to use these methods for a needs and resources assessment.

One limitation of epidemiology and survey research is that the problems that are studied can become reified. That is, constructs such as "mental illness" and IQ can be taken uncritically as some true reflection of external reality, when in fact those constructions are deeply embedded within the social context and may support the status quo. Critical analyses of "mental illness" (Hare-Mustin & Maracek, 1997) and IQ (Cernovsky, 1997) have shown that these concepts need to be contested. Epidemiology and survey research can also individualize personal problems, even if at the population-level of analysis, which may lead to victim-blaming. Claims such as "30% of women in the population have experienced a depressive disorder" need to be subjected to critical scrutiny to uncover the role of power and social factors that contribute to women's experiences of depression and to avoid the problem of context-stripping that occurs in positivist research. For example, Poland, Coburn, Robertson, and Eakin (1998) provide an excellent critique of population health research on the socioeconomic determinants of health, arguing that this research is largely ahistorical and apolitical. Alternatively, they propose a critical, historical, political economy perspective on the socioeconomic determinants of health.

The etic approach to cross-cultural research is subject to the same problems. In comparing people from different cultures or ethnic backgrounds, researchers assume that the constructs are meaningful within the different cultures that are compared. However, often the psychological constructs that form the basis of the comparison are rooted in mainstream western research and are imposed on other cultures, for which they may not be meaningful and relevant.

Activist/Interventionist Research:

Quantitative Evaluation of Programs and Interventions

Post-positivist, quantitative research is also used in the evaluation of social programs and interventions. Since the publication of Carol Weiss' Evaluation Research: Methods of Assessing Program Effectiveness in 1972, there has been considerable growth in the field of "program evaluation" (see the Resources at the end of this chapter), which has been defined as:

". . . the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future programming. Utilization-focused program evaluation (as opposed to program evaluation in general) is evaluation done for and with specific intended primary users for specific intended uses." (Patton, 1997, p. 23).

As Patton's definition suggests, program evaluation research is used to inform decision-making regarding how a program can be improved or become more efficient (formative evaluation) or whether a program should be continued (summative evaluation). Since the inception of evaluation research, program evaluators quickly learned that empirical evaluation information is but one source of information that can have an impact on decision-making in what are often highly turbulent conditions within and around social programs and interventions (Weiss, 1972). In other words, evaluation research is saturated with values and politics. Thus, program evaluators need to be clear about their values and skilled in working in with program constituents.

Many program evaluators utilize what they call a "stakeholder" (Pancer, 1997) or "empowerment" (Fetterman, Kaftarian, & Wandersman, 1996) approach to evaluation. In this approach, evaluators begin by asking pre-entry questions (Cherniss, 1993), such as: Who wants the evaluation? Who are the stakeholders? Often times, program evaluators work with program staff, managers, and funders as the primary stakeholders. Using the concept of partnership, that we have promoted throughout this book, the consumers of services, who are typically disadvantaged people, should be a major stakeholder involved in the evaluation.

Once the stakeholders have been identified, evaluators need to work collaboratively with them to answer other pre-entry questions, including: What do stakeholders want from the evaluation? What is the purpose of the evaluation? What are the evaluation questions? What type of evaluation is needed to answer the questions? Answering these questions is an important step in framing the evaluation. In Table 12.2, we provide an overview of the link between evaluation purposes, approaches, and questions. In the previous section, we discussed needs and resources assessment, which is the first approach identified in the table. In the remainder of this section, we elaborate on the other evaluation approaches in Table 12.2.

Evaluability Assessment and Program Logic Models

Evaluability assessment and program logic models were introduced by Rutman (1980) to provide a framework for evaluation. Evaluability assessment seeks to determine whether a program or intervention is ready to be evaluated. A program must meet three criteria to be evaluated: (a) clear, specific, and measurable outcome goals, (b) clearly articulated program components, and (c) a rationale that links the goals and program components. It is common for program constituents to have difficulty articulating the outcome goals of the program in question, which leads to what Patton (1997) has referred to as the "goals clarification shuffle." Program staff become confused with evaluation jargon regarding goals, objectives, and indicators. As well, sometimes the program components themselves are not clearly defined, and their link to the outcome goals is tenuous. Program staff may expect changes in outcomes to occur when the intervention components are not be powerful enough to affect a change.

The construction of a program logic model is a way of overcoming these problems (Pancer, 1997; Rush & Ogbourne, 1991). First of all, outcome goals are identified in terms of the changes in individuals or systems that the program or intervention is striving to make. Sometimes when service-providers are asked about the goals of the program, they will respond in terms of services that they provide (e. g., "The goal of this program is to provide home visitation"). Evaluators need to help program staff and consumers to differentiate between service goals and outcome goals. Service goals involve implementation of the program components, whereas outcome goals are the expected changes that occur as a result of the program components.

In Figure 12.1, we present the program logic model that was developed for an evaluation of the Ontario Prevention Clearinghouse (OPC) conducted by Geoff and Mark Pancer. The OPC is an organization that was designed to support prevention and health promotion initiatives throughout Ontario (Pancer, Nelson, & Hayday, 1990). The main program components include: (a) information and consultation (a toll-free telephone information and consultation service, with consultants and several computer data banks as resources), (b) networking referrals to other resource people pertinent to the caller's request, and (c) educational activities regarding prevention, including a series of provincial Prevention Congresses (offered every other year) and workshops. In the short-term, these program components should enhance the knowledge, support, and attitudes towards prevention of those who use OPC; while these short-term changes should lead to more proposals for prevention programs, more funding allocated to prevention, and ultimately the reduction of health, education, and social problems in the long-term (see Figure 12.1). This logic model formed the foundation of a process and outcome evaluation of the OPC, which we describe next.

Process and Implementation Evaluation

Process evaluation involves an evaluation of the implementation of the program components rather than program outcome goals. As is indicated in Table 12.2, a quantitative approach to process evaluation focuses on collecting information about the people who use the service, indicators of the amount and types of utilization of the program components, and consumer satisfaction with the program components (Pancer, 1997; Patton, 1997). Typically process evaluation is concerned with program improvement. Often times process evaluation is used to assist a program in establishing a management information system, so that the program can routinely gather data that are relevant to the consumers who use the service and the utilization of program components (Posavac, & Carey, 1997).

In the evaluation of the OPC, Nelson and Pancer (1990) assisted the OPC in the construction of a data base describing the characteristics of those people using the OPC and the nature of their requests. They also conducted a telephone survey of people who had used the OPC services to inquire about their satisfaction with the different program components. They found that the OPC received an average of 18 major requests for information per month; most (87%) requested information but few received more in-depth consultation (29%); and the overall level of satisfaction of consumers was high.

Like any utilization-focused evaluation, the OPC evaluation concluded with three recommendations: (a) that the OPC required additional funding to maintain and expand services; (b) that the OPC should provide more consultation to information requesters to help them with program planning and development; and (c) that the OPC should encourage more systems-centred approaches to prevention (Nelson & Pancer, 1990). Two years after this evaluation was completed, a second evaluation was undertaken that examined the extent to which these recommendations were implemented (Nelson & Hayday, 1995). First, the provincial Ministry of Health increased its portion of the OPC's operating budget from $50,000/year to $384,000/year. Second, the percentage of information requesters who received consultation increased from 29% to 65%. This was accompanied by increased satisfaction, contact with other resource people, utilization of the information received, and knowledge and awareness of prevention and health promotion. Third, the OPC played a leadership role in advocating with anti-poverty groups for positive reforms to the province's social assistance system.

Outcome Evaluation and Research Design

Outcome evaluation is concerned with evaluating the extent to which the expected changes occurred as a result of the program or intervention (Posavac, & Carey, 1997; Rossi, Freeman, & Lipsey, 1998). Outcome evaluation is used to provide information about whether the program is valuable and should be continued and/or disseminated to other settings so that it can be implemented on a more widespread basis. In the OPC evaluation, participants were asked about changes relevant to the short-term outcome goals, such as whether or not their knowledge and awareness of prevention increased, whether or not they were able to contact more resource people with the aid of OPC than they would have been able to do on their own, etc. While the participants indicated that they had changed has a result of their contact with OPC, it is difficult to conclude from this type of evaluation that there were changes and that they were due to OPC. More convincing demonstrations of change require the use of some type of experimental (Boruch, 1998; Fairweather & Davidson, 1986) or quasi-experimental design (Campbell & Stanley, 1966; Reichardt & Mark, 1998).

Consider an evaluation in which data relevant to the outcome goals are collected at two points in time: immediately prior to the intervention and immediately following the intervention. While this simple pre-test, post-test design can answer the question about whether or not change occurred in the outcomes, it cannot determine whether or not the change was due to the intervention. Campbell and his colleagues (Campbell & Russo, 1998; Campbell & Stanley, 1966) refer to the confidence that one can have that the intervention was causally related to changes in outcome as internal validity, and they described several threats to internal validity. History is one threat; the passage of time may account for any observed changes. Another threat is maturation. Imagine a program that is designed to improve children's academic or social competence. Perhaps the children would improve as a result of natural maturational processes. Or maybe children's improvement was due to the experience of taking the pre-test. Testing improved their performance. There are many other threats to internal validity. Campbell and Stanley (1966) also noted the need for external validity, which refers to the confidence that one has that the findings are generalizable to other contexts.

Research designs are used to reduce these threats to internal validity, and replications in different settings are used to demonstrate external validity. The most basic research design is the randomized controlled trial (RCT). In a RCT, participants are randomly assigned to intervention and control groups. That is, everyone has an equal chance of being assigned to either group. Note that random assignment is not the same as random sampling, that was discussed earlier in the section on surveys and sampling. A RCT controls for history, maturation, testing, and other threats to internal validity, because each group of participants should be equally affected by these factors. The only systematic way that they differ is in terms of the intervention.

There are several examples of RCTs in community psychology, such as the evaluation of the intervention for homeless people described in the previous chapter (Toro et al., 1997). It is also important to note that being assigned to the control group does not mean being denied services or supports. Rather individuals in the control group tend to receive the services that are typically available to them, whereas those in the intervention receive supports that are more innovative or intensive than what is typically available. RCTs are typically used for what are called "demonstration projects," in which the researchers examine the effectiveness of a new approach. When there are several RCTs demonstrating the effectiveness of the innovation, then the next step in the research and action process is dissemination of the project to other settings (Davidson & Fairweather, 1986).

In many situations, however, it is very difficult to utilize a RCT. When such is the case, evaluation researchers often choose to use a quasi-experimental design that does not provide all of the controls of a RCT, but which nevertheless approximates some of its controls. Quasi-experiments were introduced by Campbell and Stanley (1966) to provide some controls when random assignment is not possible. The most popular quasi-experimental design is called the non-equivalent comparison group, in which the evaluator seeks to find some group that is comparable to the intervention group, except for the intervention.

An example of the use of the non-equivalent comparison group design is provided in the evaluation of a multi-site, universal, primary prevention program for children and families called Better Beginnings, Better Futures, conducted by Ray Peters and colleagues, which is described in Chapter 22 (Peters, 1994; Peters et al., 2000). First, all the communities in which the Better Beginnings projects were developed were compared with demographically similar communities. Samples of children and families were drawn from both Better Beginnings and comparison communities, and these cohorts are being followed for 25 years to determine the long-term impacts of the project on participants. Another non-equivalent comparison group strategy that is being used is called the leading cohort design. Data were collected on a sample of grade 2 children and families before the projects started (these children constitute the comparison group), and they are being compared with the children who participated in Better Beginnings once they reach grade 2 (from junior kindergarten to grade 2 or four years of intervention).

One other useful quasi-experimental design is the time-series design. Like the simple pre-test, post-test design, the time-series design examines only one group over time. The difference is that data are collected at several different observation points, both prior to the intervention (baseline) and following the intervention. These multiple observations help to control for many threats to internal validity. This design can be used when the researcher has access to archival data that can be examined over time. For example, if a community development project wanted to examine the extent to which it reduced calls to police and other crime indicators in the neighbourhood in which the intervention was implemented, they would go backwards in time to generate a baseline of crime indicators, and then forwards to see if crime started to drop with the introduction of the community development project.

Cost-effectiveness and Cost-benefit Evaluation

The focus of cost-effectiveness and cost-benefits evaluation is on efficiency (the relationship between effectiveness in achieving outcomes to effort expended to achieve outcomes). This type of evaluation seeks to answer the following questions: Does the program achieve its goals at a reasonable cost? Are there less costly ways of achieving the same or better outcomes? Cost-effectiveness and cost-benefits evaluation are done in the context of an outcome evaluation with some type of control or comparison group or a time-series design. In addition to gathering data relevant to outcomes, the evaluator must determine the costs of the program (e. g., personnel, facilities, materials, equipment) and other services used by the consumer to have a fairly comprehensive comparison of costs across the different programs that are being compared (Yates, 1996, 1998). While cost-effectiveness compares the effectiveness to the costs of different program options, cost-benefits tries to quantify the benefits to participants in terms of monetary values (benefits could include employment income and savings in human service expenditures).

While difficult to conduct, cost-effectiveness and cost-benefits are important to community psychology. Government policy-makers are quite interested in these types of evaluations, and the types of interventions that are developed by community psychologists and their partners have considerable potential to demonstrate cost-effectiveness or cost-benefits. Community psychologists have been instrumental in developing community-based alternatives to institutionalization for different populations. In a cost-benefit analysis of the Program for Assertive Community Treatment (PACT) for people with serious mental health problems, Weisbrod, Test, and Stein (1980) found the community-based program (PACT) had a better benefit-cost ratio (was less expensive) and more effective than the more traditional approach of hospitalizing people in a state psychiatric institution. Community psychologists have also argued that prevention and early intervention programs are more effective and less costly in the long-term compared with traditional reactive approaches to intervention. In support of this, Schweinhardt and Weikart (1989) found that the Perry Preschool program, a primary prevention program for disadvantaged preschool children, not only had significant impacts for participants when they reached their early 20s, but it also produced a substantial cost savings. The 30-week program gave a return of $6 for every $1 invested, while the 60-week program gave a return of $3 for every $1 invested. The savings came from a reduction in costs for special education services in school, involvement in the justice system for criminal behaviour, and welfare payments and from increased taxes paid due to higher earnings.

Challenges and Limitations of Quantitative Approaches to Evaluation

In reducing programs to measurable processes, outcomes, and costs, quantitative approaches to evaluation do not tell the whole story of interventions designed to assist disadvantaged people. In such evaluations, seldom does one hear the voices of disadvantaged people and their experiences of such programs. Also, a major challenge in using this approach is that not all programs fit neatly into the straight-jacket of different experimental and quasi-experimental designs. In contrast to the logical-empirical approach of program logic models and outcome evaluation, Etzioni (1960) conceived of programs in terms of a systems model with dynamic and interacting parts. One of the implications of this model, Etzioni argued, is that programs pursue other objectives besides the rational focus on goals. Programs strive to survive and cope with dynamic changes in the larger social-political environment, including funding cutbacks from government. Moreover, many of the programs that are amenable to post-positivist approaches to evaluation are those over which the researcher has some control. Such programs tend to be ameliorative in nature. More amorphous and dynamic social interventions that strive to be transformative are not as amenable to this type of evaluation.

SOCIAL CONSTRUCTIVIST RESEARCH METHODSAnalytic Research:

Qualitative Approaches

Social constructivist research tends to rely much more on qualitative, naturalistic research. While qualitative research is used in education and many social science disciplines, it does not enjoy widespread use in psychology, which has by and large adopted a natural sciences model of research based on positivist assumptions (Kidder & Fine, 1997). While there has been more attention to qualitative research in some of the applied areas of psychology, including community psychology (see the list of special issues of psychology journals that focus on qualitative research in the Resources section at the end of the chapter), qualitative research is still marginalized in psychology. We agree with Stewart (2000) that qualitative research has much to offer community psychology, in that the two share an emphasis on diversity (including methodological diversity), understanding people in context, and collaborative research relationships.

It is also important to realize that not all qualitative research is based on a constructivist paradigm. Kidder and Fine (1997) have made a distinction between what they call "Big Q" and "small q" qualitative research. The former is based on the constructivist paradigm, while the latter uses qualitative methods as a minor supplement to quantitative methods (e. g., adding some open-ended questions to a survey) and is rooted in the positivist paradigm. The focus of this section is on "Big Q" qualitative research.

Characteristics of Qualitative Research

There are several characteristics of qualitative research (Kidder & Fine, 1997; Patton, 2002). First, qualitative researchers gather qualitative or textual data, including observations of behaviour, people's words, and pictures. Big Q qualitative researchers rely on quotes or stories that are told by research participants, and they do not attempt to quantify these data. Second, qualitative research is an immersion process for the researcher. The researcher enters the field and strives to get close to the phenomenon under study by immersing herself or himself in the context and lives of the participants for an extended period of time. There is a saying in qualitative research that "researcher is the instrument," which captures the shift of the researcher from being a detached outsider to an engaged insider. Third, qualitative research is naturalistic, with the researcher giving up control. The researcher does not attempt to manipulate any "variables," and the research is situated in people's natural environments. Holism is a fourth characteristic of qualitative research. Rather than reduce the phenomenon under study to different components, the qualitative researcher strives for a holistic and contextual understanding of the phenomenon. Fifth, the subjectivity and emotions of the researcher and the participants are legitimate data in qualitative research. The research process is reflexive, with the researcher conveying experiential as well as intellectual understanding.

Sixth, qualitative research is inductive. While the researcher may have a hunch about the phenomenon under study, she or he does not begin with a hypothesis or theory to test. Rather, the research process is more open-ended, exploratory, adventurous, and discovery-oriented. Often findings emerge that are surprising, and the researcher constructs a grounded theory to understand these findings (Glaser & Strauss, 1967). Seventh, qualitative research is typically more concerned with depth than breadth. Whereas quantitative research emphasizes breadth and generalizability of findings by collecting data on large samples, qualitative research tries to provide a more in-depth understanding of the phenomenon of interest by gathering rich, qualitative data on a small number of participants or settings. Finally, in spite of misperceptions of some that qualitative research is biased and lacking in rigor, qualitative research is quite rigorous. The process of data analysis involves both creativity in developing the grounded theory and rigor in verifying the codes, themes, and interrelationships that form the base of the theory.

Varieties of Qualitative Research

Qualitative research is quite diverse (Denzin & Lincoln, 2000; Patton, 2002; Willig, 2001), and there are several approaches that are useful for community psychology. First, grounded theory research is discovery-oriented, and the role of the researcher is to construct a theory that emerges from the transaction between the data and the researcher (Glaser & Strauss, 1967). This is one of the most widely used approaches to qualitative research. Case studies are a second approach to qualitative research (Yin, 1994). Case studies are characterized by their focus on the unit of analysis, one or several cases, as opposed to specific qualitative methods (Willig, 2001). Lincoln and Guba (1985) have suggested that the case study should be the unit of analysis in qualitative, naturalistic inquiry. The goal of case studies is to arrive at an in-depth understanding of a phenomenon in a particular context. An example of a qualitative research project that used grounded theory in the context of a single case study is Peirson and Prilleltensky's (1994) grounded theory of implementation of prevention programs and change in schools. These researchers identified factors that facilitated and inhibited the implementation of a prevention program in a high school and developed a grounded theory of the process of change.

A third method, narrative inquiry, focuses on how people make sense of their experiences through stories (Crossley, 2000; McAdams, 1998; Murray, 2000). Julian Rappaport (1993, 2000) has argued for the utility of narrative methods for community psychology. Rappaport makes a distinction between personal stories, which are unique to the individual, and narratives, which have more common themes and are applicable across individuals. He further argues that there are dominant cultural narratives that provide the context in which individuals develop their personal stories, make meaning of their experiences, and create their identities. For oppressed groups, the dominant cultural narratives are often terrorizing, stereotypical, and stigmatizing (e. g., the dangerous mental patient). In contrast, alternative settings (e. g., self-help/mutual aid organizations) can provide an alternative, positive narrative from which individual members can fashion a more positive, joyful, and hopeful personal story. Other narrative researchers have also pointed to the healing power of narratives in the context of health and illness (Crossley, 2000; Murray, 2000).

A fourth and somewhat related method to narrative inquiry is discourse analysis (Parker, 1997; Potter & Wetherell, 1987; Willig, 2001). While there are different approaches to discourse analysis (Foucauldian discourse analysis and discursive psychology), there is a common emphasis on the study of language. Moreover, what a person says, her or his talk, is constructed as an action with a purpose and meaning that must be understood in the context of the conversation. That is, the focus of discourse analysis is on what the talk is doing. This view stands in contrast to the mainstream positivist view that talk is a true reflection of the person's internal mental processes and structures (i. e., cognitions, attitudes). Discourse analysis can be used to examine existing texts (e. g., newspaper or magazine articles) or data gathered through interviews or tape-recordings of conversation. Like narrative inquiry, discourse analysis can deconstruct the functions of dominant discourses (e. g., psychiatric professionals' medical discourse about "mental illness") and alternative discourses (e. g., the stories of survivors' of mental health problems and the mental health system). Fifth, ethnographic research strives to understand the culture of a setting and people (Agar, 1986). In contrast to the "etic"stance of post-positivist research, ethnography adopts an "emic" or insider understanding of the culture and its assumptions.

While these different methods are typically associated with the constructivist paradigm, there are variations on the underlying assumptions of these methods. In some cases, grounded theory, case studies, narratives, and discourse analysis may be more closely aligned with the critical realist assumptions of the critical paradigm (Alvesson & Sköldberg, 2000; Crossley, 2000; Parker, 1997; Willig, 2001). Furthermore, the methods that we have briefly noted are by no means exhaustive of all the different methods of qualitative inquiry. For example, hermeneutic/ phenomenological research (Alvesson & Sköldberg, 2000; Richardson & Fowers, 1997) is another major qualitative method.

Sampling in Qualitative Research

Since qualitative research is not concerned with generalizability to populations, it is not necessary to have the same large sample sizes that are found in survey and epidemiological research. Rather the main issue in qualitative research is to purposefully select information-rich participants (i. e., those people from whom the researcher can learn a great deal about the central issues which the research addresses) (Patton, 2002). Often times different stakeholders are sampled to obtain their constructions of the issues. Including multiple stakeholder groups permits an examination of issues across stakeholder groups so that one can discern common and unique aspects of each stakeholder group's constructions of reality. A variety of different sampling techniques in qualitative research has been described by Patton (2002). Patton (2002) made the following statement about sample size in qualitative research.

"There are no rules for sample size in qualitative inquiry. Sample size depends on what you want to know, the purpose of the inquiry, what's at stake, what will be useful, and what will have credibility, and what can be done with available time and resources." (Patton, 2002, p. 244)

One guideline for sample size is that one samples to the point of saturation or redundancy. That is, one continues to sample until little new information is gained from additional participants.

Data-gathering Methods in Qualitative Research

Qualitative Interviews. Interviews using open-ended questions (i. e., questions that cannot be answered with a "yes" or "no") are widely used in qualitative research to understand people's lived experiences (Kvale, 1996; Morgan & Krueger, 1997). Moreover, the qualitative researcher can use in-depth individual interviews or focus group interviews or some combination of the two, depending on the purpose of the research. Sometimes participants are interviewed only once, while in other research there may be multiple interviews with the same participants. Patton (2002) has also identified several different interview methods ranging from very unstructured to very structured: (a) the open-ended conversational interview, (b) the interview guide, in which there are a number of interview questions and probes, but which are not all necessarily asked, and (c) standardized, open-ended questions, in which there are a number of interview questions, which are all asked in the same order and using the same wording. In line with the inductive and discovery-oriented nature of qualitative research, the skill of the interviewer in listening and in allowing the participant to determine the direction of the interview cannot be underestimated in this type of research. Qualitative interviews are typically tape-recorded and later transcribed so that they can be coded.

Qualitative Observation. Qualitative researchers can also gather observational data, which provides information that participants may not talk about. Moreover, the researcher may observe something of which the participants are unaware. Qualitative observation is also important for understanding context, because the researcher can see people in their natural environments. As Patton (2002) noted, there are many variations in qualitative observation, including the level of engagement of the researcher (active participant vs. more passive onlooker), the scope of the observation (very focused or narrow vs. very broad and open-ended), and the duration of observation (single vs. multiple observations). Most qualitative researchers attempt to become immersed in the setting, which means that they spend considerable time in observation and in relationships with the members of the setting. Generally, qualitative researchers take two different forms of field notes: (a) descriptive field notes about what they observe (including direct quotes) and (b) analytic or reflective field notes in which they record their impressions, insights, hunches, and feelings.

Other Sources of Qualitative Data. Qualitative researchers can also use archival data, including program records and documents, newspapers, magazines, and other sources from which they can extract textual data. In many qualitative research projects, the researcher uses more than one type of data-gathering.

Qualitative Data Analysis and Verification

Each approach to qualitative research has specific techniques for data analysis (e. g., grounded theory, discourse analysis, narrative analysis). However, these different approaches all tend to use some process of coding the data from transcripts, texts, and field notes.

One often used method of data analysis used in grounded theory is the method of constant comparison (Glaser & Strauss, 1967). The first rule of the constant comparative method is that "while coding an incident for a category, compare it with the previous incident in the same and different groups coded in the same category" (Glaser & Strauss, 1967, p. 106). The emerging coding system will be imperfect and will not always work, so "the second rule of the constant comparative method is: stop coding and write a memo on your ideas" (Glaser & Strauss, 1967, p. 107). The aim of memo writing is to uncover the properties and dimensions of a code. Next, one begins to integrate the codes and their properties. At this stage in the analysis, the researcher makes a judgment as to whether a new incident exhibits the category properties that have been tentatively defined. The focus of this step in data analysis is on rule definition and making the properties of codes explicit. Initial coding generally stops when the codes are saturated (i. e., no new codes emerge); all the data sources have been coded; and there is a sense of integration (Lincoln & Guba, 1985). An example of a code is that of "Social Isolation," which emerged as a code in Lord and Hutchison's (1993) study of the process of personal empowerment. The key properties of social isolation they found were: neglect, lack of support, and abandonment.

Once the initial set of codes has been developed, the next task in the analysis is to develop more overarching theme codes or pattern codes, which subsume and link the more descriptive codes from the first level of analysis. Whereas initial coding breaks the data down into discrete codes, theme coding puts the data back together in a way that makes sense. In grounded theory, the theme codes are the conditions, context, actions, and consequences of the phenomenon (Strauss & Corbin, 1998). In theme coding, the researcher uses questions (Who? What? When? Where? How? Why?) to create meaning out of the codes. Often times researchers use charts, matrix displays, or figures to clarify the theme codes and their inter-relationships. Table 12.3 is an example of a chart depicting theme coding in a qualitative study of the process of personal empowerment (Lord & Hutchison, 1993). Note that the themes are organized as the steps that are involved in the process of empowerment, with people moving from a state in which they experience powerlessness to a point in which they are contributing to community. The theme of powerlessness encompasses the codes of social isolation, service dependency, and limited choices.

Lincoln and Guba (1985) argue that qualitative researchers must be able to establish the trustworthiness of their data. "The basic issue in relation to trustworthiness is simple: How can an inquirer persuade his or her audiences (including self) that the findings of an inquiry are worth paying attention to, worth taking account of? What arguments can be mounted, what criteria invoked, what questions asked, and what would be persuasive on this issue?" (Lincoln & Guba, 1985, p. 290). They propose four criteria for trustworthiness: (a) credibility, which means that the one must adequately represent participants' multiple constructions of reality, (b) transferability, which refers to the extent to which the findings can be transferred to other contexts, (c) dependability, which is the extent to which findings are consistent or dependable, and (d) confirmability of the data by others. Lincoln and Guba (1985) propose several techniques for establishing the trustworthiness of qualitative data, including: prolonged involvement in the setting and persistent observation, the use of multiple sources of information, multiple researchers, and multiple methods to "triangulate" or determine the consistency of the data, checking the interpretations of the data with participants, establishing an audit trail of the steps taken in the data analysis, and providing a detailed description of the setting.

Qualitative Needs Assessment

Community researchers can use qualitative methods to assess the needs and resources of a community. Milord (1976) argued that key informant interviews, focus groups, nominal groups, and community forums are non-epidemiological methods of needs assessment. Lord, Schnarr, and Hutchison (1987) completed one of the first qualitative needs assessments of mental health consumer/survivors. They conducted in-depth interviews with 23 mental health consumer/ survivors (16 people in four provinces and seven self-help group leaders) and one focus group interview (with five to nine participants) in each of the 10 Canadian provinces. The main needs that consumer/survivors talked about were housing, employment, friends and support, and the struggle for identity. The phrase "a friend, a home, a job," have subsequently become a familiar refrain in the stories and discourse of mental health consumer/survivors (see Chapter 21).

Another technique that can be used in needs and resources assessment is asset mapping (Kretzman & McKnight, 1993). In asset mapping, participants are asked through qualitative interviews to map the strengths or assets of the community. For example, the Search Institute for developmental assets for children has identified many different assets, both internal and external to children and youth that can protect them against stressful situations (see the website for the Search Institute, http://www.search-institute.org/). Brainstorm how you could use qualitative methods to assess the needs and resources of the community described in Box 12.3.

Challenges and Limitations of Qualitative Constructivist Research

The major challenge that qualitative researchers face is the credibility of the methods. The academic community and granting agencies often do not understand or value qualitative research. Thus, gaining acceptance of these methods is a major challenge. With regard to limitations, qualitative research overcomes many of the problems inherent in quantitative research (e. g., context-stripping, narrowness, mechanistic simplicity). However, in the process, a new set of problems emerges (e. g., questionable generalizability of findings, an over-reliance on description). However, from our perspective, the main problems with qualitative research are not much different from those of quantitative research. Much of qualitative research lacks a critical perspective, fails to use participatory processes, and has little focus on catalyzing social change.

Activist/Interventionist Research:

Qualitative Evaluation of Programs and Interventions

Varieties of Qualitative Evaluation

Many of the methods and approaches to qualitative research that were described in the previous section are also applicable to the evaluation of programs and interventions. Qualitative interviews, observations, and archival data can all be used in qualitative evaluation. Similarly, grounded theory, case studies, narratives, discourse, and ethnographic methods can also be applied to evaluation.

There are two main approaches to qualitative program evaluation (Greene, 2000): the "utilitarian pragmatism" of Patton (2002) and the social constructivism of Guba and Lincoln (1989). According to Greene, Patton's approach emphasizes the values of "utility, practicality, and managerial effectiveness" (Greene, 2000, p. 984), and the primary evaluation audience is mid-level program managers and administrators. Patton (2002) borrows many of the concepts used in quantitative program evaluation, such as logic models, evaluability assessment, and evaluation of process and outcomes, but he advocates gathering qualitative data for these types of evaluation. The evaluation purposes, approaches, and questions used in Patton's (2002) approach to qualitative evaluation are similar to those that we described in Table 12.2.

In contrast, the fourth generation approach to evaluation advanced by Guba and Lincoln (1989) rests explicitly on the assumptions of social constructivism and is concerned with how the program is experienced by different stakeholders (including managers, staff, and consumers) and the meaning that the stakeholders attach to the program. This approach is guided by the principles of value pluralism, fairness in honoring stakeholder constructions and eliciting their claims, concerns, and issues, and a negotiated evaluation process. The primary method of fourth generation evaluation is the case study, which can include qualitative interviews, observations, and an analysis of program documents. In addition to the criteria for technical adequacy of the evaluation findings (i. e., the different criteria for trustworthiness mentioned earlier), Lincoln and Guba (1986) assert that the research process and findings must be fair (i. e., a negotiated process in which all stakeholders have the opportunity to share their concerns, interpret the findings, and influence the evaluation recommendations for change), authentic (i. e., leading to increased understanding, appreciation, and consciousness of the experiences of different stakeholders and their social contexts), and relevant to program stakeholders.

Challenges and Limitations of Qualitative Evaluation

While qualitative approaches to evaluation are not new (the first edition of Patton's Qualitative Research and Evaluation Methods was published in 1980), qualitative evaluation is still less accepted by government stakeholders than quantitative evaluation methods. In our experiences doing evaluation, we continue to hear from government funders and managers that qualitative evaluation is "soft" and "subjective." While qualitative methods cannot uncover causal mechanism or calculate cost-effectiveness, as can be attempted in quantitative evaluation, the more in-depth, stakeholder-based approach of qualitative evaluation is well suited to many different types of evaluation questions.

SUMMARY

In this chapter, we have reviewed a wide range of research methods that community psychologists can use in research with community groups. We examined methods associated with the post-positivist and social constructivist paradigms identified in the previous chapter as the organizing framework for this discussion. At this point in time, these two paradigms tend to capture some of the major emphases in different research methodologies. In the next chapter, we focus on methods that community psychologists use when working from the critical paradigm.

Table 12.1

Methods Associated with the Post-Positivist and Social Constructivist Research Paradigms

Table 12.2

Matching Evaluation Purpose, Approaches, and Questions in the Post-positivist Paradigm

Table 12.3

Themes in the Process of Personal Empowerment

Source: Lord and Hutchison (1993)

Figure 12.1

Program Logic Model for the Ontario Prevention Clearinghouse