Full text

The process of mixed research - involving 'mix[ing] or combin[ing] quantitative and qualitative research techniques, methods, approaches, concepts or language into a single study' (Johnson & Onwuegbuzie 2004: 17) - has developed a long way since Campbell and Fiske (1959) coined the term multiple operationalism, wherein more than one method is used as part of a validation process to help ensure that the variance explained culminates from the underlying phenomenon or trait and is not a function of the method. However, most of the biggest formalized developments in mixed research have occurred within the last 25 years, being given impetus in the 1980s by several prominent researchers (e.g. Brewer & Hunter 1989; Bryman 1988; Greene, Caracelli, & Graham 1989; Jick 1983; Kidder & Fine 1987; Louis 1982; Madey 1982; Mark & Shotland 1987; Maxwell, Bashook, & Sandlow 1986; Phelan 1987; Rossman & Wilson 1985), who called for the integration of quantitative and qualitative approaches. However, the last five years have witnessed a significant increase in the number of mixed research studies, marked by a handbook on mixed methods (Tashakkori & Teddlie 2003); several mixed research textbooks (e.g. Bergman 2008; Creswell & Plano Clark 2007; Greene 2007; Johnson & Christensen 2008; Plano Clark & Creswell 2007; Ridenour & Newman 2008; Teddlie & Tashakkori 2009); several mixed research articles contained in methodological handbooks (e.g. Teddlie, Tashakkori, & Johnson 2008); mixed research articles contained in encyclopedias (e.g. Onwuegbuzie 2007); two journals devoted to mixed research (i.e. Journal of Mixed Methods Research, International Journal of Multiple Research Approaches); several journals now routinely publishing mixed research (e.g. Field Methods, Educational Evaluation & Policy Analysis, Quality & Quantity, Evaluation, Evaluation Practice, Research in Nursing & Health, Research in the Schools, The Qualitative Report); and websites (e.g. http://www.fiu.edu/~bridges/), conferences (e.g. http://www.mixedmethods.leeds.ac.uk/) and workshops (Creswell & Plano Clark 2008; Mertens 2008; O'Cathain 2008; Onwuegbuzie & Collins 2008; Onwuegbuzie, Slate, Leech, & Collins 2008) devoted to mixed research, and by special issues (Gorard & Smith 2006; Johnson 2006; O'Cathain & Collins 2009) with another underway (Leech & Onwuegbuzie, 2010). These and other sources have helped to increase the visibility of mixed research.

Greene (2008: 8) recently asked the following questions:

* 'Is mixed methods social inquiry a distinctive methodology?

* Is the field moving in that direction?

* What is needed for mixed methods to become a distinctive methodology?

According to Greene (2006, 2008), the development of a methodological or research paradigm (i.e. qualitative, quantitative and mixed research) in the social and behavioral sciences requires a thorough critique of four interrelated but conceptually distinct domains:

(i) philosophical assumptions and stances (i.e. what are the core philosophical or epistemological assumptions of the methodology?);

(ii) inquiry logics (i.e. what traditionally is called methodology and refers to broad inquiry purposes and questions, logic, procedures and designs, quality standards and writing and reporting forms that guide the researcher's gaze);

(iii) guidelines for research practice (i.e. specific strategies and tools that are used to conduct research; the how to component of research methodology);

(iv) sociopolitical commitments (i.e. interests, commitments and power relations surrounding the location in society in which an inquiry is situated; proclamation of values- based rationales and meanings for the practice of social and behavioral science research in society).

Together, Greene's four domains provide a cohesive and interactive framework and an array of practical guidelines for a methodological or research paradigm. Although these domains have been more fully developed with respect to both the quantitative and qualitative research paradigms, this is not the case for the field of mixed research. Indeed, not all of Greene's (2006) domains have been fully articulated and developed in mixed research (Greene 2006, 2008). Consequently, more theoretical, conceptual and practical work is needed in the area of mixed research.

In recent years, much has been written about most of the 13 distinct, interactive, iterative steps of the mixed research process identified by Collins, Onwuegbuzie and Sutton (2006), namely:

(a) determining the mixed goal of the study

(b) formulating the mixed research objective(s)

(c) determining the rationale(s) for mixing quantitative and qualitative approaches

(d) determining the purpose(s) for mixing quantitative and qualitative approaches

(e) determining the mixed research question(s)

(f ) selecting the mixed sampling design

(g) selecting the mixed research design

(h) collecting quantitative and qualitative data

(i) transforming and analyzing the quantitative and qualitative data

(j) legitimating the data sets and mixed research findings

(k) interpreting the mixed research findings

(l) writing the mixed research report

(m) reformulating the mixed research question( s).

However, despite the fact that the mixed analysis step is considered by beginning researchers to be the most difficult step of the mixed research process, because typically this step necessitates competence in conducting both quantitative and qualitative data analyses, it is one of the least developed areas in the mixed research literature, with relatively few published articles on mixed analysis (e.g. Bazeley 2003 2006; Caracelli & Greene 1993; Chi 1997; Greene et al. 1989; Lee & Greene 2007; Li, Marquart, & Zercher 2000; Onwuegbuzie 2003; Onwuegbuzie & Dickinson 2008; Onwuegbuzie & Leech 2004, 2006; Onwuegbuzie, Slate, Leech & Collins 2007; Onwuegbuzie, Slate, Leech, & Collins 2009; Onwuegbuzie & Teddlie 2003; Sandelowski 2000, 2001; Teddlie et al. 2008). Currently, published mixed research textbooks (e.g. Bergman 2008; Creswell & Plano Clark 2007; Greene 2007; Ridenour & Newman 2008) - although groundbreaking - contain at most one chapter on mixed analysis. Thus, as noted by Greene (2008: 14): 'There has also been some work in the area of integrated mixed methods data analysis, although this work has not yet cohered into a widely accepted framework or set of ideas'.

Moreover, when discussing mixed analysis strategies, none of these authors discuss the philosophical underpinnings. Yet, by linking mixed analysis techniques to philosophical assumptions and stances, an iterative, interactive and dynamic linkage is provided among Greene's (2006, 2008) four domains.

With this in mind, the present article is a first attempt explicitly to provide a philosophical justification for conducting mixed analyses. First, we present several recent typologies of analyses in social science research that incorporate both monomethod (i.e. purely quantitative research or purely qualitative research) and mixed research studies. Second, we discuss what has been referred to as the fundamental principle of data analysis, wherein both qualitative and quantitative data analysis techniques are shaped by an attempt to analyze data in a way that yields at least one of five types of generalizations. Third, building on the frameworks of Denzin and Lincoln (2005), Heron and Reason (1997) and Johnson and Onwuegbuzie (2004), we compare and contrast three qualitative-based paradigms (i.e. constructivism, critical theory, participatory), one quantitative-based paradigm (i.e. postpositivism) and one mixed research-based paradigm (i.e. pragmatism) with respect to three axiomatic components (i.e. ontological, epistemological and methodological foundations) and seven issues (i.e. nature of knowledge, knowledge accumulation, goodness or quality criteria, values, ethics, inquirer posture and training). Also, we link each paradigm to data analysis strategies. Fourth, we illustrate similarities in goals between some qualitative and quantitative analyses; in so doing, we deconstruct the strong claim that analysis must be either qualitative or quantitative and illustrate that regardless of perspective (e.g. postpositivist or constructivist), both qualitative and quantitative data can be jointly analyzed. Finally, we compare and contrast 11 mixed research paradigms/worldviews, linking them to mixed analysis strategies.

TYPOLOGY OF ANALYSES IN SOCIAL SCIENCE RESEARCH

Onwuegbuzie et al. (2007) outlined the concept of monotype data, which represents the use of one data type (e.g. qualitative data) that is available for analysis - in contrast to multitype data wherein both types of data (i.e. qualitative and quantitative data) are collected and thus are available for analysis. Onwuegbuzie et al. (2007) coined the phrase monoanalysis to denote when one class of analysis (e.g. qualitative analysis) is used to analyze one data analysis type (e.g. qualitative data) - as opposed to multianalysis, wherein both classes of analyses (i.e. qualitative analysis and quantitative analysis) are used to analyze one or more data analysis types. For example, a quantitative researcher might use multiple regression (Fox 1997) to examine which variables predict some quantitative outcome of interest. Alternatively, a qualitative researcher might use the method of constant comparison (Glaser & Strauss 1967) to analyze responses to open-ended interview questions.

Three dimensional framework for qualitative and quantitative analyses

Onwuegbuzie et al. (2009) also conceptualized a typology for classifying qualitative and quantitative analysis techniques. Specifically, these authors presented a three-dimensional representation for classifying and organizing both qualitative and quantitative analyses, which involves reframing qualitative and quantitative analyses as a case-oriented, variable-oriented, or process/experienceoriented analyses.

1. Case-oriented analyses are analyses that focus primarily or exclusively on the selected case(s), wherein the goal is to analyze and interpret the meanings, experiences, attitudes, opinions, or the like of one or more persons - with a tendency toward particularizing and analytical generalizations. Although case-oriented analyses are best suited for identifying patterns common to one or a relatively small number of cases - and thus lend themselves better to qualitative research in general and qualitative analyses in particular - this class of analyses can be used for any number of cases and, as such, also is pertinent for quantitative research, leading to the use of quantitative analysis techniques such as single-subject analyses, descriptive analyses and profile analyses (Onwuegbuzie et al. 2009).

2. In contrast, variable-oriented analyses involve identifying relationships - often probabilistic in nature - among entities that are treated as variables such that this class of analysis tends to be conceptual and theory-centered from the onset and has a proclivity toward external generalizations. Therefore, variable-oriented analyses, whose '"building blocks" are variables and their intercorrelations, rather than cases' (Miles & Huberman 1994: 174), are more apt for quantitative research in general and quantitative analyses in particular. However, although the use of large and representative samples often facilitates identification of relationships among variables, small samples also can serve this purpose, making variable- oriented analyses also relevant for qualitative data along with the use of qualitative analysis techniques (e.g. examining themes that cut across cases).

3. Finally, process/experience-oriented analyses involve evaluating processes or experiences pertaining to one or more cases within a specific context over time, with processes tending to be associated with variables and experiences tending to be associated with people (i.e. cases).

Because each of these three analysis orientations represent a continuum rather than a dichotomy (e.g. variable-oriented analyses are conceptualized by Onwuegbuzie et al. (2009) as falling on a particularistic-universalistic continuum, classifying the extent to which the metainferences stemming from the variable-oriented analysis can be generalized), this three-dimensional framework supports Johnson and Onwuegbuzie's (2004: 20) assertion that 'the possible number of ways that studies can involve mixing is very large because of the many potential classification dimensions'.

Cross-over mixed analysis

Onwuegbuzie and Combs (2009), building on the works of Greene (2008) and Onwuegbuzie and Teddlie (2003), outlined the concept of cross-over mixed analyses, which involves using one or more analysis types associated with one tradition (e.g. quantitative analysis) to analyze data associated with a different tradition (e.g. qualitative data). Such an analysis can be used to reduce, display, transform, correlate, consolidate, compare, integrate, assert, or import data. Table 1 presents the cross-over mixed analysis types and strategies that Onwuegbuzie and Combs (2009) identified. Cross-over mixed analyses are distinct from other types of mixed analyses (i.e. non-cross-over mixed analyses) such as parallel mixed analysis, wherein both quantitative and qualitative data analyses are conducted separately, neither type of analysis builds on or interacts with the other during the data analysis stage and the findings from each type of analysis are neither compared nor consolidated until both sets of data analyses have been completed. Whereas non-cross-over mixed analyses involve collection of both types of data and the analysis conducted per data set represents the same paradigmatic tradition (i.e. either quantitative or qualitative) - 'within-paradigm analysis' (Onwuegbuzie et al. 2007: 12) - cross-over mixed analyses involve a between-paradigm analysis, which involves 'an analysis technique more associated with one traditional paradigm (e.g. quantitative) to analyze data that originally represented the type of data collected associated with the other traditional paradigm (e.g. qualitative)' (Onwuegbuzie et al. 2007: 12). Thus, cross-over mixed analyses involve more integration of qualitative and quantitative analyses than do other types of mixed analyses because they involve the mixing or combining of qualitative- and quantitative-based paradigmatic assumptions and stances (e.g. using exploratory factor analysis to examine the structure of themes that emerged from a qualitative analysis; cf. Onwuegbuzie 2003), which involves either maintaining an analytical-philosophical stance that the human mind/perception and mathematical algorithms can be used sequentially to examine patterns in qualitative data or adopting an analytical-philosophical stance that transcends the stances underlying both paradigms (e.g. assuming that data saturation [i.e. qualitative information repeats itself such that no new or relevant information seem to emerge pertaining to a category and the category development is well established and validated; Morse 1995] and reliability [i.e. consistency or repeatability of participants' quantitative responses] represent parallel constructs). This distinction between cross-over mixed analyses and non-cross-over mixed analyses is important because certain epistemological, ontological, axiological and methodological stances might lend themselves more to conducting cross-over mixed analyses than do other stances.

Building on the works of Onwuegbuzie et al. (2007), Onwuegbuzie et al. (2009) and Onwuegbuzie and Combs (2009), before conducting an analysis, a researcher explicitly or implicitly makes the following six decisions:

(a) the number of data types that will be analyzed - yielding either monotype data (i.e. use of one data type, namely: qualitative data or quantitative data) or multitype data (i.e. use of both data types, namely: qualitative data and quantitative data);

(b) the number of data analysis types that will be used - yielding monoanalysis (i.e. use of one data analysis type, namely: qualitative data analysis or quantitative data analysis) or multianalysis (i.e. use of both data analysis types, namely: qualitative data analysis and quantitative data analysis);

(c) the analysis emphasis of interest - comprising case-oriented analyses (i.e. analyses that focus primarily or exclusively on the selected case(s)), variable-oriented analyses (i.e. identifying relationships among entities that are conceived as variables) and/or process/experience-oriented analyses (i.e. evaluating processes or experiences pertaining to one or more cases within a specific context over time);

(d) whether or not analysis types associated with one tradition will be used to analyze data associated with a different tradition (i.e. cross-over mixed analysis vs. non-crossover mixed analysis);

(e) whether the qualitative and quantitative analyses will be carried out concurrently (i.e. results stemming from one data analysis phase do not inform the results stemming from the other phase) or sequentially (i.e. the qualitative analyses are conducted first, which then inform the subsequent quantitative analyses, or vice versa);

(f) whether the qualitative or quantitative analyses will be given priority (i.e. quantitative analyses carry the most weight or the qualitative phase carry the most weight), or whether they will be assigned equal status.

Monomethod studies involve the use of monotype data, monoanalysis and non-cross-over mixed analysis, thereby making the fifth (i.e. time orientation of analyses) and sixth (i.e. priority of analyses) decisions irrelevant. Monomethod studies also include a decision as to the analysis emphasis. In contrast, in mixed research, consideration of these six decisions yields a large variety of possible combinations (i.e. except the aforementioned combination associated with monomethod research) that is impossible to capture completely in any single typology.

FUNDAMENTAL PRINCIPLE OF DATA ANALYSIS

Onwuegbuzie et al. (2009) outlined what they refer to as the fundamental principle of data analysis, 1 wherein both qualitative and quantitative data analysis techniques are shaped by an attempt to analyze data in a way that yields at least one of the following five types of generalizations: external (statistical)2 generalizations, internal (statistical)3 generalizations, analytical generalizations, case-tocase transfer and/or naturalistic generalizations. External (statistical) generalization involves making generalizations, inferences, or predictions on data obtained from a representative statistical (i.e. optimally random) sample to the population from which the sample was drawn (i.e. universalistic generalizability). Contrastingly, internal (statistical) generalization involves making generalizations, inferences, or predictions on data obtained from one or more representative or elite participants (e.g. key informants, politically important cases, sub-sample members) to the sample from which the participant(s) was selected (i.e. particularistic generalizability). It should be noted that internal (statistical) generalization can stem from qualitative, quantitative, or mixed analyses. Conversely, with analytic generalizations 'the investigator is striving to generalize a particular set of [case study] results to some broader theory' (Yin 2009: 43). In other words, analytical generalizations are 'applied to wider theory on the basis of how selected cases "fit" with general constructs' (Curtis, Gesler, Smith, & Washburn 2000: 1002) (i.e. particularistic generalizability). Case-to-case transfer involves making generalizations or inferences from one case to another (similar) case (Firestone 1993; Kennedy 1979; Miles & Huberman 1994) (i.e. particularistic generalizability). Finally, with naturalistic generalization, the readers of the article (i.e. consumers of the findings) make generalizations entirely, or at least in part, from their personal or vicarious experiences (Stake 2005), such that meanings stem from personal experience and are adapted and reified by repeated encounter (Stake 1980; Stake & Trumbull 1982). Rather than the researcher making this sort of generalization, it is the reader who generalizes (often based on proximal similarity of the case data to the reader's focal context of interest).

LINKING PARADIGMS TO DATA ANALYSIS STRATEGIES

Over the years, in an attempt to distinguish qualitative- based paradigms from quantitative-based paradigms and to demonstrate that qualitative research represents a distinct tradition, several eminent qualitative researchers have presented frameworks that contrast qualitative-based paradigms (e.g. constructivism, critical theory, participatory) and quantitative-based paradigms (i.e. [logical] positivism and postpositivism). Indu- bitably, the most popularized framework is that developed by Guba (1990) and extended, more recently, by Denzin and Lincoln (2005). Denzin and Lincoln outlined their views of the axiomatic nature of paradigms and the issues they believed were most fundamental to differentiating the paradigms by contrasting three qualitative-based paradigms (i.e. constructivism, critical theory, participatory) and two quantitative-based paradigms (i.e. [logical] positivism and postpositivism) with respect to three axiomatic components (i.e. ontological, epistemological and methodological foundations) and seven issues (i.e. nature of knowledge, knowledge accumulation, goodness or quality criteria, values, ethics, inquirer posture and training).

In Table 2, we build on Denzin and Lincoln's (2005) table of axioms and issues. Specifically, Table 2 contains Denzin and Lincoln's (2005: 193-194) axioms of the paradigms of postpositivism and constructivism, as well as axioms of the participatory paradigms outlined by Heron and Reason (1997) (which was also included in Denzin and Lincoln's [2005] table). Table 2 also contains our proposed pragmatist paradigm, which incorporates some of the major concepts outlined in Johnson and Onwuegbuzie's (2004) seminal article on mixed research. However, Table 2 does not include the axioms of the paradigm of positivism because this paradigm, which incorporates several movements (e.g. analytical philosophy, logical atomism, logical empiricism and semantics), was discredited shortly after World War II.4 As noted by Johnson and Onwuegbuzie (2004):

Positivism is a poor choice for labeling quantitative researchers today because positivism has long been replaced by newer philosophies of science (Yu 2003). The term is more of a straw man (easily knocked down) for attack than standing for any actual practicing researchers. A term that better represents today's practicing quantitative researchers is postpositivism (Phillips & Burbules 2000: 24).

In this table, we have included two additional issues to Denzin and Lincoln's seven issues - namely, qualitative analysis and quantitative analysis - in an attempt to take the first step toward linking data analysis techniques to philosophical assumptions and stances. In Figure 1, we reproduce Onwuegbuzie et al.'s (2009) typology for classifying qualitative and quantitative analysis techniques. This figure presents an array of analytic techniques undertaken in quantitative and qualitative research as a function of the following three analysis emphases: case-oriented analyses, variable-oriented analyses and process/ experience-oriented analyses. Although this list is by no means exhaustive, it does capture most of the major qualitative and quantitative analytical techniques (Onwuegbuzie et al. 2009).

Postpositivist paradigm

Often under-emphasized in the literature, postpositivism is a rejection or modification of several core tenets of positivism. Although postpositivist researchers believe that there is an independent reality that can be studied, they assert that all observation is inherently theory-laden and fallible and that all theory can be modified. They also believe that, as a result of their cultural experiences and worldviews, people are always partially biased in their objective perceptions of reality. Given these inherent biases and perceptions and observations that are fallible, ensuing constructions are imperfect (Phillips & Burbules 2000). Thus, postpositivist researchers assert that we can only approximate the truth of reality but can never explain it perfectly or completely. Notwithstanding, postpositivist researchers believe that objectivity can be approximated by triangulating across these multiple fallible perspectives (i.e. triangulation of method, data and theory). Karl Popper (1994) identified three worlds that postpositivist researchers can address: (a) an objective physical external world (World 1); (b) an interpretative, subjective inner world (World 2); and (c) the theory world where humans mentally and physically represent the first two worlds (World 3). Popper's views of knowledge and reality are just one example of complexity and nuance commonly expressed by quantitative researchers that is not represented in simple statements found in paradigm comparison tables popular in the research literature.

Belief in the fallibility of observations renders statistics in general, and inferential statistics in particular, as suited to postpositivist research due to its emphasis on assigning probabilities (e.g. pvalues, levels of confidence or error) to observed findings. Thus, as well as utilizing descriptive statistics, postpositivist researchers use the whole array of inferential statistics for making external (statistical) generalizations (cf. Figure 1). Contrary to popular depictions of quantitative research as being deductive (and qualitative as inductive), inferential statistical generalizations are inductive. Postpositivist researchers also utilize some qualitative analysis techniques, especially those that yield frequency data such as Word Count and Classical Content Analysis (cf. Leech & Onwuegbuzie 2007, 2008). Postpositivist researchers also employ qualitative data analysis techniques that help them develop quantitative instruments.

Constructivist paradigm

As can be seen in Table 2, constructivist researchers (e.g. radical constructivists, cognitive constructivists, cultural constructivists, social constructivists/constructionists, communal constructivists, critical constructivists, genetic epistemology) often claim to believe that multiple, contradictory, but equally valid accounts of the same phenomenon (i.e. multiple realities) can exist. Thus, it is somewhat contradictory that those who ascribe to strong relativism or strong constructivism do not view the use of quantitative methods in general and quantitative analysis in particular as representing one such valid account of a phenomenon - albeit not their preferred account. However, the fact that constructivists tend to believe that time- and context-free generalizations are neither desirable nor possible, likely renders inappropriate the use of inferential statistics such as the sample mean for the purpose of making external (statistical) generalizations across populations (i.e. claiming that a statistical parameter applies broadly to nearly everyone in a population). Generalizing to a population (e.g. claiming that the sample statistic such as the sample mean is a reasonable representation of the corresponding population parameter) should be less controversial. This distinction of generalizing across versus generalizing to a population (Cook & Campbell 1979) seldom is addressed in the paradigm debates. Parameter estimation would probably be of more interest to a social constructivist than a radical or cognitive constructivist.

Descriptive statistics has long been an important part of ethnography in anthropology as ethnographers include quantitative descriptors to complement narrative description. Descriptive statistics, more generally, can be used to enhance the qualitative researcher's quest for detailed description. Sechrest and Sidani note (1995: 79) 'qualitative researchers regularly use terms such as 'many,' 'most,' 'frequently,' 'several,' 'never,' and so on. These terms are fundamentally quantitative.' Qualitative researchers also can obtain more meaning by obtaining counts of words in addition to their narrative descriptions (Sandelowski 2001). For example, in examining the lived experience in the classroom of Johnny, a child with attention deficit hyperactivity disorder, rather than telling readers that Johnny left his seat on many occasions during a class, it would be informative for the qualitative researcher to note that Johnny left his seat six times during the course of 30 minutes. This way, the readers can (if contextual detail is provided) decide whether six incidences of out-ofseat behavior are significant/meaningful and also will be in a better position to decide whether to make naturalistic generalizations.

Consistent with our recommendation for qualitative researchers to use descriptive statistics more frequently where applicable, more than one half a century ago, Barton and Lazarsfeld (1955) advocated the use of what they coined quasi-statistics in qualitative research. According to these authors, quasi-statistics pertain to the use of descriptive statistics that can be extracted from qualitative data. Interestingly, the prominent symbolic interactionist Howard Becker (1970: 81-82) contended 'one of the greatest faults in most observational case studies has been their failure to make explicit the quasi-statistical basis of their conclusions.'. Further, Joseph Maxwell (1996: 95) noted that:

Quasi-statistics not only allow you to test and support claims that are inherently quantitative, but also enable you to assess the amount of evidence in your data that bears on a particular conclusion or threat, such as how many discrepant instances exist and from how many different sources they were obtained. [emphasis in original]

Interestingly, Becker, Geer, Hughes and Strauss (1961/1977) provided more than 50 tables and graphs in their qualitative works. These tables and graphs complemented the narrative descriptions of their qualitative data.

Internal (statistical) generalizations also are possible in constructivist research. In particular, it is not unusual for constructivist researchers to utilize key informants to provide them with an insider's understanding and to provide information that the researcher is unable to experience. This often includes both qualitative and quantitative information. For example, the informant might state what many members believe or describe as characteristics of many (or only a few) people in the group. These numbers would contribute to internal generalizations. In particular, inferential statistics can be used to make internal (statistical) generalizations. Educational ethnographers Margaret LeCompte and Jude Preissle (1993) discuss enumeration in ethnographic data analysis in some depth in their qualitative research textbook.

The fact that many computer-assisted qualitative data analysis software (CAQDAS) programs allow data to be imported to statistical software programs (e.g. Excel, SIMSTAT) supports our assertion that descriptive and inferential statistical analyses are an option for constructivist researchers should they deem it appropriate (e.g. based on their philosophical stance and research question(s)) to make internal (statistical) generalizations. These statistical analysis tools also can be used to bolster analytical generalizations. One of the most popular methods of qualitative research, grounded theory, is premised on the development of theory for the purposes of generalization. Descriptive and inferential statistics can be used to facilitate rich and detailed description, and to assess and enhance trustworthiness, dependability, confirmability, transferability and authenticity. Consistent with this assertion, Denzin and Lincoln (2005) state that the training of constructivists includes both qualitative and quantitative techniques (cf. Table 2). Whatever findings emerge from descriptive and/or inferential statistical analyses utilized, it should be noted that for constructivists, these findings represent just one of the multiple valid accounts of the phenomenon.

Critical theory paradigm

Critical theory researchers seek to understand the relationship between societal structures (e.g. economic, political) and ideological patterns of thought that impede a person or group from identifying, confronting and addressing unjust social systems. Simply put, critical theory researchers primarily are interested in social change as it emerges in relation to social struggle. Moreover, critical theory researchers operate under the assumption that the knowledge gleaned from their research represents an initial step toward addressing social injustices and promoting social change. As such, critical theory research represents a form of transformative research by providing emancipatory knowledge that identifies the contradictions that are masked or distorted by our everyday thoughts and perceptions (Lather 1986). Whereas many other types of qualitative researchers (e.g. phenomenological researchers, constructivist researchers) are interested in the meanings that people attach to their own actions, critical theory researchers aim to place such actions in a broader context that is framed by social, economic, political and ideological forces that have previously not been identified or acknowledged. That is, critical theory researchers are more concerned with the forces that limit actions rather than the actions themselves. Thus, critical theory researchers tend to embrace a more etic (i.e. outsider's) stance than an emic (i.e. insider's) stance.

Although it might appear that critical theory research is similar to postpositivist researchers - where critical theory researchers seek cause-andeffect relationships wherein the actions of individuals or groups are influenced directly by social, economic, political and/or ideological variables - many critical theory researchers would consider this strategy as being inappropriately reductionistic. Rather, many postmodern critical theory researchers believe that it is not possible to predict reliably how these variables determine actions. Further, instead of focusing on foundational criteria to justify its research findings, critical theory researchers contend that, by attending to the role of power in social systems, their analyses occur at the meta-theoretical level. Drawing upon research from other paradigms, critical theory researchers identify power structures and their processes that are typically ignored in both postpositivist and constructivist research.

Like constructivist researchers, critical theory researchers believe that meaning and language are socially constructed and are neither time- nor context- free, although critical theory researchers assert that social injustice has real (i.e. objective, material) consequences. Also, like constructivist researchers, critical theory researchers primarily assess their findings with respect to the community of researchers to which they belong (i.e. authoritative consensus). Critical theory researchers contend that the subjective/objective dualism camouflages the ways in which both standpoints are constrained by power dynamics and social forces. However, critical theory researchers endorse subjectivism inasmuch as the subjective knower and known cannot be separated and that knowledge is subjective (culturally and historically embedded) and constructed on the basis of power issues (Lather 2006); yet, they reject the stance that all analyses are relative, instead, believing that rational analysis is essential to social justice. Thus, many critical theorists endorse critical realism (Morrow & Brown 1994).

Critical theory researchers believe that constructivists place too much weight on people's perceptions and too little emphasis on the more complex social forces that shape and constrain experiences, events and actions. As such, critical theory researchers deem constructivism to be just as much at risk as is postpositivism of uncritically bolstering the status quo of social injustice. According to Habermas, a leading contemporary proponent of critical theory, critical theory involves the following three types of knowledge: (a) the empirical-analytic sciences (i.e. including both the natural sciences and social sciences, which seek to manipulate knowledge for the purposes of prediction and control over nature and social structure); (b) historical-hermeneutic sciences (i.e. including the cultural and human sciences, which seek to understand communication among and within social groups); and (c) critical theory (i.e. which seeks freedom from oppression and social justice) (Blaikie 1993). Thus, Habermas advocates the use of both the method of the empirical-analytic sciences, historical-hermeneutic sciences and critical theory.

However, critical theory researchers believe that critical theory utilizes but transcends both the empirical-analytic and historical-hermeneutic sciences inasmuch as it provides knowledge that is subjected to rigorous processes of free and rationale discourse, which prevents it from being socially distorted (Blaikie 1993). This suggests that the ontological, epistemological and axiological stances of critical theorist researchers do not prevent them from using, as appropriate, all forms of qualitative analysis techniques and quantitative analysis techniques that include both descriptive and inferential statistics (cf. Figure 1). As noted by Morrow and Brown (1994: 200), 'critical theory has no basis for a priori rejection of any particular methods or techniques as such'. However, whatever analysis is used, critical theory researchers make no claims that their analyses are objective in the manner claimed by some postpositivists.

Participatory paradigm.

As noted by Heron and Reason (1997), proponents of the participatory paradigm subscribe to an experiential subjective-objective reality that stems from a co-creative interaction between the given cosmos and the way the mind connects with it. Thus, what can be known about the given cosmos is that it represents a subjectively articulated world, whose objectivity reflects how it is intersubjectively formed by the knower. A participative worldview acknowledges the following four interdependent ways of knowing: experiential (i.e. direct encounter), presentational (i.e. emerges from and grounded on experiential knowing), propositional (i.e. conceptually knowing that something or other is the case) and practical (i.e. knowing how to do something) (Heron & Reason 1997). Of these, propositional knowing is most compatible with statistical analyses. In particular, propositional knowing represents knowledge that comes to the fore by describing some person, group, entity, location, situation, process, or the like. Simply put, it represents knowledge of facts. This way of knowing is articulated via declarations and theories that stem from mastery of concepts that language provides. This suggests that propositional knowledge can be expressed not only via qualitative analysis techniques but also via an array of quantitative techniques that include both descriptive and inferential statistics. For example, inferential statistics could be used to attach levels of certainty (i.e. probability) to knowing that something is the case. Interestingly, as noted by Heron and Reason (1997), intervention research represents a form of participative inquiry that includes statistical analyses (cf. Fryer & Feather 1994).

When studying a group or community, participatory researchers might use inferential techniques to test theory that represents propositional knowledge. Confirmation of a theory then would lead the participatory researcher to make external (statistical) generalizations from the participants to the group or community from which the participants were selected. Alternatively, when studying key informants or sub-sample members, a participatory researcher might use inferential statistics to make internal (statistical) generalizations to assess propositional knowledge. Alternatively still, when studying one or a few people, then only descriptive statistics might be appropriate.

The quest for practical knowing also invites statistical analyses. In particular, practical knowing assumes a conceptual understanding of standards and principles of practice, presentational stylishness and experiential grounding in the situation within which the action takes place. In this respect, participatory research coincides with pragmatist research, which endorses a strong and practical empiricism, in which both qualitative and qualitative data are collected and analyzed to generate and to test theories against observations of the natural world. As such, participatory research does not appear to invalidate the use of descriptive or inferential statistical analyses and thus all analyses in Figure 1 are available to participatory researchers.

Pragmatist paradigm

The pragmatist paradigm offers an epistemological justification (i.e. via pragmatic epistemic principles and standards) and logic (i.e. combining approaches that help researchers optimally frame, examine and provide tentative answers to one's research question[s]) for mixing approaches and methods (Johnson, Onwuegbuzie, & Turner 2007; Onwuegbuzie & Leech 2005). Further, a pragmatist would reject the incompatibility thesis that qualitative and quantitative research are fully incompatible and cannot, in any useful way, be used in combination in social or behavioral research. A serious problem with the incompati- bility thesis is that it is an a priori argument; it seems to be based on rationalistic, foundational, deductive logic (which oftentimes is said to have been rejected in qualitative research). The incompatibility thesis does not appear to be based on observation of social and behavioral research (i.e. through an examination of how researchers actually conduct their research). In pragmatist research, research paradigms can remain separate, but they also can be mixed or combined into another research paradigm (i.e. mixed research). Johnson et al. (2007: 129) asserts that pragmatist researchers may ascribe to the view of mixed methods research is a research paradigm that:

(a) partners with the philosophy of pragmatism in one of its forms (left, right, middle);

(b) follows the logic of mixed methods research (including the logic of the fundamental principle and any other useful logics imported from qualitative or quantitative research that are helpful for producing defensible and usable research findings);

(c) relies on qualitative and quantitative viewpoints, data collection, analysis and inference techniques combined according to the logic of mixed methods research to address one's research question(s); and

(d) is cognizant, appreciative and inclusive of local and broader sociopolitical realities, resources and needs.

The mixed methods research paradigm offers an important approach for generating important research questions and providing warranted answers to those questions. This type of research should be used when the nexus of contingencies in a situation, in relation to one's research question( s), suggests that mixed methods research is likely to provide superior research findings and outcomes.

As noted in Table 2, pragmatist researchers can use the whole range of qualitative analyses and quantitative (i.e. descriptive and inferential analytical techniques) in an attempt to fulfill one or more of five mixed research purposes identified by Greene et al. (1989): tri-angulation (i.e. comparing findings from quantitative data with qualitative results in hopes of convergence); complementarity (i.e. seeking elaboration, illustration, enhancement and clarification of the results from one method with findings from the other method); development (i.e. using the results from one method to help inform the other method); initiation (i.e. discovering paradoxes and contradictions that culminate in a re-framing of the research question); and expansion (i.e. expanding the breadth and range of a study by using multiple methods for different study phases).

Decisions made regarding these five mixed research purposes and the resulting mixed research designs help pragmatist researchers to determine which of three types of mixed analysis should be undertaken: a parallel mixed analysis (i.e. findings obtained from both analysis phases are interpreted separately), concurrent mixed analysis (i.e. results stemming from one data analysis phase do not inform the results stemming from the other phase), or sequential mixed analysis (i.e. the qualitative analysis phase is conducted first, which then informs the subsequent quantitative analysis phase, or vice versa). In particular, if the purpose of mixing is complementarity, then all three families of mixed analyses (i.e. parallel, concurrent and sequential) can be used. If triangulation or initiation represents the purpose, then both parallel and concurrent mixed analyses are viable. If development is the purpose, then concurrent and sequential mixed analyses are appropriate. Finally, if the purpose of mixing analyses is expansion then a sequential mixed analysis is pertinent (Onwuegbuzie et al. 2007).

The purposes of mixed research and the resulting kinds of analyses also will help pragmatist researchers determine the number of data types that will be analyzed (i.e. monotype data vs. multitype data), the number of data analysis types that will be used (i.e. monoanalysis vs. multianalysis), the analysis emphasis of interest (i.e. case-oriented analyses, variable-oriented analyses and/or process/experience-oriented analyses), whether or not analysis types associated with one tradition will be used to analyze data associated with a different tradition (i.e. cross-over mixed analysis vs. non-cross-over mixed analysis) and whether the qualitative or quantitative analyses will be given priority, or whether they will be assigned equal status.

Pragmatist researchers also can use inferential statistics to make internal (statistical) generalizations. For instance, in the field of linguistics, political science and the like, it is not unusual for researchers analyzing qualitative data to conduct sequence analyses wherein hypotheses regarding patterns and trends in the qualitative data are tested (i.e. using p-values, confidence intervals and/or effect sizes). As an example, these researchers might test whether themes occurring among the participants statistically and/or practically significantly tend to emerge in a particular order. QDA Miner 3.0 is a computer- assisted qualitative data analysis software (CAQDAS) program that allows researchers to conduct qualitative analyses, such as thematic analysis and also to sequence analyses and other statistical and visualization tools such as clustering, multidimensional scaling, heatmaps and correspondence analysis. The fact that CAQDAS programs increasingly are allowing inferential statistical analyses to be conducted supports our assertion that inferential statistical analyses are an option that analysts of qualitative data have should they deem it appropriate to make statistical (i.e. internal and to a lesser degree external) generalizations. These inferential statistical analysis tools also can be used to bolster analytical generalizations. The descriptive and inferential statistics could be used not only to facilitate rich and detailed description but also could be used to assess and enhance trustworthiness, dependability, confirmability, transferability and authenticity.

For pragmatist research, having a postpositivist orientation does not prevent a researcher from conducting qualitative analyses - especially analyses such as word count and content analysis that involve, in part, the counting of words, codes, categories, or other aspects of the qualitative data. Such analyses can form part of quantitative-dominant mixed analyses (cf. Johnson et al. 2007), in which the analyst adopts a postpositivist stance, while, at the same time, believing that the inclusion of qualitative data and approaches are likely to enhance the findings. Similarly, having a constructivist orientation - or any other qualitativebased orientation - does not prevent a researcher from conducting quantitative analyses - especially descriptive statistical analyses that do not involve the analyst making inferences beyond the research participants at hand, which is typically not the goal of qualitative researchers. Such analyses can form part of qualitative-dominant mixed analyses (cf. Johnson et al. 2007), in which the researcher takes a constructivist-poststructuralist- critical stance with respect to the mixed analysis process, while, at the same time, deeming the addition of quantitative data and approaches as helpful in providing richer data and interpretations.

Commonalities regarding data analysis strategies across paradigms

From Table 2, the table of axioms and issues, it can be seen that the ontological, epistemological and methodological assumptions and stances representing all five paradigms allow the conduct of both quantitative and qualitative analyses - at least to a small degree - with postpositivist and constructivist paradigms having the least potential to use analytical techniques that belong to a different paradigm, the critical theory and the participatory paradigms having excellent potential to use analytical techniques that belong to a different paradigm and the pragmatist paradigm, almost by definition, having the greatest potential.

That the ontological, epistemological and methodological assumptions and stances representing all five paradigms legitimate both quantitative and qualitative analyses to be undertaken is especially apparent when one examines the similarities in goals between many quantitative and qualitative analyses rather than emphasizing the differences.5 First and foremost, both quantitative and qualitative researchers analyze empirical observations (i.e. data coming from personal experience, observation, or experiment) to address research questions (Johnson & Onwuegbuzie 2004). Sechrest and Sidani (1995: 78) note that both sets of researchers 'describe their data, construct explanatory arguments from their data and speculate about why the outcomes they observed happened as they did.'. At the level of data analysis, for example, numeric data are similar to textual/visual data inasmuch as they represent (descriptive) codes that characterize the person's meanings, beliefs, attitudes and so on; thus, both data types might be available for collection and analysis regardless of the research paradigm involved, depending on the research question(s). Indeed, both numeric data and textual/ visual data can be used to facilitate any of the following types of coding: inductive coding, deductive coding, abductive coding, interpretive coding, open coding, axial coding and selective coding (cf. Miles & Huberman 1994). As another example, thematic analysis (qualitative technique) and exploratory factor analysis (quantitative technique) have very similar goals, namely, to reduce the dimensionality of the raw data. This similarity allows mixed researchers to use analyses associated with one paradigm on data associated with another paradigm. For instance, it is not unusual for a mixed researcher to factor analyze themes that emerged from textual data (cf. Onwuegbuzie 2003)6 or to undertake a profile analysis of a set of quantitative measures stemming from one or more cases (Tashakkori & Teddlie 1998). Statistical factor analysis is an exploratory data analysis technique that searches for sets of variables that are similar to one another but are different from the other sets of variables found in the data. The researcher must label the factors that emerge from the data; that is, the research must give a name to the factors and interpret what the named factors mean. The naming part of factor analysis is an example of the use of qualitative coding within a technique traditionally viewed as quantitative. Factor analysis, we argue, has both quantitative and qualitative elements and, therefore, is an inherently mixed analysis procedure.

An even more compelling explanation for our assertion - that the ontological, epistemological and methodological assumptions and stances representing all five paradigms allow both quantitative and qualitative analyses to be undertaken - stems from the nature of qualitative and quantitative analyses themselves. Many of the core analytical techniques that are associated with both qualitative and quantitative paradigms are not as pure as is contended by proponents of monomethod research. For instance, with respect to qualitative research, the concepts of data saturation, informational redundancy and/or theoretical saturation (i.e. when no new or relevant information seem to emerge pertaining to a category and the category development is well established and validated; Flick 1998; Lincoln & Guba 1985; Morse 1995; Strauss & Corbin 1990), although inherently interpretivist, have quantitative undertones. These concepts incorporate quantitative assumptions - including the idea of internal replication (i.e. findings that are replicated by other participants in the study). In order to reach conclusions about saturation or informational redundancy, some kind of formal (i.e. conscious) or informal (e.g. subconscious) assessment of degree or amount typically takes place to determine how exhaustive the data are. Also, any conclusion on the part of the qualitative researcher that saturation or informational redundancy has taken place is accompanied by some degree of confidence or even certainty (i.e. high probability) - whether or not this confidence is estimated (which is a strategy that a postpositivist researcher likely would pursue). As the early modern continental philosopher Immanuel Kant (1781/1998) pointed out, the categories of qualitative and quantitative are necessarily part of human thought and the conclusions we construct about entities that are important to us.

With respect to quantitative research, techniques such as exploratory factor analysis and cluster analysis, although inherently postpositivist, have constructivist leanings inasmuch as for any given dataset, there are myriad mathematical solutions (i.e. factor/cluster structures) that can be constructed and the analyst has to make sense of the selected mathematical solution (i.e. meaningmaking). Indeed, there are numerous strategies (e.g. principal component analysis vs. factor analysis; correlation matrix vs. variance-covariance matrix; maximum likelihood vs. unweighted least squares vs. generalized least squares vs. principle axis factoring vs. alpha factoring vs. image factoring; eigenvalues; trace; scree plot; parallel analysis; number of iterations; orthogonal vs. oblique rotation; factor pattern matrix vs. factor structure matrix; communality estimates; internal replication techniques such as bootstrapping, jackknifing and cross-validation; cf. Henson, Capraro, & Capraro 2004; Henson & Roberts 2006; Hetzel 1996; Onwuegbuzie & Daniel 2003) and criteria that are available for deciding on the most appropriate factor structure. With so many decisions to make, two or more analysts easily can arrive at different final factor solutions, which indicates that exploratory factor analysts serve, to a significant degree, as research instruments themselves - a concept that some postpositivist researchers might be reluctant to acknowledge due to a stance that social science inquiry should be objective and that postpositivist researchers should eliminate all biases (cf. Table 2).

Interestingly, the role of postpositivist researcher-as-instrument is not restricted to analyses that are exploratory in nature (e.g. exploratory factor analysis, cluster analysis, multidimensional scaling) but also include analyses that involve null hypothesis significance testing (NHST). In fact, the more sophisticated the NHST-based analysis, the more subjective decisions the postpositivist analyst has to make, making him/her serve as a research instrument to an even larger degree. For example, structural equation modeling (SEM; e.g. Schumacker & Lomax 1996) and hierarchical linear modeling (HLM; e.g. Bryk & Raudenbush 1992) typically involve the analyst making numerous subjective decisions, including selecting how many and which models to test and selecting from the numerous goodness-of-fit indices and criteria available. Typically, the theoretical models are literally created by the researcher (after an inductive analysis of past research, current theory, hunches, etc.). Thus, it is difficult for advocates of any of the research paradigms to claim justifiably a one-to-one correspondence between ontology/epistemology and type of analysis (i.e. qualitative vs. quantitative).

Moreover, we contend that pragmatist researchers who hold a mixture of philosophical positions (i.e. belonging to both quantitative and qualitative traditions) find it natural to combine statistical analyses with an array of qualitative analyses. In order to accomplish such an embed- ded analysis, the mixed researcher has to make Gestalt switches from a quantitative lens to a qualitative lens and vice versa, going back and forth, multiple times (Kuhn 1962). According to Onwuegbuzie and Johnson (2006), this series of switching yields a new or consolidated analysis - such as cross-over mixed analyses - which, if effective, can yield more fully mixed meta-inferences that incorporate a strong use of both quantitative and qualitative assumptions and stances and that represent the strongest paradigmatic mixing. More generally in the mixed research literature, this purposeful switching is called the dialectical approach (Greene 2007; Johnson 2008). This approach can rely on a single individual trained in both qualitative and quantitative research or, if no qualified individual is available, it can rely on a research team composed of at least one qualitative and one quantitative researcher. A recent example of a qualitative-quantitative research team is found in Corden and Hirst (2008).

Mixed research philosophical paradigms

Pragmatism is only one of many stances that underlie mixed research (Greene 2007, 2008; Johnson et al. 2007). Current stances include the pragmatism-of-the-middle philosophy (Johnson & Onwuegbuzie 2004), pragmatism-of-the-right philosophy (Putnam 2002; Rescher 2000), pragmatism- of-the-left philosophy (Maxcy 2003; Rorty 1991), the anti-conflationist philosophy (Bryman 1992; Hammersley 1992; Layder 1993; Roberts 2002), critical realist orientation (Houston 2001; Maxwell 2004; McEvoy & Richards 2003, 2006), the dialectical stance (Greene 2008; Greene & Caracelli 1997; Maxwell & Loomis 2003), complementary strengths stance (Brewer & Hunter 1989; Morse 2003), transformativeemancipatory stance (Mertens 2003), a-paradigmatic stance (Patton 2002; Reichardt & Cook 1979), substantive theory stance (Chen 2006) and, most recently, communities of practice stance (Denscombe 2008).

Table 3 provides one way of representing the major mixed research stances. In this table, these leading mixed research philosophical paradigms are presented (left column), alongside a summary of their major stances (middle column) and the core analyses that are associated with these stances (right column). This table represents a first attempt to link mixed research paradigms and stances to mixed analysis strategies. However, clearly, more work is needed here to enhance our understanding of the mixed analysis choices made by mixed researchers and thus address Greene's (2008: 13) important question: 'how do the assumptions and stancesinfluence inquiry decisions?'.

CONCLUSIONS

The present article represents a first attempt, albeit a tentative one, explicitly to provide a philosophical justification for conducting mixed analyses. As noted by Greene (2008: 12): 'Our thinking about the nature and role of philosophical assumptions in our practice needs to make more practical sense, as well as offer possibilities to practitioners not yet envisioned, which is one key role of mixed methods theory'. Although we recognize that much more work is needed in this area, we hope that our article will motivate constructive and positive dialogues among mixed researchers and researchers representing qualitative- and quantitative- based paradigms (Onwuegbuzie 2002), so that, regardless of our differences in philosophical assumptions and stances, we all come to view all researchers as belonging to communities of interest (cf. Fischer 2001). Further, we hope that our article will motivate others either to refute or to build on the ideas and concepts we have presented. To the extent that our ideas and concepts make practical sense, we hope that our article will help better situate mixed analyses in the philosophy of science, thereby promoting mixed research as a distinctive methodology - consistent with the call made by Greene (2006).