Taking a complexity perspective.
The first paper in this series 17 outlines aspects of complexity associated with complex interventions and health systems that can potentially be explored by different types of evidence, including synthesis of quantitative and qualitative evidence. Petticrew et al 17 distinguish between a complex interventions perspective and a complex systems perspective. A complex interventions perspective defines interventions as having “implicit conceptual boundaries, representing a flexible, but common set of practices, often linked by an explicit or implicit theory about how they work”. A complex systems perspective differs in that “ complexity arises from the relationships and interactions between a system’s agents (eg, people, or groups that interact with each other and their environment), and its context. A system perspective conceives the intervention as being part of the system, and emphasises changes and interconnections within the system itself”. Aspects of complexity associated with implementation of complex interventions in health systems that could potentially be addressed with a synthesis of quantitative and qualitative evidence are summarised in table 2 . Another paper in the series outlines criteria used in a new evidence to decision framework for making decisions about complex interventions implemented in complex systems, against which the need for quantitative and qualitative evidence can be mapped. 16 A further paper 18 that explores how context is dealt with in guidelines and reviews taking a complexity perspective also recommends using both quantitative and qualitative evidence to better understand context as a source of complexity. Mixed-method syntheses of quantitative and qualitative evidence can also help with understanding of whether there has been theory failure and or implementation failure. The Cochrane Qualitative and Implementation Methods Group provide additional guidance on exploring implementation and theory failure that can be adapted to address aspects of complexity of complex interventions when implemented in health systems. 19
Health-system complexity-related questions that a synthesis of quantitative and qualitative evidence could address (derived from Petticrew et al 17 )
Aspect of complexity of interest | Examples of potential research question(s) that a synthesis of qualitative and quantitative evidence could address | Types of studies or data that could contribute to a review of qualitative and quantitative evidence |
What ‘is’ the system? How can it be described? | What are the main influences on the health problem? How are they created and maintained? How do these influences interconnect? Where might one intervene in the system? | Quantitative: previous systematic reviews of the causes of the problem); epidemiological studies (eg, cohort studies examining risk factors of obesity); network analysis studies showing the nature of social and other systems Qualitative data: theoretical papers; policy documents |
Interactions of interventions with context and adaptation | Qualitative: (1) eg, qualitative studies; case studies Quantitative: (2) trials or other effectiveness studies from different contexts; multicentre trials, with stratified reporting of findings; other quantitative studies that provide evidence of moderating effects of context | |
System adaptivity (how does the system change?) | (How) does the system change when the intervention is introduced? Which aspects of the system are affected? Does this potentiate or dampen its effects? | Quantitative: longitudinal data; possibly historical data; effectiveness studies providing evidence of differential effects across different contexts; system modelling (eg, agent-based modelling) Qualitative: qualitative studies; case studies |
Emergent properties | What are the effects (anticipated and unanticipated) which follow from this system change? | Quantitative: prospective quantitative evaluations; retrospective studies (eg, case–control studies, surveys) may also help identify less common effects; dose–response evaluations of impacts at aggregate level in individual studies or across studies included with systematic reviews (see suggested examples) Qualitative: qualitative studies |
Positive (reinforcing) and negative (balancing) feedback loops | What explains change in the effectiveness of the intervention over time? Are the effects of an intervention are damped/suppressed by other aspects of the system (eg, contextual influences?) | Quantitative: studies of moderators of effectiveness; long-term longitudinal studies Qualitative: studies of factors that enable or inhibit implementation of interventions |
Multiple (health and non-health) outcomes | What changes in processes and outcomes follow the introduction of this system change? At what levels in the system are they experienced? | Quantitative: studies tracking change in the system over time Qualitative: studies exploring effects of the change in individuals, families, communities (including equity considerations and factors that affect engagement and participation in change) |
It may not be apparent which aspects of complexity or which elements of the complex intervention or health system can be explored in a guideline process, or whether combining qualitative and quantitative evidence in a mixed-method synthesis will be useful, until the available evidence is scoped and mapped. 17 20 A more extensive lead in phase is typically required to scope the available evidence, engage with stakeholders and to refine the review parameters and questions that can then be mapped against potential review designs and methods of synthesis. 20 At the scoping stage, it is also common to decide on a theoretical perspective 21 or undertake further work to refine a theoretical perspective. 22 This is also the stage to begin articulating the programme theory of the complex intervention that may be further developed to refine an understanding of complexity and show how the intervention is implemented in and impacts on the wider health system. 17 23 24 In practice, this process can be lengthy, iterative and fluid with multiple revisions to the review scope, often developing and adapting a logic model 17 as the available evidence becomes known and the potential to incorporate different types of review designs and syntheses of quantitative and qualitative evidence becomes better understood. 25 Further questions, propositions or hypotheses may emerge as the reviews progress and therefore the protocols generally need to be developed iteratively over time rather than a priori.
Following a scoping exercise and definition of key questions, the next step in the guideline development process is to identify existing or commission new systematic reviews to locate and summarise the best available evidence in relation to each question. For example, case study 2, ‘Optimising health worker roles for maternal and newborn health through task shifting’, included quantitative reviews that did and did not take an additional complexity perspective, and qualitative evidence syntheses that were able to explain how specific elements of complexity impacted on intervention outcomes within the wider health system. Further understanding of health system complexity was facilitated through the conduct of additional country-level case studies that contributed to an overall understanding of what worked and what happened when lay health worker interventions were implemented. See table 1 online supplementary file 2 .
There are a few existing examples, which we draw on in this paper, but integrating quantitative and qualitative evidence in a mixed-method synthesis is relatively uncommon in a guideline process. Box 2 includes a set of key questions that guideline developers and review authors contemplating combining quantitative and qualitative evidence in mixed-methods design might ask. Subsequent sections provide more information and signposting to further reading to help address these key questions.
Compound questions requiring both quantitative and qualitative evidence?
Questions requiring mixed-methods studies?
Separate quantitative and qualitative questions?
Separate quantitative and qualitative research studies?
Related quantitative and qualitative research studies?
Mixed-methods studies?
Quantitative unpublished data and/or qualitative unpublished data, eg, narrative survey data?
Throughout the review?
Following separate reviews?
At the question point?
At the synthesis point?
At the evidence to recommendations stage?
Or a combination?
Narrative synthesis or summary?
Quantitising approach, eg, frequency analysis?
Qualitising approach, eg, thematic synthesis?
Tabulation?
Logic model?
Conceptual model/framework?
Graphical approach?
Petticrew et al 17 define the different aspects of complexity and examples of complexity-related questions that can potentially be explored in guidelines and systematic reviews taking a complexity perspective. Relevant aspects of complexity outlined by Petticrew et al 17 are summarised in table 2 below, together with the corresponding questions that could be addressed in a synthesis combining qualitative and quantitative evidence. Importantly, the aspects of complexity and their associated concepts of interest have however yet to be translated fully in primary health research or systematic reviews. There are few known examples where selected complexity concepts have been used to analyse or reanalyse a primary intervention study. Most notable is Chandler et al 26 who specifically set out to identify and translate a set of relevant complexity theory concepts for application in health systems research. Chandler then reanalysed a trial process evaluation using selected complexity theory concepts to better understand the complex causal pathway in the health system that explains some aspects of complexity in table 2 .
Rehfeuss et al 16 also recommends upfront consideration of the WHO-INTEGRATE evidence to decision criteria when planning a guideline and formulating questions. The criteria reflect WHO norms and values and take account of a complexity perspective. The framework can be used by guideline development groups as a menu to decide which criteria to prioritise, and which study types and synthesis methods can be used to collect evidence for each criterion. Many of the criteria and their related questions can be addressed using a synthesis of quantitative and qualitative evidence: the balance of benefits and harms, human rights and sociocultural acceptability, health equity, societal implications and feasibility (see table 3 ). Similar aspects in the DECIDE framework 15 could also be addressed using synthesis of qualitative and quantitative evidence.
Integrate evidence to decision framework criteria, example questions and types of studies to potentially address these questions (derived from Rehfeuss et al 16 )
Domains of the WHO-INTEGRATE EtD framework | Examples of potential research question(s) that a synthesis of qualitative and/or quantitative evidence could address | Types of studies that could contribute to a review of qualitative and quantitative evidence |
Balance of benefits and harms | To what extent do patients/beneficiaries different health outcomes? | Qualitative: studies of views and experiences Quantitative: Questionnaire surveys |
Human rights and sociocultural acceptability | Is the intervention to patients/beneficiaries as well as to those implementing it? To what extent do patients/beneficiaries different non-health outcomes? How does the intervention affect an individual’s, population group’s or organisation’s , that is, their ability to make a competent, informed and voluntary decision? | Qualitative: discourse analysis, qualitative studies (ideally longitudinal to examine changes over time) Quantitative: pro et contra analysis, discrete choice experiments, longitudinal quantitative studies (to examine changes over time), cross-sectional studies Mixed-method studies; case studies |
Health equity, equality and non-discrimination | How is the intervention for individuals, households or communities? How —in terms of physical as well as informational access—is the intervention across different population groups? | Qualitative: studies of views and experiences Quantitative: cross-sectional or longitudinal observational studies, discrete choice experiments, health expenditure studies; health system barrier studies, cross-sectional or longitudinal observational studies, discrete choice experiments, ethical analysis, GIS-based studies |
Societal implications | What is the of the intervention: are there features of the intervention that increase or reduce stigma and that lead to social consequences? Does the intervention enhance or limit social goals, such as education, social cohesion and the attainment of various human rights beyond health? Does it change social norms at individual or population level? What is the of the intervention? Does it contribute to or limit the achievement of goals to protect the environment and efforts to mitigate or adapt to climate change? | Qualitative: studies of views and experiences Quantitative: RCTs, quasi-experimental studies, comparative observational studies, longitudinal implementation studies, case studies, power analyses, environmental impact assessments, modelling studies |
Feasibility and health system considerations | Are there any that impact on implementation of the intervention? How might , such as past decisions and strategic considerations, positively or negatively impact the implementation of the intervention? How does the intervention ? Is it likely to fit well or not, is it likely to impact on it in positive or negative ways? How does the intervention interact with the need for and usage of the existing , at national and subnational levels? How does the intervention interact with the need for and usage of the as well as other relevant infrastructure, at national and subnational levels? | Non-research: policy and regulatory frameworks Qualitative: studies of views and experiences Mixed-method: health systems research, situation analysis, case studies Quantitative: cross-sectional studies |
GIS, Geographical Information System; RCT, randomised controlled trial.
Questions can serve as an ‘anchor’ by articulating the specific aspects of complexity to be explored (eg, Is successful implementation of the intervention context dependent?). 27 Anchor questions such as “How does intervention x impact on socioeconomic inequalities in health behaviour/outcome x” are the kind of health system question that requires a synthesis of both quantitative and qualitative evidence and hence a mixed-method synthesis. Quantitative evidence can quantify the difference in effect, but does not answer the question of how . The ‘how’ question can be partly answered with quantitative and qualitative evidence. For example, quantitative evidence may reveal where socioeconomic status and inequality emerges in the health system (an emergent property) by exploring questions such as “ Does patterning emerge during uptake because fewer people from certain groups come into contact with an intervention in the first place? ” or “ are people from certain backgrounds more likely to drop out, or to maintain effects beyond an intervention differently? ” Qualitative evidence may help understand the reasons behind all of these mechanisms. Alternatively, questions can act as ‘compasses’ where a question sets out a starting point from which to explore further and to potentially ask further questions or develop propositions or hypotheses to explore through a complexity perspective (eg, What factors enhance or hinder implementation?). 27 Other papers in this series provide further guidance on developing questions for qualitative evidence syntheses and guidance on question formulation. 14 28
For anchor and compass questions, additional application of a theory (eg, complexity theory) can help focus evidence synthesis and presentation to explore and explain complexity issues. 17 21 Development of a review specific logic model(s) can help to further refine an initial understanding of any complexity-related issues of interest associated with a specific intervention, and if appropriate the health system or section of the health system within which to contextualise the review question and analyse data. 17 23–25 Specific tools are available to help clarify context and complex interventions. 17 18
If a complexity perspective, and certain criteria within evidence to decision frameworks, is deemed relevant and desirable by guideline developers, it is only possible to pursue a complexity perspective if the evidence is available. Careful scoping using knowledge maps or scoping reviews will help inform development of questions that are answerable with available evidence. 20 If evidence of effect is not available, then a different approach to develop questions leading to a more general narrative understanding of what happened when complex interventions were implemented in a health system will be required (such as in case study 3—risk communication guideline). This should not mean that the original questions developed for which no evidence was found when scoping the literature were not important. An important function of creating a knowledge map is also to identify gaps to inform a future research agenda.
Table 2 and online supplementary files 1–3 outline examples of questions in the three case studies, which were all ‘COMPASS’ questions for the qualitative evidence syntheses.
The shift towards integration of qualitative and quantitative evidence in primary research has, in recent years, begun to be mirrored within research synthesis. 29–31 The natural extension to undertaking quantitative or qualitative reviews has been the development of methods for integrating qualitative and quantitative evidence within reviews, and within the guideline process using evidence to decision-frameworks. Advocating the integration of quantitative and qualitative evidence assumes a complementarity between research methodologies, and a need for both types of evidence to inform policy and practice. Below, we briefly outline the current designs for integrating qualitative and quantitative evidence within a mixed-method review or synthesis.
One of the early approaches to integrating qualitative and quantitative evidence detailed by Sandelowski et al 32 advocated three basic review designs: segregated, integrated and contingent designs, which have been further developed by Heyvaert et al 33 ( box 3 ).
Segregated design.
Conventional separate distinction between quantitative and qualitative approaches based on the assumption they are different entities and should be treated separately; can be distinguished from each other; their findings warrant separate analyses and syntheses. Ultimately, the separate synthesis results can themselves be synthesised.
The methodological differences between qualitative and quantitative studies are minimised as both are viewed as producing findings that can be readily synthesised into one another because they address the same research purposed and questions. Transformation involves either turning qualitative data into quantitative (quantitising) or quantitative findings are turned into qualitative (qualitising) to facilitate their integration.
Takes a cyclical approach to synthesis, with the findings from one synthesis informing the focus of the next synthesis, until all the research objectives have been addressed. Studies are not necessarily grouped and categorised as qualitative or quantitative.
A recent review of more than 400 systematic reviews 34 combining quantitative and qualitative evidence identified two main synthesis designs—convergent and sequential. In a convergent design, qualitative and quantitative evidence is collated and analysed in a parallel or complementary manner, whereas in a sequential synthesis, the collation and analysis of quantitative and qualitative evidence takes place in a sequence with one synthesis informing the other ( box 4 ). 6 These designs can be seen to build on the work of Sandelowski et al , 32 35 particularly in relation to the transformation of data from qualitative to quantitative (and vice versa) and the sequential synthesis design, with a cyclical approach to reviewing that evokes Sandelowski’s contingent design.
Convergent synthesis design.
Qualitative and quantitative research is collected and analysed at the same time in a parallel or complementary manner. Integration can occur at three points:
a. Data-based convergent synthesis design
All included studies are analysed using the same methods and results presented together. As only one synthesis method is used, data transformation occurs (qualitised or quantised). Usually addressed one review question.
b. Results-based convergent synthesis design
Qualitative and quantitative data are analysed and presented separately but integrated using a further synthesis method; eg, narratively, tables, matrices or reanalysing evidence. The results of both syntheses are combined in a third synthesis. Usually addresses an overall review question with subquestions.
c. Parallel-results convergent synthesis design
Qualitative and quantitative data are analysed and presented separately with integration occurring in the interpretation of results in the discussion section. Usually addresses two or more complimentary review questions.
A two-phase approach, data collection and analysis of one type of evidence (eg, qualitative), occurs after and is informed by the collection and analysis of the other type (eg, quantitative). Usually addresses an overall question with subquestions with both syntheses complementing each other.
The three case studies ( table 1 , online supplementary files 1–3 ) illustrate the diverse combination of review designs and synthesis methods that were considered the most appropriate for specific guidelines.
In this section, we draw on examples where specific review designs and methods have been or can be used to explore selected aspects of complexity in guidelines or systematic reviews. We also identify other review methods that could potentially be used to explore aspects of complexity. Of particular note, we could not find any specific examples of systematic methods to synthesise highly diverse research designs as advocated by Petticrew et al 17 and summarised in tables 2 and 3 . For example, we could not find examples of methods to synthesise qualitative studies, case studies, quantitative longitudinal data, possibly historical data, effectiveness studies providing evidence of differential effects across different contexts, and system modelling studies (eg, agent-based modelling) to explore system adaptivity.
There are different ways that quantitative and qualitative evidence can be integrated into a review and then into a guideline development process. In practice, some methods enable integration of different types of evidence in a single synthesis, while in other methods, the single systematic review may include a series of stand-alone reviews or syntheses that are then combined in a cross-study synthesis. Table 1 provides an overview of the characteristics of different review designs and methods and guidance on their applicability for a guideline process. Designs and methods that have already been used in WHO guideline development are described in part A of the table. Part B outlines a design and method that can be used in a guideline process, and part C covers those that have the potential to integrate quantitative, qualitative and mixed-method evidence in a single review design (such as meta-narrative reviews and Bayesian syntheses), but their application in a guideline context has yet to be demonstrated.
Depending on the review design (see boxes 3 and 4 ), integration can potentially take place at a review team and design level, and more commonly at several key points of the review or guideline process. The following sections outline potential points of integration and associated practical considerations when integrating quantitative and qualitative evidence in guideline development.
In a guideline process, it is common for syntheses of quantitative and qualitative evidence to be done separately by different teams and then to integrate the evidence. A practical consideration relates to the organisation, composition and expertise of the review teams and ways of working. If the quantitative and qualitative reviews are being conducted separately and then brought together by the same team members, who are equally comfortable operating within both paradigms, then a consistent approach across both paradigms becomes possible. If, however, a team is being split between the quantitative and qualitative reviews, then the strengths of specialisation can be harnessed, for example, in quality assessment or synthesis. Optimally, at least one, if not more, of the team members should be involved in both quantitative and qualitative reviews to offer the possibility of making connexions throughout the review and not simply at re-agreed junctures. This mirrors O’Cathain’s conclusion that mixed-methods primary research tends to work only when there is a principal investigator who values and is able to oversee integration. 9 10 While the above decisions have been articulated in the context of two types of evidence, variously quantitative and qualitative, they equally apply when considering how to handle studies reporting a mixed-method study design, where data are usually disaggregated into quantitative and qualitative for the purposes of synthesis (see case study 3—risk communication in humanitarian disasters).
Clearly specified key question(s), derived from a scoping or consultation exercise, will make it clear if quantitative and qualitative evidence is required in a guideline development process and which aspects will be addressed by which types of evidence. For the remaining stages of the process, as documented below, a review team faces challenges as to whether to handle each type of evidence separately, regardless of whether sequentially or in parallel, with a view to joining the two products on completion or to attempt integration throughout the review process. In each case, the underlying choice is of efficiencies and potential comparability vs sensitivity to the underlying paradigm.
Once key questions are clearly defined, the guideline development group typically needs to consider whether to conduct a single sensitive search to address all potential subtopics (lumping) or whether to conduct specific searches for each subtopic (splitting). 36 A related consideration is whether to search separately for qualitative, quantitative and mixed-method evidence ‘streams’ or whether to conduct a single search and then identify specific study types at the subsequent sifting stage. These two considerations often mean a trade-off between a single search process involving very large numbers of records or a more protracted search process retrieving smaller numbers of records. Both approaches have advantages and choice may depend on the respective availability of resources for searching and sifting.
Closely related to decisions around searching are considerations relating to screening and selecting studies for inclusion in a systematic review. An important consideration here is whether the review team will screen records for all review types, regardless of their subsequent involvement (‘altruistic sifting’), or specialise in screening for the study type with which they are most familiar. The risk of missing relevant reports might be minimised by whole team screening for empirical reports in the first instance and then coding them for a specific quantitative, qualitative or mixed-methods report at a subsequent stage.
Within a guideline process, review teams may be more limited in their choice of instruments to assess methodological limitations of primary studies as there are mandatory requirements to use the Cochrane risk of bias tool 37 to feed into Grading of Recommendations Assessment, Development and Evaluation (GRADE) 38 or to select from a small pool of qualitative appraisal instruments in order to apply GRADE; Confidence in the Evidence from Reviews of Qualitative Research (GRADE-CERQual) 39 to assess the overall certainty or confidence in findings. The Cochrane Qualitative and Implementation Methods Group has recently issued guidance on the selection of appraisal instruments and core assessment criteria. 40 The Mixed-Methods Appraisal Tool, which is currently undergoing further development, offers a single quality assessment instrument for quantitative, qualitative and mixed-methods studies. 41 Other options include using corresponding instruments from within the same ‘stable’, for example, using different Critical Appraisal Skills Programme instruments. 42 While using instruments developed by the same team or organisation may achieve a degree of epistemological consonance, benefits may come more from consistency of approach and reporting rather than from a shared view of quality. Alternatively, a more paradigm-sensitive approach would involve selecting the best instrument for each respective review while deferring challenges from later heterogeneity of reporting.
The way in which data and evidence are extracted from primary research studies for review will be influenced by the type of integrated synthesis being undertaken and the review purpose. Initially, decisions need to be made regarding the nature and type of data and evidence that are to be extracted from the included studies. Method-specific reporting guidelines 43 44 provide a good template as to what quantitative and qualitative data it is potentially possible to extract from different types of method-specific study reports, although in practice reporting quality varies. Online supplementary file 5 provides a hypothetical example of the different types of studies from which quantitative and qualitative evidence could potentially be extracted for synthesis.
The decisions around what data or evidence to extract will be guided by how ‘integrated’ the mixed-method review will be. For those reviews where the quantitative and qualitative findings of studies are synthesised separately and integrated at the point of findings (eg, segregated or contingent approaches or sequential synthesis design), separate data extraction approaches will likely be used.
Where integration occurs during the process of the review (eg, integrated approach or convergent synthesis design), an integrated approach to data extraction may be considered, depending on the purpose of the review. This may involve the use of a data extraction framework, the choice of which needs to be congruent with the approach to synthesis chosen for the review. 40 45 The integrative or theoretical framework may be decided on a priori if a pre-developed theoretical or conceptual framework is available in the literature. 27 The development of a framework may alternatively arise from the reading of the included studies, in relation to the purpose of the review, early in the process. The Cochrane Qualitative and Implementation Methods Group provide further guidance on extraction of qualitative data, including use of software. 40
Relatively few synthesis methods start off being integrated from the beginning, and these methods have generally been subject to less testing and evaluation particularly in a guideline context (see table 1 ). A review design that started off being integrated from the beginning may be suitable for some guideline contexts (such as in case study 3—risk communication in humanitarian disasters—where there was little evidence of effect), but in general if there are sufficient trials then a separate systematic review and meta-analysis will be required for a guideline. Other papers in this series offer guidance on methods for synthesising quantitative 46 and qualitative evidence 14 in reviews that take a complexity perspective. Further guidance on integrating quantitative and qualitative evidence in a systematic review is provided by the Cochrane Qualitative and Implementation Methods Group. 19 27 29 40 47
It is highly likely (unless there are well-designed process evaluations) that the primary studies may not themselves seek to address the complexity-related questions required for a guideline process. In which case, review authors will need to configure the available evidence and transform the evidence through the synthesis process to produce explanations, propositions and hypotheses (ie, findings) that were not obvious at primary study level. It is important that guideline commissioners, developers and review authors are aware that specific methods are intended to produce a type of finding with a specific purpose (such as developing new theory in the case of meta-ethnography). 48 Case study 1 (antenatal care guideline) provides an example of how a meta-ethnography was used to develop a new theory as an end product, 48 49 as well as framework synthesis which produced descriptive and explanatory findings that were more easily incorporated into the guideline process. 27 The definitions ( box 5 ) may be helpful when defining the different types of findings.
Descriptive findings —qualitative evidence-driven translated descriptive themes that do not move beyond the primary studies.
Explanatory findings —may either be at a descriptive or theoretical level. At the descriptive level, qualitative evidence is used to explain phenomena observed in quantitative results, such as why implementation failed in specific circumstances. At the theoretical level, the transformed and interpreted findings that go beyond the primary studies can be used to explain the descriptive findings. The latter description is generally the accepted definition in the wider qualitative community.
Hypothetical or theoretical finding —qualitative evidence-driven transformed themes (or lines of argument) that go beyond the primary studies. Although similar, Thomas and Harden 56 make a distinction in the purposes between two types of theoretical findings: analytical themes and the product of meta-ethnographies, third-order interpretations. 48
Analytical themes are a product of interrogating descriptive themes by placing the synthesis within an external theoretical framework (such as the review question and subquestions) and are considered more appropriate when a specific review question is being addressed (eg, in a guideline or to inform policy). 56
Third-order interpretations come from translating studies into one another while preserving the original context and are more appropriate when a body of literature is being explored in and of itself with broader or emergent review questions. 48
A critical element of guideline development is the formulation of recommendations by the Guideline Development Group, and EtD frameworks help to facilitate this process. 16 The EtD framework can also be used as a mechanism to integrate and display quantitative and qualitative evidence and findings mapped against the EtD framework domains with hyperlinks to more detailed evidence summaries from contributing reviews (see table 1 ). It is commonly the EtD framework that enables the findings of the separate quantitative and qualitative reviews to be brought together in a guideline process. Specific challenges when populating the DECIDE evidence to decision framework 15 were noted in case study 3 (risk communication in humanitarian disasters) as there was an absence of intervention effect data and the interventions to communicate public health risks were context specific and varied. These problems would not, however, have been addressed by substitution of the DECIDE framework with the new INTEGRATE 16 evidence to decision framework. A d ifferent type of EtD framework needs to be developed for reviews that do not include sufficient evidence of intervention effect.
Mixed-method review and synthesis methods are generally the least developed of all systematic review methods. It is acknowledged that methods for combining quantitative and qualitative evidence are generally poorly articulated. 29 50 There are however some fairly well-established methods for using qualitative evidence to explore aspects of complexity (such as contextual, implementation and outcome complexity), which can be combined with evidence of effect (see sections A and B of table 1 ). 14 There are good examples of systematic reviews that use these methods to combine quantitative and qualitative evidence, and examples of guideline recommendations that were informed by evidence from both quantitative and qualitative reviews (eg, case studies 1–3). With the exception of case study 3 (risk communication), the quantitative and qualitative reviews for these specific guidelines have been conducted separately, and the findings subsequently brought together in an EtD framework to inform recommendations.
Other mixed-method review designs have potential to contribute to understanding of complex interventions and to explore aspects of wider health systems complexity but have not been sufficiently developed and tested for this specific purpose, or used in a guideline process (section C of table 1 ). Some methods such as meta-narrative reviews also explore different questions to those usually asked in a guideline process. Methods for processing (eg, quality appraisal) and synthesising the highly diverse evidence suggested in tables 2 and 3 that are required to explore specific aspects of health systems complexity (such as system adaptivity) and to populate some sections of the INTEGRATE EtD framework remain underdeveloped or in need of development.
In addition to the required methodological development mentioned above, there is no GRADE approach 38 for assessing confidence in findings developed from combined quantitative and qualitative evidence. Another paper in this series outlines how to deal with complexity and grading different types of quantitative evidence, 51 and the GRADE CERQual approach for qualitative findings is described elsewhere, 39 but both these approaches are applied to method-specific and not mixed-method findings. An unofficial adaptation of GRADE was used in the risk communication guideline that reported mixed-method findings. Nor is there a reporting guideline for mixed-method reviews, 47 and for now reports will need to conform to the relevant reporting requirements of the respective method-specific guideline. There is a need to further adapt and test DECIDE, 15 WHO-INTEGRATE 16 and other types of evidence to decision frameworks to accommodate evidence from mixed-method syntheses which do not set out to determine the statistical effects of interventions and in circumstances where there are no trials.
When conducting quantitative and qualitative reviews that will subsequently be combined, there are specific considerations for managing and integrating the different types of evidence throughout the review process. We have summarised different options for combining qualitative and quantitative evidence in mixed-method syntheses that guideline developers and systematic reviewers can choose from, as well as outlining the opportunities to integrate evidence at different stages of the review and guideline development process.
Review commissioners, authors and guideline developers generally have less experience of combining qualitative and evidence in mixed-methods reviews. In particular, there is a relatively small group of reviewers who are skilled at undertaking fully integrated mixed-method reviews. Commissioning additional qualitative and mixed-method reviews creates an additional cost. Large complex mixed-method reviews generally take more time to complete. Careful consideration needs to be given as to which guidelines would benefit most from additional qualitative and mixed-method syntheses. More training is required to develop capacity and there is a need to develop processes for preparing the guideline panel to consider and use mixed-method evidence in their decision-making.
This paper has presented how qualitative and quantitative evidence, combined in mixed-method reviews, can help understand aspects of complex interventions and the systems within which they are implemented. There are further opportunities to use these methods, and to further develop the methods, to look more widely at additional aspects of complexity. There is a range of review designs and synthesis methods to choose from depending on the question being asked or the questions that may emerge during the conduct of the synthesis. Additional methods need to be developed (or existing methods further adapted) in order to synthesise the full range of diverse evidence that is desirable to explore the complexity-related questions when complex interventions are implemented into health systems. We encourage review commissioners and authors, and guideline developers to consider using mixed-methods reviews and synthesis in guidelines and to report on their usefulness in the guideline development process.
Handling editor: Soumyadeep Bhaumik
Contributors: JN, AB, GM, KF, ÖT and ES drafted the manuscript. All authors contributed to paper development and writing and agreed the final manuscript. Anayda Portela and Susan Norris from WHO managed the series. Helen Smith was series Editor. We thank all those who provided feedback on various iterations.
Funding: Funding provided by the World Health Organization Department of Maternal, Newborn, Child and Adolescent Health through grants received from the United States Agency for International Development and the Norwegian Agency for Development Cooperation.
Disclaimer: ÖT is a staff member of WHO. The author alone is responsible for the views expressed in this publication and they do not necessarily represent the decisions or policies of WHO.
Competing interests: No financial interests declared. JN, AB and ÖT have an intellectual interest in GRADE CERQual; and JN has an intellectual interest in the iCAT_SR tool.
Patient consent: Not required.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: No additional data are available.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
11k Accesses
16 Citations
3 Altmetric
Explore all metrics
In educational studies, the paradigm war over quantitative and qualitative research approaches has raged for more than half a century. The focus in the late twentieth century was on the distinction between the two approaches, and the motivation was to retain one of the approaches’ supremacy. Since the early twenty-first century, there has been a growing interest in situating in the middle position and combining both approaches into a single study or a series of studies. Despite these signs of progress, when it comes to using the appropriate research approach at the right time, beginner educational researchers remain perplexed. This paper, therefore, provides useful guidelines that facilitate the choice of quantitative, qualitative, or mixed research approaches in educational inquiry. To achieve this objective, this article comprises three distinct and underlying areas of interest, which have been structured into three sections. The first section highlights the distinctions between quantitative and qualitative research approaches. The second section discusses the paradigm views that underpin the choice of a particular research approach. Finally, an effort has been made to determine the appropriate time to opt for any of the research approaches that facilitate successful educational investigations. Since truth and the means used to discover it are both dynamic, it is also essential to foresight innovative approaches to research with distinguishing features of applications to educational research.
This is a preview of subscription content, log in via an institution to check access.
Subscribe and save.
Price includes VAT (Russian Federation)
Instant access to the full article PDF.
Rent this article via DeepDyve
Institutional subscriptions
Explore related subjects.
Åkerblad, L., Seppänen-Järvelä, R., & Haapakoski, K. (2021). Integrative strategies in mixed methods research. Journal of Mixed Methods Research, 15 (2), 152–170. https://doi.org/10.1177/1558689820957125
Article Google Scholar
Allwood, C. M. (2012). The distinction between qualitative and quantitative research methods is problematic. Quality and Quantity, 46 (5), 1417–1429. https://doi.org/10.1007/s11135-011-9455-8
Amaratugna, D., Baldry, D., Sarshar, M., & Newton, R. (2002). Quantitative and qualitative research in the built environment: Application of “mixed” research approach. Work Study, 51 (1), 17–31. https://doi.org/10.1108/00438020210415488
Antwi, S. K., & Hamza, K. (2015). Quantitative and qualitative research paradigms in business research: A philosophical reflections. European Journal of Business and Management, 7 (3), 217–225.
Google Scholar
Bailey, L. F. (2014). The origin and success of qualitative research. International Journal of Market Research, 56 (2), 167–184. https://doi.org/10.2501/IJMR-2014-013
Belk, R. W. (2013). Qualitative versus quantitative research in marketing. Revista de Negócios, 18 (1), 5–9. https://doi.org/10.7867/1980-4431.2013v18n1p5-9
Brinkmann, S., Jacobsen, M. H., Kristiansen, S., Brinkmann, S., Jacobsen, M. H., & Kristiansen, S. (2014). Historical overview of qualitative research in the social sciences. In The Oxford handbook of qualitative research (pp. 16–42). https://doi.org/10.1093/oxfordhb/9780199811755.013.017
Burke Johnson, R., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33 (7), 14–26.
Choy, L. T. (2014). The strengths and weaknesses of research methodology: Comparison and complimentary between qualitative and quantitative approaches. IOSR Journal of Humanities and Social Science, 19 (4), 99–104.
Cohen, L., Manion, L., & Morrison, K. (2018). Research methods in education (8th ed.). Routledge.
Creswell, J. W. (2009). Research design: Qualitative, quantitative and mixed methods approaches (3rd ed.). Sage Publications, Inc.
Creswell, J. W. (2013). Steps in conducting a scholarly mixed methods study . http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1047&context=dberspeakers
Curry, L. A., Nembhard, I. M., & Bradley, E. H. (2009). Qualitative and mixed methods provide unique contributions to outcomes research. Circulation, 119 , 1442–1452. https://doi.org/10.1161/CIRCULATIONAHA.107.742775
Daniel, E. (2016). The usefulness of qualitative and quantitative approaches and methods in researching problem-solving ability in science education curriculum. Journal of Education and Practice, 7 (15), 91–100.
Dawadi, S. (2017). Are quantitative and qualitative approaches to educational research compatible? The Warwick ELT , 3 (6). https://thewarwickeltezine.wordpress.com/2017/05/31/291/
Ejnavarzala, H. (2019). Epistemology–ontology relations in social research: A review. Sociological Bulletin, 68 (1), 94–104. https://doi.org/10.1177/0038022918819369
Fagan, M. B. (2010). Social construction revisited: Epistemology and scientific practice. Philosophy of Science, 77 (1), 92–116.
Farrell, E. (2020). Researching lived experience in education: Misunderstood or missed opportunity? International Journal of Qualitative Methods, 19 , 1–8. https://doi.org/10.1177/1609406920942066
Fassinger, R., & Morrow, S. L. (2013). Toward best practices in quantitative, qualitative, and mixed-method research: A social justice perspective. Journal of Social Action in Counseling Psychology, 5 (2), 69–83.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11 (3), 255–274. https://doi.org/10.3102/01623737011003255
Gunasekare, U. (2015). Mixed research method as the third research paradigm: A literature review. International Journal of Science and Research, 4 (8), 363–367.
Hodkinson, P. (2004). Research as a form of work: Expertise, community and methodological objectivity. British Educational Research Journal, 30 (1), 9–26. https://doi.org/10.1080/01411920310001629947
Howitt, D., & Cramer, D. (2011). Introduction to research methods in psychology (3rd ed.). Pearson Education Limited.
Johnson, R. B., & Christensen, L. (2014). Educational research: Quantitative, qualitative and mixed approaches (5th ed.). Sage Publications, Inc.
Johnson, R. B., & Christensen, L. (2020). Educational research: Quantitative, qualitative, and mixed approaches (7th ed.). Sage Publications, Inc.
Khaldi, K. (2017). Quantitative, qualitative or mixed research: Which research paradigm to use? Journal of Educational and Social Research, 7 (2), 15–24. https://doi.org/10.5901/jesr.2017.v7n2p15
Krivokapic-skoko, B., & O’neill, G. (2011). Beyond the qualitative-quantitative distinction: Some innovative methods for business and management research. International Journal of Multiple Research Approaches, 5 (5), 290–300. https://doi.org/10.5172/mra.2011.5.3.290
Lee, A. S., & Baskerville, R. L. (2003). Generalizing generalizability in information systems research. Information Systems Research . https://doi.org/10.1287/isre.14.3.221.16560
Levers, M. J. D. (2013). Philosophical paradigms, grounded theory, and perspectives on emergence. SAGE Open, 3 (4), 1–6. https://doi.org/10.1177/2158244013517243
Lobe, B., Morgan, D., & Hoffman, K. A. (2020). Qualitative data collection in an era of social distancing. International Journal of Qualitative Methods, 19 , 1–8. https://doi.org/10.1177/1609406920937875
Mack, L. (2010). The philosophical underpinnings of educational research. Polyglossia, 19 , 5–11.
Madill, A., & Gough, B. (2008). Qualitative research and its place in psychological science. Psychological Methods, 13 (3), 254–271. https://doi.org/10.1037/a0013220
Marshall, M. N. (1996). Sampling for qualitative research. Family Practice, 13 (6), 522–525.
Maxwell, J. A., & Reybold, L. E. (2015). Qualitative research. In International encyclopedia of the social & behavioral sciences (2nd ed., Vol. 19, pp. 685–689). Elsevier. https://doi.org/10.1016/B978-0-08-097086-8.10558-6
Mertens, D. M. (2012). What comes first? The paradigm or the approach? Journal of Mixed Methods Research, 6 (4), 255–257. https://doi.org/10.1177/1558689812461574
Meyer, D. K., & Schutz, P. A. (2020). Why talk about qualitative and mixed methods in educational psychology? Introduction to special issue. Educational Psychologist, 55 (4), 193–196. https://doi.org/10.1080/00461520.2020.1796671
Morgan, D. L. (2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1 (1), 48–76. https://doi.org/10.1177/2345678906292462
Morgan, D. L., & Nica, A. (2020). Iterative thematic inquiry: A new method for analyzing qualitative data. International Journal of Qualitative Methods, 19 , 1–11. https://doi.org/10.1177/1609406920955118
Östlund, U., Kidd, L., Wengström, Y., & Rowa-Dewar, N. (2011). Combining qualitative and quantitative research within mixed method research designs: A methodological review. International Journal of Nursing Studies, 48 (3), 369–383. https://doi.org/10.1016/j.ijnurstu.2010.10.005
Poli, R. (2018). A note on the classification of future-related methods. European Journal of Futures Research, 6 (1), 1–7. https://doi.org/10.1186/s40309-018-0145-9
Rahman, M. S. (2016). The advantages and disadvantages of using qualitative and quantitative approaches and methods in language “Testing and Assessment” research: A literature review. Journal of Education and Learning, 6 (1), 102. https://doi.org/10.5539/jel.v6n1p102
Sale, J. E. M., Lohfeld, L. H., & Brazil, K. (2002). Revisiting the quantitative-qualitative debate: Implications for mixed methods research. Quality & Quantity, 36 , 43–53. https://doi.org/10.1023/A:1014301607592
Salvador, J. T. (2016). Exploring quantitative and qualitative methodologies: A guide to novice nursing researchers. European Scientific Journal, 12 (18), 107–122. https://doi.org/10.19044/esj.2016.v12n18p107
Shannon-Baker, P. (2016). Making paradigms meaningful in mixed methods research. Journal of Mixed Methods Research, 10 (4), 319–334. https://doi.org/10.1177/1558689815575861
Symonds, J. E., & Gorard, S. (2010). Death of mixed methods? Or the rebirth of research as a craft. Evaluation & Research in Education, 23 (2), 121–136. https://doi.org/10.1080/09500790.2010.483514
Taylor, P., & Kanis, H. (2004). The quantitative-qualitative research dichotomy revisited. Theoretical Issues in Ergonomics Science, 5 (6), 507–516. https://doi.org/10.1080/1463922041233130303418
Techo, V. P. (2016). Research methods-quantitative, qualitative, and mixed methods . Horizons University. https://doi.org/10.13140/RG.2.1.1262.4886
Wohlfart, O. (2020). “Digging Deeper?”: Insights from a novice researcher. International Journal of Qualitative Methods, 19 , 1–5. https://doi.org/10.1177/1609406920963778
Yin, R. K. (2011). Qualitative research from start to finish . The Guilford Press.
Download references
No funding was received for this article.
Authors and affiliations.
Institute of Education and Behavioral Sciences, Ambo University, Ambo, Ethiopia
Feyisa Mulisa
You can also search for this author in PubMed Google Scholar
Correspondence to Feyisa Mulisa .
Conflict of interest.
Not applicable.
Publisher's note.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Reprints and permissions
Mulisa, F. When Does a Researcher Choose a Quantitative, Qualitative, or Mixed Research Approach?. Interchange 53 , 113–131 (2022). https://doi.org/10.1007/s10780-021-09447-z
Download citation
Received : 29 March 2021
Accepted : 18 November 2021
Published : 26 November 2021
Issue Date : March 2022
DOI : https://doi.org/10.1007/s10780-021-09447-z
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
Explore which research approach is best suited to the scientific method. Learn techniques to conduct precise experiments and analyze data.
The scientific method serves as the cornerstone of empirical inquiry, enabling researchers to systematically investigate the natural world and advance knowledge. In the context of scientific inquiry, choosing the most suitable research approach is vital for conducting rigorous and meaningful investigations.
This article aims to address the question of which research approach aligns best with the scientific method. It delves into the exploration of various types of research approaches, their respective advantages and disadvantages, and provides considerations to make an informed choice. By examining these factors, researchers can gain insights into selecting the most appropriate research approach that aligns with the principles and objectives of the scientific method.
The scientific method serves as the foundation for empirical inquiry, providing a systematic approach to investigate and understand the natural world. It is a structured process that scientists employ to acquire knowledge, test hypotheses, and draw reliable conclusions.
At its core, the scientific method involves a series of logical steps that guide researchers in their quest for understanding. It begins with observation, where scientists carefully observe and identify patterns or phenomena of interest. These observations lead to the formulation of research questions or hypotheses, which are educated guesses about the relationships or explanations underlying the observed phenomena.
To test these hypotheses, researchers design and conduct experiments or studies, collecting data through systematic observations or measurements. The data collected is then analyzed using statistical or other appropriate methods to draw meaningful insights and conclusions. This process of analysis involves the identification of patterns, trends, or correlations within the data.
One critical aspect of the scientific method is the emphasis on objectivity and replicability. Findings and conclusions derived from scientific investigations must be based on evidence and be reproducible by other researchers following the same methodology. This ensures that scientific knowledge is reliable, verifiable, and can withstand scrutiny and peer review.
The scientific method is iterative in nature, with new observations and findings often leading to the formulation of new hypotheses or the refinement of existing ones. This iterative process contributes to the advancement of scientific knowledge over time, as researchers build upon previous work and expand the understanding of the subject matter.
By adhering to the principles of the scientific method, researchers can navigate through the complexities of the natural world, generate new insights, and contribute to the collective knowledge in their respective fields. It is a systematic and rigorous approach that forms the backbone of scientific inquiry, enabling researchers to uncover truths, challenge assumptions, and make meaningful contributions to their disciplines.
By gaining a thorough understanding of the various research approaches and their unique characteristics, researchers are empowered to make informed decisions regarding the most suitable approach for their specific research questions and objectives.
It is crucial to align the chosen approach with the nature of the research topic, the availability of resources, and the necessity to capture both objective and subjective aspects of the phenomenon being investigated.
Furthermore, researchers should consider the adoption of a mixed research approach, which combines quantitative and qualitative methods, to attain a more comprehensive understanding of complex research questions.
Here are the common types of research approaches, namely quantitative research, qualitative research, and mixed research .
Quantitative research involves the systematic collection and analysis of numerical data to establish relationships, test hypotheses, and make generalizations about a larger population. This approach relies on structured data collection methods, such as surveys, experiments, or the analysis of existing datasets.
Quantitative research aims to quantify variables, measure outcomes objectively, and often employs statistical techniques to analyze and interpret the data. By employing large sample sizes and statistical analysis, quantitative research strives for objectivity and generalizability of findings.
Qualitative research focuses on understanding complex phenomena and exploring subjective experiences, meanings, and contexts. This approach emphasizes in-depth exploration and interpretation of non-numerical data, such as interviews, observations, or textual analysis.
Qualitative research aims to capture rich and detailed insights, uncover underlying motivations, and generate theories grounded in real-life experiences. Researchers often employ techniques like thematic analysis or grounded theory to analyze qualitative data. Through open-ended questions and a flexible research design, qualitative research allows for a deep understanding of the complexities and nuances of a research topic.
Mixed methods research integrating quantitative and qualitative approaches, recognizing the value of both types of data in providing a comprehensive understanding of a research topic. Researchers using this approach collect and analyze both numerical and non-numerical data, allowing for triangulation of findings and a more holistic perspective.
Mixed methods research enables researchers to explore complex research questions by combining the strengths of quantitative analysis (e.g., statistical significance) with the richness of qualitative insights (e.g., in-depth understanding of motivations). By utilizing multiple sources of data, researchers can gain a more comprehensive understanding and make informed interpretations.
This section provides a comprehensive exploration of the advantages and disadvantages associated with different research approaches, including quantitative research, qualitative research, and mixed research. By examining these approaches, researchers can gain a deeper understanding of their advantages and disadvantages, allowing them to make informed decisions when selecting the most appropriate approach for their specific research objectives.
Advantages:.
Choosing the most suitable research approach in accordance with the scientific method necessitates thoughtful deliberation of several factors. When making this decision, researchers must take into account a range of factors and address the question, “Which research approach is best suited to the scientific method?” When selecting a research approach. Here are some key considerations when selecting a research approach:
In summary, selecting the most suitable research approach that aligns with the scientific method requires careful consideration of various factors. Researchers must address the question, “Which research approach is best suited to the scientific method?” and give due consideration to factors such as research objectives, research questions, available resources, the nature of the phenomena being studied, ethical considerations, existing knowledge and theoretical frameworks, as well as the overall compatibility with the research design.
Quantitative research offers precise measurement, statistical analysis, and the ability to generalize findings, but may oversimplify complex phenomena and overlook subjective experiences. Qualitative research provides in-depth understanding, flexibility in data collection methods, and the exploration of subjective meanings, but findings may be subjective and lack generalizability. Mixed research integrates quantitative and qualitative approaches, allowing for a comprehensive understanding and triangulation of findings, but requires expertise and careful coordination.
Ultimately, researchers must carefully weigh the advantages and disadvantages of each approach to ensure the chosen approach aligns with the goals of their study. By adhering to the principles of the scientific method and selecting an appropriate research approach, researchers can conduct rigorous investigations, generate meaningful insights, and contribute to the advancement of knowledge in their respective fields.
Mind the Graph is a versatile platform that offers invaluable assistance to scientists in visualizing their research. Mind the Graph empowers researchers to effectively communicate complex scientific concepts and create visually appealing and accurate scientific figures for their research, enhancing the presentation and dissemination of scientific findings, allowing researchers to create perfect-matching figures that effectively convey their research insights to the scientific community and beyond.
Exclusive high quality content about effective visual communication in science.
Sign Up for Free
Try the best infographic maker and promote your research with scientifically-accurate beautiful figures
no credit card required
Jessica Abbadia is a lawyer that has been working in Digital Marketing since 2020, improving organic performance for apps and websites in various regions through ASO and SEO. Currently developing scientific and intellectual knowledge for the community's benefit. Jessica is an animal rights activist who enjoys reading and drinking strong coffee.
Training Maker
All Products
Qualaroo Insights
ProProfs.com
FREE. All Features. FOREVER!
Try our Forever FREE account with all premium features!
Market Research Specialist
Emma David, a seasoned market research professional, specializes in employee engagement, survey administration, and data management. Her expertise in leveraging data for informed decisions has positively impacted several brands, enhancing their market position.
Step into the fascinating world of quantitative research , where numbers reveal extraordinary insights!
By gathering and studying data in a systematic way, quantitative research empowers us to understand our ever-changing world better. It helps understand a problem or an already-formed hypothesis by generating numerical data. The results don’t end here, as you can process these numbers to get actionable insights that aid decision-making.
You can use quantitative research to quantify opinions, behaviors, attitudes, and other definitive variables related to the market, customers, competitors, etc. The research is conducted on a larger sample population to draw predictive, average, and pattern-based insights.
Here, we delve into the intricacies of this research methodology, exploring various quantitative methods, their advantages, and real-life examples that showcase their impact and relevance.
Ready to embark on a journey of discovery and knowledge? Let’s go!
Quantitative research is a method that uses numbers and statistics to test theories about customer attitudes and behaviors. It helps researchers gather and analyze data systematically to gain valuable insights and draw evidence-based conclusions about customer preferences and trends.
Researchers use online surveys, questionnaires , polls , and quizzes to question a large number of people to obtain measurable and bias-free data.
In technical terms, quantitative research is mainly concerned with discovering facts about social phenomena while assuming a fixed and measurable reality.
Offering numbers and stats-based insights, this research methodology is a crucial part of primary research and helps understand how well an organizational decision is going to work out.
Let’s consider an example.
Suppose your qualitative analysis shows that your customers are looking for social media-based customer support. In that case, quantitative analysis will help you see how many of your customers are looking for this support.
If 10% of your customers are looking for such a service, you might or might not consider offering this feature. But, if 40% of your regular customers are seeking support via social media, then it is something you just cannot overlook.
Quantitative research clarifies the fuzziness of research data from qualitative research analysis. With numerical insights, you can formulate a better and more profitable business decision.
Hence, quantitative research is more readily contestable, sharpens intelligent discussion, helps you see the rival hypotheses, and dynamically contributes to the research process.
Let us have a quick look at some of its characteristics.
The data collection methods in quantitative research are structured and contain items requiring measurable variables, such as age, number of family members, salary range, highest education, etc.
These structured data collection methods comprise polls, surveys, questionnaires, etc., and may have questions like the ones shown in the following image:
As you can see, all the variables are measurable. This ensures that the research is in-depth and provides less erroneous data for reliable, actionable insights.
No matter what data analysis methods for quantitative research are being used, the sample size is kept such that it represents the target market.
As the main aim of the research methodology is to get numerical insights, the sample size should be fairly large. Depending on the survey objective and scope, it might span hundreds of thousands of people.
To maintain the reliability of a quantitative research methodology, we assume that the population distribution curve is normal.
This type of population distribution curve is preferred over a non-normal distribution as the sample size is large, and the characteristics of the sample vary with its size.
This requires adhering to the random sampling principle to avoid the researcher’s bias in result interpretation. Any bias can ruin the fairness of the entire process and defeats the purpose of research.
Data analysis in quantitative research produces highly structured results and can form well-defined graphical representations. Some common examples include tables, figures, graphs, etc., that combine large blocks of data.
This way, you can discover hidden data trends, relationships, and differences among various measurable variables. This can help researchers understand the survey data and formulate actionable insights for decision-making.
Quantitative analysis of data can also be used for estimations and prediction outcomes. You can construct if-then scenarios and analyze the data for the identification of any upcoming trends or events.
However, this requires advanced analytics and involves complex mathematical computations. So, it is mostly done via quantitative research tools that come with advanced analytics capabilities.
Quantitative research is usually conducted using two methods. They are-
Primary quantitative research is the most popular way of conducting market research. The differentiating factor of this method is that the researcher relies on collecting data firsthand instead of relying on data collected from previous research.
There are multiple types of primary quantitative research. They can be distinguished based on three distinctive aspects, which are:
Surveys are the easiest, most common, and one of the most sought-after quantitative research techniques. The main aim of a survey is to widely gather and describe the characteristics of a target population or customers. Surveys are the foremost quantitative method preferred by both small and large organizations.
They help them understand their customers, products, and other brand offerings in a proper manner.
Surveys can be conducted using various methods, such as online polls, web-based surveys, paper questionnaires, phone calls, or face-to-face interviews. Survey research allows organizations to understand customer opinions, preferences, and behavior, making it crucial for market research and decision-making.
You can watch this quick video to learn more about creating surveys.
Watch: How to Create a Survey Using ProProfs Survey Maker
Surveys can be distributed via various channels. Some of the most popular ones are listed below:
Correlational research aims to establish relationships between two or more variables.
Researchers use statistical analysis to identify patterns and trends in the data, but it does not determine causality between the variables. This method helps understand how changes in one variable may impact another.
Examples of correlational research questions include studying the relationship between stress and depression, fame and money, or classroom activities and student performance.
Causal-comparative research, also known as quasi-experimental research, seeks to determine cause-and-effect relationships between variables.
Researchers analyze how an independent variable influences a dependent variable, but they do not manipulate the independent variable. Instead, they observe and compare different groups to draw conclusions.
Causal-comparative research is useful in situations where it’s not ethical or feasible to conduct true experiments.
Examples of questions for this type of research include analyzing the effect of training programs on employee performance, studying the influence of customer support on client retention, investigating the impact of supply chain efficiency on cost reduction, etc.
Experimental research is based on testing theories to validate or disprove them. Researchers conduct experiments and manipulate variables to observe their impact on the outcomes.
This type of research is prevalent in natural and social sciences, and it is a powerful method to establish cause-and-effect relationships. By randomly assigning participants to experimental and control groups, researchers can draw more confident conclusions.
Examples of experimental research include studying the effectiveness of a new drug, the impact of teaching methods on student performance, or the outcomes of a marketing campaign.
After defining research objectives, the next significant step in primary quantitative research is data collection. This involves using two main methods: sampling and conducting surveys or polls.
2.1Sampling methods:
In quantitative research, there are two primary sampling methods: Probability and Non-probability sampling.
2.2Probability Sampling
In probability sampling, researchers use the concept of probability to create samples from a population. This method ensures that every individual in the target audience has an equal chance of being selected for the sample.
There are four main types of probability sampling:
2.3Non-probability Sampling
Non-probability sampling is a method where the researcher’s knowledge and experience guide the selection of samples. This approach doesn’t give all members of the target population an equal chance of being included in the sample.
There are five non-probability sampling models:
To analyze the quantitative data accurately, you’ll need to use specific statistical methods such as:
Here are five commonly used secondary quantitative research methods:
The Internet has become a vast repository of data, making it easier for researchers to access a wealth of information. Online databases, websites, and research repositories provide valuable quantitative data for researchers to analyze and validate their primary research findings.
Government agencies and non-government organizations often conduct extensive research and publish reports. These reports cover a wide range of topics, providing researchers with reliable and comprehensive data for quantitative analysis.
While less commonly used in the digital age, public libraries still hold valuable research reports, historical data, and publications that can contribute to quantitative research.
Educational institutions frequently conduct research on various subjects. Their research reports and publications can serve as valuable sources of information for researchers, validating and supporting primary quantitative research outcomes.
Commercial sources such as local newspapers, journals, magazines, and media outlets often publish relevant data on economic trends, market research, and demographic analyses. Researchers can access this data to supplement their own findings and draw better conclusions.
Quantitative research data is often standardized and can be easily used to generalize findings for making crucial business decisions and uncover insights to supplement the qualitative research findings.
Here are some core benefits this research methodology offers.
Direct Result Comparison
As the studies can be replicated for different cultural settings and different times, even with different groups of participants, they tend to be extremely useful. Researchers can compare the results of different studies in a statistical manner and arrive at comprehensive conclusions for a broader understanding.
Replication
Researchers can repeat the study by using standardized data collection protocols over well-structured data sets. They can also apply tangible definitions of abstract concepts to arrive at different conclusions for similar research objectives with minor variations.
Large Samples
As the research data comes from large samples, the researchers can process and analyze the data via highly reliable and consistent analysis procedures. They can arrive at well-defined conclusions that can be used to make the primary research more thorough and reliable.
Hypothesis Testing
This research methodology follows standardized and established hypothesis testing procedures. So, you have to be careful while reporting and analyzing your research data , and the overall quality of results gets improved.
Below, we discuss two excellent examples of quantitative research methods that were used by highly distinguished business and consulting organizations. Both examples show how different types of analysis can be performed with qualitative approaches and how the analysis is done once the data is collected.
1. STEP Project Global Consortium / KPMG 2019 Global Family Business survey
This research utilized quantitative methods to identify ways that kept the family businesses sustainably profitable with time.
The study also identified the ways in which the family business behavior changed with demographic changes and had “why” and “how” questions. Their qualitative research methods allowed the KPMG team to dig deeper into the mindsets and perspectives of the business owners and uncover unexpected research avenues as well.
It was a joint effort in which STEP Project Global Consortium collected 26 cases, and KPMG collected 11 cases.
The research reached the stage of data analysis in 2020, and the analysis process spanned over 4 stages.
The results, which were also the reasons why family businesses tend to lose their strength with time, were found to be:
2. EY Seren Teams Research 2020
This is yet another commendable example of qualitative research where the EY Seren Team digs into the unexplored depths of human behavior and how it affected their brand or service expectations.
The research was done across 200+ sources and involved in-depth virtual interviews with people in their homes, exploring their current needs and wishes. It also involved diary studies across the entire UK customer base to analyze human behavior changes and patterns.
The study also included interviews with professionals and design leaders from a wide range of industries to explore how COVID-19 transformed their industries. Finally, quantitative surveys were conducted to gain insights into the EY community after every 15 days.
The insights and results were:
Here are some best practices to keep in mind while conducting quantitative research:
There can be many ways to collect data via quantitative research methods that are chosen as per the research objective and scope. These methods allow you to build your own observations regarding any hypotheses – unknown, entirely new, or unexplained.
You can hypothesize a proof and build a prediction of outcomes supporting the same. You can also create a detailed stepwise plan for data collection, analysis, and testing.
Below, we explore quantitative research methods and discuss some examples to enhance your understanding of them.
The surveys are meant to reach people en-masse, and that includes a wide demographic range with recipients from all walks of life. Asking simple questions will ensure that they grasp what’s being asked easily.
Choose an appropriate research design that aligns with your objectives, whether it’s experimental, quasi-experimental, or correlational. You also need to pay attention to the sample size and sampling technique such that it represents the target population accurately.
It’s crucial to select or develop measurement instruments such as questionnaires, scales, or tests that have been validated and are reliable. Before proceeding with the main study, pilot-test these instruments on a small sample to assess their effectiveness and make any necessary improvements.
Implement data collection protocols to minimize errors and bias during data gathering. Double-check data entries and cleaning procedures to eliminate any inconsistencies or missing values that may affect the accuracy of your results. For instance, you might regularly cross-verify data entries to identify and correct any discrepancies.
Select statistical methods that match the nature of your data and research questions. Whether it’s regression analysis, t-tests, ANOVA, or other techniques, using the right approach is important for drawing meaningful conclusions. Utilize software tools like SPSS or R for data analysis to ensure the accuracy and reproducibility of your findings.
Present your findings in a clear and unbiased manner. Avoid making unwarranted causal claims, especially in correlational studies. Instead, focus on describing the relationships and patterns observed in your data.
Prioritize ethical considerations throughout your research process. Obtain informed consent from participants, ensuring their voluntary participation and confidentiality of data. Comply with ethical guidelines and gain approval from a governing body if necessary.
While no single research methodology can produce 100% reliable results, you can always opt for a hybrid research method by opting for the methods that are most relevant to your objective.
This understanding comes gradually as you learn how to implement the correct combination of qualitative and quantitative research methods for your research projects. For the best results, we recommend investing in smart, efficient, and scalable research tools that come with delightful reporting and advanced analytics to make every research initiative a success.
These software tools, such as ProProfs Survey Maker, come with pre-built survey templates and question libraries and allow you to create a high-converting survey in just a few minutes.
So, choose the best research partner, create the right research plan, and gather insights that drive sustainable growth for your business.
About the author
Emma David is a seasoned market research professional with 8+ years of experience. Having kick-started her journey in research, she has developed rich expertise in employee engagement, survey creation and administration, and data management. Emma believes in the power of data to shape business performance positively. She continues to help brands and businesses make strategic decisions and improve their market standing through her understanding of research methodologies.
How to Ask Employee Survey Questions About Management the Right Way: A Guide
How to Use Open-Ended Questions: Tips, Advantages & Examples
Product Market Fit: An Ultimate Guide
Survey Question: 250+Examples, Types & Best Practices
How to Ask Sensitive Questions in Surveys
What is Diversity, Equity, and Inclusion (DEI)? 25+ Survey Questions to Ask
IMAGES
VIDEO
COMMENTS
INTRODUCTION. Scientific research is usually initiated by posing evidenced-based research questions which are then explicitly restated as hypotheses.1,2 The hypotheses provide directions to guide the study, solutions, explanations, and expected results.3,4 Both research questions and hypotheses are essentially formulated based on conventional theories and real-world processes, which allow the ...
Revised on June 22, 2023. Quantitative research is the process of collecting and analyzing numerical data. It can be used to find patterns and averages, make predictions, test causal relationships, and generalize results to wider populations. Quantitative research is the opposite of qualitative research, which involves collecting and analyzing ...
Social scientists are concerned with the study of people. Quantitative research is a way to learn about a particular group of people, known as a sample population. Using scientific inquiry, quantitative research relies on data that are observed or measured to examine questions about the sample population. Allen, M. (2017). The SAGE encyclopedia ...
Abstract. In an era of data-driven decision-making, a comprehensive understanding of quantitative research is indispensable. Current guides often provide fragmented insights, failing to offer a holistic view, while more comprehensive sources remain lengthy and less accessible, hindered by physical and proprietary barriers.
Quantitative research stands as a powerful research methodology dedicated to the systematic collection and analysis of measurable data. Learn more about quantitative research Examples, key advantages, methods and best practices. ... Quantitative research is well-suited for testing specific hypotheses or research questions, ... 7 Best Practices ...
Quantitative research is very well suited to establishing cause-and-effect relationships, to testing hypotheses and to determining the opinions, attitudes and practices of a large population, whereas qualitative research lends itself very well to developing hypotheses and theories and to describing processes such as decision making or ...
When collecting and analyzing data, quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings. Both are important for gaining different kinds of knowledge. Quantitative research. Quantitative research is expressed in numbers and graphs. It is used to test or confirm theories and assumptions.
Quantitative method is used to summarize, average, find patterns, make predictions, and test causal associations as well as generalizing results to wider populations. It allows us to quantify effect sizes, determine the strength of associations, rank priorities, and weigh the strength of evidence of effectiveness.
Mixed-methods research is a flexible approach, where the research design is determined by what we want to find out rather than by any predetermined epistemological position. In mixed-methods research, qualitative or quantitative components can predominate, or both can have equal status. 1.4. Units and variables.
1. Understanding the options. Before we jump into the question of how to choose a research methodology, it's useful to take a step back to understand the three overarching types of research - qualitative, quantitative and mixed methods -based research. Each of these options takes a different methodological approach.
In quantitative research, a variable is something (an intervention technique, a pharmaceutical, a temperature, etc.) that changes. There are two kinds of variables: independent variables and dependent variables.In the simplest terms, the independent variable is whatever the researchers are using to attempt to make a change in their dependent variable.
Quantitative research involves analyzing and gathering numerical data to uncover trends, calculate averages, evaluate relationships, and derive overarching insights. It's used in various fields, including the natural and social sciences. Quantitative data analysis employs statistical techniques for processing and interpreting numeric data.
Research in which collected data is converted into numbers or numerical data is quantitative research. It is widely used in surveys, demographic studies, census information, marketing, and other studies that use numerical data to analyze results. Primary quantitative research yields results that are objective, statistical, and unbiased.
Quantitative methods emphasize objective measurements and the statistical, mathematical, or numerical analysis of data collected through polls, questionnaires, and surveys, or by manipulating pre-existing statistical data using computational techniques.Quantitative research focuses on gathering numerical data and generalizing it across groups of people or to explain a particular phenomenon.
Choosing an optimal research methodology is crucial for the success of any research project. The methodology you select will determine the type of data you collect, how you collect it, and how you analyse it. Understanding the different types of research methods available along with their strengths and weaknesses, is thus imperative to make an ...
Quantitative research. In this type of research, the data collected is generally expressed in numbers and graphs to confirm theories and assumptions. The data collected are factual information on the topic. Under the quantitative research method, the factual information can be collected in many ways such as:
tion (Fraenkel et al., 2012). It is primarily a quantitative research technique in which the researcher administers some sort of survey or questionnaire to a sample—or, in some cases, an entire population—of individuals to describe their attitudes, opinions, behaviors, experiences, or other characteristics of th.
Quantitative methodology would best apply to this research problem. Use quantitative research methods such as A/B testing for validating or choosing a design based on user satisfaction scores, perceived usability measures, and/or task performance. The data received is statistically valid and can be generalized to the entire user population ...
Introduction. Recognition has grown that while quantitative methods remain vital, they are usually insufficient to address complex health systems related research questions. 1 Quantitative methods rely on an ability to anticipate what must be measured in advance. Introducing change into a complex health system gives rise to emergent reactions, which cannot be fully predicted in advance.
In educational studies, the paradigm war over quantitative and qualitative research approaches has raged for more than half a century. The focus in the late twentieth century was on the distinction between the two approaches, and the motivation was to retain one of the approaches' supremacy. Since the early twenty-first century, there has been a growing interest in situating in the middle ...
Quantitative research explains phenomena by collecting numerical unchanging d etailed data t hat. are analyzed using mathematically based methods, in particular statistics that pose questions of ...
Quantitative research involves the systematic collection and analysis of numerical data to establish relationships, test hypotheses, and make generalizations about a larger population. This approach relies on structured data collection methods, such as surveys, experiments, or the analysis of existing datasets.
Let us have a quick look at some of its characteristics. 1. Measurable Variables. The data collection methods in quantitative research are structured and contain items requiring measurable variables, such as age, number of family members, salary range, highest education, etc. These structured data collection methods comprise polls, surveys ...