(BJHP03).
That sample size principles, guidelines and tools have been developed to enable researchers to set, and justify the acceptability of, their sample size is an indication that the issue constitutes an important marker of the quality of qualitative research. ux2Mn[m}2M(q}[CFeY#5`sf\aI>aX}A6F56Tz|~.v)^:c35~nW|E$Kf4K]bmq42yG|a/sk Their analysis of interview data showed that code saturation (i.e. Given that older adults have been ignored in the research concerning suntanning, fair-skinned older adults are the most likely to experience skin cancer, and women privilege appearance over health when it comes to sunbathing practices, our study offers depth and richness of data in a demographic group much in need of research attention. Google Scholar.
Only occasionally was analytic or idiographic generalisation invoked to warrant the value of the studys findings [5, 17].
More recently, a quantitative tool was proposed [42] to support a priori sample size determination based on estimates of the prevalence of themes in the population. Three articles described a state of achieved saturation without using the term or specifying what sort of saturation they had achieved (i.e. Whilst the themes from the initial interviews seemed to be replicated in the later interviews, further interviews may have identified additional themes or provided more nuanced explanations.
In total, 214 articles 21 in the BMJ, 53 in the BJHP and 140 in the SHI were eligible for inclusion in the review.
studies often combined elements from different theoretical and epistemological traditions).
(SHI103). 0000011904 00000 n London: Sage; 1985. Dai NT, Free C, Gendron Y. Interview-based research in accounting 20002014: a review. Interviewer and note taker agreed that thematic saturation, the point at which no new concepts emerge from subsequent interviews (Patton, 2002), was achieved following completion of 20 interviews.
Implement Sci. All other authors have no competing interests to declare.
Psychol Health. 2077 0 obj <> endobj xref 2077 38 0000000016 00000 n
0000006193 00000 n (SHI115). Kerr C, Nixon A, Wild D. Assessing and demonstrating data saturation in qualitative inquiry supporting patient-reported outcomes research. Transforming qualitative information: thematic analysis and code development. London: Sage; 2003. p. 77108. (BJHP19).
Erickson F. Qualitative methods in research on teaching. (BJHP31). London: Sage; 2004. 1530 interviews) projects.
Book One article (BMJ18) cited a reference to support its position on saturation. We set a target of seven to 10 caregivers per site because of time constraints and the anticipated difficulty of accessing caregivers at some home based care services. 2017. https://doi.org/10.1007/s11135-017-0574-8. The funding body did not have any role in the study design, the collection, analysis and interpretation of the data, in the writing of the paper, and in the decision to submit the manuscript for publication.
Safman RM, Sobal J. Qualitative sample extensiveness in health education research. sample size) conducted and the provision of a justification (rpb=.054, p=.433).
Morse JM. The significance of saturation.
0000003258 00000 n Similarly, fewer than 10% of articles in organisation and workplace studies provided a sample size justification relating to existing recommendations by methodologists, prior relevant work, or saturation [33], whilst only 17% of focus groups studies in health-related journals provided an explanation of sample size (i.e.
Borsci S, Macredie RD, Martin JL, Young T. How many testers are needed to assure the usability of medical devices? A number of articles which characterised their sample size as small did so against an implicit or explicit quantitative framework of reference. Small scale IPA studies allow in-depth analysis which would not be possible with larger samples (Smith et al., 2009).
Following the logic of informational comprehensiveness Malterud et al.
(BJHP41).
Field Methods.
(BMJ13).
Ritchie J, Lewis J, Elam G. Designing and selecting samples.
In light of the growing methodological evidence in the area, transparent presentation of such evidence-based judgement is crucial and in time should surely obviate the seemingly routine practice of citing the small size of qualitative samples among the study limitations. The size of the sample was largely determined by the availability of respondents and resources to complete the study. 2015;26:175360.
This research, while limited in size, has sought to capture some of the complexity attached to mens attitudes and experiences concerning incomes and material circumstances. 0000006727 00000 n 2018;13(Suppl 1):14. Glenton C, Carlsen B, Lewin S, Munthe-Kaas H, Colvin CJ, Tunalp , et al. group interviews, structured interviews and e-mail interviews over a period of time were excluded), and the data had to be analysed qualitatively (i.e. [18] introduced the concept of information power as a pragmatic guiding principle, suggesting that the more information power the sample provides, the smaller the sample size needs to be, and vice versa. Other articles appeared to accept and acknowledge that their sample was flawed because of its small size (as well as other compositional deficits e.g.
Finally, three SHI articles explained their sample size with reference to practical aspects: time constraints and project manageability (SHI56), limited availability of respondents and project resources (SHI131), and time constraints (SHI113). 0000012498 00000 n qualities of the analysis, meeting sampling or research design requirements, richness and volume of the data obtained, nature of study, further sampling to check findings consistency).
(BMJ13).
(BMJ17). 0000050482 00000 n Leech NL. Saturation in qualitative research: exploring its conceptualization and operationalization.
To be eligible for inclusion in the review, the article had to report a cross-sectional study design.
Supporting data can be accessed in the original publications. Terms and Conditions,
Correspondence to 2003;15(4):386404.
We also extracted or coded quantitative data regarding the following aspects: Presence of sample size justification(s) (Yes/No), Presence of a particular sample size justification category (Yes/No), and, Number of sample size justifications provided.
(BJHP39).
the point at which no further dimensions, nuances, or insights of issues are identified) required 1624 interviews. Caine K. Local standards for sample size at CHI. Accessed 17 May 2018. Number of eligible articles published each year per journalFootnote.
Three articles (BMJ13; BJHP05; BJHP48) which all provided the justification of saturation, characterised their sample size as large and narrated this oversufficiency in positive terms as it allowed richer data and findings and enhanced the potential for generalisation. Konstantina Vasileiou. In other instances, articles expressed uncertainty about whether their results accounted for the full spectrum and variation of the phenomenon under investigation.
Theoretical sampling describes an iterative process of data collection, data analysis and theory development whereby data collection is governed by emerging theory rather than predefined characteristics of the population. 2017;27(4):591608. Expert Rev Pharmacoecon Outcomes Res.
2006;18(1):5982. 0000001056 00000 n 0000009139 00000 n We would like to thank Dr. Paula Smith and Katharine Lee for their comments on a previous draft of this paper as well as Natalie Ann Mitchell and Meron Teferra for assisting us with data extraction.
Two other articles (BJHP35; SHI32) linked the claimed sufficiency to the scope (i.e.
Employing the search function of each individual journal, we used the terms interview* AND qualitative and limited the results to articles published between 1January 2003 and 22 September 2017 (i.e.
We concur with existing views [11, 36] that the consideration of the characteristics of the study at hand, such as the epistemological and theoretical approach, the nature of the phenomenon under investigation, the aims and scope of the study, the quality and richness of data, or the researchers experience and skills of conducting qualitative research, should be the primary guide in determining sample size and assessing its sufficiency.
Luborsky MR, Rubinstein RL. Fam Pract. Using conceptual depth criteria: addressing the challenge of reaching saturation in qualitative research.
Accessed 17 May 2018. (SHI73).
We next review the existing conceptual and empirical literature on sample size determination.
Rather, sample size seemed to correlate with factors such as the journal of publication or the region of study (US vs Europe vs Asia). This study used rich data provided by a relatively large sample of expert informants on an important but under-researched topic. Manage cookies/Do not sell my data we use in the preference centre. Recruitment continued until data saturation was reached, defined as the point at which no new themes emerged.
By using this website, you agree to our We selected journals which (a) follow a peer review process, (b) are considered high quality and influential in their field as reflected in journal metrics, and (c) are receptive to, and publish, qualitative research (AdditionalFile1 presents the journals editorial positions in relation to qualitative research and sample considerations where available). A methodological study of sample-size reporting in focus group studies.
The information source preferred seemed to vary according to parents education; however, the sample size is too small to draw conclusions about such patterns. (BJHP35). This implicit reference point became more apparent when authors discussed the threats deriving from an insufficient sample size. Very occasionally, the articulation of the small size as a limitation was explicitly aligned against an espoused positivist framework and quantitative research. Certainly researchers caution that saturation is less applicable to, or appropriate for, particular types of qualitative research (e.g. .fCjYtf7#qYPcgU/:n[{/pYu.}3oH>c5|M' j \b`H :/( In practice, the number of individuals volunteering to participate dictated when recruitment into the study ceased (15 young people, 15 parents). We note, first, the limited number of health-related journals reviewed, so that only a snapshot of qualitative health research has been captured.
Finally, one article criticised and explicitly renounced the notion of data saturation claiming that, on the contrary, the criterion of theoretical sufficiency determined its sample size (BJHP16).
Although a small-scale qualitative study related to school counselling, this analysis can be usefully regarded as a case study of the successful utilisation of mental health-related resources by adolescents. Longitudinal studies were thus excluded whilst studies conducted within a broader research programme (e.g. Finally, SHI112 argued that once it had achieved saturation of discursive patterns, further sampling was decided and conducted to check for consistency of the findings. The review process should support authors to exercise nuanced judgments in decisions about sample size determination in the context of the range of factors that influence sample size sufficiency and the specifics of a particular study.
These thirty, from which we draw our findings, were chosen for translation based on heterogeneity in depressive symptomology and educational attainment. (BJHP09).
2017;17(5):57188. OReilly M, Parker N. Unsatisfactory saturation: a critical exploration of the notion of saturated sample sizes in qualitative research. We believe that such an analysis would yield useful insights as it links the methodological issue of sample size to the broader philosophical stance of the research. 0000009978 00000 n Studies that employed individual interviews (n=30) had an average sample size of 45 individuals and none of these explicitly reported whether their sample size sought and/or attained saturation. This study could be criticised for using a small and unrepresentative sample.
A sample of eight participants was deemed appropriate because of the exploratory nature of this research and the focus on identifying underlying ideas about the topic.
A systematic analysis of single-interview-per-participant designs within three health-related journals from the disciplines of psychology, sociology and medicine, over a 15-year period, was conducted to examine whether and how sample sizes were justified and how sample size was characterised and discussed by authors.
(SHI139).
Though not always explicitly articulated to what form of generalisation the articles referred to (see BJHP09), generalisation was mostly conceived in nomothetic terms, that is, it concerned the potential to draw inferences from the sample to the broader study population (representational generalisation see BJHP31) and less often to other populations or cultures.
(BJHP27). The limited potential for generalisation, as a result of a small sample size, was often discussed in nomothetic, statistical terms. 2007;41(1):10521. Indeed, recent research demonstrates the greater efficiency of purposive sampling compared to random sampling in qualitative studies [9], supporting related assertions long put forward by qualitative methodologists. statement and 2015;18(6):66984. Barroso J, Sandelowski M. Sample reporting in qualitative studies of women with HIV infection.
Are we there yet? 2012; http://eprints.ncrm.ac.uk/2273/4/how_many_interviews.pdf.
nursing sciences) as well as inter-disciplinary journals would add to the findings of this analysis. Providing a sample size justification was not related to the number of interviews conducted, but it was associated with the journal that the article was published in, indicating the influence of disciplinary or publishing norms, also reported in prior research [30]. It has previously been recommended that qualitative studies require a minimum sample size of at least 12 to reach data saturation (Clarke & Braun, 2013; Fugard & Potts, 2014; Guest, Bunce, & Johnson, 2006) Therefore, a sample of 13 was deemed sufficient for the qualitative analysis and scale of this study.
studies that quantified their qualitative data were excluded).
Recruitment and analysis ceased once theoretical saturation was reached in the categories described below (Lincoln and Guba 1985).
Recruitment continued until sampling frame requirements were met for diversity in age, sex, ethnicity, frequency of attendance, and health status. This mitigates against clear-cut guidelines, invariably applied.
2010;5(1). An experiment with data saturation and variability. Moreover, a high degree of consensus had begun to emerge among those interviewed, and while it is always difficult to judge at what point theoretical saturation has been reached, or how many interviews would be required to uncover exception(s), it was felt the number was sufficient to satisfy the aims of this small in-depth investigation (Strauss and Corbin 1990). Pairwise comparisons following a significant Kruskal-WallisFootnote 2 test indicated that the studies published in the BJHP had significantly (p<.001) smaller samples sizes than those published either in the BMJ or the SHI. Forum Qual Soc Res 2010;11(3):8. http://nbn-resolving.de/urn:nbn:de:0114-fqs100387.
That sample size principles, guidelines and tools have been developed to enable researchers to set, and justify the acceptability of, their sample size is an indication that the issue constitutes an important marker of the quality of qualitative research. ux2Mn[m}2M(q}[CFeY#5`sf\aI>aX}A6F56Tz|~.v)^:c35~nW|E$Kf4K]bmq42yG|a/sk Their analysis of interview data showed that code saturation (i.e. Given that older adults have been ignored in the research concerning suntanning, fair-skinned older adults are the most likely to experience skin cancer, and women privilege appearance over health when it comes to sunbathing practices, our study offers depth and richness of data in a demographic group much in need of research attention. Google Scholar.
Only occasionally was analytic or idiographic generalisation invoked to warrant the value of the studys findings [5, 17].
More recently, a quantitative tool was proposed [42] to support a priori sample size determination based on estimates of the prevalence of themes in the population. Three articles described a state of achieved saturation without using the term or specifying what sort of saturation they had achieved (i.e. Whilst the themes from the initial interviews seemed to be replicated in the later interviews, further interviews may have identified additional themes or provided more nuanced explanations.
In total, 214 articles 21 in the BMJ, 53 in the BJHP and 140 in the SHI were eligible for inclusion in the review.
studies often combined elements from different theoretical and epistemological traditions).
(SHI103). 0000011904 00000 n London: Sage; 1985. Dai NT, Free C, Gendron Y. Interview-based research in accounting 20002014: a review. Interviewer and note taker agreed that thematic saturation, the point at which no new concepts emerge from subsequent interviews (Patton, 2002), was achieved following completion of 20 interviews.
Implement Sci. All other authors have no competing interests to declare.
Psychol Health. 2077 0 obj <> endobj xref 2077 38 0000000016 00000 n
0000006193 00000 n (SHI115). Kerr C, Nixon A, Wild D. Assessing and demonstrating data saturation in qualitative inquiry supporting patient-reported outcomes research. Transforming qualitative information: thematic analysis and code development. London: Sage; 2003. p. 77108. (BJHP19).
Erickson F. Qualitative methods in research on teaching. (BJHP31). London: Sage; 2004. 1530 interviews) projects.
Book One article (BMJ18) cited a reference to support its position on saturation. We set a target of seven to 10 caregivers per site because of time constraints and the anticipated difficulty of accessing caregivers at some home based care services. 2017. https://doi.org/10.1007/s11135-017-0574-8. The funding body did not have any role in the study design, the collection, analysis and interpretation of the data, in the writing of the paper, and in the decision to submit the manuscript for publication.
Safman RM, Sobal J. Qualitative sample extensiveness in health education research. sample size) conducted and the provision of a justification (rpb=.054, p=.433).
Morse JM. The significance of saturation.
0000003258 00000 n Similarly, fewer than 10% of articles in organisation and workplace studies provided a sample size justification relating to existing recommendations by methodologists, prior relevant work, or saturation [33], whilst only 17% of focus groups studies in health-related journals provided an explanation of sample size (i.e.
Borsci S, Macredie RD, Martin JL, Young T. How many testers are needed to assure the usability of medical devices? A number of articles which characterised their sample size as small did so against an implicit or explicit quantitative framework of reference. Small scale IPA studies allow in-depth analysis which would not be possible with larger samples (Smith et al., 2009). Following the logic of informational comprehensiveness Malterud et al.
(BJHP41).
Field Methods.
(BMJ13).
Ritchie J, Lewis J, Elam G. Designing and selecting samples. In light of the growing methodological evidence in the area, transparent presentation of such evidence-based judgement is crucial and in time should surely obviate the seemingly routine practice of citing the small size of qualitative samples among the study limitations. The size of the sample was largely determined by the availability of respondents and resources to complete the study. 2015;26:175360.
This research, while limited in size, has sought to capture some of the complexity attached to mens attitudes and experiences concerning incomes and material circumstances. 0000006727 00000 n 2018;13(Suppl 1):14. Glenton C, Carlsen B, Lewin S, Munthe-Kaas H, Colvin CJ, Tunalp , et al. group interviews, structured interviews and e-mail interviews over a period of time were excluded), and the data had to be analysed qualitatively (i.e. [18] introduced the concept of information power as a pragmatic guiding principle, suggesting that the more information power the sample provides, the smaller the sample size needs to be, and vice versa. Other articles appeared to accept and acknowledge that their sample was flawed because of its small size (as well as other compositional deficits e.g.
Finally, three SHI articles explained their sample size with reference to practical aspects: time constraints and project manageability (SHI56), limited availability of respondents and project resources (SHI131), and time constraints (SHI113). 0000012498 00000 n qualities of the analysis, meeting sampling or research design requirements, richness and volume of the data obtained, nature of study, further sampling to check findings consistency).
(BMJ13).
(BMJ17). 0000050482 00000 n Leech NL. Saturation in qualitative research: exploring its conceptualization and operationalization.
To be eligible for inclusion in the review, the article had to report a cross-sectional study design.
Supporting data can be accessed in the original publications. Terms and Conditions,
Correspondence to 2003;15(4):386404.
We also extracted or coded quantitative data regarding the following aspects: Presence of sample size justification(s) (Yes/No), Presence of a particular sample size justification category (Yes/No), and, Number of sample size justifications provided.
(BJHP39).
the point at which no further dimensions, nuances, or insights of issues are identified) required 1624 interviews. Caine K. Local standards for sample size at CHI. Accessed 17 May 2018. Number of eligible articles published each year per journalFootnote.
Three articles (BMJ13; BJHP05; BJHP48) which all provided the justification of saturation, characterised their sample size as large and narrated this oversufficiency in positive terms as it allowed richer data and findings and enhanced the potential for generalisation. Konstantina Vasileiou. In other instances, articles expressed uncertainty about whether their results accounted for the full spectrum and variation of the phenomenon under investigation.
Theoretical sampling describes an iterative process of data collection, data analysis and theory development whereby data collection is governed by emerging theory rather than predefined characteristics of the population. 2017;27(4):591608. Expert Rev Pharmacoecon Outcomes Res.
2006;18(1):5982. 0000001056 00000 n 0000009139 00000 n We would like to thank Dr. Paula Smith and Katharine Lee for their comments on a previous draft of this paper as well as Natalie Ann Mitchell and Meron Teferra for assisting us with data extraction.
Two other articles (BJHP35; SHI32) linked the claimed sufficiency to the scope (i.e.
Employing the search function of each individual journal, we used the terms interview* AND qualitative and limited the results to articles published between 1January 2003 and 22 September 2017 (i.e.
We concur with existing views [11, 36] that the consideration of the characteristics of the study at hand, such as the epistemological and theoretical approach, the nature of the phenomenon under investigation, the aims and scope of the study, the quality and richness of data, or the researchers experience and skills of conducting qualitative research, should be the primary guide in determining sample size and assessing its sufficiency.
Luborsky MR, Rubinstein RL. Fam Pract. Using conceptual depth criteria: addressing the challenge of reaching saturation in qualitative research.
Accessed 17 May 2018. (SHI73).
We next review the existing conceptual and empirical literature on sample size determination.
Rather, sample size seemed to correlate with factors such as the journal of publication or the region of study (US vs Europe vs Asia). This study used rich data provided by a relatively large sample of expert informants on an important but under-researched topic. Manage cookies/Do not sell my data we use in the preference centre. Recruitment continued until data saturation was reached, defined as the point at which no new themes emerged.
By using this website, you agree to our We selected journals which (a) follow a peer review process, (b) are considered high quality and influential in their field as reflected in journal metrics, and (c) are receptive to, and publish, qualitative research (AdditionalFile1 presents the journals editorial positions in relation to qualitative research and sample considerations where available). A methodological study of sample-size reporting in focus group studies.
The information source preferred seemed to vary according to parents education; however, the sample size is too small to draw conclusions about such patterns. (BJHP35). This implicit reference point became more apparent when authors discussed the threats deriving from an insufficient sample size. Very occasionally, the articulation of the small size as a limitation was explicitly aligned against an espoused positivist framework and quantitative research. Certainly researchers caution that saturation is less applicable to, or appropriate for, particular types of qualitative research (e.g. .fCjYtf7#qYPcgU/:n[{/pYu.}3oH>c5|M' j \b`H :/( In practice, the number of individuals volunteering to participate dictated when recruitment into the study ceased (15 young people, 15 parents). We note, first, the limited number of health-related journals reviewed, so that only a snapshot of qualitative health research has been captured.
Finally, one article criticised and explicitly renounced the notion of data saturation claiming that, on the contrary, the criterion of theoretical sufficiency determined its sample size (BJHP16).
Although a small-scale qualitative study related to school counselling, this analysis can be usefully regarded as a case study of the successful utilisation of mental health-related resources by adolescents. Longitudinal studies were thus excluded whilst studies conducted within a broader research programme (e.g. Finally, SHI112 argued that once it had achieved saturation of discursive patterns, further sampling was decided and conducted to check for consistency of the findings. The review process should support authors to exercise nuanced judgments in decisions about sample size determination in the context of the range of factors that influence sample size sufficiency and the specifics of a particular study.
These thirty, from which we draw our findings, were chosen for translation based on heterogeneity in depressive symptomology and educational attainment. (BJHP09).
2017;17(5):57188. OReilly M, Parker N. Unsatisfactory saturation: a critical exploration of the notion of saturated sample sizes in qualitative research. We believe that such an analysis would yield useful insights as it links the methodological issue of sample size to the broader philosophical stance of the research. 0000009978 00000 n Studies that employed individual interviews (n=30) had an average sample size of 45 individuals and none of these explicitly reported whether their sample size sought and/or attained saturation. This study could be criticised for using a small and unrepresentative sample.
A sample of eight participants was deemed appropriate because of the exploratory nature of this research and the focus on identifying underlying ideas about the topic.
A systematic analysis of single-interview-per-participant designs within three health-related journals from the disciplines of psychology, sociology and medicine, over a 15-year period, was conducted to examine whether and how sample sizes were justified and how sample size was characterised and discussed by authors.
(SHI139).
Though not always explicitly articulated to what form of generalisation the articles referred to (see BJHP09), generalisation was mostly conceived in nomothetic terms, that is, it concerned the potential to draw inferences from the sample to the broader study population (representational generalisation see BJHP31) and less often to other populations or cultures.
(BJHP27). The limited potential for generalisation, as a result of a small sample size, was often discussed in nomothetic, statistical terms. 2007;41(1):10521. Indeed, recent research demonstrates the greater efficiency of purposive sampling compared to random sampling in qualitative studies [9], supporting related assertions long put forward by qualitative methodologists. statement and 2015;18(6):66984. Barroso J, Sandelowski M. Sample reporting in qualitative studies of women with HIV infection.
Are we there yet? 2012; http://eprints.ncrm.ac.uk/2273/4/how_many_interviews.pdf.
nursing sciences) as well as inter-disciplinary journals would add to the findings of this analysis. Providing a sample size justification was not related to the number of interviews conducted, but it was associated with the journal that the article was published in, indicating the influence of disciplinary or publishing norms, also reported in prior research [30]. It has previously been recommended that qualitative studies require a minimum sample size of at least 12 to reach data saturation (Clarke & Braun, 2013; Fugard & Potts, 2014; Guest, Bunce, & Johnson, 2006) Therefore, a sample of 13 was deemed sufficient for the qualitative analysis and scale of this study.
studies that quantified their qualitative data were excluded).
Recruitment and analysis ceased once theoretical saturation was reached in the categories described below (Lincoln and Guba 1985).
Recruitment continued until sampling frame requirements were met for diversity in age, sex, ethnicity, frequency of attendance, and health status. This mitigates against clear-cut guidelines, invariably applied.
2010;5(1). An experiment with data saturation and variability. Moreover, a high degree of consensus had begun to emerge among those interviewed, and while it is always difficult to judge at what point theoretical saturation has been reached, or how many interviews would be required to uncover exception(s), it was felt the number was sufficient to satisfy the aims of this small in-depth investigation (Strauss and Corbin 1990). Pairwise comparisons following a significant Kruskal-WallisFootnote 2 test indicated that the studies published in the BJHP had significantly (p<.001) smaller samples sizes than those published either in the BMJ or the SHI. Forum Qual Soc Res 2010;11(3):8. http://nbn-resolving.de/urn:nbn:de:0114-fqs100387.