KACD
Articles
- Page Path
- HOME > Restor Dent Endod > Volume 49(1); 2024 > Article
- Statistical Review Article Sample size determination for conducting a pilot study to assess reliability of a questionnaire
-
Mohamad Adam Bujang1
, Evi Diana Omar2, Diana Hui Ping Foo1, Yoon Khee Hon3 -
DOI: https://doi.org/10.5395/rde.2024.49.e3
Published online: January 10, 2024
1Clinical Research Center, Sarawak General Hospital, Ministry of Health, Kuching, Sarawak, Malaysia.
2Sector of Biostatistics and Data Repository, National Institutes of Health, Ministry of Health Malaysia, Selangor, Malaysia.
3Institute for Clinical Research, National Institutes of Health, Ministry of Health Malaysia, Selangor, Malaysia.
- Correspondence to Yoon Khee Hon, BPharm, GradDipClinPharm. Institute for Clinical Research, National Institutes of Health, Ministry of Health Malaysia, Jalan Setia Murni U13/52, Seksyen U13 Setia Alam, 40170 Shah Alam, Selangor, Malaysia. yoonkheehon@gmail.com
- Citation: Bujang MA, Omar ED, Foo DHP, Hon YK. Sample size determination for conducting a pilot study to assess reliability of a questionnaire. Restor Dent Endod 2024;49(1):e3.
• Received: September 27, 2023 • Accepted: October 30, 2023
Copyright © 2024. The Korean Academy of Conservative Dentistry
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
- This article is a narrative review that discusses the recommended sample size requirements to design a pilot study to assess the reliability of a questionnaire. A list of various sample size tables that are based on the kappa agreement test, intra-class correlation test and Cronbach’s alpha test has been compiled together. For all calculations, type I error (alpha) was set at a maximum value of 0.05, and power was set at a minimum value of 80.0%. For the kappa agreement test, intra-class correlation test, and Cronbach’s alpha test, the recommended minimum sample size requirement based on the ideal effect sizes shall be at least 15, 22, and 24 subjects respectively. By making allowances for a non-response rate of 20.0%, a minimum sample size of 30 respondents will be sufficient to assess the reliability of the questionnaire. The clear guideline of minimum sample size requirement for the pilot study to assess the reliability of a questionnaire is discussed and this will ease researchers in preparation for the pilot study. This study provides justification for a minimum requirement of a sample size of 30 respondents specifically to test the reliability of a questionnaire.
INTRODUCTION
A pilot study is a preliminary investigation conducted before proceeding to the actual survey or experiment specifically designed to address many aspects of the research study. In survey research, a pilot study is mostly conducted to test the suitability of a study instrument before conducting an actual fieldwork phase by using the instrument. Besides that, a pilot study can also evaluate the overall performance characteristics of the chosen study design, study measures, research procedures, recruitment criteria, and various other operational strategies that are being considered for use in a subsequent, often much larger study [1].
It is usually very costly to conduct large survey research. Any major flaws that occur when conducting the full-scale survey are very likely to result in a waste of resources such as time, manpower, and money. Therefore, a pilot study shall serve as an important prerequisite step before undertaking the survey research by ensuring that it can feasible to conduct the main study with a realistic ability to address its research objectives. One of the main concerns for planning a pilot study is determining the minimum sample size requirements of a pilot study for assessing the reliability of the questionnaire.
Three common statistical tests are used to determine the reliability of the questionnaire: The kappa agreement test, intra-class correlation test, and Cronbach’s alpha test. Cohen’s kappa coefficient is a test statistic that determines the level of agreement between 2 different evaluations by a variable that is expressed in a categorical form. For test-retest reliability testing, an evaluation shall be performed by the same rater at 2 different times (say time 1 and time 2) usually with a lag time of about 1 to 2 weeks [2,3]. The goal of performing the test-retest reliability test is to determine to what extent the responses elicited at time 2 are agreeable with those responses elicited at time 1. The range of kappa’s coefficient values shall usually lie between −1 and 1 wherein a value of −1 indicates perfect disagreement and a value of +1 indicates perfect agreement [4].
Cohen’s kappa measures the level of agreement or its test-retest reliability for a variable that is expressed in a categorical form, whereas the intra-class correlation is a statistical test that can be used to measure test-retest reliability for a variable in a numerical form. Some other studies have also used Pearson’s correlation coefficients to measure the level of test-retest reliability [5,6,7]. However, it must be cautioned that it might not be appropriate to use a correlation test to examine the test-retest reliability because it can often be misleading to use Pearson’s correlation coefficient to examine test-retest reliability since the correlation test only provides a measurement of the correlation between 2 different ratings and it does not take into account the presence of any systematic biases in both ratings [8]. The range of values for which the intra-class correlation coefficient shall lie is between 0 and 1 wherein its value being equal to 0 indicates perfect disagreement and its value equal to 1 indicates perfect agreement [9].
Cronbach’s alpha is a measure of the internal consistency or reliability between several different items, measurements, or ratings. For questionnaire reliability testing, Cronbach’s alpha measures the internal consistency of a questionnaire or at least the domain(s) of a questionnaire. The response variable is usually measured in an interval form that is usually based on a Likert scale. The test was developed by Cronbach and was originally used to measure the reliability of a psychometric instrument [10]. The value of Cronbach’s alpha ranges from 0 to 1 with the higher values implying the items are measuring the same latent variable or dimension. On the contrary, if Cronbach’s alpha value is low (near 0), it means some or all of the items are not measuring the same dimension and so the questionnaire does not exhibit reliability or internal consistency [10,11].
One of the main reasons for conducting a pilot study for survey research is to test the reliability of a questionnaire. Therefore, this study aims to provide useful recommendations on how to determine the minimum sample size to design a pilot study to assess the reliability of the questionnaire. Such recommendations will assist researchers to conduct proper sample size planning to prepare for a pilot survey.
SOFTWARE
A list of various sample size tables was compiled together which were based on minimum sample size requirements for performing a kappa agreement test, intra-class correlation test, and Cronbach’s alpha test. These statistical tests are selected for this compilation because they are most commonly applied in a wide variety of pilot studies. The minimum sample size requirement was estimated by using PASS 2022 Power Analysis and Sample Size Software (NCSS, LLC, Kaysville, UT, USA). The PASS software is a commercial software that provides a list of tools for determining sample size requirements for over 1,100 statistical tests and confidence interval scenarios. For all calculations, the significance level (alpha) and power are set at the values of 0.05 and 80.0% respectively.
This calculation for performing a kappa agreement test was based on a formula introduced by Flack et al. [12]. Besides the significance level (alpha) and power (i.e., 1 – beta), there are 3 other parameters are also required for sample size calculation, namely: ‘K1’ refers to the value of the kappa coefficient in the null hypothesis, ‘K2’ refers to the value of kappa coefficient in the alternative hypothesis and ‘category’ refers to the number of categories for a particular variable. K1 is set to be equal to 0, K2 is set to range from 0.2, 0.3, 0.4, 0.5, 0.6, and 0.7 and category is set to range from 2 by 2 up until 10 by 10. To simplify the calculation, it is assumed that the proportion in each category is proportional to each other (i.e., for a 2 by 2 table, the responses are assumed 0.25 in each category). Most scholars agreed that the minimum Cohen’s kappa coefficient should ideally reach at least 0.40 [13,14,15].
The calculation was performed by using a formula introduced by a previous study [16]. Besides the significance level (i.e., alpha) and power (i.e., 1 – beta), there are 3 other parameters for the sample size calculation such as R0 refers to a pre-specified value of the intra-class coefficient in the null hypothesis, R1 refers to a pre-specified value of the intra-class coefficient in the alternative hypothesis and the number of total observations which are designated as 2 shall refer to the observations for performing the test-retest reliability. R0 is set at 0 and R1 shall range from 0.2, 0.3, 0.4, 0.5, 0.6, and 0.7. Most scholars agree that the minimum value for the intra-class coefficient should ideally reach at least 0.50 [8,17,18].
The calculation was done by using a formula introduced by a previous study [19]. Besides the significance level (i.e., alpha) and power (i.e., 1 – beta), there are 3 other parameters required for sample size calculation such as CA0 refers to the value of Cronbach’s alpha coefficient in the null hypothesis, CA1 refers to the value of Cronbach’s alpha coefficient in the alternative hypothesis and category (k) refers to the number of test items. CA0 is set at 0, CA1 is set to range from 0.5, 0.6, 0.7, and 0.8 and the number of test items shall range from 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, 25, 30, 35, 40, 45, 50, and 55. The minimum value of Cronbach’s alpha coefficient should ideally reach at least 0.60 [20,21].
Ethical approval is not required since this is an article discussion from a methodology perspective. This article has been granted approval for publication by the Director General, Ministry of Health, Malaysia.
For Cohen’s kappa agreement test, the minimum sample size requirement shall range from 3 to 194 subjects which depends on the setting of a list of various conditions (i.e., total number of categories for a response variable). As a general recommendation, this study shall plan to estimate the ideal sample size which is based on the number of categories as 5 (i.e., to represent a 5 Likert scale). By setting the K1 as 0.0 and K2 as 0.4, the minimum required sample size is 15 (Table 1). For the intra-class correlation test, the minimum sample size requirement shall range from 10 to 152 subjects which depends on the setting of the conditions (i.e., values of R0 and R1). As a general recommendation, this study shall estimate the ideal sample size to be based on R0 as 0.0 and R1 as 0.5 which follows that the minimum required sample size is 22 (Table 2). For Cronbach’s alpha test, the minimum sample size required shall range from 7 to 68 subjects depending on the setting of the conditions (i.e., values of CA0, CA1, and number of test items). As a general recommendation, this study shall estimate the ideal sample size to be based on CA0 as 0.0, and CA1 as 0.6 along with a group of 5 test items which follows that the minimum required sample size is 24 (Table 3).
Table 1
Determination of minimum sample size requirement for performing kappa agreement test
Category refers to total number of categories from a particular variable.
K1 refers to value of kappa coefficient in the null hypothesis.
K2 refers to value of kappa coefficient in the alternative hypothesis.
The calculation considered the proportion in each category are proportionate to each other (i.e., for a 2 by 2 table, the responses are assumed 0.25 in each category).
Table 2
Determination of minimum sample size requirement for performing intra-class correlation test
| Number of observations | R0 | R1 | No. |
|---|---|---|---|
| 2 | 0.0 | 0.2 | 152 |
| 0.3 | 66 | ||
| 0.4 | 36 | ||
| 0.5 | 22 | ||
| 0.6 | 15 | ||
| 0.7 | 10 |
Table 3
Determination of minimum sample size requirement for performing Cronbach’s alpha test
DISCUSSION
For survey research, a pilot study is usually conducted to assess the suitability of a study instrument or questionnaire for research purposes. This is very common for a survey that involves using a newly developed questionnaire or validating an existing validated questionnaire. This pilot study for a survey aims to assess whether or not the questionnaire can be correctly understood by the subjects, and also if the subjects are able to provide logical responses as valid and reliable feedback to each of the questions. Scientifically, it determines whether the questionnaire is feasible and also sufficiently reliable for use in a new research study. A poorly developed questionnaire for use in a survey may lead to a waste of resources because the information gathered from the study respondents will probably not be adequately valid and reliable.
Very few studies have discussed sample size requirements for conducting a pilot study that emphasizes scale development for a study instrument [22,23]. However, the previous studies also did not discuss sample size requirements for conducting a pilot study for performing an agreement test which is the ideal statistical test to examine test-retest reliability. When assessing the validity and/or reliability of a questionnaire via a pilot study, a researcher will usually evaluate the test-retest reliability by performing the kappa agreement test or intra-class correlation test. As discussed earlier in the introduction section, correlation is not an appropriate statistical test to measure the test-retest reliability of a study instrument because the items are expressed on a categorical scale [8].
Besides setting the significance level (i.e., alpha) and power (i.e., 1 – beta) to be 0.05 and 80.0% respectively, a minimum required sample size shall also depend on certain pre-specified conditions and parameters set by researchers. For the kappa agreement test, intra-class correlation test, and Cronbach’s alpha test, the recommended minimum sample size requirement is calculated to be 15, 22, and 24 subjects respectively. Say, with the provision of an additional non-response rate of 20.0% and quoting the highest minimum sample size of 24, then the minimum required sample size shall be at least 24/0.8 = 30 subjects.
Here, this study proposes sample size statements based on a recommendation from a previous study [24]. Bujang in his paper recommends 5 steps (step 1: to understand the objective of the study, step 2: to decide the appropriate statistical analysis, step 3: to estimate or calculate the sample size, step 4: to provide additional allowance to cater for the possibility of non-response rate, step 5: to write a sample size statement) for sample size determination. The examples are as follows;
This study aims to determine the test-retest reliability of questionnaire A. Since, the response variables are measured in categorical form and therefore, the kappa agreement test will be used for analysis. The response variables are in 5 categories representing a 5 Likert scale. For the calculation, type I error (alpha) was set at a maximum value of 0.05, and power was set at a minimum value of 80.0%. By setting the kappa coefficient in the null hypothesis (K1) as 0.0 and the kappa coefficient in the null hypothesis (K2) as 0.4, the minimum required sample size is 15. To incorporate a non-response rate of 20.0%, a minimum sample size of 19 respondents is required.
This study aims to determine the test-retest reliability of questionnaire A. Since, the response variables are measured in numerical form and therefore, an intra-class correlation test will be used for analysis. For the calculation, type I error (alpha) was set at a maximum value of 0.05, and power was set at a minimum value of 80.0%. By setting the intra-class coefficient in the null hypothesis (R0) as 0.0 and the intra-class coefficient in the alternative hypothesis (R1) as 0.5, the minimum required sample size is 22. To incorporate a non-response rate of 20.0%, a minimum sample size of 28 respondents is required.
This study aims to determine the internal consistency of domain A. For this purpose, Cronbach’s alpha test will be used for analysis. For the calculation, type I error (alpha) was set at a maximum value of 0.05, and power was set at a minimum value of 80.0%. By setting Cronbach’s alpha coefficient in the null hypothesis (CA0) as 0.0 and the alpha coefficient in the alternative hypothesis (CA1) as 0.6 along with a group of 5 items, the minimum required sample size is 24. To incorporate a non-response rate of 20.0%, a minimum sample size of 30 respondents is required.
In a nutshell, assessing the reliability of a questionnaire will usually require a smaller sample size of less than 30. This is in line with researchers’ expectations because they expect to recruit a small sample size for a pilot study so that a large portion of patients or respondents can be reserved for a real survey. A larger sample size is often necessary to test for the validity of the questionnaire by using either exploratory factor analysis for determining its construct validity or by performing a sensitivity and specificity analysis for a questionnaire that is developed for screening purposes [25,26,27,28]. Therefore, the researcher can only assess the reliability of a questionnaire in a pilot study whereas the validity of a questionnaire will have to be assessed by the actual full-scale surveys because the validation process will necessitate a much larger sample size [29,30].
CONCLUSION
In conclusion, the determination of the minimum sample size requirement for a pilot study shall depend on the aim of the pilot study itself by giving careful consideration to all the statistical requirements. Hence, this paper provides a list of useful recommendations regarding the determination of minimum sample size requirement when designing a pilot study to assess the reliability of the questionnaire. Generally, a minimum sample size of at least 30 respondents shall usually be sufficient to assess the reliability of the questionnaire.
-
Conflict of Interest: No potential conflict of interest relevant to this article was reported.
-
Author Contributions:
Conceptualization: Bujang MA.
Formal analysis: Bujang MA.
Investigation: Omar ED, Hon YK.
Methodology: Bujang MA, Omar ED.
Project administration: Hon YK, Foo DHP.
Resources: Bujang MA.
Supervision: Bujang MA.
Validation: Omar ED, Foo DHP.
Writing - original draft: Bujang MA, Hon YK.
Writing - review & editing: Omar ED, Foo DHP.
ARTICLE INFORMATION
- 1. Moore CG, Carter RE, Nietert PJ, Stewart PW. Recommendations for planning pilot studies in clinical and translational research. Clin Transl Sci 2011;4:332-337. ArticlePubMedPMC
- 2. Jalaludin M, Fuziah M, Hong J, Mohamad Adam B, Jamaiyah H. Reliability and validity of the revised Summary of Diabetes Self-Care Activities (SDSCA) for Malaysian children and adolescents. Malays Fam Physician 2012;7:10-20. PubMedPMC
- 3. Yunus A, Seet W, Mohamad Adam B, Haniff J. Validation of the Malay version of Berlin questionaire to identify Malaysian patients for obstructive sleep apnea. Malays Fam Physician 2013;8:5-11. PubMedPMC
- 4. Cohen JA. Coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37-46.
- 5. Feldman RS, Douglass CW, Loftus ER, Kapur KK, Chauncey HH. Interexaminer agreement in the measurement of periodontal disease. J Periodontal Res 1982;17:80-89. ArticlePubMed
- 6. Lemasney J, O’Mullane D, Coleman M. Effect of fluoridation on dental health in 5- and 11-year-old Irish schoolchildren. Community Dent Oral Epidemiol 1984;12:218-222. ArticlePubMed
- 7. Mann J, Goultschin J, Call RL. Assessment of inter-examiner agreement in scoring periodontal disease. J Periodontal Res 1985;20:86-90. ArticlePubMed
- 8. Bujang MA, Baharum N. A simplified guide to determination of sample size requirements for estimating the value of intraclass correlation coefficient: a review. Arch Orofac Sci 2017;12:1-11.
- 9. Bartko JJ. On various intraclass correlation reliability coefficients. Psychol Bull 1976;83:762-765.Article
- 10. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16:297-334.ArticlePDF
- 11. Leontitsis A, Pagge J. A simulation approach on Cronbach’s alpha statistical significance. Math Comput Simul 2007;73:336-340.Article
- 12. Flack VF, Afifi AA, Lachenbruch PA, Schouten HJA. Sample size determinations for the two rater kappa statistic. Psychometrika 1988;53:321-325.ArticlePDF
- 13. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-174. ArticlePubMed
- 14. Fleiss JL. Statistical methods for rates and proportions. 2nd ed. New York, NY: John Wiley; 1981.
- 15. Bujang MA, Baharum N. Guidelines of the minimum sample size requirements for Kappa agreement test. Epidemiol Biostat Public Health 2017;14:e12267-1-ee12267-10.ArticlePDF
- 16. Walter SD, Eliasziw M, Donner A. Sample size and optimal designs for reliability studies. Stat Med 1998;17:101-110. ArticlePubMed
- 17. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol 1994;6:284-290.Article
- 18. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016;15:155-163. ArticlePubMedPMC
- 19. Bonett DG. Sample size requirements for testing and estimating coefficient alpha. J Educ Behav Stat 2002;27:335-340.ArticlePDF
- 20. Nunnally JC. Psychometric theory. New York, NY: McGraw-Hill; 1967.
- 21. Bujang MA, Omar ED, Baharum NA. A review on sample size determination for Cronbach’s alpha test: a simple guide for researchers. Malays J Med Sci 2018;25:85-99. ArticlePubMedPMC
- 22. Hertzog MA. Considerations in determining sample size for pilot studies. Res Nurs Health 2008;31:180-191. ArticlePubMed
- 23. Johanson GA, Brooks G. Initial scale development: sample size for pilot studies. Educ Psychol Meas 2010;70:394-400.ArticlePDF
- 24. Bujang MA. A step-by-step process on sample size determination for medical research. Malays J Med Sci 2021;28:15-27.Article
- 25. In: Bujang MA, Ab-Ghani P, Soelar SA, Zulkifli NA, editors. Sample size guideline for exploratory factor analysis when using small sample: taking into considerations of different measurement scales. 2012 International Conference on Statistics in Science, Business and Engineering (ICSSBE); 2012 Sep 10-12; Langkawi, Malaysia. Piscataway, NJ: IEEE; 2012 12;p. 1-5.
- 26. Bujang MA, Adnan TH. Requirements for minimum sample size for sensitivity and specificity analysis. J Clin Diagn Res 2016;10:YE01-YE06.ArticlePubMedPMC
- 27. Bujang MA, Ghani PA, Soelar SA, Zulkifli NA, Omar ED. Invalid skewed responses contributes to invalid factor solution in exploratory factor analysis: a validation approach using real-life data. J Behav Health 2019;8:152-160.Article
- 28. Bujang MA. An elaboration on sample size planning for performing a one-sample sensitivity and specificity analysis by basing on calculations on a specified 95% confidence interval width. Diagnostics (Basel) 2023;13:1390. ArticlePubMedPMC
- 29. Bujang MA, Mastura I, Bariyyah K, Baharum N, Othman SH, Mat-Lazim SS, et al. Validation of the summary of diabetes self-care activities (SDSCA) in Malay language for Malaysian adults. Malays J Public Health Med 2016;16:227-234.
- 30. Bujang MA, Lai WH, Ratnasingam S, Tiong XT, Hon YK, Yap EP, et al. Development of a quality-of-life instrument to measure current health outcomes: health-related quality of life with six domains (HRQ-6D). J Clin Med 2023;12:2816. ArticlePubMedPMC
REFERENCES
Tables & Figures
REFERENCES
Citations
Citations to this article as recorded by 
ePub Link
Cite
XML DownloadSample size determination for conducting a pilot study to assess reliability of a questionnaire
Sample size determination for conducting a pilot study to assess reliability of a questionnaire
Determination of minimum sample size requirement for performing kappa agreement test
| Category | K1 | K2 | No. | Category | K1 | K2 | No. | Category | K1 | K2 | No. |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 × 2 | 0.0 | 0.2 | 194 | 5 × 5 | 0.0 | 0.2 | 56 | 8 × 8 | 0.0 | 0.2 | 35 |
| 0.3 | 85 | 0.3 | 26 | 0.3 | 17 | ||||||
| 0.4 | 47 | 0.4 | 15 | 0.4 | 10 | ||||||
| 0.5 | 29 | 0.5 | 9 | 0.5 | 6 | ||||||
| 0.6 | 20 | 0.6 | 7 | 0.6 | 4 | ||||||
| 0.7 | 14 | 0.7 | 5 | 0.7 | 3 | ||||||
| 3 × 3 | 0.0 | 0.2 | 117 | 6 × 6 | 0.0 | 0.2 | 49 | 9 × 9 | 0.0 | 0.2 | 32 |
| 0.3 | 52 | 0.3 | 23 | 0.3 | 15 | ||||||
| 0.4 | 29 | 0.4 | 13 | 0.4 | 9 | ||||||
| 0.5 | 18 | 0.5 | 8 | 0.5 | 6 | ||||||
| 0.6 | 12 | 0.6 | 6 | 0.6 | 4 | ||||||
| 0.7 | 9 | 0.7 | 4 | 0.7 | 3 | ||||||
| 4 × 4 | 0.0 | 0.2 | 71 | 7 × 7 | 0.0 | 0.2 | 46 | 10 × 10 | 0.0 | 0.2 | 29 |
| 0.3 | 32 | 0.3 | 21 | 0.3 | 14 | ||||||
| 0.4 | 18 | 0.4 | 12 | 0.4 | 8 | ||||||
| 0.5 | 12 | 0.5 | 8 | 0.5 | 5 | ||||||
| 0.6 | 8 | 0.6 | 6 | 0.6 | 4 | ||||||
| 0.7 | 6 | 0.7 | 4 | 0.7 | 3 |
Category refers to total number of categories from a particular variable.
K1 refers to value of kappa coefficient in the null hypothesis.
K2 refers to value of kappa coefficient in the alternative hypothesis.
The calculation considered the proportion in each category are proportionate to each other (
Determination of minimum sample size requirement for performing intra-class correlation test
| Number of observations | R0 | R1 | No. |
|---|---|---|---|
| 2 | 0.0 | 0.2 | 152 |
| 0.3 | 66 | ||
| 0.4 | 36 | ||
| 0.5 | 22 | ||
| 0.6 | 15 | ||
| 0.7 | 10 |
Number of total observations which are designated as 2 refers to the observations for the purpose of performing a test-retest reliability.
R0 refers to value of intra-class coefficient in the null hypothesis.
R1 refers to value of intra-class coefficient in the alternative hypothesis.
Determination of minimum sample size requirement for performing Cronbach’s alpha test
| k | CA0 | CA1 | No. | k | CA0 | CA1 | No. | k | CA0 | CA1 | No. |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 | 0.0 | 0.5 | 68 | 8 | 0.0 | 0.5 | 37 | 30 | 0.0 | 0.5 | 33 |
| 0.6 | 40 | 0.6 | 22 | 0.6 | 19 | ||||||
| 0.7 | 24 | 0.7 | 13 | 0.7 | 12 | ||||||
| 0.8 | 15 | 0.8 | 8 | 0.8 | 7 | ||||||
| 3 | 0.0 | 0.5 | 50 | 9 | 0.0 | 0.5 | 36 | 35 | 0.0 | 0.5 | 33 |
| 0.6 | 29 | 0.6 | 21 | 0.6 | 19 | ||||||
| 0.7 | 18 | 0.7 | 13 | 0.7 | 12 | ||||||
| 0.8 | 11 | 0.8 | 8 | 0.8 | 7 | ||||||
| 4 | 0.0 | 0.5 | 44 | 10 | 0.0 | 0.5 | 36 | 40 | 0.0 | 0.5 | 33 |
| 0.6 | 26 | 0.6 | 21 | 0.6 | 19 | ||||||
| 0.7 | 16 | 0.7 | 13 | 0.7 | 12 | ||||||
| 0.8 | 9 | 0.8 | 8 | 0.8 | 7 | ||||||
| 5 | 0.0 | 0.5 | 41 | 15 | 0.0 | 0.5 | 34 | 45 | 0.0 | 0.5 | 33 |
| 0.6 | 24 | 0.6 | 20 | 0.6 | 19 | ||||||
| 0.7 | 14 | 0.7 | 12 | 0.7 | 12 | ||||||
| 0.8 | 9 | 0.8 | 8 | 0.8 | 7 | ||||||
| 6 | 0.0 | 0.5 | 39 | 20 | 0.0 | 0.5 | 34 | 50 | 0.0 | 0.5 | 33 |
| 0.6 | 23 | 0.6 | 20 | 0.6 | 19 | ||||||
| 0.7 | 14 | 0.7 | 12 | 0.7 | 12 | ||||||
| 0.8 | 8 | 0.8 | 7 | 0.8 | 7 | ||||||
| 7 | 0.0 | 0.5 | 38 | 25 | 0.0 | 0.5 | 33 | 55 | 0.0 | 0.5 | 33 |
| 0.6 | 22 | 0.6 | 20 | 0.6 | 19 | ||||||
| 0.7 | 13 | 0.7 | 12 | 0.7 | 12 | ||||||
| 0.8 | 8 | 0.8 | 7 | 0.8 | 7 |
k refers to number of test items.
CA0 refers to value of Cronbach’s alpha coefficient in the null hypothesis.
CA1 refers to value of Cronbach’s alpha coefficient in the alternative hypothesis.
Table 1 Determination of minimum sample size requirement for performing kappa agreement test
Category refers to total number of categories from a particular variable.
K1 refers to value of kappa coefficient in the null hypothesis.
K2 refers to value of kappa coefficient in the alternative hypothesis.
The calculation considered the proportion in each category are proportionate to each other (
Table 2 Determination of minimum sample size requirement for performing intra-class correlation test
Number of total observations which are designated as 2 refers to the observations for the purpose of performing a test-retest reliability.
R0 refers to value of intra-class coefficient in the null hypothesis.
R1 refers to value of intra-class coefficient in the alternative hypothesis.
Table 3 Determination of minimum sample size requirement for performing Cronbach’s alpha test
k refers to number of test items.
CA0 refers to value of Cronbach’s alpha coefficient in the null hypothesis.
CA1 refers to value of Cronbach’s alpha coefficient in the alternative hypothesis.
