Sample Size Calculations for Population Size Estimation Studies Using Multiplier Methods With Respondent-Driven Sampling Surveys (2024)

Reviewed by Lisa Johnston and Wolfgang Hladik

Overview

We briefly outline multiplier method size estimation, the approach to estimating uncertainty in the resulting population size estimates, and integrate this with advice on design effects and sample size requirements for RDS surveys.

Multiplier Method Population Size Estimation

Multiplier methods use 2 sources of data to estimate population size as described above: (1) a count of unique individuals from the target population receiving a service or unique objects distributed among this population, M, and (2) a representative estimate of the proportion of the target population in receipt of the service or object, P. The count is divided by the proportion as in Equation 1 (Figure 1) to obtain the population size estimate.

Johnston et al. [4] suggest using the Delta method to estimate the variance of the PSE, which combines variance in P and variance in M. We assume that M, as a count of target population individuals on a roster or unique objects distributed to the target population, follows a Poisson distribution for which the mean and variance are equal to µM [8]. The variance of P depends on the sample size of the RDS survey.

Sample Size Calculations

RDS is a structured, peer-referral recruitment method assuming a model for estimating each participant’s probability of inclusion; thus, allowing weighting of responses to be used to approximate a random sample [9]. Existing guidance for estimating proportions from a RDS survey suggests that the sample size required for a simple random sample must be multiplied by a design effect (DEFF) to account for the RDS design [10]. Empirical reviews of RDS surveys have found most DEFFs to lie between 2 and 4, though some studies have found higher DEFFs [5-7,11]. The sample size for the RDS survey used to estimate P can be calculated as Equation 2 (Figure 1) given that n is sample size, µ_P is the estimate for the proportion we wish to estimate, and se(P) is the standard error of P. Recognizing that PSE are often required for small sites, we additionally suggest using a sample size n_adj that has been corrected for an estimated finite population as Equation 3 (Figure 1), where N is the estimated population size.

Rearranging Equation 2, and using n_adj as obtained in Equation 3, se(P) as corrected for finite population size can be calculated as Equation 4 (Figure 1), and the effect on the variance of the PSE can be obtained by inserting se(P) into Equation 5 (Figure 1). The 95% confidence interval (CI) around the PSE can then be obtained by taking the square root of var(M/P), multiplying by 1.96 (assuming an approximately normal distribution) and subtracting/adding to the PSE.

We examined the relationship between sample size, P, and the width of the 95% CI obtained for a population size estimate of 15,000, fixing this estimate so that M varied with P.

Application to Estimating the Number of Female Sex Workers in Harare

To estimate the number of FSW in Harare, we planned a RDS survey of FSW aged 18 and older who had resided in the city for at least the previous 6 months. For service data, we planned to use Sisters with a Voice clinic attendance records. FSW attending this clinic, which provides sexual and reproductive health services for self-identified FSW, are given unique identification numbers and their visits recorded and dated (described further elsewhere [12]). For M, we planned to record the number of unique women attending in the 6 months prior to the survey.

To identify a reasonable estimated FSW population size for sample size calculation, we used previous estimates from a systematic review of FSW prevalence among 15- to 49-year-old women in sites from sub-Saharan Africa (.07%–4.3%) and multiplied them by the number of women of this age in Harare [13]. The 2012 Zimbabwe census estimates that 30.2% of the population of Harare is female aged 15 to 49, and that the total population of Harare is 2,123,132 [14], giving a FSW population size in Harare of 4488 to 27,572, with a plausible midrange estimate of 15,000, or 2.3%, of the adult female population.

We examined the number of sex workers who visited the program for different reference periods up to April 23, 2015 to generate likely values for M and P given an assumed PSE of 15,000. We then examined the impact of reference period on sample size requirements assuming these values of M and P. Finally, we investigated the effect of DEFF on the width of the 95% CIs around the PSE for different sample sizes of the RDS survey. We developed a Web-based tool to implement the methods described here [15].

Relationships Between RDS Survey Sample Size, P, M, and Width of the 95% Confidence Intervals

For all values of P and M, increasing the RDS survey sample size decreases the width of the CI around the PSE, Figure 2. The precision of the PSE also varies by the values of P and M, such that much larger sample sizes would be required to estimate the PSE with the same level of precision if P is small rather than large (and correspondingly, M is small rather than large).

In Figure 2, values of M are varied with P so that M / P is always equal to 15,000. For instance if P=.05, M=750, or if P=.4, M=6000.

Application to Planning a Population Size Estimation Study

For our Harare example, we were able to review earlier service attendance data to see how the value of M might depend on the reference period chosen. The value of M in turn affects the sample size required via the impact on P, as shown in Table 1 and Figure 3, which assume a population of 15,000 FSW in Harare. Depending on whether we chose a period of 1 or 24 months, we might be estimating a proportion of .006 or a proportion of .148. For a given sample size, the width of the 95% CI will increase if the reference period is shorter and P is smaller. Higher DEFFs increase the uncertainty around the PSE, Figure 4.

We used previous service attendance data to observe how M varied by reference period, and therefore to predict how our estimate of P, the proportion of women attending, might vary by the reference period we chose, see Table 1. Figure 3 shows the relationship between these values of P with the width of the 95% CI’s around the PSE for different sample sizes.

Based on changes in the width of the estimated 95% CIs with increasing sample size (Figure 3) and on choosing a reference period that would both reduce the likelihood of recall bias while preventing P from being too low, we chose a sample size of 1500 FSW for the RDS survey and a reference period for Sisters service attendance of 6 months, for which we estimated P would be approximately .06.

Summary and Discussion of Findings

We have applied current guidance on RDS and multiplier methods to propose an approach to planning population size estimation studies and determining sample size. We have given an example using the SMM, similar principles of which can be applied to the UOM.

Even for large sample sizes, 95% CIs around the PSE are wide. The uncertainty around the PSE is more sensitive to the uncertainty in P than in M, which is evident from the formula for var(M/P). Researchers cannot choose a value of P, but they can encourage it to be higher by encouraging M to be higher. Concerned only with random error, it would improve the precision of the PSE to choose a longer reference period, and thus likely obtain a larger P in the case of the SMM, or to distribute a greater number of unique objects for the UOM. However, for the SMM this approach needs to be balanced against the potential for recall bias on estimation of P. It is also possible that the relationship between M and the reference period will differ across service types and according to whether individuals visit frequently or sporadically, and that bias in M might vary by reference period. If there are errors in unique identification of individuals in the service data, a longer reference period could lead to a higher likelihood of duplicate identification numbers, which would bias the PSE. For the UOM, care is needed to ensure that more objects distributed did not increase the likelihood of dependence between methods of distribution and RDS survey recruitment, a key source of potential bias.

We used DEFFs of 2 to 4 in our sample size calculations, but it is possible that a higher value would be more appropriate. Previous research has found that high levels of hom*ophily (similarity) between recruiters and recruitees in RDS surveys is associated with higher DEFFs [7]. In SMM studies, the RDS survey is intended to measure program attendance, a characteristic that is likely to exhibit high hom*ophily as it is a route by which participants might know and recruit each other. High hom*ophily is also likely when the same social networks are used to distribute unique objects and to later recruit individuals to a RDS survey. Higher DEFFs might therefore be required, though in a previous population size estimation study of 9 communities in Zimbabwe, we found evidence of high hom*ophily by program attendance for some sites but not all [8].

RDS surveys must have sufficient recruitment waves in order to reach stable estimates. There should also be sufficient numbers of seed participants to reflect diversity of the target population [16], concerns that need to be considered alongside the total sample size [17].

Recommendations

This short paper considers random error around size estimates and does not discuss a consideration of bias resulting from unmet assumptions of both the multiplier and RDS methods, which we consider elsewhere [8]. We agree with advice that researchers should use more than one multiplier and more than one method of estimating population size [18,19]. However, justification for sample size is often not given. Based on our findings, we strongly recommend conducting sample size calculations for estimating population size and considering the relationship between reference period or number of objects distributed and P for potential impact on uncertainty.

Conflicts of Interest: None declared.

1. Holland CE, Kouanda S, Lougué M, Pitche VP, Schwartz S, Anato S, Ouedraogo HG, Tchalla J, Yah CS, Kapesa L, Ketende S, Beyrer C, Baral S. Using population-size estimation and cross-sectional survey methods to evaluate HIV service coverage among key populations in Burkina Faso and Togo. Public Health Rep. 2016;131:773–782. doi:10.1177/0033354916677237. [PMC free article] [PubMed] [CrossRef] [Google Scholar]

4. Johnston LG, Prybylski D, Raymond HF, Mirzazadeh A, Manopaiboon C, McFarland W. Incorporating the service multiplier method in respondent-driven sampling surveys to estimate the size of hidden and hard-to-reach populations: case studies from around the world. Sex Transm Dis. 2013;40:304–310. doi:10.1097/OLQ.0b013e31827fd650. [PubMed] [CrossRef] [Google Scholar]

6. Johnston LG, Chen Y, Silva-Santisteban A, Raymond HF. An empirical examination of respondent driven sampling design effects among HIV risk groups from studies conducted around the world. AIDS Behav. 2013;17:2202–2210. doi:10.1007/s10461-012-0394-8. [PubMed] [CrossRef] [Google Scholar]

7. Wejnert C, Pham H, Krishna N, Le B, DiNenno E. Estimating design effect and calculating sample size for respondent-driven sampling studies of injection drug users in the United States. AIDS Behav. 2012;16:797–806. doi:10.1007/s10461-012-0147-8. http://europepmc.org/abstract/MED/22350828. [PMC free article] [PubMed] [CrossRef] [Google Scholar]

8. Chabata S. Estimating the size of the female sex worker population in urban and rural communities in Zimbabwe: Project Report for Masters of Science in Medical Statistics [master's thesis] London, UK: London School of Hygiene and Tropical Medicine; 2015. [Google Scholar]

9. Heckathorn D. Respondent-driven sampling: a new approach to the study of hidden populations. Social Problems. 1997;44:174–199. doi:10.2307/3096941. [CrossRef] [Google Scholar]

12. Hargreaves JR, Mtetwa S, Davey C, Dirawo J, Chidiya S, Benedikt C, Naperiela MS, Wong-Gruenwald R, Hanisch D, Magure T, Mugurungi O, Cowan FM. Cohort analysis of programme data to estimate HIV incidence and uptake of HIV-related services among female sex workers in Zimbabwe, 2009-14. J Acquir Immune Defic Syndr. 2015 doi:10.1097/QAI.0000000000000920. [PubMed] [CrossRef] [Google Scholar]

15. Fearon E. Planning a multiplier method population size estimation study with RDS: a tool for calculating width of the 95% confidence intervals for a given set of inputs 2017. 2017. [2017-08-30]. https://fearone.shinyapps.io/rds_ss_shiny/ webcite.

16. Johnston LG, Whitehead S, Simic-Lawson M, Kendall C. Formative research to optimize respondent-driven sampling surveys among hard-to-reach populations in HIV behavioral and biological surveillance: lessons learned from four case studies. AIDS Care. 2010;22:784–792. doi:10.1080/09540120903373557. [PubMed] [CrossRef] [Google Scholar]

18. Abdul-Quader AS, Baughman AL, Hladik W. Estimating the size of key populations: current status and future possibilities. Curr Opin HIV AIDS. 2014;9:107–114. doi:10.1097/COH.0000000000000041. [PMC free article] [PubMed] [CrossRef] [Google Scholar]

19. Wesson P, Reingold A, McFarland W. Theoretical and empirical comparisons of methods to estimate the size of hard-to-reach populations: a systematic review. AIDS Behav. 2017 doi:10.1007/s10461-017-1678-9. [PubMed] [CrossRef] [Google Scholar]