The impact of response timing, mobile telephone use, and data collection tools on survey bias : evidence from Rakai, Uganda

Background

A credible public health survey or surveillance program is judged by its ability to measure intended outcomes accurately and cost effectively. However, in the pursuit of operational efficiency, such programs risk introducing bias in their estimates. This thesis examines several key potential sources of bias in public health surveys, focusing on three main domains: the characteristics and selection of respondents paying attention to the hard-to-reach late respondents, the use of mobile phone technology for data collection, and the adaptation of standardized measurement tools-specifically the Framingham Risk Score (FRS) and the Finnish Diabetes Risk Score (FINDRISC). The tools are internationally recognized for estimating the 10-year risk of cardiovascular disease and type 2 diabetes, respectively. By exploring these domains, this work aims to highlight critical methodological considerations that survey designers and implementers must consider minimizing measurement bias and enhance data validity.

Methods

The study was conducted within the Rakai Community Cohort Study (RCCS), a long-standing surveillance program in south-central Uganda. Covering 34 diverse communities, the RCCS focuses on HIV and other health priorities. In each round, 70-80% respond on time, while late respondents require extra effort from the researchers to track them. Study I examined whether late respondents differ from early ones and if their (late respondents) exclusion could bias results. We conducted descriptive analyses and compared these groups using Chi-square tests. Multivariable logistic regression was used to adjust for confounders, reporting adjusted odds ratios (AORs) with 95% confidence intervals. We also modelled interactions with age to estimate marginal predictive means (MPMs), illustrating how the likelihood of late response varies across age groups. The results are reported in constituent manuscript I. The use of mobile phones for population-based surveys is an ongoing global discussion, particularly in resource-limited settings. We leveraged RCCS Rounds 19 and 18 data, collected around the COVID-19 period, and data from a concurrent mobile phone-based surveillance in the same population to compare participation rates by survey modality (face-to-face vs. mobile phone surveys), we assessed phone access, and characterized the differences between respondents reached and missed by each method. The primary independent variable was interviewing modality, with adjustments made for sex, age, community type, HIV status, and other covariates. Robust standard errors accounted for clustering at the individual level. We also examined longitudinal trends in mobile phone ownership, including the frequency of mobile phone number change, the time to change of numbers and multiple SIM ownership. Phone number change was defined as the complete replacement of previously recorded numbers between survey rounds, excluding partial changes. Rates and associated factors were estimated using Poisson regression with generalized estimating equations (GEE), while time to number change was assessed using the Andersen-Gill Cox model to account for repeated measurements. The results are reported in studies II (face-to-face vs mobile phone surveys) and study III (mobile phone ownership, numbers change and time to change analyses) We conducted qualitative exit interviews to assess participants' experiences with the FRS and FINDRISC risk score tools. Interviews were held on the same day, immediately after participants completed the questionnaires and measurement procedures. The analysis, based on 15 interviews, followed an iterative approach. Thematic coding was used to identify and summarize key themes that we report in constituent study IV.

Results

Overall, 68,263 participants were included in the late respondent analysis. Men were 22% times more likely to be late respondents than women (23% vs. 19%, respectively). This was true for younger participants 15-19 years old relative to 40+years old participants. Compared to the unemployed, participants engaged in formal employment followed by those engaged in fishing were more likely to be late respondents (28% and 27% respectively). When comparing response times between people living with HIV and those who are HIV-negative, no significant difference was observed (AOR=1.06; 95% C.I .: 0.99, 1.13). In women, HIV status had no effect on the marginal predictive mean (MPM) of being late, which declined with age in both groups. Among young men, MPM of being a late respondent was significantly higher in those with HIV but decreased with age. Conversely, in HIV- negative men, the MPM increased with age. In study II, we observed 90% access to mobile phones, with lower access among older people, and people living with HIV. When including only individuals who participated in the previous RCCS survey round, participation in the face-to-face survey (82%) was higher than participation in the mobile phone survey (75%, p < . 001). Survey participation was higher among people living with HIV compared to HIV-negative individuals (84% vs 82%, p< . 001) in the face-to-face survey. In study III we assessed mobile phone ownership. A total of 97,034 participants were included in this analysis. Overall, 61.8% reported owning a phone. Mobile phone ownership increased by 33% over the study period, from 51% in 2010 to 68% in 2020 (p<0.001). Ownership was higher among men. There was no difference in phone ownership by HIV status (adjPR=1.01, 95% CI 0.97-1.05). About 75% of participants who reported more than one sexual partner in the past 12 months owned a mobile phone, compared to 58% who reported zero or one partner in the same period (p<0.001). Phone number change is as high as 15-19/100 person years and consistent across the ten years of observation.

When we examined participants' experiences with FRS and FINDRISC, we found that overall, the tools were clear and acceptable in this setting. Participants were comfortable with having their weights, waists, heights and other measurements taken if it was by a health worker. There were a few challenges with interpretation and measurement of physical activity, and dietary practice questions given the seasonal variations in their availability in the community.

Conclusion

The studies reveal that late respondents are different and cannot be ignored therefore means to speed up their participation should be devised. The use of mobile phones is feasible, there are some differentials in those who are missed by phone surveys (especially men). Meanwhile, mobile phone ownership is on a rapid rise across both sexes with men slightly higher and women rapidly catching up (by 2020). However, the rate of phone number change is quite high and this needs to be considered. The FRS and the FINDRISC scores are generally acceptable, however, some adaptation to match local practices and the seasonal dietary practices need to be addressed during survey planning, data analysis and interpretation.

Key terms: Late respondents, Mobile phones, Bias, Framingham, Finnish Diabetes Risk Score.

List of scientific papers

Paper I: Characterizing late respondents in the Rakai Community Cohort Study, 2012-2020, Uganda. Robert Ssekubugu, Zangin Zeebari, Hadija Nakawooya, Ping Teresa Yeh, Anthony Ndyanabo, Fredrick Edward Makumbi, Anna Mia Ekström, Grace Nalwoga Kigozi, Eli. M Binder, Kate Grabowski, David Serwadda, Victor Ssempijja, Larry W Chang, Godfrey Kigozi, Helena Nordenstedt [Manuscript]

Paper II: Use of Mobile Phone to collect data on COVID-19; phone access and participation rates in Rakai, Uganda. Robert Ssekubugu, Anthony Ndyanabo, Fredrick Edward Makumbi, Anna Mia Ekström, Laura Bere, Grace Nalwoga Kigozi, Hadijja Nakawooya, Joseph Ssekasanvu, Maria J. Wawer, Fred Nalugoda, Nelson Sewankambo, Victor Ssempijja, Betty Nantume, David Serwadda, Godfrey Kigozi, Ronald H. Gray, Larry W Chang, M. Kate Grabowski, Helena Nordenstedt & Joseph Kagaayi Glob Health Action. 2024 Dec 31;17(1):2419160. https://doi.org/10.1080/16549716.2024.2419160

Paper III: Mobile phone ownership, number change, time to change and the implications to public health services delivery in Uganda, 2010-2020. Robert Ssekubugu, Ping Teresa Yeh, Hadija Nakawooya, Victor Ssempijja, Godfrey Kigozi, Joseph Kagaayi, Fred Nalugoda, Anna Mia Ekström, Betty Nantume, David Serwadda, Philip Kreniske, Zangin Zeebari, Michelle A. Moffa, Larry W. Chang, Kate M. Grabowski, Fredrick Makumbi and Helena Nordenstedt [Manuscript]

Paper IV: Cardiovascular (Framingham) and type II diabetes (Finnish Diabetes) risk scores: a qualitative study of local knowledge of diet, physical activity and body measurements in rural Rakai, Uganda. Robert Ssekubugu, Fredrick Makumbi1, Rocio Enriquez, Susanne Rautiainen Lagerstrom, Ping Teresa Yeh, Caitlin E. Kennedy, Ronald H. Gray, Lilian Negesa, David M. Serwadda, Godfrey Kigozi, Anna Mia Ekström and Helena Nordenstedt BMC Public Health. 2022 Nov 29;22(1):2214. https://doi.org/10.1186/s12889-022-14620-9