Can improved paediatric pneumonia diagnostic aids support frontline health workers in low resource settings? Large scale evaluation of four respiratory rate timers and five pulse oximeters in Cambodia, Ethiopia, South Sudan and Uganda

Abstract

Background: Pneumonia is the leading cause of infectious death in children under-five in sub-Saharan Africa and Southeast Asia. Currently, the diagnostic criterion for pneumonia is based on increased respiratory rate (RR) in children with cough and/or difficulty breathing. Low oxygen saturation, usually measured using pulse oximeters, is an indication of severe pneumonia. Health workers report finding it difficult to accurately count the number of breaths and current RR counting aids are often difficult to use or unavailable. Improved RR counting aids and lower-cost pulse oximeters are now available but their suitability in these settings and for these populations are untested.

Objective: The studies sought to identify and evaluate the most accurate, acceptable and user-friendly respiratory rate counting devices and pulse oximeters for diagnosis of pneumonia symptoms and severity in children by frontline health workers in low-resource settings.

Methods: Three sub-studies (I-III) were conducted among health workers, children under five and their caregivers, and national stakeholders. Sub-study I uses an explanatory qualitative approach with pile sorting and focus group discussions with frontline health workers and national stakeholders to explore their perspectives regarding the potential usability and scalability of seven pneumonia diagnostic aids. In sub-study II (a & b) four RR counters and five pulse oximeters were evaluated for performance by a cross-sectional sample of frontline health workers in hospital settings against reference standards in Cambodia, Ethiopia, South Sudan and Uganda. In sub-study III the same nine devices were evaluated using mixed methods for usability and acceptability in routine practice, over three months, in the four countries.

Findings: Frontline health workers and national stakeholders’ universally valued device simplicity, affordability and sustainability. They prioritised different device characteristics according to their specific focus of work, with health workers focusing more on device acceptability and national stakeholders’ being less accepting of new technologies (Sub-study I). In sub-study IIa most CHWs managed to achieve a RR count with the four devices. The agreement with the reference standard was low for all; the mean difference of RR measurements or breaths per minute (bpm) from the reference standard for the four devices ranged from 0.5 bpm (95% CI -2.2 to 1.2) for the respirometer to 5.5 bpm (95% CI 3.2 to 7.8) for Rrate. Performance was consistently lower for young infants (0 to <2 months) than for older children (2 to ≤59 months). Agreement of RR classification into fast and normal breathing was moderate across all four devices, with Cohen’s Kappa statistics ranging from 0.41 (SE 0.04) to 0.49 (SE 0.05). In Sub-study IIb, although all five pulse oximeters tested in the field had performed well on a simulator (±2% SpO2 from the simulator), their performance was more varied when used on real children by frontline health workers. The handheld pulse oximeters had greater overall agreement with the reference standard, ranging from -0.6% SpO2 (95% CI -0.9, 0.4) to -3.0% SpO2 (95% CI -3.4, -2.6) than the finger-tip pulse oximeters, which ranged from -3.9% SpO2 (95% CI -4.4, -3.4) to -7.9% SpO2 (95% CI -8.6,-7.2). This was particularly pronounced in the younger children, where handheld devices had -0.7 SpO2 (95% CI -1.4, -0.1) to -5.9 SpO2 (95% CI -6.9, -4.9) agreement, compared to fingertip devices, which had -8.0 SpO2 (95% CI -9.4, -6.6) to -13.3 SpO2 (95% CI -15.1, -11.5) agreement. First level health facility workers had better agreement in classification of hypoxaemia with the reference standard (𝛋=0.32; SE 0.05 to 𝛋=0.86; SE 0.07) for all five devices, when compared to CHWs (𝛋=0.15; SE 0.02 to 𝛋=0.59; SE 0.03). In Sub-study III health workers reported being better supported by assisted RR counters, which provided more support than their standard practice ARI timer in counting and classifying RR in sick children under 5 in these settings.

Conclusions: Frontline health workers were able to use the nine test devices to measure RR and oxygen saturation in children under 5, but with variable performance, and found it more difficult to get a successful measurement in younger children. Frontline health workers were better supported by assisted RR counters, such as Rrate and respirometer, compared to their standard practice diagnostic aid, MK2 ARI timer. Handheld pulse oximeters with multiple probes performed better than fingertip pulse oximeters, especially in younger children. The views of different stakeholder groups should be considered when looking to take these types of pneumonia diagnostic aids to scale. A consensus view on a robust research method and reference standard to evaluate future pneumonia diagnostic aids needs to be reached. While laboratory testing of new diagnostic aids can be valuable it should not replace field testing with frontline health workers in routine practice. Automated, easy to use, robust and affordable pneumonia diagnostics aids need to be developed and launched at scale to better support frontline health workers to address the high pneumonia burden in resource poor settings.