Study design and setting
This prospective, observational mixed-methods study validates the integration of a DDSS within a government platform in SA, examining its impact on health-seeking decision-making among pregnant women and mothers. Enrolling participants between May and June 2023, the study adopts a decentralized approach, enrolling participants from various regions across SA.
Eligible participants were women aged 18 years or older, registered with MomConnect and using the platform via WhatsApp. Inclusion required participants to conduct a symptom assessment for themselves or their children. Exclusion criteria included age under 18 for registered users, inability to complete an English assessment due to literacy or language barriers, and incomplete surveys. Participants were advised not to use the cAI for emergency situations.
A sample size of 184 for the primary endpoint analysis was calculated using a maximum acceptable unsafe advice threshold of 10% and an expected unsafe advice rate of 5%, an alpha level of 0.05 and a power of 0.8. To ensure at least 100 participants completed all surveys, we aimed to enroll 800 participants, with the possibility of increasing the target to 1,000. No additional power calculation was performed for the secondary outcomes.
DDSS design and validation
The DDSS’s conversational artificial intelligence uses a Bayesian network framework where disease probability estimations are derived through approximate inference algorithms.
Information-theoretical methods dynamically select questions to optimize diagnostic information gain based on evolving probability distributions. The medical knowledge base comprises disease models of common and rare conditions with symptom–disease associations derived from peer-reviewed literature through a standardized curation process by medical doctors. Disease models incorporate epidemiological data to derive population-appropriate prior probabilities, with continuous knowledge base expansion following established protocols.
Validation uses a multitier methodology: automated testing against thousands of internal test cases spanning medical specialties and diagnostic complexity levels, validation using physician-authored cases kept confidential from internal teams, and continuous post-market monitoring through user feedback and research studies to ensure clinical safety and effectiveness.
The DDSS design was informed by health-seeking behaviour theory and models35,36, behavioural design thinking, human factors and usability engineering studies, which focused on how the user journey affected ultimate patient health choices37. In early validation, the DDSS’s design was informed through a novel measure, iterate measure mixed-methods study design, which combined survey-based quantitative insights with qualitative insights from applied methods of contextual inquiry, observations, contextual interviews and cognitive walkthroughs with users38.
Data collection procedures
The DDSS was promoted to eligible users on MomConnect. After providing informed consent, participants completed a questionnaire via the platform to collect baseline data on socioeconomic status (household income, education, employment, cost and distance to healthcare) and initial health-seeking intentions. They then used the DDSS to input symptoms and associated risk factors, guided by its algorithm. The DDSS provided a summary message and PDF report detailing probable symptom causes and recommendations for care.
Post-assessment, participants reported changes in their health-seeking intentions and provided feedback on the DDSS’s usability and utility. One week later, a follow-up survey assessed whether they sought care and, if so, the quality of service received.
Phone survey
A subset of participants, selected through stratified random sampling, participated in phone surveys 1–2 weeks after their DDSS assessment. Sampling included 50% random selection, 25% with symptoms classified as ‘serious’ and 25% who changed their care-seeking intentions from seeking professional care to self-care. These surveys explored DDSS experiences and health information priorities. Responses were thematically analysed based on recurring patterns and key phrases.
Physician panel evaluation
A panel of three senior physicians (specialized in gynaecology or family medicine) assessed a random subset of 184 cases, evaluating the appropriateness and safety of DDSS recommendations. Each case was reviewed by two physicians, with a third resolving disagreements. The eight-level DDSS’s urgency advice was mapped to a four-level scale following the simple mapping scheme set out in Supplementary Table 1 and compared with the panel’s recommendations and participants’ intentions before and behaviour after DDSS use.
Assessment of socioeconomic barriers
To capture heterogeneity in participants’ access to healthcare, we developed a composite measure of barriers drawing on both district-level health system indicators derived from participants’ registration location on the government platform and individual-level survey data (Supplementary Table 2). District-level data were obtained from the District Health Barometer 2022/20233, a nationally recognized source of health system performance data across all 52 districts in SA. Indicators included: (1) antenatal first-visit coverage, (2) MMR in facilities, (3) early neonatal mortality rate in facilities, (4) density of medical practitioners per 100,000 uninsured population and (5) hospital bed availability per 10,000 population. These were dichotomized using the national averages reported in the Barometer (for example, below-average ANC coverage or hospital bed density, and above-average mortality rates) to flag districts with poorer health system access or quality.
At the individual level, participants provided information on distance to the nearest facility (>60 min), cost of travel (>50 ZAR), monthly household income (<1,600 ZAR, approximately US$97), and employment status (unemployed or still in school). Each indicator was scored as ‘0’ (no barrier) or ‘1’ (barrier present), yielding a total possible score of 0–9 across all domains. Individual-level variables answered with ‘Skip’ or ‘I don’t know’ were scored as ‘0’. We classified participants as experiencing minimal barriers (0–2), moderate barriers (3–5) or severe barriers (6–9). The indicators were selected to reflect both structural health system capacity (availability of services and providers) and household-level constraints (financial, geographic and socioeconomic), which together are known determinants of maternal health access and outcomes in SA. In addition, the geographic distribution of participants was analysed in relation to MMR and IWI data for SA provinces.
Data analysis
Data analysis was conducted using Python (v3.12) and Microsoft Excel (v16.92). Descriptive statistics were used to summarize participant demographics, socioeconomic characteristics and consultation information. The safety of advice, intentions and behaviours was evaluated by comparing them with the panel’s determination of the lowest safe urgency advice for each case. The primary endpoint was analysed using a one-sample exact binomial test to determine if the proportion of unsafe advice was significantly below the hypothesized 10% threshold (n = 184), with 95% CIs calculated via the Clopper–Pearson exact method. In addition, advice, intentions and behaviours were compared with the panel’s assessment of the most appropriate urgency rating. The Stuart–Maxwell test assessed changes in behaviour, and chi-squared tests evaluated the alignment between intended and actual behaviour and the advice provided. Comparisons between barrier levels and safety outcomes were performed using Fisher’s exact test, with effect sizes reported as RRs with 95% CIs. CIs for differences in proportions were calculated using the Newcombe–Wilson method.
Phone survey data were analysed using descriptive statistics to compare IWI scores24,25,39, calculated via principal component analysis. Closed questions on participants’ socioeconomic background and healthcare access were summarized, while open-ended responses about DDSS integration and health information seeking were thematically coded and quantified to identify common patterns and their frequency.
To explore factors associated with changes in healthcare-seeking intent and subsequent behaviour, three multivariable logistic regression models were developed. Two models examined directional shifts in intent following DDSS use: one assessing an upward change versus rest, and a second assessing a downward change versus rest. A third model examined predictors of actual safe care-seeking behaviour.
Predictors included categorical variables for employment status and pregnancy status. To ensure model stability and address data sparsity, employment status was collapsed into three categories (unemployed, employed or in education). Similarly, pregnancy status was collapsed into three categories: not pregnant, first/second trimester and third trimester.
Continuous variables included age, household income (1–4), time to reach care (1–5), cost of care (1–4) and the DDSS urgency advice level (1–8). The participant’s pre-DDSS intent (1–4) was included as a baseline adjustment variable in the intent-change models.
Standard maximum likelihood logistic regression was used for all models. The model results are presented as AOR with 95% CIs and associated P values.
There were no missing data for the primary outcome. Missingness in secondary outcomes was mainly attributable to loss to follow-up at the 1-week survey (n = 183; 18.9%), with additional missing data arising from skipped responses for health intent before use of the DDSS (n = 61; 6.3%), health intent after use of the DDSS (n = 18; 1.9%) and income (n = 57; 5.9%) or missing location data (n = 51; 5,3%), while all other variables had ≤1% missingness.
All regression analyses were conducted using complete-case analysis, and participants providing ‘Skip’ or ‘I don’t know’ responses to the intent/behaviour questions were excluded. To assess the potential impact of complete-case analysis on our regression models, participants included in the regression models were systematically compared with those excluded due to missing, skipped or ‘don’t know’ responses using chi-square tests or Fisher’s exact tests for categorical variables, and Mann–Whitney U tests for ordinal and continuous variables. Included and excluded participants were comparable across age, employment status, pregnancy stage, income, cost to care and DDSS urgency advice. A statistically significant difference was observed for time to the nearest healthcare facility (P = 0.0107); however, the absolute difference was small (mean 1.53 in included participants versus 1.66 in excluded participants, with both groups having a median of 1).
Reimbursement
Participants received a R30 (approximately US$1.80) airtime top-up upon completing each survey stage.
Generalizability
Subgroup comparisons ensured findings were generalizable to the broader MomConnect user population. Comparisons included MomConnect users utilizing the DDSS but not enrolled in the study and users not using the DDSS.
Data management and ethics
Data were securely stored in a validated electronic system, anonymized with unique identifiers and managed under the Protection of Personal Information Act. Ethical approval was granted by the Pharma Ethics Review Board (no. 220624611) and the SA NDOH. The study was registered at ClinicalTrials.gov (NCT06790069) and complied with the Declaration of Helsinki and ISO 14155:2020 guidelines. Informed consent was obtained from all participants. STROBE guidelines were followed throughout22.
In accordance with the provisions of the Protection of Personal Information Act, 4 of 2013 (‘POPIA’), Reach is mandated as the Operator to the SA NDOH in the provision of the MomConnect WhatsApp Service to the end users of the MomConnect Service. Reach, under the terms of a written letter of mandate, entered a subprocessor agreement with Ada. The processing activities of Ada rendered it both a suboperator under POPIA as well as a subprocessor under the General Data Protection Regulation.
Reporting summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.
