Ethical Oversight of Artificial Intelligence in Health Research

Context

The World Health Organization (WHO) has released guidance titled Artificial intelligence-related health research: ethics review and oversight. It proposes a comprehensive framework for ensuring that ethical safeguards keep pace with the rapid development, testing and deployment of AI in healthcare.

Understanding the WHO Framework

AI is increasingly influencing how health data is analysed, how research is conducted and how clinical interventions are developed. The WHO framework therefore advocates end-to-end ethical oversight, extending from research design and data collection to clinical application, publication and post-market monitoring.

The framework broadly identifies three categories of AI-related health research:

1. Health research using AI for data analysis: AI and machine-learning techniques are used to examine large health datasets and generate new insights.
2. Research conducted with AI tools: AI becomes a research instrument, for example through large multimodal models, chatbots for participant engagement or synthetic-data generation.
3. Research conducted on AI systems: Studies evaluate AI tools themselves, including their safety, diagnostic accuracy, effectiveness and usefulness through clinical trials and implementation research.

Why Conventional Ethics Review Needs Reform

Traditional Research Ethics Committees (RECs) were largely designed around conventional biomedical research. AI introduces risks that can emerge continuously rather than only before a study begins.

• Rapid deployment: AI systems can move from experimentation to real-world use much faster than conventional medical technologies, reducing the time available to identify risks.
• Risks from supposedly anonymous data: Even de-identified datasets can create privacy concerns through re-identification, data linkage and group-level harms.
• One-time review is insufficient: AI models may change through retraining, software updates and changing datasets, requiring continued monitoring.
• Technical expertise gaps: Ethics committees may lack sufficient expertise in machine learning, model validation, algorithmic discrimination and AI-specific risks.
• Private-sector oversight gaps: Commercial developers may treat health-related AI development as internal product testing and consequently avoid independent ethical scrutiny.
• Lower barriers to experimentation: Generative AI and accessible data-science tools allow individuals and organisations without traditional clinical research backgrounds to undertake health-related experimentation.

Major Ethical Challenges

1. Algorithmic Bias and Inequality

AI systems trained predominantly on data from particular populations may perform poorly among communities that are inadequately represented in training datasets. This can reproduce or intensify existing inequalities in healthcare.

2. Hallucinations and Research Integrity

Generative AI can generate fabricated references, inaccurate medical information or misleading statistical interpretations. If such outputs enter scientific literature without adequate verification, they can undermine the reliability of health research.

3. Automation Bias

Researchers and healthcare professionals may place excessive confidence in machine-generated recommendations. Such dependence can weaken independent professional judgement and allow AI-generated errors to influence clinical decisions.

4. Synthetic Data Concerns

Synthetic data can reduce privacy risks, but it is not automatically free from bias. Models generating synthetic datasets may reproduce historical inequalities or create unrealistic relationships between demographic and clinical characteristics.

5. Labour and Human Rights Concerns

Data labelling, content moderation and other AI-support activities are often outsourced to workers in lower-income countries. Poor working conditions and exposure to disturbing material raise questions about fair remuneration, worker protection and psychological well-being.

6. Health Data Colonialism

The extraction of health datasets from low- and middle-income countries without adequate local participation, regulatory safeguards or benefit sharing can create an unequal research relationship. Ethical governance must therefore address not only individual consent but also community interests and data sovereignty.

Towards Shared Responsibility

The WHO approach places responsibility across the entire research ecosystem rather than solely on researchers.

• Researchers and institutions should incorporate ethics into AI system design, conduct bias and robustness testing, assess social impacts and ensure fair working conditions for data workers.
• Research Ethics Committees should develop AI-specific risk categories, bring technical experts and patient representatives into the review process and move towards continuous oversight.
• Data Access Committees should examine requests for secondary use of health data even when a study does not require conventional REC approval.
• Funders can make ethical certification and capacity-building requirements part of research grants, particularly for collaborations involving LMIC institutions.
• Publishers and medical societies should strengthen disclosure requirements, ensure transparency about AI use and require information on data provenance and relevant computational methods.
• Governments and regulators should modernise data-protection and health-technology regulations, strengthen cross-border data governance and connect ethical assessments with regulatory approval processes.

Way Forward

A robust governance architecture for AI-enabled health research should focus on lifecycle-based oversight rather than one-time approval. International standards, including established research-ethics frameworks, need to incorporate AI-specific requirements relating to model development, data provenance, bias, transparency and reproducibility.

AI-related health research should also be supported by specialised training and certification in AI ethics for researchers, clinicians and technology professionals. At the same time, partnerships with LMICs should emphasise local research capacity, data sovereignty and equitable sharing of benefits.

Commercial health-AI systems that directly affect patients should not fall outside ethical scrutiny merely because their development is classified as private-sector product improvement. Independent assessment, continuous monitoring and accountability mechanisms are particularly important where AI systems undergo frequent updates or may influence clinical decisions.

Conclusion

AI can substantially expand the capacity of health research, but its benefits depend on the quality, fairness and accountability of the systems through which it is developed and deployed. The WHO framework shifts ethical oversight from a single pre-research checkpoint to a continuous, multi-stakeholder responsibility. Such an approach can help ensure that innovation in digital health advances alongside privacy, equity, scientific integrity and respect for human dignity.

Source : WHO