TLDR: A recent qualitative study highlights significant trust deficiencies in AI-based diabetic retinopathy screening, pointing to critical shortcomings in data collection transparency, patient consent, data privacy, and regulatory oversight. The research, involving diverse healthcare stakeholders, underscores the urgent need for robust ethical frameworks and transparent practices to ensure the responsible and effective deployment of AI in healthcare.
Artificial intelligence (AI) holds immense promise for advancing individual well-being and societal progress, particularly in healthcare. However, its integration, such as in diabetic retinopathy (DR) screening, introduces a complex array of ethical, legal, social, and technological challenges. A new study, titled ‘Evaluating Trustworthiness in AI-Based Diabetic Retinopathy Screening: Addressing Transparency, Consent, and Privacy Challenges,’ emphasizes that building trust in AI systems is paramount for their acceptance and effectiveness, ensuring equitable and ethical healthcare solutions.
The qualitative study investigated the perspectives of various health system stakeholders, including ophthalmologists, retina specialists, program officers, legal experts, bioethics experts, and AI developers. The research utilized pretested semi-structured questionnaires and analyzed data using Atlas.Ti, adhering to the Consolidated Criteria for Reporting Qualitative (COREQ) Research guidelines.
Key findings from the study revealed critical shortcomings in the data collection practices of AI companies. These include a notable lack of transparency, inadequate patient consent processes, insufficient attention to data privacy, and the absence of robust regulatory frameworks. These gaps have reportedly led to unchecked data privacy breaches and raised serious concerns about emerging ‘data colonialism’ practices within the healthcare system.
Six key themes emerged from the interviews regarding the perceived trustworthiness of AI in DR screening: the effectiveness of the AI algorithm, responsible AI concerning data collection, ethical consideration and approval, explainability, challenges of AI implementation, and accountability and liability.
Several quotes from the participants underscored these concerns. An AI developer noted the lack of ethical requirements in current practices: “We started going out into the market conducting camps on our own, and during the entire process, we’ve screened more than 94,000 people till date, and there is no ethics requirement to conduct camps.” Conversely, an ophthalmologist highlighted AI’s potential, stating, “The primary purpose of AI is its use for screening purposes, and it can cover up to 80 percent of all community diabetic retinopathy-related problems without a needless referral.”
However, legal experts voiced significant concerns about current regulatory landscapes. One legal expert in AI remarked on the self-regulatory nature of data handling: “Right now AI companies are quickly creating large data sets. They are self-regulating and saying we are anonymising, and not revealing it into public domain, don’t get into trouble.” Another AI developer added, “The available ethical framework does not impose any compulsion to have any ethics clearances for AI training datasets.” A legal expert further elaborated on the regulatory environment, stating, “Regulatory policy regulations are vague or gray or not completely well-defined or not completely tested.”
Also Read:
- Global Health Organizations Unveil New Ethical Principles for AI in Medical Care
- Capgemini’s New Guide Emphasizes Urgent Need for AI Ethics Governance
The study concludes that trustworthy AI necessitates transparent data practices, robust patient consent mechanisms, and strict adherence to ethical and privacy standards. Addressing these areas is crucial for overcoming current shortcomings and ensuring the reliable and ethical deployment of AI in healthcare. The researchers recommend the development of robust data governance and privacy frameworks, along with reviewing and updating current organizational data governance infrastructure to foster strategic collaborative partnerships for implementing AI best practices.
