Presagen publishes in Nature Scientific Reports: novel AI algorithm that can automatically, efficiently and accurately detect errors in medical data

AI healthcare company Presagen has published a novel Artificial Intelligence (AI) algorithm in the journal Nature Scientific Reports, that can automatically detect errors in data without human oversight. The AI algorithm’s accuracy and efficiency was verified on various types of medical data, including images and record-based data.

Medical data is typically prone to errors due to expert subjectivity, inaccurate test results or confounding biological complexities. It is often difficult or infeasible for medical experts to detect these elusive errors manually, particularly in large datasets. More importantly, with tightening consumer and patient (medical) privacy laws, manual verification of data is not always possible. Thus a human-free (i.e. private) alternative which is scalable and accurate is the only way to ensure that global medical datasets used for AI medical applications or other purposes, such as clinical trials or studies, are free from errors to ensure high quality outcomes.

Presagen’s AI-based error detection algorithm, LDC, uses deep learning in conjunction with an approach called label-clustering to analyze datasets and identify inconsistencies in the data. These inconsistencies are typically driven by errors, or mislabeled data.

Presagen Co-founder and Chief Scientist Dr Jonathan Hall said “Say you have an image of a cat that was erroneously labeled as a dog. The AI algorithm would compare that cat to all the dogs in its database and recognize significant inconsistencies to what a dog should look like. The algorithm would then flag the image of the cat labeled as a dog as an error. This data can then be re-labeled to a dog or discarded as questionable data.”

The patented algorithm called the LDC was tested and shown to be accurate and highly efficient, and thus scalable and low cost to run.

Senior AI Scientist Dr Tuc V. Nguyen said “In one medical dataset, the LDC identified errors with an accuracy of up to 85%, whilst requiring up to 93% less computing resources compared with a previous algorithm that we developed, called the UDC. When we trained AI models on this error-free data, AI training exhibited greater stability and up to a 45% improvement in accuracy.”

Removing erroneous medical data without human oversight has become critical for developing reliable, accurate and scalable AI applications using global datasets. The use of global datasets is important to ensure diversity is represented in the AI, minimizing bias.

The LDC is one of the core technologies used to develop Presagen’s up and coming healthcare product Life Whisperer Oocytes, which uses AI to assess images of female eggs in IVF or for egg freezing to help inform on the quality of the eggs and likelihood of developing a usable embryo for fertilization. Life Whisperer’s existing embryo assessment products are currently being used by IVF clinics globally.

Paper Title

Efficient automated error detection in medical data using deep-learning and label-clustering

https://www.nature.com/articles/s41598-023-45946-y

Authors

T.V. Nguyen, M. A. Dakka, S. M. Diakiw, M. D. VerMilyea, M. Perugini, J. M. M. Hall, and D. Perugini

Paper Abstract

Medical datasets inherently contain errors from subjective or inaccurate test results, or from confounding biological complexities. It is difficult for medical experts to detect these elusive errors manually, due to lack of contextual information, limiting data privacy regulations, and the sheer scale of data to be reviewed. Current methods for training robust artificial intelligence (AI) models on data containing mislabeled examples generally fall into one of several categories –attempting to improve the robustness of the model architecture, the regularization techniques used, the loss function used during training, or selecting a subset of data that contains cleaner labels. This last category requires the ability to efficiently detect errors either prior to or during training, either relabeling them or removing them completely. More recent progress in error detection has focused on using multi-network learning to minimize deleterious effects of errors on training, however, using many neural networks to reach a consensus on which data should be removed can be computationally intensive and inefficient.

In this work, a deep-learning based algorithm was used in conjunction with a label-clustering approach to automate error detection.  For dataset with synthetic label flips added, these errors were identified with an accuracy of up to 85%, while requiring up to 93% less computing resources to complete compared to a previous model consensus approach developed previously. The resulting trained AI models exhibited greater training stability and up to a 45% improvement in accuracy, from 69% to over 99% compared to the consensus approach, at least 10% improvement on using noise-robust loss functions in a binary classification problem, and a 51% improvement for multi-class classification. These results indicate that practical, automated a priori detection of errors in medical data is possible, without human oversight.

About Presagen

Presagen is an AI healthcare company that is changing the way clinics, patients, and medical data from around the world are connected through AI. Its platform, The Social Network for Healthcare, connects clinics and patients globally, and enables collaboration and data sharing to create scalable AI healthcare products that are affordable and accessible for all. The decentralized network democratizes the creation of AI products, promotes collaboration through incentives, and protects data privacy and ownership. With a focus on improving Women’s Health outcomes globally, Presagen’s first product, Life Whisperer, is being used by IVF clinics globally to improve pregnancy outcomes for couples struggling with fertility. With a vision of creating the largest network of clinics, patients, and medical data from around the world, Presagen is driving the future of AI Enhanced Healthcare.