Presagen Webinar Series: Knowledge and masking to address AI bias and performance

In Feb 2023 Dr Sonya Diakiw presented the webinar “Knowledge and masking to address AI bias and performance”. Presagen’s webinar series presents new technologies and challenges related to AI in healthcare, women’s health and fertility. Below is the transcript of the presentation, the webinar video, and the presentation slides.

Slide 1

Hi everyone, Thanks for joining me.

I’m Sonya Diakiw from Presagen. Today, I will be talking about how knowledge and masking can help reduce bias in AI, and help improve AI accuracy when used in practice.

Slide 2

Two significant issues with creating AI that can be reliably used in practice, and on mass, is AI bias and AI performance.

With AI bias, AI should work consistently across different types of demographics, and in healthcare, different clinical settings.

The AI should work equally as well on hardware or medical equipment used in large prestigious clinics, as well as small remote clinics.

The AI also needs to perform to a certain level of accuracy to provide clinical benefit, which for hard medical problems, can be challenging.

Slide 3

Building AI tools that are useful and work well in the real world is not always straightforward.

It is not as easy as throwing data at the AI and expecting the AI to just work.

The AI does not understand the problem you are trying to solve, or the data you are using to solve it.

The AI simply looks for patterns in the data you give it, to classify, predict or identify something of interest in the data.

For example, using AI to identify cancer in medical images.

Slide 4

Unlike what most would think, greater detail in the data can be counter-productive.

When the AI is looking for patterns in the data, and it is presented with a massive amount of detail or background noise, it can find patterns in almost anything.

This means it can find patterns that are irrelevant to the problem you are trying to solve.

Slide 5

An example of this is using AI to identify polar bears in images.

Most polar bear images contain snow in the background.

If you train AI using these images, the AI may learn that images with snow are polar bears, rather than identifying the polar bear itself.

The AI may get most of these images correct, but it is not doing the right thing and can have unintended consequences.

Slide 6

As you can see with the example, the AI may not always find what it is looking for on its own – it needs help from us.

We have knowledge about the problem and the data.

We know what is relevant and what is not, and what is just background noise.

And we can use this knowledge about the problem domain to target the AI and focus on what is important.

We can use masking to hide the background noise, in this case hide the snow in images, so the AI can focus on learning to identify a polar bear.

This helps ensure that the AI performs well and as expected.

Slide 7

So how does this help with addressing bias?

Let’s use an example of using AI to identify potatoes and carrots.

From this image, you would think it is easy to distinguish potatoes from carrots, by both their shape and color.

Slide 8

However, is it?

Some potatoes can have an orange like color similar to carrots, and carrots a lighter color similar to potatoes.

So, color in this case, if used as a variable in AI training, can confuse the AI.

Slide 9

Masking can again help here.

Masking can be used to hide the color of the vegetables in the images.

Therefore, the AI can focus on the shape of the vegetable, not the color.

The AI will more accurately identify potatoes from carrots, without any bias due to their color.

Slide 10

So how does this apply to a real-world example, using medical images?

Let's consider an example of using AI to assess images of embryos from IVF patients.

Using these images, we want to identify if the embryo is likely to be viable, which means that it is good enough quality to lead to a pregnancy for the IVF patient.

And we also want to see if the embryo is likely to be genetically normal, that is, whether it has the correct number of chromosomes.

Slide 11

We know that different parts of the embryo are important for assessing viability and genetic integrity.

For viability, the outer part or rim of the embryo is important.

For genetic integrity, the inner part is important.

This knowledge can be used to target the AI for each assessment to ensure optimal AI performance.

Slide 12

With regards to bias, images of embryos are taken from different camera systems.

Some are taken with a standard microscope-mounted camera, and others a time-lapse incubator system.

The background in these images look different.

We want to ensure that we eliminate these background effects so that the AI works on both camera systems, and is not significantly biased to one or the other.

Slide 13

We can use masking to remove the background, and to allow us to focus on either the outer layer of the embryo or inner part of the embryo.

We can then train the AI and be confident it is focusing on the right part of the image to solve the problem.

Slide 14

Other techniques can also be used to remove bias.

We can synthetically impose different colours on images, to help remove bias from color.

And we can flip and rotate the image to ensure that the orientation of the embryo is not a factor in training, because we know that the orientation is irrelevant to the problem of embryo viability and genetic integrity.

Slide 15

We can use these masking, or segmentation, techniques, and augmentations, in the AI training process for embryo images.

So in this particular example, the AI was only trained on one type of camera – a standard camera.

There were no timelapse camera images used in training.

So using these techniques, how well did the AI do on embryo images from different camera systems, even ones it was not trained on?

Slide 16

It did very well.

Clinical studies showed that the AI performed just as well on different camera systems, even those such as time-lapse cameras that it was not trained on.

Knowledge and masking in this case helped ensure reliable and consistent performance across different clinical settings.

And how was the performance?

Slide 17

Again, international clinical studies showed that the AI performed very well.

Here are a list of peer reviewed publications.

For embryo viability, studies showed that the AI can increase accuracy for pregnancy prediction by up to 25%, and reduce the number of cycles needed to achieve pregnancy by 12%.

For embryo genetics, studies showed that the probability of selecting a genetically normal embryo as the top ranked embryo was 82%, which we believe is impressive for a non-invasive and instant assessment using only images.

These publications are available on our website at Presagen.com.

Slide 18

To conclude, knowledge and masking can help improve AI performance and reduce bias, particularly for difficult medical applications.

For healthcare, this can improve scalability and reliability of the AI when used in clinical practice.

And importantly, it helps ensure that the AI is applicable to everyone, without bias.

Slide 19

I would like to thank you for joining this webinar.

This presentation and scientific publications are available on our web site at Presagen.com.

Thank you.