Episode 48: Using Artificial Intelligence As A Tool: Part 2

In this episode, Dr. Michael Kraus, Associate Chief Medical Officer of Fresenius Kidney Care, continues our series on Artificial Intelligence. Dr. Kraus once again chats with experts from our Global Medical Office Dr. Luca Neri, Senior Director and Data Science Lead, and Dr. Caitlin Monaghan, Data Science Manager, about the growing abilities of artificial intelligence and how it’s impacting medical technology and clinical practice. You’ll also learn about how machine learning is being used in our field of nephrology and how Fresenius Medical Care is using artificial intelligence to improve patient outcomes.

Dr. Michael Kraus: Welcome, everyone, to this episode of Field Notes. I'm Dr. Michael Kraus, the Associate Chief Medical Officer at Fresenius Kidney Care, and your host for this discussion today. Here we interview the experts, physicians and caregivers who bring experience, compassion, and insight to the work we do every day. 

Today, we're getting back to our ongoing discussion about artificial intelligence and machine learning. Be sure to go back and listen to Part one of the series entitled “Using Artificial Intelligence as a Tool” with Dr. Len Usvyat and Dr. Luca Neri from our Fresenius Medical Care's Global Medical Office. In this episode today, we'll be diving into the growing abilities of A.I. and how it's impacting medical technology in clinical practice. And since we're a company focused on kidney care, we'll take a look specifically at how machine learning is being used in our field of nephrology today.

I'm very happy to have some experts from our global medical office on board today to talk about this topic with us. I'd like to welcome back Dr. Luca Neri, Senior Director and Data Science Lead. And I'd like to introduce Caitlin Managhan, Data Science Manager. Luca, Caitlin, thanks for being here today.

Dr. Luca Neri: Thank you for having us, Mike.

Dr. Caitlin Monaghan: Thank you for having me.

Dr. Michael Kraus: If you're listening to mainstream media, you hear about all the concerns about AI. We're going to focus on the benefits of A.I. and how it improves care of our patients, not only tomorrow, but even today. So, Luca, let's start with the recap on how we use A.I.in medicine. In what capacity has A.I. been around and how long have you been using it?

Dr. Luca Neri: Well, my A.I.is used in many, many ways in medicine, even though the applications that are really in the market and the clinics are not so many. But let's try to have an overview of the many ways we can use it. 

So we can use it to predict the outcomes or complications of the patients by using their clinical data and also center’s data to understand the health status and try to project in the future how they will be and how the health, well, let's say progress for this patient. But also we can use it to suggest treatments, to suggest the dosage of drugs for specific diseases, or even to select them on different drugs, to understand the best clinical pathway for the patient. We can also use to interpret imaging or diagnostic test and to help physicians have a quicker and more holistic view of the diagnostic process. We can use it also to understand which clinical pathway may bring the best care for the patients when we look at clinic level rather than a patient level. So having an advanced benchmarking tool that help us compare a clinic to clinic practice, to practice, even physician by physician in a way that all physicians recognize when they do well so that their good clinical practice can be taught to other physicians and to other clinical practices.

Dr. Michael Kraus: It goes from community based practice—looking at the entirety of a community, could be a clinic, or even higher, all the way down, obviously to the bedside and how to prescribe medicine. Thanks, Luca, that's tremendous. Caitlin, how are we using machine learning to improve care today?

Dr. Caitlin Monaghan: We're using it in a lot of different ways. So, many of the ways Luca touched upon, and we really try to implement it or think about it at different aspects of a patient's journey as well. So, whether it's predicting kidney function decline or progression through CKD and ESRD. We use it to identify outcomes that maybe we want to prevent, so things like hospitalizations or leaving a modality. 

We also use it to identify good events too. So, it's not always just looking at doom and gloom, but identifying patients who might benefit from being on different modalities. So, treating at home, in particular, trying to make sure that we are giving everyone the opportunity to go home. So, there are many, many ways that we're using it, and we're always working with the end users, we're working with clinicians and others in the field to identify ways that we can help them by taking advantage of these advanced techniques that we know how to use, making sure that we are helping the patients and helping provide the best care that we can.

Dr. Michael Kraus: Luca, we don't live in the dark ages. I mean, we've been doing bio stats and epidemiology and we think about how to take care of patients in multiple ways, and we have for a long time. What's the difference about A.I. and why are we using it as a tool today?

Dr. Luca Neri: Well, A.I. is a very broad term and umbrella term. And some of the traditional statistics, like logistic regression, for example, can be used to create models, predictive models. And of course, there are specific techniques like machine learning or let's say, generative A.I. that we are listening about a lot in these days that are a bit different from traditional statistics. But at the end, both traditional statistics and these new machine learning techniques aim at detecting patterns in the data that repeat and by detecting this pattern, we can, let's say, learn from the past to project in the future and do simulations. Artificial intelligence now does it in volumes of data that were not possible several years ago. So now we can elaborate and crunch such a volume of data that the insight that we can get from these data are much more deep than what we could do before with simpler statistical models.

Dr. Michael Kraus: Caitlin, you've taught me this. We're able to look at a wide amount of data and look for patterns that my brain certainly is not good enough to pick, and that's useful. So, let's focus in on some things that you're really excited about and things you've been working on. Caitlin, we'll start with you. Let's focus on what you've done to predict patients on peritoneal dialysis who are at risk of dropping off therapy.

Dr. Caitlin Monaghan: This is one of my favorite projects and I think it has the potential to have a huge impact on improving the patients’ lives and the care that we give them. Essentially, we wanted to be able to identify patients who look like they're going to be leaving peritoneal dialysis. And it's actually two models that we display together to look at different time horizons.

So, look at the likelihood of a patient leaving PD in the short term, so the next 1 to 3 months. But also we wanted to make sure that we could kind of forecast out a little bit farther to identify patients who are leaving in the longer term, so 3 to 6 months, in order to identify patients who are beginning to experience technique decline. 

And the hope of doing this is to really get at two things. So, one, be able to prevent patients from leaving PD, if at all possible. So maybe they just need a change in their prescription because their kidney function has declined more. Maybe they need a change in their medications or other interventions and just getting ahead of that before they hit kind of the point of no return of being too sick to continue on. PD can help prolong their time there and keep them healthy on PD, out of the hospital and prolong that treatment modality. 

That being said, oftentimes PD is not forever. It's one step on a patient's dialysis journey, and we want to help identify when maybe a patient is beginning to start to experience technique decline so that you can prepare the patient both physically, by getting access placed, and also mentally, making sure that they're educated and can make informed decisions about what's next on their journey and getting ahead of that, so it's not just an urgent need or crashing back into in center. They're in the hospital, they're really sick, but really making sure that we're taking the best care of the patient by getting ahead of this and preparing them for the next step on their journey.

Dr. Michael Kraus: I've been practicing nephrology for almost 40 years now, and I can't tell you the number of times somebody said if only we had a way to predict who's going to drop in PD because that is a big problem. As you said, you need to get access, you need to be educated on your next journey in your life, and prediction is important. How good is your tool?

Dr. Caitlin Monaghan: It's good. I'm a lot more look at the numbers like it could always be better and you're the good balance when we're presenting the same, you know, this is actually a good tool. We are able to identify these patients with, you know, a high degree of certainty, especially in the short term. So, one example of the performance is if our model says a patient is at high risk of dropping in the short term, one in two patients drop. So, half of our patients at that risk level drop. So, it really is a good indicator of patients who are high risk and really do need additional resources directed towards them.

Dr. Michael Kraus: And if you tell me they're low risk so I don't have to worry about them, what percentage of them drop in the next three months?

Dr. Caitlin Monaghan: It's low. It's a lot lower than you'd expect. I should have pulled up the numbers.

Dr. Michael Kraus: I think you've told me it's one in seven or 7%. It's a low number. That's correct. It's quite impressive, the differentiation between the two. It's six or eightfold difference. So, it allows the physicians to focus exactly on what the issues are. If you tell me the high risk that drop for the next six months, one in four of them drop, which is absolutely incredible and much better than the one in six that would drop if you tell me that they're low risk. So, I think it's useful. 

Let's move to Luca. You're excited about the AV fistula failure risk model, which is for our audience, the fistula is the hemodialysis access where the artery and the vein are tied together and you cannulate and that has problems down the road and it's expensive and worse yet, it's morbidity for our patients, so understanding when that fails is critical. Tell me about your model.

Dr. Luca Neri: So, it's a predictive model that aims at predicting whether the AVF would fail within three months, where failure is switched to another vascular access, thrombosis or any procedure that is done to reestablish the patency of the fistula. The model is quite good, the accuracy is around 80% and enables stratifying patients in four classes. So, if you go from the very low risk patients, you have around 40% of the sample with a failure rate within three months below 2% and then go down to the risk classification, you have moderate risk and high risk and the high risk class, you have 20% of the sample with 20% of risk of failure.

And the very high risk you have a tiny group of patients that are really, really very high risk patients with, say, 66 to 75% of risk of failure within three months. 

And the basic idea that that we add in in building the model is that we wanted to have a technical surveillance method that was completely free of effort for the physician so the physician can see the results without doing any additional action on the patient. And that is meant to help the physician in diagnostic process. So, if we describe our physician does a diagnosis of it first, when the physician is in front, the patient, the physician try to recall, let's say, the event, a rate that he would expect in that population. It is an automatic process that the physician doesn't think. He uses memory to recall the cases and he has embedded a pattern of events. And then the physician had additional diagnostic testing to corroborate the hypothesis he starts with. We wanted to help the physician with the initial step because that is something that machines do better than humans is recalling. And so by using the ability to extract automatically information to the medical chart, complex information, the model uses more than 30 parameters and then combines them.

Then we can provide a pretest probability to the physician that will inform the diagnostic process later on. Now, we are working also on other, let's say, companion project, because this one is one brick of the bigger house that we want to build for physicians and patients to manage the AVF. We are trying to do another model that would allow the detection of stenosis by analyzing the sound of the AVF through an electronics data scope. We have promising results with the first bunch of training, we have 90% sensitivity and 90% specificity for this new tool. And the next step is combining the two and see whether the two combined can help reclassify patients and do a better and more accurate diagnosis with less effort.

Dr. Michael Kraus: So, both of these tools are just that. They're tools, right? It's sometimes people think that we get an answer and then we just respond to it. So, I'll start with you, Luca. You tell me my patient’s fistula is at risk. As a physician, what do you expect me to do with that information?

Dr. Luca Neri: Essentially, with that information, you can do two things. The first thing is that for patients that are very low risk, you can reduce the number of unnecessary testing that you would do to this patient because also unnecessary test not only is inefficient, but also can cause harm to the patient or it's a discomfort to the patient. But for those that are of high risk, well, you may increase the level of attention and you may have an additional justification to do a more intense diagnostic process for this patient.

And of course, this is the judgment of the physician that integrates this information, this prognostic information. It's not reality. So, we are trying to understand from the past possible, let's say, futures for the patient. And then it is the physician that integrates these data into a knowledge, into his knowledge to create the diagnosis. And then you can differentiate the diagnostic pathway for the patients based on this baseline information, because as we know, the positive predictive value and the negative predictive value of any test depends on the sensitivity and specificity of the test, but also on the prior probability that the patient has a disease or a complication or a future event.

Dr. Michael Kraus: So, as a physician, I look at it differently. I've got lots of patients to take care of. I need to know who to focus on. You are telling me this is a risk, it then allows me to put more critical thinking, more attention to that patient, decide what the next best steps are. So, I think that's excellent. 

And that moves me back to you, Caitlin, because clearly in your model, that's the case. In fact, as I get it, you want me to make your model fail. What do I mean by that?

Dr. Caitlin Monaghan: We want you to prove the model wrong by keeping the patient on PD, by keeping them healthy, improving their overall health so their risk decreases. And what's nice about how we deliver the report is we give you the breadcrumbs as to why, you know, we don't just say someone's high risk and then leave it at that. We really try to emphasize the need for team collaboration when reviewing this. So, we tell you the reasons why, so that there is that critical thinking element that this really is part of clinical decision support. It's not—we predict the patient's going to leave PD in six months, so we're going to automatically schedule this, you know, access placement and we're going to automatically schedule this. No, we want you to, you know, prove the model wrong, keep the patient on PD when healthy. But when that's not the case, help the patient transition off as appropriate. So having the clinical interventions of critical thinking, using the knowledge that the clinicians have of their patient that models don't have is really key into making the best use of this tool.

Dr. Michael Kraus: And Luca, going back to you because you've studied the physician response a little bit. A tool is good for a couple of reasons. One, it tells me who's at risk, but even more important, it tells me who's at risk that I may not have known just by rounding on the patient, doing my usual doctor stuff. When you've used this tool and you've interacted with your physicians, what kind of responses have we gotten? What are the thoughts of the usefulness?

Dr. Luca Neri: For example, for the AVF risk score, we did a quite large usability testing in 15 clinics asking physicians to look at the risk score after the evaluation of the patient and provide their feedback first on whether they agree on the assessment of the risk score, but also whether they found it useful in the sense of reducing unnecessary procedure or detecting patients that were not detected before. And in both cases we had very, let's say, positive responses for detection of undetected cases. We had more than 60% of physicians believing that the tool may help in that direction. And even more physician stated that they believed the model would help them reduce unnecessary procedures. And also the agreement was very high. I would say that it was around the 90% of the agreement between the physician and the model.

So, these three data together make me think that, well, maybe the physician would have made the same decision if provided with enough time. But given the fact that the work in the clinic sometimes is hectic and also, let's say working at high intensity and attention for 8 hours is very difficult, then you may have the need for an assistant that keeps the information for you, crunches them, and then make you think on the case on a more focused way and more efficiently. So of course, we don't know yet whether the impressions of the physicians would really translate into real benefit. So, for doing that, we will need further studies.

Dr. Michael Kraus: And we'll have to see because I think this is the important thing: early intervention matters because later intervention with access could lead to thrombosis, infection, all sorts of morbidity for the patient. So, I think with your tool, what we'll see is we are actually making that intervention a few months earlier, which I think is beneficial for our patients.

Caitlin, also similar, I know that the nurses tell us it's a great tool and they see things they didn't see before. What are the problems you're noticing in putting out something such as this tool?

Dr. Caitlin Monaghan: There are always going to be difficulties with the implementation of any tool, even the best one out there, because there really are the two components. You need a good model. You need one that that does a good job at identifying patients who are at risk. But you also need a good intervention or good, you know, what to do with this information. And certainly that's something that we have to work with the clinicians on and users on to really identify what key interventions should be done with this information, because that is ultimately what matters is acting on the information in order to help the patient.

Dr. Michael Kraus: And that's the beauty of this. I tell you what's the problem, I'll give you tools to help you when I tell you there's an issue. But everything is dependent on critical thinking of the nurses and the physician, social workers and dietician, the whole IDT team. And that's the beauty of medicine, right? That the art. So, I think A.I. can drive us to help us spend the time appropriately. So while we’re on that, Luca, how do you think A.I. will change clinical practice in health care? And then I'm going to ask Caitlin the same question so leave her some room for answers.

Dr. Luca Neri: I will say a couple of things. I think that I can help us provide health care more equitably and efficiently because now we have a big problem in healthcare worldwide, which is the scarcity of health care professionals, long waiting list, and so access to care when the patient really need is a key component of health care. So, we may have the best drugs, but if patients do not reach medical care, well, they don't get them. So they do not have the benefit of technology. If we don't make health care more efficient and more equitable by, let's say, recognizing the real needs of the patients, bear on the health care trajectory that they have. I think that one other way that artificial intelligence would help us would be, for example, within generative A.I. to remove the burden of administrative work as much as possible from physicians and nurses. We need to use them for what they're trained for as well, taking care of the patients and not writing paperwork.

Dr. Michael Kraus: Caitlin, what are your thoughts on where we're going with this and how it can improve health care?

Dr. Caitlin Monaghan: I think similar to what Luca said, a huge part is improving the efficiency of the clinicians. You know, resources are very, very scarce and so we want to do what we can to help improve, you know, efficiency, help, you know, make the time used more carefully. So not only just, you know, removing burdensome administrative tasks that can automate, automatically fill in information, things like that, but also help pivot from being reactive to, you know, issues that are arising with the patient and constantly having to put out fires to be more proactive to prevent those from happening in the first place. And that's also a timesaver, better for patients clearly. And really is a timesaver because you're able to get ahead of problems, prevent them from being problems and having to, you know, divert resources, you know, quickly to address something by preventing it from happening in the first place.

Dr. Michael Kraus: Sufficiency, it's recognizing problems as they come up maybe a little earlier and reducing morbidity. But also, as you said, we can help with the other things that are sucking our physicians’ and nurses’ times like prior authorization. You can automate that, and it's done and taken care of and those are the things that don't make us happy, doctors, nurses, anyhow. So, I look forward to how we can change our lifestyles and improve and I think A.I. is a huge benefit. This has been great. Luca, anything else you want to add?

Dr. Luca Neri: No, I'm okay. Mike. I think that's it.

Dr. Michael Kraus: Very good, Caitlin.

Dr. Caitlin Monaghan: I just want to emphasize how much I appreciate all the work that's been done on the ground, you know, boots on the, you know, on the ground and working with the patients directly. It's something that I don't have experience doing. And so just a lot of respect and appreciation for all the patient care that's going on out there.

And I know Luca feels the same way where we really just want to help support clinicians and those caring for patients as much as possible. And so certainly we're open to ways to collaborate and build a better future.

Dr. Michael Kraus: And I think all of us working together actually is a wonderful experience and will help tomorrow. I mean, this has been an excellent discussion and something that is so relevant to the future of medical technology and frankly, the present circumstances of medical technology. A.I. is a tool that's alive and well and being used today. And it's important we stay up to date and what's happening. It's a lot of fun, actually, just like we've learned today. Luca and Caitlin, thank you for being here.

Dr. Luca Neri: Thank you for having me.

Dr. Caitlin Monaghan: Yeah, thank you for having us.

Dr. Michael Kraus: And to our audience, thank you for joining us today. If you're new to the Field Notes podcast, you can download past episodes on the Apple Store, Google Play, or wherever you download your favorite podcasts. Please remember, subscribe so you can receive the latest updates as they happen. Until next time, I'm Dr. Michael Kraus, and you've been listening to Field Notes by Fresenius Medical Care.

Take care, everyone and let's begin a better tomorrow.