About This Episode

Episode two of our special look at Translating Science Into Medicine.  Analytics expert, Dr. Len Usvyat talks about his team’s work in using advanced analytics to support clinical decision-making.  His team is at the forefront of turning the aspirations of AI into applied medicine.  By leveraging big data to accelerate AI solutions for kidney disease, researchers, data scientists, and practitioners are developing clinical tools that enhance patient care.

 

Video player requires JavaScript enabled. You can watch this video here: https://youtu.be/sQaBep6DLYY


 


Featured Guest: Len Usvyat, PhD

Len Usvyat chairs Fresenius Medical Care’s Advanced Analytics Steering Committee and works closely with the global MONitoring Dialysis Outcomes (MONDO) initiative, an international consortium of dialysis providers. His team provides analytical support and functions in the liaison capacity with Fresenius’s integrated care assets, such as its pharmacy, vascular care centers, urgent care facilities, and the Fresenius health plan. Len has published over 70 manuscripts in peer-reviewed journals. He holds a master’s degree from the University of Pennsylvania and a doctorate from the University of Maastricht in Netherlands.

 


 

Listen to this episode
 

Episode Transcript:

Artificial Intelligence and advanced analytics are helping make advances in the treatment of chronic kidney disease care and across healthcare at large. Analytics expert, Dr. Len Usvyat, is at the forefront of turning the aspirations of AI into applied medicine. In part two of our special look at Translating Science Into Medicine, Dr. Usvyat talks about his team’s work in using advanced analytics to support clinical decision-making. By leveraging big data to accelerate AI solutions for kidney disease, researchers, data scientists, and practitioners are developing clinical tools that enhance patient care.

Dr. Maddux
So Len thanks for joining me. You lead the Applied Advanced Analytics Group in our global medical office. You want to describe a little bit about what that group does and what your all’s purpose is?

Dr. Usvyat
Yeah, absolutely. So first of all, thanks for having me here and Applied Advanced Analytics team is a, it's a group that really at the end of the day, our team is responsible for looking at data and gaining insight, out of the data that we receive and not just getting the insight out of the data that we get but also translating it into clinical practice, translating it into publication and research activities that are associated with the work that we're doing and we have a very diverse set of skill sets that we that we utilize to gain that insight out of the data that we have.

Dr. Maddux
So, one of the things that I've talked a lot with you about over the years is that the work that we do needs to be applied. It needs to have high utility in the care of patients and describe a little bit about the skills and the capabilities that the team has. What kind of people are you looking for to, to do this kind of work?

Dr. Usvyat
Yeah I think one of the essential elements to the work that we do as you mentioned is turning it into applied practice. I think at least certainly from our perspective just creating these predictive models or creating other image recognition algorithms or other things that we may be doing is not enough for us. It's really important to say how we're going to apply this in real clinical practice and so to do that we don't just need to have people who are data scientists per say, or people who can pull out data and do data analysis. I think it's important for us that we have folks who can also understand clinical practice and if we're applying something to a dialysis clinic setting then also understanding how clinicians will utilize this in day to day operation if we're working with our vascular care centers, it's also understanding how vascular care centers, would apply this in real clinical practice so I think it's it's truly is in my mind, very applied data sciences, I would call it.

Dr. Maddux
Yeah. So, I know, and that you have a diverse background and how you got to the role that you have. Talk about the kind of background that you or others in the team have that might not naturally be thought of as being pertinent in kidney disease, analysis, but certainly has turned out that way. What are some of the backgrounds of some of the folks on your team?

Dr. Usvyat 
Yeah, our team is you know, and even though we're very, relatively speaking a very small team it's extraordinarily diverse. We have folks who have background in IT, we have folks who have background in statistics, we have folks who have background in pharmacology. We have people who have done work in data engineering and neuroscience, for example, and so it's a, it's a very very diverse skill set and I think we, I don't know if that was necessarily done on purpose but I think it, I guess partially it was on purpose but we really wanted to make sure that we have these very diverse skill sets so that folks are A: there's different opinions that, that people can offer when we create these models and then B: I think when we apply them to clinical practice that they're not just laser focus to the very traditional thinking of how dialysis clinics have functioned, you know, for the last 50, 60, 70 years so I think it's it's really important to me I think this diverse set of skill sets is, is, is very very critical.

Dr. Maddux
Yeah. Before we talk about some of the details of AI and some of various offshoots of artificial intelligence, talk a little bit about what you see the mission as of your group with regard to our global medical office and working with the many other people around the company that both interact with data and are also creating insights, can you just talk about that some?

Dr. Usvyat
Absolutely. I mean, I think, as we all know it's, Fresenius is a very large, diverse company and there's different structures of where these analytical teams, physically located or where they report into different organizational structures, so one of the key things for us, when the global medical office was founded, is, is really to create a structure where in some cases would actually be providing assistance to some of the regions with our with true maybe even data analytics services. In some cases just providing some more guidance that we have developed through knowing how to apply these models and in real clinical practice. In other cases, it's really what I would consider this bridge building activities, or I think as you refer to them, which I think really very well portrays what we're trying to do. We try to create this team, where in some cases we truly do the work needed to support these activities and in some cases it's doing more of this bridge building activities, understanding what other folks are doing, Making sure that we may be sharing some processes that are the same across the globe, without necessarily saying we're going to be actually taking on the role and let's say applying some of the models that we have developed in other parts of the world.

Dr. Maddux
We have as one of our core values as a company collaboration, and I think that the advanced analytics groups around whether they’re in GRD, whether they're in the operational teams in various parts of the world, whether they're in the medical office have shown a tremendous amount of collaboration, not just opportunity, but actually collaboration and practice, and I think it's, it certainly is one of the areas that I think is most important for us to sort of be the example and help demonstrate that between your group, and other groups that are similarly positioned in the company. Let's shift gears for a second and can you describe for the folks that are listening and watching this today a little bit about what's the distinction between artificial intelligence, which we hear about as a buzzword all the time, but AI versus machine learning versus deep learning? What are these disciplines and how do they distinguish themselves?

Dr. Usvyat
Sure, so I mean I think AI and as you said it is a buzzword it's been it's been now around for a while I think was actually first turned in I believe early 1950s, but at the end of the day, AI is really more of this umbrella term and what it, what it's trying to say is it's trying to basically have computers mimic human behavior so at the end of the day it's computers trying to mimic what a human mind would actually be doing and so how does this actually work? Well, AI is the umbrella term so it may actually include devices that have some sort of AI like capabilities where there's some component of this human like thinking going on in the computer of the AI product for example. Machine learning is a subcomponent and that's the one that we actually do most of at least in this organization and to be honest in most other companies. That's really taking data and applying traditional or less traditional statistical methods and other mathematical methods to process that data, and to be able to gain some insight, out of the data that we have. It is not just the most kind of the most traditional linear regressions of logistic regressions, neural networks, Xg boost methods, a variety of other techniques that can be applied to data. And that's what's called machine learning and then a deep learning is really the sub component is kind of the smallest piece of this big AI umbrella, but it's probably the one that you know most people get particularly excited about because that's where you need some super heavy duty processing to be able to process the data that we have and so typically when people think of deep learning people think of image processing or speech recognition where you need much more complicated neural networks, for example, to be able to process them, and you know so as we all know now I think you know computers have a fairly good ability to recognize images and to differentiate between, you know what, what the image may be showing and I think that's a, that's something that you need some as I say more heavy duty computing processing power to be able to process it.

Dr. Maddux
So that brings up the question of why now why is this popular now, when the term AI has been around since, you know, 60 years ago 70 years ago, and I just would be interested in what's changed in our ability to have computing power at hand, or have the techniques evolved to make them simpler or more accessible. I mean, what do you think the reason is that AI is such a popular dimension right now and looking at how we gain intelligence and insight?

Dr. Usvyat
So I can think of at least three things that came together and I think we're you know we're, we are living at a time where we have this confluence of these three different things and so the first one I would certainly say is data. AI by definition, requires lots and lots of data to be able to learn on something just like the human brain needs precedent, to be able to know how to deal with a situation before the same for computers and AI and machine learning and deep learning is you have to have precedents so that means that you have to have a lot of this data and I think as we all know, we're now generating more data exponentially by day than we did in the past and so we're collecting lots and lots more data. So that to me is number one number. Two is the fact that the computing power has become exponentially more powerful. I think we're able to do processing of computers, processing of information so much faster than we were ever able to do that before you know hard drive space has become very inexpensive and then of course, all these statistical methods that have actually existed, many of them have existed for a very long time. Now, in the past that might have taken you a month to be able to run one simple algorithm and now you can run it within minutes sometimes because of this processing power so I think these three different things coming together has really resulted in my mind I think in why AI has become such a hot topic right now.

Dr. Maddux
What are the opportunities to move this into real time? We've in the past done retrospective analytics to look at what performance was on, you know, as an example. Now we're moving closer and closer to a period where real time analytics become sort of a necessary imperative to get the most out of whatever the tool that's being developed or whatever the application is. What's your sense of how quickly Are we going to be at a point in time where the expectation is that real time analytics that will influence the decision in the next minute, for example?

Dr. Usvyat
Yeah, and I think that to me I think that's absolutely crucial. I mean I think we, certainly our development of these machine learning algorithms has been a very much of an evolutionary process. We started off with these predictive models that, you know, might have been distributed only monthly or quarterly or even once every six months and I think we now have moved very closely and in some cases I would even argue, we have gotten to a point where we are able to do some real time analytics so we have done some work with Quickline for example and working with our IT organization and working with RRI where we're able to provide some feedback real time. in fact, on what's happening with the patient based on the information that we're receiving real time from Quickline devices. So the question becomes, now we can do this in two, three clinics. The question is how do you do this in all 24, 25 hundred clinics, but then also the question is what do what you do with that once you have that information. I think this is where that loop back to our applied concept comes in. It's not enough just to provide prediction, even if it's real time, you need to be able to bring that back into the workflow of our clinicians and make sure that they know what they would be doing with this information and then also long term monitoring how they're using this information and making tweaks to the product that we're developing.

Dr. Maddux
So let's step back for just a second and talk about data sources for a minute. What are some of the unusual data sources that we might not traditionally think of in our healthcare world where we're generally thinking about the typical table one graphics of demographics of the patient and comorbidities and clinical elements that we might typically measure in a treatment, but what are some of the things that actually do you see as highly useful data sources that people might not be thinking about as critically important?

Dr. Usvyat
Yeah and that's a wonderful question so of course number one is I think we would probably, most Fresenius employees would say that we are very very data rich as an organization, I think that is very true. We're very unique as probably any healthcare segment where we are collecting so much information about our patients so you could argue, just looking at it, why do we need any additional data? And I think this is maybe this was the thinking maybe a few years ago where we'd say well why would we apply, look at other data sources, but I think we've realized that there's other data that is so critical to us that we should be leveraging and so for example and I think the list is pretty long we've, we've tried all kinds of things over the last several years, but I can certainly tell you the most basic things such as weather you know is air pollution for example, the traffic patterns, the walkability scores in a given neighborhood, things that we may be able to get from Twitter especially if they're  geocoded things where we're able to understand maybe what folks are tweeting about. So things like happiness indices for different zip codes around the country, in those little things actually provide an important picture about where the patients are being treated. It's not necessarily just the traditional you know albumin and blood pressure and things that we are used to, it's more about where the patients are geographically or in terms of their family and other determinants of the social determinants associated with them.

Dr. Maddux
So one of the, one of the data sources that you and I talked about several years ago, you've now created almost as a name data source is the acoustic fingerprint, you want to just describe that as an unusual way of looking at our environment?

Dr. Usvyat
Sure, absolutely. In the environment we're in physically and any space that we may be in has a very unique signature and that is a sound signature of a place and so any environment would have a certain unique, what we termed as acoustic signature essentially of a place, and so the question becomes how do you actually numerically collect that data, how do you actually measure what the sound ambience, what does that look like, how do you translate  into numbers and also how do you do this in a way that cannot be deciphered to anybody's conversation or to anything that can be tracked back to a patient or to an employee and so what we, what we've been able to do is create this device that we call an acoustic sensor but what it does is it can sit in any environment for example a dialysis clinic, it will collect information on what's happening in the dialysis clinic without taking any kind of voices or any kind of conversations and immediately translating that sound wave coming out of the clinic all the different sound waves coming out of the clinic into a numeric wave and then the question is, can we take that data and translate it into some new insight about what may be happening in a given dialysis clinic and so we have shown some initial data demonstrating that yes there's something about the clinic, there's something about the clinic itself, the sound of the clinic that may be predictive of what may be happening in the clinic. I can also mention one thing that you know with COVID and with masking for example there's something there may be something unique about the audio signature of a clinic once we, you know, required that our employees and our patients everybody's masked who is in the clinic so when we started doing this in the beginning of March, they can see different things happen,

Dr. Maddux
And I recall that some discussion within our engineering group and our research and development, engineers thought that maybe the noise of our machines might actually also have a signature that might predict potential, heart failure or other things that might require maintenance and so a lot of these data sources don't necessarily lead to something that we prospectively know what the value of that is, and I think with much of the advanced analytics group it's quite creative and looking at what can we learn from those insights and it’s sort of interesting.

Dr. Usvyat
No, absolutely. And I think I mean there's nothing wrong with failing certainly so I think part of it is actually trying and sometimes it doesn't work and sometimes it does work and I think as you mentioned, the acoustic fingerprinting actually there's so many other uses of data and as you say, you know, figuring out when you know certain PD cyclers for example they may have some sort of maintenance problem, you have a might if you have this sensor inside the machine I mean that may really be helpful, and I understand I think our GRD team is one of those things that they're working on is trying to figure out if they can install this as part of the machine.

Dr. Maddux
So, shifting gears, just a little bit. I've heard you describe that the development of a predictive model, and its ultimate evolution to where it gets into use, and has high utility mimics in many cases the phased process that a new drug comes to market with. You want to describe that a little bit?

Dr. Usvyat
Absolutely, I mean I think we, you know, we, since we started doing this quite a long time ago, we realized that one of the things that happens often in a company, and a company of our size is, you know, people get very excited about new good things coming to our patients or to our clinics and I think there's often a desire to say that we should if we have a predictive model we should roll it out to everybody immediately, you know we shouldn't just have a small pilot where, where we can test how the model is performing but actually roll it out to everybody and so we realized while there's nothing necessarily wrong with that, that it would make a little bit more sense to go through some sort of a phased approach to the model development, where we try to learn different things in different phases and so one of our team members actually said “well maybe we should be utilizing this drug development process as the analogy.” Now of course, I will say immediately that we don't think of predictive models as some sort of a research or like a real drug development because you know this is all like QI activities that were involved in. But nonetheless, the analogy of these phases made a lot of sense to us and so we now essentially say that our, our predictive modeling development, generally speaking, should follow this four phased approach where in the first phases it's more about understanding the need, what is the actual clinical need for example that we're trying to answer and maybe the second phase of our rollout is more about can we have a very small number of people who would be utilizing this model and let's learn, let's work with those clinicians very carefully, participate in their daily discussions, and really understand how they're using this model that we have developed for them. And then as we move down to these phases you know phase two phase three may become more about rolling it out to a larger audience where we actually have some P values and we can really demonstrate that our predictive model made a statistically significant difference. And then for us I think phase four becomes you know you just roll that out to the whole organization because by then, we know that this has actually made a difference on patient outcomes so we like that analogy. It doesn't mean that we always utilize it, it doesn't mean it always happens that way because again, I think there's, you know, in an organization that really does care about I think clinical outcomes we often get really excited and we want to roll it out to everybody, immediately but I think we tried to sometimes say well let's actually make sure that statistically this made sense for example or that clinicians really will know exactly what to do with this particular model. So that's why we like that analogy.

Dr. Maddux
So you and I have talked a lot about single purpose use of some of these and that I think one of our challenges in the process has been creating that environment where a model is available to multiple systems or multiple geographies or different types of people in the clinical experience. How do you think the maturity of the process is developing with regard to being able to effectively create a product around a given model that can be used by different parts of the organization that might have an interest in the same question?

Dr. Usvyat
Yeah, no I mean that's ta great question. We, you know, back in the day again and I'll give a little bit of an evolution of course back in the day, the way we would run these predictive models is somebody in my team or somebody in RRI would physically have to get you know would have to run some code to download the data have run another code to process the data you know run another code to actually get the result of the predictive model well of course that's not a scalable solution in any shape or form. So what we've been able to do is that what we do at this point is pretty much all of our models, almost all of them are, they are run on a routine basis, nobody physically runs them, none of our data scientists or our data engineers actually run these predictive models. We have worked very closely with our IT organization. This collaboration has been really key, where we're able to populate a certain data source with all the results of the predictive model so this score, whatever this predictive model says, basically becomes an additional variable associated with a patient and can be queried just like any other albumin or hemoglobin value because it's in that in the database and it's populated on a daily or weekly or monthly basis. We also have had other cases where for example we worked with acumen group to build a predictive model for CKD progression and there we just provided them with a code, we don't necessarily have access to Acumen’s data but we've been able to work with them to just give them the code that they're that they can run. And then there are other cases where it's simply an API where you know you're, if you need if you don't need a certain result frequently, for example, and if the computation can happen quickly enough, you just have the whatever the solution that that the EMR that may be utilized it can call an API.

Dr. Maddux
Len, tell us what an API is.

Dr. Usvyat
API is Application Programming interface.  It allows for the ability for applications to pick up certain information that needs to be communicated back into the tool the clinician will be using.  In our case it would be saying, I be a clinician saying I want the results of the predictive model, an API would go out and say great, I need to get all this information, I need to get these 200 variables, I need to do something with them, and I need to compute the probability that a patient will let's say go to the hospital and it will bring it back with just that probability on a, on a tray. And that's an API and that should be happening, pretty much real time because I don't think anybody wants to wait for a probability for you know for two hours to show up in the computer so this is why, this is why, in some cases API is very useful and in other cases, as I said, we just populate we just compute all these probabilities routinely we store them in a database because it just becomes another data element that clinicians can use.

Dr. Maddux
So, as you're developing these models and we're deploying more of them, the question will always arise, to what degree will the regulatory environments around the world require regulatory approval for a model, where is the state of that today and what's happening there?

Dr. Usvyat
Yeah, so we've worked very closely with our regulatory folks and we're also working very closely with our colleagues, globally, because I think this is where this global medical office has become extraordinarily useful, because there are colleagues in China, for example, who are in Geordie who are going to some very similar discussions. Our European colleagues have actually went through a regulatory process for, certainly for their anemia algorithm and I think there's some other ones that they're trying to go through the regulatory process to actually get some of these things approved for these devices. When we spoke to our folks here in North America, as long as our models are being utilized internally. And as long as they're the types of models that the clinician still has the ability to override anything that the computer or any recommendation that the computer is doing, you know, we've decided that for now we won't necessarily put them through regulatory pathways, but I think that option is always open for us if we wanted to do that. We're certainly have the capability to be able to put those through regulatory pathways.

Dr. Maddux
We've seen some other organizations commercialize some of their models in one fashion or another, not strictly in the scientific community but as a commercial enterprise. We've done a little bit of that certainly with some of the European machine learning activities. Where do you think the opportunity is? Is that a business opportunity for the company?

Dr. Usvyat
I mean I think for it from my perspective, I do think there is real value in it and I think the value comes not just from the fact that we can, we have number one we have the data, which I think in that case it makes a little bit more unique to dialysis organizations. Number two is I think we have the skill set certainly to be able to build these models, and I think what really is critical in our, in the teams that we have globally that are involved in these activities is really the ability to apply them, and I think this is not anything that is small, there's many of them, there's small data science shops that are you know there's hundreds of them out there. And many of them as you suggest they you know they tried to commercialize these products but what I think most people don't have is the ability and the skill set to really see how to apply it in real practice, and especially if it's a dialysis organization to me it's a no brainer that I think we have more experience than anybody else out there in this field.

Dr. Maddux
So I know that it's never politically correct to ask people which child they think is their favorite, but which model is your favorite?

Dr. Usvyat
That's a tough one. You know I would certainly say this, what we call IHPM it goes by this real funny name I think it was, it opened a lot of-

Dr. Maddux
what's that stand for?

Dr. Usvyat
I should  describe what that stands for, it stands for Imminent Hospitalization Predictive Model, it was built by Andy Lon who was a data scientist in our group who has since moved on to live on the west coast. But I think one of the reasons I thought it was a great model is first of all, everybody in the team was really involved in building that model but also involved in understanding how that model is functioning. And what we've done when the model was put into use I think it was developed in an agile manner so it was using all the different agile principles and we worked so closely with the clinicians, I mean literally Andy as well as others have spent time on the phone, sometimes an entire day with a clinician seeing how they're utilizing this model so I think that model truly turned this applied name that we have into real real practice and it, I can't imagine getting more applied than that. I certainly think it was, it was a real eye opener to me in many many areas and that's why it kind of, you know, I think it was, I think it was a very interesting model.

Dr. Maddux
So we've been on, in my career, the most rapid learning curve in the last six months with the COVID-19 pandemic. Tell me a little bit about what your team's been working on with regard to COVID-19, and some of the work and trying to understand how we can respond better to the pandemic and the ongoing effects of what began as in early 2020 and is now extending into either extended first phase second phase third phase, whatever it might be.

Dr. Usvyat
I think when it comes to COVID-19 I would certainly say, and I may have mentioned this before but I think we're in a very unique position, you know, Fresenius has a lot of data on COVID positive patients, and one of the unique things about Fresenius is that we actually have seen these patients before they even had any symptoms. And we've seen them quite routinely for days and weeks and months before they actually developed any symptoms for COVID. And so that puts us in this very unique position to say what is actually different about these patients, you know, when they developed symptoms versus a month ago that is a very very unique opportunity that I can't imagine too many other healthcare industries, would be able to say. Our nurses documenting chairside what some of the symptoms that the patients may be having and again, comparing that when they have symptoms versus two months ago I think becomes very useful so Caitlin Monahan who is our senior data scientist has built a number of predictive models and there's been many iterations, but it's really to say, Well, can we predict whether the patient will have COVID-19 in the next two or three days, we don't have a long window for this you know from the time the patient actually picks up the virus to when they develop symptoms so we're trying to have enough data on what may be changing in the patient to when we can actually, when we can identify disease before the patients even develop symptoms because once they develop symptoms. You know it's less, there's not much more than the predictive model can necessarily do so we have done quite a bit of work in that area at this point.

Dr. Maddux
And others in the extended organization I know have done quite a bit of work in understanding the impact of pool testing and certain things. So I think there are a number of these sort of analytical mathematical research areas related to COVID, independent of the pure epidemiology of which we have learned quite a bit.

Dr. Usvyat
Absolutely, I mean I, you know, I know that RRI, They were doing testing antibody testing on a lot of their patients and so one of the questions becomes as if we want to do antibody testing for example, can we utilize the predictive model to kind of change that pretest probability, because I think you would know a little bit more as to who you want to potentially do a test on so there's a lot of other applications of this type of model.

Dr. Maddux
So going back to the use of artificial intelligence in true clinical care. Where do you see some of the risks that exists in this field which is relatively, although the terms have been around a while, I think it's been a relatively recent advance to where it's become much more routine to use these things. What are the risks?

Dr. Usvyat
Well I mean I think you know of course most people somehow have this fear of Terminator like, you know, situations when they think of artificial intelligence and applying artificial intelligence in clinical practice. I will say one of the trickier things is you know medicine has been around for thousands of years you know clinicians who went to medical school nursing schools and other schools and, you know, the usual approach is not to say I'm going to rely on some black-box like algorithm, they're more likely to apply their knowledge that they have gained. And so to me I think one of the things that we always when we work with our clinicians is really to, to tell them that I think this model is not supposed to take anything away from the but potentially highlight things that they may otherwise may not have time to necessarily see you know we don't have the ability, I think we all know we don't have the ability to process 10,000 variables in our head and say oh this patient actually needs something and this is to me whether computers become helpful, and we very often use this analogy of pilot, and you know the airplanes and I like it so I still use it. To date, as we all know, I think the pilots, or the computer, pretty much does a lot of the flying. But I think probably none of us would want to get in an airplane if there wasn't a pilot in the cockpit and so to me I think it's a very similar analogy to healthcare that I think physicians will be there, were there, and will be there. Even if we use this data to really help them actually optimize their therapy and actually have them pay attention to things that are a lot more critical as opposed to the more mundane things and so this has been shown with some of the algorithm development work that has happened in the last five or 10 years.

Dr. Maddux
Well certainly advanced analytics has taken on a substantial life that has allowed us to think about how we actually benefit from what we've learned from taking care of so many patients for so many years. Any final thoughts and comments that you would want to share with our Dialogues audience on this aspect of what we do as a vertically integrated company and any sense of what you see the vision being in the future for the possibility of this?

Dr. Usvyat
Yeah I think I would say to anybody watching this I would certainly say that I think a few here of some of these approaches and things that we're trying to do in the clinics I would be number one is I would personally want to hear about the things that folks think that we need to improve because as I said, it is part of the process is making sure that we are making the supply, and so I think getting this feedback can also be very open minded about the types of activities that we're trying to do, and providing as I said, providing all kinds of feedback that folks are seeing of how these things are being applied, how these things potentially can be applied better because the data and the analytics behind it is really just the engine but I think we need that clinical input to be able to apply them in a way that's intelligent and smart and will improve our patient outcomes and make it easier for physicians and clinical staff to be able to do the type of work that they're doing.

Dr. Maddux
Well, good. Len,  thanks for talking to me today about artificial intelligence and advanced analytics.

Dr. Usvyat
Thanks so much.