Data-Driven Healthcare: Transforming Cancer Care

[00:00:00] Joy Rios: Hello, and welcome to the HIT Like a Girl podcast. My name is Joy Rios, the show's host. This is a podcast where we get to talk about how we're doing. complicated the world of healthcare is. I like to liken it to a 30,000 piece puzzle. It started smaller, and when I first started, it's like a thousand piece puzzle. Nope bigger than that.

There's a lot that gets into it. And so each one of our guests, they say, brings a piece of the puzzle and helps us and our audience understand the big picture. Would you mind taking a moment to introduce yourself and tell us about your piece of the health IT puzzle?

[00:00:44] Carolyn Starrett: Wonderful. That's such a good analogy. I always say no silver bullets in healthcare, right? My name is Carolyn Starrett. I'm the CEO at a company called Flatiron Health. At Flatiron Health, our mission is to improve and extend lives by learning from the experience of every person with cancer. And what that means for us is that we build a software that's used to manage cancer patient care, clinical treatment decisions, and clinical trials at cancer centers.

And then we also have kind of the other half of our business in which we aggregate, anonymize, curate data from real world patient stories to try to think about how we can use that data for good, how we can inform development of new and novel therapies, accelerate path to market, and ensure reimbursement and access.

[00:01:31] Joy Rios: That's not a small thing at all. That's a huge thing. How long have you been at this?

[00:01:35] Carolyn Starrett: I personally am coming up on eight years at Flatiron. I joined shortly after Series B funding rounds. We were a couple years in at the time that I joined, and then I stepped into the CEO role about three years ago after our two co founders transitioned out.

[00:01:52] Joy Rios: Okay, so I mentioned before we pressed record that I used to have a client that used the Flatiron EHR and that was in the time frame of 2015 to 2018. Were you around during that time?

[00:02:03] Carolyn Starrett: I joined in early 2016 and I did a lot of work with our Cancer Center clients.

My first role at Flatiron was actually building out the customer experience and onboarding and professional services teams working with cancer centers who used our EHR.

[00:02:20] Joy Rios: Okay, and how much has the system changed since then? And what I remember at the time is being able to support folks. My job is to help them through MIPS reporting basically.

And the dashboard and the access to data and information was very sophisticated. I really liked. How everybody seemed to be connected and there was community aspect to your EHR that I didn't see in others.

[00:02:42] Carolyn Starrett: Oh, I appreciate hearing that. Yeah we've invested a lot to try to connect the sites that use the technology because I think one of the benefits we offer is that it's incredibly configurable and so as a result sites can get really strategic on how to design workflows and how to use the technology to their advantage.

On the flip side, it takes a lot of expertise. And there's a lot of kind of learning through experience. And so we've really tried to invest in connecting folks to, to make sure we can share our best practices, identify what they look like.

[00:03:12] Joy Rios: And considering the great work that you're doing, you want to get everybody access to great information as soon as possible.

[00:03:18] Carolyn Starrett: Yeah. And let's be honest, EHRs are technology that in general are not beloved to say the least. And we've worked really hard to try to continue to improve the technology, make it more intuitive, make it assist in care delivery and assist in the patient experience rather than get in the way. And yet there's that is an ever going journey.

[00:03:41] Joy Rios: I don't know that we will ever be perfect. There's no such thing in this world, but I'm curious about some of the stories that you said that you have learned from being in your time at Flatiron, so learning from cancer patients and what it is that is part of, I don't, honestly, I feel like, please tell me some of those stories that are, I don't know, driving the decisions around how, what the direction you want to go in.

[00:04:02] Carolyn Starrett: What really precipitated founding Flatiron and drives us to this day is the idea that we don't want those stories to be lost. And despite the rollout of technology and all these digital data streams that exist and all the advances, it still is very hard to make actual use of the critically important data elements that live in these different siloed systems and PDF files and get stuck, right?

So a lot of the work we do is parsing the data, extracting it, anonymizing it. Standardize and cleaning it, building in endpoints and outcomes so that we can actually pose really interesting research questions to the data to give a couple of, so you can think about the types of questions we might explore are how does a drug work in a really rare population that is too small to be picked up in a typical clinical trial or how long a might one need to be on any given treatment for it to impact overall survival?

As an example, one of our partners, Penn, just published an article in JAMA where advanced lung cancer patients who go on immunotherapy, there's no clinical guideline or evidence for how long they should be on immunotherapy. And do you want to stay on it for the rest of your life? If you're doing well if you're healthy what?

What's the impact of going off therapy? That's a question that hasn't been answered in a clinical trial, but we can look into the data sets that we curate and say, is there a difference in overall survival depending on whether you continue on this drug, which has side effects and as in is not, has real impact on patients for their need to take it. And what we found or what they found in their research was that there's no difference in overall survival if you stop after two years versus if you just keep taking it indefinitely. And so I think those are the types of questions which we can now look at and answer.

[00:05:46] Joy Rios: How involved are you in clinical trials and like how much of that is part of your world like tangentially or do you guys actually get involved in some of them?

[00:05:54] Carolyn Starrett: It's one of the biggest areas of growth and investment that we're making. We work very closely with almost half of the independent cancer clinics in the U.S. And then we work very closely with all of the large oncology drug developers.

And what we found over the last few years is that on both sides of our customer base, the sites want to do more research and want to provide more and greater access to trials for their patients. And the sponsors need to be able to tap into community sites to fill their trials and to hit diversity and representation goals.

And so we really got pulled into this from both sides. And we've tackled it from two dimensions. One, those real world data that I just talked about are very helpful in thinking about how to design a trial. How do you optimize the protocol and the inclusion exclusion criteria? What sites should you go to find patients?

And then the expertise we have in oncology workflows and technology has enabled us to help build technology to use ML to prescreen patients for those eligibility criteria and flag them in the EHR and then to manage all of the data collection required for the trial in a way that is much less burdensome to sites and hopefully to patients.

And so we're pulling all sides of flat iron together and we think that could be a really exciting vector for learning and innovation because there should be ways to both tap into patients who really struggle to access clinical trials today and do so in a way that is accessible to them and less burdensome, and at the same time just make it much easier for sites and faster and more efficient to catalyze this learning.

[00:07:21] Joy Rios: I was recently on a three day cross country trip with a friend who works in the clinical trial area. That is my area of level of expertise is three days of being in a car with somebody talking about what it's like to from her experience what it's like to implement a clinical trial and also like identifying the right clinics and the right diagnoses and the patient populations and what I gathered was it is very challenging and it is a big mess of an area and it's okay that's how much I know.

Can you help me fill in some blanks on just for people who don't understand clinical trials, their importance and what's the magic number of people that you need to be included in a clinical trial? Is it a templated type of thing? I feel like there are so many hurdles in that entire process and problems that need to be solved and I would imagine that you are right in the center of a lot of the solution.

[00:08:13] Carolyn Starrett: You talked about a 30,000 piece puzzle. I think that metaphor is very apropos and the answers shift depending on what question you're trying to answer. If you think about a new drug approval, the clinical trial is really the gold standard for how you understand safety and efficacy.

The trial needs to get powered for the size of the impact. And there are statisticians who can tell you all of the unique parameters for how you think about how many patients you need and how you design and structure the trial. But ultimately, when a trial is designed and you agree on a protocol and an optimization, you have to then figure out how to find patients who meet those criteria, how to find them at the right time in their patient journey that it's actually of interest and possible to run on a trial.

And so you think about in a cancer context, this is actually a really hard piece of the equation because once you go on a standard of care treatment, you're no longer eligible for the trial. And so you actually need to think about clinical research at the same time as you think about standard of care.

What we've seen in oncology is sometimes it actually isn't an issue of health equity that the best treatments are the right treatment for a patient might be a clinical trial, not a standard of care thing. So if you miss the opportunity to consider them at the right time, that's actually a real challenge to equity in terms of access to the best possible outcomes.

So technology can help solve for that problem. The other like mind boggling part of clinical trials is you have to collect very precise data elements in very precise ways. We have seen that often 70 percent of the data elements you might need for a clinical trial are already captured in the EHR as part of standard of care.

And yet typically those systems don't talk to one another. So you've basically are doing this duplicate data entry. They, we joke the swivel chair effect. You go from one screen to another and you just reenter data back and forth in different systems. And one system is collecting the data for the trial.

And once it's so it's just It's so wildly backwards that we don't have better options for all of the people that it takes to do this and then to validate the quality of the data and validate the data entry. And it's incredibly burdensome and that's the role we think we can help with. We have a background in data.

We've created an automated data transfer tool. We can stitch in the data that already lives as clinical care. And each one of those, as I said, no silver bullet, but at each one of these steps, there's a smarter and better way to think about the problem.

[00:10:32] Joy Rios: When you think about empowering patients, so how, in addition to identifying patients but how can we give them an opportunity to either self select to be part of a clinical trial or like to even know about them before to be able to make that choice?

[00:10:49] Carolyn Starrett: Right now, less than 5 percent of people are able to participate in trials despite the fact that they can be a really powerful treatment option.

And so I think there is a broad education problem and there's a lot of distrust in the health care system at large, distrust in clinical trials. So I think one really important ongoing vector is engaging people in their communities and helping educate around the pros and cons and potential benefits that might be available to them.

I think the other piece is actually making sure clinicians are aware of when those options might behoove, might benefit a patient of theirs and can have that conversation and just explore and that doesn't mean you have to enroll, but to not even have the option is a real miss. And broadening that understanding, making, helping clinicians to better understand what may be possible is something that our technology can help with now.

[00:11:39] Joy Rios: This is random, but not random. Are you at all involved with the cancer moonshot? Do you guys can you talk about that? And in what ways?

[00:11:47] Carolyn Starrett: Yeah, so exciting. I think to see the administration taking such an interest in this field and cancer. And I think there are really important goals that they've outlined to improve cancer mortality and to improve access to diagnostics and testing. And so I was actually able to go to the White House for the launch of the recent initiative and hear Biden announce his goals. 

And it's just exciting to see that type of attention and energy and the role it can have as a catalyst for bringing people together to try to motivate innovation and motivate collaboration more recently like we're involved in the Cancer X initiative, the offshoot, which is a collaboration across a number of different organizations involved in cancer.

And I think what these initiatives do is align public and private energy and focus. There's funding with the community through government programs, which is important for innovation when we're doing new things and it's creating a spirit of collaboration and these are not problems that get solved with one player.

This is a space that we all need to come together and rally around.

[00:12:48] Joy Rios: So what is your piece of that? Where do you guys get involved? Are there committees? Are there action forces?

[00:12:55] Carolyn Starrett: We are one of the organizations that's part of the current CancerX initiative and we are a voice at the table.

We're helping to think about pilot projects. We have a couple of different stakeholders who attend.

[00:13:08] Joy Rios: How did you get, okay, let's talk about your journey. Let's transition. How did you get into this and where did you start? Did you know in a did you know from a young age I wanna work in, work on cancer and save people's lives?

[00:13:22] Carolyn Starrett: I didn't. I grew up, my dad was a vascular surgeon and I grew up watching him in the early days of, he had a homegrown EHR that he was using for his little private practice and watched him, in a PHI non compliant way, maybe, enter charts on his home computer when he was home on the weekends.

It wasn't certified technology, but that didn't exist, so it was okay. Exactly, this was his own access database, right? So I always had that mental model back in the day. However, I spent most of my career before Flatiron working in data and enterprise technology companies. And I was always fascinated by transformative technology that, that kind of worked at the infrastructure layer.

I like really complex problems. The first company I worked at was, building internet routers and that ended up making the backbone of what we know today is the internet and all of the kind of data exchange that exists. And I had a number of different experiences in that vein, did some consulting working with healthcare and pharma organizations, and I just woke up at one point.

I had three very early stage melanomas. I have a lot of cancer in my family, and I looked at the problems we were applying these really important technologies to, and I just felt like I really wanted, I felt very called to work at the intersection of data technology and health. And followed a mentee of mine to Flatiron who'd been working, I'd been advising him and he'd come over to Flatiron from the company I was at prior to and followed him to Flatiron and it's been such an enormous privilege and an opportunity to get to work with our sites.

And our customers and partners, yeah applying my background, but to this really important field. That feels very meaningful.

[00:15:07] Joy Rios: You are all over. It's all, are you global as well?

[00:15:10] Carolyn Starrett: We are. We have now affiliate operations in the UK, Germany, and Japan.

[00:15:15] Joy Rios: Wow. That's a big responsibility.

[00:15:17] Carolyn Starrett: It is. We've tried to we've been very selective and intentional in how we've thought about international expansion.

Any given country that we enter, there's a whole. unique context that we have to think about the regulatory context, the privacy, the security, the patient engagement models, the ethics reviews. And so we had to be really thoughtful about where does, where do we think we can really make a difference? Where are the cancer patient populations that are, that we need to and can most benefit in learning from, and how do we do so in a way where we can bite off, we don't bite off more than we can chew.

So we're now probably three to four years in, we have partnerships signed and each of our three global markets, we are looking at data sets. And I think one of the things that's really unique that we think can be a very helpful value add and the way we're approaching it is that we can blend those data sets in the learning from those patient populations internationally with the larger and more established data sets in the U.S.

And so we're working with the mhm. guidance and the regulators in each market to think about like data transportability and how do we build the right evidence package for various different use cases, blending what we already know from U. S. patient populations with those country specific.

[00:16:30] Joy Rios: It would be so interesting to see the trends that, you know, from population health from each of those regions and what is affecting them health wise.

[00:16:38] Carolyn Starrett: What happens in cancer most often is actually are approved in the U. S. first. And then kind of, they roll out around the rest of the market, but for example, in Europe, one of the big questions is, are they going to be reimbursable?

And so a lot of what we work with there is building a case for what's called HTA, health technology assessment and which drugs might get reimbursed by each of the healthcare systems that exist there. And so they want to understand U.S. results and outcomes and compare that to their populations.

[00:17:09] Joy Rios: You said it's such an interesting point because you also likely get to see the data that's related to precision drugs and precision medicine and how effective it is and whatnot. Can you speak to that at all?

[00:17:21] Carolyn Starrett: I was talking about this earlier today with someone or maybe on the panel that I was on.

It's an interesting paradigm shift because there's such this critically important moment when a new drug is approved. And if you think about the context of a pivotal clinical study it's, can you prove that the drug is safe and effective? But we talked earlier about, like, how many patients do you need to look at? And how do you know when you have enough data? And how do you make those determinations?

In reality, I think the answer is we're really just getting going now. When the drug is approved, yes, we may be able to prove safety and efficacy, but you've looked at this tiny narrow slice, and that population may or may not be representative of the broad population of patients that might benefit from the drug.

And then there are all these different complex considerations in terms of unique factors that are personal to each of us. And so I think one of the really powerful things we can do with real world data is learn from a much broader population and start to look for those smaller sub cohorts and what happens at those levels.

[00:18:17] Joy Rios: I can only imagine the dashboards that you guys have available on a company wide scale. I also am curious about how are you guys, if at all, the big AI question, right?

[00:18:28] Carolyn Starrett: Every AI has been core to our business since the very beginning because part of our job is to extract data that is not clean from EHRs and then figure out how to standardize and make that usable.

We've always built ML models that help us with data extraction, natural language processing. Increasingly now, we've been able to fine tune our ML models per variable. And the legacy of Flatiron is we had built up this team of oncology trained abstractors who log into a compliance system and do a lot of the clinical validation of the data points that we curate for us.

What's been super cool to see over the last year or two is that we can actually build on that labeled data set that we have and measure performance of ML against the human labeled abstracted data and see and what we found is that in some cases for some variables, ML actually performs better than the humans.

We think about like smoking status or areas of metastases. There are these elements in cancer where data points are just always written in prose and we find that sometimes the ML models are better. And we can now have a dial and configure where do we want to use ML for data curation, how much human abstraction do we want as and then measure quality over time, and so it's been a huge accelerant to us. 

[00:19:45] Joy Rios: And when it's considering, when you're looking at data sets, I get it. How about bias? It's also are there ways in which AI bias shows up or not necessarily because they're closed systems or closed data sets?

[00:20:00] Carolyn Starrett: Nuanced and complex question. A bias is always a risk we have to think about in terms of training and so our data science teams, I think, do a really incredible job of thinking about the frameworks for

understanding and measuring and monitoring drift and how do you have to come back to the data sets. We've done some experimentation, we'd built some models to do risk stratification in patient populations. And you start to get these interesting questions really quickly, if I build a model based on a community oncology representative set of sites, is that actually valid in an academic medical center context and vice versa? Or is the model that's trained on data from patients in New York valid for patients in New Mexico?

And so I think in the broader AI arena right now it's a really interesting question that we need to collectively grapple with, which is how do you, how do we think about this?

No one has any, no one has a crystal ball or perfect answers on how we're going to regulate it.

[00:20:57] Joy Rios: Exactly. When it feels like the train has already left the station, it's happening.

[00:21:02] Carolyn Starrett: In the context of our business, I think we're really lucky in that we are, it is closed loop and so we can look at provenance.

We know where the data comes from. We know what our models are doing. We know we are able to validate and track back and then measure and monitor quality. So we've tried to be very careful in how we apply.

[00:21:19] Joy Rios: There's consequences for, especially if it's applied in ways that are biased to the underrepresented folks.

Curious your answer around how you would empower patients. and their families. You people who go through cancer journeys, it's one of the most stressful, if not one of the highest stressful things that a person and their family and loved ones can go through. What would you like them to know? How can we empower them?

What is something that could give them a little bit more feeling of, I don't exactly know the way, it's been a long day.

[00:21:52] Carolyn Starrett: I know where you're going. We don't directly touch patient care. So we build software that is used to inform patient care. And the thing that I am really inspired by is that we, and we do these patient panels of every quarter we bring cancer patients to talk to folks at Flatiron and what we hear over and over again is that they feel empowered by the opportunity for others to learn from their stories.

And I think that's really the power and potential of using real world data for good is these stories go lost if we don't take the time and effort to structure the information and make it available for research and make it available ror learning for the broader good, and I think that's a really empowering thing that I hope we can continue to advance.

[00:22:37] Joy Rios: Carolyn, thank you so much for taking this time to talk with me. And if people want to connect with you, follow you, purchase Flatiron, be part of the Flatiron community, we're ready to direct t:hem?

[00:22:48] Carolyn Starrett: You can always go to our website. www.flatiron.com or find me on LinkedIn is probably the best place.

[00:22:55] Joy Rios: Fantastic. I really appreciate your time. Thank you so much.

[00:22:58] Carolyn Starrett: Thank you so much for having me.

[00:23:09] Joy Rios: Thanks for listening. You can learn more about us or this guest by going to our website or visiting us on any of the socials with the handle HIT Like a Girl pod. Thanks again. See you soon.

Again, thank you so much for listening to the HIT Like a Girl podcast. I am truly grateful for you, and I'm wondering if you could do me a quick favor. Would you be willing to follow or subscribe to this podcast or maybe leave us a rating or review? Or if you're feeling extra generous, would you share this episode on your Instagram stories or with a friend?

All those things help us podcasters out so much. I'm the show's host, Joy Rios, and I'll see you next time.

I'm the show's host, Joy Rios, and I'll see you next time.