Code & Cure
Decoding health in the age of AI
Hosted by an AI researcher and a medical doctor, this podcast unpacks how artificial intelligence and emerging technologies are transforming how we understand, measure, and care for our bodies and minds.
Each episode unpacks a real-world topic to ask not just what’s new, but what’s true—and what’s at stake as healthcare becomes increasingly data-driven.
If you're curious about how health tech really works—and what it means for your body, your choices, and your future—this podcast is for you.
We’re here to explore ideas—not to diagnose or treat. This podcast doesn’t provide medical advice.
Code & Cure
#24 - What Else Is Hiding In Medical Images?
What if a routine mammogram could do more than screen for breast cancer? What if that same image could quietly reveal a woman’s future risk of heart disease—without extra tests, appointments, or burden on patients?
In this episode, we explore a large-scale study that uses deep learning to uncover cardiovascular risk hidden inside standard breast imaging. By analyzing mammograms that millions of women already receive, researchers show how a single scan can deliver a powerful second insight for women’s health. Laura brings the clinical perspective, unpacking how cardiovascular risk actually shows up in practice—from atypical symptoms to prevention decisions—while Vasanth walks us through the AI system that makes this dual-purpose screening possible.
We begin with the basics: how traditional cardiovascular risk tools like PREVENT work, what data they depend on, and why—despite their proven value—they’re often underused in real-world care. From there, we turn to the mammogram itself. Features such as breast arterial calcifications and subtle tissue patterns have long been linked to vascular disease, but this approach goes further. Instead of focusing on a handful of predefined markers, the model learns from the entire image combined with age, identifying patterns that humans might never think to look for.
Under the hood is a survival modeling framework designed for clinical reality, where not every patient experiences an event during follow-up, yet every data point still matters. The takeaway is striking: the imaging-based risk score performs on par with established clinical tools. That means clinicians could flag cardiovascular risk during a test patients are already getting—opening the door to earlier conversations about blood pressure, cholesterol, diabetes, and lifestyle changes.
We also zoom out to the bigger picture. If mammograms can double as heart-risk detectors, what other routine tests are carrying untapped signals? Retinal images, chest CTs, pathology slides—each may hold clues far beyond their original purpose. With careful validation and attention to bias, this kind of opportunistic screening could expand access to prevention and shift care further upstream.
If this episode got you thinking, share it with a colleague, subscribe for more conversations at the intersection of AI and medicine, and leave a review telling us which everyday medical test you think deserves a second life.
Reference:
Predicting cardiovascular events from routine mammograms using machine learning
Jennifer Yvonne Barraclough
Heart (2025)
Credits:
Theme music: Nowhere Land, Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 4.0
https://creativecommons.org/licenses/by/4.0/
One image, two diseases, zero extra tests.
SPEAKER_00:Hello and welcome to Code and Cure, the podcast where we discuss decoding health in the age of AI. My name is Vasan Sarathi.
SPEAKER_01:I'm an AI researcher and cognitive scientist, and I'm here with Laura Hagopian. I'm an emergency medicine physician and I work in digital health. And uh today's topic is related to women's health. So I'm psyched for that. Um, we're gonna be talking about using mammograms to actually predict cardiovascular events.
SPEAKER_00:Ooh, fun.
SPEAKER_01:Yeah. I I I think this is great because we don't see enough research, I think, about women's health. And um things in women don't always present the same way that things in men do. And um cardiovascular disease is is like that. You know, we talk about atypical symptoms sometimes, and sure you can get them in men too, but I've had people show up and say, Oh, I'm dizzy, oh, I'm just tired, and they're in the middle of having a heart attack. Oh my god. Um yeah. And so uh a paper like this is exciting to me because it's like, hey, why don't we look at women in a different light and see if we can figure out something different about them from the data that we have available?
SPEAKER_00:That's great. That's great. Yeah. No, I skimmed the paper too, and I found it to be very interesting. I mean, there was a lot of stuff that I didn't understand. So I'm looking forward to hearing uh your perspective on it.
SPEAKER_01:Yeah. I mean, I think the first thing to dig in with is like, hey, um, instead of talking about mammograms right from the get-go, it's like, hey, how do we usually assess someone's cardiovascular risk? And when I talk about cardiovascular risk, I'm talking about uh, you know, risk involving the heart and the blood vessels. So anything that can affect that. Um, so heart attack is probably the first thing that comes to mind.
SPEAKER_00:All of this falls within the cardiovascular health umbrella.
SPEAKER_01:Yes. I mean, there's some nuance there, but yes, let's let's go with yes as the answer to that. It's like, hey, if you have, you know, narrowing of your blood vessels, if you've got plaque buildup, um, that doesn't just affect the vessels in your heart, right? It can affect all of your vessels. So it can affect the blood vessels going to the brain, and that's that could lead to stroke. Right, right. Right. It could affect the blood vessels going to the heart, which can lead to heart attacks and heart failure. It can affect the blood vessels going to the legs. Um, and so, you know, people can get peripheral vascular disease where it, you know, it hurts their legs when they walk, they're not getting enough circulation there. And so that's the whole, the whole concept here is like, hey, can we predict what someone's risk of that might be over 10 years, over 30 years? And then can we do anything to like mitigate that risk? Can we do anything to help someone improve? And so in the United States, we often use a calculator called the prevent calculator to figure out what someone's risk is of cardiovascular disease, of heart failure, et cetera. Um, and that takes into account a bunch of different things, which means you'll you need a lot of data to input into it. Okay. Um, you need someone's sex, you need to know whether or not they're a smoker, whether they're taking cholesterol medication. You need to know their age. Um, you need to know what their cholesterol levels are, their total cholesterol, their HDL cholesterol. You need to know their body mass index, so their height and their weight. You need to know their blood pressure. You need a lab test for their kidney function. You need to know whether they have diabetes, whether they're on blood pressure meds. Um, and you need potentially some urine tests. Um, if they're diabetic, their hemoglobin A1C level. And you can even include um basically there's their zip code to understand what their social situation is. Okay. Um, because that can affect your health too. And so I know I just listed off a bunch of things, and part of that is just to show that it's it's a little bit of a complex calculation and it requires you to have lab testing done. It requires someone to input all of that, and then you get out this output that's like, hey, this is this is what this person's risk of cardiovascular disease is.
SPEAKER_00:What's the output of a prevent calculator look like?
SPEAKER_01:Yeah, it's like, oh, well, I mean, we could do it for you. We could do we could do it live for you. You're like, no. But it's like, hey, this is the this is what your tenure risk of having an event is. Maybe it's 10%, maybe it's 2%, maybe it's got it. You know, 40%. Who knows? Like it could be very high. And if it's high, then what we do is we're like, well, how can we lower that risk?
SPEAKER_00:Yeah.
SPEAKER_01:What can we do to lower that risk?
SPEAKER_00:Well, one of those numbers, right? Probably those numbers that you just listed out has to be adjusted so that you can lower the risk.
SPEAKER_01:Right. And so that could involve, for example, maybe if someone's a smoker, you're like, hey, you can lower your risk significantly if you stop smoking. Or if someone's cholesterol values are high, maybe they need to go on a satin, a cholesterol medication. Same true for blood pressure. If their blood pressure values are high, uh even you know, even they're already on one, maybe they need to go on a second blood pressure medication. Lifestyle changes can factor in here too, right? Uh eating less salt, doing the DASH diet, exercising more, et cetera. So the whole, the whole concept is like if someone's risk is high over 10 years or over 30 years or whatever it is, like it's like, hey, let's intervene. Let's do something about it. If you, if your A1C value is high, maybe you need to do more to control your diabetes. And you you and your provider can work on that. So the whole the concept is like, let's understand what your risk is like, and then let's figure out how to decrease that risk.
SPEAKER_00:Got it. Okay.
SPEAKER_01:But here's the problem. I mean, these tools are not that well used in women.
SPEAKER_00:Let's say this seems like prevent calculator seems to be doing just fine. What's the issue?
SPEAKER_01:It's I mean, it's a great calculator if people use it.
SPEAKER_00:Yeah.
SPEAKER_01:Right. Um, but like maybe someone hasn't had all those lab tests done. Um, maybe someone hasn't used that calculator in in every single woman that's out there, right? And so then there's this question of, okay, well, is there a way, is there a different way that we can understand someone's cardiovascular risk? That's like a two-for-one deal, right? That's where our mammogram comes in because people are getting screening mammograms already. Yeah. Every year, every two years, starting at the age of 40.
SPEAKER_00:I guess my naive mind is like, what what our uninformed mind is like, what does a mammogram have to do with cardiovascular risk? It's cool, right? It's yeah, I don't understand it though. I don't understand what the relationship is. And why would somebody would even think to do that, to make that connection? Well, I think I get so many other tests, right? Like that they're not all related to cardiovascular health, but this one is.
SPEAKER_01:It is related to cardiovascular health. And and even before this paper that we're talking about, one of the things that they've looked at in general is looking at the breast arterial calcification. And so we talked a little bit earlier about how if you have disease in the blood vessels of your heart, like plaques and buildup, et cetera, you you may have it in the blood vessels going to your brains and going to your brain and going to your legs and all of that stuff. Yeah. Well, you could have that in the blood vessels that are in your breast too.
SPEAKER_00:Right.
SPEAKER_01:And so the whole idea here is that if you have calcifications in the arteries in your breast, it's probably present in the arteries in the rest of your body too. And so it's not this like giant leap. It's actually like a just a different way to look at it.
SPEAKER_00:It just so happens that part of the mammogram is measuring calcification. And calcification is in every blood vessel that could happen. And that's a sign of obviously, obviously, a sign. Calcification in my head is sort of the stuff that makes the blood vessels narrower. Yeah. Right. Yeah.
SPEAKER_01:So, you know, the intention of a screening mammogram is you take an x-ray of the breast and you're looking to see if there's anything that looks like it could be cancer. Right. But you're taking a whole picture of the whole breast, right? From multiple different angles.
SPEAKER_00:So you're just inherently getting this information, anyways.
SPEAKER_01:Exactly. And so it's like, well, if you could use that information in a different fashion, like why not? Yeah. There's your two for one deal. And, you know, this all started with this breast arterial calcification, but there's actually other characteristics. And I I'm not even sure exactly how they do correlate. And this is where like the deep learning models come in and can sort of like recognize this. Where there's other characteristics from the mammogram, like, oh, what's the you know, architecture of the breast? What's the dense density of the breasts? And those can get factored in too. So it's not just the breast arterial calcification, although that's the one that like makes the most sense in my mind. When you take um a deep learning model like this, you can incorporate all these characteristics for you know the model to take from and figure out okay, how does this how does this relate or what is someone's risk?
SPEAKER_00:All right. So so let's let me just take a moment to recap. So we have uh we're interested in cardiovascular health, and part of that is understanding the heart, how the heart functions and how the blood vessels are, and it can influence every single part of the body, obviously, because the blood goes everywhere. Um, and historically there's been a prevent calculator, a measure that takes in a whole bunch of different variables, some many of which, uh some of which are just things like uh your behavior and and your other things that you do outside, and some of those are related to lab tests and so on. And all of those collectively are fed into this scoring system that then tells you um your, you know, what is what's your cardiovascular risk. And it's been okay, but there's been some issues because it doesn't have full coverage. There's some issues with maybe it doesn't cover women as well.
SPEAKER_01:Um Well, I wouldn't, I think you're pushing a little too far there. Like it's it's actually a really great tool. Okay. If it's used.
SPEAKER_00:Oh, okay. Oh, I see what you're saying. It's a great tool, but the problem is not everybody gets tests, not everybody uses it. Yeah, exactly.
SPEAKER_01:I mean, the prevent is like the next generation tool. There was stuff before it, which was also good, but kind of tended to overestimate people's risk. We don't need to go into a ton of details about it, but I I think what's what's different here is that it's like a novel approach and it's something that we already have. Yeah, I mean many ways.
SPEAKER_00:But that's it. So the prevent requires you to do a whole host of tests, which are separate and maybe only serve purposes for the prevent in some ways. Whereas you're already the point is to say how much can we sort of extract from existing tests so that the person or the patient or anybody doesn't have to take any extra effort in in in like doing additional things, they're just doing their other tests as part of the routine test. And can we get more information out of what we already have?
SPEAKER_01:Or as much information, really, just like as much, yeah, so that we can figure out how to lower someone's cardiovascular risk.
SPEAKER_00:And one of the cool aspects of this, uh the calcification, breast calcification is that now you have mammograms that kind of relate to uh things that are cardiovascular health related, and people have started doing that as well. Now, that is sort of the starting point for this paper, right? Because this paper takes it to a different level.
SPEAKER_01:Yeah, exactly. Exactly. And so that's where they bring in this deep learning model, and they, and you'll probably have to feed in some of the details here, but basically they took all of this data from like 50,000 women and they knew, you know, they knew all this background information about them so they could actually calculate their prevent scores. They had their mammograms and all the imaging characteristics about them. And so they were able to, you know, feed that into this new model that they were creating and see if that could predict someone's cardiovascular risk as well. And so what the end model used was their the person's age and then their mammogram characteristics. And again, going beyond just breast arterial calcification and also looking at other things like breast density.
SPEAKER_00:Yeah, and it's it's it so so that's also interesting to me because what they're saying is hey, here's this other test that's relevant. Why not extract uh what we think is important from that? No, no, no, but let's take it one step further. We're doing this test anyways, so why should we restrict ourselves ourselves to extracting only the things that we think are relevant? Let's just extract everything, right? That's kind of what is is being implied here by this test.
SPEAKER_01:Yeah, exactly. It's like if it's already being done, why not try to use it for another purpose? And and honestly, like at the end of the day, that's what they were able to do. It was like it it worked pretty much just as well as that prevent calculator that we've been talking about in terms of predicting cardiovascular risk. So the whole idea is like, hey, we can, if someone you can't calculate their prevent score, or maybe that's like a routine thing that's delivered in the future along with your mammogram. It's like, here's your mammogram results. Right, right. Right. Um, and with the mammogram results in the US, now you get like, here's your here's what your breast density is. Um, when I've had them before, like they'll give me my my uh risk of breast cancer calculated with a couple of different calculators. It's like you could put your cardiovascular risk on there too. And then it's like a flag to you, to your provider, like, hey, is there something I need to do about this? Like, hey, can we talk about this? Um, you know, your blood pressure is a little bit high and your cardiovascular risk is high. Let's do something about it. So there's like this trigger to maybe have that conversation if you can calculate based on a mammogram what someone's cardiovascular risk is.
SPEAKER_00:Yeah, and that's kind of where they used um these deep learning systems to help with that. And it's a it's a little different from the deep learning systems we've used in the we've talked about in the past, but it's it's similar. It's still a neural network, it's still a deep learning system. But the way they calculate it and the way they set it up is a little bit different. And they use something called Deep Surve, S-U-R-V, and it learns a risk score from images. Um, it's it's a neural network that extracts features from your mammograms, your routine mammograms, plus things like age, and outputs a single number that represents how high um how high risk a particular woman is uh for a future cardiovascular event. Um it doesn't predict the exact time of event, it just says how high the risk is, which is exactly kind of what's what the prevent calculator does too.
SPEAKER_01:It's not like uh, hey, in in five years, in two days, we predict you're going to have your heart attack.
SPEAKER_00:Yeah, and and and what the basis of this neural network model is something called the Cox Proportional Standards model, which is a model you use for risk analysis. And and I think a way to think about that model is um and you're when you're watching a race, right, in the Olympics or whatever, a 200 meter race or something, people are running. You can at any given point observe what people are doing, and you know that this person is ahead of this person, this person is ahead of this person. So you can like um you can extrapolate from that, right? You might not know exactly what time they'll finish, but you might know relatively who's ahead of whom, right? That's kind of what uh this model does in predicting, you know, in the in that race example, the event is the end of the race or whenever you stop watching.
SPEAKER_01:Okay, but here we're talking about cart like cardiovascular event, heart attack, stroke, right, right, right, right. You know, death from cardiovascular disease, et cetera.
SPEAKER_00:Yeah. So the deep serve model keeps the Cox proportional standards framework, um, basically ranking who is riskier than whom over time, uh, but replaces sort of there's like a mathematical detail here where there's a non-linearity in the model uh that allows you to capture these like complex patterns that might exist in the in the data. You know, maybe the patterns are implicit in the breast tissue and so on and so forth, but like it it you need the deep network, deep deep learning network to be able to capture kinds of the weird relationships that might exist in the data itself.
SPEAKER_01:Okay, so is that something that's like explainable? Like, do we know exactly what the deep learning model has found?
SPEAKER_00:I mean, there's some of it is a little black boxy, obviously, but because it uses this approach and because we know what the inputs are, we could potentially explore, and we've talked about this before, uh, which factors are more influential than others and things like that. I don't think they've explored maybe maybe they have, but I think that that's something that would be a very exciting work that if if someone were to do it. Um it also naturally handles sort of uh situations where you know maybe the women did not experience a cardiovascular event during the study. Um, but their information is still useful uh up until their last follow-up because the model can still learn from incomplete outcomes. Like, because I can't remember that that's a lot of people.
SPEAKER_01:Well, you don't want everyone to have a cardiovascular event. You want to like figure out who will be, though.
SPEAKER_00:We always think about deep learning models as being or any kind of machine learning models is you want the quote unquote right answer in the data set, right? You want a data set of here are 12 factors, and these people had cardiovascular events, these people, you know, you you uh historically that's how we think about it, right? We think about these models as provided, giving giving them an input and giving them an output and giving them lots of examples of inputs and outputs, and then saying um and the outputs typically are things like cardiovascular events or those occurring. So you'd want a data set of that, but the model here is set up differently where it doesn't care about when the event happened, it's just taking what it's taking a snapshot of what it is, or the state what the state of the world is right now, and so that allows you to feed in incomplete data and still be useful.
SPEAKER_01:Yeah, because if you well, I think about the pervent calculator, and it's like everyone's gonna get an output, even if they ultimately don't have a cardiovascular event, it's like your your chance isn't zero percent. So, like right, someone's chance could be 40%, another person's chance could be one percent. And so you want to be able to differentiate between those, but also like some people are are not gonna meet that endpoint. Yeah, yeah. And so the model's able to account for that.
SPEAKER_00:Yeah, the model's able to account for that. And you know, they're it's not just one model, they had a whole system that they built that had to take in not just uh all the features that were extracted from the mammogram, but even potentially handcrafted features that the human experts were able to kind of put in. And that's kind of the explainability part because there was knowledge from before about the other things that might be relevant, so they were able to feed all of that in.
SPEAKER_01:And did they feed in like the image itself or do they feed in like an interpretation of the image?
SPEAKER_00:There are pieces of the architecture that takes in the image directly as well.
SPEAKER_01:Interesting.
SPEAKER_00:Yeah. Um, but again, like I said, it's not just one model, right? It's there's a lot of stuff. You take all these different pieces and you put them together and eventually concatenate all of the different features that you care about and then feed that into the model. Um, and so I I think that that's the really cool part. And the results seem to suggest that they it did it did pretty well, right? That it was able to um that this model performed pretty well.
SPEAKER_01:Yeah, it performed like pretty much as well as the prevent calculator did, which is awesome. Right. It also makes me think, like, in general, is there data out there that we're just like not fully using that AI could help us take advantage of? Like we do so, you know, we do so much testing and this and that and the other thing. And is there information that we have like as humans haven't said, hey, this is connected? Um, where like AI could come in and be like, hey, hey, there's a connection here, or I can predict some, I can do some predictive modeling off things that we didn't necessarily even like notice were related.
SPEAKER_00:This is it reminds me of our podcast episode on drug repurposing. Yeah. Right where we We're able to use AI to find new uh targets for existing drugs. Um, and that was really cool. And and you're sort of sort of saying the same thing is there is there are opportunities for cross-pollination. Data that we already have, of course it must be, right? Because there's a of course your body that's all connected. So why wouldn't there be?
SPEAKER_01:Um and it's like you as a researcher, you might you might be like, oh, I wonder if these things are connected, and it's like a little bit of a shot in the dark, and then you spend all this time doing that research. Whereas AI could function completely differently. There's so much information and we may just be like tip of the iceberg in terms of using it. Yeah and AI could help us figure out how to use the information that we already have for things like predicting risk.
SPEAKER_00:Yeah. I mean, it makes me also think that shouldn't we be thinking about applications where uh people, I mean, doctors and other uh healthcare providers and such are saying things like, Oh, I wish my patients tested for this more often, or I wish there was more um, I wish, you know, like places where you really wished there was more uh data in order to fill in things like the prevent calculator, right? Are there other examples like that where you can you can you because it seems to me seems to me that those are the low-hanging fruit, because then you can go say, okay, these people are not doing tests for this, it's really important. Maybe we can just what other tests are they doing, anyways? Okay, they're doing these other tests, anyways. So can we just get data from that and apply it here? You know what I mean?
SPEAKER_01:Yeah, and when you think about like anything at like the public and population health level, like you could bring some of this stuff in. Oh, what's someone's risk of like colorectal cancer, for example? We recommend colorectal cancer screening in people. Um, it's like, well, what kind of tests should they get? There's stool-based tests, there's colonoscopies. Um, mammogram is like actually a really good example of this, too, where some people who have a higher risk of uh breast cancer, maybe they need more frequent screenings with mammograms. Maybe they need uh an MRI, right? And so it's like it's not necessarily a one size fits all kind of thing.
SPEAKER_00:Yeah.
SPEAKER_01:And I do wonder if AI can kind of help us get there. You could also think about other applications too, like, oh, um, maybe we can use a model to predict who's gonna go ahead and get their vaccines. Um, and if someone is, you know, not gonna get their flu vaccine this year, who are the ones that we really need to try to convince because they have a high risk of hospitalization. Yeah. Right. Like they've got COPD or asthma, they're elderly, whatever, they're immunocompromised, and they didn't get their vaccine. Now we need to like reach out to those people because of the high recommendation score. Yeah. Yeah. I mean, there's just there's so much out there that could be done. And and there's so much data that we do collect on people that you know, maybe we're just not maximizing the use of right now.
SPEAKER_00:Right. Agreed.
SPEAKER_01:So I think this is a a very cool application. Um, prevent calculator is something that's simple, but not necessarily as widely applied as we may want. And so being able to use uh a mammogram, especially in this in this population that isn't always of women that isn't always um getting screened for cardiovascular risk, it's like, hey, this is a basically a two-for-one deal, where a mammogram, which is already being done, can also be used to predict cardiovascular risk, which then, like in turn, can be used to make the changes necessary, whether that's you know, tobacco cessation, whether that's new medication, whether that's lifestyle changes, et cetera. And so I think this is this is cool. And this is something you can extrapolate from too, and say there are other use cases like this. This is the tip of the iceberg.
SPEAKER_00:Yeah.
SPEAKER_01:So with that, we will see you next time on Code and Cure. Thank you for joining us.
