Emma Rocheteau, a PhD student in Pietro's lab, presents a project that predicts intensive care unit (ICU) length of stay using a novel Temporal Pointwise Convolutional (TPC) neural network. The motivation is to lower hospital costs, reduce infection risk, and improve resource allocation by forecasting how long patients will remain in the ICU. The study leverages a large multi‑hospital dataset (over 200,000 ICU stays from 208 U.S. hospitals) that includes frequently sampled vital signs, irregularly sampled lab results, static demographics, diagnoses, and medication records, and the model is also validated on the MIMIC‑IV dataset. Inspired by clinicians’ focus on temporal trends and feature interactions (e.g., blood oxygen vs. inhaled oxygen), the TPC architecture combines temporal convolutions to capture trends over time, pointwise convolutions to model relationships between variables at each time point, and skip connections that allow each feature to have its own receptive field and independent parameters. The network consists of nine stacked TPC layers. Evaluation uses mean squared logarithmic error (MSLE) as the loss and mean absolute percentage error (MAPE) as an interpretable metric. Compared to several deep‑learning baselines—including a transformer—the TPC model achieves substantially lower error. Ablation studies reveal that temporal convolutions provide most of the performance gain, pointwise convolutions add a 10‑15% boost, and allowing per‑feature (non‑shared) weights is critical; weight sharing across features degrades performance. The talk concludes with acknowledgments and contact information.
Hello everyone, I'm Emma Rocheteau and I'm a PhD student in Pietro's lab.
It's a pleasure to be talking to you about my most recent project, predicting length of stay with temporal pointwise convolutional networks. I'll just make myself a little smaller for the rest of the talk. All right, okay, so why predict length of stay? Well, it's a key determinant of hospital costs, so it's expensive to keep patients in for a long time. Also, longer stays increase the risk of getting hospital-acquired infections such as COVID-19, or in more reasonable times, multiple drug-resistant bacteria. We can mitigate for some of this But we need to do this by knowing how long the current set of patients are going to remain in the ICU so we can remove bottlenecks for them being discharged. And, and we can also plan to get the next patient in to make the best efficient use of the resources. So we're using some data from the United States for this project. We have over 200,000 ICU stays from 208 different hospitals across the US. This kind of data includes vital signs such as heart rates and blood pressures, often things constantly monitored in the intensive care unit, and then also lab results such as, say, a COVID-19 swab, and also all kinds of blood tests. We also have data that doesn't change over time, and then diagnoses and medications. And I should also say that we've recently confirmed our results on the newly released mimic 4 dataset as well. So this is some example data from EICU. So along the bottom we have time, and on the left I'm showing 6 different variables that we have in our data. So the first 3 are frequently updated, so we have a new measurement every hour for most of these. And these are the kind of things that are constantly monitored by machines in the intensive care unit. And then the next 3, these are much more irregular And they're much more sparsely sampled. And this is much more typical behavior of the lab results.
And we need to make sure that any model that we design can get good use out of the different kinds of time series data that we have. So in the design of the model, we really did think back to how do doctors solve problems. So one of the things that we found out when I went to the intensive care unit is that doctors tend to think about what is the trend of the patient. Are they deteriorating over time or are they improving? And also, are there ways to combine different variables together to get a better understanding of what the patient's health state is? So that can be more clear with an example. So say that if we have a COVID-19 patient and they have some lung failure, so they're struggling to breathe, the doctor might be interested in how's their blood oxygen content, the partial pressure of oxygen in the blood down here in red. And they're going to want to know, is this improving, getting worse? But they also want to know how much oxygen the patient's breathing in because it could just be that they're on 100% oxygen and actually a high value for the oxygen content of the blood doesn't mean that they're actually ready to be discharged just yet. In terms of our temporal trends that we want to extract, we use temporal convolution. So we can see that along the bottom we have the blood oxygen content. And the arrangement of the neurons in the network can allow us to get a picture of what's happening over time with this method. And so by the time we get to this red output neuron, we can see all along the time series. For the relationships between features, we use the pointwise convolution. So here, instead of sharing information across time, we're looking at all of the variables that we have for a given time point and getting a higher-level understanding of the patient's health state. So this is where we can look at what's happening in the blood oxygen and the and concentration of oxygen that the patient's breathing in at the same time. We also add skip connections to the temporal convolution. So this allows us to skip certain parts of the network if that seemed to be beneficial for learning. And in the case of temporal convolution, this allows us to actually choose how far back we want to look in time. So we have a choice as to whether or not we can extract trends for the entire period, or we can tend to look more at more recent values. And this kind of thing really helps us to cope with, say, vital signs where we want to look at the more short-term trend versus lab results where we either want to look a long way back in time or perhaps if there's no variation at all and we only have one measurement, we just take the most recent value. We actually arranged these methods in parallel. So this is showing just one layer of our TPC model. Temporal pointwise convolution. But we actually have 9 of these in the complete model. So just to recap why we expected our model to do well, it's been specifically designed to be able to extract trends over time and also relationships between features. And it can theoretically choose its own receptive field independently for each feature because we have different parameters for each feature in our data, like different parameters for blood oxygen compared to, say, the blood pressure. And that's because of the skip connections that we can do that. So just to show you some results, we have in blue, I'm showing the mean squared logarithmic error, which is actually the loss function that we minimize in our work. And on the right in red, I'm showing the mean absolute percentage error, which is a bit more human interpretable as a metric. So along the bottom here, I have a median model, which will just always predict the median value of the training set, no matter what the test set data is saying. And that just shows us how much performance we get as a bare minimum just by predicting in a reasonable range. And the next three, these are all deep learning methods which have been used before for this task over the past 5 years. And actually, it's difficult to see much difference, but the transformer is actually the best model. But our TPC model is much, much better than any of the baselines that we tried. So we wanted to find out a bit more about why this was. So we tested the TPC model under different sets of conditions. So we find that when we just use temporal convolution, so this temp method, we get almost as good performance as the TPC. So most of the benefit is actually coming from the temporal convolutions. The Pointwise on its own doesn't do very well, but it does add about 10 to 15% of the performance to the TPC model. Interestingly though, the Temporal Convolution, it only does very well on its own if we allow the network weights to learn different weights for each feature. So this TempWS means that we do weight sharing between the different features. So we constrain the model to process heart rate and COVID-19 tests in exactly the same way, and that significantly impairs the model. So it's very important to allow the flexibility to learn differently for each feature. So, yeah, so thank you for listening to my talk. And if you do have any questions, please do email me. My email's at the top. Thank you to my funders, the Armstrong Fund, Frank Elmond Clinical School, and thank you to my wonderful supervisor Pietro, and also my mentor at Microsoft Research, who's my collaborator there. So yeah, thank you very much for listening.
