Machine Learning for Real-Time Hydrate Risk

Good afternoon. Good morning everyone. We're depending on where you are. I believe I have the, the go ahead to get to get started, so I, I'll go ahead and do that. Thank you for attending this this talk. My name is Andrew Yule and I work at a company called Pontem Analytics. Talk about Pontem here in a minute. But the point of this talk is to discuss discuss how I used Wolfram language and some of its, particularly some of its machine learning prediction capabilities to, to help implement a real time monitoring solution for hydrate risk in an FLNG or in an LNG facility. So I will jump into that a real quick kind of an intro into quantum analytics. The company that I work for, we are a specialist consulting company and we focus on combining basically engineering with data driven analysis. We're kind of a unique group of employees. We've got strong backgrounds in engineering, chemistry and then data science. And we kind of use all those tools that we have to solve problems for our clients in terms of kind of the areas that we operate actually kind of go this bottom bottom up here. We have some, some work that we do in the industrial space around kind of predictive analytics and predictive maintenance and things like that for large industrial applications. We do a small bit of things in the performance space around computer vision and tracking motion and things like that for optimal human kind of movement and, and less injury. We do a little bit in the, in the health space. Similar similar idea was kind of what can you do with data around, you know, preventing illness and kind of increasing longevity. Things like that. We have some, some assets that, that we, that we own in the, in the mineral space. And then kind of our, our main focus or, or what we do most more than anything is, is focused in the energy space. And so there again as I mentioned, we're, we're kind of by background we, we have the strong expertise in engineering chemistry, so things like production chemistry, understanding fluids, things like that and then tying all that together with both physics based modeling as well as data driven modeling, things like machine learning and things like that. So it's kind of the areas that we, we operate in kind of coming off the back end of that, where we operate, where we focus a lot of our work is in, is in the offshore oil and gas environment. And so in that space you're dealing with very extreme conditions as far as your temperatures and your pressures. Just kind of for reference, if you're, if you're producing, you know, hydrocarbons from, from the bottom of the, you know, body of water like The Gulf of Mexico, you're trying to get know those hydrocarbons from, from the ground which could be, you know, 50,000, 15 to 20,000ft below the surface. And you're sitting in, in around 8,000ft of water on top of that, right? So the pressures of that water sitting on top of that and the temperature that you're dealing with can cause all kinds of issues with, with the fluids that you're trying to produce. One of the one in particular issue that we, that we routinely have to have to deal with and mitigate and provide strategies for, for, for how you, how you avoid is, is something called hydrates. So hydrates are unique structure. They're, they're actually, they're what's called a classic clathrate compound in which when you have ice, when you have, sorry, when you have water in the presence of, of a, of a hydrocarbon gas like you know, methane ethane, things like that, what happens is that those gas molecules can get trapped by the water and that water will typically kind of form this, this cage like structure around, around the, those molecules and they become very problematic for your pipelines. Got an example here on the right here. So you can, you can form these hydrates in your, in your pipeline. And if you have enough of the, the right conditions and enough of the mass, you know, required the water and the gas, those, those hydras will continue to agglomerate together and they'll, they'll basically lead to a blockage of the pipeline. And if you can imagine, you know, in an isolated pipeline subsea, that's, that's an incredibly problematic situation. So I wanted to give this background about hydrates because it's going to feed into what we were doing on this particular project. So a little bit more just about kind of hydrates and why they're, they're interesting. So this, they obviously they look like ice, right? They, they kind of, they, they feel like ice and things like that as you're, as you're looking at them. But you can see here, so I've potted. This is a, this is a temperature versus pressure plot. The blue line here is actually, is actually ice, right? So, so water ice forms at 32 degrees. As you increase the pressure, there's a small deviation where you can actually go a little bit lower before you start to form ice. So kind of to the left of this line is ice. To the right is, is, is no ice right now hydrate curve is actually this, this red line here, right? And what you can see is that you can form A hydrate at temperatures well above freezing and actually at temperatures that are relatively, relatively moderate, you know, somewhere around 60 to 70 degrees if you have the pressure, right? So there's a strong pressure component and that's what ultimately allows hydrates to form in situations that you, you wouldn't necessarily intuitively expect a hydrate to form. If you, if you thought about it like, like ice. So that's some background about, about hydrates. What I wanted to focus on now is actually the project that we were involved in. So this was, this was not an offshore project. It was actually a, an onshore LNG facility. And so LNG stands for liquefied natural gas. So what you were doing in lng, I'm sure most people have heard of this. You, you probably see it, see it on the news occasionally, things like that. It's, it's essentially you're taking methane, right? Which is, which is a gas, and you're, you're converting it into a liquid form that you can transport it easier, right? So we've heard of like exporting lng. That is essentially where you're taking this methane. You're converting it into a liquid form and then you're able to ship it on these large tanks kind of around the world. And it's, it's one of the most effective ways to transport gas. The process of actually creating LNG is, is rather difficult, though. Methane by itself does not want to be a liquid, right. At really, really at any temperature that, that we're used to. Right. So this, this again is kind of that same, same plot I was showing earlier. But this is a, this is temperature versus pressure. But this is the phase on low for, for pure methane, right? So at, basically at temperatures above about, you know, minus 300 degrees Fahrenheit, that methane is going to want to exist as a gas. So the only way you can get it to a liquid to actually cool it is to actually cool it down to these temperatures. So you typically see LNG plants operating at around -260 degrees Fahrenheit. That's the only way that they can get that, that ethane into a liquid form. Now one of the other things that, that occasionally these, these plants will do is they will also include an ethane stream in it. Right? So methane is a single carbon, single, single carbon molecule. It's a carbon and four hydrogens. Ethane is two carbons and six. Six hydrogens. Yeah. And so what you end up with when you do that is you can increase the heating value of that, of that Methane that you're shipping and all of the LNG prices and things like that are generally, they're, they're costs based on, on how much heating value you actually have. So if you have ethane available as a source for you, you can inject that into the, into the methane and do your cryogenic processes. You actually increase the heating value of that, of that lng. So here I've got that same methane phase envelope and here I've got the, the, the, the ethane phase envelope, right? And you can see ethane is, you know, much more likely to form. You know, over here it will be a gas, but at relatively reasonable temperatures you can, you can convert ethane into a liquid much more easier, much, much easier than you can with, with methane. So now when, when you inject that ethane, which in the case of this project was coming in as a liquid stream because it was at a higher pressure and slightly cooler temperature, right now, now you have a methane and ethane stream and you actually have a multi, multi phase or a multi component system, right? And so the phase behavior of that system is fundamentally different from when you have either methane or ethane together. You can see here, this is a representative phase envelope for what a 80, 20 mixture of methane ethane looks like, right? So you can see that we shift ethane, we shift, we shift shifted to the right compared to methane, we shift left compared to ethane. We also have this, this envelope, right? And what that means is that in between these two bands we'll actually have kind of a multi phase. So we'll have a, a liquid and a gas phase present present at the same time. The important thing though is that when you're injecting that ethane, injecting it into liquid, into a, as a liquid state, and when it, when it reaches, when it mixes with that methane, there is a vaporization process that's to, going to happen to that ethane. And through that vaporization process the ethane will cool, right? And, and sometimes that cooling could be quite substantial. And so this kind of, kind of tying this back to what I was talking about earlier, if that coolingness is substantial enough and you actually have some moisture content in either the ethane or the methane stream that you're mixing, you suddenly have a risk for performing hydrates. So same idea that I was, I was talking about before. Now in this case, these streams that we're mixing when you're going into an LNG facility generally are very pure. They're, they're, they've already gone through some kind of processing or, you know, some kind of separation of different components within, within their streams. However, there's still, there's still a, you know, strong chance that you'll end up with, you know, enough water content in those streams to actually form, form hydrate. Generally when you're, when you're looking at forming hydrates, you're in a situation where you have excess, excess water. So water is not a limiting component to the hydrate formation. However, in this case, because we are at such low water contents, the. How the conditions that you'll form hydrates at are actually limited by how much water you can form. And some of the, some of the theory and some of the calculations that testing that are out there show that there's a bit of uncertainty at this, at these kind of conditions. So for example, I've plotted here, you can see it kind of moving from the right to the left. You can see how that hydrate curve that I was showing originally, how that changes as you move on the far right, you, you're basically, you're saturated with water. You have so much, you have enough water that you're not limited in the hydrate formation on water. And as you move down to smaller and smaller amounts of water, you can see that curve starts to shift to the left. As part of this project, we did some testing of some running some experiments at, at a, at a water loading that was kind of somewhere in that middle range there. And you can see we, we ran multiple tests on, on multiple, multiple tests at multiple conditions. And you can see the error bars there or somewhere around 8 to 12 degrees Fahrenheit, right? So this says that there's an 8 to 12 degrees margin or, or uncertainty on, on what the actual hydrate formation conditions are when you're in this water limited situation that we're trying. Okay, so jumping kind of into the, the heart of the project and, and you know, what, what we were given and what we were looking to do. This first, this first image here is an actual drawing of where this mixing takes place. Right. It's actually relatively simple piece of equipment. It's, it's a, it's a long, long pipe about, about 20ft long. Pretty substantial, substantial diameter. There's about, there's two injection nipples up here on the top. And this is where, where the ethane streams come in. So the ethane rates are, we've got equal, equal liquid rates of ethane coming into each, each port up here, the mixing, mixing happens. And kind of by the end of this, you've got a Kind of a homogeneous gas phase of your original natural gas with that vaporized ethane. As part of the project, there was some really interesting computational fluid dynamic simulations that, that were run to kind of understand this, this process in detail. So what I'm showing here is you can see that you've got, you've got your, your natural gas coming in at around 71 degrees Fahrenheit. These, these ethane streams are also coming in around 70 degrees Fahrenheit. Now that mixing happens and it happens relatively quickly. But during that initial vaporization, you can see that you get some ethane coming in here and you get temperatures dropping around minus 32 degrees or so now only around 5 to 10, 10ft down that line. You can see that all the temperature, all the mixing, the full mixing has occurred and you're back up to the temperatures around around 40 degrees Fahrenheit or so. So there is a, there is a strong risk of kind of what, what might be happening here when you're forming those incredibly cold temperatures for a brief period of time. And the, and the key thing that we wanted to analyze and look at is the, are you forming that, those cold temperatures as close to the center of the pipe as possible? Because if you're forming those on, on the outer walls of the pipeline, you're giving a spot for hydrate agglomeration to, to occur. And if you're doing this kind of mass volumes that are being injected into this, into this pipe, you're just continually feeding that hydrate formation. And it wouldn't take long before the agglomeration process would become large enough that you could cause a blockage in this. Right. So the thing we wanted to make sure is that this process, yeah, you're going to hit these low temperatures, but it's going to occur as close to the middle of the pipe as you can. And then very quickly we start to seed all that phase, all that phase mixing, and you end up with a much warmer process. Right. So this detailed simulations were performed. Just understand and validate, validate the design. And I look at the, really the detailed fundamentals of what was going on. Once we ensured the design was fit for what we needed it to, we could take a step back and take a more traditional approach that didn't involve computational fluid dynamics and instead use just a normal process modeling approach. So here we've set up a simple model in a process simulation around this whole process. And this, the speed here of how we can model with this is orders of magnitude faster than what we can do with computational fluid dynamics. And so this gives us the ability to basically change, change inputs, right? Change all the inputs that we care about in the model. So that's, you know, the temperatures and the pressures and the flow rates of both the ethane streams and the methane streams, the compositions, the water contents, things like that. And then ultimately what we wanted to do was to monitor for this hydrate risk after all this mixing, mixing has occurred, knowing that this process model that we've set up here has been validated by the more, the more computationally heavy approach up here. So all in all, we ended up with, with a, with a highsis model that was capable of simulating 200,000 cases, right? And we set this up in a, in a parametric study, like I said, varying all the possible inputs that we care about that could change within the system. So kind of just recapping where, where I'm, where we are at, right? So we, we've defined, define the risk for the project, right? There is a risk of hydrates that we want to, we want to completely avoid, right? We don't want, we don't want that risk to be present while we're operating. We defined what the ideal margin was that we need to stay above. Again, from some of that lab testing, we know that there's an uncertainty around 8 to 12 degrees Fahrenheit. So our prediction, if our prediction was, you know, as best as it could be, we could still expect that there's 8 to 12 degrees of uncertainty. So we want to operate around whatever our prediction is essentially, plus, you know, plus 8 to 12 degrees of a margin of uncertainty to ensure that we do stay outside of that risk for hydrates. There was detailed CFD modeling done to simulate this process in detail. And, and once we understood that and we, and we, we could then build a faster process simulation model that was, you know, ensured to be aligned with what we'd done on the, on the computational fluid dynamics approach. Right. And so at this point, the client said, you know, what we would like to be able to monitor is, is our hydrate formation margin in real time to alert operations whether they need to cut the ethane injection or stop it all together. Right? So we're getting to now this, the, the part where, where Mathematica and Wolfram language actually came in. So we, we ran this large parametric study in ISIS. Again, we have over 200, 000 data points of all the possible ways that this system could behave. And, and we were tasked with, how do you build a model that can, that can be implemented into a real time system to, to predict that, that hydrate formation temperature for the client. So the first thing we did with the Wolfram language was we put all this data in and we used some of the predict functions. Right. And what that did is that gives us a really easy starting point to say okay, what kind of accuracy can we achieve with relatively simple models like something like a linear regression and then increasing in complexity to something like a neural network? And then ultimately, right, because this is a parametric model run from a simulation, we controlled the inputs. So we have a, essentially a squared off arrays of inputs that we're putting into this. So you can actually use multi linear interpolation here. Right? And that actually ends up being the most accurate way that you can, you can model this. But the problem there is if you want to embed something like that into, into a relatively, you know, simple automation control system, you're talking about being able to take all 200,000 of those points and put them into the, into the model. Right? So where we kind of compromised was on. Okay, let's, let's look at this neural network approach. Right? Linear regression model just doesn't have the accuracy we need. Especially if we're looking at 8 to 12 degrees of margin on our results. If we're seeing that in the error alone, it's not going to be feasible. Right. The neural network looked like it had the highest levels of accuracy right out of the box. And we figured that maybe there was a good way that we could, we could tweak some of that to improve it further. So we use the Wolfram language. Building a relatively simple neural network with, with, with three different linear layers and some, some ramp functions for, for the activations. We were able to achieve basically the following results down here, right? So this is a parity plot showing how our predictions for the hydrate margin with this neural network compared to the actual predictions from from HEIS model. Our overall mean absolute error was around 0.207 and the root mean squared error is 1.306. Right. So so very good performance probably, you know, within, within what we would want for this type of model to be deployed into, into the field. And so with that we kind of set out, set out to do that. So we said how can we, how can we most easily put this neural network into, into practice in the field? Right? We don't have the ability to call, you know, web APIs and things like that. So we can't, we can't deploy this model into the cloud. So Here again, what we were able to use some of the net extract functions in the Web Wolfram language to pull out those weights, the biases, things like that. And then we could actually construct the math, the matrix multiplication operations that the neural network was, was doing. And we work with the automation control to get all that, that math programmed into, into the automation control. So got, got a little screenshot here. But you can see this is the real life system, right? It's receiving its data and in real time and all that. And then right down here we've actually got our, our model kind of running, running in the back down, telling them what the hydra temperature is and what the hydrate margin is. You can see here, you know, it's around 48 degrees, kind of well, well outside of, of where we would want to be during, you know, during steady state. Become some kind of, you know, hydrate. Hydrate. Monitoring risk. Yeah. So just recapping, right. So kind of were wolf and language played a role, right. We actually analyzed years worth of historical data. I didn't, I didn't get a chance to talk about that. Just time and things like that. But it was a big play, big component of understanding the possible risk in this system. Once we knew the high, once we had this process model, we knew what conditions were going to be problematic and we looked at historically, do we ever kind of reach some of those conditions? And it turns out that we do not, not, not that often, but it certainly was a risk, a risk for the, for the facility and something to be planned for. The Wolfram language also allowed us to use, you know, very easy screening of different, different models to give us a place to start with before just diving into something more, more specific. The neural network framework within the language is very, very easy to use. It's very flexible. It's a lot easier than some of the more traditional, traditional languages out there. When you're trying to build these neural networks. It also gives us the ability to, to kind of write scripts. Right. So I mentioned one of the neural networks that we built for this project. There was actually two, two more and I didn't want to spend too much time on them, but we were able to build those in a very automated fashion using just normal code that's in the Wolfram language. And then lastly, just the symbolic architecture of language. It makes it so much easier to pull out the information that you want from those different layers that you've trained. Things like that. It's all things that can be done in other languages, but it's the consistency and the cohesiveness that's in the Wolfram language that it allows it to be this kind of one stop shop for executing a project like this and seeing it from start to finish. So that is all I have. Let me quickly jump over here to see if there are any questions. Not seeing any. So, yeah, that's it. Thank you all very much for attending.