Herb Zien - CEO at LiquidCool Solutions

Welcome to another CEO Wisdom podcast. We have Herb Zien with us today. He's CEO at LiquidCool Solutions, has been doing that since 4 years, been board member at various organ— not, not 4 years, almost 20 years, sorry, and been board members at other organizations, MIT educated. Thought I'd have him on to talk about cooling these very expensive data centers that apparently are losing out and building sort of, I would say, like broken systems, right? If the heat can escape and the fans, they just dissipate the heat elsewhere and they don't really kill the problem at the source. So Herb, welcome to the podcast. Tell us a bit more about yourself and what you've been building.

My background is principally, although I have a business degree, I've 2 engineering degrees based on thermal engineering, and my career has principally been a serial entrepreneur. I've owned all kinds of stuff, done some crazy things over the years. One of the things I did was a friend and I started what became the largest owner and operator of district heating and cooling systems in the U.S. So we owned the downtown energy systems in Boston, Baltimore, Philadelphia, Los Angeles, Atlanta. 11 cities. We sold the company. I exited with dignity and became an angel investor. And when I was running Trigen, the district energy company, I would laugh. I'd walk into these— in fact, I wrote a paper called A Thermal Engineer Walked Into a Data Center and Couldn't Believe What He Found. But the, but the, um, you walk into these places, you put the earplugs in, and, uh, you got a hot thing in the middle of the room and they're trying to blow the heat away, which is stupid, actually, is the, I think, the proper adjective for that. It's just a dumb thing to do because fans are, uh, use a huge amount of power. Air is an insulator, not a— earlier in my career, I used air to keep heat from moving around. But it's an insulator, and the waste heat and fans is basically also adds to the burden of the data center cooling system. So when we sold the company, I became interested in addressing that problem, one of these few change-the-world opportunities, and I became involved with a company that was just starting up that basically submerges the electronic boards and components in a dielectric liquid, which conducts heat and not electricity. By doing it, they can overclock the chips. It's about 1,400 times more efficient at moving heat away than air is. Go ahead.

Why do people still use fans? Is it the cost? Right? Like, and why did people use fans in the first place? I mean, as you know, most— there's a lot of electrical engineering and smart people involved in this. Why did they use fan and not like, I don't know, liquid water tubes? There must have been a reason.

Interestingly, the earliest computers were liquid cooled, but many of them were. But, you know, there's a lot of inertia to these systems. People think of data centers as high tech, but actually They're so expensive and there's— they're big bets. Engineers like me, the smart thing to do in many cases is just to do what we did before but tweak it a little bit. The reason the change is now occurring is because the chips are too hot. Air cannot do it anymore. It just can't be done. It's physically impossible. So data centers are converting to liquid. And basically they're doing it— there's 5 different liquid cooling technologies. One of them is called rear door cooling, and it brings— it's an attempt to make the volume of air you're refrigerating smaller. So what it does is it brings a heat exchanger to the back of the rack, but it's air cooling inside the rack. So there's a challenge with the hot chips in that— with that technology. There's direct-to-chip, which where they put a heat exchanger actually on top of the processor with the intention of cooling that heat, moving that away with liquids, but there's still 30% of the energy has to be removed a different way. And typically that means you're running two— an air-cooled system right beside a liquid-cooled system in the same server, which is expensive and wasteful. It gets the job done, but it's not very easy. And the third, a third way is what they call tank immersion, where you actually invest, you install the boards in a tank of dielectric liquid. It's bulk cooling for the most part, or you can use a refrigerant in the tank and that becomes a whole different story. And then there's our technology, which is a rack-based immersion technology, which is what I think will win the race at the end of the day. But, you know, only time will tell.

And to come back to the question about the fans, right, because we've all used fans at some point. I don't know, we were down on vacation in Mexico and we used a fan. The fan only— well, Well, it touches your skin and then there's evaporation going on and that cools you down. But at some point, especially if the air doesn't touch you, the fan and the ventilator actually heats the room, right? So it makes it—

Yeah, that's exactly right. The only reason fans work, as you just described, is because of the evaporation.

Right. But we can assume that if inside like a data center and inside a rack, the fan actively also heats the whole thing. So, right, it pushes the air out, but then, I mean, everything is pretty hot in like a data center because of that. So at the end of the day, does it really dissipate the heat? So why is it still being used, if you play, uh, devil's advocate here?

Why are they building data centers in Phoenix, Arizona? I mean, the truth is The truth is, if you were an engine— if you were citing a data center and you were serious about it, you would, among the boxes you would check, are: is there water? Now, we don't need water, but generally most of the technologies do. Is it— is it cool? Is it cool outside? Yeah, and of course there's other boxes you check as well, but those two would be— you'd be It's surprising to me that there's— they're locating data centers in some cases where they are, because it really adds to the problem. And frankly, a lot of the opposition to data centers because of the power they need or the air, the water they consume is justified. It's— the engineers are not doing anybody a favor by using a technology that's basically obsolete. There are solutions to this problem, including ours, but they're not easy. They're being only gradually introduced to the marketplace now.

Right. But if we think of it from a cost perspective, right, like fans are incredibly cheap, running water tubes throughout the full data center and having a system to control that full water is way more expensive, right? So isn't that like a clear cause, like, because it's an upfront investment? Is it like the solar electricity dilemma? Like, should I install some panels on my roof? Because yeah, at first I'm gonna have to pay a lot, but then eventually I'm gonna save. Is it the same economic— economical dilemma?

Well, no, and the reason it isn't is because air cooling is more expensive, not less. Air is—

installation, the installation of the full system is pretty costly, no?

No. So here's an example. In the old days, 5 years ago or 6 years ago, a hyperscale data center was 30 megawatts. Now they're talking about 300 megawatts or even a gigawatt, which is 1,000 megawatts, which is incredible. But, um, so Facebook at the time, now Meta, built a data center in Iowa, and it was a well-designed data center for the time, around 2016 timeframe. So 30 megawatts at that time was considered a hyperscale facility. In order to cool it, in Iowa where it doesn't get real hot all year round. They are moving from the central air handling system 1.5 million cubic feet of air per minute through that facility. The, the fans in the racks, which we eliminate completely, liquid-cooled solutions, are moving another 5 million cubic feet of air per minute. So there's 6.5 million cubic feet of air per minute moving through that facility to cool those servers. That's crazy. The amount of heat they generate, the fans generate, use 40% of the energy. Just cooling the fans, running the fans and cooling them uses 40% of the energy of the whole data center. It's nuts.

I asked my AI while I was asking you, Well, the first thing is that yes, the AI revolution brought like much more cooling needs. So yeah, you check that box. But it says, um, retrofitting existing facilities is enormously expensive, right? So not true.

That's not— we do the plumbing. No, like, uh, AI slops that we're talking about here. It's Fable 5.

It's Fable 5. Like That shouldn't be—

it turns out, it turns out that, uh, well, it's unfortunate. I wonder if I can share my screen here. Let me see.

You can, there's a plus icon, but do explain while you do that.

Well, so, um, here are the slides and PDFs.

Um, so you don't have to like move all kinds of pieces. I have an existing data center refitting it with water cooling wouldn't necessitate me to move boxes around and wouldn't cause downtime?

No.

That's crazy.

Tell me if this shows up here. Let's see. It's processing. It's making progress here. This is— you see it?

Yeah.

Okay. No, I mean, I learned something. Thanks. Uh, so So right now what's happening is, well, for example, a new data center today, greenfield data center, costs, in addition to the IT equipment, just to build it is about $13 million per megawatt. So when you're talking about a 300-megawatt facility times $13 million, that's the kind of number to build the facility. To support the data center. Now, in our case—

billion, right? 10 billion or something? 20, 30 billion, right?

And the IT equipment would be even more expensive. So these are huge bets. So in our case, um, ours is half the cost because we literally eliminate the infrastructure you need for air cooling except for ventilation. So there, here's what a data center looks like in our world. Now, what's happening is because the, the chips are so hot, existing data centers, if they want to continue to operate in, in, in the marketplace, have to add liquid cooling to the, to their system. It's now not an option, it's necessary. So for decades, the operators did not want to bring liquids into the white space. Now they have to. So in our case, the only infrastructure we need if it's cooler than 110°F outside, 45°C, is a dry cooler, which has got fans in it and radiator basically. And the pumps we need to move the facility coolant, which is typically water with glycol added for antifreeze. To the racks. So this is what our system looks like. Now, typically the dry cooler and the fan, the pumps, are operated and owned by the data center operator. In our case, if it's— the heat recovery is becoming a big topic, and there's some great applications like desalinating seawater and such as that. In our case, if you have a concurrent need for heating and computing, for hot water and computing, we can recapture over 90% of the input energy in the form of 140-degree water. So it's pretty effective.

So you rebuilt the full technology from scratch instead of tubes, is like these liquid-cooled server racks. So you've built that.

Yeah, so that you— so you'd have pumps and you'd have pipes to get to the racks. So that's the— that's the— so let me see if I can go to the next slide.

And you keep the fan, right? Because if one fails, then the other can still be a backup. Is that the thesis?

Well, you need fans for ventilation, and you need fans for if there's direct-to-chip systems in this facility, which we view as— frankly, I view as a transition technology. Okay, you need fans to cool the 30% of the heat of the board that isn't covered by the chip, the processor heat exchangers. So one of the other things to think about is that air is actually not very friendly to electronics. If you open up a server after a few years, or even a few months, It's got dust in there. It's got particulates in there, pollution.

Yes.

And it's got oxidation. There's corrosion, because air is not a friendly medium for cooling.

And don't they filter air inside data center? Don't they have like special ventilation systems?

Well, they use the white space. It's a clean room to a large degree, but still, it's not perfect. And also, The problem— there's a problem of oxidation that occurs when air touches electronics.

Couldn't they have like a special kind of air that actually doubles the cooling inside these white rooms?

No, not really. I mean, they, they, uh, there are some electron— there's some refrigeration concepts that are being employed, um, two-phase cooling, but fundamentally air is basically filtered and it runs through and then it's exhausted.

Okay, got it. All right, go on.

Uh, so the company, we shifted from sci— we have 72 patents on our technology globally, and the company shifted from science to sales a couple years ago. So there are now, there's a catalog commercial products, but the, so the, it's located in Minnesota. Among the things— and it's been tested by Lawrence Berkeley Labs, by what was then called NREL, the National Renewable Energy Lab, now it's called the National Laboratory of the Rockies. It's been tested by Intel. We're one of the 1,000 Points of Light by the Solar Impulse Foundation. So it's, it's been pretty well vetted by laboratories in our own practice as well, in the field and in the lab. During the development, the science phase, we built a— we installed a rack of our servers in the back dock of our facility in Minnesota and ran it for 5 years at 100% CPU load, the whole rack of all the servers. And we just, during the course of that, we— this is one of the servers that we took apart after 3 years just to look, see what it looked like. And as you can see, having run 3 years, 24 hours a day, 7 days a week at 100% CPU load, you can see that all the parts look like they're brand new.

Okay, let me ask you questions. So the common data center, AI data center nowadays is 300 megawatts, right? Like that's the most common one. If I start one of these from scratch, right? What do I have to pay you initially? Are there installments planned as well? And how much can I save per month if I would use you versus if I wouldn't use you?

So, so the— those are important questions. So you would, you would— the cost of a brand new data center using this technology, it's not— it's— you wouldn't pay us, but using this technology, the kind the cost of building it is about $6 million per megawatt instead of $13 million. The maintenance cost is reduced by 90%. Okay. The energy cost is reduced by, at least by the lab testing that's been done, by 40%.

That's insane.

And it is insane. And the servers will last a lot longer. Now there's going to be a replacement for technological obsolescence, but those servers that would have otherwise been discarded because the parts would have been oxidized and such, you can— secondary market for them.

Your lab test, were they done on the latest, I don't know, Blackwell chips or—

Well, we are using GPUs. Blackwell chips are pretty pricey, and they've gone beyond that now. So we are working on a server, a 6U server that would fit in a standard rack for a major worldwide automobile manufacturer that would use 8 GPUs, but we were talking about H200s. That may have gone up now to the— Gotcha.

To the— Isn't NVIDIA building its chips with cooling in mind? Wouldn't they integrate these mechanisms in the near future, right now?

Well, NVIDIA— what NVIDIA is doing is they're using direct-to-chip, direct-to-chip cooling, and they're using air, principally air, or alternative liquid cooling system for the rest of the board. We would simplify that immensely by just taking care of all of it with the same fluid. So all of the components you're looking at in this, in the slide, have been immersed for 3 years. And this cold plate, or the, the heat exchanger, sitting on top of the processor— there's a 20-watt pump in each server, and that 20-watt pump pushes the fluid using what we call forced convection to pick up the heat in the processor, but it also removes all the rest of the heat in the server, and it moves it to a liquid-liquid heat exchanger at the back of the rack. So what's happening is our cooling distribution unit is basically a series of liquid-liquid heat exchangers on the back of each rack. Every server is independent, so you can slide a server out or push it out or drain it right inside. It takes about 3 minutes to drain it with a pump kit we provide, and you can slide it out and, and all this other servers work fine. The only thing the contractor needs to do is to hook up the inlet and outlet to the facility coolant when he's installing the rack.

Got it. Is that the end of the presentation?

I think so, if I can figure out how to get out of it.

Yeah, yeah, okay, I'll just unshare it. Why isn't— why is not everyone jumping towards this? And I I mean, it's pretty significant the cost you can have him save. So why isn't like all the big data centers flocking your way?

Well, I think they will. I think it's a matter of showing it at scale because it's one of these things that seems too good to be true. I mean, what's happened is with the, introduction of the AI chips, which are too hot for air, people were looking for a simple solution that they can implement immediately. And this has happened before in a different context. If you think about lighting, for example, the, you know, incandescent lights, which were 90% heat, 10% light, were around for 100 years. There was a need to change that because of the because it was wasteful energy-wise. And so I think most people realized early on that light-emitting diodes, LEDs, were going to be the endgame, but they weren't quite ready yet. So this whole compact fluorescent, the little spiral bulbs, that whole market blossomed, and factories were built and patents were filed. But that whole, whole thing only lasts about 4 or 5 years. I mean, when was the last time you could find a CFL in a hardware store? Because it's not as good. It doesn't solve all the problems, whereas the LEDs do. So we're in a situation, we, we, we would— the metaphor, we're the LED, we think. We like to think we're breathing our own exhaust for a long time. We're the LED in this transition period where cold direct-to-chip is the transition technology. In our view, it's got to go away because it's too expensive. It's too much. If you look at the wiring and the piping and everything else involved in having two cooling systems for every server, it's ridiculous.

And your story, right? You were around for a while, you had an exit with dignity, you said. Became an angel investor, um, but then you started seeing like the AI revolution happen, right, like around 2019, and then you're like, I need to go all in into this, right? Was there like a revival in the business and a shock of like, yeah, this is, uh, another big shot, a potential home run for me?

Well, that's that. I mean, the original idea was to fix the problem of data centers, having watched the way they were built and operated before when I was running Trigen. But the market is taking care of itself, and we believe that we're being swept up along with all the other technologies, and we're getting traction. I mean, you know, our pipeline— one of the important things to know is that we started with a clean sheet of paper. I call it Occam's razor, was the principle that I learned in engineering school. Start with a clean sheet of paper, keep the number of components small, and, and see what you get compared to what's out there now. And frankly, as I mentioned before, people who are in our industry think if you can cut the price in half, if you can cut the energy by 40%, cut the space by 70%, cut the maintenance by 90%, you'd think, as you mentioned, they'd be lined up around the block. But we, we're trying now to prove it at scale. So we're working on a data center in Maine that would showcase our technology, uh, and we're also working with other vendors. We have other configurations Because once you cut the tether from all the infrastructure you usually need, we can literally— if there's power and connectivity, we can literally go anywhere, right? So that's the important point. We can go anywhere.

So you still need a case study to, yeah, like Trojan horse yourself into the, the next ones.

And then we believe that the Yeah, you must have seen Odyssey just as well.

Uh, see the movie?

Uh, yeah, the Odyssey movie.

So yeah, uh, I'm about to. Apparently it's very good.

It's, it's worth— it's 3 hours of your life, but it's pretty good. Yeah, it's okay.

Perfect. Yeah, I'll definitely go check it. But no, Trojan Horse, I've used that. Okay, another good one is Beachhead, right? It's like—

right, the Beachhead is right. So the Beachhead, what we're What we're doing now is pilot projects. We know these are big bets. We know it's hard to believe that it could be as good as it is, and so we did a pilot project with an electric utility. We installed a small rack— they installed it, actually— in the turbine room of a coal-fired power plant, and we charged them for it, and then at the end of the test, we bought it back. And if we didn't do it that way, the test probably would never have ended. So we— the reality is they, they were so impressed with the results that they bought 40 servers, and they told us, these are the processors, this is the storage, this is the memory we want you to build into these servers. And so now what they're doing is they're installing these servers as other equipment comes up for refresh in existing racks that they already own. And so it's a very easy transition because you can literally migrate from air to liquid one server at a time in the rack you already have.

Right. Selling to these guys, uh, is pretty hard as well. It's, um, call them, uh, hyperscalers, right? Data center operators, AI infrastructure companies, enterprise CIOs. Like, uh, it's definitely a hard, uh, GTM play. What's your plan to get in touch with them?

The—

what's beachhead in the GTM cell side of things?

Well, that is the operative question. So we're working principally right now on relationships. You need to find that individual in these companies that is willing to try something a little different on a small scale. So what we're really looking to do right now is pilot projects where there's a relatively small investment, for example, on heat recovery. We're working with a water heating manufacturer that their— the proposal is they buy a rack from us for testing purposes for $40,000, and at the end of the test, we'll buy it back for $25,000. And they're going to develop an appliance, but from their perspective, I mean, in hospitals and university campuses and places like that where there's a concurrent need for hot water and computing, the, the product they would have otherwise sold, a hot water system, might be $5,000 or $10,000. Suddenly they're selling a $500,000 system that can be really installed by a plumber and electrician in a day. So that's being tested right now. So there's all kinds of things that are— I mean, it's just a whole different way of looking at the world once you know that you can cut the tether all this infrastructure that really never should have been there in the first place.

And are there competing solutions? Um, I mean, not the same exact system, right? But like one system that can achieve similar results, the one you mentioned, so that minus 90% maintenance, minus 40% cost. Are there competing existing solutions to, uh, what you're building?

There's one, uh, there's a company in the UK that actually stole our technology, frankly. We were at a computer show called SC15 Supercomputer Show in Austin, Texas in 2015, and this company had a, had a booth next to us in the emerging technologies room. And a year later, our technology showed up with their name on it. Anywhere there's litigation involved with patent infringement, and we won. So we'll see what happens there, but it's a gigantic market. There's room for a lot of people in it. So the only other company offering a similar approach is really this firm in the UK. We think we, because of the patents that we have, we have an advantage even in Europe. But, you know, only time will tell.

Right. What's your role in the company? Like co-chairman, it's pretty broad.

Vice chairman. I'm in charge of all the vice. I'm the vice chair.

What does that mean?

Well, we have a small board and I'm— we have obviously a chairman of the board and I'm the second in command.

Okay, so you analyze decisions and they come to you when they have important decisions to make and you advise Right. Okay, okay, got it. Um, what does a day in your life look like? You're also like a board member at Milwaukee Symphony Orchestra, so an admirer of music, probably someone that plays music. What does a day in your life look like nowadays?

Well, today I'll be out in the yard after this call. Uh, a day in the life is, is interesting because there's so much It's in the news, and people are contacting me like you did, expressing an interest in whether this has legs or not. But, you know, my principle— I mean, I could have retired when I exited the other company, frankly, but this is too juicy an opportunity. I mean, you have a universe of people doing stupid things. Really. Now, maybe we aren't the right answer for everybody, but we are the right answer for many people. And this has happened before. The horse manure crisis of 1894 is a funny story, but basically, if we have a few more minutes, the London Times published an article that said in 1894 that unless something is done, The City of London will be under 9 feet of horse manure in 50 years. And of course, all the cities had this— major cities had this problem. And so an international conference was called in New York City in 1896. They couldn't solve the problem. However, the last horse-drawn trolley in Manhattan was retired in 1913 because while the people in New York were worrying about horse manure, Henry Ford in Michigan was worrying about, was thinking about the horse. He was looking at it from a whole, from a clean sheet of paper. Let's start from scratch. What would we do today knowing what we know if we were trying to build a system? And the entire world flipped in 15 years. It's astounding how fast that can transition when the old solution is just inferior in every way.

The horseshit analogy, uh, is intriguing for sure. Herb, thanks for being on the podcast. Where can people find out more about you?

Well, they can look at Liquid Cool, www.liquidcoolsolutions, one word, liquidcoolsolutions.com. And I'm at herb.zien@liquidcoolsolutions.com.

That was