The Picks and Shovels Businesses Quietly Driving the AI Build Out

Today on the AI Daily Brief, we're talking about liquid data center cooling and other interesting opportunities in the AI buildout. Before that in the headlines, Meta says that AI has helped its ad revenue. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show. Notes. Welcome back to the AI Daily Brief Headlines edition. All the daily AI news you need in around five minutes. We kick off today with yet more reporting from earnings week, where the market is responding to Meta's earnings in a more positive way than they did earlier in the week with Microsoft, CNBC writes Meta's advertising growth is proof that hefty AI spending is already paying off. This of course, gets to the big Question, which we've been following all summer and which is also the subject of today's main episode around ROI and AI hype and market cycles and et cetera, et cetera, et cetera. But in this case, Meta's advertising sales in the last quarter increased 22%, which was double the growth rate at Google. Zuckerberg said that AI is quote, improving recommendations and helping people find better content as well as making the advertising experience more effective. Said Zuckerberg, those are already products that are at scale. The AI work we're doing is going to improve that. Overall, Meta's revenue was up over 39 billion, with 98% of its sales coming from advertising. CNBC writes that Zuckerberg pointed to AI as the foundation behind Meta's refreshed online advertising platform, which was battered after Apple introduced an iOS privacy update in 2021, said Mark Mahaney, Internet analyst at Evercore ISI. They rebuilt their ad tech stack using AI and they changed their user interface and generated a lot more user engagement because of AI. It's showing up in the revenue and profits now. After the report, meta shares were up 7%. And so now with this, we have the second area after Cloud where AI is actually showing up in roi. Now, do I think that this will make much of a difference when it comes to the narrative competition around whether AI is in a bubble or not? No, because I don't really think that this comes down to evidence that people are seeing. I think it's much more about preconceived notions and previous pattern matching. Still good to see that Meta, who is investing so heavily, particularly in open source AI, is leading not only in their models but also in their revenue performance coming from AI. One additional thing, Zuckerberg repeated some of what we've heard from other CEOs like Google CEO Sundar Pichai when he said it's hard to predict how this will trend multiple generations into the future, but at this point I'd rather risk building capacity before it's needed rather than too late. Next up There are rumors that XAI is considering acquiring Character AI, but Elon Musk has denied them. The report originally came from the Information, who tends to have better sources on this sort of thing than just about anyone else. And I think that part of the reason that the story got traction hold aside the details is that it does seem to reflect something that people think, broadly speaking, is coming, which is of course a wave of consolidation, as AI companies that have started over the last couple years and have spent tons and tons of money have to figure out either business models that actually work or new sources of capital or or frankly, just a new place to land, the Information writes. That discussion may not result in a deal, but the internal debate at XAI points to the types of pairings that are likely to become more common as smaller AI startups such as Character face the steep costs of training and running their models while competing against deep pocketed rivals. There is also a talent dimension of this, of course, there's only so much AIML talent available with top engineers, and fascinatingly, it does seem like something is going to happen with Character, the Information says. An offer from xai, should one materialize, would end months of uncertainty over Character's next steps. Although it isn't at risk of running out of cash soon, it hasn't raised a new round of venture funding in more than a year, a contrast with the rapid funding pace of many AI startups. Character in recent weeks has sought to cut costs and has instituted new processes for approving budgets and headcounts, according to an employee. Ultimately, Elon used X to say that they are not considering this acquisition, but who knows? An update from a Previous story the EU's AI act finally goes into effect today, four years after it was initially proposed, CNBC writes. The legislation applies a risk based approach to regulating AI, which means that different applications of the technology are regulated differently depending on the level of risk they pose to society, said Charlie Thompson, a VP at apn. This will bring much more scrutiny on tech giants when it comes to their operations in the EU market and their use of EU citizen data. Now, notably, one thing that is disallowed under the AI act are things including social scoring systems that rank citizens based on analysis of their data and predictive policing. The predictive policing was interesting to me because the Guardian was reporting that Argentina is considering using AI to predict future crimes, they write. Argentina security forces have announced plans to use artificial intelligence to predict future crimes. This week, new President Javier Milei created the Artificial Intelligence Applied To Security Unit, which the legislation says will use machine learning algorithms to analyze historical crime data to predict future crimes. It is also expected to deploy facial recognition software to identify wanted persons, patrol social media and analyze real time security camera footage to detect suspicious activities. Unsurprisingly, there are lots of folks who are very concerned about this. Amnesty International Argentina says large scale surveillance affects freedom of expression because it encourages people to self censor or refrain from sharing their ideas or criticisms if they suspect that everything they comment on, post or publish is being monitored by security forces. A new study from the ILO and the World bank said that between 2 and 5% of jobs in Latin America and the Caribbean are threatened by AI in a way that makes them at risk of being fully automated. Overall, they think that between 26 and 38% of jobs in the region could be exposed to generative AI and impacted by it in some way. The report also suggests that AI could improve the productivity between 8 and 14% of jobs. Lastly, today Taco Bell is set to begin rolling out AI drive thru ordering in hundreds of locations by the end of the year. Taco Bell is owned by Yum Brands and apparently hopes to roll out AI to all of its drive thru lanes in the future eventually. Now other companies have tested similar technology including Wendy's and White Castle. However, not all of these experiments have gone well. McDonald's ended its trial of Automated Order Taker which was an AI that had tested in partnership with IBM. Worth noting however, that it seems that the issue might not have been AI in general, but the specific partnership as McDonald's says that they're planning to turn to other vendors instead. I fully expect that the first time you get an AI ordering experience at a Taco Bell you come over here and tell us. For now though, that is going to do it for the AI Daily Brief Headlines edition. Next up, the main episode. Today's episode is brought to you by superintelligent. As you guys know, Super Intelligent is a platform we are building to help everyone, individuals and teams maximize their use of AI. We help you figure out how to use AI tools as well as what to use AI for. And this is really important. The whole goal of Superintelligent is not just to give you tutorials and lessons, but to show you how other people like you are actually getting value from AI right now. For those of you who are still out there working, learning and grinding deep in the summer, I'm excited to share our best offer ever. If you sign up with code YEAR50 right now, you will get 50% off the already reduced annual price. Due to popular demand, we have extended this offer for just a couple days. We were going to close it at the end of July. Now we are running it through Friday August 2nd. But if you want this 50% off discount, head on over to be super AI and take advantage right now. Like I said, this will close on Friday August 2nd. The code one last time is year 50 for 50% off the already discounted annual rate for superintelligent. See you there. Today's episode is brought to you by Venice. Venice is a private, uncensored, generative AI app. It accesses open source models to enable text, image and code generation without the fear of being spied on or having your data exploited. Discuss anything with Venice without concern about it being monitored, sold or given to advertisers and governments. Venice is different because your conversations and creations are kept securely within the browser, never stored or accessible by Venice. Unlike other AI apps, Venice won't tell you what's okay to say or not. Venice won't patronize you. It simply provides direct access to machine intelligence. No topics or off limits, no ideas are taboo. With Venice, you're in control of the AI as you should be. Pro subscriptions are available for $49 a year or $8 per month. AI Daily Brief listeners receive a 20% discount on Venice Pro. Visit Venice AI NLW and enter the discount code NLW DailyBrief. That's NLW Daily Brief, all one word. Welcome back to the AI Daily Brief. One of the big themes for quite some time now, certainly one of the biggest discussions of this summer has been whether the market has gotten out ahead of itself when it comes to AI valuations. Now, this has sometimes been translated broadly into a larger question of whether AI is overhyped. But as you've probably heard me say before, AI, I think it's largely about specific market dynamics and the fact that Wall street is trying to figure out how to price the future in stocks that are available to trade today, a thing that they're very uncomfortable with. Now. This week has been kind of an interesting one on that front as we've had sort of a mixed story. On the one hand, we saw that Microsoft's results earlier this week were sort of a mixed story. AI was increasing as a percentage of Azure revenue, but Azure revenue didn't quite hit the estimates. But then Meta, as you heard in the headlines today, started to show how AI was paying off in the form of its advertising revenue. And in between all of this, we also had a big burst for Nvidia yesterday. Nvidia shares were actually up 13% on Wednesday, although the exact reason for that I think is a little bit debatable. On the one hand, it could have been about Nvidia. Specifically, CNBC characterizes the jump as happening after, quote, remarks from top customer Microsoft and rival chip maker AMD signaled there wouldn't be a slowdown in the multi billion dollar build out of AI servers based around GPUs. Another way to look at this, if we're putting on our skeptical hats, is that even if there is a bubble in the hyperscaler buildout, in other words, companies like Microsoft and Google not being able to get the return on their investment in AI. In the meanwhile, Nvidia and other chip makers continue to benefit because they're cashing checks all along the way. UBS analyst Carl Kirsid wrote in a note. Microsoft earnings report may encourage most Nvidia and semiconductor investors as Microsoft's CapEx came in much hotter than expected at 19 billion in the quarter. Other analysts argued that their peers weren't thinking about growth and AI in general the correct way. Morgan Stanley analyst Joseph Moore, for example, wrote, our perception is that the market is taking a very glass half empty view of some of the hyperscale comments where there is a clear desire on the part of customers to continue to commit resources to developing multimodal generative AI. In other words, this particular analyst is arguing to some extent that other investors may be missing the forest for the trees. Now, it wasn't just Nvidia who had a good day. AMD reported better than expected sales and earnings and CEO Lisa Su said that the company expected 4.5 billion in AI chip sales this year year, which was an 11% increase over what it had previously estimated. Goldman Sachs analysts said, we believe AMD's positive data center GPU business outlook and Microsoft's comments indicating a sustained increase in capex through fiscal year 25 bode well for Nvidia. Now of course we also had a very dovish FOMC meeting that seemed to all but confirm that rate cuts were coming in September. So that also could have had some impact on the particular pop, but I think that the larger dynamics remain. What's more, the betting odds are that there's a lot more AI infrastructure build out to come before any Even theoretical bubble pops. One of the big stories earlier in the year was OpenAI CEO Sam Altman flying around the world trying to convince companies to go in on a $7 trillion scheme to increase AI chip capacity. And so the question that we want to ask for the rest of this show is to the extent that there are questions around whether the hyperscaler big tech companies themselves, the Microsoft stocks, the Google stocks, et cetera, are the right place to bet on AI. Are there other less obvious areas? I will also use this moment to remind you that nothing I ever say on this show or any other is financial advice. But one interesting question might be what else goes into data centers other than just the chips themselves? In May, Goldman Sachs posted a report. AI is poised to drive 160% increase in data center power demand. They write, on average, a ChatGPT query needs nearly 10 times as much electricity to process as a Google search. In that difference lies a coming sea change in how the U.S. europe, and the world at large will consume power and how much that will cost. So basically, this report estimates a few things. First of all, they're betting that there will be a significant increase in the percentage of global power that data centers consume, from 1 to 2% today to 3 to 4% by the end of this decade. This, of course, has lots of implications. As Goldman Sachs points out, since 2020, the efficiency gains appear to have dwindled and the power consumed by data centers has risen. Some AI innovations will boost computing speed faster than they ramp up their electricity use, but the widening use of AI will still imply an increase in the technology's consumption of power. By 2028, they expect that AI will represent around 19% of data center power demand. This has other implications as well. US power demand, which has been around zero for the last decade, is likely to rise 2.5% over the next decade, which, as they point out, is a type of spike that hasn't been seen in the US since the early years of the 20th century. And so, as they point out, there is going to be a lot of work to get data center infrastructure and the power grid ready for this sort of new demand. They estimate that Europe needs a trillion dollars plus of investment to prepare its power grid for AI. And of course, if they're right, that creates lots of interesting opportunities for investors to hone in on. Now, one of these little areas that I've recently been exploring is data center cooling. A recent report called Moving beyond the Limitations of Traditional Data Center Cooling digs into the state of this field and how it's Changing Analyst Stephen Hill in spite of the remarkable advancements we've witnessed in the world of computing over the last few decades, one of the most persistent challenges that still remain for data center operators lies in the basic problem of removing heat. It's a simple fact that as servers and other infrastructure devices get faster and more powerful, they also consume an increasing amount of electricity. And as everyone in the IT industry already knows, the majority of the electricity that passes through a computer exits in the form of heat. For data centers, it also requires nearly the same amount of energy to remove it. This, he said, leads to the delicious irony of computing no matter how powerful our systems get, one of the key challenges still comes down to the relatively primitive issue of heat management. Hill then goes through what he calls the options air cooling he identifies as ubiquitous but inefficient. He writes, the majority of data centers have continued to depend on the tried and true raised floor model of air cooling. Much like a refrigerator, raised floor cooling depends on the circulation of cooled air throughout the entire machine room. Hot air rising from the systems is gathered from the ceiling and passed through computer room air handlers, who which are devices that are located at strategic locations at the periphery of the room. CRAH is based on chilled water or computer room air conditioners based on refrigerants, cool the air and then deliver it below a tiled floor that's raised with a metal framework between 24 and 48 inches above a concrete subfloor. One of the problems, he points out, is that you can only push so much air through the holes in a given tile. This has become more of a problem as the design of data centers have changed, hill continues. The majority of high density data center operators have adopted air as the model for cooling massive cloud data centers, likely because the substantial evaporating cooling solutions they require continue to be energy efficient at scale for moderate workloads. But he says this also presents problems because of the extraordinary water consumption requirements of these facilities. A large cloud data center can consume billions of gallons of fresh potable water per year. Now you, the listener, may have heard this start to come up as one of the critiques of the environmental impact of AI. And so again, we're left with the question of what other solutions might there be. The next that Hill points to is water based direct liquid cooling, which he calls better but challenging. Hill points out that water is about 24 times more efficient than air when it comes to transferring heat. But as he points out, water based cooling isn't without its challenges simply because water and electricity don't really mix particularly well. Hill writes the majority of in server water cooling is based on the direct to chip or D2C model where pressurized water is passed through a water block covering the chip and then returned to a radiator or a chilled water Source for recirculation. D2C water cooling, he writes, is clearly an improvement on air only cooling for hosting higher density applications within an existing air cooled data center, but he writes it's hardly common and almost never used as the primary source of cooling. Next he comes to dielectric liquid cooling, which he considers evolving as a viable option. He writes one of the earliest uses for refrigerant cooling was the Kray 2 supercomputer that was introduced in 1985, an incredibly powerful system for its time. But the only way to keep the extremely high density circuit boards cool was to bathe them in a flowing dielectric coolant that wouldn't react with energized components. The fluorinert coolant developed by 3M was pumped through the chassis and then bubbled up through a dramatic see through waterfall chamber almost as large as the computer itself, where the refrigerant reverted back from gas to liquid for recirculation. It seems that everything old is new again on a revolving basis in it, and the same holds true for refrigerant based cooling. Based on a similar principle as our air conditioners and refrigerant freezers, this model leverages a cycle where heat is extremely efficiently transferred through the evaporation and condensation of a fluid with a relatively low boiling point. One of the versions of this he points to is called immersion cooling. There are two varieties of this single phase and two phase, which he say both require a substantial tank filled with liquid, with the main difference being whether or not it's based on the evaporation or condensation process of refrigeration. Now the latest entry he says, is something called refrigerant based 2 phase DTC cooling. Try saying that 10 times fast, he said. It offers some of the best attributes of earlier options, in particular the way it leverages the heat transfer enabled by the phase change of the refrigerant fluid from liquid to gas and back again in a closed cycle. Like a water based DTC system, primary cooling is provided by a cold plate heat sink that is attached to the primary heat sources such as CPUs, GPUs and potentially any other high density ICs that may become major heat generators. However, unlike water based systems, 2 phase can remove heat by taking advantage of both sensible and latent heat evaporization. It also requires four to nine times less flow rate at the chip compared to a single phase water based cooling. The example he points to is a company called Aselsius Nucool Platform. He writes, their process is specifically optimized to remove heat from relatively small electronic components. The cold plate, known as the vaporator, in contact with the processor is supplied with a liquid refrigerant where it converts to gas in the presence of heat. The warmer gas flows through a manifold as part of a closed loop to an integrated heat exchanger where it condenses by transferring its heat to facility water or another refrigerant. And then there's a heck of a lot more about this specific technology now. In January of this year, Datacenterdynamics.com wrote a piece called 2024, the Year of Liquid Cooling. The tipping point for liquid cooling has arrived, they say, with a few choices on technology in that they quote Josh Klayman, who's the CEO of that company, as Celsius, who says liquid cooling as a sort of pragmatic solution has always been niche, but we're seeing the dynamic rapidly changing. Mainstream CPUs are becoming much hotter after years of incremental increases in wattage. So suddenly we're seeing a J curve in terms of the heat that these chips produce. Now there's a lot more that's interesting about this. I kind of went down a little bit of a rabbit hole. But the point relative to this show, which is obviously much more technical than we normally get, is that I believe that one of the things that's going to happen as Wall street digs deeper into these questions of the short, medium and long term of AI, I think that whatever conclusions they come to when it comes to hyperscaler valuation and even base level infrastructure valuations in the form of companies like Nvidia or AMD or intel, they're likely quickly going to expand their field of view and horizon to other parts of the infrastructure buildout that are going to enable this whole transformation. Which regardless of what Wall street thinks, the hyperscalers don't seem to show any signs of slowing down anytime soon. I think that there's going to be a lot more focus on companies like this as Celsius and other competitors in that space. And I think there's going to be really interesting opportunities to start to dive deeper and figure out what that infrastructure build that's actually going to look like. Anyways, guys, that will do it for today's episode. Like I said, a little bit more exploratory and ponderous. But since it's the summer and since we've been having this conversation over and over about bubble or not, valuations too high or not, I think digging a little bit deeper, peeling back the surface, understanding what else is happening, what industry areas are coming to the fore to actually enable this transformation, is a really interesting place to spend some time. For now, though, that is going to do it for today's AI Daily Brief. Until next time, peace.