#205 Nick Loui on Transforming Chaotic Data into Actionable Intelligence

Welcome to The Cognitive Crucible, produced by the Information Professionals Association. Our website is information-professionals.org, where you can find links and information about today's conversation and get access to members-only content. Join John Bicknell and explore all aspects of our generational challenge, cognitive security. The Cognitive Crucible is a forum that presents different perspectives and emerging thought leadership related to the information environment. The opinions expressed by guests are their own and do not necessarily reflect the views of or endorsement by the Information Professionals Association.

My guest today on The Cognitive Crucible is Nick Louie, who is the co-founder and CEO of Peakmetrics, a leading company specializing in AI-driven narrative threat intelligence. Nick Louie, welcome to The Cognitive Crucible.

Hey, John, good to see you, and thanks so much for having me. I'm a huge fan of the IPA and all of the events. So, uh, very excited for this conversation.

Excellent, excellent. Likewise. So the, the conversation I'd like to have with you today, Nick, will unpack how Peak Metrics helps clients transform chaotic data into actionable intelligence. But before we get into this topic, could we start by getting your assessment of our strategic landscape, or if you prefer, the times in which we live?

Yeah, absolutely. I, I love this question because I think it's it speaks so much to all the different perspectives of, of such a complicated, you know, broad issue. Uh, the way that I think about it is really by looking at the multiple phases of the internet. Uh, so my old boss would actually start this back to, you know, to like communication and cavemen, which, which I won't bore you with. I'll really focus on the different phases of the internet. Uh, so to me, there was basically this rise of the, the first generation of the internet. We successfully brought e-commerce online. We successfully brought news publications online. But if you think about the media ecosystem, it was really still a reflection of, of that more traditional top-down media ecosystem in that first generation of the internet. Over the last 10 to 15 years, we saw the rise of social media and how that was able to give voice to the people. We saw how everyone can become their own mouthpiece. I, you know, some of my original roots were working in basically the intersection of entertainment technology. So I did a lot of work around celebrity leveraging their voices online and how they could create online influence more directly. And perhaps most interestingly to me, actually more recently, was what I believe to be a reversal of the flow of information. So whereas in the past you had traditional media sort of dictating content and then spreading to social, What we've seen now is how it's a bottoms-up ecosystem, and you have the rise of social media actually dictating the news cycle, or perhaps adjacently related, this concept of the trumpet of amplification from Claire Wordle out of First Draft, where it's this flow of information from anonymous web to fringe web to then eventually professional media. And that's that pattern that we see again and again. And now I think we're in this completely new generation of web, which is basically the rise of generative web. So if you look at studies, we're seeing how ChatGPT is already proliferating content on the internet. Some 90% of online content is going to be synthetically generated in the next couple years. And on top of that, we're in this world that is, that the synthetic content is of course just infinitely more believable than previous generations of AI-generated content. So I think actively right now, we are going through the start of sort of the next generation of the internet. And there's a whole bunch of implications that I can speak to, but maybe I'll pause there before I keep going.

Yeah, yeah, that's a really fantastic perspective. There's a guest that is coming up. The episode has not aired yet. By the way, to our audience, we're recording this on October 10th, 2024, just so people can have a marker in mind. But this fellow talks about information as having like a demand side problem. And by that, I asked him, what do you mean by that? He thinks it's, you know, part of the challenges that we're facing these days is that, you know, people are having a problem either filtering out bad data. And I think you're kind of talking a little bit about that with all the AI-generated content that's coming as well. So a lot of which could be intentionally deceiving. But also, you know, to your points about, you know, the internet going through an evolution, well, people in society are going through an evolution on how they consume content as well. So I'm in my mid-50s. And so when the internet first became a thing, for me at least, it was like in the mid-1990s. We were first able to surf the web, right? That's what we were doing, surfing the web. And you could just spend hours and hours just, you know, checking out stuff. And it was all brand new. It was like, wow, this is cool. But nowadays, I don't surf the web anymore, right? I only go to find something on the internet when I need it, right? So it's a different way of engaging. But anyway, what are your thoughts on thinking about some of the challenges we face as like a consumer demand side problem?

Yeah, I mean, I think that's right is when you look at then the implications and the challenges and you look at these at least 3 competing forces simultaneously, you have the, to your point, fragmentation of platforms. One of the trends that we've been tracking is that post-January 6th, we saw how a lot of what was previously on social media is now moving into your platforms like your Telegrams of the world, your iMessages, your WhatsApp. Plus, of course, the rise of TikTok for the younger generations. You combine that with a national decline in trust and the sort of rise of cynicism from a societal perspective. And then you combine that with all of what we've talked about and will continue to talk about around the potential implications of generative AI. And it creates this very complicated information space that I think is only compounded. On the content side, one of the things that really interests me around this whole sort of intersection is actually that I also think that we're at the beginning of a flywheel for online content where you basically have this situation where AI is, of course, trained on large public information, you know, huge, huge parts of the internet. And then what we've seen is that because of AI, credible publishers, particularly traditional media, are now blocking models from training on their data. Fair enough. That's, that's their prerogative. But what you have now then is that the latest iteration of those models will be continued to get trained on, on that publicly available data, but with a bias towards potentially bad data or even worse, you know, poison data by some of our adversaries that are taking advantage of the fact that there's this exploit. And on top of that, you have a move from right now where the vast majority of software companies are basically using commercial models like the OpenAI models and moving to more open source or alternative models. And, and then, you know, you, all of that creates this a little bit scary flywheel where actually the models we're deploying into production combined with sort of the content that could be potentially damaged creates this really scary flywheel. So I think that, that combination of factors creates a, a pretty fascinating and terrible in some ways, context for sort of the information environment as it's today. With all that said, to be clear, I'm an optimist. I think that this is one of the most exciting and fun fun times to be working in this space, but it is a really interesting challenge.

Yeah, yeah, I guess as an entrepreneur, you have to be an optimist to a certain extent. Otherwise, otherwise you wouldn't be thinking that you could add value or build something new and valuable that the marketplace will appreciate. So for sure, for sure on that.

I think it's a combination probably of optimism, depth of knowledge, and a little delusion, right?

That's right. Hey, hey. You know, right, 100% on all that. Yeah, and I totally agree with you too. Again, back to, I like the way you're looking at things as like an evolution or starting things off as looking at things evolutionary in that people would once just freely or not even realizing that they're giving their data away to now people realize it and now people are starting to do something about it. But it's the same thing organizationally as well. And so organizations are now starting to guard their data and not just freely give away data that can get hoovered up and turned into valuable LLM models. So yeah, a really interesting trajectory there for sure. I wanted to get your thoughts on, you know, speed and the way things are today. So we've all heard the phrase that knowledge is power. And I think this is kind of maybe building off of some of the stuff you were just talking about. But in today's world, it feels more like speed is power. And so how does the sheer volume and velocity of information challenge decision makers? And how can actionable intelligence help them regain control.

Yeah, I think that's completely true. Obviously, knowledge is power, and data is certainly still important. But I think that's completely right that combining that with speed now more than ever is incredibly important. Unsurprisingly to, I'm sure, many of the people in this audience, a typical TikTok trend reaches its peak some 3 to 7 days after it gets published. If you look at platforms like X or Twitter where you have influencers promoting content, a viral trend can spread in the span of hours. And so I do think for practitioners working in this space, there is now this expectation of a fast response time. One of the examples that I think are incredibly relevant today is actually looking at the Kamala Harris campaign, which from a digital and social perspective, I would argue is an incredibly effective strategy. And I think there's a few reasons why it's been successful. One is that the team is run by Gen Z. Gen Z understands.

Is that right?

Yeah, that's right. The TikToker, I should say, they're pretty smart. They fragment it by channel, right? So Facebook is run by millennials. Gen Z is run by, or TikTok, excuse me, is run by Gen Zers. The typical approval process that they try to hit for publishing a social media post is less than 15 minutes. That's how fast they're optimized to be able to execute on digital content for the TikTok algorithm. And then they're also pretty intelligent about how they think about the content strategy. So for example, they will not use a trending sound. For those of you that don't understand how TikTok works, what you can do is create a new TikTok video and you replicate a sound and that's how you basically get a sound to go viral. So one of the things that they will not do is use a TikTok sound that has been used over 200,000 times because what they've realized is that while it seems to make sense that you would want to hop on a viral trend, if it has over 200,000 reposts, by the time your version of that goes live and starts to spread in the algorithm, you're actually old news and you're no longer cool.

From, I mean, just, just, just from a sound perspective.

So, well, yeah. So the way it works is basically you, you create iterative versions of a video. So there could be a sound that's a particular soundbite of someone saying something, and then you can create a new version of that video using that sound. And that's how you get like music to now spread. That's how you get these weird, funny soundbites that are spreading through viral trends. But that's right. Basically, if it's been used too many times, then the perspective of the campaign is basically, well, by the time our video goes viral, potentially, it actually makes us look no longer cool. So we have to have a different content strategy.

Oh, oh, oh, oh, I think I see. Huh. Okay. No, that's interesting.

But I think that, yeah, I— speed, right? That 15-minute approval time. Can you imagine, you know, most organizations, most companies, certainly not anyone working in national security could possibly imagine being able to get an approval through that quickly. But that's the speed at which information flows today. And so, of course, that becomes a difficult challenge. How do you How do you pull intel or, or respond in an environment, uh, that's moving that quickly?

Right. So follow-up question on that, or just making sure I'm understanding. So, um, the Harris campaign, so the TikTok platform, or there's been studies of TikTok consumers, which tend to be relatively young people. Gen Zers, let's say, they've observed that content does not get as much traction or go viral as often if the seed content, the, you know, original clip or the original sound has been sampled too much and TikTok starts starts downgrading that or the consumers of TikTok stop listening or stop watching and listening to it as much as well. And the Harris campaign understands this as a TikTok dynamic and they are making sure that their content production honors that dynamic.

It is. It's actually more a reputation risk to the campaign. So if you think about a viral trend that has sort of this curve, maybe historically a viral trend could go viral and could stick for longer. What you're seeing with TikTok is basically the cycles with which viral trends recycle is so fast that the concern from the campaign is that if they hop on a viral trend, but it starts to spread at the end of the life cycle of a trend, then they are no longer—

I see.

You're a boomer type of—

You're no longer in the cool club. Club, right?

Right. It's a reputation for the, for the account, I guess.

It's like you, you don't want to be the last one laughing at a joke, right?

Or that's exactly right.

Or, or the last one to come to a party, right? It's like, hey, hey guys, I'm here. It's like, uh, the party was just ended.

Sorry, you told that joke. It's— we're past that, right? Oh wow. That's really the concern around the— to your point about speed, you know, these trends move so quickly and they— I think they acknowledge that.

That is, that is something. Okay. Wow. Great to know. Maybe we can talk about misinformation just a little bit, which can obviously, you know, spread like wildfire. Do you have any real-world examples or thoughts about how delayed insights have either led to a crisis or, you know, how actionable intelligence could have change the outcome in a misinformation context?

Yeah, I mean, I think if you, you know, we do a lot of work in this space of understanding, I'll call it spread of information. I think our particular perspective is looking primarily actually at online trends. And for our particular vantage point, we are less concerned or interested in the concept of, for example, true versus false. Perspective is really, if you're a practitioner working in the space, how do you, how do you understand the trends that are spreading and why they're spreading and start to understand and break down those origination, those propagation patterns? And if you think about some of the evolution for that, I think that where we sit today is, is very similar to where cybersecurity was in some ways several years ago, where we're basically good at media aggregation and being able to pull content together. But if you are an operator looking at that data, you're looking at these streams of content. And it's still very difficult to actually acquire the context around those larger patterns. And I think that's part of what causes basically this delay into insight. And that's what creates a lot of these problems. We can talk more about that in a second. But I think from To your point, some of where we've seen how those delayed insights can lead to a crisis in, in this space, one of my favorite examples is actually in the commercial space, which is a few years ago, a hacking group called Lapsus$ hacked the cybersecurity company Okta. It was a pretty famous, well-known hack. Shares of Okta dropped some 20%. What was really interesting to me about that particular story was that there was a delay of first, actually multiple months where Okta didn't actually make the disclosure. But then even when they made the disclosure, what happened was Lapsus$ created an entire social media campaign around the hack. So they published screenshots on Telegram. They then started to post to Twitter. And, and what was fascinating about that particular example was that they were simultaneously exaggerating the extent of the issue. So the actual breach that happened to Okta was relatively minor compared to basically the claims that Lapsus$ were making in the public ecosystem. It was almost this disinformation campaign by a hacking group against a, you know, against a company. And in my particular opinion, if Okta had been— had better intelligence about some of those early publications that Lapsus$ was putting out and realized that they were running basically the social media campaign against them and been more public and proactive about disclosing the information that they had, they could have controlled that narrative better. Because now, months and years later, people know that Okta was hacked, but a lot of the time, the narrative that exists around that was actually the one that Lapsus$ put out. And so Okta basically lost control of that narrative. So I think that's a really interesting example. And I think how you're starting— what you're starting to see now are that companies are starting to realize that that is a problem and they're starting to debunk a lot of these rumors early on or get ahead of them. You see some of the research coming out of Google Jigsaw on how that's actually an effective way to get ahead of these issues. You also see how companies like BlackRock are actually setting up specific newsrooms on their website. In their case, it's called Setting the Record Straight, where they're tracking rumors around them and just getting ahead of it, basically, which I think not only from a comms perspective, but from an SEO perspective is actually very helpful and very smart so that when people are searching for these rumors, they can actually build their own narrative as part of a response. So optimistically, I think we've seen how, at least in the private sector, companies have been affected and how companies are now starting to actually be proactive and react better to these types of issues. Apologies for the long answer. Hopefully that was some helpful context.

Yeah, that is fascinating stuff, Nick. I'm just curious. So we haven't talked real specifically yet about your company, Peak PeakMetrics. But are, are these the kinds of things that you guys are plugging into in order to, to help mitigate these brand attacks or, you know, to help organizations with brand reputation management?

Absolutely. You know, we, the way that we think about this information space is that there's a lot that, that the public sector and the private sector can learn from each other. You know, I think the public sector historically has been very good at intelligence collection and thinking through things from a geopolitical perspective. The flip side is that I think the private sector has done a very good job of understanding this concept of reputation, understanding the importance of social media trends, being able to move very quickly and react to these different types of online trends. And from our perspective, we're right in the middle. You know, we plug into both to support to support that spectrum because we think that there's increasingly an overlap now anyway, unfortunately, I guess, for better or worse, of the private sector, these geopolitical issues, the social media trends that are emerging. And so there is some strategic advantage, I suppose, to being able to pull in from both sides of that equation.

Mm-hmm. We haven't gotten too much explicitly into the cognitive aspect of your technology and how you look at the world along those lines. But, you know, obviously there's a big, huge— you're talking about like a competition for attention with, you know, how to go about and choreograph content on TikTok, for example. But do you think you can paint a picture for our listeners about what the cognitive burden looks like in practice and how it impacts people's ability to respond effectively, or organizations for that matter?

Yeah, I mean, I think the starting place that everyone talks about is, of course, that intersection of increased volume and velocity and that speed of content, of course, that affects it. Dialing more specifically into it, I actually feel like it's almost like an inadvertent laziness is the term that I would use in a lot of the way that we think about the actual practice of processing through these large amounts of datasets. And so what I mean by that is very tactically, the way that most organizations think about analysis in this space is really focused on that aggregation of data. And so you're scrolling through feeds and feeds of social media content or online news content, and you're trying to look for an anomaly, and you're trying to look for something that could be problematic. Maybe you're trying to identify a trend. Then maybe say you identify a trend and you go deeper and you set up additional queries and analytics for that particular trend. And I think while that is successful and while that helps you paint a story of what's happening, it's also a reflection of so much human bias based off the analysts that are actually performing that workload. And of course, that's important. You need subject matter experts in any type of analysis in this space. But I think the problem there is that if you look at the platforms, everything is just becoming more algorithmically driven. And of course, combine that with what we talked about earlier where a lot of conversations are moving into private and semi-private communities. And so what you lack there then is the ability to actually process and understand this data at scale and in real time, which is a lot of our focuses. And also looking at the context of that conversation to actually hyper-enable an operator to acquire that context faster and get past those human biases. So that's a lot of what I think about where the challenge is, is not so much in the data availability or even the workflows now being able to process through that data because you could certainly read through tons and tons of content and that is an important first start. But you, you don't necessarily succeed at seeing those known unknowns, which I think are going to increasingly become larger over time because of so many of the reasons we talked about earlier.

Hmm. It is. And is this something that Peak Metrics is helping customers grapple with?

Yeah, that's a lot of what we think about is how do you then apply more effective uses of machine learning against that data to help? Well, I guess it's, it's maybe if I take a step back, the way that we think about it is a pretty simple, in many ways, framework. It's detect, decipher, defend. So on the Detect side, that's right. It's, it's how do we help our clients better aggregate data from disparate sources, which again, I think a lot of people can do to some degree. And, and part of the problem actually with this space is that there's a lot of words that are used by the same people, but then when you actually dive down into it, you have to really check the quality of that data. So for example, a lot of people say that they're pulling all types of different social media data and news data, but there's a lot of discrepancy between the quality of that data. So anyway, detect is step 1. How do we help our clients more effectively aggregate the widest set of data for their particular use case?

Yeah.

Step 2 is what we call decipher. And our approach is using machine learning to identify different trends emerging in that data or what we call narratives, trying to extract different narratives almost as a node or as an entity in itself.

Mm-hmm.

And then breaking down the factors behind those narratives. So why is it spreading, you know, Is it particularly toxic? Is it, is it connected to a bigger geopolitical issue? Basically breaking down that narrative. And then the defense side is really how do you respond? And so that's how we think about that particular issue. And that, that decipher component in the middle, I think, is really what speaks to some of the challenges that exist in the current ecosystem.

Do you guys get involved in sentiment analysis as well? And how do you look at you know, how sentiment integrates into this problem set.

Yeah, I feel like sentiment is such a— it's like the boogeyman in this space. In many ways, everyone talks about sentiment. What does sentiment actually mean?

Yeah.

Okay.

Yeah.

I think my particular perspective is really twofold. One is everyone wants sentiment, but the reality is that there's— it's a much more complicated issue than that. So we think about it, one, as you should actually prioritize the narrative because the narrative gives you a much more granular example of the issues surrounding a particular entity.

Do you think— sorry to interrupt, Nick, but do you think you could just, for probably me more than the rest of the listeners, what could you separate out what Narrative, compare narrative versus sentiment. What do you mean by the two?

Yeah, so I think when we think about narrative, it is a collection of phrases, stories, articles that all relate to a particular idea or theme. And if we're succeeding successfully, the narrative actually has a particular angle. Or a bend to it that it's trying to drive toward. I think sentiment traditionally is more looking at positive, negative, neutral, things like favorability. I think they go hand in hand.

So does, does, would, would it be fair to say that narrative is something with that, that has some kind of intention behind it? Or yeah, what narrative well done there is you know, some, some sort of a purpose behind it.

Yeah, correct. I would agree with that. Yeah, if, if you're successfully identifying and driving narrative, it has— it's not just about aggregating content together, it's understanding basically the, the why behind, you know, the narrative. There's usually something more to it.

Yeah. And, and, and presumably, um, narrative— from what you were describing to me, it sounds like narrative is bigger than just sentiment because perhaps a narrative campaign could have a number of different sentiments integrated throughout it or like complementary sentiment that doesn't necessarily mean that it's all good or all bad. Or there's some example I'm trying to think of here where like injecting disagreeability at appropriate moments can actually be helpful in kind of an odd way. Or it's almost like even in cakes and cupcakes and sweet treats, the recipe calls for a little bit of salt, even though salt is not the same thing as sugar, right? And so a mixture of sentiment can actually be helpful for an overall narrative campaign success. There's a question in there somewhere.

I mean, I think that's right. I mean, I think what strikes me as important about narratives, I suppose, is that if you look at today, the way that we think about analysis of, say, open source content or publicly available information, if you are only looking at, say, social media posts or news stories and ranking that for traditional sentiment, say positive, negative, neutral, you are missing so much context as to what is driving that sentiment change, for example. Whereas if you think about narrative and then you look at, say, narratives emerging around the Middle East conflict, you all of a sudden start to understand what are the different themes that people care about, what are the audiences that are affected, it's really getting at the why behind that. And then you can start to classify and say, okay, these are the set of narratives that are favorable to me as an organization. These are the set of narratives that are not favorable to me. And that's what actually can drive the insight and the response. And so that's where when we think about sentiment, one, I think, is actually the importance of narrative as part of that, as part of that, as part of that analytic, I suppose. And then two, you know, more tactically, I think if you look at the question of sentiment, then yeah, it's more complicated than that. You have to look at actually not just positive, negative, neutral, but favorability. We're doing a lot of work now around building out specific favorability models based off of who you are and what you're trying to accomplish. And I think this is where a lot of newer technologies like large language models are actually showing really impressive numbers around their ability to actually contribute towards the sentiment problem. So from a technology perspective as well, it's a really interesting time to be looking at this, this question of sentiment.

Okay. Cool. So I, I believe you guys here relatively recently just launched a new capability where customers are able to identify threats in, in ways that have not been possible before. Do you think you could talk a little bit about that?

Yeah, we are— we're currently investing a lot of work into basically hyper-enabling an operator to determine context quickly. I guess.

Okay. All right. Tell me more. Yeah. Excellent.

And so what that means is if I look at the framework that we use of that detect, decipher, defend, really, as maybe I've harped on a lot now, but like really, we believe that the center node of a lot of this analysis is around that concept of narrative. And, and so our goal is how, as an operator, do we help you determine what the narrative is and why and the why behind it to determine basically a level of threat against all the many different narratives that exist within the world that affect you. So, for example, the way that our workflow works is that let's say that we're looking at 200,000 posts about an organization or an entity. Over the last month. We use machine learning to break down and identify, say, 50 narratives out of that data. And then with the threat score component of what we're looking at, we are automatically ranking those narratives for different factors that are concerning. So we look at things like the volume of the narrative, the velocity. We partner with organizations to determine the credibility of different publications. We look at the reach. We look at bot-like activity. We look at, to your point earlier, different forms of sentiment against that narrative. And all of that is scored automatically and programmatically so that our clients can basically determine what the most threatening areas are at any particular point in time and critically in real time. I think if you compare that to what is done today, so much of what we do as practitioners is trust, is based off gut. So maybe it's like a gut feeling that you can then back up with data, which is problematic for a few different ways. But also, if you're trying to create some type of scoring system manually, we see this in the corporate sector where companies will basically spend all of this resource to try and figure out a complex scoring system for how they actually are gonna try and rank, you know, a particular threat at, at a particular point in time. And then the reality is that you actually get to try to put that into practice, and it's just too much effort because who is gonna put— who's gonna fill out their checkerboard of, of score for every single viral trend that's gonna happen? And so I'm really excited about this feature release around our ability to basically start to automatically score these different emerging narratives and patterns, again, in real time. Because I think with the speed, with the volume, with the lack of resourcing, we need to be relying more on machines and methodologies to support this content.

Right. So the kinds of workflows that that you guys are envisioning or that your clients are using, do they tend, tend to be like human-in-the-loop workflows? So like if they receive from your services a threat indicator, then they take that as a piece of evidence that they should be considering on how to, you know, continually manage their business, their, you know, brand, their, you know, military operation, et cetera. Is that a good read?

Yeah. I mean, I'm a firm believer, particularly at this particular moment in time, that a human in the loop is incredibly important, and we should be focused on hyper-enabling an operator to be more efficient with their time and to basically reduce the time it takes for someone to discover something to actually respond. Yeah. You know, I think there's people in our space that are trying to figure out at what point do you move more of that response side to automation. And I think that, again, for better or worse, it's probably inevitable that we'll start to automate more of the response side of this type of work. But the, the risk is still too great. So I, I do think that at this point, at least, having that human in the loop bringing in that subject matter expertise to basically help provide additional context, but automating upfront as much as possible around, around creating the, creating sort of the story and the context for that human to verify and confirm. That's, that's really the focus for today.

Right, right. Well, it sounds like you guys are, you know, really providing some value already in terms of time savings and insight generation already. But what are you doing? I mean, clearly, this is, this is a fast-moving space, you know, back up to the top, things are evolving, right? What are you guys doing to, you know, try to stay ahead of the curve in order to help your clients continue to manage these problem sets?

Yeah, thanks, John. I definitely appreciate that. And I think it's probably also important to acknowledge that, you know, we, we understand that this is a massive problem and there's a lot of different people thinking about it from different perspectives. Our particular focus is on that real-time nature of the problem. The folks that we're working with are working, you know, to your point earlier about speed, you know, in real time, daily, weekly. You know, I do think that's in contrast to some degree to like the academic world where, you know, the academic world has an incredible place in this as well. We donate a ton of data actually into the academic world as part of our work just because we believe it's important. But the timelines for research in academia are just still so much slower. So I think acknowledging that we are definitely coming at this from a particular vantage point of how do you get ahead of this today, like as we speak in real time. And then to your point, how do you actually stay ahead? One I think is of course putting the methodology and technology into practice. I'm always excited when the type of work that we're doing is deployed in, you know, in a real-world scenario and seeing the outputs being leveraged for improvements to strategic communications plans to, in some cases, take down notices where appropriate or whatever the subsequent actions might be. So it's always exciting to see it actually being deployed in the real world. I think from a landscape evolving, you know, the big thing I'm tracking right now, which I know a lot of people are tracking, are basically this move from large language models to large multimodal models. So if you're not familiar with that as much, LLMs is, of course, the acronym that everyone knows and loves today.

Right.

LLMs refer to large language models. And what we're seeing now is the deployment of LMMs, which are basically multimodal models. So the combination now of text, image, video, audio, et cetera, and being able to not only independently analyze these different modalities, but actually pull them together. So if you think about, you know, the sort of concept of I/O and you look at even something as simple as a meme, that's of course not just simply text. That is actually the combination of the image and the text. And increasingly, as we move towards this video-forward social media world anyway, that's a lot of what I'm starting to track is what are the implications of LMMs and how do you actually incorporate that into into how we analyze this content. So that's what's interesting to me right now. And it's evolving so quickly. I think if you're tracking any of the new OpenAI models, the combination, frankly, of the new models, some of the— it's October 10th, so who knows what it'll be when this comes out. But the new models that OpenAI has released to basically put an LLM against itself to actually handle reasoning, combined with, I believe that the newest model actually also does support multimodal analysis, so it can actually read images and videos and audio, etc. Like that, that is a fascinating intersection of technology. And, and that's where I think the, on the analysis side and the intel side and the tools for processing this type of content need to catch up very quickly. Of course, it's already going to be deployed on on the content creation side.

Yeah. Wow, that is cool. So large language models, my understanding is that they process text in order to build their knowledge and generate insights. But you're saying that the large multimodal models will not only ingest text, but they will also ingest imagery, video, and other, other types of data in addition to text and build their, their insight generation off of a larger variety of data sources.

Yeah. So if you look at LLMs, basically the, the technology beneath LLMs that is, is a technology called Transformers. And over the last, say, 4 or 5 years, what you've seen is that LLMs have been deployed the fastest. That's where you're seeing it used everywhere, of course. But they've been simultaneously looking at that technology for, well, I think actually initially was even deployed for image use for different types of reverse image search use cases. But you can deploy it, that exact same methodology towards image, towards video, towards audio. And, and now what you're starting to see are these companies successfully actually bringing those models together. To actually work in orchestra, which is pretty exciting.

That is exciting. Wow. Okay. That was a first on the Cognitive Crucible, a mention of large multimodal models. So thank you for that, Nick. All right. Well, you guys are doing some really, really interesting work, and it'll be great to hear about all of your success going forward. Yeah, if you don't mind, we can look to close things out in the typical fashion. So do you think we could ask you for a research question that an interested student might take a crack at if they want to try to advance the frontier of knowledge in the problem space that you guys are working in?

Yeah, absolutely. Well, so first, An offer, perhaps. We do a lot of work with the academic space. One of our big supporters is a nonprofit research institution called Harmony Labs, which is created by John Johnson, the co-founder of BuzzFeed. They invested in us early. As part of our collaboration, we donate data to them. So if you're an academic researcher, they actually have a research network where they've aggregated really fantastic datasets around this issue. We're part of that dataset that's available. So, would encourage you to reach out and try and, you know, access that data if that's interesting to you, because it's available. And it's partnered with basically major universities to be able to access that. So that's one thing I'll throw out. And then the research question that I think I continue to be curious about is there's still a lot of work to be done around correlating earned media impact to outcomes. So how does information that is spreading on these different social media channels, how does PR or earned media actually correlate to an outcome? And I think if you contrast that with the advertising world, where one of the, I think, brilliant things that Google and Facebook did was realize that they had enough data, privacy issues aside, to track consumers across the internet and basically create attribution so they could track you're an advertiser, dollars spent through to the ad network that they've deployed through to purchase data, you know, they could successfully, even if the purchase occurred later, they could successfully basically create the attribution. I think there's, we're starting to do a lot of work around trying to successfully create that attribution layer on the earned media side. But I would be very curious to hear more academic research around trying to correlate, you know, the earned media to outcomes, as I mentioned.

Okay. Yeah.

Clearly, the privacy implications now where, like, as we kind of touched on earlier, there's increased consumer protections around privacy, which I, I think in general is, of course, correct and good. But it, it makes it an even harder challenge from a practitioner perspective.

Yeah. No, a very, a very ripe area of research for sure. And then lastly, could we get uh, your book suggestion, or it could also be like an online resource that, uh, you think our audience might appreciate.

Yeah, um, we've talked a lot about the importance of narrative, and the one that I always go back to is, uh, is Joseph Campbell's Hero with a Thousand Faces, which is the classic story of the hero's journey and describes this universal arc that runs through almost every modern myth and storyline. You know, all the way going back to the Bible through to Star Wars and Harry Potter. And my perspective is that if you're working in the information space, understanding narrative, understanding how stories are actually structured can be a very valuable thing. And you can apply that in a lot of ways. So I really love sort of that, you know, very, in some ways, fundamental concept of storytelling.

Yeah, awesome, awesome book. I can double down on that one. And Joseph Campbell, I've mentioned him several times on the podcast, was, I don't know, maybe one of my inspirations for this whole portion of my life and being— Yeah, yeah, yeah. I thought that just his life's work in general, I found pretty inspiring. From, you know, he was actually one of the characters portrayed in the book by John Steinbeck. I'm forgetting it right now. What is John Steinbeck's, like, most— Most—

Is that what it was?

No, no, no, no. Steinbeck had a book where he— Cannery Row. Canary Row. So Joseph Campbell was one of the characters in that book where they're talking about the cannery operations in Monterey, California back in the, like, early 1900s. And so he— but so Joseph Campbell has this interesting life story where he did work in the canneries and he got integrated into that whole like Central California coast, mid-century, mid-19th, mid-20th century scene, which I just thought was really fascinating. And all of his work on myth and comparative religion, I thought was really fascinating. Yeah, to your book suggestion there on A Hero with a Thousand Faces.

Interesting. I did not, I didn't realize that. That's awesome.

Yep, yep. I believe he's mentioned by name. You know, he talked, you know, Steinbeck talks about, maybe he doesn't use the last name, but he talks about Joe in Cannery Row, and that is Joseph Campbell.

How funny.

Yeah, yeah, yeah. So good stuff, good stuff. Well, that was a bit of a sidebar, a little bonus round for—

I like it.

We'll go ahead and keep it in. So what the heck? So, but with that, Nick Lui, thank you so much for being a guest on The Cognitive Crucible.

Thank you, John. Appreciate it.

The Cognitive Crucible is the only podcast dedicated to increasing interdisciplinary collaboration between information operations practitioners, scholars, and policymakers. To find out more about the Information Professionals Association, visit us at information-professionals.org. Please support our podcast by giving us a 5-star rating and leaving a review.