#259 Getting the Data For Your Data-Driven Decisions | Jonathan Bloch & Scott Voigt

Behavioral data now lets you start to fold in intent in a weird way by programmatically understanding how was someone's mouse moving? Was it moving with intent when they hit the site? Did they know where they were going? Were they frustrated or confused? And believe it or not, those heuristics are in every visit to be able to understand the behavior of the visitor even without knowing them. And then you can start to cluster those behaviors together to start to either change the experience and give a more personalized experience or, or remedy some sort of frustration.

Hi Scott and Jonathan, welcome to the show.

Thanks for having us, Richie, thanks for having us.

Big question to begin with, where do organizations get their data from?

Obviously from lots of places. Captain Obvious would say in my world. So I'm CEO of a company called fullstory. And so most of our customers are paying a lot of attention to where they're getting their first party digital data. And in our case, that really comes from the visitors to their websites or their web apps or even, you know, the, the mobile device. And so they're just thinking about every specific action that a visitor is taking place and making sure that they're paying attention to the good ones and ignoring the bad ones or the private ones.

Okay, that seems fairly straightforward. You get some data directly when people come to your website and then there's going to be data from elsewhere. Jonathan, do you want to talk a bit more about where people get their data from in your world?

Sure. So I'm CEO of a company called Exchange Data International, and we aggregate data from all exchanges around the world. So we get data from exchanges from regulators, from central banks, from ministries of finance. All in all, we have probably 700 sources coming in daily. And then we have to translate them into English. So we translate 39 languages a day because the world does not speak English.

Okay, that's a very good point, is that there's data all over the world and you want it in the right format that people in your organization can understand.

Okay.

So Scott, you mentioned the idea of the difference between first party data, third party data. So the third party data coming from elsewhere. That's maybe a little murkier on what that involves. So Jonathan, you mentioned some examples of just banks and exchanges that provide data. Where else might you get your data from? What else constitutes third party data?

Well, in the digital realm, there are all sorts of third party sources where marketers can go to sort of buy profiles, if you will. If you've ever accepted some weird terms of services on different sites out there, they're collecting data on you Facebook at all. And those are aggregated in different ways, sometimes anonymous, sometimes mapping back to you or your phone. Those sort of things are third party data where you're collecting data from around the web. First party really is, if you want to think about it, in sort of the real world, if someone were to walk into your store, they're in your store, your relationship is with that customer, you know them, you've created that relationship. That's first party data. And so the same would be true if somebody went to your website or your web app. You have that relationship with them, they've sort of accepted a terms of service, usually on the visit. Certainly in the eu, that would be true. And that's first party data.

All right, so if it's coming through a separate organization, there's going to be third party data where you've not got that direct relationship with the customer. All right, and so with this third party data, it seems like there's going to be some sort of licensing involved. Jonathan, these sort of regulations around data, your area of expertise, what sort of data licenses are there for getting hold of this third party data?

Well, I mean, in terms of data and licensing, it's a very fraught area because there's a whole issue of copyright, there's a whole issue of trademarks, and increasingly you find lawsuits about this. So for example, the New York Times and various other publications are suing the various AI operators, saying this is our data, how can you use it in your AI machines? And those cases are still to be decided. And I spend a lot of my time dealing with this issue because the exchanges often say we own the copyright in this data, but you can't own copyright in figures and numbers. So what they try and do is they try to circumvent the copyright in a very underhand way. When you sign a contract with these exchanges, they say you recognize that they have copyright, so they then bind you contractually. And I think you're going to see that in other areas start happening now because as data becomes more valuable, particularly for AI, people are going to assert their rights over things they do not own, but then they'll bind you contractually. So I suspect terms and conditions on websites and various other places will change significantly because of that.

That's fascinating. I didn't realize you couldn't copyright numbers, but it sounds like there's going to be some extra legal language on top of that that says, okay, this is my data, you're going to have to pay to use it. And I Guess just as a follow up to that. If you are trying to monetize or sell your own corporate data, what do you have to do to protect it?

The only way you can protect it is, is by building moats around it, making it so unique or making the process of obtaining it is so difficult that you become a unique provider. So we specialize in corporate actions data and we really only have four competitors worldwide because it's so expensive to accumulate and then you've got all the translation costs. So you need to somehow make your data unique and find a niche where it's very difficult for others to enter.

It seems like we're entering a bit of the wild west right now, Jonathan, if I'm reading you right, we actually headquarters, we brought some of our best engineers and our best product managers on site this week to really just talk about AI, what we're doing with it, how we should think about it, how we should think about it on our customer's behalf. And again, coming back to the idea of behavioral data. Well, when, when we as humans go to websites and go to a site and read, we're having that first party relationship. If a bot goes to that same website and reads and navigates around. Well, in the past we didn't, there weren't, I don't think that was a big copyright violation if the Google bot hit your website. But if all of a sudden the, you know, the OpenAI bot is hitting your website or the anthropic website is doing something, it gets a little bit murkier there. And so we, we were trying to figure out, well, like you said Jonathan, you can create a moat around your content. But I do wonder if there isn't going to be some interesting moments where sites are trying to understand when a AI bot is hitting the site and then in real time watch that AI as it's moving through the site and trying to understand, okay, do we want to embrace this one and reject this one? Do we want to create a different experience for the bot with different prices if you're an E commerce customer or different watermarked type data for the, for a publisher that maybe has something out there. And again it, you might be able to put some terms and conditions. Good guys will honor them, bad actors won't honor them. So are we going to see this sort of, this moment where websites are putting up, you know, chaff to kind of like confuse people when they come across? I don't know.

I think, Scott, I think you're absolutely right. It's going to be the Wild west, because it's a bit like if you look at the invention of drones, so now you've got anti drones to shoot down the drones. Right. And the issue with AI is AI is going to be ubiquitous. Whether Google will exist in 10, 15 years time is a real open question because your search engines are probably going to be AI powered for sure.

They are. For me already as a human, I like, I feel bad for Google because I just use OpenAI to say, hey, what time is the football game on? And it just gives you the answer. And if you go to Google now and ask that same question, it doesn't give you the answer. Unless they're using Gemini, at which point they're taking. They're spending money to give me the answer at expense of not letting me go click through the sites where they would get paid. And that's. That's tough.

Well, I mean, if you look at the history of technology, who remembers Netscape, AltaVista, WAN Computers? It's the way of all flesh.

Yeah.

To create, you must destroy.

Exactly.

Absolutely, yes. Dangerous game being in tech. Okay, so a lot of great points there on what bots should be allowed to do. And I think, yeah, historically most websites were happy for Google to scrape them because they benefited from being in search results. But now, yeah, with AI, they're not necessarily getting things back. Now, Scott, you mentioned the idea of behavioral data before. Can you just explain what that is? Is it the same thing as customer data?

Yeah, I would say it's probably a click deeper than just customer data. I mean, to us, you know, customer data is who you are and how much you spent. Behavioral data is, at least in the digital realm of something, is everything you did on your journey to buy that thing or accomplish that goal. And in the past, that was sort of a series of discrete steps. I went to a page, I went to another page, I completed a transaction. Behavioral data now lets you start to fold in intent in a weird way by programmatically understanding how was someone's mouse moving? Was it moving with intent? When they hit the site, did they know where they were going? Were they frustrated or confused? And believe it or not, those heuristics are in every visit to be able to understand the behavior of the visitor even without knowing them. And then you can start to cluster those behaviors together to start to either change the experience and give a more personalized experience or, you know, remedy some sort of frustration if all of a sudden there's a sudden spike and people. We created this concept we would call a rage click. Where people just hammer on the mouse in frustration. We've all done that. That's a behavioral signal that people are now paying attention to so they can say there's been a spike in rage clicks. And what do we need to do to remedy that so our customers can have a better experience.

You mentioned personalization. That's a very hot topic. But what sort of personalization might you want to provide for different customers?

Well, I'm going to start with a very simple example and then you can start to extrapolate it. But um, it's Friday night, you want to go buy a pizza. And so you go to your, your favorite online pizza place. And perhaps I'm, I'm in the mood for a deal. And so I'm slowly scrolling through the page, I'm kind of hovering with behavioral intent over the coupons that are available there, but I'm not really doing something and my tab goes dark all of a sudden because maybe I've gone to my second favorite pizza place. That's a behavior you can start to pay attention to during that experience. And perhaps if I light that tab up, given that I'm on the fence as to whether I buy or not, I can give me a really good discount and save that play. Now I don't give that same really good discount to everyone because on a different night I might go there, I go straight to my previous orders, I click on it and there is high intent, high behavioral signal that I am going to convert come hell or high water. What do you do there? Well, you offer me the great cross sell on that, that, those chicken wings that you know if I eat, I'm going to be hooked to them and I'm going to get them every time going forward. And so it's, it's really starting to personalize that experience around the, the intent and behavior of what those visitors are doing. And you can imagine really starting to, to, to evolve that concept in, in many ways.

Okay, I like the idea of just having a last minute save for the undecided customer is going to bring you more business. And Jonathan, is there an equivalent personalization in the financial data that you deal with? Do you everyone standard issue stuff or do you customize for different groups?

I mean, we do in terms of data, one of our characteristics or one of our major selling points is the customization of data. Because our big competitors, and we have only four competitors, which are three stock exchanges and Bloomberg, they hate doing customization. And the reason they hate it is first, it's expensive to set up. Secondly, it's even more expensive to maintain because you can write a special script. Sometimes they don't work. Things go wrong. So you're going to have a lot of engineers to deal with that. So customization becomes very important. So for example, somebody trades only a certain sector on the stock market. Say they trade motor vehicle manufacturers. They don't want a stock market feed of all the stocks on that exchange. They just want the motor car manufacturers. And so we do that type of customization and I think there's going to be a growing market for that. So on the one hand, you, you have the people providing what I call commodity information. They give you everything at a low price. There'll be increasing role for the boutique, which customizes to people's preference and interests.

That's interesting. So the value of the data, you can get basically higher margins if you are giving customers exactly what they want. So the personalizations are going to basically bring you more money that way. All right, so since you sort of mentioned that personalization is expensive, Scott, do you have a feeling on, like, what's a good way to get started with personalization of experiences for customers?

My instinct and what we see with our customers is you start with the biggest points of pain. And it turns out that if you haven't been paying attention to behavioral data and you start to pay attention to behavioral data, you'll quickly see that there are areas where you're just frustrating your customers. You haven't been paying attention to a particular flow or journey. There are errors and lots of low hanging fruit. Get those out of the way and let's get customers, remove the roadblocks for them to be able to be successful in whatever they're trying to accomplish. On the flip side of that, I think what you can start to do is understand intent earlier in a journey and start to pull forward the things that meet what it is that that customer is aiming for. And we see lots of customers that are just taking the baby steps towards that through understanding and then design.

Okay, I like that. Just start with don't annoy your customers. Stop being frustrated.

Doesn't it feel like there are too many places out there that still just annoy their customers?

Absolutely. Many terrible websites, many terrible store experiences as well. Yeah, shopping can be hard. In fact, visiting most websites can be hard. All right, so I'd like to get back to data sourcing. So suppose your chief data officer says, okay, we need to be more mindful about where we get data from. We need a new strategy. How do you form a data sourcing strategy.

Okay, well, I mean the important thing to do is firstly to define exactly what data you want, the scope of it. Secondly, once you define the scope, you should go out and search for all the options that are available. And then you need to start looking at things like quality, delivery mechanism, price, frequency of delivery to take all the factors into account. Often what people do is say I just want this data set and don't do exact definitions, et cetera. And that costs them a lot of money. In the end, you also got to define what is the purpose for which you want the data so you don't land up with extraneous data. So clearer definition is the key for data sourcing.

Okay, so a lot of planning up front. Think about what you actually want in order to solve your goal rather than just going and buying things. All right, so once you've done your planning, what's the sort of next step to implementation?

Well, the next step would be usually what people do is they go onto Google and they will search for people providing the data. Often what they will do is what they call an RFP request for quotation and they will set out exactly what data they require in how it should be delivered, when it should be delivered and for what purpose. And then they start get the proposals in and then they start negotiating. They should then ask for sample data or even better, a trial of the data because what people say the data is what it actually is. There could be a huge gap between the two.

Yeah, I can certainly see how you want to sample and just make sure that you are getting what you asked for. You think there'd be some sort of like free sampling process like you get when you go to the stores. They give you a little taster of what you're going to get. Okay, so who needs to be involved in this? Like who is typically responsible for sourcing data?

Well, I just want to say one, the reason why you've got to be so particular. Most data is supplied on subscription, so you not just buy the one off, you committing for usually a one year period, might even be longer. So you want to make absolutely sure that you're getting what you expect because it's not a one off payment and people involved quite a few, the user, firstly, second of all, your compliance officer or your legal officer to do the contracts. Thirdly, and most importantly your IT people because usually the data has to be integrated into your systems and that is often easier said than done.

Okay, yeah, I can certainly see how integration is going to be tricky. Are there any tools or Technologies or platforms that make sourcing data easier?

Well, there's several. I mean, firstly, you got the whole concept of APIs now, which is really a format agnostic and you're just taking the data you require. Second of all, you've seen the growth of these various storage companies which add a layer on top of AWS or Azure, people like Snowflake, people like data bricks, and there's several of those. And we come across a situation where a lot of financial institutions will only take data which is now stored in one of those storage facilities because it makes it so much easier for them.

Okay, so use one of these sort of big cloud data warehousing platforms and that's going to make your life easier. And is the same true when you're looking at this first party behavioral data?

I actually think it's a little bit different and I think the reason is just the underlying cost. For Jonathan, I don't know your business very well, but rich financial information that has been cultivated and translated sounds like a heavy process if you think about the amount of data in a visit and the storage cost to collect that data, it really is just not that expensive. And so I, we sort of, as I talk about our team to our team, you know, we believe that we, we should, we believe our customers should collect every valuable visit. And by and large that's every visit. Because if they hit your site, you want to know, even if it was a bot that bounced, who was it? Why was it in the cost? So our job is to collect every visit and then extract the value from, from each one of those visits. The problem with that is there is so much that can happen within every one of those visits, you might as well collect it all. I mean, let's use Google as sort of a parallel. Google wouldn't have been successful if they said let's only spider the biggest websites out there, or let's only spider the first page of every website, or let's only spider the top of each website. They went out and collected the entire Internet, all of it, every word on every page sits in an index. And they were able to do that cost effectively because you didn't, they didn't know who might want to search on what, how that would fill into the algorithm. And so I think if you're a company that cares remotely about digital experience, then you want to collect every bit of behavioral data that you can, even if you don't know exactly how you'll use it. Because the cost to process, the cost to store, the cost to infer they're relatively reasonable at this point. And you know, waves hand says AI we're getting into this moment where perhaps over the past few years you had to have a human that had a hypothesis about the data to go query it to get a, to get an answer or result. With AI, you're now going to have machines that are constantly looking who is happy, who was sad, who was confused, how do we make this better? And you're just going to need every bit of data you can to make that good and true.

That's really interesting, the idea of just being able to ask more questions than you have data scientists for, because you can use AI to automate a lot of the exploratory data analysis.

Yeah, that's coming. It might be here at this point. Data scientists are expensive and I don't know how much you guys have played with Claude and chatgpt, but they were very, very good at just kind of plowing insights out of things.

Absolutely. And so since both of you mentioned the cost of these things, how can you assess the sort of relative value of data versus the cost of storing and processing it and getting insights from it?

It's a big, big. It depends, of course. I mean, if you're, if you're a. The bigger the company, more the data, the more the data, the more the cost, the more complex the data. You know, one of the reasons, Jonathan, when you mentioned Snowflake and databricks, I mean, people like to take all of their data and then push it together and then querying all of that data together is still expensive. That querying process in limited scope, if you only want to query, say the financial data and nothing else around it, it's probably relatively cost effective. If you only want to query the behavioral data, it's relative cost effective. But, you know, the more complex, the more expensive. And so therein lies the rub.

Well, I think what you've seen already is the three major cloud providers, Azure, AWS and Google, being disrupted by newcomers. So you've got people like Akamai where the costs are 1/20 of AWS because I costed something out.

Wow.

So what you'll see, as what we said before about potential demise of Google in the next 10, 15 years, you might also see some disruption in the cloud storage business. And that's also one of the, to.

Be clear, for the record, I'm long on Google. They also happen to be one of my investors, so I'm big on Google. They're going to be great.

Hopefully around long enough to keep funding you.

Yeah. But I mean, I'm not suggesting that they will totally demise, but what I'm suggesting is that AI will become much more ubiquitous. And in fact, that's bad news for all those companies with AI in their name. Those 17,000 companies who grazed a lot of money for venture capital. I don't see much future for them. Because what will happen is the big guys, because of the cost of computing power for AI, they will dominate. So it'll be yet another area of technology where big guys dominate, be it desktop software with Microsoft, Google with search engines. AI will have big players, certainly at.

The foundation level, where it just costs tens, hundreds of billions of dollars to drain the thing, then, yeah, there's only going to be space for a limited number of players.

But you know, one of the reasons why we see the growth of APIs is because people do not want to store super plus data because of the cost. They very, how can I say, Skillful in just extracting the data they require and only storing that.

I've had quite a few guests on the show talking about like, what should your data storage strategy be? And the opinion seemed to vary wildly from you should store absolutely everything, no matter how insignificant it seems. Now it might come in handy 10 years down the line and someone like, well, you know, there's no point in storing data that you're not, that you don't know you're going to get value from. Just store what you want. It's going to save your cost, simplify your life. Do either of you have an opinion on where you land with this?

I'll take a first stat. If 00 to 10, if 0 is only collect what you absolutely know you need and nothing else. And 10 is any bit of data that you can get, you should grab it no matter what and stick it in a warehouse. I probably am somewhere in the 77 realm. I tend to believe that you don't know what's going to be important and that the costs will continue to move in the right direction and the ability to mine data, mine extract value from data will continue to increase. But if you, if you're sure that there's never going to be value in it, then don't collect it. Like, be pragmatic.

I agree with Scott, but accepting you could never recreate history, and we have done various exercises or people who wanted to recreate history and it's so expensive, it's almost unobtainable.

What do you mean by recreate history? Just so I'm on the same page.

Okay, so for example, somebody says give me 20 years of stock market prices and you've got 10 years. You now have to go back and get another 10 years. And that is almost impossible.

My version of that, by the way, is, you know, there are customers out there that will, they will look at costs and they might say, well, you know, we're pretty sure that those visits have no value at all. And so we're going to just, we're going to make sure that we don't ever collect the bot visits. We've been talking about bot visits, right? We're not going to collect them. It's not even that it's expensive to reconstruct history. Once you don't collect certain things, you can never collect those sort of things. And so there's no way back machine if you didn't collect a certain bit of data. And so I agree with Jonathan there.

Okay, so better to collect stuff just in case. And I mean, I guess you can always delete stuff later on.

Within reason. Yeah, that's my cheat code. That makes. Okay, as long as I say within reason.

Actually, this reminds me, there's like a sort of famous example at the BBC, the British Broadcasting Company. So Doctor who, like the early episodes they recorded in the 60s, and then at some point in the 1970s, someone was like, you know what, tape is expensive? And they recorded over the Doctor who episodes and those things, I was like, just saving a few British pounds. And those things are worth a fortune now, but they just don't exist because someone recorded them.

There you go.

So, yeah, best not save a small amount of money for something that could be valuable in the future. All right, so Jonathan, you mentioned that you need to translate a lot of the data you receive in order to increase the quality and the value. What sort of data quality metrics do you need to look for, particularly when you're getting external data?

So, I mean, the problem you have is no data provider can be 100% accurate. Anybody who says that 100% accurate is basically bullshitting, put it bluntly. And there's reasons why you cannot be 100% accurate. Firstly, somebody can mistype something and it's only picked up a lot later because it doesn't correlate with some other sources of data. Second of all, translation is not a science, it's also an art. And words can mean different things in different contexts. And that's a big problem with AI, for example, that it doesn't often understand context. I can give you a very real example which I've dealt with. So we pay a third Party company to use AI to scour the web for companies who are rumored to be going to ipo. So they've not yet registered to ipo. So a very big client comes back to me and says we've received details of an ipo, but it comes out of a television script because the AI picked up some television script which said such a company is going to IPO in five years time. So to miss this context.

Okay, yeah, I can certainly say there's probably room for disaster with episodes of Succession or something or Silicon Valley.

The way we measure quality of our output is by looking at the support tickets raised by our clients to see are there any systemic errors that appearing. And that's the only way that we've discovered you could really keep a nab on quality without having two sources or three sources for every piece of data, which often is not possible.

Okay, that's interesting though, the idea of two or three sources. So I guess if the different sources match and they're hopefully independent, then you've got some guarantee that the number's going to be right. Okay. And Scott, is the same true for first party data, like how do you assess the quality of it?

Yeah, I think it's a little bit different. So first party data, you can just think of it as a machine taking record keeping and it is just a straight record of what that visitor did. And machines are just very good at keeping clean records. There's you visited a site, you went here, you went here, you went here. And that's a very elegant, elegant log if you want to think about it. And so quality doesn't really play into the quality of the behavioral data. Where it does start to creep in is the interpretation of the quality of the behavioral data. So it might not be intuitive if you run some sort of report that talks about a journey where you thought something was going to be true and it wasn't true. And so then the human has to check their intuitions and you can spend a lot of quality cycles just validating. Yes, it recorded everything. Yes, it recorded everything correctly. It just didn't happen to line up with what you thought was going to be true.

Okay, so you got some sort of machine checks to say, is this the same as what I thought it was going to be? If it's different, then that's going to give you some impetus to go and look at it.

In essence, it might be, I mean the situation would be, well, did you miss something? Like this is not a, did you miss something along the way? And there's probably A tier two, tier three, check. Oversimplifying. Was the machine running? You know, did it see this? Was there a secondary source that also validated that? Yes, it saw this. Pretty rare in that world.

I guess the related thing is, so suppose you've got high quality data, but how do you, what do you need to worry about with. Regarding data privacy? Maybe the customer data. That's going to be a big thing because it's got kind of some personal information in there.

Yeah, you hit it right on the head. You know, the way that behavioral data is collected is by harkening back to our earlier discussion, basically capturing everything, where everything is everything except very private information. And you just need to be very cautious of it. So we've had to develop lots of tools and techniques and process to make sure that when you get to a page and you're a company that wants to not understand anything personally identifiable now it's first party. So you know your customers. And often it's quite a right to know that Richie visited the website and he typed Richie in the first name. But if you're a company that does not want to know that information, then you have to have tools that say for this block, don't ever collect that data ever. And that's one of those things that you constantly have to pay attention to, unfortunately. Building those tools, learning those patterns, understanding those have given us a bunch of different approaches that say, for example, we would call it private by default. You can still capture all of the behavioral data when a visitor hits a site, but you wouldn't collect any text for what the visitor saw, anything that they typed in. It's just, it's just no text, all behavior. And then you can start saying, well, we know those are some safe areas, so we'll allow that text to come through or that image to come through if that helps paint a picture. Okay.

Yeah. So I guess most websites, it's like I'm happy if people know my name that I visited. If it was, I don't know, trying to think of an example, like I don't know if it's like a thing therapy website or something like that where it's like, okay, I'm looking for mental health advice or something, you maybe want that to be more anonymous. I'm sure there's plenty of examples.

Health information, financial information, like we serve, we have a number of bank customers and again, that bank already has that data. They understand your account balance. But you just need to be so thoughtful about the propagation of financial data. Where else might it show up and what other systems and who has access to it often best just left alone. Let's not bring that data across ever.

Okay, that makes sense. Just be really mindful about who gets to see what. And Jonathan, since you're dealing with financial data, do you also see data privacy issues?

Well, we try not to gather any information relating to the customer's holdings which identify them as the holder of that information. So when we get data from banks, we ask them to delete any customer information and just give us the securities for which they want information. Because it's better not to own that data. Because in case you get a data breach, the repercussions could be enormous.

Yeah, certainly. I think most people wouldn't want their financial holdings, their portfolio being made publicly available. So that seems pretty important. And so one of the big hot topics at the moment is synthetic data. So data generated either programmatically or with AI. Have either of you seen any examples of its use? Because I think this is one of the things that can help with data privacy.

Yeah, I mentioned the AI summit that we did in the office here and we were actually talking a fair bit about it. Is, is there a way to take the abstracted visit of a visitor and create sort of a synthetic visitor, if you will, that mirrors mouse movements and patterns and all of those things such that you can step around those. Privacy. You just don't have to worry about privacy because you can understand interaction by using an artificial visitor that, that really mimics human interaction as opposed to just, you know, synthetic. Button clicking. We haven't seen much of it yet, but, you know, as of the writing of this yesterday or the recording of this yesterday, Anthropic launched sort of a computer interaction layer where you can download Claude on your machine and tell it that you want it to fill out your expense report and it can just open up spreadsheets, open up pictures, do what it needs to, and fill out an expense report. And so this new frontier of having. Is it synthetic? You know, having a machine be you and do things for you, it's going to be pretty interesting.

Okay, so machines pretending to use websites, that's fascinating.

Oh, it's coming. Yes. Okay, cool.

And Jonathan, does synthetic data exist within the world of finance?

Well, I mean, I think people are beginning to use AI to write reports. And you've already seen a couple of lawsuits where the lawyers have been reprimanded because the AI has produced some lawsuits which some precedents which didn't exist previously.

Yeah, when you're making use of AI to generate things Then you absolutely want to make sure that some humans checked it to make sure that it's SME factory accurate.

I mean, I think anybody using AI now has got to put a human layer over it because it's not quite there yet. And I think, you know, it's great if you want to do a wedding speech or something like that, but if you wanted to do institutional or business information, you need a human overlay.

Absolutely. I mean, certainly for writing questions for the podcast, I will use AI to research companies and come up with some questions and then I mostly ignore them and just make up my own.

Rishi, we just assumed you were in Mayhap this whole time. You're not. You're real.

Yeah, I'm gonna clone myself shortly and then I can record more episodes. All right. So I'd also like to talk about regulations because it seems like there are so many regulations around data, around AI as well. So are there any regulations you need to be aware of when you are trying to find data sources?

Well, on the behavioral data front, certainly. I mean, there are lots of regulations. In Europe, GDPR has been a really good standard. In the States, we're seeing each state, ccpa for example, covers California, have its own privacy regulations about what data can be collected, how most of that data is really, most of those regulations rather are aimed back at third party data and the collection of third party data and buying and selling data. But there are things that you certainly have to pay attention to on a first party basis. Don't want to collect health information, don't want to collect financial information, those sort of things. And you want to make sure that the visitor is aware of what data is being traded. And a lot, you know, I mean, the emergence of every except cookie banner that is in the world, I think in large part because of those regulations. We all. Jonathan probably knows more about this than I do, but it does seem that as it pertains to AI, there's a lot of talk about regulation, but there isn't anything that's very solid yet. So everything's just kind of being decided in the courts. That's going to take forever for it to shake out. So we know what that really looks like.

Yeah, certainly. I mean, you've got the EU AI act, which is sort of coming into force gradually, but the rest of the world is sort of still playing catch up, I think. And Jonathan, are there any regulations for finance that people need to be aware of?

Well, I mean, the huge amount of regulations which cover financial data, for example, insider trading. So the Use of data to give somebody an advantage because they traded on non publicly available data. And how do you define non publicly traded data becomes a very big issue.

Yeah, I'm curious about that because I always thought insider trading was very much about. Well, I overheard someone having a conversation about an ipo, which probably wasn't actually a TV show, and then I made a trade on this. So, yeah, if you're just using data sources, tell me what can go wrong here.

Well, for example, what people have done is they've supplemented the data sources with the hire of consultants who have insider knowledge. And that's how data has historically been conveyed. And there have been several court cases as a result of that. So data can be communicated both verbally, by machine imprint, etc. So that's one issue. The second issue, there's a lot of data which is mandated by regulation. People can manipulate that data, and we've seen court cases about that.

Okay, all right. So basically, be careful of what your consultants tell you and just make sure you're not breaking any laws if you act on that. But yeah, that's pretty interesting that you just need to be a little bit careful about making sure that you've not got data that's not publicly available when you're making your investments.

In fact, these days, when you supply data to hedge funds, they ask you to complete long due diligence questionnaires to make certain that they are not in receipt of any data they should not be receiving.

Okay, that sounds very, very sensible. I know these hedge funds have good legal departments in general, so, yeah, best pay attention to those people. All right, super. So just before we wrap up, what are you most excited about in the world of data at the moment? And you're not allowed to just say generative AI.

I think more and more data is becoming available. I think more and more things are being measured. More and more companies are realizing that the data they gather in their normal course of business can be made commercially available because it has other uses. So the general availability of data is increasing, and we are definitely not short of projects of bringing more data to market.

I do like that. Yeah, there's more data, more uses of data, and it's no longer big data. It's just data now, and you do whatever you want with it. All right, super Scottish.

Jonathan, thank you for giving me time to think. Because my answer, generative AI. Look, there has been a paradigm shift in the way that companies are collecting this behavioral data, and it's exciting to see. So up until a few years ago, the paradigm was if I ran a website, I had to know as a product owner which moments in the customer's journey were important. Right. Add to cart, change the size, hit checkout complete. So I had to know that. And then I had to go convince an engineer to write code around each one of those moments, wait for that data to populate, and then get sort of a lackluster report telling me who did. The problem with that is that it's a very biased set of data and that people have a hypothesis what's important that may not be what's important. It's expensive and then it's incomplete. There's been this movement to something called full capture, where by default you're just collecting all of that data privacy friendly way, and then you're able to get retroactive. And so I think the result of that is going to be just better digital experiences for most of the users out there, because site owners, app owners are going to finally know what was working and what wasn't.

That's very cool. That sounds like it's sort of radically changing the whole AB testing approach that product teams have.

I mean, just even think about the idea of A versus B testing. Why is it not A, B, C, D, E, Z testing? It should be all of those things as you develop towards personal experiences. But if you didn't have the data to understand the behaviors, you couldn't really do that.

Wonderful. Okay. Yeah, Sounds like we've got a revolution on our hands. So thank you both. This is a fascinating conversation. So, yeah, thank you both for your time.

Yeah, Richie, thanks for having us.

Thanks for having us.