Coolhunting: Chasing Down the Next Big Thing | Peter Gloor | Talks at Google

Thanks everyone for coming. We're very excited to have Peter Gluer here today with us. Peter is a research scientist at the MIT Sloan School of Management and he's also the co author of Cool Hunting, Chasing down the Next Big Thing. So what is cool? Well, in his book Peter's gonna talk about what is cool hunting, how we can kind of harvest that process of cool hunting and how we can explore cool hunting for solved. So thanks give everyone. Let's give a hand to Peter.

Gloria. So Jimmy, thanks a lot for inviting me here. What I really would like to do is talk about swarms and how swarms become creative and how we can discover what cool things those creative swarms are up to. And I would like to start with a really cool example. So I think that gives you enough of an impression of this opera which has been composed by a swarm, a swarm of three jazz composers, jazz musicians that work together long distance over the Internet. They met sometime 10 years ago in the Swiss mountains playing in the same orchestra. And afterwards they took off to South Africa, to the UK and to Scotland. But they continued to work together to develop things like this opera project together. And as you can tell from listening to sounds like a very homogeneous piece of art put together by an unbelievably committed group of people. I just stumbled through personal cool hunting having coffee with a friend somewhere in my hometown of Arau in Switzerland. And she told me about her conductor who also composed operas together with two other people. And the conductor, he was doing the voice and the text and then they had one guy who did the melody and another one who did percussion and together they worked and then they went off to the Ukraine to record this stuff because they didn't do it for the money. When I bought this cd, I got personal thanks because they told me I was buyer number 15. So people that are up in sworn creativity for really cool stuff, usually, at least in the beginning, are not doing it for money. And I don't have to talk to you about those examples. They are particularly in this group, absolutely well known. However, again, the point is nobody here is paid to do those sort of things. And they very nicely illustrate the basic principles of swarm creativity. Namely, we are not hiding what we are up to. And if any of you is a little bit involved into Linux and has read the Debian constitution, anybody is active in here? Linux, Debian, nobody. Okay, so then you can tell that this constitution is really very detailed. It tells we will not hide anything. And the community has reinvented itself quite a few times by Firing its leaders and very democratically choosing other ones. And that's another of those principles of swarm creativity. So in my talk, I will first describe a little bit more in detail what I mean by strong creativity. Then I will introduce to you a tool we have developed over the last five years which has had a couple of names. And the latest version is called Condor, where we do social network analysis of people networks, but also of content networks to hunt for those creative swarms. And in particular, what we are hunting for is what I call the coins, the collaborative innovation networks. Now have been pointed out that I better call them cins sins, but scenes doesn't really sound very sexy, whereas coins sounds much more attractive. So cool hunting for me means finding those collaborative innovation networks, which, as we will see, are very particular types of, of social network structures. And once we have found them, we hunt for them. We also would like to nurture them to help grow those collaborative swarms. And that's what I call cool farming. So in the very last part, I will talk a little bit about that one. Already making a little bit of advertisement for the next book on the way, which will probably be called Cool Farming. So what do I mean by swarm creativity? Obviously, if you look back at the three examples I gave, the Internet is the big enabler of global swarm creativity, because only now is it possible to connect small and large groups of intrinsically motivated people that don't have to have big budgets to work together over long distance. However, I would like to make the point that you don't really need the Internet to hunt for those collaborative swarms. And I discovered that really impressively when I was in Ghana last summer. I was there for a charity where we collect used computers and then we try to install them in schools and hook them up to the Internet. So I was there last August and we were on the road with our truck when suddenly it broke down. And how things work in Ghana. Out of nowhere, there comes a little boy. This little boy calls somewhere, I don't know whether they use bush drums or whatever. And then a mechanic came on his bike and he went under the car and he banged on the car and he exchanged some mysterious parts and the car worked again, sort of. So in the end, we had to go to the main distribution or shopping center in Ghana, which is the Makola market. Huge creative chaos. And what they have there is they have heaps of spare car parts ripped out of cars and clustered by brand and by type. It turns out our car, the clutch cable, had a leak because a stone had probably made a hole in it, so we had to find another one. And on this market there, it's all, it's a network staffed by people. So again, a little boy, we told him, we have a Nissan who broke down, we need a clutch, cable. And then he knew what heap to go. And we bought this part from a vendor, we put it in the car. Again, out of nowhere, mechanic came on his bike and fixed our car, sort of. So in the end we had to go to the only single Nissan dealer, authorized dealer in all of Ghana. And they tried to replace our part, but they couldn't. And they could only find out because they had the Internet. And what they found is that the Nissan of my friend was imported from Switzerland, whereas all the other Nissans came from Japan. And so the feat was not 100%, but just banging at some magic places. You got the car to work, sort of. But in the end, to definitively fix it, we needed the Internet. So I think this story illustrates very nicely creativity of swarms, what you can do, even if you have very low budget. But sometimes there is just no way about getting connected together. So what's the basic principle of swarm creativity here? I have put it very simply and it's not me that invented it. If you and I swap a dollar, you and I still have only a dollar. If you and I swap ideas, each of us will have two ideas. And so this attitude, this mindset of sharing, of working together, for me that's the most important point about swarm creativity. And if you think back at this three people team of opera composers, there was no leader. Artists are all prima donnas. So it was very important to them that there would be no leader. But still they figured out a way of working together. They shared what I call the same genetic mindset because they had played jazz. And if any of you is in chess, you know that you get to know your colleagues really well and then you are able because it's a very free floating way of doing music. So you are able to pass on the control, you pass on the melody to your colleague and so on. And this goes around in circles. And that means you very well know how each other ticks. Same system as in the Makola market, where you know very well how this market work, where this little boy is capable of finding the heap of car parts. To me it's just one big chaos. But for them it's all one, like a beehive, one creative swarm. So that's sort of the big picture. And now I would like to Talk a little bit in more mathematical terms. How can we find those creative swarms and my sort of hammer, the tool to look for those people is social network analysis. Who of you is familiar with social network analysis? Okay, so that means I will show you the first picture that's sort of a one picture introduction of a social network. It's fittingly a music example. We have three musicians and they play together. And then it shows the network of people who have ever played with them and with their friends. And what, that's usually what you do in social networks. You have, they are called actors, which of course in la I have to be careful using this term. But in social network analysis every person is called an actor. Or it can also be gorillas, it can be soccer teams, it can be countries. And the closer they are together, the stronger the tie. And if you get entire networks, you apply graph theory. And that allows you to do all sorts of interesting things and to discover graph structures, hopefully of innovation. That's what we are looking for, of new ideas and how they spread. And if you have found those patterns, we can do cool hunting. That's the simple idea. So what we have developed is a tool which was a year ago called TechFlow. And then my daughter said to me, it's like a big bird that flies above large groups of people and tries to get the pattern of network. So let's call Condor, okay? We are always open to creative suggestions. And what it does, you get those graph structures as movies and you see how they change over time. It also loads all sorts of communication archives automatically, like the web, like large email archives. So just exchanging emails, that establishes a social tie. And the more you and I exchange emails, the stronger the tie. So that's the basic idea. And now we look at those structures over time. By the way, if there is a question, please, I don't want to keep this warm back, so just ask it anytime. And also what Condor does, it combines content. That means actors can be people, but they can also be terms. And I don't have probably to tell you about tf, IDF and these sort of things. So that's no rocket science. We just do out of the box information retrieval to extract most significant terms and relationships. And we mine your search engine quite a bit to get good ties. And then we are able to do things like what you see here quite in a small picture. This is the one below in the lower left, what I call the contribution index. And if we for example, take an email network, it tells you who you are within the community. What it means or how it's done is the farther to the right you are, the more active you are. It's basically just messages sent and received. Plus it looks at are you more of a sender or a receiver? And in each community you usually have one person which is in the upper corner here, the most active, that's the community coordinator. And then you usually have people here, and those are usually the community leaders because they get more messages than they send. They get almost all of the messages, but they do not respond because they have no time to respond. So they are put on cc. So looking at this sort of picture gives you a very good impression of what's going on in the community. So I would expect if here you have certain project groups, you could just look at email archives and you would discover who are your hidden leaders, who are the knowledge sources and who are the knowledge sinks. You also have the people up here, those are the ones that send messages, but unfortunately they are not responded to very frequently. And then you have the people down here and those are the ones that are the knowledge sink, sort of. They know a lot, but they don't respond very rapidly, which is not a good thing either. And if we do that over time, we get those curves and we can look at positions of people inside a social network. And there are two metrics which are very useful inside such a social network. Basically what we are looking for is for the centrality of a person inside the network. The more central you are, the more important you are. That's the theory. Now, there are different types of centrality. For example, you can just be connected to lots of people, but as you know, that's easy to spam. Like if you take PageRank or whatever that means you just get people to send you lots of email messages, your friends, and you send lots of messages to your friends. And that will boost your. It's called degree centrality. But there is something else which is called betweenness centrality. And this basically looks at this graph and then it takes all the shortest paths through the graph. And the more you are on this shortest path, the more important you are because you control the flow of knowledge. And that's a much better metric for importance. So depending on the. And there are more of those metrics, but those are the two we use most. And they will tell you how important you are for the later you can think of. If you have two friends on the world, you have a degree of two. If those two friends are George Bush and Bill Gates, you have a betweenness which is very high because you sort of inherit their betweenness. And our tool now takes all sorts of input because you can do this sort of thing for websites, for online forums, for email, for phone archives. We even did projects where everybody is wearing a social tag, a badge. And we measure interaction using infrared sensors. And the more two people interact, the stronger their tie. I will show you what you can do with those things. All sorts of very interesting cool hunting applications. And after that, we get the movies. So let me just show you here how this looks like. I could run it a live demo, but it would take some time to start up the system. So I will just give you an impression here. So this is an online forum of drug abusers and they talk about how to use oxycodone to get really high, really quick. So the upper window is sort of the content window and the lower one is the people window. And we have just parsed the forum, loaded it into MySQL database, which you can run on this laptop or any place else. And then you see, for example, who are the most active, the most important people talking about oxycodone abuse. Or you can also look what a certain person is talking about most. So this works both ways. And the beauty is that the social network concept can be combined for actors being people and actors being concepts or terms. So. It's amazing what you can find on the web. Okay, Anyway, so you can do more things with this sort of tool. You can even get very characteristic structures out of the community. And here, what we have done here, we have traced how the structure, the social network structure changes over time. And that will tell you whether there is an innovation group, a learning group or an interest group. And the way how this has been done is we have here, Sorry, we knocked it off by accident. So the screen, it'll be right back up. Okay, should I go on or should I hold back? Yeah, you just don't have the screen for a second. Okay, give me just one minute. Sorry everybody. Well, I think I will just not talk about these technical details too much anymore. But it's all on our website. Lots of stuff about this tool and what you can do with it. And go on and talk about the coins, the collaborative innovation networks. Because now we know what we are looking for. Swarm creativity. We know the tool. And now the question is how can we map properties of creative people on their social networking behavior? And to do that I would like to introduce a very well known creative person. No picture yet. So I have to introduce it by saying that is Ben Franklin, great role model. No Internet, but he had his own coin. If any of you has seen some read his autobiography or movies about him, you know, he had this was called the Junto or Junto or whatever, which was a group of people that he brought together, applying all those principles of keeping himself in the background, trying to give credit to other people, very democratically organized. And so here we have him, and here we have one of his innovations, the Franklin stove, which also is a great example of how he decided to give away cool ideas for free. He refused to patent it. And that means that after just a few years after his invention, it was already spread out through the entire US and Europe because it was such a superior way of heating a house compared to having this open fire pit if you couldn't afford a more fancy stove. So he's a great role model of innovation and of spreading out innovation. The way how he got out his town, libraries, postage stamps, printed money was by seeding his coin, his group with ideas, then keeping himself in the background and let the other people carry those ideas. What does this mean in communication terms? I structure or characterize communication behavior of people along three dimensions. And the first one is connectivity. So the more friends you have, the more connected you are, the better you are in spreading out your ideas. And that's something which is very easy to measure in social network terms. So if you look at mailboxes, if you look at your communication pattern through phone archives, through RFID tags, we can tell those things. Now there is of course the big question of privacy. And we have spent lots of time of coming up with ways of looking at structures while still respecting individuals privacy. Some of those ways are actually hidden in the tool and I'd be happy to talk more about this in the Q and A session. The second dimension is how interactive you are. And if you look at your own co workers, you want people that respond to your emails and your phone messages very quickly. So the more responsive the better. And again, something easy to measure automatically. The third thing you need to be a real Ben Franklin and willing to share applying these ideas of swarm creativity. And that's something we can figure out if you look at the content. So and what we are looking for is those people. And again this is something which we can very easily do automatically if we now look at how those people collaborate. We come to the coins. And as an example, look at the way how Linus Torvalds created Linux, he teamed up with a few friends, he sent out his first source code as soon as he has written it to solicit feedback. And he created the first coin. Everybody talking to everybody according to those well connected, highly interactive and sharing principles. Over time there was a group of people that weren't the core hackers anymore and they joined the team as a learning network. And then over time there came even more people which were just interested in the stuff without necessarily wanting to learn all the details and without wanting to join the coin. But this is a great engine of innovation and it works extremely well. Other example, the World Wide Web, I was really lucky. I was part of. I was working at the Lab for Computer Science at MIT at the time when the web was created. I was even at this Hypertext 91 conference. I even had a paper there about a system called CyberMap, which was sort of before the Web times. So we were building cyber maps of the Bible because there were no big good data collections. And there was this guy who asked me whether I had a paper. I had a booth, we were allowed to bring along our computers and demo our stuff. And this guy told me, I'm also from Switzerland, can I come up and put some of my papers up at your booth so that other people will see it also? And also put up a poster besides your poster. And because such a nice and humble fellow, I was very happy to do that. And so Tim Berners Lee, he did that with other people also, by the way. And he succeeded in getting off his ideas almost single handedly and in a very similar way to Linus Torvalds and to Ben Franklin. He had this group of people, he had close to zero budget, like my three jazz composers. But because he was so excited, because he shared everything from day one, it worked so well. And now companies like yours are doing extremely well thanks to the ideas of those coin creators. And so the point is, how can we find those people? I got some exposure into that myself. When I was after my time at MIT working as a consultant for Deloitte. Our job was to get the E business concept off the ground in 96, 97, 98. And after I had left Deloitte and went back to MIT, I still had my email boxes. And that was perhaps 2001. I loaded them into the tool and I got those structures and that was the inspiration for the coins. Because you see the core team, the coin that were the people working with me in creating cool stuff in consulting, which as some of you might know, just means coming up with buzzwords and slide decks. And so that's what we did. And then there were the people in each of the European countries. My job was to lead the E business practice for Europe. And they took our stuff and they created local consulting groups so they were the learning network. And then they attracted other people in each of the practices to work with them and get it off the ground. Same structure. Here you see some more snapshots over time. How that's again, my own mailbox, the coin started, how it grew over time until we had those coins at all European countries. So that's the big picture. And now the question is, how can I find the Ben Franklins inside of those networks? And here, the insight we came up with, which I discovered when teaching a course about this stuff with students in Helsinki, in Germany and at mit working together long distance, which means I had great email data of the students working together. So the insight was don't be a star, be a galaxy. And what that means, if you look at the communication structures of those students, don't communicate as a star, but communicate as a galaxy. And it's really amazing. You can load your own mailboxes and you will discover how you communicate, and you can load group mailboxes. And again, you will see whether you are stars or whether you are galaxies. And Ben Franklin obviously was a galaxy, Tim Berners Lee was a galaxy. And if you have those galaxies, you find the innovative people. So that's the basic idea. Now let me show you some examples. The first one is a research project we did in Israel together with a postdoc who had come to MIT two or three years ago. And she looked at software startups and what she had done. She had asked the CEOs of 100 startups with whom of their competitors they were talking. So she had collected manually at that time the social networks of the executives of those little companies, which means they had between five and 200 employees. And that was in 1998. And in 2005, we checked which of the companies were still around, and that was really amazing because this social network structure of the executives tells you which of the companies survive. The first interesting thing was that the ones that refused to answer the questions of Ornit in 1998 had a much higher chance of not surviving after the E business bubble burst. So just being cooperative, being a good guy willing to work on a research project is a strong predictor of success. But there are also the other ones, the social networking predictors, looking at the strong and the weak ties, because she had asked, with whom do you have informal exchanges? And with whom do you have official alliances? And both of them were a strong predictor for success. So that means if in startup mode you talk with your competitors, that will help you survive. But it's really nice because you can, in principle, you can now go back, you can analyze the communication network today, and then that gives you some statistical prediction whether you have good or bad chances. Now of course there are some others, but that's how it works. Another application of the same principle. Just using your search engine and constructing the social network of websites. The way how this has been done is we run, let's say a query for gay marriage, for war in Iraq, for gun control, for abortion. And then we look at what websites come up and then we look how they are linked. And because as you I'm sure know, there is a scale free property of the web, which means that there are a few websites which are almost linked to anything. Wikipedia usually comes up on top. New York Times is very heavily linked. You will get one connected network and that will tell you whether the Americans care more about the war in Iraq, about gay marriage, about abortion, or about gun control. And here I have another example where we did the same for the presidential candidates. Now it's a movie which is already half a year old. So I'm sure it will look a little bit different today. But what it does, it will here over time you see some of the candidates go off on one side together. In particular, I think Rudy Giuliani and Neil Gingrich at that time. What that tells you is that those guys were always mentioned together in some very prominent spaces on the web. Then you can also look at how important they are by measuring their betweenness. And you can repeat that at any given point in time, while at the same time looking at what websites are the most important to boost their importance. And usually Wikipedia, New York Times and so on, those will come up. But there are others which are extremely problem specific. Like Oval Office 2008, which is a blog that you are only interested in. If you are interested into presidential politics, you would not discover that because in the searches it would be in the long tail somewhere hidden farther down. But here, because we look at between us of website, we construct this network. It will be very central. And so you can discover this sort of information. So that's one way. And then you can do even cooler things. For example, you can look at movies, you can analyze the Internet movie database, you construct a social network of people talking about those movies and you look what they are saying about. And then you can combine sentiment analysis, you combine what they are saying with the Social network position of people. And then we correlated that with movie success. And what this permitted us is to predict what movies would be making most money at the box office. And that was very strong correlation. This looks again very similar. It's just a database. Before for the politics was the entire web. Here we are taking the Internet movie database and we see. So we just parse one forum, previews and reviews. And the top window, that's the actors, which is not the movie actors, but the social network actors, the people participating in the forum. And that's a great group to do a sampling because they care, they care a lot about movies. And so they will talk very openly about whether this is a good movie or a bad movie. And then we just check what the most central people are saying about Pirates of the Caribbean, Little Miss Sunshine, the Devil Wears Prada and whether they are using positive or negative terms. We track that over time and that tells us with very strong correlation whether a movie is going to be successful or not. So what I do here, I look for the Departed, which as you might remember, won for Oscars. I believe we also did it for Oscars. And I had a student project in December. They predicted eight out of which five got an Oscar, two were nominated and one was a mission. So statistically again, very nice result. So social network gray means they do not talk about the departure, but you will see how over time it will become bluer and bluer because the buzz about this topic really picks up. And the bottom network gives you just the terms, the concept they are talking about same movie. And so you see Vendetta is another one. It's a very nice way to get a very quick overview about the field you are cool hunting in. So that's about cool hunting. And I'm looking at the watch and I don't know how much more time I have. So I will go ahead and just say a few words about how you now can put this to use. Not just to discover cool trends, but even to farm them. And this here is another example where we looked at social networks of open source programmers. And looking at their social network structure over time allowed us to predict their creativity. The way how we measured creativity was numbers of enhancements of new features put into Eclipse. It was the Eclipse community. We looked at the advantage of Eclipse. It has a very well structured mailing list format, so it was very easy to parse and also lots of groups. So the sample was big enough. And performance we measured as numbers of bugs fixed, normalized over time. And what we were able to say is if you look at how betweenness centrality of a community oscillates is an indicator of performance. The less it oscillates, the more high performing, the more bugs you fix. The more it oscillates, the more creative you are, the more new features you have. And intuitively that makes sense if you are a person, very creative, you go into a group, you brainstorm, and then you go back into your office, close the door and think. Then you repeat that process. And that's very indicative of creativity. I will show you some more examples because it's really amazing. Did another project in a German bank where we got their entire email archive, where we got a group of 22 people to wear for one month the social badges so we could measure their individual people social network. And then again we looked at performance of the entire bank, we looked at creativity of individual employees. And for example, we were able to track how new outsourcing providers were integrated over time into the community, how to make that, to speed up that process. These are just some screenshots from the email archive. And here we have some of the people, they still look reasonably happy wearing those clunky social badges that we had soldiered together at a joint project with Sandy Pentland at the media lab. And because we placed some of the sensors in the rooms, we were also capable of knowing where they are. Then we got movies like this one. This is one day in the life of this group where you see in the beginning some social interaction around the water cooler and the coffee maker. Then you see team meeting, you see lunch, which has very interesting structure because you face a person and just look at him or her while you are eating. So the lunch pattern we found is very typical. And then you had some ad hoc meeting in the afternoon. And we also had the people take psychological tests, scientific version of the Myers Briggs test called New FFI 5 Factory Inventory. And what we got out is that based on your network position within the community, with very high correlation, we can tell you whether you are an extrovert, whether you are neurotic as defined by this FFI test, whether you are open for new things and whether you are easy to get along with. And it's again taking those social network metrics. And remember the contribution index which I introduced some time ago, telling you how much of a sender or a receiver you are in the community with infrared tax, it just measures how close we are and whether I look into your face or not while talking together. And so the more I'm around, the more I look at you, the more an extrovert I are and the more I talk to you when not looking at you, the more neurotic I am, the more my betweenness centrality fluctuates. The more creative I am, the less it fluctuates, the easier, the more steady is my communication structure, the easier am I to get along with. So that works extremely well and you can then put that to quite amazing practical uses. So I see that time is quite progressed and that means I would like to wrap up with this one picture. What is the trick of cool farming? I think this resumes it extremely nicely. Just be part of the swarm, keep yourself in the background, immerse yourself. And that's a very good way getting new trends off the ground. Thank you very much. Yes. Have you tried analyzing scholarly networks like for example, taking publications from dblp. Absolutely. Identifying like, you know, for example, Eric DeMain from MIT? Yeah, no, we did lots of those academic networks. And it gets particularly interesting if you combine for example the patent network and the citation network and the co authorship network and you overlay those two, you can even take the blocking networks and then that will tell you who are the trendsetters. And we have done that for example in bioinformatics to trace new trends and also the people that push those trends. So that's a very good application. I should perhaps say that this entire work is of course not just my own, but it's a very big swarm of the people that has worked together here that I'm just part of. And there are two books so far that describe this entire what we have been doing. Yes.

So in this talk you focus mostly on social networks and their topology, how to analyze them to find the key people and so on and so forth. I wonder if what you analyze this enough to characterize the group. So you said that, well, a group of friends with the same focus can produce good stuff and that gives you some interaction that you can analyze. On the other hand, we being told or have experienced firsthand that it's designed by a committee team seems to start pretty much. Also you have the same group of people, maybe not the same, but you also have a group of people and they interact closely, talk to each other. So how can we distinguish between a.

Group of.

Making good stuff here, a committee that creates crap just by analyzing the social interactions?

Well, I mean There are about 20 or 30 more projects I didn't have time to show you. And what is particularly interesting is if you compare advice networks, official reporting networks, project networks, and just Overlay them and then measure the differences in position and that will, for example, bring up the dysfunctional committee or the networks extremely nicely. The question how you do collect those networks, it's always very problem specific, the easiest way. And we have done that a couple of times. You just have an online survey where you say I work with X, I report to X, I'm with X in the same team. And then if that, everybody in the organization does that, then you get your social networks and you just measure the differences. And that's eye opening and very frustrating for the bosses very frequently.

What about the one or two guys that just avoid all social interaction and then hide in their basement and then create good stuff?

Well, the point, yes. Do they create good stuff or not? Now in this one project there were two people who were hiding away in a room and you could not. I mean it became very clear that in the movie you would see how they move farther and farther away. And the question is, do they compensate by email interaction? Is their work taken on later on? And I mean the bottom line is what we are doing is statistics and using it for an individual person. That always has to be treated very carefully. And that's when we do MIT research projects, we always refuse to give back any individual data on a person because it can be misleading. And the problem you say is an extremely critical one. And the best way to figure it out in a smaller group is to talk to, to the people and find it out if it's in a big community. The trick that we found is to combine different networks. Like you take the blog network, the online forum network, the Wikipedia network, the quotation network, the co authorship network. And then you might find some people which are, they produce only very few papers, but they are so important, important and they get quoted by everybody. You get other people which are very active bloggers. You get people which are spoken about in the online forums. And then combining this information hopefully tells you more than if you just have one communication channel. But with the social badge project it's very clear. If you only look at who talks to whom, you lose the introverts. And that's why we took the email archive and combined the two and that gave us the creatives and that took care of the two people hiding away in the room because taking the email network to get their oscillations and then you see that they are creative people. Okay, so no more questions, thank you very much.