FINAL Episode: Gaga for Google CSE - Schema and Knowledge Graph

At Shelly's Alley, welcome. It's Friday. It's 1:00 PM Eastern time, so I must be in front of my computer staring at my computer, uh, uh, staring at myself on my computer screen in my webcam. Hello everybody, happy Friday. So, uh, a little sad and a little excited at the same time. Sad-cited, is that a word? I don't know. Well, it is now. The reason is this is the final episode of Gaga for Google Custom Search Engines. And as such, um, you know, it's kind of sad to wrap up a story, kind of like an old book that you put down and you've been reading for a while, and well, now you're gonna miss it. But the good news is that this completely wraps up Google Custom Search Engines, and I don't think there's anybody out there that has done this much coverage, this extensive, on Google Custom Search Engines. So by the end of the series, I'm thinking that you're probably going to be able to claim that you are genius-level custom search engine developers or creators or, I don't know, whatever the proper word is for that. So without further ado, let us catch up to where we are with regard to custom search engines and kind of do a quick, um, a quick recap here. So we talked about setting it up. You need websites, or at least a website to start. You can also, as you'll see today, have a custom search engine with zero websites, but you do need one to start. You need websites, then you need refinements, which are either labels or post-search criteria you can add. Then you need synonyms. So websites, refinements or labels, and synonyms. Then we talked about how to customize it and make it look good. So you can add your logo and make the design look more comfortable and even surreptitiously hide the ads if you want to do that. We also talked a little bit about promotions, but that's not as useful for us, although if you are going to use this on your website, you can promote your own services, and that may be useful. Now today we're going to talk about the fine-tuning. Let's get into that fine-tuning. Catching up to the place. Here we go. Okay, so Inside of your custom search engine, there are two ways that you can add restrictions to your search without having to get very specific on, say for example, the websites that you want to search or search strings that you have constructed. So these are kind of large, large ideas in concepts. I call them categories, or perhaps they're better off as entities like Google calls them. And you can do this in two different ways. You can use Google's Knowledge Graph entities, or you can use the schema.org construct. But both of them, in true Google fashion, have caveats, and both of them have been implemented differently inside of a custom search engine than they are externally to others. So for example, Google's Knowledge Graph entities can be called upon by APIs with a much more nuanced detail and with a lot more control because now it's going through the API, so there's a lot of other switches that you can use. Inside of the custom search engine, there are some limitations to the entities. And when it comes to schema, not all schema are supported. So if you go to schema.org and look at the list, you'll find that many of the things that are on that list are not populated in the custom search engine schema. So I'm going to show you which ones are and which ones— well, I'm going to show you which ones are because the ones that are not, well, there's no point in showing you, right? So before you jump into this, I want to show you the outcome. I'm going to switch over here to my Google custom search engine. Hopefully it remembers me from the— there we go, excellent. Now what I did in preparing for today is I actually created two examples, a schema-based sourcing search engine and then one with the entities specifically. And these are experimental, so I'm just going to kind of show you what I've done so far so you can learn from it. and, you know, ask some questions around that as well. Let me start with the schema-based sourcing search engine. You'll see that in here, first of all, I haven't done anything with the description or the keywords or anything, but you'll see that I do not have any sites. This is possible by adding a site and then removing it. You can't create a new search engine without adding a site. At least I haven't found a way to do that. There probably is a way. But I just created one with one simple website, and then I came back and I removed it. You can remove sites from your search engine now and, and get to the state. Why do I want to get to the state? I want to get to the state because this is how I'm going to show you what I've done with schema.org. So in the first example, this is a search engine with zero websites. So it's a custom search engine with zero websites and only restricts setup via schema.org. In full transparency though, I did configure some search features, so there are a handful of refinements that I'm going to demo, and there are of course some synonyms— very, very few because I was just creating this as a demo. I do have a live version of this that I use every day, um, that is— that has websites and labels for those websites and so on. But since this is kind of a pure demo, I wanted to show you the power of what this does by itself, just schema.org. So I've got a few refinements and some synonyms. Let's back up to the setup. You'll see that down here there's the choice of knowledge graph entities or schema.org. As the PowerPoint identifies, you can have two options with regard to knowledge graph. There's a difference between knowledge graph and schema, and this becomes the biggest difference. A knowledge graph entity is a specific entity. So you are asking Google to come back with results that include that entity. Your search criteria apply to that entity or only web pages that contain that specific entity. So rather than a website or a keyword, you're asking for a concept in an XML kind of schema, right? You can apply an entity search to the whole CSE so that it works across all of your refinements, or you can apply an entity search to a specific refinement. In my personal opinion, it is much more useful to use entities in refinements as labels than it is to apply an entity to the entire search engine. However, the exception might be if you have a very customized focus search that you want to do and you don't want to limit anything else but that one particular entity, then it may make sense to use an entity in the whole CSC, meaning right here. For now, the first example I'm going to do is I maxed out the 10 allowable schema types that Google permits, and I added them all here. To add them, you start typing and you'll see it come up as a dropdown. So if I added person, you'll see that it comes up. You choose it and it gets added. There's a maximum of 10. As soon as you hit enter, you'll see it saves it. And although supposedly Google says if you add a derivative or if you add a child and in schema that has a parent, then the parent won't be searched. But if you add a parent, then all the children will be searched. So person is a schema entity. Here. Ah, come on. So you'll see here there's these types of commonly used entities, right? Well, I'm going to go to person. And you'll notice that person belongs to the schema of thing. There's basically, um, really at the top of the, of the, of the organizational structure, there's just a couple of choices. Once you go into thing, that's where you'll see person. So theoretically, if I add person as a schema, it's supposed to search all these other criteria, such as, um, you know, spouse, image, actor, author, buyer. All of these are supposed to be adopted. However, I didn't necessarily find that to be the case. Also, the other thing I didn't find to be the case is not all of these properties are actually transportable. So I tried to, for example, add email, which is one of these properties here, but email addresses cannot be added here as a restrict. So if you start typing email, the only choice you get is email message, not email address. So it's limited. And the other thing that I found is when I just tested organization, I got different results than when I just tested organization role. The idea here is this is a role within an organization, this is the organization itself. So you would think that organization role would be included in organization. It is not. I had to call it out. So these are job titles. That's probably the closest that you'll get to a job title. The other one is employee role. Between organization role and employee role, I captured most job titles, not all of them, but a lot of them. Then I have person, company, business function is sort of like employee role, but when I tried it, I only got a couple hundred results. I threw it in there anyway, because it does help. So I've got person, company, event, job title, and these two contact page and contact point are kind of like the contact us pages, which sometimes will have additional information like phone numbers and emails and so on. So that's all I added without any websites. The rest of this is essentially up to the refinements and the synonyms, the refinements that you see here. So what did I add for refinements? Well, Let me go back to that for just a second after I show you what the public search engine looks like. So this is the public search engine. Notice I did not do any customization. This is just a rough example for the purposes of experimentation, but this is the public search from here. And for those of you trying to follow along at home, I will put this into the, um, chat for Glenn. Hopefully Glenn can publish it out to the Facebook group. So if you can grab it there. Otherwise, um, at least it'll be in the notes. We are fortunate to have Glenn today taking notes. So here's the 4— I'm sorry, 5 refinements, and here's the programmable search. Let's say for example I'm looking for a Java engineer. I hit enter and I can select— first of all, there's 299— 298 billion results, which we know isn't really true. As you scroll through this though, you'll notice that the content looks very different. But when you click on SDE, this now reduces the results to 176,000. SDE is a, um, a refinement that I added for software development engineers, and I'll show you how I did that. But what you can see that does is it starts bringing back things like blogs, examples of code, um, and other content like that. So it's getting much closer to people than we would have expected, and I didn't have to set up a bunch of different websites. Um, obviously the other examples are not going to be as relevant for Java, but we can do resume for Java, and you'll see that what this is doing is it's looking for entities that are like a resume. I'll show you what that looks like. For accountant, obviously Java is not going to work, so let's try tax and then accountant, and you'll see it starts to bring back blogs, again, bios, people being mentioned. So it's not perfect, but it is definitely narrowing things down quite a bit to the entities that it knows as accountants. Once I show you the code, you'll see what I'm talking about. And then finally, I have manager, which are, you know, basically just people that have managerial job titles— manager, director, leader, you know, etc. And then HR, and this is just, you know, the entity known as human resources. So this will be recruiters, these will be people that are other types of HR professionals, etc. So very loosely, that's what it's doing. Let's take a look at what these are. The SDE refinement is not searching only the websites with a label because there are no websites, so it's giving priority to websites with the label, which basically translates to nothing because there's no websites. But I added the knowledge graph entities. Now remember when I said that a graph entity is a singular entity? This is what I meant. Whereas with schema, you're putting together a, a schema that encompasses all of those types of objects from schema.org with all of the properties of those objects like alumni and follower and funder and all that. With knowledge graph, you're only applying a specific construct or a concept that is that entity, and only if Google has recognized it as an entity. By the way, if you're wondering how do you find those, there's a link at the bottom here that will allow you to search all of the knowledge graph entities so that you can figure out which ones actually exist. And, you know, that way learn which ones you can put in here. There's a limit of 5. You can have only 5 entities in your refinement, but you can have lots of refinements. So narrow them down. Here I chose software developer, software engineer, web developer, and Java developer, and that's it. So when I go to do the search and I select, um, something like React.js, hit the search and choose SDE, It's looking for this keyword, React.js, across all of the pages that it has indexed where somewhere on the page it has recognized the existence of an entity known as a software developer, Java developer, et cetera. Sometimes that's a blog, sometimes that's someone's profile, and sometimes you get job postings. We can defeat that with some adaptations, but at least for now you have the major concept of how this is working. Accountant, same idea, but the difference is with this one I showed you an example of what you can do when you add post-search refinements. So this refinement adds the keywords he, she, they, me, my, about, bio, profile, CV, or resume to every search that contains accountant, chartered accountant, CPA, CMA, and, um, well, I guess I don't know what a uniform CPA is, but I added that one in there as well. So this is different than the other one because it's adding those extra keywords. Let me show you again what happens. You go to the search engine and put in tax and accountant. Try that again. There we go. Okay. And you can see here it's looking through content and it's looking at, you know, resumes other tax preparers, people who are CPAs. The other ones are very similar. Resume is basically the same idea. Here are the knowledge entities known as resume, curriculum vitae, and profile of a person. Now I don't know if this is actually a silhouette or if this is an image of a person. I'm not exactly sure. I didn't have time to test it, but I do know that these other two produce resumes. I also eliminated all of those annoying samples by doing a negative logical disjunction here, so I'm removing examples, samples, and so on. And for manager, I added the bios and the concepts of manager, management, leadership role, and for HR, very similar graph entities for HR, recruiter, et cetera, and also the bio keywords. So you can play around with this and tweak it until you get to the right, combination, and then you have synonyms. So I added person and CV. Let's see what happens when I put, let's say, person here. Now I've got tax and person. Go to accountant, start scrolling through, and I notice the results are getting a little bit better. Not this one in particular, but Now it's talking about the results are from people. There's even some bios starting to populate. So you can see how just a little bit of a tweak by adding a synonym can make a big difference. And the other one would be CV. That would be slightly different because that would not be person. That would be more useful, say, on the resume side, for example. And you can see I'm actually getting resumes. So imagine if I change this to React. I guess, JS before, and go to resume, starting to get resumes without adding websites. See, I didn't add Sean Kemp's website, it didn't add Dandy Zhu's website, etc. This is just based on the knowledge entity. So that's one way to do it. That is, um, Knowledge Graph entities added as refinements. For this other example here, I added a bunch of websites. I'll get to that in just a second. And I added a few schema.orgs. I limited to person, role, contact point, uh, etc. The idea here is I just want to find people and profiles, and I wanted to limit it, not include as many things. For example, all the organization, um, pages are removed, so no companies, just theoretically people, contact points, people with a role, employees with a role, etc. But then I added a bunch of websites, about 50 of them. Now none of these are actual websites. They're all top-level domains with particular patterns like profile, profiles, resume, CV, speakers, speakers, panelists, members, team, etc. A whole bunch of those. I also removed some websites. I got rid of LinkedIn because this is not a LinkedIn search. I already have one. I'm searching for other things. I got rid of anything that's careers, employment, jobs. Probably should also eliminate Indeed and Glassdoor and all that. So I added lots of patterns, pretty open-ended, and removed a few things and made this a much more defined person schema. And then on top of that you can add some refinements. I just experimented with React.js for a friend, and some synonyms, which I haven't done any right now. So this one is a very different setup, and let's see what happens. I'm putting this on Glen's chat here so he can publish it. So this is a different search than the one that we were just doing before. This is now the kind of website patterns and knowledge— knowledge graph is not included, it's just schema.org. Let's try React by itself. Okay, so on the first pass, seems to be kind of all over the place because React is not just a JavaScript tool, it's actually also an English word. But when I click on the React tab, that reduces it significantly. Why? Because on that tab, I'm only looking for the 3 related concepts: React, Redux, and ECMA. For those of you that don't know what this is, just think of it as JavaScript or, or software tools that are related to React. There's, there's these 3. There's probably more, but these are the 3 that I found. So in this particular case, the refinement is only looking for these 3 elements and there's not a lot of them out there, so it doesn't bring back a lot of results. But look what happens if I change this to engineer or something like that and then go to React. I get profiles of people. Again, this is not very broad. I would have to add more Entities to include a few others, I probably could add JavaScript or whatever, but see what an immediate effect this has, very, very accurate. Again, it really depends on how you program that, that, I'm sorry, that graph entity here. So if I were to add maybe JavaScript and try to do my search again, Forget when you're doing this, it's probably a good idea to refresh your search. It looks like I'm getting 6 results now because I added another entity which doubled me from 3 to 6. Then I also have, because of the list of websites here, I also have a huge collection of websites that have things like profiles in them. So now I don't really need to go that specific. I could really just go back to the beginning here and use synonyms. Let's go ahead and create one since we haven't done one for this, and let's use synonyms, for example, for job titles that are related to project manager. So project, project M-J-R, we got project manager, program manager, IT project relationship— I know that's not a good one— engagement manager, and so on, right, just as an example. And I'm using an abbreviation here. Now if I put that into my search, I start to get people. Why? Because of the patterns. So let's see what happens when I add another refinement with a new entity. Keep in mind, going back to the setup here, there's only a universal schema, there's no knowledge graph entities. So I would have to add that as a refinement. Let's add the refinement of make sure you give priority and don't search only the websites. Let's add the refinement of people. Well, actually, I should call it person because that's more accurately what the element would be. So it's per-son. Yep, not coming up. Interesting. People? Oh, okay, interesting. So it is people. There you go. Interesting because schema doesn't have people, it has person, but anyway, there you have it. So I'm gonna add that. Let's see what happens when we add that refinement to our search. Oh, wow, only one. Probably not a lot of quote unquote people for that. Just out of curiosity, let me do project manager and search for people and see what it does. Eh, only a few more. So you can see that that knowledge graph entity is really limiting my search. Not a very good way to do it. The other thing I wanted to try is to just do it as a knowledge graph entity from the very beginning. So for that, I'm just going to create a new search engine, start with LinkedIn.com, and I'm just going to come back and delete it later. But what I'll do is I'll delete LinkedIn, and I'm going to add just the the one knowledge graph, which we now know as people. Now this is not a refinement. It's actually the whole thing. Oh, it's saying it's not. I did something wrong probably. All right. I'm going to not, um, not continue down this road because as I didn't think this would work, but knowledge graph entities I was trying to show you are very limited. So it's actually not even allowing me to do it. Um, let me add a website just to make sure that's not it. So I'm gonna add.com, and hopefully that alleviates the error. Let's just try one more time, people. Okay, let's see if it doesn't error out this time. Yeah, it looked— it worked. All right, let's try it now. Uh, I was doing project manager, so let's do project manager. Oh, just as I suspected, same thing. So that's the difference between the knowledge graph and the schema. I'm going to remove it here, and I'm going to add it here, As the schema person, go back to my search, same project manager, and I got a little bit more, not that much more, but I actually got a couple results. So this is why now you understand why I added all those other schemas as well as person. Right, I'm going back to that particular one and show you that one. So this is where I had person, organization, role, et cetera. That's how I got it to work. Just one of these alone doesn't do the trick. So now you know, you can search the entities here and you can only add 5 at a time, but that's okay. You can add them as a refinement, which I recommend, but you can also try experimenting with them in the whole CSE. And then here's a little list of the useful schemas that I found for you to prepare. So these are the ones that are in separate rows. There's person, role, and business function, which I found to be very similar. Organization role and employee role, similar to role and business function, but brought other results. So like it didn't overlap, there was additional results. Then org and corp, similar. Contact page and contact point, similar. The difference being one is a person, and the other one is a webpage basically. Then gender type was actually productive if you're doing diversity search, and email message brings back only email messages. Believe me, there's not that many out there. I tried it and there's just not a lot of entities in the schema that have that. So that's it. That is everything that you need to know about custom search engines. Go out there and, you know, and try it, experiment with it. Try different combinations of graph and schema. And just, you know, let me know if you have any questions. I guess that's really all I can tell you about that. Let's go back to— oops, here we go. Oh no, it's picture in picture in picture in picture. I can solve that, just a second. There we go. Alrighty, don't forget to remind other people to join us at the Facebook group, although we are planning on launching a new home destination where we're going to put all the files. Right now, unfortunately, all the files continue to be in the Facebook group. Once we launch that, we'll export everything. For the time being, if you're watching this on LinkedIn or Twitter and you want the handouts, the notes, you'll, you'll have to grab that from the Facebook group. Unfortunately, if you can't get into Facebook for some reason, or you absolutely refuse to, you know, shoot me a note and, and we'll see if we can get you the notes, um, until I'm able to actually get this all published. So with that, have a wonderful, wonderful weekend. I hope you enjoy the nice weather that we're having here today, uh, in your part of the world. Good day, good night, whatever it is. Take care. Love you all. Bye-bye.