Using Data to Build Seamless Digital Experiences | Developer Day 2021

Everyone, Happy Developer Day. This is always my favorite day of the year because I love hearing about what you all are up to and seeing the awesome stuff you've built since last year. Thank you so much for sharing and letting us be a part of it. I really love it. Hopefully this session using data to build seamless digital experiences will get you excited about what you're going to build next because we have some exciting product updates and betas for all of you. Before we get started, I wanted to quickly introduce myself and let you meet the other speakers as well. My name is Will Spiro, My pronouns are he, him, and I'm a product manager at HubSpot focusing on how customers can leverage data to power their digital experiences. I've been working at HubSpot for a little over five years and when I'm not working I generally like to do anything outside. I'm also joined by Steph and Gobi today, Stephen, senior software engineers at HubSpot working on the same problem, who will introduce themselves in a little bit. Just a quick agenda before we get started. I'm going to start by providing a quick overview of the concept of using data to power digital experiences. Next, I'm going to talk about the data layer of the CMS and some exciting feature updates. Then I'm going to pass it off to Steph and Gobi to do some live demos. We're going to talk a little bit about what's next and then we'll have some time at the end to answer some questions. But before we get started, just a quick legal note. Please note that during this presentation we may make forward looking statements and it's possible circumstances might change after the fact. That might change if something we say actually happens or not. So for this reason, please do not make any purchasing decisions based on any of these forward looking statements and please keep any sensitive information to yourselves until something is widely announced. Okay, but now to the fun. Let's dig into why you should care about using data to power websites and digital experiences. For anyone who's not read it, I highly recommend checking out this blog post. Angela DeFranco, a former VP of product at HubSpot, wrote, It's called the Evolution of Web How Websites Are Becoming More Than a Pretty Face. In this blog post, Angela talks about the future of websites and how websites have evolved from brand assets to growth tools. No longer do we just need a website or just a beautiful website, but we need a website that drives business results. And with this evolution, we've learned that companies aren't just Looking for brochure websites. They're looking to design and build sophisticated, interactive digital experiences that will power their businesses. And this is cool and makes sense, but it begs the question, what does a sophisticated interactive digital experience actually look like from an end user's perspective? How does it work? How does it drive business results? And also how do you build it? But let's take a big step back and actually jump into the problem here. Why is it a problem for websites to just be beautiful in nature? From the perspective of our website visitors, what problem do they face? Well, the reality is that many digital experiences don't live up to their expectation and are anything but seamless. Just because a website is beautiful does not mean it is functional. And that often comes at the expense of our visitors. And we can all relate to that, right? We've all had frustrating, slow, two dimensional customer journeys that take much more time than you expected. You might get as far as to the point of being able to make a purchase, but by then you might not be super excited to continue your relationship with this company. Or even worse, your bad experience might start after becoming a customer. And as Anthony, Laurie and Gregory mentioned earlier, we want to make solving this problem for your customers and prospects easier. We want to enable you, our customers, to build remarkable digital experiences for your customers. You should be able to turn that frustrating journey from before into something that looks more like this. You land on a website and instead of getting stuck on a form, you're able to log into a personalized interactive portal that can guide you throughout your entire journey. Once you're logged in, you're presented with all the information that's relevant to you. You might browse recommended products, engage with the sales team, generate a custom quote, and maybe make a purchase. The journey should be quick, easy and painless. And and it shouldn't end when someone makes a purchase. A great digital experience can act as a guide throughout the entire customer journey. And now, obviously, every product and service should not have the same journey. There are many factors that might change the experience depending on the nature of your business, but you get the idea right? The key is creating as seamless a customer journey as possible. Based on what we know about customers and prospects, you might be wondering what actually is the key to these seamless experiences. At its core, the key to building these seamless experiences is all about using data to power them. As your business grows, you curate more and more data from your customers, from your prospects, from your marketing, sales and service teams. Using this data to power personalized digital experiences is how you can create that amazing experience for your prospects, for your customers, and also your internal teams as well. And you're probably thinking to yourself, if I'm using HubSpot and all my customer and business data is in one system, it should be easy to use my data to power content, right? Well, it sounds simple, but we're not making it as easy for you as we'd like. First you need to learn Hubble. Then to get blog data, you need to use our blog Hubble functions. To get CRM data, you need to learn the CRM objects Hubble function and then separately pass data into the CRM Associations Hubble function. But first you need to hit some API to find out the association definition ID of your CRM objects for Hub DB more functions. You might even need to learn a whole other API for knowledge base. Well, I guess you can't really use your knowledge base data. Okay, this is a lot. This is all adding up to a super complex web of things to learn, code to duplicate, docs to read, and confusion to be had. We're not making it as easy for you all to access your data to power digital experiences as we'd like. Let's dig into the new data layer of the CMS and how you can utilize it to better access data in your websites and create richer experiences for your customers and prospects. We've been focusing on the presentation layer of the CMS for the past couple years with the introduction of themes and improvements to modules. But we're now investing in a refined data layer to sit next to the presentation layer. We want to provide a cohesive layer for you to interact with to power your digital experiences with data as you grow, integrate more data sources into your business and try to build more seamless user experiences for your customers. This refined data layer of the CMS will make utilizing data across your digital experiences easier to implement and learn. Now, Gregory gave you a little bit of a preview of this, but at its core of this data layer is a brand new GraphQL API. This is going to let you query for all of your HubSpot data and its relationships in a single unified layer, and you can use it in CMS hosted content, external content such as jamstack sites or apps and integrations. It unlocks all data under a unified API and provides a straightforward way to connect that data to your websites and apps. This unified layer is going to make it much easier to interact with all of your HubSpot data across channels, rather than needing to learn a bunch of disparate ways to access your data. The core queries can be used throughout websites, apps and integrations, meaning less learning, less coding, and more time solving for your customers. Today we're going to be focusing on how you can include GraphQL queries as part of your theme for use in templates and modules. But you can use the same GraphQL queries in our GraphQL endpoint to use your data in any channels or integrations. So you might be wondering why GraphQL instead of Hubble for Data Query There are a few main benefits. First is that GraphQL is an open and widely accepted data query language that's rapidly becoming the standard for API based data access. Second is itself documenting, which we'll show you a little bit more about later. And lastly, it will allow you to get all of your HubSpot data and all of its relationships in just a single query. Rather than needing to learn separate functions, make round trip API calls to get associated data. So let's take a look at a simple example. If you wanted to create a portal style page that shows user their tickets behind CMS memberships, you'd need to make multiple round trip Hubble function calls and for loops. And you might even run into the Hubble function limits trying to get all the data you need for that portal, let alone any other blog posts or knowledge base articles. With GraphQL, a simple single query allows you to fetch exactly what you need without making Raptor API calls or looking up strange IDs. But enough talking for now. I'm going to pass you off to Steph and Gobi to check out some of these features.

Hi, I'm Gobi and I'm an engineer in the CMS team and I have been at HubSpot for three years now. I have Steph as well here.

Hi, I'm Steph. My pronouns are she, her, and I'm also an engineer on the CMS team. I've been with HubSpot for a little over six months now.

Steph, I learned that you built a new app using new data layer features. Can we talk about what that app does?

Absolutely. So we created an app for a fictional recruiting agency to demonstrate the new data layer features that Will just outlined. The concept is that this recruiting agency could post jobs for a number of different companies in a listing page. Candidates that view the page can filter the jobs that they want to see by using the location department and company filters at the top of the page. So for example, if we wanted to see jobs available at HubSpot, we could just change the Dropdown. If we wanted to see jobs available in the engineering department at HubSpot, we can filter it down further. So then if we click Apply on one of these cards, you'll be taken to an instance page where you can see more details about the role, such as salary, range, location, more detailed description about the role, and an application form for candidates to apply. Also, at the bottom of the page, there's related roles at the same company and same department that the candidate might be interested in. So once a candidate applies for a role using the form, which I'll do now upload a sample resume that I have available and they hit submit, a contact record is created in the CRM and the application that I submitted is associated to that contact record. So if I preview this contact record, you can see all the jobs that this applicant has applied for. From there, contacts are automatically added to a list which allows candidates to create a login for the site. So this list automatically adds any new contacts that have an associated job application. As you can see on the filter to the side here, it'll send them an email to register an account. Once the candidate signs up for an account, they can log in and do all of their submissions. Since we know who is logged in, we can pass that user context into GraphQL queries to fetch data associated with the logged in contact to create scalable personalized portal type pages. For example, if we go back to the site where I'm already logged in, there's this application listing page which will allow me to see all the applications that I've submitted.

That's cool stuff. And I guess all these pages are powered by these new GraphQL queries. Let's dive into the listing page that you built. How did you implement that listing page with the GraphQL?

So I created two CRM custom objects to store all the role details and the job applications that have been submitted. And I created associations between them. So if we preview one of the job applications, you can see we have associated contacts, the person that applied for the job, and an associated role. Now if we preview that, you can see that I've associated the company, the role that the or the company that the role is being posted on behalf of, any deals, if there were any, and then the applications that have been submitted for that role. I wrote a GraphQL query to get all the data I need for the listing page. So let's hop over to the design manager real quick. This is the GraphQL query that retrieves all of the listings for the role listing page. This query gets the roles we're going to display as well as the company they're related to. This query also retrieves the values for populating the filter dropdowns.

You are bringing all the data for your page, including the associations, in just one single query. For a minute, can we discuss how we used to do this before without the GraphQL and compare it with this new approach?

Sure. As will mentioned, we have to use multiple Hubble functions to grab all the data that we needed to render the page. I have here what it might look like if you were to write the role listing page using vanilla Hubble. You'd have to know the CRM objects functions to grab the roles and the companies. You need to know the individual CRM object function to grab a single company, and then you'd have to know the CRM associations function to grab the company related to a role to display the name on the card. And for that you need to know the association id. All of this requires me to write more code and I would even run into the Hubble function limit for the association functions. Now I can use just one GraphQL query to bring all the associations in one go, which makes my code much less complicated. Let me show you how you can create and execute queries for your portal using graphical. So this is the graphical UI for my portal. This is a playground where you can write queries and try them out. With this, I can quickly create a query to grab roles and the data I might need to display them on a page. Graphical has two ways to edit your query. On the left is an explorer which shows all your HubSpot data, where we can simply click on the pieces of data we want to add to our query. So for the role listing page, we'd want to retrieve the role collection, the items in that collection, and then some pieces of data we might want to show on the listing page, such as department description, location and title. In the middle of the screen here you can see the query being generated, but you can also modify it directly right here with auto complete. So if we wanted to add the associated company, we could start typing associations. And then for this we're going to get the associated company and then we'll retrieve the items in that list and we'll get the name of the company. Now we can execute the query to see how it looks. So on the right here you'll get the results of the query using the Data in your HubSpot portal and you can also test this with variables. So on the bottom here you can see this query variable section. I have a variable set up for filtering already and this will filter any roles that are in the remote location. All we need to do is add the filter here, the variable here, and this is going to be of type role filter. So if we start typing you can see all the different filter types available, and then we're going to apply that filter to this collection. Now we can run the query again. And as you can see, the roles returned now are only those in the remote location. Then I could just create a GraphQL file in the design manager or the local file system and just copy and paste this query there.

Nice sir. This graphical UI looks very self explanatory and I think we really don't need much API documentation to understand all the available properties, associations, filtering options, etc. I think we just can simply use the left bar as well as this autocomplete options to learn about all these things. I have a question here, Steph. If you want to use my own local development tools such as VS code, is there an easy way for me to write the query without leaving my VS code?

Yeah. So to use VS code, all you have to do is install the GraphQL extension shown here by the GraphQL foundation and then you set up your GraphQL config using this GraphQLRC file in the root of your directory. And this is just defining the API endpoint that it'll use the extension we'll use to get the schema. And so we have an API endpoint available that takes your portal ID and an API key to retrieve the schema for your portal. Once that's done, you can write your queries directly in VS code with autocomplete. So I have this demo query here. Again, it's retrieving from the role collection and say I wanted to grab salary information. I could just start typing salary and I would see all the available properties on the role object.

That sounds pretty cool. I assume we can use any code editor and any supported GraphQL plugin for our local development. So now that we have this working query in our hand, how can we use this query to build a listing page in CMS?

So GraphQL data queries can be attached to both templates and modules, or even used outside of HubSpot on a website hosted externally. Similar to how you use the public HubDB API today, you can attach it to the template using the template annotation data query path and then you can access the data returned from the query using the context Key dataquery. So back to the Design Manager, we go to the application listing template. You can see here in the template metadata, we've added this data query path, which is just a relative path to the query in your folder structure. And then if we go to the module that's used on that template, you can see we're accessing the results of the executed Data query using DataQuery to get the contact and their associated applications. So you can use the attached query anywhere in your templates code and as we just showed in your modules code, if you want to. Also, you can attach it directly to a module by defining DataQueryPath in your module meta file. For modules, we only support this via the local file system for now and not in the Design Manager. This is the metafile for one of our modules, the role listing module. You can see data query path. Again, it's just a relative path to the data query in your folder structure. You can access the data query response inside the module code using the context query, the module data query. So if we go to the role listing module that has that data query attached, you can see we're retrieving the data using module data query. So for the role listing page, I created this module and attached a query to it, and then I simply added the module to the page.

Impressive. I understood how we can connect the query to the templates as well as modules. So I've seen that you shown us, like the filter dropdowns in the listing page. Can you explain how we implemented the filtering functionality with this graphql?

Sure. So, to customize the data query, I used variable bindings. So back to the role listings query. Here at the top of the GraphQL file, there's this comment section. We have a number of variables that use Hubble to set their value, just like you would in a module or a template for filtering, we retrieve the selected filter values from the query dictionary when we execute this query. In this case, we're binding the variables location, department, and company to values retrieved from the query dictionary in the request. This is the same Hubble that you would use to retrieve this in your modules or templates. Today. When the query is run, the Hubble we have written will be executed on the server and bound to the query variables, which are then used in the GraphQL query itself to filter the results of the query. For example, in this role collection, we're using the location and department variables to filter these roles based on what the user has selected. In this case, we can also choose which part of the query to run via the Skip and include directives. You can see on the right here we have the skip directive for this first section if we're searching by company, and for the second section we're including it if we're searching by company. So for the first section, if we're not filtering by company, we run it because we want to start from the roles and grab the associated company. But when we're filtering by company, we have to use this section because we have to start from the company and grab the associated roles in order to appropriately handle paging in our role listing. This is a cool way you can use Skip and include to dynamically change the query you're executing. One other thing to note in the variable bindings for this query we utilize the module field in the Role Listing module, which let you select the number of cards per row and the number of rows per page in order to pull back the correct number of roles to display. This gets down to this limit variable, which is then passed to the collection to limit the number of roles that are being returned. So let's hop over to the page editor real quick so you can see it in action. So this is the role listing page with the role listing module. And you can see on the left we have two module fields, cars per row and rows per page. And if we update those and click Apply changes, the query automatically retrieves more roles to display.

Nice. I see how we use the variable binding to supply the argument values to the query from both query dictionary as well as the module fields. So now that we spoke about the listing page, can we talk a little bit about the instance pages for each role that you create?

Steph sure. So for the instance page, we attach the data query in the Dynamic Pages section of the page settings. So if we go over to the page editor for the Role Details page and we scroll down to Advanced Options and then scroll down further to Dynamic pages, you can see here that we have a new section titled Data Queries, and this section shows only the data queries that can be used for a dynamic instance page. So queries that retrieve a single object by the unique identifier of that object. One thing to note, since we're selecting the data query here in the page settings, we don't need to set the data query path in the module or the template. The data query we selected uses the variable bindings we just discussed to pull the unique identifier for a role from the dynamic portion of the slug which we defined up here in the settings using this square bracket syntax so in Context slug you can define the dynamic slug portion using square brackets. We can access that dynamic portion through the path param dictionary, similar to the query dictionary from earlier. So back to the Design Manager we go to the role query. You can see we're using Hubble to retrieve the dynamic slug from the path param dictionary. We're binding it to this role identifier variable and we use that to retrieve a single instance of a role. Additionally, we can have modules on the page that are driven by their own separate data queries. For example, the Featured Roles module that shows roles related to the one being displayed on the Role Details page uses this query here. And this query is using the data query that we bound to the page to retrieve the department and the company that's currently being viewed so that we can show other roles that the candidate might be interested in. This is a really cool way you can use multiple data queries to build a truly data driven page.

This all pretty cool stuff. Still, I think these are the features that I have been waiting from HubSpot. Actually we discussed a lot of new concepts today such as GraphQL, data queries, dynamics and various ways to use these queries in cms. Do you have any documentation for our developers to start learning about all this cool stuff and then build their own data driven experiences?

Yep, we've open sourced this sample GraphQL theme project with all the queries, modules and templates we've shown along with some step by step documentation that'll walk you through setting up all the pages we've gone through today so you can have a head start on developing your own website using GraphQL. So finally, here are a few thoughts on how to get started with the data layer. 1. Fill out the beta form. We'll get in touch with you when you've been added to the beta so you can try this all out. 2. As I said, check out the sample theme and the related documentation on our GitHub. And third, and most importantly, have fun and let us know if you have any feedback. Now I'm going to pass it back to Will to briefly discuss what's next.

Awesome. Thank you so much Stefan Gobi. While I'm not able to divulge too much information about what's coming, here are a few hints at things we're thinking about. First, we're going to be further fleshing out this GraphQL layer and adding all the HubSpot data sources to it like HubDB, Blog and KB. This is going to better support omnichannel readiness and general headless capabilities with HubSpot as a data store, we also want to work on a more marketer friendly editor for structured content and additionally make it easier to edit the layout of dynamically generated instance pages for structured content. Lastly, we're always working on improving the capabilities of our data sources, so as always keep your eye out for any updates and betas. Before we wrap up, here is our CTA to you. What data do you have that can be unlocked and employed to help create a more seamless experience for your visitors, prospects and customers? What features that we talked about today can you use to help make that happen? We are so excited to see what you build and can't wait to see how to utilize the features that wraps up our presentation. Thank you all so much for joining us today. Please get in touch if you want to chat, talk about the future, or provide any feedback on anything we shared today. Now we're happy to take any questions.

Thank you all so much. What a super exciting presentation. There's so much potential there. One of the questions from the audience was what are the limitations around associated objects? For example, if 1000 associated objects, will that return them all or is there going to be some type of pagination? What's that experience going to look like?

Good question. One of the things that we're hoping to implement with the help of the developer community and beta participants is a reasonable query complexity. Ultimately, we don't want you returning too much data in a single query as to not slow down your pages or apps or stuff, but we also want to expand what's possible. Today, with help from the beta participants, we're hopefully going to come up with a reasonable query complexity limit. It's something we'll also expose to you all how complex your query is so you can know how far you can go. Then beyond that, you'd ultimately use pagination or some of the concepts that that Steph and Gobi showed today to return different data in different circumstances.

Awesome. We have a lot of questions about GraphQL. Will it be possible to make GraphQL calls using client side JS?

Yeah, good question. So right now there's an endpoint available to make GraphQL queries in addition to what Steph and Gobi showed. Right now it does require authentication using an API key. I saw there was a couple questions about integrations. You can definitely use the endpoint in apps or integrations you're creating. We don't currently support OAuth on that endpoint, but that is something that we're going to be working on implementing in the near term. As far as client side JavaScript goes, it wouldn't really be possible today because it does require authentication. But we are looking into how. We could provide a mechanism for you to basically create an unauthenticated endpoint for you to use on the client side. But you can definitely use that endpoint on server side JavaScript I saw. Or a serverless function. You could hit the endpoint there.

Great.

How about where are folks going to be able to use this data? Is it going to be available? Are there going to be custom email modules available? What's that look like?

These things we can use in both templates as well as modules and it can be any module. It can be an email module or a webpage module or anything actually. Generally you pull the data using our existing associations Hubble functions or appdb Hubble functions. I think in all those places you'll be able to make use of these GraphQL query functions.

Awesome.

Another question about custom objects. Are there any plans to add other file types to custom objects? Right now files and images are unavailable. So Andrew says it would be nice to make CRM field options work like module fields. What are we thinking around that?

That's a super cool idea. By the way, comparing CRM object properties to module fields. As I mentioned in the presentation, we're always trying to make the data sources more sophisticated. Both Hub DB and CRM objects we actually do in a small private beta right now have a rich text property available in the CRM. And we also want to better support other content types like media and images and video and other stuff you see in the CMS and in Hub db. So definitely more work to do on that.

Awesome. One final question. What tiers or you know, Hubs subscriptions will have access to GraphQL and what's the best way for folks to get started? We will see you in the next session which will be our lightning rounds. Thank you all.