In this introductory lesson, the instructor explains why a concise, program‑friendly syntax is required to translate human‑readable HTML navigation into code for web scraping. The focus is on XPath, one of the two most common selector languages. The speaker draws parallels between XPath’s single forward slash (/) and directory or URL navigation, showing that it moves one level down the HTML tree. Bracket notation is introduced to index sibling elements, illustrated by selecting the second <div> among several children of <body>. The double forward slash (//) is then presented as a way to select all descendants of a node, such as all <table> elements anywhere under a specific <div>. The lesson emphasizes that XPath is language‑agnostic, usable from Python, R, and other environments, and that these basics are sufficient to start writing simple scraping code.
Chapters
At this point we've run through the basics of HTML with some wordy ways to describe how to navigate to particular elements. However, if we want to describe where these elements are within our programs programs made to navigate and scrape HTML, then we need to build up a standard program friendly language or syntax to do so. You have noticed that all your exercises at this point have been multiple choices, and this is because we didn't have the necessary understanding of how to turn our wordy navigation of HTML into a variable for the computer to ingest. That changes now this lesson will give a crash course in some basics of what's called XPath notation, one of two common choices for this purpose. And in the next chapters we will go deeper into both syntaxes with many more examples jumping right in a simple XPath string we could write in Python is given here. One nice property of XPath notation is that you might already have some familiarity with similar syntaxes because it uses a single forward slash in an analogous way as you do when you are navigating directories or typing a URL into your browser. The single forward slash moves us forward one generation. In fact, if we think of the tag names as the directory names, then these simple xpaths will look very much like navigating between directories. What might seem unfamiliar are the brackets. These brackets are used to help specify which element or elements we want to direct to. For example, there could be several div elements which are children of the body element, that is Several div siblings. So we can use the brackets to narrow in on the div element. We want to illustrate the sample XPath string we wrote in the last slide. Here we have highlighted the div element which would be selected within a tree representation of some HTML. Notice that the number 2 in the brackets of our XPath expression refers to the second div element of the three div elements ordered from top to bottom. As usual, paying attention to the fact that the first child of the body is a span element, so is not counted when looking at the div elements. Another important feature of XPath notation is the double forward slash. Using the double forward slash tells us to look forward all future generations instead of one generation like the single forward slash. So for example, we could navigate to all table elements within an HTML document by simply typing double forward slash table. Or we could want to restrict to a specific div element, say the one we learned how to navigate to in the last couple slides, and navigate to all table elements which are descendants of that div element. And that's it for now, we have only just scratched the surface of XPath notation, but we've gone deep enough that you can begin to write some code and get your feet wet navigating HTML computationally. Let me emphasize here that XPath is general, meaning that it is not Python specific. So if you decide to start scraping the web in R, say most libraries there will also be able to read and interpret your.
