PySpark Tutorial : Recommendation Engine Types and Data Types

Content-based filtering uses item attributes like genre and actors to match user preferences, while collaborative filtering like the ALS algorithm groups users by rating patterns. Recommendation models work with explicit ratings (stars), implicit ratings (confidence from interaction frequency), and binary ratings (0/1 interactions).
Chapters
In the world of recommendation engines, there are two basic collaborative filtering engines and content based filtering engines. Both aim to offer meaningful recommendations, but they do so in slightly different ways. Content based filtering, as the name suggests, tries to understand the content or features of the items and makes recommendations based on your preferences for those specific features. For example, a movie streaming service might go to great lengths to add descriptive tags to their movies, such as the genre, whether it's animated or not, the language spoken in the movie, the decade it was filmed, and which actors were in it. So when a user like you gives five stars to a really dramatic Portuguese movie with specific actors from a specific decade, they can infer that you like movies like this and will also like other dramatic movies in Portuguese with those same actors and recommend those movies to you. Collaborative filtering is a little bit different. As explained in the previous video, collaborative filtering is based on user similarity. However, unlike content based filtering, manually created tags are not necessary. The features and groupings are created mathematically from patterns in the ratings provided by users. When you provide ratings for a product or item, whether it be a thumbs up or thumbs down, or even if you just watch a video without even giving it a rating, you are providing meaningful insight about your preferences from this behavior. The ALS algorithm can mathematically group you with similar users, predict your behavior, and help you have a more effective customer experience. While ALS can have content based applications, this course will focus on its application to collaborative filtering. But many of the principles of collaborative filtering can be applied to content based applications. Now let's talk about ratings. In the realm of recommendation engines, there are two main types of explicit ratings and implicit ratings. Explicit ratings are pretty straightforward. Examples of these are when you input a number of stars or something like a thumbs up or thumbs down. These are explicit ratings because users explicitly state how much they like or dislike something. Implicit ratings are a little bit different. They are based on the passive tracking of your behavior, like the number of movies you've seen in different genres. Fundamentally, implicit ratings are generated from the frequency of your actions. For example, if you watched 30 movies, and of those 30 movies, 22 are action movies and only one is a comedy. The low number of comedy views will be converted into low confidence that you like comedies, and the high number of action movies will be converted into a high confidence that you like action movies. These confidence scores are then used as ratings. The logic behind this is in essence, the more you carry out a behavior, the higher the likelihood that you like it and thus a higher rating. Additionally, in some cases, you may not have access to user behavior counts like this. A simpler form of ratings that still works with the ALS algorithm is the use of simple binary ratings. Rather than having a count of user actions, binary ratings just show whether a user has done something like watched a comedy represented by a one or not watched a comedy represented by a zero. These types of ratings aren't nearly as rich, but they still can provide meaningful insight and still work perfectly fine with the ALS algorithm. Now let's look at some actual data.
More from Richie

#359 My Best Friend is AI with Valerie Tiberius, Professor of Philosophy at University of Minnesota

How AI Agents Will Work While You Sleep | Ruslan Salakhutdinov, Professor at Carnegie Mellon

#358 How AI Agents Will Work While You Sleep | Ruslan Salakhutdinov, Professor at Carnegie Mellon







