The credits roll after the last season of How I Met Your Mother, and Netflix conveniently suggests that you might enjoy Friends or New Girl. Where did that suggestion come from?
Netflix's recommendations are based on algorithms that identify features of shows you've watched in the past and match them with other shows in the company's database.
But according to Shipra Agrawal, a professor of industrial engineering and operations research, Netflix shouldn't just recommend shows based on what you've watched in the past. Instead, Agrawal has proposed an algorithm that would create better recommendations based on both exploitation—capitalizing on a user's past responses—and exploration—introducing a user to new material.
Many users rely on recommendation systems to find new content, which can range from movies to books to clothing. As a result, these algorithms have a major influence on the spread of information, as well as a user's experience with a product.
Although previous recommendations are a good predictor of what will be successful in the future, they don't account for a user's unexplored preferences.
"I could keep recommending you chick flicks and maybe you keep liking 70 percent of them," Agrawal said. "But that doesn't mean that there isn't another genre—let's say, political fiction— which you would have liked 90 percent of them."
This balance between exploitation and exploration is especially important when platforms try to improve long-term recommendations. For streaming services like Netflix, it is advantageous to expose a user to new genres early so that the user's response can be used to shape future recommendations.
"Your recommendations should take into account that these are the only data points that you're going to have available in the future," Agrawal said. "You should incorporate exploration now, which means that you should think about what kind of data you want to have, and that should also be part of deciding what movies you should recommend."
However, there is also risk associated with exploring too much when users are already confident in their likes and dislikes.
Agrawal's algorithm can calculate the amount of exploration that a user should face along with the genres that the user has already sufficiently explored.
Her algorithm differs from others currently in use because it can take into account variables for the many complications that arise in real-life applications. Variables can range from the changing preferences of a single user to the limits of a finite amount of content.
"The algorithm automatically figures out which are the areas that you need to explore, which are the areas you are confident about," Agrawal said. "It will observe your response, and it will then tell whether the recommendation was good or not. But it will also keep account of what kind of recommendations it doesn't have data for."
The utility of combining both past success and future possibilities in making decisions has applications beyond film recommendations: Casinos, advertisements, and basic inventory management all feature choices based on combining past data and unknown risks.
"Think of these possible choices as different machines in the casino. … One machine is better than the others. If you pull that machine all the time, then you will have maximum chances of earning," Agrawal said. "But you have only a limited amount of money—you cannot try every machine enough number of times and then decide which one's the best."
"Should I explore the other machines or should I stick with this one?" she said. "At what point should I switch?"
These questions of choice do not exist within an academic void. Agrawal will present her work alongside other researchers at Data on a Mission, an event hosted by the Data Science Institute, as part of a larger discussion of how data can be analyzed to reveal connections and make better-informed decisions.
"In my opinion, data science is about using the response to decisions to improve decisions," Agrawal said. "I think of data as a response to what we offer and what we do. ... Can you learn over time to improve your future decisions?"