Why the streaming platforms need to build new technology and not just rely on the AI-based recommendation algorithm?

It has been a couple of months since I last binge-watched on Netflix. After trying to identify reasons, one thought that started hitting my mind is – Is this because of the recommendation row we see on top of the OTT platform? And to my surprise, the answer was - Yes!

I believe this is one classic example, where even after having AI involved for years, we are not being able to surpass the complex behavior of the human mind.

Rise of the Video Streaming Industry

Amazon was the first company to introduce a recommendation algorithm. I remember how great it felt to get personalized recommendations while making my grocery list. The algorithm knew what I should add next based on my previous purchases and items already in my basket.

Around ten years later, in 2007, a company that used a similar recommendation algorithm for renting DVDs made a big change and entered the world of video streaming. No one expected it, but this move ended up tapping into a massive industry worth USD 100 billion. As a result, we saw the rise of streaming platforms like Netflix, Disney+, Amazon Prime Video, Hulu, and HBO Max. It completely changed how we watch movies and shows, and it had a huge impact on the entertainment industry overall.

Content Recommendations on OTT Platforms

Although Netflix's algorithms extensively utilize years of user data to develop their recommendation system, it still operates using both closed-loop and open-loop systems. In the closed-loop system, recommendations are generated based on user behavior within the platform, while in the open-loop system, the user provides specific search inputs or prompts for recommendations.

This mechanism often leads to algorithmic amplification, where the user is repeatedly presented with the same content at the expense of other content. The algorithm strives to optimize recommendations to their highest quality. However, once this optimal point is surpassed, the algorithm enters a "Filter Bubble." In this stage, the user is confined to seeing content that aligns with their past preferences, limiting the discovery of new content. Feedback loops further perpetuate this process, as the algorithm learns from the user's choices, which, in turn, are influenced by the algorithm's recommendations.

For instance, if a user frequently watches action movies from the year 2000, the algorithm will predominantly suggest more action movies from that specific year. This can result in a cycle where the user continues to watch similar movies or eventually leaves the platform. Personally, I often choose the latter option. The feedback loops reinforce the effects of the filter bubbles, encouraging users to remain within their limited content bubble, causing the bubbles to shrink.

Way forward

Although Netflix revised its algorithm in 2008 and 2014, relying solely on past viewing history or user information for recommendations may not be enough to keep growing in this industry. The next technology in this space should be able to compete with human creativity and complex thinking. Predicting a user's specific content preferences on a given day poses challenges that cannot be easily overcome by algorithmic recommendations alone.

Marcus du Sautoy - a British mathematician (whose studies focus on algorithms and AI) also asked in one of his books ‘The Creativity Code’ whether “AI can complete creative tasks better than humans.”