TLDR: A study on a major Norwegian news organization, Aftenposten, found that “controlled personalization”—combining editorial curation with modest algorithmic recommendations—significantly increased user engagement (higher click-through rates, reduced scrolling) and supported journalistic values. It led to greater content diversity, broader article coverage, and reduced popularity bias, demonstrating a viable path for traditional media to adopt personalization without compromising editorial principles.
In the evolving landscape of online news, striking a balance between engaging readers with personalized content and upholding core editorial values is a significant challenge, especially for traditional news organizations. A recent study delves into this very dilemma, exploring a strategy called “controlled personalization” within legacy media online services.
The research, titled “Controlled Personalization in Legacy Media Online Services: A Case Study in News Recommendation,” was conducted by Marlene Holzleitner, Stephan Leitner, Hanna Lind Jorgensen, Christoph Schmitz, Jacob Welander, and Dietmar Jannach. It highlights how traditional news outlets often combine editorially curated content with algorithmically selected articles, a cautious approach to personalization that respects journalistic principles.
Unlike large news aggregators that prioritize automated content selection for maximum user engagement, legacy media like Aftenposten, a major Norwegian news provider, operate with a different set of values. These include promoting diverse viewpoints, fostering democratic engagement, and maintaining public trust. These values can sometimes conflict with purely algorithmic optimization for clicks or retention.
To evaluate the effectiveness of controlled personalization, an A/B test was conducted on Aftenposten’s mobile website over 34 days, involving approximately 58,000 paying subscribers. The test compared the existing non-personalized ranking system with a personalized variant where 20% of the ranking score was influenced by a personalized recommendation algorithm. This modest level of personalization was a deliberate choice to ensure editorial control remained paramount.
Impact on User Engagement
The findings from the A/B test revealed several substantial benefits. Users exposed to personalized content demonstrated a higher click-through rate (CTR), increasing by over 14%. This suggests that personalized recommendations made it easier for users to discover relevant content. Interestingly, the Canceled Click Rate (CCR), which indicates if users find articles irrelevant after clicking, remained constant, ruling out the possibility of “clickbait” recommendations.
Furthermore, personalized content led to reduced navigation effort. Users in the personalization group scrolled down less in the news feed (fewer impressions per user) but still clicked on a slightly higher number of articles (increased clicks per user). This indicates that personalization helped users find what they were looking for more efficiently.
Regarding reading behavior, while the average reading percentage of an article remained similar between groups, the average activity duration per click slightly increased in the personalized group. This suggests that users spent a little more time actively inspecting articles they clicked on, implying sustained interest.
A subgroup analysis showed that these positive engagement effects, including higher CTR and reduced impressions, were consistent across users with low, medium, and high activity levels.
Upholding Journalistic Values
Beyond engagement, the study also examined how controlled personalization influenced journalistic values, focusing on content diversity, coverage, and popularity bias.
The results indicated that personalization contributed to greater content diversity. The distribution of both viewed and clicked articles across different sections (categories) became more even in the personalized group. This was supported by a lower Gini Index and a statistically significant difference in section distributions, suggesting that users were exposed to and consumed a wider variety of topics.
In terms of article coverage, the daily click coverage was significantly higher in the personalized group. This means that readers with personalized recommendations explored a broader range of unique articles each day, ensuring that a larger fraction of the available content pool was engaged with.
Crucially, controlled personalization also helped reduce popularity bias. The personalized recommendations, on average, included less popular items, and readers were more likely to click on these less popular articles. This is a significant finding, as popularity bias is often an undesirable characteristic of recommender systems, potentially leading to “filter bubbles” where users are only shown content similar to what is already popular. The study found that personalization helped address this, promoting a more varied consumption pattern.
Also Read:
- Spotify Revolutionizes User Acquisition with Advanced Machine Learning and Automated Content Marketing
- AI’s Role in Peer Review: A Framework for Systematic Evaluation
Implications and Challenges
The study concludes that even a modest level of controlled personalization can yield substantial benefits, successfully aligning user needs with editorial goals. It offers a viable path for legacy media to adopt personalization technologies while upholding journalistic values. The authors emphasize the importance of multi-dimensional analyses, moving beyond just click-through rates, to fully understand the impact of personalization.
However, implementing such systems comes with organizational challenges. These include the need for continuous adaptation due to the dynamic nature of news, potential for algorithmic bias, and the crucial role of editors in maintaining control. Skepticism among journalists regarding algorithmic recommendations also necessitates careful communication and collaboration between editorial, product, data, and curation teams to build trust and ensure alignment with journalistic missions.
For more in-depth details, you can read the full research paper available here.
