Authors: Longyu Zhao (Staff Machine Learning Engineer), Gwendolyn Zhao (Staff Machine Learning Engineer), Peng Yan (Senior Machine Learning Engineer), Yuanlu Bai (Senior Machine Learning Engineer), Yuan Wang (Senior Machine Learning Engineer), Yao Cheng (Staff Machine Learning Engineer), Ang Xu (Principal Machine Learning Engineer), Zhaohong Han (Manager II, Ads Lightweight Ranking)IntroductionPreviously¹, we launched the next-generation serving stack for standard ads, which we call Nexus. Nexus decoupled candidate generation from scoring and moved us beyond the classic two-tower-only world,…
Why we think LLMs can be useful and why we will not replace all of our models with themContextAt Medium, we have many Machine Learning models that we use to label stories automatically. These affect what stories we recommend to readers.Here’s some examples:a few of our text classification models. All diagrams and charts made by the authorSome Clarifications on our Machine Learning policyBefore we go deep on this project, I just wanted to clarify a few things about how we stand regarding AI in general.Medium has been training internal models with user and post data for a long time now. We…
Expedia Group Technology — DataWhat happened when we treated a framework migration as an architecture modernisation — and cut P99 inference latency by two-thirdsSt Paul’s and millennium bridge, LondonExpedia Group™ has always been a market leader in providing personalised search experiences for travellers. As our ranking models evolved, we saw an opportunity not just to migrate to Keras 3, but to modernise the broader stack around it so we can better serve travellers. This led us to rewrite key parts of our pipelines that made model training 30% faster and cut P99 inference latency by…
Authors: Ying Li, Arjun Rao, Shradha SehgalIntroductionRecommendations sit at the heart of the Netflix experience. Our current production models rely on thousands of hand‑crafted features over users, items, and interactions, along with specialized architectures for sequence modeling, feature interactions, and multi‑task objectives. This stack has evolved over many years to support diverse content types (movies, series, games, live, podcasts) and product surfaces, but its complexity makes it costly to onboard new use cases: adding a content type or surface can require significant feature…
netflixgenaiexcerpt only · body stays at the source
Part 1 of 2AuthorsPersonalization (Homefeed): Yuke Yan, Chuxi Wang, Andreanne Lemay, Olafur Gudmundsson, Anna Kiyantseva, Krystal Benitez, Jongho Kim, Jiacong He, Rahul Goutam, James Li, Dylan WangUser Understanding: Simin Li, Sufyan Suliman, Yingjian Ding, Hongbo DengData Science: Armando Ordorica, Yan Chen, Ellie Zhang, Karim WahbaIntroductionPinterest’s mission is to help people discover the inspiration to create a life they love. Our recommendation system serves hundreds of millions of users, surfacing billions of Pins across interests ranging from home renovation to meal planning to…
How we built a Transformer-based sequence model that encodes years of guest behavior to surface the right listings at the right time.By: Daochen Zha, Chun How Tan, Xin Liu, Bin Xu, Han Zhao, Xiaowei Liu, Jun Shi, Tracy Yu, Hui Gao, Huiji Gao, Liwei He, Michael Kinoti, Stephanie Moyerman, and Sanjeev KatariyaIntroductionPlanning a trip on Airbnb rarely happens in a single session. A guest searching for a place to stay in San Francisco might browse dozens of listings over several days, leaving behind a trail of views. Typically, over a period of years, that same guest will have accumulated many…
Authors: Lequn Wang, Jiangwei Pan, and Linas BaltrunasFigure 1. Autoregressive homepage generation. GenPage builds a Netflix homepage one row or entity at a time, each one conditioned on what’s already on the page and the user’s context.IntroductionThe Netflix homepage is the first thing users see when they open the app and the primary way they discover content to enjoy. Almost every part of it is personalized, including which rows appear, which entities show up within those rows, and how everything is arranged on the page.Constructing that homepage is a genuinely hard problem. It is not…
Key Contributors: Karuna Ahuja, Marko Avdalovic, Soroush Sobhkhiz, Shrikar Archak, Xiyu Wang, Ji Chao Zhang, Hao YanIntroductionEvery time a user opens Instacart, they see product recommendations: on the retailer home page, in search results, and alongside their cart. Many of these recommendations are sponsored products surfaced by a retrieval model that decides which products to show from a vast ads product catalog. A relevant ad helps users discover products they didn’t know they needed; a less relevant one generates friction.Two years ago, we introduced Contextual Recommendations (CR), a…
Key Contributors: Shrikar Archak, Karuna Ahuja, Soroush Sobhkhiz, Marko Avdalovic, Xiyu Wang, JiChao Zhang, Hao Yan, Chris HartleyIntroductionOperating a grocery catalog at Instacart’s scale means managing millions of products across thousands of categories. Every product is assigned to a category in our hierarchical taxonomy like “Dairy > Cheese > Parmesan”. These categories provide broad classification, but they miss the connections that drive how customers actually shop.For example, a customer is building a cheese board. They’ve added Parmigiano Reggiano, and now they need accompaniments.…
Authors (listed alphabetically)Ads Feature Engineering Infra team: Ajay Venkatakrishnan, Le ZhangCore ML Infra team: Eric Shang, Pihui WeiML Data team: Connor Votroubek, Yi HeUser Understanding team: Camilo Munoz, Simin LiIf you work on ranking, retrieval, or recommendation systems, you’ve probably asked for some version of the same thing: “Give me the last N meaningful actions this user took, with the right enrichments, in a format that’s easy to train and serve ML models.”On paper, that sounds simple. In practice, “user sequences” often become one of the most expensive and fragile parts of…
How we made our filtering 10x cheaper by removing our Bloom FiltersBloom Filters are great tools to make fast and cheap filtering. They also come with plenty of problems and can easily get expensive and cumbersome. We switched to user-based direct database queries, which made our filtering cheaper and easy to maintain. Here’s the full breakdown of that migration.Intro: This is a 4-part series breaking down improvements to the algorithm behind the Medium’s Daily Digest over the past year. When we started this work, the Digest was suboptimal — and since it’s a huge distribution surface,…
How we made our email story recommendations betterIn this Part 1, you’ll understand how we improved one of the main ways our users are exposed to our product and how that led to a massive 7% increase on the average reading time for the digest users.Intro: This is a 4-part series breaking down improvements to the algorithm behind the Medium’s Daily Digest over the past year. When we started this work, the Digest was suboptimal — and since it’s a huge distribution surface, reaching millions of readers every day, we started working on incremental improvements.By the end of these projects, the…
Cross-Digest diversificationIn this part 4, we’ll see how we went from investigating a few complaints from digest power users to improving our digest recommendations across the board.Intro: This is a 4-part series breaking down improvements to the algorithm behind the Medium’s Daily Digest over the past year. When we started this work, the Digest was suboptimal — and since it’s a huge distribution surface, reaching millions of readers every day, we started working on incremental improvements.By the end of these projects, the digest was 10% more likely to convert users to paying members, less…
Authored By Rohit Gupta & Siddhartha DevapujulaIntroductionMillions of users visit Myntra daily to upgrade their wardrobes and millions of items are listed on the platform at any given time. Users neither have the time nor the capability to scroll through this vast list of items. Even after applying category and attribute filters, usually the number of items is still in thousands. Hence it becomes critical that the top search results for any user are both relevant and personalized. Just like search, many other recommendation widgets across the platform face the same challenges.Fashion…
Optional Google Analytics helps us understand visits. Microsoft Clarity records masked interactions to improve the site. Optional tools stay off unless you choose them. Privacy details.