1
Building Reproducible AI Evaluation Workflows with Docker Sandboxes (opens on the source site)
Learn how Docker Sandboxes can make AI evaluation workflows more reproducible with consistent execution, structured artifacts, and runtime evidence.
#genai
6 posts · 6 companies · newest first
From the web
2
GenRec: Towards LLM-Native Recommendation at Netflix (opens on the source site)
Authors: Ying Li, Arjun Rao, Shradha SehgalIntroductionRecommendations sit at the heart of the Netflix experience. Our current production models rely on thousands of hand‑crafted features over users, items, and interactions, along with specialized architectures for sequence modeling, feature interactions, and multi‑task objectives. This stack has evolved over many years to support diverse content types (movies, series, games, live, podcasts) and product surfaces, but its complexity makes it costly to onboard new use cases: adding a content type or surface can require significant feature…
3
Eval-driven development: Lessons from evaluating GenAI at scale (opens on the source site)
How Airbnb teams build trustworthy Generative AI products by treating evaluation as a first-class engineering discipline; not an afterthought.Nestled into the lush hillside, this stunning modern retreat features striking natural wood architecture, terraced balconies, and a serene landscape.By: Rohit Girme, Dan Miller, Mia Zhao, Lifan Yang, Clint KellyIntroductionGenerative AI breaks a lot of the assumptions that used to hold true for software testing. Unlike traditional software, LLM outputs are non-deterministic, and “correct” is subjective. Because so much judgment is involved, you often…
4
Evaluating AI at Scale: How Thumbtack Approaches Reliability, Safety, and Quality in GenAI (opens on the source site)
A practical look at how Thumbtack navigates evaluation for emerging AI experiences and what we’ve learned along the way.By: Shishir Dash, Director of Applied Science & Teja Venkat Kolli, Senior Applied ScientistEvaluating AI at ScaleIntroductionAI is reshaping how people interact with products, and Thumbtack is no exception. We’re introducing AI into more aspects of our customer and local service professional (pro) experiences — from helping customers articulate what they need, to generating helpful summaries, to offering clearer explanations of how pros may fit those needs.But evaluating…
5
Structured outputs with Pydantic AI (opens on the source site)
One of the challenges of working with LLMs is getting them to respond with a consistent format, such as a given JSON schema. Anyone who has tried to solve this issue with prompt engineering knows how frustrating it can be. You add a ‘MUST’ here and an ‘always return JSON’ there, but still the output […]
6
The GANfather: Using Malicious GenAI Agents to Combat Money Laundering (opens on the source site)
Digital systems have become deeply integrated into many aspects of modern life, particularly within the financial sector. While digital banking simplifies day-to-day operations for clients, it also creates new opportunities for malicious actors to exploit these systems. As a result, money laundering has grown particularly prevalent due to this digital expansion.Banks are required to monitor for money laundering activities and issue alerts when suspicious transactions are detected. Typically, monitoring is performed by rules-based legacy systems. A better approach would be to use Machine…
6 shown