7 posts · 6 companies · newest first
Cloudflare's global network is immense but not limitless. As we look for small ways to trim our resource usage, we sometimes get lucky and we can cut significantly more. Here’s how we reduced one of our Pingora-based service's RAM usage with statistics.
Five Rust-level memory optimizations to the DNS cache layout of Big Pineapple cut per-entry memory by 56%, freeing approximately 100 TB of memory across Cloudflare's fleet.
We used DSPy to improve LLM judges and optimize our chat experience, creating an evaluation-driven feedback loop that produced better outputs.
A practical loop for optimizing GPU inference without guessingIt wasn’t an incident. That made it interesting.A few days after a cloud migration, one of our highest-traffic ML services looked healthy. Traffic was flowing. Latency was fine. The rollout had gone the way you want these things to go: boringly.And yet a Slack thread started anyway.Not because anything was broken. Because something had become easier to see.On-prem, GPU efficiency tends to get folded into bigger conversations: capacity planning, hardware timelines, headroom for the next model. In the cloud, the same question…
Optimizing FastAPI at Scale: Lessons from OLX’s Recommendation PlatformPhoto by Rosy KoIn distributed systems, there is a motto that says ‘you are as slow as your slowest tasks’. In Python, thanks to the notorious Global Interpreter Lock (GIL), this issue is amplified: ‘your slowest task will make every other task slower’. In this article, I’ll walk you through the optimizations we made to scale a FastAPI service that now handles tens of thousands of requests per second, achieving a p99 latency under 10ms.IntroductionOLX is a global online marketplace that enables users to buy and sell goods…
Introduction In this article, I’m going to show you how to optimize the queries executed by a Spring application running on MySQL using Releem. For a short introduction about Releem, check out this previous article, in which I explained how to set up Releem to collect metrics from your MySQL database. Spring Petclinic application The application under test is going to be the well-known Spring Petclinic that we are going to run using the MySQL profile: After starting the application, we are going to run the petclinic_test_plan.jmx JMeter test plan that is... Read More The post MySQL Query…
Large Screen Devices - The New Frontier SoundCloud large screen optimized Recently, the Android team at SoundCloud took on a project to…