Optimizing FastAPI at Scale: Lessons from OLX’s Recommendation PlatformPhoto by Rosy KoIn distributed systems, there is a motto that says ‘you are as slow as your slowest tasks’. In Python, thanks to the notorious Global Interpreter Lock (GIL), this issue is amplified: ‘your slowest task will make every other task slower’. In this article, I’ll walk you through the optimizations we made to scale a FastAPI service that now handles tens of thousands of requests per second, achieving a p99 latency under 10ms.IntroductionOLX is a global online marketplace that enables users to buy and sell goods…
OverviewThe Rule Executor Service at Myntra stands out as a pivotal tool in the domain of rule engine. While, at a glance, one might wonder what sets it apart from the myriad of rule engines available today, the difference is profound. In Myntra ecosystem, supporting numerous pricing use cases necessitated building a capability that is engineered to handle vast volumes of rules, processing millions of facts against similar scale in terms of rules for multiple tenants with high reliability, performance, availability and consistency. In order for the system to be performant and use as a broker…
Bazaarvoice notification system stands as a testament to cutting-edge technology, designed to seamlessly dispatch transactional email messages (post-interaction email or PIE) on behalf of our clients. The heartbeat of our system lies in the constant influx of new content, driven by active content solicitations. Equipped with an array of tools, including email message styling, default […]
Strava challenges offer a fun way for athletes to compete against themselves and others! Back in 2020, our legacy challenge leaderboard system was running into bottlenecks and scalability problems on a regular basis, and we often found ourselves putting out fires to keep the system stable. In late 2020 and early 2021, I worked on a project to replace the old leaderboard system with a new one that could handle a much larger number of athletes competing in challenges. This blog post is about that project. I drafted most of this post when the project wrapped up in 2021, but didn’t get it…
This post highlights 6 important rules to keep in mind when developing performant web applications: avoid premature optimization, do the minimum required work, defer non-critical tasks, leverage caching, avoid N+1 queries, and design for horizontal scaling. Following these guidelines will help you write efficient code from the start and build apps ready to handle growth.
Optional Google Analytics helps us understand visits. Microsoft Clarity records masked interactions to improve the site. Optional tools stay off unless you choose them. Privacy details.