Recorded September 3, 2026. Quotes are lightly edited for clarity. Maybe this sounds familiar. You run infrastructure at a company that isn’t American. Your workloads are on AWS, Azure, or Google Cloud, probably more than one, because that is what everyone picked. Until recently nobody asked you where the data lives or who can reach it. Now you’re getting questions. Legal wants to know what NIS2 means for where your systems run. Someone on the leadership team read that the US government locked the cloud accounts of judges at the International Criminal Court and wants to know if that could…
kubernetesawsexcerpt only · body stays at the source
With Discovered Stacks, Pulumi Cloud does the bookkeeping for a CloudFormation migration: every resource in the stack gets an explicit migration status, and the migration is done when the code provably matches the cloud. In this tutorial, we take one real CloudFormation stack from discovered to migrated and managed by Pulumi IaC, end to end. What we’re migrating Our example is payments-api, a CloudFormation stack with 61 resources: a VPC, an Aurora ledger database behind an RDS Proxy, an assets S3 bucket, a DynamoDB ledger table, a Kinesis payment-events pipeline, and the IAM roles, KMS keys,…
You usually find out in the postmortem: the alarm fired, but it paged a schedule nobody was on anymore. Or the service had been running in production for three months before anyone created the matching PagerDuty service, so the first person to notice the outage was a customer. The infrastructure was code, reviewed and versioned. The incident response setup was forty clicks in a web UI, done once, by someone who has since changed teams. PagerDuty’s own engineering team has been making the case for managing PagerDuty as code for years, and the OneUptime folks recently published a hands-on guide…
pagerdutyawsexcerpt only · body stays at the source
Over the past few years, we’ve been on a journey to modernise how we run Amazon Elastic Compute Cloud (EC2) instances at Slack. In our first post, Advancing Our Chef Infrastructure, we shared how we moved from a single Chef stack to a resilient, multi-stack setup with versioned cookbook deployments and safer promotion workflows. This…
By Alvin Bao, Alex Petrov, Jennifer Lai, Aidan Sherr, and Samartha ChandrashekarAs a part of the journey to transition Netflix’s compute infrastructure to be more Kubernetes-native, we have leaned into incorporating components from the Kubernetes ecosystem into our container platform Titus. One example of this is our use of Kueue, a cloud-native job queueing system for batch workloads, which has largely replaced the custom queuing and scheduling logic in our homegrown managed batch solution Compute Managed Batch (CMB). In this post, we’ll give an overview of what motivated the migration, how…
kueuenetflixexcerpt only · body stays at the source
Photo by Corinne Kutz on UnsplashBefore we knew betterOur orchestration system started as a simple internal solution to manage event pipelines and trigger downstream jobs. Over time, as more workflows and dependencies were added, it gradually evolved into a tightly coupled monolithic scheduler that became increasingly difficult to understand and maintain.Understanding how a workflow executed often meant looking through multiple files, configurations and database tables.For newer team members, onboarding into the system took time because much of the workflow context was distributed across…
In early 2023, Slack faced a foundational challenge: serving Large Language Models (LLMs) at enterprise scale with the security, reliability, and performance our customers expect. Over three years, we evolved from basic infrastructure to orchestrating a sophisticated multi-cloud architecture. We didn’t just want shiny new models; we needed a system resilient to regional outages and…
Excerpt By 2024, Slack’s data platform had accumulated 700+ SSH-based operators orchestrating critical data pipelines. We’re talking daily search indexing that processed terabytes of data, analytics jobs powering business intelligence, the whole shebang. Every single one of these jobs required direct SSH access to production AWS Elastic MapReduce (EMR) clusters. We had a massive security…
Text classification is often done through fine-tuning of a pretrained foundation model with domain-specific data. In FreeAgent we use transformer based models to automatically classify incoming bank transactions. Specifically we use a DistilBERT model that is fine-tuned on hundreds of millions of bank transactions with customer-labelled accounting categories. The model inputs are currently text-based, built from a combination of bank transaction descriptions and amounts. In this post we describe an approach to fine-tuning the DistilBERT model and training the classifier including the…
data---mlaiexcerpt only · body stays at the source
Written by Stefan ZierFor years, Lyft’s localization infrastructure relied exclusively on human translation. While this model usually ensured excellent quality, it was bound by multi-day turnarounds and costs that scaled linearly with every new language. For the few languages Lyft initially supported (Spanish, Portuguese, and French), these limits were acceptable.However, Lyft’s expansion goals quickly outpaced what traditional workflows could support. Lyft’s recent Québec launch required compliance with Bill 96 (legislation mandating French-first user experiences) which demanded faster…
Learn how to debug silent failures in AWS API Gateway HTTP when your OIDC provider doesn't implement the .well-known/openid-configuration endpoint. Enable FailOnWarnings to catch these issues before they break your production deployment.
awsserverlessexcerpt only · body stays at the source
Supporting developers to debug and resolve issues with datastores in the Self-Service ecosystem.Welcome to the third blog post of our Self-Service Datastore series, where we share our journey towards creating a more efficient and reliable way to manage datastores at Zendesk.Previous blog posts:Unlocking Efficiency: A New Era for Datastore ProvisioningSimplifying Datastore Provisioning with Kubernetes OperatorsWe need reliable, fast, and compliant self-serve methods to provision datastores. Furthermore, we need to ways to access those datastores from applications; otherwise, they won’t serve…
In our previous posts, we explored the building blocks of AWS API Gateway, including its integration with Lambda Authorizers, Usage Plans, and CORS configurations. These discussions laid a foundation for understanding the critical components of designing secure, scalable, and efficient APIs within AWS. Now, we are bringing everything together by introducing OpenAPI YAML — a powerful way to define APIs declaratively and integrate them with AWS API Gateway. This final article in our series focuses on the practicalities of using OpenAPI specifications to manage your APIs more effectively,…
open-apiawsexcerpt only · body stays at the source
Discover the journey behind FullStack Bulletin, a weekly newsletter for full-stack developers with 404 curated issues over 8 years. Learn about its origins, technical implementation, and future plans.
IntroductionWelcome to the second blog post of our Self-Service Datastore series, where we share our journey towards creating a more efficient and reliable way to manage datastores at Zendesk. In today’s dynamic application development landscape, the ability to swiftly provision datastores is crucial for maintaining agility and delivering exceptional user experiences.Provisioning encompasses all steps involved in requesting a datastore: configuring it to meet company standards, ensuring security and compliance, and managing access credentials. In this article, we’ll explore how we have…
In this blog post, I’ll walk through how using LocalStack with Docker and Docker Compose helped us speed up development and testing, making the entire process more efficient and seamless. When building modern cloud-native applications on AWS, having a reliable and cost-effective approach to development and testing is essential. For this project, we utilised LocalStack, a platform that simulates AWS services locally. This enabled us to work more efficiently, speed up development, cut down on costs, and quickly test integrations without constantly interacting with the actual AWS cloud.…
Welcome back! In this fourth installment of the blog series, we’re shifting our focus to a crucial aspect of API management: configuring Cross-Origin Resource Sharing (CORS) in AWS API Gateway. As your applications grow and interact with different domains, handling cross-origin requests effectively becomes essential. In this post, we’ll delve into the importance of CORS, walk through the steps to set it up in AWS API Gateway, and share best practices to ensure your APIs are accessible and secure. By the end of this article, you’ll have a solid understanding of how to manage CORS…
In the evolving landscape of digital communication, email spoofing poses a significant threat to the integrity of online identities. As a hobby project, and to practice AWS and software design, I have been developing an application called MovieBooker. This application streamlines the process of booking movie tickets and managing movie programs for both moviegoers and staff. Email Integration with AWS SES As part of the infrastructure for the MovieBooker application, I implemented Amazon Simple Email Service (SES) to send emails when a user pays for a ticket. However, I encountered a challenge…
Welcome again! As the third part of this blog series, we will explore AWS API Gateway usage plans! As businesses rely more on APIs to power their applications and services, efficient management and control become crucial. With this comprehensive guide, you will be able to confidently manage your APIs and ensure optimal performance for your business needs. In this post, I will cover the essentials of AWS API Gateway usage plans, from core concepts to advanced configurations and best practices. By the end, you will confidently be able to implement the API according to your specific business…
Bazaarvoice notification system stands as a testament to cutting-edge technology, designed to seamlessly dispatch transactional email messages (post-interaction email or PIE) on behalf of our clients. The heartbeat of our system lies in the constant influx of new content, driven by active content solicitations. Equipped with an array of tools, including email message styling, default […]
The AWS Solutions Architect Professional certification is one of the toughest IT certifications. This post shares preparation tips, exam strategies, study resources, and sample questions to help you succeed.
This post explains how to conditionally create resources in AWS CDK using CfnCondition. It provides a practical example of creating an S3 bucket based on an SSM parameter value. The post covers defining a condition, attaching it to a low-level CDK construct, and importing the conditionally created resource.
The AWS Solutions Architect Associate exam covers a wide range of AWS services. This post shares helpful notes and tips for studying key concepts like EC2, S3, VPC, DynamoDB, and more. It provides advice on the exam mindset and lists official and unofficial preparation resources. The notes summarize important details around provisioned throughput, instance types, database replication and more that are helpful to know for the exam.
For the past 1 month, I had a chance to work with AWS Lambda. During the period of work with Lambda, I collected a lot of thoughts about this technology and would like to share them with you. Getting started So if you don’t know anything about AWS, I recommend starting with official docs: Amazon has a very rich documentation which will explain all the details about Lambda. If you don’t want to read the whole doc, then Lambda is a technology which allows you to deploy your code in a so-called Lambda functions - a containers somewhere inside AWS infrastructure. This gives a lot of benefits: you…
The AWS CLI s3 cp command supports streaming content to and from S3 using stdin/stdout with the - argument. This enables powerful pipelines without intermediary files.
I have heard about AWS Lambda and all the cool things happening in the serverless world. I have also deployed Go functions using the Apex framework for serverless deployment. But recently I have started working on some Python projects again and decided to see how well the Python community is adapting to the serverless era. […] The post Deploying A Flask based REST API to AWS Lambda (Serverless) using Zappa appeared first on Abu Ashraf Masnun.
Optional Google Analytics helps us understand visits. Microsoft Clarity records masked interactions to improve the site. Optional tools stay off unless you choose them. Privacy details.