Spark
We track 16 posts about Spark from 9 engineering blogs. Most active: Nvidia, Vercel, OpenTable. Latest post: Oct 15, 2026.
Companies writing about Spark
Recent posts
Be There for the Beginning: Celebrate the Launch of NVIDIA RTX Spark (opens on the source site)
Nvidia ·
A new beginning for Windows PCs starts tomorrow with the launch of NVIDIA RTX Spark — one PC built for creators, gamers and AI agents. All are invited to celebrate with NVIDIA and partners at launch events around the world. Share Your Spark Every PC has a story. What was your first Windows PC, and […]
Muse Spark Open now available on AI Gateway (opens on the source site)
Vercel ·
Muse Spark Open from Meta is now available on AI Gateway. The open-weight reasoning model is designed for coding and agentic workflows, including multi-file refactoring, debugging, and understanding large codebases. Muse Spark Open supports text and image input with a 1M-token context window, allowing agents to work with code and screenshots in the same request. It is available with ZDR and no training, without creating separate provider accounts. Use meta/muse-spark-open as the model name across all API formats and coding agents. The model is available across the AI SDK, the…
Running Apache Spark experiments in my sleep (and on a plane) (opens on the source site)
Netflix ·
By Prashanth Gedde NarayanaswamyAs one of Netflix’s largest and critical data pipelines, member-sessionizer consolidates tens of billions of hourly client events, encompassing user taps, playbacks, and engagement signals, into individual viewing sessions per profile.A quick two-week tuning effort spiraled into two months when standard approaches hit a wall. Because the failures only happened at full production scale, each test took hours, limiting me to just three or four attempts daily. The turning point was changing the goal: instead of trying to speed up a single run, I built a way to run…
All Systems Go: Global PC App Ecosystem Ready for NVIDIA RTX Spark (opens on the source site)
Nvidia ·
NVIDIA RTX Spark is a new beginning for the PC — a reinvention combining the capabilities of a gaming PC, a creative studio and a personal AI supercomputer into a single superchip. With up to 128GB of unified memory and 1 petaflop of AI compute, RTX Spark delivers the full stack of NVIDIA RTX and […]
NVIDIA, Microsoft Kick Off a New Beginning for Windows PCs With RTX Spark and AI Agents (opens on the source site)
Nvidia ·
At a Microsoft event in San Francisco on Wednesday, Jensen Huang and Satya Nadella outlined how NVIDIA and Microsoft are co-engineering hardware and software for AI agents to run on Windows PCs. NVIDIA was founded because of Windows, Huang said. Now AI agents are coming to Windows. “If you look at the entire journey of […]
NVIDIA DGX Spark 64GB Gives Developers More Ways to Build and Scale Local AI (opens on the source site)
Nvidia ·
Local AI is becoming more useful by the token. As AI agents move from experiments into everyday development, increasingly capable open models are shrinking to fit on more devices, giving builders more to run locally. Coming this month, NVIDIA DGX Spark will be available with 64GB of unified memory from top manufacturer partners — Acer, […]
Partition Finalization in Pinterest’s Next-Generation DB Ingestion Framework (opens on the source site)
Qianrui Zhang | Sr Software Engineer, Logging PlatformKanchi Masalia | Software Engineer II, Stream Processing PlatformLiang Mou | Sr Staff Software Engineer, Logging PlatformYi Pan | Principal Engineer, Agent PlatformIntroductionThis is the third post in our series on Pinterest’s next-generation database ingestion framework. Part 1 introduced the DB ingestion framework built on Kafka, Flink, Spark, and Iceberg, and Part 2 covered automated schema evolution. This post tackles another challenge in migrating downstream customers to the new ingestion framework: knowing when data is complete…
Muse Spark 1.3 now available on AI Gateway (opens on the source site)
Vercel ·
Muse Spark 1.3 from Meta is now available on AI Gateway, in both the standard and contributor pricing tiers. This model improves on prior Muse Spark models at agent work and coding, with a 1M token context window and text, image, and PDF input. On coding it takes fewer turns and writes less filler than the previous release. To use Muse Spark 1.3, set model to meta/muse-spark-1.3: Copy link to headingContributor tier Muse Spark 1.3 Contributor is a pricing tier on the same model rather than a separate one, with the same weights, capabilities, and context window. The difference is that Meta…
Leading Publishers Bring Blockbuster PC Games and Technology to NVIDIA RTX Spark (opens on the source site)
Nvidia ·
NVIDIA is bringing the next wave of RTX gaming to the Gamescom conference running this week in Cologne, Germany, with support for new games, anti-cheat technologies and increased visual quality. Electronic Arts, Embark and Ubisoft are among the latest game publishers and developers bringing their blockbuster titles to NVIDIA RTX Spark ahead of its launch […]
Scaling Grab's Data Lake: Our journey to Apache Iceberg adoption (opens on the source site)
Grab ·
Introduction: The evolution of Grab’s Data Lake At Grab’s scale, managing petabytes of data across billions of S3 objects demands more than a storage layer. It demands a robust architectural primitive that supports the high-concurrency needs of a modern “Lakehouse.” Our goal is full storage-compute separation, leveraging S3 as an elastic foundation for both near-real-time metrics and large-scale batch transformations. For years, the vast majority of our tables were Hive Parquet, managed through the Hive Metastore with a directory-based layout. This model served us well, but as data volume…
Using LLMs to Analyze Spark SQL Plans: A Practical Approach to Debugging Long-Running Jobs (opens on the source site)
Expedia ·
Expedia Group Technology — InnovationUsing large language models to reveal bottlenecks in Spark SQL execution plansPhoto by Luis del RíoIf you’ve ever stared at a 300-plus-node physical plan at 2 a.m. trying to spot a missing broadcast or one cursed skewed partition, this is for you.Spark makes it deceptively easy to write complex SQL that looks correct but quietly turns into a performance and cost problem at scale. A query that runs fine on day one can slow to a crawl as data grows, joins get wider, and aggregations become more nested. Suddenly, jobs take hours instead of minutes, clusters…
Automated Schema Evolution in Pinterest’s Next-Generation DB Ingestion Framework (opens on the source site)
Yisheng Zhou | Software Engineer IILiang Mou | Sr Staff Software EngineerGabriel Raphael Garcia Montoya | Staff Software EngineerIstvan Podor | Staff Software EngineerIntroductionIn the first post of this series, we introduced Pinterest’s next-generation CDC-based ingestion platform built on Kafka, Flink, Spark, and Iceberg. In production, upstream schemas are constantly evolving, and in a distributed CDC pipeline, schema is not just metadata — it is a cross-system contract spanning ingestion, transformation, storage, and historical backfill. A schema change that is not handled carefully can…
Rain: A key-value store for Strava’s scale (opens on the source site)
Strava ·
Much of our heatmaps are built on batch data outputs stored in RainAt Strava, we love maps — some of our most loved features are nestled on map surfaces. My team, the Geo team, is focused on building and improving these products. On the Geo and Metro teams, we tend to work with large datasets: aggregations of open source map data via OpenStreetMaps, GPS data points from uploaded activities, third-party datasets for properties like elevation, and beyond. This aggregated dataset eventually turns into Geo features we know and love, like the global heatmap, Strava Metro, the routing product,…
Leveraging Spark 3 and NVIDIA’s GPUs to Reduce Cloud Cost by up to 70% for Big Data Pipelines (opens on the source site)
Paypal ·
By Ilay Chen and Tomer AkiravAt PayPal, hundreds of thousands of Apache Spark jobs run on an hourly basis, processing petabytes of data and requiring a high volume of resources. To handle the growth of machine learning solutions, PayPal requires scalable environments, cost awareness and constant innovation. This blog explains how Apache Spark 3 and GPUs can help enterprises potentially reduce Apache Spark’s jobs cloud costs by up to 70% for big data processing and AI applications.Our journey will begin with a brief introduction of Spark RAPIDS — Apache Spark’s accelerator that leverages GPUs…
Mesos at Opentable (opens on the source site)
How we got started with Mesos: Pain-points in a hybrid environment and surviving the transition The post Mesos at Opentable appeared first on OpenTable Technology Blog.
Using data science to create a dining expert (opens on the source site)
We can build expert knowledge of cities with our corpus of unstructured reviews The post Using data science to create a dining expert appeared first on OpenTable Technology Blog.
Related topics
This page is generated automatically from the engineering blogs we follow. Every post links to its source, where it was published. See all sources.