1
Agentic QA for data: Bots go beyond the rules (opens on the source site)
Traditional quality assurance catches only the failures you anticipate. An agentic approach finds unexpected gaps and helps spot missing data.
#scraping-practice
6 posts · 1 company · newest first
From the web
2
The trouble with variants: Solving the web's infinite ecommerce explosion (opens on the source site)
Product variants can turn a straightforward ecommerce scraping project into millions of combinations. Learn how to classify, scope and control them.
3
How I use Zyte MCP to plan web scrapers (opens on the source site)
Zyte MCP lets my AI agent investigate a site before I write the scraper, so I can decide where the data should come from, compare an API with the rendered page for missing fields, and end up with a crawl plan I can act on.
4
Stopping a crawl when the data stops looking real with Jev (opens on the source site)
A Scrapy pipeline that asks a fast, calibrated AI model whether each scraped field still looks real, and stops the crawl when too many don't. What it caught, what it misses, and whether building it was worth it.
5
Podcast Ep08 - Scrapy, Python and mushroom soup (opens on the source site)
Scrapy's core handles crawling well and deliberately leaves almost everything else out: no bundled browser, no opinion about how you shape your data, no built-in answer for every anti-bot wrinkle. What it gives you instead is a clean way to add those things at the edges, exactly when you need them and never before.
6
The missing middle ground in scrapy-playwright just got filled (opens on the source site)
You can now choose which Python browser library you want to use with scrapy-playwright. I go through why this is a huge deal for the right scraping demographic.
scraping-practiceexcerpt only · body stays at the source
6 shown