A few people asked why I built the football scraper against ESPN specifically instead of a stats API or one of the sites that actually sell football data. Short answer: I wanted real data, for free, without waiting on anyone's approval.
The API route didn't work for a side project
Most of the "real" football data providers are built for people already running a business around it. Pricing is quote-based, there's a sales call before you get an API key, and most of them assume you want play-by-play data for a betting product, not box scores for a personal project. That's a lot of friction for something I was building to learn from and put in a portfolio.
ESPN is consistent and it's everywhere
ESPN's site structure barely changes year over year, which matters a lot for a scraper. Their scoreboard, box score, and schedule pages follow the same layout across leagues, so once I had the scraping logic working for one league, extending it to others was mostly reusing the same selectors. It's also just a site people recognize, which makes the project easier to explain.
What the scraper actually does
It's a fairly standard scrape-and-normalize pipeline:
1. Fetch schedule/scoreboard pages
2. Parse box scores per game
3. Normalize team names and stats into a shared schema
4. Write to JSON / a local storeNothing exotic. The interesting part was handling the small inconsistencies: bye weeks, postponed games, and stat tables that don't render the same way for every game type.
The catch
Scraping a site instead of using an official API means it can break whenever ESPN changes their markup, and I'm relying on their public pages rather than a supported data feed. That trade-off was fine here. If I were building something people depended on daily, I'd look at this differently.