Hacker Public Radio

Your ideas, projects, opinions - podcasted.

New episodes Monday through Friday.

HPR3656: Importance of Small toy projects

Hosted by norrist on 2022-08-08 00:00:00
Download or Listen

Importance of Small toy projects

Inspired by

My recommendation for learning a new language.

  1. Think about a small problem you want to solve with code.
  2. Start banging away at it.
  3. Redo over and over, its OK
  4. Same project but better
  5. Learn new tech and practice

An example of one of my small projects

  • Original intent of my podfaded project
    • Mostly Tech/Linux podcasts
    • Find every podcast I could
    • Track release cadence
    • Rate how podfaded the feed is
    • Use search API to find RSS feeds

First attempt plan

  • Scrape podcast networks
    • used beautiful soup - something I wanted to learn anyway
    • HTML
      • Jinja templating - something I wanted to learn anyway
      • Bootstrap - something I wanted to learn anyway
      • Different colors based on how podfaded

First attempt problems

  • individual script per network
  • 4-5 different scripts
  • took forever to run
  • Broke when networks changed their page layout


  • Test Driven Development and pytest
    • Red Green Refactor
    • Confidence to change code
  • Simplify - one scrapper script
  • Embed audio player for latest episode

How it works - Scraping pages for feeds

  • scrape_for_feeds.py
  • Provide a list of websites that have lists of podcasts
  • Search the page for any links
  • Check if the feed is valid by trying to parse it with feedparser
  • Add feed to database

Podcasts pages I scrape

How it works - Checking the feeds

  • feed_info.py
  • loop through feeds
  • use feedparser to find latest episode
    • title and enclosure (audio file)

HTML Output

Feedback Welcome

  • https://gitlab.com/norrist/podfaded2
  • Bugs
    • Sometimes the title is missing, so some of the CSS coloring doesn't work
    • Excluded podcasts still occasionally show up on the list
  • Looking for more curated lists of podcast pages
    • not search sites
  • Maybe scrape HPR podcast recommendation episodes



More Information...

Copyright Information

Unless otherwise stated, our shows are released under a Creative Commons Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0) license.

The HPR Website Design is released to the Public Domain.