The Voice Memo Graveyard: Building a Second Brain for Musicians with Python

Keep Audio XYZ is a web platform that helps musicians turn fragmented voice memos into structured song ideas. Rather than generating music, the system focuses on organizing and connecting what musicians already have.

The talk follows a musical idea through the full system lifecycle:

1. Audio upload and local Whisper transcription
2. User-driven metadata tagging and domain modeling (Idea → SongSeed → Song)
3. Deterministic matching engine and song-seed generation

Along the way, several engineering tradeoffs are examined:

• Why a single FastAPI + SQLite monolith running on an Oracle Cloud VM was chosen deliberately over distributed infrastructure during the MVP stage.
• Why explainable deterministic similarity beats embeddings when your domain has explicit structure.
• How deterministic scoring with explicit Python logic works by modeling musical constraints, providing more predictable behavior.
• What breaks when CPU-bound ML inference runs inside async web frameworks. We will cover purely Pythonic deployment gotchas: how async def inadvertently blocks uvicorn workers during ML inference (and how to fix it), plus navigating Python 3.8 vs 3.12+ type-hinting mismatches when moving from local development to cloud VMs.

The talk is based on real implementation tradeoffs discovered while building and deploying the system as a solo developer. Attendees will walk away with explicit takeaways and patterns they can reuse in their own small Python monoliths.