7 items
A podcast about machine learning and data science hosted by Katie Malone and Ben Jaffe. Approachable explanations of ML concepts without dumbing them down.
Back with new episodes on Substack. Also available on Spotify.
Explains model distillation as both a legitimate technique for building smaller, cheaper models and a contested way to clone a rival lab's model via its API, plus why proving distillation happened is forensically hard. Covers the 2015 Hinton/Vinyals/Dean paper's soft-target vs. hard-label distinction.
Applies Jevons Paradox to AI inference — per-token costs dropped 1000x since 2022 but total spend is up 10x because usage exploded. The key insight is that agent costs grow quadratically with turns, not linearly, because each step feeds the full conversation history back as context.
Temperament matters more than talent in AI research — a meditation on the daily practice of reading and building, and why equanimity is the real prerequisite.
The Zen framing isn't just a metaphor — he quotes Suzuki directly, structures the piece like numbered koans, and the equanimity point is genuinely it: sit with failure the same way you sit with success, neither attached to the outcome.
Why scaling up multi-agent AI systems doesn't deliver proportional benefits — collaboration turns out to be a distinct capability, and adding agents to sequential tasks often makes things worse.
Strong on coordination cost — agents communicating and handing off work isn't free, and that overhead often swamps any gains from parallelism.