Listen: building with ai models

The queue

The shading behind each date shows whether an episode is still current. It empties after two years for agents and tools, retrieval and in production; five years for everything else.

1. 1 Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix) Lenny's Podcast 1 h 23 min The author of O'Reilly's AI Engineering lays out what actually makes AI apps better, from pre-training versus post-training to RAG, before any tooling enters the picture. A plain-language map of the primitive for a developer new to the field.
   Oct 2025 Foundations Mark as heard
2. 2 Prompting 101 | Code w/ Claude Anthropic (Code w/ Claude) 25 min Video only Two Anthropic applied-AI engineers build one prompt from scratch against a real task, calling the model in the console with nothing in between. Watch it as a demonstration of iterating on the raw primitive, one visible failure at a time.
   Jul 2025 Foundations Mark as heard
3. 3 AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt) Lenny's Podcast 1 h 38 min Built on a survey of more than 1,500 prompting papers, this separates the prompting techniques that still work from folklore. Its product-prompt and prompt-injection sections matter as soon as your prompt runs inside an app instead of a chat window.
   Jun 2025 Foundations Mark as heard
4. 4 High Agency Pydantic > VC Backed Frameworks — with Jason Liu of Instructor Latent Space 52 min Makes the case for a thin layer (typed Pydantic models over the provider SDK) in place of a heavy framework. It fits the stage of learning the call itself: structured output is the first thing you need from a model.
   Apr 2024 Foundations Mark as heard
5. 5 Episode 59: Patterns and Anti-Patterns For Building with AI Vanishing Gradients 48 min An early GitHub Copilot engineer lists the 'seven deadly sins' of LLM development and what he does instead. It sets good defaults before you add retrieval or agents.
   Sept 2025 Foundations Mark as heard
6. 6 Episode 45: Your AI application is broken. Here’s what to do about it. Vanishing Gradients 1 h 18 min The core argument of the whole path: debugging an AI app starts with reading its outputs, not with a dashboard. Hamel walks through the spreadsheet-level workflow he uses with clients.
   Feb 2025 Evals Mark as heard
7. 7 Episode 50: A Field Guide to Rapidly Improving AI Products -- With Hamel Husain Vanishing Gradients 28 min A 28-minute version of Hamel's 'Field Guide to Rapidly Improving AI Products'. It covers how error analysis tells you what to fix and when an eval is worth writing.
   Jun 2025 Evals Mark as heard
8. 8 Why AI evals are the hottest new skill for product builders | Hamel Husain & Shreya Shankar (creators of the #1 eval course) Lenny's Podcast 1 h 47 min The clearest end-to-end walkthrough of error analysis on real traces from a property-management assistant: writing notes on failures, then grouping them into a taxonomy. It is this path's stage 2, done live.
   Sept 2025 Evals Mark as heard
9. 9 AI Evaluations Clearly Explained in 50 Minutes (Real Example) | Hamel Husain Behind the Craft (Peter Yang) 53 min Video only A screen-shared session in which Hamel goes through 100 real production traces and builds the eval criteria in a spreadsheet. Video only, and worth watching because the tool is so ordinary.
   Sept 2025 Evals Mark as heard
10. 10 Episode 29: Lessons from a Year of Building with LLMs (Part 1) Vanishing Gradients 1 h 30 min Five practitioners behind the 'What We've Learned From A Year of Building with LLMs' report discuss its lessons. The recurring theme is data literacy and looking at outputs. The YouTube link is the full livestream, which also covers part 2.
    Jun 2024 Evals Mark as heard
11. 11 Episode 60: 10 Things I Hate About AI Evals with Hamel Husain Vanishing Gradients 1 h 13 min A catalogue of the mistakes teams make once they start writing evals, and the data-centric playbook that replaces them. It includes a method for building LLM judges you can trust.
    Sept 2025 Evals Mark as heard
12. 12 Episode 57: AI Agents and LLM Judges at Scale: Processing Millions of Documents (Without Breaking the Bank) Vanishing Gradients 41 min A researcher who builds LLM document pipelines explains how to make them trustworthy at millions of documents. Error analysis comes first, then guardrails and LLM judges, with cost kept in view throughout.
    Aug 2025 Evals Mark as heard
13. 13 Why you should write your own LLM benchmarks — with Nicholas Carlini, Google DeepMind Latent Space 1 h 10 min A DeepMind security researcher argues for personal benchmarks built from tasks you actually needed done, not public leaderboards. It is the eval habit applied to choosing models.
    Aug 2024 Evals Mark as heard
14. 14 Production AI Engineering starts with Evals — with Ankur Goyal of Braintrust Latent Space 1 h 57 min The founder of an eval platform explains why evals, more than logging, are where teams get serious about production. It is long, but it shows how eval tooling fits into the AI development lifecycle.
    Oct 2024 Evals Mark as heard
15. 15 Episode 53: Human-Seeded Evals & Self-Tuning Agents: Samuel Colvin on Shipping Reliable LLMs Vanishing Gradients 45 min The creator of Pydantic argues that most LLM failures come from fuzzy data and drifting prompts, not the model. He shows how small amounts of human labelling can seed continuous evaluation.
    Jul 2025 Evals Mark as heard
16. 16 Creating tested, reliable AI applications Practical AI 50 min A short, practical conversation about closing the gap between an app that works 80% of the time and one you can rely on, through behaviour testing from prototype to production.
    Nov 2024 Evals Mark as heard
17. 17 Next Level AI Evals for 2026 Vanishing Gradients 54 min Takes evals from unit-test thinking to statistics: calibrated judges, confidence intervals, and evals as release gates. Listen once you have a first eval and want to know whether its numbers can be trusted.
    Apr 2026 Evals Mark as heard
18. 18 Tool calling and agents Practical AI 45 min A plain explanation of tool calling as the mechanism under every 'agent': how a model 'talks to' databases, APIs and apps. Good for fixing the vocabulary before the agent episodes.
    Feb 2025 Agents and tools Mark as heard
19. 19 How We Build Effective Agents: Barry Zhang, Anthropic AI Engineer 15 min Video only A 15-minute talk from the co-author of Anthropic's 'Building Effective Agents' post: when not to build an agent, and how simple the loop should stay. Video only.
    Apr 2025 Agents and tools Mark as heard
20. 20 While loops with tool calls Practical AI 45 min The PromptLayer co-founder describes how prompt engineering became context engineering, and an agent became 'a while loop with tool calls'. It covers the testing and versioning that keep that loop manageable.
    Oct 2025 Agents and tools Mark as heard
21. 21 Context Engineering for Agents - Lance Martin, LangChain Latent Space 58 min An engineer who builds agents at LangChain breaks down how agent contexts fail and how teams manage them. It connects Manus's, Cognition's and Anthropic's published lessons.
    Sept 2025 Agents and tools Mark as heard
22. 22 Episode 51: Why We Built an MCP Server and What Broke First Vanishing Gradients 48 min A PM who shipped one of the first LLM features in SaaS (Honeycomb, 2023) and later an MCP server explains what broke when tools met real production data.
    Jun 2025 Agents and tools Mark as heard
23. 23 How to Build a Coding Agent with Nico (Amp Code) Vanishing Gradients 1 h 53 min An Amp engineer takes a coding agent apart: the loop, and the harness around it that runs tools and manages context. The practical point is that every harness component is an assumption that expires as models improve.
    Sept 2026 Agents and tools Mark as heard
24. 24 Why Your RAG System Is Broken, and How to Fix It with Jason Liu - #709 The TWIML AI Podcast 58 min A consultant who repairs companies' RAG systems describes the warning signs he looks for and how he diagnoses them. He treats retrieval as something you measure, not tune by feel.
    Nov 2024 Retrieval Mark as heard
25. 25 Episode 52: Why Most LLM Products Break at Retrieval (And How to Fix Them) Vanishing Gradients 29 min Moderna's data-science research lead explains why LLM features break at the context, not the model. It includes a case where a plain YAML-based system beat a 'smart' retrieval pipeline.
    Jul 2025 Retrieval Mark as heard
26. 26 Generative Benchmarking with Kelly Hong - #728 The TWIML AI Podcast 54 min A Chroma researcher shows how to evaluate retrieval on your own data by generating realistic queries, instead of trusting public embedding benchmarks. It is the eval habit applied to RAG.
    Apr 2025 Retrieval Mark as heard
27. 27 Episode 68: A Builder’s Guide to Agentic Search & Retrieval with Doug Turnbull & John Berryman Vanishing Gradients 1 h 29 min Two search veterans (Reddit and Shopify search; GitHub Copilot) explain how retrieval changes when an agent does the searching, and why measurement against real user intent still decides quality.
    Jan 2026 Retrieval Mark as heard
28. 28 Four Months Inside a Production AI Agent: What Real Users Changed Vanishing Gradients 1 h 5 min A rare follow-up with a team four months after launching a healthcare agent, covering what real users did and how each production failure became a test. It is the most concrete episode here on running an AI system in front of people.
    Jul 2026 In production Mark as heard
29. 29 Episode 70: 1,400 Production AI Deployments Vanishing Gradients 1 h 10 min Patterns from a database of about 1,400 real LLM deployments: what breaks, what costs money, and which architectures survive. A practical guide to failure modes in production.
    Feb 2026 In production Mark as heard
30. 30 Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon Lenny's Podcast 1 h 26 min Two engineers behind more than 50 enterprise AI launches (OpenAI, Google, Amazon, Databricks) describe what separates products that last from ones that stall. It includes a pointed section on what evals cannot do.
    Jan 2026 In production Mark as heard
31. 31 How to Find the Agent Failures Your Evals Miss with Scott Clark - #767 The TWIML AI Podcast 53 min Your offline evals only catch failures you already know about. This covers how to find the unknown ones in production traces and feed them back into evals.
    May 2026 In production Mark as heard
32. 32 Notion’s Token Town: 5 Rebuilds, 100+ Tools, MCP vs CLIs and the Software Factory Future — Simon Last & Sarah Sachs of Notion Latent Space 1 h 17 min How Notion rebuilt its agent four or five times before it was ready for production, and how the company organises evals and failure analysis as a job in its own right.
    Apr 2026 In production Mark as heard
33. 33 How To Hire AI Engineers — with James Brady & Adam Wiggins of Elicit Latent Space 1 h 4 min Elicit's head of engineering on what an AI engineer is and how they hire one. For an experienced developer it reads as a checklist of which existing skills carry over.
    Jun 2024 Career Mark as heard

30 of 33 episodes have a podcast version, which keeps playing with the screen locked. Video plays through YouTube and stops when the screen does. Nothing is hosted here: audio streams from each publisher. What you have heard is kept in this browser only.