PostHole
Compose
Login
SE Radio 697: Philip Kiely on Multi-Model AI
Philip Kiely, software developer relations lead at Baseten, speaks with host Jeff Doolittle about multi-agent AI, emphasizing how to build AI-native software beyond simple ChatGPT wrappers. Kiely advocates for composing multiple models and agents that take action to achieve complex user goals, rather than just producing information. He explains the transition from off-the-shelf models to custom solutions, driven by needs for domain-specific quality, latency improvements, and economic sustainability, which introduces the engineering challenge of inference engineering. Kiely stresses that AI engineering is primarily software engineering with new challenges, requiring robust observability and careful consideration of trust and safety through evals and alignment. He recommends an approach of iterative experimentation to get started with multi-agent AI systems.
Brought to you by IEEE Computer Society and IEEE Software magazine.
Philip Kiely, software developer relations lead at Baseten, speaks with host Jeff Doolittle about multi-agent AI, emphasizing how to build AI-native software beyond simple ChatGPT wrappers. Kiely advocates for composing multiple models and agents that take action to achieve complex user goals, rather than just producing information. He explains the transition from off-the-shelf models to custom solutions, driven by needs for domain-specific quality, latency improvements, and economic sustainability, which introduces the engineering challenge of inference engineering. Kiely stresses that AI engineering is primarily software engineering with new challenges, requiring robust observability and careful consideration of trust and safety through evals and alignment. He recommends an approach of iterative experimentation to get started with multi-agent AI systems.
Brought to you by IEEE Computer Society and IEEE Software magazine.
Show Notes
Related Episodes
- SE Radio 633: Itamar Friedman on automated testing with generative AI
- SE Radio 603: Rishi Singh on using Gen AI for test code generation
- SE Radio 626: Ipek Oskaya on Gen AI for software architecture
- SE Radio 680: Luke Hinds on Privacy and Security of AI Coding Assistants
From IEEE Computer Society
Transcript
Transcript brought to you by IEEE Software magazine.
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number and URL.
Jeff Doolittle 00:00:18 Welcome to Software Engineering Radio. I’m your host, Jeff Doolittle. I’m excited to invite Philip Kiely as our guest on the show today for a conversation about multi-agent AI. Topics around machine learning and artificial intelligence have come up in previous episodes. Some examples are Episode 633 when Itamar Friedman appeared on the show to discuss Automated Testing with Generative AI. Episode 603, featured Rishi Singh on using Gen AI for test code generation and Episode 626, featured Ipek Oskaya on GenAI for software architecture. Philip Kylie leads software developer relations at Baseten. Prior to joining Baseten in 2022, he worked across software engineering and go-to-market for a variety of startups outside of work. You’ll find Philip practicing martial arts reading a new book or cheering for his adopted bay area sports teams. Philip appeared previously on Software Engineering Radio in Episode 426 for a discussion about writing for software developers. Philip, welcome back to the show.
Philip Kiely 00:01:20 Hey Jeff, thank you so much for having me.
Jeff Doolittle 00:01:22 Glad you’re here. Multi-agent AI. What are we going to talk about in this episode, Philip?
Philip Kiely 00:01:29 Today we’re going to talk about how to build things that aren’t ChatGPT wrappers when you want to build what we call at Baseten AI native software. So that’s companies that are started to build AI native products as well as existing companies who are adopting AI and trying to become AI native. It’s about more than just producing a wrapper around a single model call. It’s about composing multiple models together into a unique and differentiated system that can actually achieve your user’s goals.
Jeff Doolittle 00:02:03 So give us more details on what you mean. It sounds like you’re saying most teams start by doing what you’re calling a wrapper around something like ChatGPT or maybe some other models. What exactly does that entail?
Philip Kiely 00:02:15 Yeah, so when you look at the sort of evolution of an AI product, you might have something that starts out as an add-on. Let’s say you add a Chat window or a search window to existing product.
Jeff Doolittle 00:02:28 Everybodyís doing this right?
Philip Kiely 00:02:30 Not exactly, but then there are a lot of companies now that are building really exciting and new products that weren’t possible before we added all of these AI capabilities. an example of that is something like DS Script, which is a tool that I use all the time as a content creator where I upload the video that I made, I get a transcript, and then I can type in the transcript to actually edit the video. This is a brand-new capability and that requires more than one model. It requires transcription, it requires language models, it requires the ability to actually have a model that takes action on an object like the video sequence. Or if you think about a company like, I don’t know, source Graph or Zed who are building code editors, it’s more than just ask a question, get an answer, paste it into your code base. You’re actually integrating the context of the code base into an agentic framework and then powering that with multiple different models, not just one. all of these products, they’re using a bunch of models, they’re coordinating and orchestrating them together, and they’re also figuring out how to do all of that in a way that’s fast for the user from a latency perspective and creates like feasible unit economics from a business perspective.
Jeff Doolittle 00:03:49 Okay. for people that are just getting started, are there benefits sometimes to just using maybe off the shelf AI tools and just wrapping them?
Philip Kiely 00:03:58 Yeah, a hundred percent. When you’re getting started and you just need to introduce some kind of intelligence to your product, you should absolutely go grab an off the shelf model. Something like Gemini, Claude, GPT-5, all of these are great models. They’re very capable, they have a wide range of capabilities and they’re going to show you immediately what’s possible at the frontier of intelligence. But as you advance in the product development cycle, as you roll out to production to hundreds of thousands, millions of users, you start to understand the specific needs of your product and your audience and need to build a tailored AI strategy to accommodate those.
Jeff Doolittle 00:04:40 Okay. Now, some of this stuff is still new. Things change so fast that I don’t want to assume that every listener knows what the word agentic means, which you just use that word. So could you just describe briefly what is agentic AI as opposed to just what we’ve thought of as like Chat AI, this sort of stuff that’s been common and is now very well known?
[...]