PostHole
Compose
Login
Episode 542: Brendan Callum on Contract-Driven APIs
Brendan Callum, engineering manager for the Pinterest developer platform team, discusses the "spec first" approach to API development and how it's different from "API first." Brendan speaks with host Kanchan Shringi about the challenges and advantages...
Brendan Callum, engineering manager for the Pinterest developer platform team, discusses the “spec first” approach to API development and how it’s different from “API first.” Brendan speaks with host Kanchan Shringi about the challenges and advantages of adopting the “spec first” approach based on the launch of the v5 spec at Pinterest. The discussion explores learnings from the launch, internal APIs, team structures and approvals, as well as deprecation of APIs.
Show Notes
- Episode 387: Abhinav Asthana on Designing and Testing APIs/
- Episode 383: Neil Madden On Securing Your API
- Episode 376: Justin Richer On API Security with OAuth 2
- Episode 143: API Design with Jim des Rivieres
Transcript
Transcript brought to you by IEEE Software magazine.
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number and URL.
Kanchan Shringi 00:00:16 Welcome all to this episode of Software Engineering Radio. Today we welcome Brendan Callum to the show to explore the “spec first” model of API development. Brandon is the engineering manager for the Pinterest developer platform team. Pinterest is a visual discovery platform and social commerce network with a mission to inspire. Brendan has also led other engineering teams at Pinterest. As a technology architect and strategist, Brendan specializes in helping organizations scale to match rapid growth. Welcome to the show, Brendan. It’s really great to have you here today. Would you like to add anything else to your bio before we get started?
Brendan Callum 00:00:55 No, thank you, Kanchan. That was great and I’m very excited to be here today to talk about “spec first” development, so let’s get into it.
Kanchan Shringi 00:01:02 Awesome. Just a couple of things before we jump in. I’d like to mention a few related episodes we have done in the past: Episode 387, Abhinav Asthana on Designing and Testing APIs; Episode 383, Neil Madden on Securing APIs; and 376, Justin Richer on API Security with OAuth2. I’ll put these links in the show notes. So, Brendan, what is the “spec first” API approach? How is it different from “API first”? Maybe you could start with explaining what is “API first” and then explain what is “spec first”?
Brendan Callum 00:01:40 Sure, yeah, great. Yeah, I see “API first” as sort of a product strategy or philosophy and “spec first” design — or “spec first” co-generation, as we’ll get into — as kind of a practical or tactical way to put that into practice. So, if you’ve got alignment from your product team, which is not always easy to get, but if you do and you have understanding that APIs are a powerful way to grow and scale your product, then the next thing that’ll come to mind is how do we actually make that happen? How do we ensure that the APIs we deliver and launch live up to our product goals, that they’re consistent and high quality? And “spec first” is, is one way to organize the process of development to ensure those kinds of outcome? Yeah, so I can talk a little bit more about how “spec first” works or some of the details there.
Kanchan Shringi 00:02:34 Absolutely.
Brendan Callum 00:02:34 Absolutely. So yeah, getting into it one level deeper, when you think about the different aspects of what an API is, one of the things that we often did in the past is kind of start with the business logic and work backwards from there as to what the specification or the interface should look like for partners. And when you do that, you often end up with an API that isn’t consistent with the rest of the endpoints that you’ve already released that meets the business goals internally, but maybe doesn’t make sense externally. And you’ve already made decisions as to how the wiring and the business logic should work. So, you’re kind of boxing yourself in to what the interface should look like. “Spec first” says no, like, first things first is how is a client of the API, how is a consumer of the API going to experience what you’ve built? What is their request going to need to look like and what kind of response should they expect? And then, everything internally that happens from the business logic perspective and on the back end is secondary to that. And that allows us to answer questions like consistency and standardization early in the process, rather than having to kind of at the last-minute figure that all out and throw out a bunch of work. So yeah.
Kanchan Shringi 00:03:49 Did you happen to have any example of being not consistent before you started on this spec-first approach, and what was the issue?
Brendan Callum 00:03:57 Yeah, great question. So, a lot of this was actually driven from partner feedback too. So, this is not our first current, as we call it v5 of our public API. This is not our first attempt at building external APIs. I’ve been at the company for almost about eight years now and I got to see some of the first iterations of our external APIs back in 2014, 2015. And we’ve continued to evolve and build and learn from our partners. And one of the biggest things that different developers would tell us is that they had a really hard time predicting how the API would behave from our documentation. They’d read the docs for one end point, and they’d tested out and it would tell a completely different story. And likewise, they learn how to use one set of endpoints for one use case, say campaign management and ads analytics.
Brendan Callum 00:04:51 And then they’d want to expand their application to call some of our other endpoints maybe around shopping. And when they would do that, they’d see a completely different design pattern, a different set of standards for how the interface works, sometimes even different rate limiting mechanisms. And so, what they’re really asking for was consistency, predictability, and not having lots of kind of undocumented changes that cause them to thrash and break their apps. Yeah. So, we considered all that feedback when we were launching this new version of the API, and we decided that we really need to flip our development process and we need to have a consistent set of core reviewers who look first at the spec design and the quality of the interface that we’re presenting to partners. Do the timestamp fields here and the formatting of this request match the kind of conventions we’ve landed on for our other APIs? So, are we doing request batching in the same way that we are across these different endpoints? And things like that need to be front loaded in in the process.
Kanchan Shringi 00:05:59 Makes sense. A follow up question on that is, so that did require a change to the process as you mentioned. Did that require other changes to the team structures, org structures, any kind of cultural changes you needed? Did you have to overcome objections or any hurdles?
[...]