How We're Preventing Breaking Changes in GraphQL APIs at Buffer — and Why It's Essential for Our Customers

As part of our commitment to transparency and building in public, Buffer engineer Joe Birch shares how we’re doing this for our own GraphQL API via the use of GitHub Actions.

⚡

At Buffer, we’re committed to full transparency — which means building in public and sharing how our engineers work. You’ll find more content like this on our Overflow Blog here.

We’ve all experienced it at some point — a change is deployed to an API and suddenly, clients stop working.

User experience deteriorates, negative reviews start coming in, customer advocacy starts dealing with requests, multiple engineers start digging into the issue, and it quickly becomes all hands on deck.

Not only does this lose the trust of our users and interrupt their workflows, but it also costs an organization a lot of time and money to resolve these issues.

One way to prevent all of this from ever happening is to detect these breaking changes before they are merged into our repository at a Pull Request stage.

This way, we can prevent such changes from ever being merged, avoiding a breaking experience for our clients and reducing downtime for our users.

As part of our commitment to transparency and building in public, I’m going to share how we’re doing this for our own GraphQL API via the use of GitHub Actions.

When it comes to detecting breaking changes, we can detect these by taking the schema representation on the branch of the pull request and comparing it with the schema representation on the main branch. We can then use the result of this diff to determine whether breaking changes exist in our schema changes.

When it comes to this workflow, we can break this down into several steps:

• Generate schema for the current branch: This will give us a schema that represents the changes we have made in our pull request
• Generate schema for the main branch: This will give us a schema that represents our current production API
• Perform verification of the current branch schema against the main branch schema: This will tell us what changes exist in our schema comparison and if any of them will break clients
• Post the result to the Pull Request: This will allow us to ‘fail’ the pull request to prevent it from being merged, along with alerting the author of the breaking changes

With this in mind, we’re going to build out an automated workflow that will run these operations for any Pull Request in our repository. For these pull request checks, we’re using GitHub Actions, but most of the following code will work for whatever CI setup you are using.

Note: We won’t be diving too much into the concepts of GitHub Actions here. If you are not familiar with Actions, I suggest following the quickstart tutorial.

Setting up the workflow

We’re going to start by setting up a new GitHub Action, we’ll create a new file named breaking_change_check.yml and start by giving our Action a name.

name: Schema Change Verification

Next, we’ll want to specify when this action is going to be run — this will essentially allow us to define when we want to perform the checks in the PR.

We’ll not only want to do this on opened events (when the PR is initially opened), but also if it is reopened or synchronized — which will allow the checks the re-run if there are additional commits pushed to the pull request.

This ensures that we are always checking for breaking changes on the latest commits pushed to the branch.

name: Schema Change Verification:
on:
pull_request:
types:
- opened
- reopened
- synchronize

We’ll also only want to run these checks when .graphql schema files are changed, so we’ll specify this rule using the paths property.

name: Schema Change Verification
on:
pull_request:
types:
- opened
- reopened
- synchronize
paths:
- 'graphql/*/**.graphql'

graphql//**.graphql* is the path where our GraphQl schema files are located; you will need to adjust this according to your project.

Generating the current branch schema

Now that we have the foundations of our action configured, we can move on to defining the jobs that will be responsible for generating and verifying our schema.

We’ll start here by defining a new job generateChangedSchema and defining the use of the ubuntu-latest runner.

name: Schema Change Verification
...

jobs:
generateChangedSchema:
runs-on: [ubuntu-latest]

Next, we’ll need to perform a couple of setup operations for our job. We’ll want to start by checking out the repository at the branch of our PR, for which we’ll use the checkout action.

Our action is also going to utilize node, so we’ll need to install this using the setup-node action.

name: Schema Change Verification
...

jobs:
generateChangedSchema:
runs-on: [ubuntu-latest]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v3
with:
node-version: '18'

At this point, we’re now ready to move on to the generation of our schema. For this, we’re going to need to load our schema files and then merge them into a single schema file. This makes the verification process much simpler as we only need to work with a single file instead of multiple.

To make this process easier, we’re going to utilize a couple of dependencies from graphql-tools. We can see that these are named according to what we need them for, we just need to add them to our package.json file as a dev-dependency.

"@graphql-tools/load-files": "7.0.0",
"@graphql-tools/merge": "9.0.1"

With these dependencies in place, we’re now going to write a small script that will load and merge all of the .graphql files in a given directory.

import * as fs from 'fs'
import { loadFilesSync } from '@graphql-tools/load-files'
import { mergeTypeDefs } from '@graphql-tools/merge'
import { print } from 'graphql/language/printer'

const mergeFiles = async (): Promise<void> => {
try {
// using the provided path, load the types from the schema files
const typesArray = loadFilesSync(`../../graphql/src`, {
extensions: ['graphql'],

// merge all of the types from the recieved schemas, compressing all of the types     // into a single place
const result = mergeTypeDefs(typesArray)
// write the schema to a single file to be used for diffing
await fs.promises.writeFile('generated/schema.graphql', print(result))
} catch (e) {
console.error("We've thrown! Whoops!", e)

;(async (): Promise<void> => {
try {
// the merged schema file will be created in the generated directory, so create
// the directory if it does not yet exist
if (!fs.existsSync('generated')) {
fs.mkdirSync('generated')
await mergeFiles()
} catch (e) {
console.error("We've thrown an error! Whoops!", e)
})()

We’ll then add a new [task] to our package.json file so that we can easily execute this with the required arguments. This script takes a single argument when being executed, which is the path for the merged schema to be saved.

If you need to provide paths for the location of schema files, this can be done through additional arguments. Our schemas are located in a single directory, so we hardcode this inside of the script itself.

"graph:generateSchema": "ts-node scripts/generateSchema.ts"

With this in place, we can now execute this command from our generateChangedSchema job. For this, we’ll use a bash script step where we’ll need to navigate to the directory where the generateSchema command exists and then use the node command to execute it.

name: Schema Change Verification
...

jobs:
generateChangedSchema:
runs-on: [ubuntu-latest]
steps:
...
- name: Generate Schema
run: |
cd services/api-gateway
node graph:generateSchema

At this point, we will have a merged schema file that contains the contents of all our schema files. To wrap up this job we’re going to attach this schema file to our workflow run, this is so that we can download that file for use within the next job in our workflow.

[...]

*Original source*