PostHole
Compose
Login
SE Radio 590: Andy Suderman on Standing Up Kubernetes
Andy Suderman, CTO of Fairwinds, joins host Robert Blumen to talk about standing up a kubernetes cluster. Their discussion covers build-your-own versus managed clusters provided by cloud services, and how to determine the number of kubernetes clusters an organization needs. Andy describes best practices for automating cluster provisioning, and offers recommendations about customizations and opinionation of cloud service providers, choice of container registry, and whether you should run complementary services such as CI and monitoring on the same cluster. The episode also examines the day 0/day 1/day 2 lifecycle, cluster auto-scaling at the cloud service level, integrating stateful services and other cloud services into your cluster, and kubernetes secrets and alternatives. Finally, they consider the container-network interface (CNI), ingress and load balancers, and provisioning external DNS and TLS certificates for cluster services.
Andy Suderman, CTO of Fairwinds, joins host Robert Blumen to talk about standing up a kubernetes cluster. Their discussion covers build-your-own versus managed clusters provided by cloud services, and how to determine the number of kubernetes clusters an organization needs. Andy describes best practices for automating cluster provisioning, and offers recommendations about customizations and opinionation of cloud service providers, choice of container registry, and whether you should run complementary services such as CI and monitoring on the same cluster. The episode also examines the day 0/day 1/day 2 lifecycle, cluster auto-scaling at the cloud service level, integrating stateful services and other cloud services into your cluster, and kubernetes secrets and alternatives. Finally, they consider the container-network interface (CNI), ingress and load balancers, and provisioning external DNS and TLS certificates for cluster services.
Show Notes
Transcript
Transcript brought to you by IEEE Software magazine and IEEE Computer Society.
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number and URL.
Robert Blumen 00:00:19 For software engineering radio. This is Robert Bluman. Today I have with me Andy Suderrman. Andy is the CTO of Fairwinds, a Kubernetes service provider. He’s previously held roles as SRE, principal, engineer and director of R and D and technology. He works with infrastructure spanning major cloud providers and verticals. He is a graduate of the Colorado School of Mines. Andy, welcome to Software Engineering Radio.
Andy Suderman 00:00:46 Thanks for having me.
Robert Blumen 00:00:48 And today Andy and I will be talking about setting up and managing Kubernetes cluster. We’ve done a few episodes on Kubernetes already, 446, 334 and 319, and it was mentioned in 440 on GitOps. We also have some recorded content on Kubernetes coming up that we don’t have an episode number yet, so we’ve covered it quite a bit. I’d like to just do one background question. If you could give a really brief synopsis of what Kubernetes is and what problem it solves, then we’ll be talking more about how to set it up.
Andy Suderman 00:01:23 Yeah, sure. Happy to. So Kubernetes at its core is a container orchestrator. We use it to run containers across multiple machines and do lots of things with containers. So at its heart, it’s an API that allows us to describe the desired state of containers running across multiple machines. So that’s probably the simplest way to define Kubernetes and how we think about it.
Robert Blumen 00:01:45 So I wanna start out with, let’s say an organization has decided they want to migrate to Kubernetes or adopt Kubernetes as their orchestration platform. How did that conversation go to get to the point and what alternatives did they consider and rule out?
Andy Suderman 00:02:03 I think it’s a really interesting way to ask that question because most of the time I get asked, what should we think about when we’re moving to Kubernetes? People have already made the decision. I think it’s important to think about the reasons why. So lots of different alternatives to consider. I think one of the biggest things to think about with moving to Kubernetes is taking on complexity. You’re adding so many layers of complexity to your stack. Do you really need that level of customization? Do you need that level of control? Are you building a platform on top of that? Are you serving multiple teams in multiple apps? If you just have one app and it’s already containerized and you don’t need to run it across, you don’t need a ton of control over how it’s run and you only have one. Maybe don’t use Kubernetes and use something like Cloud Run or Fargate on EKS or one of the other, many other ways to run containers. So I think thinking about the balance of complexity versus features that you get from running Kubernetes is super important.
Robert Blumen 00:02:59 I’m gonna ask you a question where the answer’s gonna be. It depends, but do the best you can. A medium-sized organization that has some different products and they want to get all in on Kubernetes, how many clusters are they gonna end up with and what are the driving factors in triggering when you can run certain things on the same cluster when you need a new cluster? And how much overhead is there for each cluster?
Andy Suderman 00:03:27 Yeah, this is a question we get a lot and the answer is almost always two. You need one non-production cluster and one production cluster. And beyond that, Kubernetes has so much built-in ability to segment workloads in different ways and control who has access to what that it’s very uncommon to really need, especially in a medium to small-sized organization, to need more than just the non-prod and the prod cluster. You have to have that separation between non-production and production because you need to be able to test changes that are cluster wide and you can’t safely do that in production. I have seen companies run massive single clusters for the entire organization, prod and non-prod, and that usually turns into a bit of a disaster. So things to think about when you’re segmenting workloads, are they particularly noisy in one particular area of resource utilization? There’s different ways to segment that out, but sometimes a separate node group is necessary. You should always utilize namespace as much as possible because they give you a very cheap segmentation line to draw between different areas in your clusters. I think I hit all the points of the question.
Robert Blumen 00:04:28 Yeah. Now my understanding it, maybe I’m wrong about this, but Kubernetes is single region?
Andy Suderman 00:04:35 Generally that is the case. Most implementations of Kubernetes allow you to run multiple availability zones in the same region, but running cross regions is generally not recommended, mostly because of network transit issues and not being able to sort of make the cluster be completely aware of what network topology looks like between different segments of the cluster.
Robert Blumen 00:04:57 If I have a product and I wanna run it on multi regions, that would imply I’m gonna need one cluster per region. Is that correct?
Andy Suderman 00:05:05 That’s typically how we recommend folks do it. I’ve seen solutions where, especially in in Google where networking is a little bit flatter, where you can run multi-region clusters, but typically we run one per region.
Robert Blumen 00:05:18 A small company that starts because they have one product idea. So you put that out on your Kubernetes cluster, medium sized company that has multiple products. Are you going to run several products all on your same prod cluster or are there gonna be different kinds of considerations of, could be anything and maybe you could include it in your answer of why you’d need to put each product on its own cluster or maybe not, maybe not all end to one.
[...]