Conf42 Cloud Native 2022 - Online

Taking a step back and leveraging GitOps to wrangle your clusters and projects

Video size:

Abstract

You’ve moved to the cloud-native space. You now have multiple Kubernetes and/or OpenShift clusters, possibly spread around various clouds, and it’s gotten legit unruly. Don’t worry; I’m here to help and give you the tools to help wrangle your clusters so you can focus on building for the business instead of fighting infrastructure. Leveraging GitOps and the principles around a mash-up of traditional infrastructure management and software development practices creates an auditable/accountable/enforceable way to interface with your fleet.

In this talk, I’ll be walking through (a strong possibility of your situation) why GitOps has become the de facto way to maintain your cluster(s), advantages and struggles going down this path, and strong suggestions on “fastest time to dopamine” paths. You should know that you can take bits and pieces from this talk and build this up piecemeal, but the sooner you leverage this, the sooner you can start focusing on what makes your business win instead of losing time with administration.

Summary

  • JJ Asghar: Taking a step back and leveraging Gitops to wrangle your clusters and projects. And again, JJ, developer advocate. If you have any questions or thoughts, never hesitate to reach out directly to me.
  • If you're at this talk, you've probably made your adventure into cloud native. I'm here because I'm slow but surely taking over more and more stories inside of IBM around Gitops. It's becoming a de facto way to do work with our cloud native space.
  • CI stands for continuous integration and CD stands for Continuous delivery. CI is pretty much industry standard now across the board. Continuous delivery is a rather robust, mature process. Gitops is basically the other half of CI.
  • Gitops is a way to leverage the typical git workflow in configuration of one or many kubernetes. Clusters can also be openshift. It leverages something called a pull mechanism instead of a push. I genuinely believe this is going to become the de facto way that we are going to maintain more than one cluster.
  • Gitops is a pull based mechanism instead of a push based mechanism. This means that main is always deployable, right? Like again, CD continuous deployment. There's multiple different ways of making this happen. The first one we need to talk about is pull versus push.
  • One's Argo CD, the other one's Jenkins X. The one that kind of grabbed me and really made me understand the beauty of Gitops is Argo. If you're already in the Jenkins ecosystem, Jenkins actually is a natural progression and more power to you.
  • You can leverage Argo CD to maintain some level of consistency through all of your different Openshift clusters, or Kubernetes clusters. The idea is to add a link to Google to every single cluster. You can do this through multiple different ways.
  • The idea is that now I have my configuration inside of my GitHub repository, which could be any git repository, and be able to change that now if I have enough time. That moment that you see everything go from out of sync and then back to sync all of a sudden makes Argo CD makes so much more sense.
  • I genuinely believe this to be the next generation of how we're going to maintain our Openshift or Kubernetes clusters. People do say that Gitops is DevOps two or 3.0 or whatever you want to call it. But DevOps is still encompassing so much more than just maintaining infrastructures.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hello, my name is JJ Asghar, and as you can see up over there, you can find me at jgasgar on Twitter or more or less anywhere else. And awesome@ibm.com. Yes, I really do have that email address, which is always fun. So you've come here today to learn about some githubs. I'm just kind of stepping myself into this world and the things that I finally learned Andor embraced. Frankly speaking, I'm shocked it took me this long to get here. I'm very excited about this talk, and we're going to go over some basic things to make sure we're all talking about the same stuff, and then we're actually going to play with some Gitops and two very simple use cases that as you build off of it, youve be amazed on what you can actually get done. And it just seems like a natural progression for all our cloud native lifestyles. So let me go ahead and flip my screen. There we go. Youve should be able to see IBM there. Yes, as I said earlier, I do work for IBM. And here we go. Taking a step back and leveraging Gitops to wrangle your clusters and projects. And again, JJ, developer advocate. So please, if you have any questions or thoughts, never hesitate to reach out directly to me. My life is to be a personable nerd to help you out here. All right, so let's talk about the introduction. Basically, if you're at this talk, you've probably made your adventure into cloud native. You probably have a kubernetes or a couple of kubernetes clusters or maybe even some openshift clusters, and you have this ability to deploy stuff. You understand what pods are, you understand what the advantage of cloud native is, and you're doing well. But this git ops thing keeps popping up in a lot of different conversations, and you probably want to learn and or leverage it, and that's why you're here. I'm here because I'm slow but surely taking over more and more stories inside of IBM around Gitops. It's becoming more and more a de facto way to do work with our cloud native space, and it really is just a natural progression. And I'm really hoping this talk kind of brings that to you. The natural thing you need to start talking to after about understanding this, you need to start talking about CI and CD. If you've never seen these two terms before, I'm shocked. It's kind of can industry standard now, but CI stands for continuous integration and CD stands for continuous delivery. CI is pretty much industry standard now across the board. If youve writing some software somewhere, you probably have some level of a CI pipeline that constantly either builds it or runs your unit tests or runs. Youve integration tests or things like mean by default things like GitHub actions and Travis and Circle CI. They're very low overhead to get running, and it gives you a peace of mind that your code is doing what you need to do. As I said, it's pretty much the standard. Now, CD, on the other hand, which is continuous delivery. As much as people want to say it's CI and CD in one breath, CD isn't industry standard. Continuous delivery is a rather robust, mature process that you know, you can trust your code to continually get deployed. There's philosophies out there that the main branch should always be deployable. There's also a lot of different philosophies, like the production branch should always be deployable, things like that. And that type of gating on there actually kind of goes in the face of CD, where your application should just be deployable. I guess arguably the production branch or main branch you target and then it's always deployable, and then you can always make sure it works. But the point being is that when CI finishes, CD happens. So if everything's green, it just goes out there. So as soon as you merge it to whatever deploy branch you have, which should be main, in my personal opinion. When you hit that big green button, the tests kick off one last time to verify everything hasn't changed. Nothing, no regressions. And then the theory was is to deploy it. And that's where we kind of come into this conversation around Githubs. Gitops is basically the other half of CI. It is the CD portion of it. So let's bring this over here. So what is Gitops? Simply said, it's a way to leverage the typical git workflow in configuration of one or many kubernetes. Clusters can also be openshift. In the applications inside of them. It leverages something called a pull mechanism instead of a push, which is very important to talk about. We'll talk about more in a minute. You can deploy something from like GitHub actions or Jenkins, or even Openshift webhooks. The biggest pain point that GitHub takes away is that now you have a true history of your deployment and how things happen. There are real problems with not leveraging Gitops completely. This is more or less a whole or nothing process. You can't kind of just do a Gitops workflow right. It doesn't work that way. When you decide to go down the Gitops path, you are making a change to your deployment process and your deployment mechanisms that will be complete whole hog. One reason why we like it on the openshift side so much is because we do pick the best of breed of things from the red hat standpoint. And the Gitops workflow actually fits really well in that space, which is really nice. So why should you care, right? As long as our app is up and running, why should I bother with all this overhead? Well, if you have one clusters, okay, maybe this isn't worth it, right? But as soon as you start building multiple clusters, you start building a dev cluster, building a QA cluster, or for that matter, you have a process where you need features out quickly. The idea of doing manual deployments is tough. Obviously youve moved to the cloud native space and now you have multiple kubernetes or Openshift clusters. It could even be spread around various clouds. Right. Like you're not all on IBM cloud, which you should be. I'm just saying. Kidding. I work for IBM. Of course I'm going to say that. But it's got unruly. You have clusters, you have your dev environment Andor Azure your production, and IBM Cloud Andor for some reason you have a bunch of s three buckets. I dont know. It's unruly. It's frustrating. And I'm here to tell you not to worry. One of the beauties of Gitops is the ability to maintain all these clusters. Because what is a Kubernetes cluster but a bunch of yaml and leveraging Gitops to edit that Yaml Andor whatnot allows for some way to wrangle around clusters. IBM here to tell you and give you these tools to help make this happen, to help make this happen so you can focus on building for the business instead of fighting your infrastructure. You've already spent so much time Andor effort learning how to get your application inside of kubernetes. Now you got to fight maintaining the infrastructure all over again. And Gitops kind of takes some of that pain away and in turn gives you that CD that we were talking about earlier that you desperately should be moving towards. Leveraging Gitops and the principles around is a mashup of traditional infrastructure management andor software development practices that your developers have been using for almost decades now. And it gives you an audible, accountable, and enforceable way to interface with all of your Kubernetes fleets, which is really important. So from what I've been learning about Gitops and all that. I genuinely believe this is going to become the de facto way that we are going to maintain more than one cluster. Okay. Arguably youve can maintain one, as I said earlier, you can maintain one cluster with this if you really want, but for me, this really feels like what we are going to be doing moving forward. So let's talk about a typical git workflow, right? Like I mentioned about leveraging into main earlier, this is a very simple example, right? And of course you're going to have a lot of different arguments around all the different stuff, but let's just stay with this very simple, very simple workflow that I'm about to show as a de facto way to use git. Okay? I mean, if you want to talk about git flow and other weird ways of doing it, sure, fine. You can absolutely implement those inside of the space, but I'm looking for just showing people how to get started so they can mold it to their normal workflow. So this is it, right? You've got your main branch, where that's all your code, and then you have a feature branch or a bugfish branch that branches off, does the fixes, and then you merge it back into main. That's it, right? It's nothing too crazy. You work in isolation, which is really nice. Your feature branch and youve bugfish branch are off on bits own branch, as you can see from here. And that's important to know. So that means that main is always deployable, right? Like again, CD continuous deployment. So this simple branch, right, it's a simple branching mechanism, is all you need to do to start with to leverage Gitops in your normal workflow. So there's multiple different ways of making this happen. And the first one we need to talk about is pull versus push. So I mentioned earlier that Gitops is a pull based mechanism instead of a push based mechanism. So when you run your CI pipeline and you push out your code to your main branch, and now it does all the tests and whatnot, traditionally you would push your deployment out to the new clusters to push it out to your new servers or whatever, right? Well, Gitops comes and flips the narrative. What they do now is pull. So constantly there's a little agent that sits on side of your cluster pulling against wherever youve Git repo is looking for changes. You can obviously change how quickly and how slow you want to do it, and that's fine. But the idea is that you pull into your cluster any changes, which means your application needs to be aware of that it could be updated at any time, which can be a little bit level of, could be a little bit of an engineering problem. I mean, that's the reason why you should run things through QA and test and all. But the idea is that it pulls constantly, which actually opens up some really interesting conversations because that means now if you have can air gapped environment, you can have this agent running on an air gapped cluster somewhere that pulls off of a cache that's been security scanned, for instance, which is really neat. So instead of you just pushing your code and it pushes out to that environment, now you can cache the things and verify that it's the stuff you expect it to be. This also gives the ability to pull in a, you don't have to push everything off at the same time. You pull things at a rate that you feel comfortable with, which allows for better, like, you know, uptime and whatnot in the long term, because youve know, if you pull Andor something breaks, youve can have the intelligence in there to make sure it doesn't change it. But if you push on top of it, pushing on top of it, unless youve made some really interesting engineering choices, you could take your application down. So it's important to recognize these things. So let's talk about the three major players in the space, and one's Argo CD, the other one's Jenkins X. And frankly speaking, the granddaddy of them all is something called Flux CD. Argo is the first one, Jenkins X, and obviously flux. So flux CD is CLI based and is very lightweight and does exactly what you want it to do. It is unbelievably powerful. And IBM very, very impressed with the work it does. You have Jenkins X, which is kind of the next generation of the Jenkins ecosystem, and it's very impressive, but it's very prescriptive, or at least the last time I played with it, it was expecting some very specific things. And if you're already in the Jenkins ecosystem, Jenkins actually is a natural progression and more power to you. The one that kind of grabbed me and really made me understand the beauty of Gitops is Argo. And in the rest of this talk, I'm going to be actually focusing this conversation around Argo CD because it really did just kind of make things make sense to me, and that's important to say. So let's go ahead and install one, and I'm going to go ahead and bring this small. No, that's not what I wanted. Let's see here. We're going to make this small, this over here, and then bring this over here, and then probably make this a little bit bigger so youve all can see. And then if I do cubectl get ns. So as we see here, I'm on an openshift cluster and this should work. There we go. As you can see, and I built an upset cluster about 17 days ago. Got a bunch of projects in here, but if you look here, kget pods, I'm in the default namespace and there's nothing in there, which is exactly what we wanted. Where is my notes? So the first thing first is I need to. What the. So there are some really nice straightforward ways to install Argo CD. And right here is I've actually created a nice little markdown file. But the first thing we're going to do here is we're going to create a namespace, Argo CD namespace, and then I'm just going to pull it directly down from the stable branch and kubectl apply it. So there we go. So you see, we creates the namespace and we threw in some custom resources, which is nice. And that's actually all you need to do. Argo CD is just two commands, which is always fun. So if I do kgetpods and Argocd, as you see, we're creating our containers. I did not pre bake this, I wanted to show this live, which is important. But that's actually it, that's actually all the installation, which is pretty nice. So let's see here, bring this back over here. And as that is going. So let's talk about some infrastructure management. So the first thing I wanted to talk about obviously has the way to deal with a bunch of fleets. So as I've been seeing with different customers out there, as much as I tried to tell people have one, big clusters use namespaces and projects to split stuff up. Frankly speaking, people didn't adopt it right. They started spinning up multiple clusters for dev, QA and prod or whatever. So that's actually where Argo CD comes into play really nicely. You can leverage Argo CD to maintain some level of consistency through all of your different Openshift clusters, or Kubernetes clusters. I'm going to be showing Openshift because it's just a little bit easier with the GUI, but all these things work on kubernetes just fine too. So this right here, if you can read it, is basically creating something called a menu link inside of our console on our Openshift cluster. I'm going to show here in just a second, plugging it in and making it happen. But the idea is, let's say for instance, you wanted to change, I don't know, a logo everywhere. Or for that matter, you wanted to add a link to Google, to every single openshift cluster. If you bring, take a look here, just so I'm completely transparent, here's an openshift cluster here. And you see here, there's this nice little drop down up here. There's this manager managed cluster right here. The idea is to add a link to Google, right? It's not great, it's not a great example, but the idea is now I can shotgun this across every single cluster. You have an internal website that you all need to add for like privacy, I don't know, whatever you imagine it. So now every single cluster that's running Argo CD and I add this little configuration to will now get that link, which is pretty nice. So if I come back over here and see that Argo CD is up, there we go, we're running, everything's great. So the interesting part about it with Argo CD is that you don't have to expose it to the Internet, right? It can just sit on the cluster by itself. So one of those times you might want to actually do it is you go ahead and create a port forward here. If I bring this back over and then I go to 127 80, 80, there we go. It's like, hey, youve not secure, blah, blah blah. And as youve can see here, here's Argo CD, right, like I did port forward to it. It's sitting there. IBM not exposed to the real Internet. It's nice and secure, which is always good. But before I go any farther, IBM going to need that password. So I'm going to go ahead and grab this and there it is. Don't worry. You do not worry about stealing this from me because I will be blowing up this cluster after the fact. But if I go to admin here and then pop that password in, and there we go, we got argo CD sitting in here running, and by default it has cluster admin rights, so it can create namespaces, andor do things and whatnot, which is always nice. So I have this Gitops tutorial repository, which is important to show this is what we're actually going to be linking up against. I actually have my installation instructions in here too, so you're more than welcome to play with this in the future if you like. If you go into the OS thing here, and then if I go into tenants. And then here's my link to google. Here is my customization file which is if you need that, know what that is. But here is the actual yaml which is kind of nice. I don't know if you can read that but hopefully youve can. So if I go ahead and bring this up over here, I can go ahead and just create a new app here. Now you can do this through multiple different ways. I'm just going to do it through the GUI because it's nice and easy. So I'm going to do link to google and then my sync policies, I want to have it do automatic so it's constantly building against it. And then the source repository is this whole thing here. If IBM going to go ahead and go ahead and here copy this and then you can actually change it to whatever revision you want which is nice. And did I erase that? All that oops link to google. I want this to be automatic. It can actually self heal which is nice too. That's important to recognize. So if something goes funky you can always try to do it. And then grab that there and then the path and then I'm going to go directly to here which is OS tennis main copy. Put that in there and then the cluster URL, you can actually have argo CD control a bunch of remote clusters if you wanted to, which is neat too. But we're just going to do this for the local one and we're going to go ahead and do this in the default namespace and then that should be it. So let's go ahead and create default. And there we go. We go ahead and create it. And if I've got my notes correct, this should make it happen. The path needs to be absolutely create. How about that one? Maybe. Let's see. This did go faster in my previous testing, so I'm so sorry this is taking so long. One more time. So then I believe by default this checks every five minutes, but we'll see. And there we go. So now it looks and then it sees this and it's in cluster. And then what we're going to do is go ahead and hit that sync button. Synchronize. There we go. As you can see immediately it said it's out of sync. So it's figuring that out. It synced it. And then if I've got this right, we should be able to see okay, that's synced. So then if I come back over to our cluster here and I refresh this page. There we go. We got our link to google. Youve see that? Pretty neat. Cool. So if I wanted to change this all of a sudden. Hang on. The idea is I could be able to change this to whatever I need it to be. So if I go back over here to this link copy, throw it in here, and then go back into os, and then tenants link to google and change this to bing, for instance, because I'm crazy or whatever, I commit that again, I should have created another branch, do all that stuff, but you get the point. And now if I go ahead and say, sync this one, and then I synchronize again and it'll say it's out of sync here in a second. Last sync. There we go. So it's out of sync. It recognized. Hey, it's out of sync. Now, this does self heal. So if I had a bunch of clusters, I wouldn't have to synchronize every single one of them. But the idea is that, right? And now we should see this change here in a second refresh maybe. There we go. So now if I come back over to my openshift cluster refresh, and if I click on this one and link to google, there I am. At mean, point taken. It's not super crazy, but the idea is that now I have my configuration inside of my GitHub repository, which could be any git repository, and be able to change that now if I have enough time, which I don't think I do, unfortunately, I wanted to show off the ability to do a simple application here, which in that same repository I can easily add. And this little simple python app, as you can see, is so much more advanced. But it's all in there too, where you can play around and sync it all in, and then you can just make a small change to the flask app or the deployment or whatever, and then it actually federates throughout the whole thing. That moment that you see everything go from out of sync and then back to sync all of a sudden makes Argo CD makes so much more sense. It really truly does. So I'm coming right up to 30 minutes, or actually 26 minutes. So let's just talk about a conclusion real fast. I genuinely believe this to be the next generation of how we're going to maintain our Openshift or Kubernetes clusters. Fleets and fleets of clusters out there need consistency. If I learned anything from the configuration management space, which was when you spin up a bunch of EC, two instances or vms or whatever, you need to make sure that they all have the same version of OpensSL or whatever on top of it well, in the Kubernetes space and Openshift space, you're going to have a bunch of devs touching a bunch of different clusters, and they're going to want the same interface and know that the same stuff is there on each one. And Gitops is just going to do that for us as administrators has people who maintain the infrastructure for your cloud native users. It's just a natural progression. People do say that Gitops is DevOps two or 3.0 or whatever you want to call it. I think that's a little bits unfair. I think Gitops is a natural progression, but not really another version. It's got some strengths to it, and I believe it's going to be the next one. But DevOps is still encompassing so much more than just maintaining infrastructures. So thank you for your time. Hopefully you did learn something new, and please don't hesitate to reach out. My job is to be a nerd that you can find, as I said at the beginning, and hopefully this tickled your fancy and play around on that Gitops tutorial. There's a few things in there. I am updating it pretty regularly with new stuff or whatever, and I do want to say Christian Hernandez from Red Hat has some amazing tutorials on how to use Argo CD. Actually, the Gitops tutorial I have there is from his tutorial, which is a lot more in depth than what I did. But if this spurred something, I strongly suggest going through his thanks y'all and to I hope to see youve everyone in the real world soon.
...

JJ Asghar

Developer Advocate @ IBM

JJ Asghar's LinkedIn account JJ Asghar's twitter account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways