Let's talk about DevOps and internal developer platforms and how those might be the only thing that matters for DevOps engineers or DevOps culture, what terraform that's called today. My name is Victor. I work for upbound. We're a company behind Crossplane and quite a few other things you will see crossplane later. But that's not the focus right now. When I said DevOps, most of you are probably thinking about containers. Kubernetes, cloud, GitHub, CI CD logging, monitoring, infrastructure as code troubleshooting, security service mesh and so on and so forth. But that's not what DevOps is about. Those are operations tools, right? Those are the tools that if they did exist in the past, some of them did. We would be using them as members of OpS departments or Ops people or security experts or sysadmins, whatever we were called in the past. But DevOps is about combining development and operations and enabling developers, application developers, to be selfsufficient, to move faster, to be able to do things by themselves. And when we talk about developers, do we really expect them to use and know all those things? Are they all going to become experts in containers? Kubernetes, cloud, Githubs, CI CD logging, monitoring, troubleshooting, et cetera, et cetera? They're not, because that would be impossible. No single person can know all that. And yet we want everybody to be selfsufficient, to self serve their needs. And that's the core of what we are trying to do in DevOps. We are trying to shift left to shift operations and everything that is happening on those right side of application lifecycle to the left. Now we cannot do that by saying, hey, now you do it. Here's AWS account. Create a cluster, create node groups, create vpcs, subnets and so on and so forth. Learn kubernetes, create manifests for deployments or stateful sets and virtual services and services and ingresses and so on and so forth. We cannot shift left by saying, now you do it. So the core, the essence of DevOps is about creating services that use ops tools but are meant to enable developers to be able. Application developers, just to be clear, because everybody's a developer, right? Everybody writes code. So to enable application developers to do whatever they need to do, you need a cluster, create it. There's a service for that. You need an application. Should it be back end or front end? This or that. Anyways, there's a service for that and so on and so forth. And that's how we are getting to the concept of internal developer platforms, or idps. They're internal because they are meant for internal usage. They are developer because application developers are the customers of those providing a platform. And it's a platform because that's how it all comes together. So IDP or internal developer platform is a layer on top of the technology and the tooling that we already have. It is about shifting left by enabling developers, application developers to be self sufficient by consume services of such a platform. That's what operations and SREs and DevOps and whatever other names we have for people on the right side of application lifecycle, that's what they're doing to enable developers to self serve themselves. And the reason why we want to do that, the reason why we want to create services is to free operations from doing mundane tasks. Kind of hey, there is an email, I should create a cluster for this team. Or there is another email or Jira ticket I should create manifest for application deployed here and there and so on and so forth. No, that's a waste of time. What is a good usage of time is to put all that knowledge into a platform and let everybody use it. Hey, this is how you create a cluster. Don't ask me. There's a button, you fill in those three fields or four fields, you get a cluster, you want application. Again, there is a form with a few fields, you need to specify whether it's this or that. Click a button, you get your application, and so on and so forth. And the reason why we want self service is actually there are quite a few reasons. We want to increase productivities, we want to reduce lean time, we want to increase deployment frequency, we want to decrease meantime to repair or MTTR, we want to decrease change failure rate or CFR, and so on and so forth. There are numerous studies that show that selfsufficient teams are performing better than teams, that depend on other teams who depend on other teams, and so on and so forth. And then we get to the question, what should such a platform do? And the simple answer is everything. Right? And there are certain stages, certain groups of operations that we need to think about. We need to be able to change those desired state. That's the state that is stored in git. We need to perform certain actions like building container images, running tests and so on and so forth. We need to converge the actual into the desired state. You know, if it's in git, that's what it should be. And we need to be able to observe what's going on and all that can look more or less like this, we have an IDP at the top, that's those interaction, that's how we interact with the whole system. And it's tailor made for the needs of a company. It contains a user interface which could be web or CLI or ID, I don't care whatever people are using. And it needs to have an API. And this is now the core of the issue, that API must be Kubernetes, simply because that's the only API that we have today that is widely accepted and has a tendency real potential to become a universal API. And then we have the desired state, that's what we're storing it. And then we have the actual state, that's AWS and Azure and on Prem and Google and Splunk and elastic and whatever else we are doing, those are the providers, and we have infrastructure on top of those providers which are servers or clusters or databases. And we have applications which could be ours or third party applications. So we have the desired state, we have the actual state, and we have internal developer platforms that enables people to observe those actual state to see what's going on and to be able to define what the state should be by pushing stuff to git in a way that is very easy to define and push as well. And then we have a Kubernetes cluster. Now don't think about Kubernetes cluster in this concept as a cluster where our applications are running. I mean our applications could be running in Kubernetes, but that's not the purpose here because our applications could be running a services like Lambda or Google Cloud functions, it could be running on vms, it could be this, it could be let's say a database managed database like RDS. In this concept I'm not talking about kubernetes where applications run. I'm talking about kubernetes where we can define custom resource definitions, define what something is, because we already established that things are too complicated for an average developer. Defining all the resources for an application Kubernetes is too complex for people who are not dedicated to kubernetes to understand defining all the resources in AWS, again too complex. We need to simplify that and we simplify that by creating new custom resource definitions. Hey, in my company this is what it means to run a cluster. And I'll show you that in a second. And this is what it means to run an application. Again, I will show you that in a second. And those CRDs have associated custom resources. Those are the manifests that we store in git that are creating into custom resources. And the goal of those CRDs is to define what something can be. The goal of CRS custom resources is to define what something is. And the goal of kubernetes and controllers in kubernetes is to convert those custom resources into something which is the actual state, which could be in our providers, infrastructure, applications and so on and so forth. This is what unites it all. And to do that we need tools, we need pipelines, we need GitHub tools. That's the tools like ArgosDM flux that synchronizes desired state into what is in Kubernetes. And Kubernetes synchronizes what is in kubernetes with the actual state. And we need infrastructure and we need Arbec and a bunch of other things. And I will show you all those things in a second. Actually, not all those things, but some of those things in a demo, because I get nervous speaking for too long without touching my keyboard. Actually. Now I'm going to think about it. Why not go into demos right away? I had five, six, seven slides prepared to talk about it, but that's boring. So let's go straight into those demo and let me show you how all that looks like, how we can combine couple of tools to do just that, and it's not going to be hard, so bear with me. So let's say that I'm a developer and I want to have a cluster in somewhere. Let's say aws. And that cluster should have all the things that I do not really understand, but it should be a cluster, kubernetes cluster. Eks cluster. How would I do that? How would I do that to the level that is easy for me to define and consume it as a service. And by the way, that cluster should not be just a cluster, it should be production ready cluster, meaning that it should not be only an eks cluster with node groups and kubernetes and vpcs and so on and so forth. But everything that I need inside of that cluster already baked in. And I could do that in a very easy way by showing you this file. Infrawks. Here we go. And this file is relatively simple. Bear in mind, if I would do this without the way how I'm doing it right now, which is by the way, crossplane, I would probably end up with 500 lines of yaml or hcl or something like that. But in this case it's much easier because I'm saying, hey, I want to create or I want to define something called composite cluster. Now what is composite cluster, well, that's what other people defined and already prepared. In my cluster there is a custom resource definition, or what we call in crossplane XRD or crossplane resource definition that is called composite cluster. And I can just say, hey, I want to consume that thing and I will give it an id and I will say, hey, this should be an AWS cluster. And that's where we are going into compositions because this custom resource can have many different compositions and I can choose, hey, do I want in Google? Do I want in AWS? Where do I want? And finally, I have a couple of parameters. I'm saying, you know what, I do not know what is t two something something or whatever are the sizes in AWS and Azure and Google. I just want the nodes of that cluster to be medium size, whatever that means. And I want to have a minimum of three nodes because hey, you should have a minimum. And I assume that it will auto scale up, right? We are defining something that doesn't exist in crossplane. It's what we call crossplane resource definitions, which are basically base for operations in my company, or sres or DevOps or whatever they're called to use to combine different resources. And I will show you those resources later into something that is meaningful to everybody else. In this case me and I have a second xrd called Gitops because I want this cluster to be Gitops ready and I do not want really to install argo CD. I do not want to create argo CD applications and projects. I do not want to configure git repository myself and so on and so forth. So I'm using yet another custom resource definition that was created by others. It's called Gitops. And I'm saying, hey, you know what? I want to use Argo CD. I could have other implementations of this like flux, but in this case argo CD and I wanted to run in a cluster that is identifiable through this secret and secret namespace, which happens to be the one created by this guy over here. And most important, this is what really matters. Hey, I do not want to go into details. What I do want to do is to tell you that those is the repository you should monitor and make that cluster the one defined above. GitHub's ready. Now all I have to do is push this manifest to git repository. Argo CD will pick it up, it will create whatever needs to be created in the management cluster, and then crossplane will do everything else. But I already did that in advance because it takes 20 minutes, 30 minutes or something like that. So let me show you the result of me pushing that to git. I did that in advance. And the result from operational perspective can be this get managed. And then we can see that behind that simple definition many things happened. And I got all those role policy attachments and roles and different objects installed in that cluster and different helm charts installed in that cluster, and a node group and a cluster and subnets and all those other things that are really necessary. But I do not want to bother with somebody else create a composition that combines all those resources and exposes them through that composition that I showed you a minute ago. Now that was from development perspective, from operational perspective, from SRE perspective, there are a couple of things that I had to create, somebody else had to create in advance. And the first one is definition saying, hey, I want to create something completely new, completely new definition. And that definition will be called composite resource definition. That does not matter. What matters is that the name from user perspective will be composite cluster. That's the one I used a second ago. And there is open API schema that creates a completely new definition. It's a typical open API schema, nothing really special. It says hey, you should have an id and you should have some parameters, and within those parameters you should have version and node size and minimum number of nodes. And that's what crossplane took to create that new custom resource definition that I used at the very beginning of this demo. And then I need to create a couple of implementations of that. Something I will show you eks, even though I have the implementation of that same resource definition for Google and for Azure and so on and so forth. But eks is like this. Here we go. This is a composition. So one composite resource definition can have many compositions. So we can pick and choose which variation we want. And in this case it is composition. That is called cluster AWS. That's the one I used. And here we have eks cluster defined, and we have a bunch of parameters here taken from those user and node group, and again with some parameters hard coded and some parameters provided by users. Like in this case I'm saying, hey, whenever somebody writes in their manifest parameters, node size, that should be translated to instance types, because that's what AWS understands. And whenever somebody says small, it should be t three small, and whenever somebody says medium, it should be this and large should be this and so on and so forth. I have here what else? Role and another role and role, attachments and so on and so forth. Those are all the things that you need to do. No matter which tool you're using if you want to be in control. But we can hide this from end users, from application DevOps, all those things. And it's not only about AWS over here. I'm also deploying some helm charts and kubernetes resources and so on and so forth, everything that is needed to run a cluster production ready, cluster in production. And I did those same thing with Githubs. The variations, whether it's flux, whether it's argo CD, what happens when argo CD is installed? How do we create this? How do we create that? And from the user perspective, all those things, from application developer perspective, all those things are digested into something that I showed you at the very beginning. I do not need to worry about those things. I know that this is what operators in my company, sres did. I can just consume those services by defining penetrate lines of yaml, like here. Now let's verify that the newly created cluster really works. And remember I pushed it to git and I'm not going to interact directly with my kubernetes cluster at all. It's all githubs only by pushing to git. But first let me double check that I can connect to the new cluster and I'm going to do that. You can ignore this is fetching the secrets stored in the management cluster. What does matter here is that I can confirm that the new cluster is up and running by saying hey, get all the nodes, for example. Let's go, come on. And were we go, this is the newly created cluster, right? With everything inside and among other things it already has argo CD up and running and I can prove it to you by opening those port. I should have created those domain for that, but I didn't, I was too lazy I guess. So here we are, this is Argo CD already baked into that cluster and I can prove it to you. Admin admin one, two, three, there we go, it's running. I'm not going to save the password and it's empty. Nothing there going on, right? Nothing here. And if I go here, nothing here, right? Because I did not yet push any application manifest to my git repository, but I will do that right away. So I want to have an application running in this cluster. And let's say that it should be a backend application with database. And let's say that that database should be AWs RDS database. How would they do that? And again, that would be very complicated because if I would like to do that myself, I would need to define stateful set or deployment. I'm not sure because hey, Kubernetes is complicated a service, ingress virtual service, this and that. And then I would need to define VPC and subnets and what's or not for RDS database and RDS database itself. Create some users in that database, choose this or that, it would be complete. But the operations of my company, the SRE or DevOps already created a service for that. Remember this is crossplane now still and I can show you that service. I can define a new application based on that service which will be this one, and says hey, instead of all that nonsense, I'm going to define something called app, operator, already created crossplane composition that is called app. And I'm going to just say that I want to be a backend with a database. This is the image that I care about. This is the those it should be exposed to and this is the port that the application is listening to. And I want also something called SQL. And that SQL should be in AWS and PostgreSQL and it could have a couple of parameters and it should write the connection string to this secret so that this application can connect to that database through that secret to authenticate there. Right again, I'm reducing hundreds and hundreds of lines of YAML or HCL or whatever you're using to a few lines, 30 lines of Yaml. So what should I do? Well, I should just copy this file from examples and then what was it? Backend app, backend backend AWS, PostgreSQL and no claim. That's the one I'm going to copy to the directory apps. That's the directory in repository that ArgoCd is already configured to watch. And I'm going to add that file and I'm going to commit with message that says something and I'm going to push that to git. And that's it. That's all I need to do to have my application up and running in the newly created cluster. And that application is very complex. There are many moving pieces, many resources, but I'm using a service provided by others. So let's see whether it's working by opening it in Argo CD. Here we are. Is it working? Let's see. There we go. It's coming right? It found two manifests, app and sql. And so far it's not doing anything with them. But look at it. This is now crossplane expanding those compositions into some objects and provider configs and DB subnet groups and RDS instance and Internet gateway and so on and so forth. All those things could need to be created by me, but I don't have to do that because somebody created a service that I can consume and that service is called. One service is called sql, another service is called app. And I can just define what I want. Those let the details and the complexity into the hands of whoever is creating those crossplane compositions and just push simple YAML into a git repository and let Argo CD get it to the management cluster and let Crossplane do the rest. Actually, it's not going to the management cluster. It's going directly with Argo CD from git to the cluster that I created before I started those demo. And I showed you already how. And if you don't believe me that all those things are happening, I can prove it to you. I can go to that newly created cluster, say, hey, this is the kubeconfig that I fetched before, and get all the apps and sqls in that cluster. And there we go. You see, they're not yet ready. Application is ready, but database is not ready because it takes like ten minutes, I think for AWS to create airds instance. And I can also do something similar with get managed all the resources managed by crossplane. I would not do that as a developer because for me this is what really matters. But as operator or SRE, I can see all the details. What did Crossplane create from those two? And this is what it did. Those are all the resources that it created for us. Here we are. All those two were expanded into those RDs instance in AWS, DB subnet group. Some objects creating in Kubernetes cluster like deployment, egress service, Internet gateway, Kubernetes, VPC, security groups, route tables. Right. All the complete is now moved to the cluster. It is grouped into compositions so operations, accessories, DevOps care about creating those definitions and controllers behind it. It's relatively easy, as you saw, and then exposing them through a simple interface to everybody else so that everybody can consume without dealing with such a complexity. Thank you so much for watching. My name is Victor. Let me show a couple of slides. I didn't show many slides. I work for outbound and what you saw in this demo was very brief, but probably gave you a glimpse of what you can do with Crossplane to create your own platform for your own people in your own company. And that is called Crossplane. And the company behind Crossplane, actually the company that founded Crossplane is upbound. I work there, but Crossplane is open source. It was donated to cloud Native Computing foundation, CNCF, so you can use it freely, at least in this concept. You might want to check what we are doing at upbound, but Crossplane is open source. You might also want to listen to the podcast Devil's paradox. You might want to come to my YouTube channel, DevOps Toolkit. And that's about it. Thank you so much for watching, for listening. See you next time in the next conference or in my YouTube channel. Come there. Thank you so much.