Welcome to this talk about Kubernetes manifests and everything you can do with them that you weren't aware of. Going to walk you through a bunch of things that are important to understand and know when you're working with kubernetes and manifests when you're just getting started, really a lot of things to know. I'm going to be showing do a little bit of a demo as well. But first of all, just going through 101, what is a manifests? Talk about the manifest lifecycle, talk about how to create manifests and validate them. Talk about templating tools and just tooling in general and some best practices at the end to round things off. Hope that will be valuable to you. I have with me the ever lovable Kubernetes mascots here to the right. Say hello to them and let's jump on straight in. So first of all, what are Kubernetes manifests? And if you've worked with kubernetes, I'm sure you know what they are. If you haven't, now you know basically kubernetes. Any object that you want to create in your Kubernetes clusters, you have to specify a manifest that defines the state of that object. These manifests are either in JsON or YAml format. A file can contain multiple objects defined, and once you've defined the object, you apply it to your Kubernetes cluster and then kubernetes will do all its magic and create those corresponding objects for you. So straightforward, a manifest is a YAml or JSON file. Can work with any editors of course that can work with those kind of files. Basic structure of a manifest as you can see here, very straightforward. This is a YAmL example. And generally people mostly use Yaml. Although JSON is valid, every manifest has to have a kind. So telling what kind of object we're creating, the version of the kind. So this is a v one service that I'm creating in this manifest, and then some metadata and the name of the object is required and the namespace is optional. And the name and namespace have to be unique in combination. So you can only create one service with the name pet store. For example, in the default namespace you could have a pet store service in another namespace, but you won't be able to have two pet stores in the same. So good to be aware of. And then you can also add other metadata. You don't see them in here like labels and annotations commonly added by tooling like helm or other tools that you might use can also be helpful, just for your own sake, to organize your objects when they're in your cluster. And once you've kind of passed that initial block of Yaml, coming to the actual meat of the Yaml file or the manifest is what's called the spec or the specification, it could be also data in some objects. This specifies the actual properties for the objects that I want to create. So here you can see this is a service listening on port 80 and targeting port 80 80 using the name of the port is HTTP and then it's a selector telling to which other objects this service is applied. So obviously the content in the spec can be varies a lot between different kinds of services and somewhat between versions, and can be really short like you can see here, but it can also be super super long and elaborate and complicated, depending totally on the kind of object that your manifest is describing. Important to understand, API version and kind, and since those are used to kind of define the schema or the content of the manifests, as we already mentioned. So based on the kind and version, that's where you know kind of which properties and maps and arrays and enumerations, all things you can use to define the state of your resource. And the actual schema languages used underneath to define and validate those properties or specs is derived from OpenapI 30. So it's JSON schema with some limitations. You can read about this on Kubernetes IO. Obviously one thing to be really aware of here is that this shouldn't be confused with the version of Kubernetes itself, right? So currently I think Kubernetes is at version 125, or could it be 124? I don't remember. That version supports multiple versions of the actual object. So for example the crone job kind, you can run either API version, version one, or version one beta one, which is an older version, but you need to keep track of this. Of course if you're using V one beta one for your chrome jobs and you're about to update to Kubernetes 125 that no longer supports that version. So you needed to be on top of these versions and obviously always try to use the latest version. But of course if you're targeting for example a cloud provider, make sure that they support the Kubernetes version that you're using with the objects that you're creating. So always good to be on the safe side and make sure you've got that yourself. That's all covered. Obviously many of these objects in Kubernetes don't live by themselves, right. They refer cto other objects. It's very common with kind of references between objects. So three most common types of references are just by name. So for example here you can see a config map ref. So this is referring to a config map with this name. Another more complex reference is what's called an object reference. So this not only by name but also by kind and API group. So this role ref here could be referring to other kinds of objects in other API groups, whereas this ref up here is always locked to a config map with the current version. Selectors are also very common. We saw a service previously, here's another one. This service applies to any object whats has this label defined. So Kubernetes IO name with this value. So this is a way of applying or creating references of one to many, right? So this service would apply to any object having these labels. So it's a powerful way of kind of referencing more than one object and creating more complex dependencies between objects. Also good to be aware of, obviously the status of your object. So when you're working with your object locally, it doesn't have a status because it's not deployed in Kubernetes. And as soon as you deploy your manifests to Kubernetes, Kubernetes itself is going to add a status property to your yamls or to the manifest. And I'm going to show this in a little while, kind of showing you the current state of the object and any events kind of related to the object in your clusters. So good to know that if you kind of look at the manifest that you have locally, maybe that you created and then the manifests that's actually running inside your cluster, you're going to see some differences there just because Kubernetes could apply some transformations and add some additional metadata, but it'll also add status metadata that kind of lets you know, kind of keep track of the projects. If you were to look at the manifest itself and this is not added for all object types, once again you have to play around or read the documentation just to figure out kind of what values and properties are added for which types of objects. So trying to kind of drill this into a lifecycle, because we all love lifecycles, is usually you start by creating your manifests using some tools or get back to tooling. Later. You edit them, you do validation. I'll get back to that soon. And once you're kind of happy with what you've created, you apply the manifest files to your Kubernetes cluster. And then what Kubernetes does is it takes the manifest and then it tries to, using its internal objects and logic, which I'm not going to go into, create the corresponding object in your cluster and then, in line with how you've defined it, and then there's a loop that can happen in cluster. So let's say the object isn't working as you desired, or you want to make some changes. You can do that straight directly by modifying the manifests in the cluster, and then Kubernetes will kind of update the state of the object accordingly. You can also, if you're a little bit more structured, let's say you made a mistake, whatever mistake you might make, that change out here to the left, and then reapply the manifest to your cluster, which makes for a little bit more of an auditable workflow lifecycle. But usually hot fixing is something that you'll end up doing, at least initially while you're getting to grips with Kubernetes. And at the end you'll delete the object from, you might at least delete it from kubernetes, and then of course the manifest will be removed as well. Not a super complex lifecycle, but good to understand the relationship between your local files containing the Yaml Json and then the actual object generated and created inside the cluster when you apply those files to Kubernetes. So creating can editing manifest is a straightforward thing, right? You can use any editors, obviously you can copy paste from other manifests, which of course is maybe not the best way to do it well, but it's a very productive way of getting things started, and there's a lot of examples out there, so it's a good way to get started. And there's a lot of plugins and code snippets and generators and wizards and stuff online and as plugins to your ides to help you getting started with your manifest. The easiest way maybe is to use Kubectl, which I'm guessing or hoping you're aware of. Kubectl is the command line tool for kubernetes, and there's a simple command that you can use to generate basic manifests for different types of objects. I'm going to show you this in just a little while and walk you through kind of the whole thing just to see how that works. Finally. Not finally, but let's talk a little bit about validating. Super important to validate your manifests, obviously syntax validation. You want to make sure that your manifests are valid YamL or JSON schema validation. You want to make sure that you've defined the right properties, and the properties have values that are in line with the type of properties that they are, and otherwise, obviously Kubernetes is not going to be able to handle the manifest when you apply it. Even though it might be a valid yaml, it might not be a valid schema. Another step of validation is around links. So as you saw as we talked about earlier, super important that if you're referring between objects, that those links are all lined up, that you're referring to the right name. Sometimes you might be referring to objects in other namespaces. You need to make sure that those objects are also there. And this can be tricky because just like I said here, the objects that you're referring to might not actually exist locally in your files. They might already be running in your cluster, so it might be hard to validate these kind of references. Also, some references, as you can see here, can be optional. This is depending on the schema of your object. So for example, there's a secret reference here which is not optional, so this actually has to exist. Others references can be optional so you won't see any errors. You might get a warning, at least depending on the tooling you're using, if that reference is not satisfied. But once again, definitely something to check. And this is a common cause of spending a lot of time figuring out why things didn't work. And just because you might have misspelled some link reference to some config map and it can take a while to figure whats out. The error messages you get from kubernetes are not always super helpful and it can take a while to figure out why these things didn't work. So if you can use some tool initially to validate these references, that can help you save a lot of time. Then finally policy validation policies are another kind of rule set on top of the first three here, and this can be policies related to performance, to resource allocation, to security, networking, et cetera. Or like just local. You might have policies defined by your teams around naming, around labeling, around things that are applicable more to your project than at a kubernetes level or general level. And a common technology used to validate manifest is called OPA open policy agent. It's used by a lot of open source projects out there to validate kubernetes configurations, and these policies can be applied either before you deploy. Right? So in your editor, which is the best place to do it, because that'll save you most of the time fixing those issues. But it's also pretty common to have some kind of gate or validation in kubernetes that just makes sure that you're not deploying things that don't align with your policies, which is still super helpful. Right. But it's a little bit late in the process because then you'll have to kind of go back and make changes and redeploy, et cetera, et cetera. OPA uses a language called rego, which is straightforward. It's not a super complex. Oops, sorry about that construct. And as I said, this could be used to validate anything related to resource, right? Name networking, resource allocation, best practices, et cetera. I'm going to show you in just a little while how what these policies can kind of look like, and there's a lot of them available online on GitHub. And there's also a nice vs code plugin available if you want to create your own and debug your own policies so you can really piggyback off what a lot of the existing open source tools have done, but then adapt those policies to your needs that you might have in your teams. Okay, finally you've created your manifest, you've validated them, and now you're ready to deploy. And usually deploying is something you might initially do manually. Right. You're using Kubectl or helm or customize. Eventually you're going to look into maybe automating that using CI CD or even a Gitops approach. Gitops meaning that you manage all your manifests in git, and then you have a tool like Argo CD or flux that constantly makes sure that the state that you've defined in git is actually the state that your cluster has, which is a super nice approach, but it's a bit of a maturity curve to get there. It requires that everyone's on board, so maybe that's not where you should start, but it's definitely a great endpoint to aim at. Okay, enough talk. Let's have a quick look. I'm going to use a couple of tools just to demo a little bit here. So I'm going to start by just using a Kubectl command to create a very simple NgInx deployment. And I'm going to use a tool called demonical, which is a free open source tool to have a look at that. So you can see here created this conf 42 yaml. It contains one deployment. Let's hide the file explorer. This whats the yaml that was created. You can immediately see here that there are some errors by the generated in the generated yaml. So let's just remove those because null is not a valid value by the schema. So this was the schema validation kicking in. And you can see here there's a reference to an image in Nginx image. You can also see that by default Kubectl adds an empty status property which will then be populated by kubernetes when we deploy, which we're going to do in just a little while. But I did want to show you a little bit around OPA validation. So let's go over here and let's just start enabling some OPA policies. So for example here a process can elevate its own privileges. I've enabled that OPA policy and you can see immediately here that we got an error. Let's hide this because it's a little bit crowded. So you can see the error message saying that this process can elevate its own privileges. And there's actually a recommendation that we should set security context, allow privilege escalation to false in our containers. So let's see if we can do that. Sorry, was that security? Sorry, I don't remember continuous security concepts. Let's set that to false. And now you can see that the error went away and now I can. Let's see if there's more. Let's try. Default capability is not dropped. Oops, we got another error. So we can kind of use this kind of tooling to interactively find those OPA policy validations that we're interested in and fix those before we deploy anything to our cluster. But I am going to. Oops, let's just turn that validation off again so we don't turn them all off to avoid any errors. And let's deploy this. I have a local mini cube cluster running on my machine, just going to press here, deploy. Let's put this in the default namespace and press. Ok so it's been applied to my cluster and so let's have a look at the cluster itself. So you can see here is the actual object, the NgINX deployment. Let's just filter on the default namespace. So this was the object that was applied to Kubernetes, my local mini cube cluster. And as you can see there's a lot more in here now than the initial manifest that you saw. Kubernetes has added a lot of different things. You can see that this property that we set to false is here, but it's also added the status property that I mentioned earlier on. So this is kind of what our object now looks like when it's running in Kubernetes. While obviously very different from the yaml that we had defined outside Kubernetes that we can see here. Okay, great. So we were able to create the manifest, we did some validations, fixed some OPA validation errors, deployed it to Kubernetes, and then connected to our cluster and saw that everything seemed to be up and running. All good. Let's get back here. Next step is to talk a little bit about templates. Templating is a very common need that you'll run into when you start working with multiple clusters or target environments. So let's say you have a dev environment, a staging environment, production environment, and when you deploy your different applications, you'll want to configure them somewhat differently, right? You might want to have different scalability settings, different security settings, different network settings for the same kind of source of manifests. And there's a lot of tools to help you do that. So have one source of manifests and then either kind of parameterize those or kustomize them or whatever to make sure whats, when you actually deploy to different environments, you have the values that are applicable to that environment. So yaml, there's the yaml native approach which is customized. I'm going to show that in a while it's built into Kubectl, so it's a very native way of doing things. Helm and JSON at helm is super popular, which uses a more custom templates language based on go templates. There's a couple of frameworks to actually instead of writing Yaml, you write code and then that code generates your yamls for you. Similar to like terraform would generate infrastructure. You can use things like cdks or decorate to write code to generate manifests. And then of course you can use all code related constructs that you might be happy to use when you're writing code. And there's also abstraction layers like Acorn and others to help you to kind of provide a higher level of abstraction that would then get converted of course to manifests before they are applied to kubernetes. So ultimately all of these result in a YAML or JSON manifest that is applied to kubernetes. They just help you do those in different ways. And whatever works for you and your team and what you think is best is always a good start. Talk a little bit about kustomize. Kustomize basically works by asking you to define the set of base manifests like the one that we looked at and then defining overlays or patches specific for a target environment. So I might have a base set up a deployment for base setup, and then I might have overlays for dev and production environments where I might have different replica accounts or different security settings or namespaces or whatever. And then kustomize when you apply to your cluster customize, you'll tell customize, okay, I want to use the production overlay. So it'll overlay the configuration in production over your base configuration and then take the result and apply that to your clusters. This is a very nice approach. It's Yaml native, so any Yaml tooling will work with it and it scales fairly well. Depends on your needs. Helm I'll mention that as well because it's so widely used and popular for good reasons. Helm uses a more custom templating language based on go templates and then packages those into what's called a helm chart. And these helm charts can contain not just templates, but also tests and a bunch of things that kind of turn into a package for an application. So helm is a really nice way of kind of packaging your entire application and then helping others install that application and then using what's called a values file, which once again is a yaml file as input, you can parameterize those helm templates. So a user would generally use your helm chart, create their own values file where they might specify custom ports or custom namespaces or whatever other ways you might allow your helm chart to be configuration, and then they apply those value files to your helm chart. And then helm can do its magic and take the output and apply that to kubernetes. And these helm charts are distributed through helm repositories, and there's a bunch of them out there. So if you look at artifactory or just search for helm repositories, you'll find a lot and you'll find that there are helm charts for many common infrastructure tools like databases and web gateways and game. Even Minecraft has its own helm chart. I'd like to mention a really neat practice that's been coming up lately and is about using helm and kustomize together. The workflow here is that you would use helm to package your application and then use customize to overlay any runtime configurations, right, because specific CTO your environment, because the creators of the helm chart obviously are not always aware of how you're going to deploy that helm chart. And you might have very specific needs around once again, security, networking, et cetera, which those might not be configurable in the helm chart itself. So what you would do is use helm as an initial phase and then use customize to overlay any runtime specific projects over the output of helm. Sounds complicated and it is complicated, I'll grant you that. But it's a very powerful concept because it really makes the most of both the tools what they're really good at. So definitely something exploring as you kind of venture deeper into the depths of kubernetes, deployments and applications. How do these tools fit into the lifecycle that we managed earlier? Let's just look at that here. Now, instead of jumping straight from edit to validate, there's a phase where you will want to preview, meaning that if you're using helm or customize or any of the other tools, you'll want to run those tools and get the output of them, which is what would be applied to your cluster and then run your validations on that. So you would want to validate the output of customizer helm and make sure that all the OPA policies and anything else are in place. You might also want to validate pre preview of course, but really important to validate what's actually going to be applied to your cluster and not what's kind of input to another tool, whats does something before it applies to your cluster. And I wanted to show that quickly what that looks like. Let's jump back to Monaco. I'm going to go to a Kubernetes example repository which has a customized example. This is just filter on that and have a look here. So here we can see, we start with the file Explorer. You can see there's a base folder here with a deployment, an Nginx deployment and an NgInx service. As you can see here, you can see also links between them marked out in here. And then there are overlays for dev and prod. And if we look at the overlays themselves, you can see here that the dev overlay sets the replicas to one, the production overlay sets it to five, and the excitement, the staging overlay sets it to three. So what this basically means that if I take the dev customization and preview that, so what that means is now we're looking at the output of customize for that customization. You can see that replicas here is set to one. Let's exit and let's have a look at the staging customization and preview that. And then as you can see here, replicas is now set to three. Right. And just for sake of getting it all going through everything, now I'm going to preview the prod customization, and we will hopefully see that replicas is set to five. Right. Ultimately, though, the deployment itself by default actually had replicas set to two. So if I wouldn't have used any of these, I would have gotten replicas of two. But this is a very basic example of showing how you can use customize to set different property values for different target environments. In this case, replicas for my deployment. Okay, that was a really quick demo because we are really a little bit short on time, just quick on tooling. There's a lot of tools out there. Just to kind of break this down a little bit. Here's the manifest lifecycle. We talked about these different phases and then the object lifecycle in your clusters. Kubectl is great for creating manifests and for deploying them. It doesn't really have any support for editing, previewing, validating. Of course, then it's great for updating and deleting projects running in your cluster. So that's where it really shines. Next step here would be ides and plugins. So if you're using visual studio code or intellij, et cetera, the core ide doesn't do much. Right. You're very reliant on plugins, and that's where I've put parentheses. So if you kind of cobble together a bunch of plugins, ids can do most of these things for you, right? They can create, obviously edit. That's what they're good at. Validation plugins. There's plugins for deploying, there's plugins for doing all these different things. The challenge here is that the plugins will maybe not always be very compatible with each other or up to date, et cetera, et cetera. And we're having some transition problems here. Not an issue. Validation tools obviously are great at validating. Templating tools like helm and Kustomize can help you create. They obviously do have support for previewing and applying to your cluster, and creating and updating and deleting what's already running in your cluster. You might be familiar with tools like lens or k nine s or other cluster management tools, great tools. They're very focused on working with Kubernetes itself. So if you're connecting to a clusters, you want to see what's running inside that cluster. They're not really geared at creating manifests, editing them, previewing or validating them. They have some support, maybe for applying helm charts, for example. But a big gap if you're the person that's authoring the pre deployment phase of your applications. And then there's a more niche set of tools, Monaco, which I was showing earlier. You can call them Kubernetes ides or configuration management tools or whatever, but I think the pros of these tools is that they try to support the entire lifecycle in a very productive and integrated way, right? So instead of having to cobble together all these things using more manual tools and Clis, you can just use a tool like Monocle, which is basically a wrapper for all these common tools that you're already using with a nice UI. But once again, do what works best for you and what feels most productive. Not trying CTO say anyone is better worse than another. It's all up to you and the team. Finally, some best practices. First thing, understand manifests I hope this talk helped you do exactly that. Always try to use the latest stable API version for your manifest, obviously in sync with the cluster that you're working with targeting. Keep them simple. Don't add a bunch of default values unnecessarily. It's just going to be confusing for people to work with them. Policies talked about that. Try to use policies. You might not even use a tool to validate policies, just agreeing and discussing. Having a meeting about howto work manifests and some policies around naming is super helpful and creating a shared common knowledge around how people are going to work with manifests. Use templating tools, customize helm, whatever else, whatever works. Don't use them if you don't need them because they do add a level of complexity. Validate manifests before you deploy I know there are great tools for doing in cluster validation, and many CI CD tools can validate for you. From my point of view, that's a little bit too late. You shouldn't be deploying things that have errors, trying to deploy things that have errors in them. Try to deploy as early on in your process, similar to if you're using linting for your source code. That's something you do in your ide, and preferably not only as part of CI process. That being said, you should automate validation as well, right? So you want to have the automated as more as a safety check, but try to do it early on because it'll save you a lot of time. Then finally, I really didn't talk a lot about Gitops, something I urge you CTO adopt when you've come for a new journey. If you've come through all these first seven bullets and you're happy with that, then it's probably a good time to look at. Start using Gitops. Gitops is similar to maybe like adopting agile. It's not something that just one person does. The entire team has to do it, and it does impose some constraints on your workflows, so be aware of those talk within your team so everyone's on board and go for it because it's really awesome. And that's it. All done here. Thank you so much for listening. Hopefully this was helpful. Please reach out to me at this email address or on discord or you can obviously find me in any monocle related assets online. Thank you so much for listening.