Hello, my name is Chris Ayers and today I'm going to talk about CI CD with GitHub Actions. I'm a senior customer engineer at Microsoft and I work on the fast track for Azure team. We're part of Azure engineering and we help customers build and deploy software to Azure securely, reliably. I have a big focus in DevOps and what I do, I'm on Mastodon, Twitter, LinkedIn. I have a blog and all of my code samples are out on GitHub, so feel free to go out to my repo and take a look and I'll be sharing that. So I first want to talk about YaML and then we'll dive into CI CD, go into some of the discussion around actions, and then I want to show some demos and have a fun time playing with those. So GitHub actions are part of workflows and those are all written in Yaml, yet another markup language. And you have lists that can start with a dash. We have key value pairs that are separated by a colon in a space. Then we have objects which are very similar to key value pairs, but all of the properties are below the object and indented. And I'll show some examples of that shortly. I just want to make sure that those three kind of basic things are covered. And something that's very important is if you get your indentation wrong or your spacing wrong, it can cause problems with your workflow. So having good tooling like Vs code with the YAMl plugins and extensions can be really helpful in this. And another thing is I like to talk about, we have CI CD. Well, there's two types of CD that come into play, and you can do all of these things with GitHub actions through the workflows. Like you can run your builds, you can run your unit tests, you can deploy things out, and there are ways to stop for approvals and see if you want to proceed. Maybe you deploy to dev and you want someone to sign off before it goes to QA, or you deploying to QA and you want some approvals or time windows before things go to prod. You can do some of that. But usually I think some of the differences between continuous delivery and continuous deployment are youll do have push button deployment, but things slow down and you have a human in the loop that can approve or not approve. Now, actions are event driven, so there's a large number of events that can trigger if a workflow runs. All GitHub actions workflows live in the GitHub workflows folder and they're all defined in Yaml. Now what's different about GitHub actions from maybe something like an Azure pipelines is in Azure pipelines you have to specifically say I want a new pipeline using this file. But with GitHub, if youll drop a Yaml inside that GitHub workflows file, and it's a valid workflows YAml, it will start running whenever it's supposed to be triggered. Now if that's a manual trigger, if it's on a push, if it's on a schedule, as soon as you put the file in the repo and it's a valid yaml that matches the schema, it'll be run whenever it's supposed to be triggered. Now, events trigger workflows, and I'm going to show you a big list of events. I'll probably bring it up in a moment. But a workflow contains one or more jobs. So we have a job. When the workflow gets triggered, a job is going to be kicked off and that job can have one or more steps. Those can be like command line steps, like a bash script, they can be actions, they can be shell commands, it can be a number of different things. Now, each of those jobs runs on a runner, and like I said, you youll have one or more jobs, and those jobs by default run in parallel. There's no dependencies between them, they're just jobs. If you have like five or ten jobs, they could all try to kick off right at the same time, unless you have some explicit dependencies between them or you have some needs between them. Now, each job is using to run on its own runner and what a runner is, and I'll talk about those in a little bit more depth. It's like a vm or a docker container, something that is spun up for the purposes of running your job. Now, if it's using a GitHub hosted runner, which I'll show you some documentation about, it comes with a ton of tools. If it's a self hosted runner that you built and are running yourself, it is going to have whatever tools you put on it. So let's get into some demos and just see what we're talking about. This is where all of my demos live out on GitHub. But I wanted to talk a little bit first about those events, those actions. So in GitHub we talked about events that trigger workflows. So like I said, there's a workflow, there's a ton of different events that can trigger stuff like someone tried to push something against main or they created, we can go right here, they created a branch or a tag. You can see what happens if someone forks your code. That can be a trigger. So triggers are not just limited or the events are not just limited to things like pushing or pulling code. It can go far, far beyond that. Like someone created a project or moved a card or made a comment, or you want to do it on a schedule. So you want like a nightly build. This one is one of my favorites. It's super important. Workflow dispatch out of the box. There's no manual way to trigger a workflow. You have to add workflow dispatch if you want to manually trigger a workflow. So let's take a look at one. So I've got up here in my GitHub actions demos, you can come here to actions and you can see I have a whole bunch of different workflows listed here. So if we just start on this first one, I can take a look at it and I can open up what a yaml file looks like. In this particular case, this workflow. So we've got a name and you can see I've got on. So these are those events that are going to trigger the workflow to execute. So on a push, on a pull request, and on workflow dispatch, so I can manually trigger it. And then I'm going to list out my jobs. So I'm going to have one or more jobs. And in this case I'm going to name my job build. So I have a build job that's indented under jobs and I have to tell you what it runs on. It needs an image for this job to run on. So this one is going to run on Ubuntu latest, and I've given it some steps, like it's going to check out the code base and it's going to run an echo command. Well, maybe I want to build net or maybe I want to build Java or go or something. Do I need to go and set up all those tools and configure them? Well, I don't, because like I said, they already exist on a lot of the runners. So if I just do another quick search for hosted runners, again, great documentation comes right to it. And I can see kind of an overview of what I'm getting. I've got my workflow, it's running my jobs. This is giving you an example of different ways to specify if you want to run on Windows or Linux, but out of the box you pretty much get like a two core cpu, seven gigs of ram, some space, and there's lots of different tags for different versions, even Mac if you need to do like a mobile app or a Mac application. But trying to understand what's installed on it, I can look at my runner history and I can see what's on that machine, or I can also come here and see what's available. So on my run, if I look at my build, I can see the log for it and I can expand out setting up the job. I can see the operating system, it was an Ubuntu 20 machine, and I have a runner image and there is a link right here for my build on the included software. Now this has so much stuff, it's kind of funny how many languages and tools and packages are available out of the box like Docker and Kubectl NPM. We have CLI tools for pretty much every cloud out there. So right out of the box, using one of these hosted agents, I can run and do lots of things very quickly across a wide range of languages, and I don't have to do a lot of configuration to do it and I can get status. So let's go back to our actions. We had a basic trigger. I showed that we might need the workflow dispatch event, so if I want to run this, I can say run. And that enables the button here so I can manually run my workflow whenever I want to. So I'm triggering this run and what we'll see momentarily. There we go. It's manually run. It's queued up right now. I can go in and see kind of in real time as it executes, so I can get real time logs, I can get the output of individual commands. Cool. Well what about doing something a little bit more complex? Well I do have a more complex job, but let's say I don't want to build it on every push. I can have filters, I can do limitations, so I only want it to build on the main branch when I push on the main branch, or if I use a pattern of release something so the starstar will do things under that path. Maybe I want to only build it on v two tags or using star. I only want to build it when I touch Javascript files. So I have a lot of different options for controlling when things run. I youll do a schedule like a cron job, so I can specify cron syntax, so I have a lot of different capabilities. And just like with all the others, I can come out here and look at GitHub action workflow syntax and this is going to give me a detailed breakdown of all of the structures of the workflow. Yaml so I can see on pull request on branches, including branches, excluding branches using the not symbol. It's all very well documented out on the GitHub documentation page, so we have a lot of different options to build and record. Let's see what else we got. So what about multiple jobs? And I will go ahead and kick this one off and we'll take a look at it when it triggers. So I mentioned a workflow is made up of one or more jobs. I also said they all kind of run in parallel. So here I've got not one but two, three jobs, and job three finished first, so we had no real idea what was going to go when they all ran in parallel. And what that looks like if we're looking at a job, is I've got job one and it runs on Ubuntu and I've got job two and it runs on Ubuntu. So I have this list of jobs that I've built out, but what if I want some sort of dependency between them? What if I don't want job three to finish before job one? Maybe I want job one to always run first and then I don't care if job two or three finish first. That could be I want to do a build and then I want to run some acceptance tests or some unit tests. We can structure our jobs in any way we want. GitHub actions and the workflows are an amazing toolkit that lets us kind of automate and trigger our workflows as need be to match our working style or our requirements. And the way we can set up these dependencies in our workflows is using the needs keyword. So job two needs job one. So we can establish that dependency very easily. Let's keep going, let's see what else we got. So steps, there are lots of different steps. And I'll go ahead and run this guy and we'll go into what steps look like. So with steps they can be, like I said, commands tasks. Once this job loads we should see all the different steps. So we got all sorts of stuff. So I'm running, if we look at the setup job, I'm running on an Ubuntu machine, so it's Linux and I can just say echo my path. So I'm doing a bash variable environment, variable reference and it's spitting out my path. And try doing CMD. But CMD won't work because CMD is a Windows specific thing, but out of the box. The Linux machines come with Powershell because Powershell is cross platform, but not Powershell like the Windows 5.1 version. I can also specify random shells, like I want to use Python, or I want to do something with Perl. Or I can specify a checkout action. Like I want to use the checkout action. I want it to be locked to a specific hash, a specific shaw like commit. For this checkout action. I can also do different versions, like I want to do version two or three based on tag or based on a branch. I can check out things. So how do I know about this hash, this version? Well, actions besides my simple commands that I want to run. If we scroll all the way to the top of the GitHub site, we'll see the marketplace. So the GitHub marketplace is where we have all of those great actions that are built by the community, by people, by companies to help do things. In this case we had the checkout command. So if I look it up, we've got a blue check mark to show it's creator verified by GitHub. We've got stars. We can see version information about the individual actions. And the latest version we have tags on if it's built and test. Any action in the marketplace needs to be in a public open source repo or a public repo. Anyone pretty much can go and see issues or do pull requests. For now, almost every well written action you'll find has really good documentation around the usage of it. The options that you can pass into it, scenarios like they might give you scenarios on how to leverage it, to do certain things that you're trying to do. So looking for an action in the marketplace is a great thing to look for. And we can go right to the code base so I can click on the action and it's going to take me to the repo that has that action, so I can look at how somebody implemented something or how something was done. I can also go right to the open issues. So if I'm having a problem, I can go look at the issues that are affecting that particular version or that action. And so we have a lot of ways to look for and find capability that people have already implemented and consumed that now, something that you might have noticed and I kind of skipped over, I was passing information into the action. Well, we also can do environment variables, and these all kind of flow together into what are called contexts, and we'll talk about those momentarily. But let's say I want to pass information, I want to have some variables I want to pass something down into an action into my job should trigger it already ran. I can run a script and say I have environment variables that I'm going to call, and you can see I passed in my environment greeting hello name world. So I can pretty quickly, pretty easily pass information from my top section in my workflow. I can pass the name of a service or do some sort of dynamicism like string manipulation, and pass it down into lower level jobs and have those picked up. Now, something else that I think is really important to cover when we're dealing with variables is there is a hierarchy that exists. So when we have workflow level variables, or we have things like job level variables, or even step level variables, they can override each other. Like if you invoke the same environment key like environment name, this is saying hello from location step, and I've passed in information. But if I go and I look at my workflow, you'll see what's happening. I've defined at my workflow level a location workflow. But as I start executing on the specific job, the job overrides the workflow value. As I get down to an individual step, the step overrides the job level value. And so when I'm running my command hello world from step, I'm getting information from my workflow, from my local step in my environment. So understanding how youll can leverage values defined higher up in the hierarchy and how you can override them lower down in the hierarchy is important. Now we're dealing with variables. We're starting to pass stuff. Let's get into conditionals. So conditionals are a fun thing where we can add a job, we can conditionally execute jobs or steps based on values, and those values can be everything from the branch we're on or the environment we're in. So here youll notice run goodbye did not execute, but run hello did, and we can see why that is when we look inside the file. So we can do stuff like if, so if this expression is true, we can run this step, we can do things around jobs like maybe we don't want to do our deployment job if we're not on the branch main, if we're not in an environment like production. So we can have some conditional control of our different workflows, of our different steps. And conditionals are really a form of expression. So let me just show you what some expressions look like where we can do string formatting and we can do string interpolation. We can put values together and we can build up things. So if we want patterns and naming, we can leverage that. We can check if stuff ends with or contains strings. We can do array manipulation, so we can give it like a JSON String and make an object out of a JSON string. And then if it's an array, we can check to see if something contains an entry in the array, not just a string, but the whole array. And then we can do stuff like joining and concatenating, and we can print those out. So those all work very much like you would expect. We get some basic dynamicism and we can play around with string manipulation in some of our workflows, and some of our actions and expressions are essentially a type of context. So I've got a couple of contexts here, and I will show like the GitHub Context gives you a lot of information about the repo and the check ins and the commits and the users. So there's a ton of information there about the repository and the, you have stuff about individual jobs, like if they've run steps. So if we look at our GitHub workflow context, contexts are something that comes up quite a bit when we're building out things. We've got our environment. So if we look at our environment, we can see all of the different values that have been defined. If we're doing a job, we can find out information about the container and the services, if we list it out, so we can define different things. So contexts are going to be an object that we reference as we're doing some more complex interactions with the different workflows. And then secrets are a type of context. So secrets exist in GitHub under settings. So if we come over here to settings in my repository, you'll see down here under security, we've got secrets and we have actions. And you'll notice I have environment secrets which are listed here in this top section. And then down below I have repository secrets, which are for the repository. And just like we had a variable hierarchy, there is a secret hierarchy. Like you can have level secrets and repository level secrets and environment level secrets. So you can have different secrets that can override each other. I can edit this, which will change the value, or I can remove it. And the way you access, and you can see I've got some environment secrets here, I'll show those in a moment. But the way you access your secrets is relatively simple when you're looking at the workflow. So if we're in our workflow and we want to get to a secret, we do the dollar kind of handlebars secrets name. And if this did not exist as a secret, it will just come back as an empty string. It doesn't want to give you an error that that secret doesn't exist. That could kind of give people a way to hunt around to see if you can find the secret. Now, notice I'm passing it two different ways. I a lot of times will pass it by environment variable over passing it in the command line, just in case something is like listing processes on the system. Though I know there's arguments about passing secrets via environment variables. It's a little bit of a debate. Whichever way you want to do it, GitHub actually tries to protect you a little bit and tries to not print out things. It knows our secrets. So if I go into the logging and I look at these two things where I've passed it by context, it knows that was a secret and doesn't print it out. Same with environment, but with the environment, it knows I'm pulling the environment name, it just doesn't print out the value. So this can give you a little bit better information. If you're troubleshooting things, this might give you a little bit less, but there's multiple ways to do that. Now, I mentioned that the secrets come from the environment or from the individual actions. So environments are listed here on the left. So we have under code and automation, environments and environments give you a couple of different things. You can just hit new environment and make a new environment. So I can do something like this, and I've created an environment. Now I can ask for approvals. I mentioned that earlier with the CD part, and I can say I want someone to be a reviewer before you deploy into that environment. I don't have to have secrets or resources, it's just a tag. But if my job says it's using this environment comp 42 environment, it'll ask for approvals. I can also have protection on this environment, so I can say I can only deploy to this environment if it comes from my main branch. My main branch is the only one that's allowed to deploy into comp and maybe have secrets that are specific to this environment. Now, a lot of times what I'll do is I will have multiple environments with the exact same secrets. It makes it much easier for me to reuse my jobs or to copy and paste them. I'm still leveraging all of the same secret values. I'm just referencing them from a different environment. So that's how I can interact with my environments. If I have multiple environments like this, maybe they have multiple secrets. And that's okay, I've got secrets and I access them the exact same way. So when I'm accessing my secrets, it doesn't matter if I'm accessing a secret from my repo, from my, or from my environment. And I know Linux environment, shell environment versus environment, a little bit of reuse and I'll show some end to end examples in just a few moments. I want to talk about matrixes, matrices for a second. So matrices are a really cool feature that is available in workflows where maybe you're a library builder or you're building an application and youll want to be able to test it on multiple versions of software or multiple OSS. Well I can define a matrix of I'm building a python library or a node library and I want to test it on node 1214 and 16. And I need to make sure it works on Windows and Linux. So I can define and you can see I've kicked off a bunch of node builds and a bunch of windows builds, but I can go in and say I'm building node and I have a strategy, it's my matrix. So here are my node versions and here are my windows oss, and I'm leveraging these just like I do secrets and other contexts, Matrix OS and matrix node version. So I can access the same values because it's part of my matrix strategy. This is going to be my matrix context and I can access the thing. I'm only building out the job template essentially one time, but this gets copied three times, two, six times for the full matrix. So I think that's a really cool feature that some people need to leverage. Not everybody does, but it's a good to have. So let's look at some end to end stuff. I've got three workflows left and I just want to walk through a couple of examples. So like net, for instance, I do similar patterns to what I've done in other pipeline CI CD platforms, which is I'm a big fan of separating out my build and my deploy jobs stages. I like to have a build that compiles tests and then gives me an artifact. And I like that because I can get really fast feedback on it and it gives me an artifact that I can redeploy later. I can youll back and forward pretty easily and just redeploy that one task without having to rebuild everything. Because I know I had a good binary. I usually also do infrastructure as code. So you'll notice I upload two artifacts here one is my web app and one is my infrastructure as code. So it's IAC. So I'm uploading both infrastructure's code and my compiled application so I can deploy it out. Now I said environments can have approvals. Well I set up an approval here on this environment. And before I approve it, if we go take a look at the Yaml file, I'm doing some interesting things where I'm only going to run this workflow if I like no paths. So remove all the paths and only add in the. Net sample. So only do it when I touch that folder. And I'm doing some builds, I'm setting up net and giving it the specific version I want. And these are cool actions that you can look at. Anytime you look around on GitHub and you find someone using an interesting action, you can come out here and you can look up the action, you can see use cases and how that interacts. So I poke around a lot of open source projects to see how they're doing things. Cache defines one. It provides you with this type of syntax for compiling nuget packages and. Net packages a little bit more quickly that cache things. But I'm just doing individual commands to restore, build, test, publish and upload artifacts. Now this is where it kind of gets cool on my deploy, it needs build. So my deploy has to wait till the build finishes and I'm using an environment. So this is where I have an environment called net and this is where it's asking me do I have an approval to deploy into my net environment. I'm going to download infrastructure as code and I'm going to log into Azure because I want to deploy an arm template or a bicep file and I deploy out my infrastructure as code. There is an id here which I'll touch on momentarily. And then we get the web app and it deploys to the output of my deploy step. So because I gave this task an id of deploy, I can go steps deploy and I have an arm template here that has outputs so I can do outputs. Web app name I don't know what the name of this web app will be at the time I write it. Like my dev environment might give a different name than my prod environment because it's driven by a secret. So I don't know what to deploy to. So I need to get the dynamic output of this step right here in order to pass it to this step right here to deploying out my code. So let me go ahead and approve this so I can give an approval. I can leave a comment. This will now start kicking off the job and it'll spin up infrastructure, get the dynamic name, pass that into my deploy and deploy software onto that web service that got created. You can see it's pulling down the actions it needs and it's going to do that task. Now this is using the older style of Azure login, using the credentials with a service principle. They've got a newer style that's really nice called Openid Connect. They actually link to it right there in the conversation or in the log. And what's awesome about OpenID connect is you don't have to pass a secret, so you don't need a password like Azure and GitHub trust each other. And I might show this if I have a few minutes, but really look into OpenID connect. It's pretty neat, but yeah, I'm running this deploy. I don't know the web app name, but it's creating my deployment and then when that's done it's going to download the web app and deploy that out there and we can see kind of what that looks like in an older run that I did a little while ago. So if we look at our deploy here, it's getting the deploying and because I had a secret as part of it, it's obscuring it, but it gives you the URL to access the deployment and passed it in and it worked. Well, what about containers? What if I want to deploy a container? Well, containers are very easy to work with. In GitHub actions. We can log into Docker Hub. So if we want to log into Docker hub or ACR for Azure, the Azure container registry, you can use the Docker login action for both of those. If you want to do a build and push or you want to use build to compile a container, I'm giving it a folder of container example and I'm saying I want youll to push this code with these tags so you can have multiple tags on a container. Now I do have another sample out on GitHub that you can look at if you're interested in. That one is kind of more slated directly to containers. Anyways, I will find that in a minute. Could have sworn I had it right on my main page. But that one's another fun one where you can see a couple of steps like I want to do trivia scanning, I want to scan my container, maybe I don't push it when I first build my container and then I scan it and then I push it up. GitHub also provides a superlinter, and a superlinter will pretty much look at all of your code base in pretty much every language you can think of. It's really neat for just trying to standardize types along your code, just some weird stuff. Best practices. It has so many linters, it supports so many languages, and there's some config files you can check in that will help standardize how you want to do it. So if you want to turn on certain ones or turn off other linters, you have full control over how those different linters work. Now you might say, these actions are awesome. How do I get started? Well, you can just go to actions and say new workflow. And what this will do is it'll look in your code base and try to give you an idea. It'll scan your code and try to give you a starting point, like it's a docker image or net app or a grunt app. And you can search for these workflows. If I was doing like, oh, maybe I want to do NPM, well, there's a whole bunch here about oh, I want to do a node JS package and I can hit configure. And what this will do is it'll already set up your repo GitHub workflows. It'll give you this and it'll start you out with a reasonable place to start from. You can also search the marketplace right there. So I said I'm doing Azure. I can click an Azure button and add in an action potentially, so I can find a version and I can copy it and I can paste it over here and then I can delete the stuff I don't need. So it gives me a starting point. Another thing that I think is really interesting is the paths up here. Not everybody. It's interesting that it might have a whole bunch of stuff there. You can name it whatever you want, it'll automatically pick up those actions. Another really cool thing, GitHub has build essentially a training platform based on GitHub actions. So skills. GitHub is a full training thing built on this. Remember I said there were events around what happens if you fork it, or what happens if you comment or push? Well, when you use this template and you make a fork, it starts doing stuff and then it starts prompting you via pull requests, issues, comments. And if you come in here and you're curious how it works, look, they have some scripts and they have workflows. So this is going to unpush to main, it's going to check on a conditional is this GitHub a template, so you can see some cool tricks based on how things are working in the actions. And you can look across GitHub. So if you're looking at like GitHub workflows, you can go see what other projects are doing, doing codeql analysis, checking for spelling. So you can go take a look and see what people are doing. Oh, they're using a c spell action. That's cool. I didn't know I could do that. That's all I had. Please feel free to, like I said, follow some of my repositories and take a look. Feel free to clone some of these repos and play around. That's the best thing you can do, play around with actions, try them out. It works really cool, really well. And there's even a way, if you're really interested, to test some of these locally. You can see here, I'm doing a build, I'm doing a scan. So there's a tool out there called act that I like to recommend to people. If you're playing around, it is super cool. It runs in a container, but essentially this will let you run your GitHub actions locally, like you pretty much say act, and it'll pull in your workflows and it'll run them inside these containers. So you can kind of test things locally while you're trying to get stuff working. I've used this many times and it's a really cool repo, it's a really cool project. And it's something that I think if you do all these a lot, if you make a lot of workflows, that this is something that is a helpful tool. And I did not get into some of the reusable workflows or the templates, but again, search for the documentation and it'll work out. You'll find it. The documentation is incredible, so you'll be able to find that pretty quickly. And so with that, I'd like to thank everybody for joining me today, and happy deploying and playing around with GitHub actions. Thank you.