Hi everyone, my name is Bianca Popo and I am a DevOps engineer at Analog Devices. Thank you for joining me today for my presentation on GitHub integration within a private Jenkins server. The problem that we're going to tackle during this presentation is integrating a privately hosted Jenkins server with GitHub. The biggest hurdle in achieving this would be ensuring that the server is not compromised. library tags, while also providing as much information as possible to contributors about the status of their projects. The first question someone may ask is, why not just use cloud? There are so many available CI CD cloud services that offer out of the box solutions that are already integrated with GitHub, for example, GitHub Actions, Azure Pipelines, GitLab, and so on. And these are also highly secure, so why not just use these? The answer is that the software product that is tested may impose some limitations on the CI CD infrastructure. These are only some examples that I've encountered so far. For example, the produced artifacts might be too big, and most cloud services impose a limit on the artifact size that they can store. or the software product may require third party tools for building or testing, which may be too big or they may require a paid license. And last but not least, building the software project may be too resource intensive. There's also a possibility that some developers might just prefer a self hosted open source CICD platform. Based on a 2023 study made by JetBrains, Jenkins is the most widely used CI system among developers and companies. So this presentation is going to mostly focus on Jenkins. Let's visualize our goal. And for that, I've prepared this diagram. In the center there is our Jenkins server that is inside a private network. This server contains all of our build and test pipelines. Now, we could have this server run the pipelines once in a while, just to check that our application doesn't contain any bugs, or we could manually run it every time a commit is added to the main branch. But this approach would not work for large repositories, which have frequent commits and many pull requests. We could drastically improve the bug finding process if we could associate a specific build with a commit. This means that every time a commit is made to an important branch, such as main or release branches, the pipelines on our server should be triggered and verify that very commit. Next, we would like to make these results available to all contributors and non contributors on GitHub. But we can't just provide them access to our local Jenkins instance, because that could expose us to attacks. So now that we've defined our goals, let's take a look at what's stopping us from achieving them. In this next diagram, we can see that our local Jenkins instance within our private network is protected by a firewall. This means that while traffic can go from our server to the GitHub servers, our server can't accept any incoming traffic from GitHub. If our aim is to trigger a Jenkins pipeline each time a commit or PR is made, we would need to enable webhook triggers inside our repository settings. These webhook triggers can be configured to send a notification to a certain URL every time a certain event occurs. For example, push, pull request update, new issue, and so on. If we try and configure a webhook trigger to send notifications to our local Jenkins server, we will notice that our server does not receive any data sent from GitHub, due to this firewall being in place. The second issue would be displaying the results of the pipeline runs on GitHub, without compromising our server security. We'll talk more about that in a second, but first let's try and solve the first issue. We've established that we need to enable communication from GitHub to our local server. Now, we wouldn't necessarily want to disable the firewall, since that's what's keeping our private infrastructure, well, private. There are a number of solutions to this problem, but the most straightforward one would be to set up web tunneling. Web tunneling is used to create a secure path between your local machine and another server to send data safely. It works by encrypting data, routing it through the tunnel, and forwarding it to its destination. The most popular web tunneling service is NgRock, because of its reliability and security. We can see in this diagram that running the data through this web tunnel safely bypasses the private network's firewall. When running the ng rok client on the local server with a chosen port, ng rok creates a secure tunnel between that specific endpoint on your machine and the ng rok servers. Their servers then forward this data to the correct location, in our case, the github servers. This doesn't mean that ng rok can see your data though, because it offers end to end encryption of your data. With enabled access through our web tunnel, we can go ahead and set up our pipelines to be triggered by GitHub events. The webhook triggers sent by GitHub will have to be interpreted and processed by Jenkins. There are several plugins that offer such functionalities, but these are the most popular ones. A GitHub plugin offers a lot of functionalities, but the one we're interested in is the processing of push events sent from GitHub. So, this will only work for new commits. Each one of these plugins has a different endpoint that accepts requests. In the case of the GitHub plugin, the endpoint is github webhook. You can see in the image below that I've configured GitHub to send requests to the interrupt tunnel, followed by this specific endpoint. After the plugin is installed on Jenkins, inside the Job Configuration tab, you will now have the option to trigger the job automatically. The same is valid for the GitHub Pull Request Builder plugin, only this plugin accepts the issue comment or pull request events. The endpoint for this plugin is ghprbhook. The option to select inside the Jenkins Job Config is GitHub Pull Request Builder. Just a heads up, this plugin has some security vulnerabilities, so you should probably make sure you configure it properly. The generic webhook trigger plugin is a plugin that accepts all kinds of HTTP requests, so it's more flexible, but less configured for GitHub itself. Nevertheless, you get more control over what happens to your triggers. After configuring GitHub webhooks, you should see a job get triggered every time a commit or pull request is made. We still have one more issue left though, and that is how we're going to let contributors know the test results. So, our goal here is to update the status on each commit or pull request after the Jenkins job is finished. Inside the Jenkins job, you could use the GitHub plugin once again. It offers highly customizable solutions for updating the status. You can run it with parameters, such as the status message, the context, the result, or the reference URL. The latter is the most important one in my opinion, since it's what populates the hyperlink under the details section of the GitHub check stack. By default, the value of this parameter is set to be the URL of the Jenkins server itself, so not the ngrok web tunnel. This is an issue, because it could expose your private IP to anyone on GitHub. You could set it to the ngrok web tunnel URL, but then anyone who wants to see the results would have to either authenticate on your Jenkins server, Or you would have to provide anonymous users with read all access rights. Neither of these options are secure and could expose malicious actors to your internal infrastructure. So how can we make sure that anyone can see the test results without compromising on infrastructure security? You could process these results and then host them on a public webpage. A quick and easy solution would be to use guests. These are small Git repositories hosted by GitHub, and they are primarily used for easily sharing code segments. Gists support markdown, so you could pre process the Jenkins output so that it's easier to read and understand by all contributors, even those that are unfamiliar with Jenkins. You should also sanitize this output before publishing the gist, so that no credentials or secrets are accidentally posted. Creating GISTs is also quite easy to integrate into your existing pipeline scripts, since GitHub offers a command line interface that has such a functionality. In this way, you could easily add a PostBuild stage into your pipelines, which handles preprocessing and posting the job output to GIST. In this example image, I had to create pipelines for a repository which contains multiple projects that need to be built and tested separately. So, my approach with this was to generate a markdown table containing the status of each project and whatever errors or warnings were thrown up during the build or the test stages. You could post the entire Jenkins console output, or You could get creative on how to automatically prepare the output into a nice final form for your repository's contributors. In the rightmost column in this example, you can see the results from another pipeline actually. So this method can be used to merge results from multiple pipelines as well. This other pipeline tests each available project directly on the hardware on a remote server. If you're interested in that, one of my colleagues has a presentation on that topic called Advanced Test Harness Infrastructure for Validating ARM and FPGA Based Systems, which is also a part of this event. I would recommend setting up a bot account for your repository, which will be responsible for setting the status. This account will need reading and writing rights. Now that we tackled the last issue as well, you should get a fully automated process of building and testing your GitHub projects, and have the Jenkins results posted back to GitHub in a nice format. The screenshot is of a pull request that shows our Jenkins job result posted inside the Checks tab. Before ending this presentation, let's recap what we achieved and highlight some benefits that came from all this. We managed to automate the process of building and testing our GitHub projects on our local Jenkins server. We have also set up a process of automatically posting back the Jenkins job results, and we did all this while maintaining a secure testing infrastructure. This means that it will now be easier to catch bugs, since we can identify the faulty commit at any time. We also have a more detailed history for projects, because each commit now has the full test details attached to it. And in the case of pull requests, the review process is much easier, because each pull request is tested in the same controlled environment. So we won't get any more it works on my computer bugs. Both the reviewers and the contributors will have access to the test results, so everyone is on the same page when reviewing the proposed changes. If you have any questions or if you'd like to send me some feedback, feel free to message me on LinkedIn or hit me up on my email here. Thank you so much for tuning into my presentation and happy coding.