Hello everyone and welcome to my session. Today I'm going to talk about the potential risk of using Terraform providers and Terraform modules in cloud environment. But before we jump to that, I would like to introduce myself. I'm Uri, the CEO and co founder of Zest Security. I spent over decades working in the cyber security, specializing in both offensive and defensive security practices. As the former lead security architect, I've tackled countless security challenges in my career. And today I will be sharing some of those experiences with you. So the agenda for today. First, we will discuss Terraform and why I began researching its security application. Growing from my experience leading cloud security strategy and vulnerability management in large enterprise organizations, I will explain how critical it is to evaluate the risk of new tools introduced into a production environment. Next, I will share the analysis we conduct on Terraform providers modules and the key insights we uncover. After that, we will explore the potential impact of this risk. and what they mean for your organization. Then I will walk you through two real world attack flows that demonstrate how this risk could cause significant damage to your organization. And finally, we'll wrap up by discussing best practice and mitigation strategy to reduce and prevent these risks, ensuring a more secure environment. So, as a security architect or security engineer, your main goal is to ensure your organization is secure. To do that, you often need to understand how the software or tool work in your environment to spot the potential security gaps. In this case, the tool we are looking at is Terraform. Terraform is a powerful tool that allows DevOps or SRE to manage their cloud resources by code or ISE, infrastructure as code. With great power comes a great responsibility. So, to find the security gaps in Terraform, I went and read the documentation. By the way, Ashikov did a great job with the documentation. One section really caught my eye about the malicious Terraform providers and modules. Terraform configuration will have full access to the variable and Terraform state. Also, Terraform cannot prevent malicious providers and modules from exfiltrating sensitive data. That raised a few questions for me. What is exactly Terraform providers? What is Terraform modules? And how can an attacker exploit this? Let's break down these questions. Thanks. Terraform is based on plug in called providers to interact with cloud providers, SaaS providers, and other APIs. Each provider has a set of resource types and data resources that Terraform can manage. It's important to mention, without providers, Terraform cannot manage any kind of infrastructure. In other hand, modules is a collection of TF files kept together in the same directory. Modules are the main way to package and reuse resource configuration with Terraform. When a user creates or deploys a resource to the cloud, what happens is the user will define the desired resource he wants to create by creating a modules or a Terraform file. Once you execute the command Terraform unit, Terraform will check which providers need to perform this action, and will download it to the machine. Now, the user executes Terraform plan, and after, apply. This is where the provider sends the instruction to the cloud. From a tag surface perspective, Terraform provides a rating in Go language program. The communication between Terraform core and the Terraform provider is with RPC. And finally, the Terraform providers communicate with the target cloud API using HTTPS. Terraform is built to make managing cloud resources easy, but it doesn't mean it's bulletproof. And let's be honest, the attack surface here is something we cannot ignore. Before we get into the potential risks, it's important to note that HashiCorp managed providers in modules into three different tiers. The official, provided on and maintained by Ashicorp itself. Partners, provided modules on and maintained by other company that has a partnership with Ashicorp. Community, provided modules published and maintained by individual contributor. Ashicorp recommends to use provided modules from the official or partner as they are more secure than others. All the reports are stored in GitHub as a public repository. This makes it incredibly easy for attackers to exploit the code and for vulnerabilities and even take it a step further. Crafting a malicious pull request that could sneak its way into the providers or modules. Think about it for one second. Open access to the code means open access to new opportunities. Let's keep those tiers in our mind because we will get back to it. Let's talk about the analysis we performed. We closely tested 10 of the most widely used providers modules across the three tiers I mentioned before. What we found was a critical vulnerability exists across all tiers. However, community provider had the highest number of open critical vulnerability, almost 60%. While community provider present the most significant risk, the reality is that any type of provider or module can introduce risk. We have 12 of critical vulnerability related to the official and another 29 percent of critical are part of the partners. Even if the risk is low, which we'll explore later, it still creates a potential of attack back. So, does having more and non vulnerability make you more or less secure? It's a tricky question. If you have a lot of vulnerability, maybe it's because you are actively looking for them in order to fix them. But if you have very few, it's because you're, secure or because maybe, you're not even checking. So, I have another question for you. Are you using Terraform in production? Do you actually know which providers or modules you are using? And last, are you scanning those providers or modules for vulnerabilities? Let's continue and see why I ask those questions. Let's take a closer look at the community providers specifically. They are one of the biggest potential risk factors. To put it in perspective, the downloads number are huge. In just one month, we are talking about millions of downloads. Millions of downloads per month. Now thinking about this, every single download could be introducing a vulnerability to production environment around the world. And these aren't just the numbers. They represent real opportunity for attackers to exploit insecure or malicious code. It's a powerful reminder of knowing what you are pulling into your environment. So why is it so important? As we saw earlier, the millions of downloads of each month, these providers modules become a goldmine for attackers. The problem? They are often left out of the loop. When it comes to the security, providers models aren't typically scanned for vulnerability as part of the SDLC or application security processes. And while IC scanners are great at catching a misconfiguration, they don't dive deep into the vulnerability scan for this third party component. And let's face it, remediating security issues manually is nothing new. It's time consuming, expensive, and far from efficient. We have already covered what error proof providers modules are, and I've shared with you the risk analysis we perform on them. Now it's time for the exciting part. I'm going to walk with you through two attack flows. The first one is exploiting non vulnerable provider that happened to me, in my career. Second one is using malicious terraform modules that also happened to me in my career. So, let's dive into the first attack flow, exploiting vulnerable provider. For this example, I'm going to use CVE from 2021. It may be a bit old, but I actually ran into this in production just two years ago. In this case, the Terraform Vault provider had a vulnerability where it failed to properly configure GCE type bound labels for Vault GCP authentication method. As a result, security policy meant to bind resources to specific GCP labels were ignored, which opened the door for unauthorized access to Vault managed secrets and services. Remember when we talk about the different tiers? So, Vault is part of the official tiers, which should be more secure than others. Yet this vulnerability slipped through. Now let's take a look a how an attacker could explore this vulnerability to gain full access to the cloud. First, the attacker obtained the JWT token, which read only access As we have seen in previous attacks in the world, it's pretty common to find secret and tokens in public repositories making this step easier. Next, the attacker identify that the user has configured vault using Terraform. By the way, the user set everything up correctly. From the code at the top, we can see that the user has a defined bound service account, in this case the admin role, with bound label. But when you run terminal type command, The Vault provider mapped it incorrectly in the configuration, leaving the bound label empty. At this point, all the attacker has to do is to authenticate with Vault using the admin room. And surprisingly, it's going to, it's going to work. From here, it can escalate privileges to admin by using gcloud API to bypass the authentication and do whatever you want. Lateral movement, data exfiltration, service disruption, you name it. And again, I'm sharing this case because it's actually happened to me. We were lucky enough to cut it in and no one tried to exploit it before us. But sometimes, attackers don't need to put in much effort to exploit vulnerabilities. As you can see here, many times the provider themselves exposed sensitive information, like secret and token by mistake. The interesting part is that you won't find any related CVE for those issues. However, if you dig into the Terraform provider repository, you will find security issues. Some of them have been closed, but others are still open. And many don't even have a CV associated with them. This is just to emphasize how important it is to perform code review before you use any providers or modules. Even if there is no CV, it doesn't mean the code is secure. Now, let's move on to the second example, malicious modules. An attacker creates a malicious module. When it's ready, the attacker has plenty of options to convince, how to manipulate the user into installing the malicious module from many resources, such as GitHub, Terraform Registry, and S3 bucket, or even local path. In this example, I'm going to show you a malicious module that creates an EC2 instance in AWS, but also injects a backdoor using user data. Let's take a look at how, how that works. By the way, if you don't, if you're not aware of the user data, user data is a script of configuration file that executes automatically when the EC2 instance is launched, allowing you to perform initial setup tasks, like installing software or configuring settings or even a backdoor. The attacker uses a social engineering technique to convince the user to install the malicious modules. From the victim perspective, the code looks legit. Nothing. Nothing seems, suspicious can expect the, except the word malicious, which in reality would not be there. Once the victim execute the Terraform, apply command. Terraform will not only create a C two stances expected, but also deploy the back door for the attacker. Why? Because no one reviewed, the modules before, use it. Now let's take a look at how the malicious code looks like. As you can see, the Terraform malicious module is actually quite simple. It's create an EC2 instance with the user data scripts that is obviously using Base64 encoding. If we will decode the user data, in this case, we will get the command line that initialize the backdoor for the attacker. In my career, I've been, I've seen many DevOps teams use user data to run legit scripts. And they often use Base64 encoding to make the code more compact. This will just, show how easy it is for an attacker to create a malicious module that can bypass security. So, we have covered two attack scenarios. The first one involving a vulnerable telephone provider. The second one involving a malicious modus. Now, let's discuss how can we reduce this risk by following best practices in implementing effective mitigation. So, for best practice, what you need to do is to read the documentation and look for potential security gaps. Read the code carefully to identify any code injection or other vulnerabilities. Check for feedback from other users. Thank you. It can provide valuable insight into any issues or concerns with the repository. Keep scanning the repository you use for vulnerability. Even if it's difficult to exploit a vulnerability, it will help you understand how the repository owner handles vulnerabilities. Pin the version of your providers to avoid unexpected changes. Enable state locking to prevent unauthorized user from modifying or manipulating your state file. This can cause a real damage to our file version. Control the Terraform lock eight CM file to keep track the or, to keep track of Terraform providers version audit your Terraform plan and state file for misconfiguration that could create a security risk. And last use IC security tool to scan your configuration and fight potential security issues earlier. Now let's talk about the mitigation. You can implement control to reduce the risk in your environment. You can use IAM role and policy to limit access to your CI CD system and application logs, ensuring that only authorized users have access. Use only temporary credentials to minimize the risk of long lived credentials being compromised. Implement controls like VPC, load balancer, and WAF to restrict access to, to only, to only known, communication. Leverage the CWPP or SUS to detect any malicious communication with your environment. And last, set up CloudWatch alert in AWS to monitor and detect any attempt to, to read or write to your Terraform state file. This will help you to prevent unauthorized modification of your state file. I hope this session was insightful for you. Feel free to reach out to me for any question. Enjoy the talks on Conference 42. Bye bye.