Hi everyone, my name is Josh Stella. I'm the CEO CTO of Fug. Fug is a company that secures cloud infrastructure structure toand checks it for compliance. For the last ten years, I've been living purely cloud security, both at Fugue as a co founder, also at AWS, where I was a principal solutions architect focused on national security. So that's a little of my background. Oh, I'm primarily a software developer and software architect prior to being a founder. Okay, today we're going to be talking about locking down the security of IAM. I'm going to pull my slides up here. Locking down the security of IAM in the AWS cloud. The focus today is purely on the AWS cloud. If you are new to IAM, this is actually going to be a pretty good introduction because we need to establish a mental model, an understand of what IAM is and how you use it if you're more experienced. I will be getting down into some details and hopefully this will be a really useful session. Okay. It's mostly going to be interactive session. Our whole agenda. I have five slides. Most of this is going to be hands on at the whiteboard, hands on at the console. Okay. But I do have five slides and they're really. But why cloud security is different than traditional data center security. We're going to do that deep dive into securing IAM, and there will be, of course, Q A. All right, so why cloud security is different? For lots of reasons. I'm going to really focus on a couple of these as they relate to this topic we're discussing today. So the way hackers approach kind of the attack surface of the Internet has changed over time. In the old days, sort of the Hollywood version of hacking, which, by the way, does still happen. The hackers pick but an organization and then find a way in, right? So they identify the target and they search for vulnerabilities, or create vulnerabilities through things like social engineering. This does still happen. Some famous examples of this are the north korean breach of Sony's network, Sony motion Pictures. They made a movie they didn't like and so they went after them. But far more often, it is what we're calling here the cloud exploit strategy, which is the hackers are using automation to just look at every single object they can reach via the Internet and looking for misconfigurations very often on cloud, looking for things like IAM and s. Three misconfigurations, IAM being the permissions that are granted, but also bucket policies. There's a lot of things they're looking for. And then their automated tools essentially give them a shopping list that they can then use to go pick, but who to attack from. And some of the really high profile big attacks, like the capital one breach, that's how that was done. She didn't target capital one. Her automations discovered that. Now, John Breeden here is saying that within a few hours of deployment, these assets are being scanned by hackers and bad guys. We think that is an extremely rosy estimate. It is more like five to seven minutes. So what that means is when you put an object, a resource, a cloud resource, out onto the public cloud within a few minutes, it has been examined by hackers who intend to discover its vulnerabilities. And if it has vulnerabilities, they've probably found them. So a whole lot of these breaches probably go unknown and unpublished, or unpublished toand unknown in some of the notable breaches we've seen in recent years. The only reason the victims found out about the breach at all is because the hacker bragged about it on social media or something like this, even months after the actual breach occurred. So it's a brave new world. All right, so the main attack surface in the cloud is not the operating system of servers that you have hosted in the cloud. It's not the configuration of your applications or the security of the applications that's much more relevant to the data center. You still need to do it in the cloud, but you have this new attack surface in cloud, which is the vast majority of hacks, which is the misconfiguration of the cloud resources themselves. And very often in the press, these things are portrayed in overly simplified ways. I've been interviewed with some frequency by journalists trying to get their head around how a bunch of data was exfiltrated out of s three. And they always go into it thinking, well, the s three bucket was left just open to the wide world. Very often it wasn't. There was some combination, usually using IAM, misconfigurations. CTO, get to the s three data. Okay? So at Fugue, we cover all these different compliance standards for you more than anyone else, ISO, toand, NIST and HIPAA, GDPR, CIs, et cetera, et cetera. And what we've learned is that those compliance standards, even in their most conservative interpretation, which is how we do it at very. I mentioned I have a national security background. We are exceptionally paranoid in our interpretations. In a good way. We're going to look for everything. But even when you do that, a lot of the techniques that the hackers are using are not caught by those compliance standards. A lot of stuff is, and you want to use those, but a lot of things aren't. So even if you're sitting there thinking, well, I know IAM is secure because I've examined it against the best practices and I've used the AWS IAM analyzer. IAM analyzer. You probably still have highly vulnerable IAM configurations. It's hard not to. And they're usually missed by security tools, okay. Because they're not recognized as vulnerabilities. One example, and I'll come back to this, is there is one action in EC two. It's not even can IAM that allows an EC two instance to change which IAM policy applies to itself. A lot of people have this turned on by accident. That is a massive breach potential to it, okay? Same thing with things like s three list in these IAM permissions, really, really dangerous in production environments, really dangerous, but probably looks fine to your security tooling. So what this means is, well, also they're only apparent in the full context of an environment. You can't just scan an Infrastructure's code file and say, we're good. You have to look at when that terraform or cloudformation script runs and does a deployment or whatever you're using, you're doing it manually. The same thing is true. You have to look at it once it lands in these overall misconfiguration of the account, the know the whole combination, because that's what the hackers are going for. And the primary network that they're using is not the TCP IP network, it's the IAM network. And I'm going to explain what I mean by that later. But IAM forms a network in the cloud, okay? Because of all these issues, these things are really easy to overlook, and that makes them incredibly common. And here it says, increasingly exploited in major breaches. I am not aware of any major breaches that were not carried out through misconfiguration of cloud resources. You look at some of them, I have a whole talk, I give on thinking like a hacker and walking through details of breaches. But when you look at, like, for example, the Uber breaches or the Imperva breach, and sorry, I'm naming names because you have to be able to go, look, these things know Imperva is a great company. These make a great product. They had a great response to their breach where their CTO went and put a blog post up saying exactly what happened. And guess what? It was poorly managed configuration of IAM keys almost always is. So IAM is central to getting cloud safe. All right, now that's the slides we're going to go to actually digging in. All right, let's do that. So we're going to start. You might feel like this is kind of remedial, but I think it's actually going to be quite useful. I'm going to start with just the AWS documentation on understanding how IAM works I mentioned earlier. I believe it is just absolutely essential for you to have a functional mental model, a way to really understand what's going on in IAM and in cloud security in general. And to do that, well, we'll start with understanding how IAM works, and then we're going to go to the whiteboard and iam going to show you a simplified form of this. All right. This diagram, to me is not super helpful. It's literally correct. But that's not really useful to forming a mental model on how to think about it. It does show us a number of things, like principles, firing off requests, needing to get authorized, and of course, authenticated and authorized CTO perform actions on resources. But this diagram is just really, really muddy and it's easy to get confused with it if youll don't already know what they're talking about. But I do want CTO talk some about the terms here because we're going to use these as we go to the whiteboard. So the first term we're going to focus on is principal. So iam a principal toand in AWS, a principal is an actor. If any of you used to do uml, it's an actor. Okay? That can be a human, it can be an application, it can be a component of infrastructure. All right. That principle are the only things that can create requests. There's an asterisk on that, but don't worry about it. Simple mental model. Right. Something we can work with. So you got principals who do actions. They do them by sending requests. Okay, so a request is what it sounds like. I'm going to skip over those things because it's just authentication and authorization, in that order. Yeah, the authorization is where you're going to find out if you can perform an action. So you've got a principal performing an action on, typically a resource, which is an object. Okay, let's switch to the whiteboard. I lost my whiteboard. Let me get it back and let's switch to it. Okay, cool. My whiteboard's working. Excellent. All right. You never know when you've got these fancy, complex setups. Okay? So let's just get some of what we just discussed onto this whiteboard. So we have principles and we have actions that's doing things in the cloud, and we have resources. Okay, there's a reason I'm putting this on the whiteboard, because a lot of the complexities in designing secure IAm are not constrained to just the policy understanding. They're really cases by the interactions of these things. And there are lots of scenarios that you can get into where you can overly compliance things, or worse, you can make them overly permissive and therefore dangerous. So we're going to talk through really two scenarios today. The first one is where the principal is a human, okay? And this is what most people think about. It's kind of a hunchback human. Sorry about that. I'll call him Josh, since he's got a bum shoulder. I don't have a bum, just, anyway. So human actors are who most people immediately think about when they think about IAM, identity and access management. They're thinking about know active directory users with their authorizations. Right. And that is one type, and we are going to cover that. But we're also going to cover applications as principles, or components as principles, resources as principles. So for example, we can have an EC two instance or a container or a lambda. I always get it backwards. Let's see if I get it backwards this time. I hope not. Might have. Anyway, you can have an EC two instance acting as a principle to do things like read out of s three. And the way you define that is through IAM. So this is how IAm becomes a network. And the same with. Same with our lambda function. Okay, so what are actors? We have a kind of high level fuzzy idea of that. Well, when you think about the cloud, what you really have are APIs that you can use CTO, manipulate infrastructure to create it, or at least configure it and modify it over time, et cetera. So creating an EC two instance would be an example of an action. All the verbs you can do in the cloud, read from s three, get object from s three, list s three buckets. So your actions, and there's a lot of them. For the purpose of IAM, understanding IAM, I think it's best to not overly focus on which service things are in, but to just understand that for any given function that you want to perform in the AWS cloud, there are a list of actions that that principle needs access to, and very often those are not constrained to a single AWS service. So it's best to think of actions as just a big long list out of which you're extracting the ones that are the minimum needed to perform the function, the business function you're trying to do. There's a feedback loop in architecture here, too, which I'll talk about a little bit. But actions, let's say like getobject, create instance list buckets. Now you can go on the web and search for AWS actions, and there are thousands of these things. And that's actually really good design because it gives you the ability to really scope your permissions down. But what it does do is put a lot of burden on you to really understand the actions. And there's a lot of these. That's where things like the analyzer can be helpful, but it can also give you a false sense of security, because if you've architected your system in a dangerous way. So for example, your Ec two instance needs to list s three buckets. Don't do that, by the way. Put it in a database, separate the actors. These tools aren't going to find that. So you really have to understand what the attack vectors are. And then resources are named things in the cloud. Like there's a service s three, right? But you have an individual s three bucket. What did I get up to? Let's call it Baz. Or you have a container or an RDS instance, a database instance. Call it my data. What do we call our container? We'll call it whale. All right, so we've got our principles, we've got our actions, and we've got our resources. Let's think about this from the perspective, first of Josh here, of this guy. If I'm a user, a human user, I mentioned I'm a developer, so I need to do things like create instances. Create Ec, two instances. Okay? So I want to be able to call create instance. However, there is a default AWS policy for all of these actions, and it is deny. And there are really two walls of denial. There's the wall of denial between any principle and any actors. So your default is you get none, right, good design. And the Other is between the actions toand the resources. So there's a wall over here too. And so the mental model for IAM is poking holes in these walls. Okay, youll need to get from the left to the right on the diagram, and you need to do that by poking holes in the walls. And that's what in IAM policy are allows, or I call them allowances. I think that's the correct English to poke through the walls of denial. Now, any deny you can get. So for example, let's use a different example. Let's use the, well, let's do a list bucket, all right? Which is different than list buckets because in list bucket I'm listing the controls of one s three bucket. In list buckets, I'm looking at what are all my s three buckets. So let's talk about list bucket for a second. So this guy got blocked. If I need to poke a hole through here for list bucket, I want to poke a hole through here for list bucket. Right. So I can do that with Iam. I can grant the action to Josh using a policy to be able to list bucket by default. You also then have to say which buckets may Josh list. And very often people just put star in there, which is actually often wise for many use cases. We'll get into some complexity of things like attribute based access control for scoping. But you can also, in your IAM policy, just enumerate a list of buckets Josh is allowed to list, and that's reasonable to do too. So if Baz is one of those, then the request, which is the thing that caused this whole chain of events to take place, will work because it got allowances all the way through and from Baz, will be returned CTO. Josh, a list of the contents of the bucket. Right. Seems pretty clear. Okay. So as you're doing this, your goal has to be to have these minimum number of these actions. Okay. And even in the AWS documentation, they will tell you, be as constrained as you can, least permissive. Right. Well, let's take a look at that. I mean, that's great to say, but this is one of the places where people really get burned with Iam, and it's really easy to do. And every time somebody gets burned and ends up in the news due to IAM exploits, I feel bad for them because there is a lot to know and do when you're dealing with IAM. Okay, I'm on the screen here for the IAM service and I went to policies. Policies are where you're going to be effectively defining the allowances and the denials. Right. Toand you get a whole bunch of these that are called AWS managed. That means AWS put them in your account already because the AWS services themselves need them in some way, shape or form because AWS services talk to AWS services using policy too. All right, so let's take a look at one of these policies. All right, here we've got, you can see here at the top effect is allow. Remember, everything's a deny. And then here's the list of actions. Access analyzer, service role policy. Okay, so this is a policy that's intended to be used by an access analyzer tool that AWS has. So, what it needs are these very specific EC two collection here. These are describe calls. Those are similar to lists in other places. So describing the addresses the VPC endpoints describe vpcs. So, vpcs are actually inside the EC two service. Just a weird thing, a side effect of how AWS evolved. And then we see some for IAM. Remember I said at the beginning, usually to perform a necessary function, a business function, you're going to have more than one service in your list of actions. Okay, so here we've got some EC two, we've got some IAM, which is just getting roles and listing roles. And then we've got kms, right? For our keys, we've got lambda, because Lambda is an alternative to EC two. I mean, fugue is almost completely built out of lambdas, but lambdas are computational actors. They're principles in here. So we got that. And you also have a bunch of stuff for organizations, which describes, obviously, how the topology of your organization, its permissions, and so on, map out. And a bunch of stuff in s three. Okay, so you can see here it's a fairly long list of actions, but you notice in resources, right, well, resource in this case is any. All right? So that means any resource that these actions apply to, it's not scoping down the list of resources here, it's saying any of them, but these actions are the only ones that can be called on them. I want you to notice here that this star is the only star in this policy. Star means any. Okay? And as you go up here, you're just not seeing big, broad star cases, and you shouldn't use them yourself. Iam looking at an AWS managed policy, because these generally, or often anyway, contain a lot of best practices. And you really do need, if you want secure iam to fully understand which actors are truly necessary for the business function. And limit to that. And I'm going CTO. Explain why here, now. Okay, well, that sounds simple enough. Just list the actions you need. Well, why do people very commonly make the error to be overly permissive? Let's take a look at this is a policy I'm building for interacting with EC two. So remember, we're talking about Joshua user. I needed the create instance. So that's probably here in write somewhere, right? Well, I selected all of write which would be write star. And I just said don't do star. There are 289 write actions in EC two alone. So somewhere will be create instance, I hope, right? You would think create Internet gateway, create image. See, I don't remember the name of it off the top of my head. So somewhere in here are the permissions I might need to use the actions that are needed to meaningfully create an EC two instance. It's not just going to be about creating the instance, whatever they've called it, I'll go find it later. It's also probably about like creating an EBS volume and attaching it and some other objects that other resources. An EC two instance, practically configured, is a collection of resources. Really? Okay. So when you're designing these policies for your users, my strong recommendation, and that of AWS as well, is to use roles, not map what are called inline policies directly to josh these user, but instead create a role like developer or team a developer and have in one place that list of permissions of allowances that are really needed for that developer role. And then if you decide to change that or evolve it, you can do that in one place and the managing will be preserved. Another thing I see that is usually an error in securing IAM is just proliferating iam policies. I mean, a lot of folks, I think, look at the fact that AWS sticks like 700 of these things in your account already and think, well, no harm in that. I could have another 700 or 1000. Gets really hard to keep track of. So doing this is star, okay, that's star. Now you could have star for everything. Ec two star, right? But we're not doing that. We're doing here. Ec two, write star. Still do not do this. Enumerate the specific actions. I'll give you a couple of reasons why. The reason why people do it is they look at this 289 and they go, this is going to be a lot of work. It is. It's going to be a fair amount of work. And you really need to develop the expertise toand follow best practices as they're available from places like AWS. I'm going to argue that not all AWS managed policies are actually all that securely designed, but it's work you have to do. And there are tools that will help you. We can help you. There are tools in AWS like the access analyzer and other things that can help you, but you really do need to develop your own chops. Now, missiveness that you do not want to use. Star. Like maybe for example, the, oh yeah, ec two has a whole lot of describe calls, too, 124 of those. That's not a good example. How about read? How many are in read? Okay, 23 toand read. So that's kind of a reasonable number, maybe. I think to myself, well, that's fine, I'll just do read star. Well, what happens when AWS adds a new action into read? It gets inherited. Right? So I mentioned earlier, there is a particularly dangerous. Here, I'll show you a diagram of this. This is fugue. We automatically do these diagrams so you can kind of go spelunking around your cloud infrastructure to understand what's going on with it. But here I've got can EC two instance right here, and it has this IAM instance profile. Okay, we're getting into our second use case. Instead of Josh as the principal here, it's this EC two instance. That's the principal. And what we've done here is we've given it a mapping to an IAM policy to allow it to go read and write from s three, and hence the policy name s three, read write. So it gets a policy, just like a user through a role would get a policy. And that's a lot of what goes on in AWS hacking is people being overly permissive, or just having the wrong configuration, or even the wrong application design, which causes the policies that are running in the actual environment to create huge, exploitative, lateral movement data exfiltration, illegitimate crypto mining, all that stuff is done through these IAM configurations. Very often they're not getting them from users. Sometimes they do. Very often they're getting them from running infrastructure that has a policy. And so in this case, we've got s three read write. That probably means that EC two instance has the ability to do things like list all these s three buckets over here and their contents. So that'd probably be pretty attractive to a hacker. Right? So, other point I was making though is, if I've given this like s three list star, right? Can s three read star? When new actions come out on the services, iam going to inherit those because of the star. And the one I wanted to point out that I spoke about earlier is there is one permission in EC two that allows you to remap which IAM instance profile you have, so you can create an EC two instance, give it a nice safe IAM profile, but for one error, which is the ability to perform the action, which is kind of buried, that allows you to switch which IAM profile these EC two instance has. So you can think of it as lateral movement or permission escalation. But if that's in there, then hackers are going to have a field day. Now, that didn't always exist in EC two. In fact, when I was at AWS, the beginning part, there was no feature like that. There was no action like that. So if I had created a policy in whatever it was seven, eight years ago where I had, I believe it's in the rights. So if I had given it right star, for example, and these were a much shorter list of rights back then, I might not know, it might have been actually safe for my use case then. But as that feature was added to EC two, I might not even notice that. How many services are there now? Hundreds. And the release notes are great and everything, but it's hard to keep up. Well, all of those EC two instances that had my once safe policy have now inherited an incredibly dangerous action because I put a star in there. So a couple of reasons to not use star. So that's really how we get into trouble with IAM, is through over permissiveness and also through things like losing track of the number, all the different IAM policies and resources that you've got in them. I'm going to go back to my whiteboard here if I can. Okay, so let's whiteboard a little more, because if you remember in that example IAM policy, we had a very concretely enumerated list of actors. The AWS example that I showed you for the access analyzer, but then for resources, it said all, you don't have to do that. So in that IAM policy, you can list resources. You could say, like, if you wanted s three in this case, list bucket permissions, but you only wanted Josh to be able to list bucket permissions on Baz. Well, you could put that in the IAM policy instead of saying resources star, you would spell, but Baz there using its arm, okay, now, a lot of folks do this, and you'll end up sometimes with policies that have a long list of resources in that resource field, and they feel like, well, IAM getting this right, that can get challenging at scale. A lot of our customers have hundreds of thousands or more cloud resources. We manage millions of cloud resources for our customers. And when you see these really large deployments and you imagine, can every named resource be encoded, hard coded into these IAM policies toand kept track of? Because there's constant churn in the environment and the answer is often no, that becomes very challenging. Just to wrap up here, and we're doing kind of an abbreviated form of this. There's lots to talk about in IAM, but let's talk about something called attribute based access control. Attribute based access control is where, instead of having to list in the resources part of the policy, every resource that principle may act upon, it's based on tags. And so you can tag resources. If I'm on team a, I build application a for my business, and there's a team B-C-D-E-F et cetera. As long as those resources that are relevant to team a are being tagged in some way, CTO team a, I can then create new resources with those tags, and suddenly team a will have access to them. Okay, well, that sounds really great. Right, so let's give the tags, we'll make them blue. So I've got these tags over here that know which resources. So blue is a and I don't know. We'll do rainbow is b. And so if Josh is on team know we're on a here, we can only see the team a stuff and operate on the team a stuff. Okay, well, that makes sense, right? Here's the catch. If you do this, youll need to really restrict access to writing tags, okay? And even reading tags, maybe now that carries its own set of architectural burdens. Put another way, if I'm limiting resource access based on tagging and many different products and functions in cloud use tags, and I have to block tags access to writing them, particularly because then any hacker who gets in who has right tag permissions can open up whatever resources they might want into different groups. Now, obviously, you might say, well, isn't that going to be limited? If a hacker gets in and has a team a set of policies and permissions, will they be able to even see Team B stuff to retag it? No, that is fair. But there are scenarios where a hacker gets a more permissive set of roles and can retag things such that they are appearing in places that you don't want to restrict. So you really need to be careful about you're now including tag management as part of your security boundaries. That said, with that caveat, I think it's a really smart way to go for a lot of things. Maybe not the only thing to do, certainly not the only thing to do, but in a lot of use cases, it's really smart. Okay, one other thing I want to talk about are these are so IAM is thought of as like the method of security, and it's critically important to it by many folks. But really, it's a citizen among other citizens who have policy expressions in AWS. And remember, I said at the beginning that any request by a principal, if it hits any, denies, the whole request is denied. Well, IAm isn't the only place we can put allowances and denials. In fact, it interacts with things like s three bucket policies. Toand I recommend every s three bucket have a bucket policy. So, for instance, there would be, oh, let's go back. CTo the browser and my diagram, for instance, would be that this VPC, this EC two instance, because it's in this VPC virtual private cloud network, is allowed to read write CTO, this important records bucket. But I don't want this EC two instance which has the same role as this one down here. Just, it only can speak CTo this other s three bucket. Okay. And they can't see each other's s three buckets. Well, there's a couple of ways to do that, right? One is you could put the resource, you could have two different IAM policies with different resources enumerated, or those two s three buckets cloud have their own what are called s three bucket policies. So these are resource policies, not principal connected policies, if that makes sense. So we've been talking so far about these IAm policies that get attacks to the principles column over here on the left. Now we're going to talk just briefly because it's not IAm. The name on the tin here is IAm. But it's important to understand you have options. You don't have to solve everything through IAM. You have other places you can put policies and so on. This resource right here, this Baz, I can have what's called a resources policy. And the way I would do the pattern I was just describing where the one EC two instance that's in the correct VPC can get there is in that bucket policy. I can restrict access based on the VPC endpoint, which is this guy right here. And that's how this EC two instance is going to come out through this endpoint. To talk to this bucket, you can put in the bucket policy only allow requests coming from that VPC endpoint. So now if you've got two EC two instances that both think they have permission to all the s three buckets, you're okay, because your bucket policies have introduced another layer of denial. So you can say, no, you're not getting to me, I'm not the right s three bucket. I'm not the s three bucket you're looking for. So there are a lot of ways to strategize and to design your IAM policies and your practices with IAM understand there is no one right way. There are some general right things like being least permissive that you want to do, but there's no one right strategy. And you really have to think of IAM in these context of all the other AWS services and how you intend to use them together. So if you were looking for a silver bullet here, I can't give you one, but I can at least tell you some best practices and some ways to really get your head around the problem. The most common error I see moving my whiteboard here with IAM and with cloud resource configuration in general, but it's particularly true of IAM. The biggest problem that I tend to see is people never develop a mental model. They never develop a way to understand the problem. Instead they look at examples alone. Examples are great, but you can't imitate somebody else. You have to understand this stuff to do it well and make decisions. There are a million ways to do security well in AWS. I'll give you one example of this. Here's my email address and my twitter handle. I'm not a big Twitter guy, but reach out to me if you want to talk about this stuff. I'll leave that up. The example I was going to use of different ways to very capably accomplish secure systems in the cloud is to talk a little bit about how we've done it. And we've had two major iterations of our software at Fuge. The first one we started working on back before Lambda existed, and it used EC two as a runtime environment. That itself, that decision itself had lots of implications for how we would make things secure. When we did version two of the product, we switched to basically only lambdas. And we have obviously a database and other components, but for computation, mostly lambdas. And that had created a very different scenario in terms of how we would deal with IAM. It caused us to automate a lot more of our role creation for all those lambdas, because you don't want a lot of sharing, you want to limit that blast radius. And so that caused engineering work, right? So depending on the decisions you make in how you're architecting on the cloud, it's using to affect how you're going to do it securely, and it's going to affect how you're going to use IAM and think about those features. Okay, well, that's what I've got today. I hope you enjoyed it. I'm around to take questions. AWS, we're were talking and you've got my contact info now so thanks for attending, and good luck on your secure AWS. You've got the mental model, you got the piece parts. You can do it.