No, wafs don't use a web application firewall, and when you should anyway. I'm Joshua Fox, senior cloud architect at Doint International and please contact me by email. This is remote and I always like to talk to my audience, so please reach out. I advise customers in Google Cloud, AWS and Azure. I'm certified in all three and I'm also a Google developer expert, which means I travel the world talking about Google technologies. But I'm not a Google employee. And in this talk I will mention a few different web application firewalls. We at Doit support our customers who are digital cloud natives. We give them advice in the clouds and we offer them software for analyzing costs, reducing cost, understanding the cloud systems, and we also resell cloud services, sometimes at a very nice discount. This talk is based on an article which you can see at the end. I'll give you a link to these slides and a QR code so you can read the article as well. But the talk goes into more detail. The scenario is that the chief information security officer is shouting about the vulnerabilities in your web application. Very worrying, and nobody knows what to do. Quick, we need a solution now. So somebody says WaF and the CEO says, yeah, go ahead, let's protect our system. That sounds like an easy solution. What is a waf? It's a service that tries to protect your web application. And later on I'll get into detail on how it does it and what it tries tries to do. But first, let's talk about some drivers for getting a web application firewall. The biggest one is a hacker attack fire drill. The system is under attack, data going out. There are data being encrypted by an attacker. We have to do something fast. Another reason is similar. It's a penetration test. This is a hacker attack that you paid for. And I have seen this happen when I've hired penetration testers and I asked one of them, would you like a password to our system to go into it and see what vulnerabilities there are? He said, well, let me try. Without a password. Within ten minutes he was tap dancing through the database. And so I sent him to another business unit because it was really scary what was happening to our business unit. Yeah, that's right. Same thing. The state of the art is not good. This is always motivated with urgency. And the problem with that is that quick and dirty solutions tend to stay that way. And that means that not only do you never get a proper fix for your insecure system, but your team doesn't get a fix. You do not develop the expertise as a team that you need to keep your application secure. Another driver for WAF is outside requirements, which often expresses itself as an audit. You have to fulfill the audit, whether it's because of a government regulation or the customer says you need it in a request for proposals. Maybe a partner, maybe the cloud provider itself requires that if you want to sell together with them, or a standard certification may require a WAF. Well, can't argue with that. They demand it, you're going to get it. Although that's not security. And another thing that isn't security is a security blanket, something to make the CEO feel warm and cozy. Because we have the WAF, so we've protected our system. That is the most dangerous driver for getting a WAF. To understand what a WAF is supposed to protect against, and often does, let's look at some potential threats. The top ones to look at are called the Owasp top ten. Owasp is a standards body, and the reason they list ten is because in reality there are thousands of possible vulnerabilities, tens of thousands depending on how you count it. Nobody can handle all of that, so they give you ten to look at and to learn. And we'll review a small fraction of that here, as well as some other threats. I created a demo. This is live running system at knowaf dot joshuafox.com and with waf dot joshuafox.com dot I'm going to walk you through it here in my talk in slides because it helps us focus on the key points. But if you want to play around there, you can. Now I'm the attacker and I'm going to create a game for the innocent victim. This is actually Scrabble, and the name I choose for myself is pretty strange. Script Alert hacked you script, that is JavaScript, and I'm going to make that JavaScript run in the browser of the innocent victim. In an actual hack it would steal your cookies, steal your password, but this is the standard example used to keep it simple. Once you can run arbitrary JavaScript in the victim's browser, game over. This is the architecture. One server can be reached through a load balancer that does not have cloud armor, or through a load balancer that does have cloud armor from the same browser. Two different routes, actually, two different subdomains with and without protection. I made the code unsafe by taking the code in the lower right, which wraps up the person's name in a div. So when the hacker sends a chat message to the innocent victim, the chat message will be rendered safe by escaping it. I'll show you what that looks like later. Then I broke the code on the upper left. The attacker's name is just going to be rendered onto the victim screen and execute it. Whoops, hacked you. That's JavaScript running in the victim's browser. However, if we access the system through the WAF, it will be blocked. So again, I try to create a game and I make my name into a script and boom, access is forbidden. So that's nice, but it's kind of a crude weapon to just break a request in such an ugly way. If you're sure you're working with a hacker here, well, I guess that's good, but I will explain later why this is a cruder system than you would really like. Another example of an attack is SQL injection, famous comic strip where the mother gets a call about her son and her son's name is Robert. Quote parenthesis, semicolon, drop table, student, semicolon, dash, dash, and little Bobby Tables has been entered in the database of the school and they've lost all the student records. Why is this? Well, if there is a bit of SQL like this where the first and middle name are inserted into the database, then it comes out like this. After you put that weird name in Robert, quote close parenthesis. Everything after the is a comment and therefore is ignored. But this drop table students will be run. Not good. The entire database table is dropped. Another example of an attack is distributed denial of service. That's where they pound your server with so many requests that it can't handle it anymore. Why distributed denial of service? Why multiple attackers? Because if it's only one attacker, then your one server can probably handle it. It's one on one. Your server can probably handle the load. But if there are hundreds of attackers pounding your server with requests, it might crash under the load. The worst type of threats are application level in your application, broken access control, incorrect authorization of page. So what if you have hundreds of pages, hundreds of PHP's, jsps, and some of them have not correctly defined who's allowed to see them or search. Pages are often very vulnerable this way because if you're letting your user search in some gigantic mess of text, and then of course the search results include snippets of text, do you know where those snippets came from? Is each one of them authorized to the person who was searching? But hey, let's toss in Oauth to protect against those threats we mentioned earlier and many others. How does cloud armor work how does it protect your system? Here's a sample architecture. There are end users on the left, and cloud armor attaches to the load balancing, which importantly decrypts the HTTPs. That's necessary for cloud armor to look inside and see if it appears to be a threat. Cloud load balancing then sends the request onwards either to a backend on Google Cloud compute engine, which is the case from Aida Scrabble, or it could be kubernetes or many others, or even outside of the Google cloud. It only sends on the request. Of course, if it doesn't block the request, you define the control using policies and rules. A policy is a collection of many rules arranged in order of priority. The ones with a lower priority number have a higher priority. They're going to take precedence over the ones with a larger priority number. And here's what the rules look like. Each rule has a match condition and action. Action could be deny the obvious one. If you see a threat, allow once you've created a rule that says this certain particular string really isn't a threat, so let's allow it, or just log and do nothing else. And the match condition is often a regular expression, but it can be many others, which we'll talk about. It could be an IP address blacklist or whitelist. Think about your firewall in Google Cloud or in Amazon. That would be an access control list, a security group. Or you can block certain IP addresses, but here you're doing it in the WAF, on the edge so it's faster blocking before the request even gets into the cloud. And also the more sophisticated tools for defining blocks of dangerous IP addresses. You could block entire geographies. If you only sell in one country, you might want to block every other country to just say, well, I don't want anyone from that country. It's not even a question, let's exclude them. Or a scan of HTTP content, often with a regular expression where you look for those scripts or other dangerous strings. Please use the pre configured rule sets. So Google Cloud armor has some for SQL injection, which I showed you earlier, cross site scripting which I showed you, and many, many other potential types of attack. Some smart people put these rule sets together and they are complicated, complicated rule sets. You can then configure them with sensitivity. You can either say let's be hypersensitive and block, but then you risk some false positives. You might block something you don't want to, or let's make it at a very low sensitivity just to get the worst, most dangerous looking ones. But then we risk false negatives, but you can make that trade off. I showed you a minute earlier, SQL injection. It's way more complicated than that. Each one of these here is a definition from OwAsp of a different type of SQL injection attack. The second one, common DB names that detected well, you saw the database table students. I don't know how common that is, but the good people at OWASp determined that if database table names are in there, if database names are in there, it's more likely to be an SQL injection. And there are hundreds of these. And in each one of those there are signatures. I am not expecting you to read this. In fact, I'm showing this to you to show how complicated it can be. That is a regular expression as you can detect. Don't try to read it right now unless you're smarter than me and tags severity many other definitions of this signature. Now you can write your own rules if you want. So in this case we're looking at the user agent header and looking for WordPress case insensitive in that we might imagine that a hacker is trying to break into WordPress so we write that rule. I would suggest that you use the rules the experts wrote at a far greater degree of sophistication, which addresses false positives and false negatives better than you probably will be able to. But when it comes down to it, the WaF won't protect you. It will not bring you the security you want and need. The first reason is that you may be blocking your own application. Here's an example. You have a software engineering forum. People are discussing snippets of JavaScript. Do you want to block those? Well, I once entered a bug in a bug reporting system. There was a security bug. It involved a JavaScript for cross site scripting and my request disappeared. I couldn't even report it. Well, of course there was looking at my report about cross site scripting and blocked it. So you can't just block these things because they look somewhat dangerous. And it's not just software engineering forums or bug reporting systems. Anybody could write some string that looks a bit like an attack string. You get false positives in certain cases. If your application is badly written. Some applications pass JavaScript back and forth from client to server and then execute it. That is bad. But if your application is already written that way, your product manager is not going to like it when you explain why you're blocking your own functionality. Or more simply, if your application has hundreds of text fields and unvetted code written by perhaps unskilled developers. Or it could be nothing wrong with your system. But the rules are imperfect. You saw how complicated they are, and they might miss something, and they might label a request as an attack when it is not. Job zero is a secure application. That is your job. You must have somebody on the team who is responsible for security. Doesn't mean everything is secure, but you can go to that person and ask what's the situation, what are our main vulnerabilities? And then every team member must have some degree of training. After all, you take care of performance in your code, you take care of usability. At the same time, you have to always be thinking of security. I can't define how to secure your app in a few minutes. It is a complicated, a very complicated profession. But here are a few simple ideas. One is to escape all strings, so the top string, as you know, is a potential attack. The lower string will still be rendered visually as the top string, but is no longer interpreted as JavaScript. Or there's a function in the react framework, dangerously set inner HTML. Yes, the react team was very worried that you would use it. They're telling you not to use it. When developers are under stress and it seems to work and the other one doesn't work, they use it and they will insert, let's say, JavaScript without escaping. You can try sanitizing, which means deleting anything that looks like an attack script. But I told you a minute ago, why deleting those all could create false positives. The only way to handle this is with a security mindset. It must be in your mind and in the minds of your team members at all times. That's what they're paid the big salaries for. Let's take another example of false positive DDoS. So we're looking for big spikes which probably indicate an attack. So here we see that there are gigantic numbers of requests that come all of a sudden on a low traffic website. And maybe the DDoS system should block it. But oh no, that is the Australian Open tennis match. And if you block those spikes, you've blocked the only reason the website exists all year. There are ways around that. First of all, by planning in advance and turning off the DDoS protection. Perhaps another is to rely on advanced machine learning driven features which do exist in the wafs. IP addresses also can bring false positives. It's just a number. Do you know who's using it? Maybe the numbers behind an NAT and shared by hackers and non hackers. Does your product manager want you to block even r1 customer who in a b two B situation might get very angry to you, get very angry that the system is blocking them. Do you want to block an entire country and completely deny yourself customers from that country? One way around this is with a dry run, so you can set the WAF to preview, and then even though the action is in principle deny, you will only get logs like this log. This is an abbreviated log and it will basically say we would have denied the request. And you can look at that and say oh, we really shouldn't have denied it. My code is awful. It passes JavaScript on purpose, or we have to accept that JavaScript here, or we have to change our code. And therefore you will not endanger yourself with those false positives. But the problem with preview is uncertainty. I have seen teams turn on the preview and wait for months in which they're paying a lot of money as they try to figure out if given strings that trigger the Waf are really a danger, not a danger, and they are scared of blocking because they don't know their own application backwards and forwards. So this is not a cure all. But in addition to false positives, you could also have false negatives. They just let the attacks through. Reg exes, as you learned back in college, are limited as a language. They cannot catch everything. Besides which, they have to be fast. Remember, every web request is being scanned by potentially hundreds of Reg xs before it even gets through. And for that reason also, only a few kilobytes can be scanned by these reg xs. And that means that any attack after those kilobytes will simply be let through. But the worst thing really is broken access control. If you left some of your pages open to unauthenticated users, to all authenticated users, or to the wrong authenticated users. If you left them open to write for users who should be only read only, the WAF is helpless. It doesn't know what you intended when you wrote your application. And the attackers can shift around too. They can change their IP addresses using a VPN or whatever. They can also change the country they appear to come from, and they can change the attacks. Tiny string changes may get through the regular expression while still carrying an attack. They are smart, they are human, they are always scheming, and the WAFs have predetermined rules. There are new machine learning systems that are flexible, but they are not magic either. Well, what if you leave yourself some flexibility? The Wafs do offer that you can configure the policies. You can exclude a rule from the policy because you think it's making false positive detections exclude a field, not scanning a field. But then remember, you're not scanning the field. Raise or lower the sensitivity. You can create your own rules. That looks good, but I suggest you don't do that because the experts have already worked on the tough balance between false positives and negatives, and if you try to reduce the false positives, you may be increasing the false negatives. You don't have a very special case you think you do, but in fact these attacks are most often widespread across the industry. On the other hand, if you have some deep hackers, then probably nothing you do is going to help. If state level hackers are coming in, your tweaking is not going to make the difference. If you do have a very special case in your code because your code does something weird, probably you need to take a good look at your code, not tweaking the WAF rules. But the WAF is not just imperfect, it may make things worse. Remember, it looks at your decrypted messages of each HTTP request. It has to to see what the risks might be. What if the waf has a bug? The biggest risk though is complacency as the temporary becomes permanent as you don't learn new skills. I compare a WAF to a jewelry store with giant holes in its concrete walls, and then the jewelry store owner puts a chicken wire fence around and says, well, it's not perfect, but at least we have a fence. Yeah, maybe you should be looking at those big holes in your concrete walls. Besides all that, the WAF risks performance because it has to run those rules on every single request. The more rules to run, the more of a risk to performance. If we talk about the cost of a system, we should also talk about its cost. And I'm not going to talk dollar amounts. You can look those up, and plus they're different wafs. But generally in the basic form, you pay for each rule you define, each policy you add, or for a very, very large number of requests. As with many things in the cloud, so long as your systems are still under test, so long as you haven't gone viral, these costs are negligible. When you go big, you can get an enterprise system in which you pay a monthly fee and not pay on demand. There are variations of this and pay perhaps a lesser amount if there are massive amounts of resources going through. The real risk is if you commit to pay a monthly fee and then find that there are so many false positives and negatives that despite the risk, your product manager tells you to back out, then you are paying a lot. I've said all these bad things about Oauth. What is it good for? If you have external requirements, like I mentioned, you have no choice. Then it's not a security choice, it's a compliance choice. Go ahead. If you are running a third party app, perhaps a commercial system where you cannot alter and fix the code, perhaps open source where you don't have the skills to fix the code, then you can put the WaF in front of it. You still risk complacency. The WAF isn't magic. Another use case is central supervision. So you have a security team in a big company. It hears about a new threat CVE, like the famous log four j, and it steps into action and uses the WAF to block it, meanwhile shouting very loudly at the product team. Now the immediate response should be, you know, is the product team really going to fix it? But in this case we're seeing the WAF as a monitoring tool. And if we choose deny, allow or preview, then that's actually to help the central team understand what's happening, temporarily control it, monitor it, and not just tune the protection per se. It's part of a process across a large organization. Another reason to use the WAF is if you absolutely know what you're blocking, you know there's a hacker coming from a specific IP, or you have a business partner and you want to whitelist your IP addresses and let no one else through, and you've chosen not to use a normal firewall. Just some examples. When you know what you want to block the one go to feature. So here it's pretty straightforward. I do recommend OAuth. There are other services that could come in useful. You can pay to have a human team on standby from Google Cloud, from Amazon and so on. You can get event services where you see the attacks as they increase over time. Let's say they come in waves where they provide third party IP address lists, for example for tor nodes, and where they adapt the protection using machine learning in a more flexible way than in the basic services. Regardless, if you are going to do a WAF, let's say for ddos, do it now, not doing a fire drill. I suggest you prefer the WAF offered by your cloud service provider, Google Cloud armor, AWS Shield and so on, because the HTTPS is probably being decrypted at the load balancer in typical architectures. So you don't want to decrypt it in two completely different vendors and they offer pay as you go. Some of the other vendors do too. But the importance of pay as you go is that your ramp up period may be far longer than you expect. In summary, the minuses of of are first and foremost complacency that you will not put the resources on it, you will not develop the skills, you will get false positives and block your own application. Product managers are willing to take risks or false negatives, and you'll let the hackers through even though you paid all that money for Oauth. And they do add risks and slowness of their own. However, the pluses are that they're easier than do it yourself. You don't need to change your code and essential tream can control and monitor and understand security through the WAF. There are also advanced features that you probably can provide yourself, like the machine learning driven adaptive protection or DDoS protection. In conclusion, security is job zero. Your job in your app. You cannot hand off responsibility to anyone else, but there are some cases where a ref is relevant. You can see my slides here bit ly waf don't waf dont. Or you can scan this QR code and see the slides. And we are hiring. I love my job, so please email me there whatever you want to discuss.