Hi, I'm Brian Contos. I'm the chief security officer with phosphorus cybersecurity. And this presentation is cameras, caCs, and clocks. XioT security sucks, a story of 2 million interrogated devices. So let's go ahead and just jump right in. So, first off, what is XioT? So fundamentally, IoT is a combination of three disk but overlapping areas. The first is what we might consider enterprise IoT, or Internet of things. Those are things such as printers, voiceover, IP phones, KVM switches, ups, any kind of sort of enterprise dumb, if you will, device that's operating on the network. In addition, two that those are network devices such as network attached storage, wireless access points, load balancers, switches, so on and so forth. And then finally, it's the OT or industrial control system side SCADA devices. These are things like plcs or programmable logic controllers that typically operate in batch and discrete manufacturing oil and has production power and energy and many other sort of utility or critical infrastructure type organizations. Collectively, we call those IoT or attendee Internet of Things. Now, these are purpose built devices, typically with specific hardware and software. They're usually running operating systems we're all familiar with, like Linux, Android, BSD. Sometimes on the OT side, it's a real time operating system like Vxworks. They're network connected. Not all OT devices are network connected, most are these days, but certainly on the enterprise IoT and on the network device side, hence the name network, they're network connected and they can't run endpoint security, which is a big one, right? I mean, we talk about our servers, our rotations, our laptops, a critical database, whatever the case might be, even in a virtual machine, we're going to have some of endpoint security, anti malware, intrusion prevention, localized firewalls, so on and so forth. These devices, you cannot do that. So at Fosfer cybersecurity, we've been researching this space for over five years. We've looked at millions and millions of devices across public companies, private companies, healthcare, financial services, military, government, you name it. And in doing so, we found some really interesting sort of trends and statistics and use cases, as well as a group of devices that we like to call the devices that suck, not because they sucks in terms of how they operate, but their security certainly is lacking. So hence the title for this presentation. But we'll get into that in just a little bit. So I want to share just a couple kind of interesting things here. I recently went on showdown, and I think most of you are familiar with that. It's like a Google search, if you will, for devices that are connected. Two, the Internet. I just typed in words like camera, voiceover, IP, phone, printer ups, things like that. It's not a very scientific study, but I just wanted to see roughly how many of these devices weve exposed. And there's probably some percentage of honeypots and some percentage of mistakes there. But even if it's 5% or so, still we have about 5 million cameras. We have over a quarter million voiceover IP phones, printers, about 83,000. And I have no idea why, but over 13,000 ups systems connected. Now, IoT devices don't need to be Internet accessible to be attacked and manipulated and used for a number of nefarious things, but it's certainly one way in. In fact, most attacks we're seeing today, interestingly enough, like a phishing attack, they're going after someone's laptop, and then once they get in through it, they're looking for XIoT devices to IoT to. And then from there, they're attacking both XIot and it assets, and even cloud assets from those XIoT devices because they can maintain persistence and avoid detection. But we'll talk a little bit more about that in a compiled of use cases in just a moment. But a funny thing about this, maybe it's not too funny, but take upss, for example, so uninterrupted power supplies, generally speaking, if you have something plugged into that, it's because it's really important because you don't want the power to be interrupted. And one of the most popular brands is APC. It's a very common ups that we see out there. Now, if you go onto Google and you just type in default password for APC ups, you'll find very quickly it displaying, saying, hey, the password is APC and the username is APC, and they're both lowercase. I've never seen an APC UpS system where the password is changed. In fact, we have a running joke at phosphorus. That is, if we ever actually find an APC ups system where the password and username aren't APC, APC and lowercase, we'll buy everybody in the company a steak dinner. We've been eating a lot of chicken. So all this to say that one of the easiest ways to get into a lot of these devices are, number one, is it Internet accessible? And number two, does it run the default creds? That's all the hacking is involved, if you can call it hacking. Now, nation states get this, cybercriminals get this, too nefarious operators have taken notice of XioT over the last few years. And a great example of this is Russia. So Russia, the russian FSB actually hired some contractors to build an XioT tool that would actually go out and discover XioT devices, compromise those devices, and allow them to be controlled for multiple nefarious things, for spying, for distribution of malware, for ransomware attacks, to add to botnets, and again, to my point earlier, to actually use to attack it assets. In a lot of these cases, it's used to attack IoT assets to then exfiltrate sensitive data. So these Russian FSB had this XIoT hacking tool built. It's called frontin. Now, frontin is a military grade, nation state built XioT hacking tool. Unfortunately for Russia, the digital revolution hacking group got wind of this. They actually stole a copy of the tool and they released it online. So if you go to some of your favorite torrents and places where you get such tools and you can read Russian, you can have access to a nation state grade, military designed XIoT hacking tool available to the world. So again, nation states are taking interest in these types of tools. Another example, nation states, is China. So China actually kind of skips the middle ground. So instead of having a device to go out and find IoT devices or malware to compromise those devices, they actually manufacture the devices with the malware pre built. So it's already inside you. Just skip that middleman. And I'm not calling out China or all chinese companies. There's just a few companies that certainly have been identified here. In fact, in the United States, weve actually passed a law that prohibits the use of many of these devices in government organizations and with government contractors. Those are things like Hickvision, Huawei ZTE, and a few others. And some of the problems with a few of these, and I'm not going to call it the specific brands, but some of them have a little light on them that's green or red. So green means I'm recording video and I'm recording audio. Red supposed to mean I'm not recording video, I'm not recording audio. Pretty simple, right? But what we found in some of our testing is when you actually say, stop recording video and stop recording audio, it does turn the green light to red, but it still continues to record audio and video. Furthermore, it's piping that information out to some location remotely, which theoretically is making its way back to whoever designed this malware in the beginning, this organizations in China. So these are architectural capabilities that have been built into these devices specifically to spy on organizations. And a lot of the compromised that you see in Iot or a large number of them, are about spying. I can see you, I can hear you, I can use this to capture data. You see other attacks that are more physically based, like, I'm going to shut your power off, stop your elevator, modify your HVAC system, open all the doors to your building. Those are kind of physical arena attacks. And then there's the other attacks. The ransomware compromised 10,000 devices so you can do cryptojacking. So mining cryptocurrency. Actually, a Iot of organizations sadly detect that these 10,000 security cameras have been compromised and been uploaded with a crypto minor, not because of some great cybersecurity tool or some xiot security tool or some great incident responders. It's their power bill. Their power bill is a lot higher because, as most of you know here, if you're mining crypto, it takes a lot of power and energy. So certainly attackers like that. But again, the biggest ones are using these devices to attack it assets and hide, maintain persistence and evading detection. These allowing me to attack other tools. So this is just one example. China is not the only country that has devices like this that are doing nefarious things, but it was interesting that Russia built a tool specifically to attack these tools, and China ships the tools already with nefarious capabilities built in. Now, going back, one of the cameras here that's been banned is hickvision. So we already know hickvision kind of makes a malicious camera. Well, in addition to that just announced, and this was on August 23 of 2022, over 80,000 hickvision cameras were found to be exposed online. So just like we saw in that search that we did for cameras a little bit earlier, using showdown 80,000 cameras that were hickvision specifically that were vulnerable to a critical command injection flaw. CSO, not only do they ship with kind of malicious design, but they also have vulnerabilities on them, too, that can be exploited by other attackers. In this case, it was discovered that over 2000 organizations across about 100 countries had not applied the security firmware patch that had been available all the way as far back as September 2021. This doesn't surprise me at all, and I'm going to share some statistics with you in a little while. But a lot of the firmware that we come across is like six, seven years old, and a lot of it has actually been end of life beyond six or seven years. So the fact that this patch was around for over a year, we think of that for maybe a mobile phone or laptop. Oh, why would you wait a year for a lot of these devices, they're simply not being managed. So I'd be remiss if I didn't talk about the Marai botnet. If I'm talking about Xiot security, this was kind of the Godfather, or the grandfather, if you will, of Xiot attacks. IoT started back in 2016. So that's why I call it these grandfather of these attacks. So what was it essentially? So people were using Shodan or other tools to find Internet connected cameras. At least that's where it started. So once they found the cameras, they said, furthermore, I want to see if they're running the Cleartext protocol Telnet. Okay, so I found the cameras. Now they're Internet connected, they're running Telnet. Furthermore, I want to see are they running default passwords or maybe one of know, eight or so pretty common passwords that we run across. And sure enough, they were able to find these cameras. Well, just with those basic searching. And remember, we're not hacking anything. We're looking for a camera that's connected to the Internet, that's running Telnet with a default or a relatively common password. That's it. No hacking involved, nothing was exploited. Just creating a botnet predicated on that set of cameras. They're able to, nefarious users were able to use the Marai botnet to control these cameras, to conduct DDoS attackers that were very impactful on big companies with big networks, right, Reddit, Netflix, Sony, PayPal, Twitter, GitHub and a few others, especially some big telecoms as well. Internationally, these, it took them down just a few little cameras, right? But the cameras collectively had massac process and network capability. But this is what's even more interesting about Mariah Botnet, because there's a lot of white labeling and shared libraries in the Xiot world. That means the build that went into things camera might be very similar to what was used in this printer or this voiceover IP phone or this audio video equipment. So they're being explored running Telnet with the same default passwords, maybe the same vulnerabilities and things like that. So even though I'm sharing your library, that's allowing me to do ABC, that library also has vulnerabilities, XYZ. And I'm sharing that and sharing that and sharing that. So what we found is even though they started with cameras, this expanded was past cameras and started including printers and phones and other devices as well. What's also interesting about this is it was way back in 2016. We still see devices today when we're out at customer sites and we're all over the world. So this isn't just specific two, one country or another that are still vulnerable to these. Marai botnet back from 2016, they're still vulnerable. And we're even finding devices that have Mariah Botnet still on them because they've never been fixed. Hey, as long as the printer is printing or the camera is recording, we don't really know how to manage it or who's managing it or what to do, so just let it do its thing. They're not being monitored, they're not being managed. So again, maintaining persistence, evading detection is a big deal. This is back in 2016, and things really haven't gotten any better. So this is a little bit more werent. So this is can attack. That was announced by Mandian called quiet exit, and this was May 2 of 2022, so pretty recent. And this is all about achieving that persistence from this apt. So what these attackers did is we feel they got in through phishing attacks, again through a laptop, social media, whatever the email or whatever it was, they got in through some type of attack that got into someone's laptop. We all know there's a gazillion ways to do that. So once they're on the laptop, they started looking to pivot to XioT devices. Why? Because no one's looking at them. Also, they generally have default passwords. If they don't have a default password, they're probably running in an old firmware with lots of vulnerabilities that could be easily exploited. A lot of these guys, we see levels, eight, nine, and ten level cves on a score of one to 1010 being the worst. So in this particular case, once they got into the environment, they started looking for these IoT devices, and they found some, and they were looking for some network devices, wireless access points, load balancers, NAS switches. A lot of these systems run BSD, and they're also looking for traditional IoT devices. Voiceover, IP, phone, security cameras, printers. Most of those run Linux, some of those run Android, but very popular operating systems. So they recompiled, reconfigured their solution. And their solution, the attacker solution, was dropbear. Ssh. Nothing too crazy about dropbear, but they installed it on these particular devices, which allowed them to have a reverse SSH tunnel, CSO client, server, communicate and control these devices remotely. So typical command and control, c two tunnel back and forth over SSh. But they were communicating with these XIoT devices, which made it a little bit more novel, right? Once they got onto these devices, they used those devices not to hack really, but to make API calls to the local exchange server and the Microsoft 365 in the cloud. So both on Prem and in cloud, API calls were made to extract sensitive information. And what was the sensitive information? Email between corporate executives, corporate development folks think m a think BD people, and even security staff. So they Iot in through it, pivoted to XIoT, used that to attack both cloud and on Prem. It exfiltrate the data. And the worst part of this, in most cases, these guys maintain persistence and evaded detection for over a year and a half. Again, because no one's looking at these devices, no one's paying attention. The bad guys know it, the nation states know it, the cybercriminals know it. So they're really taking advantage of these. And I can almost guarantee, as we all know, there's probably plenty of organizations out there that have been compromised with this, where data is still being exfiltrated as we watch this video. So another one I wanted to share with you was a russian xiot botnet. And this was a takedown. So this was a success case. The US, the UK, Germany, Netherlands, and actually a couple other companies or countries working together took down a botnet. And as we all know, botnets are kind of like DNS, right? Or a Christmas tree. It's very scalable. So you take out a piece here, a piece here, maybe it pops up someplace else, but if you get to the top of it, you take that down, maybe you've taken it down now for a few weeks or a few months, you're not going to take it down forever. People can rebuild someplace else, but it does cost time, money and resources for the bad guys. So it is a success case. But the reason I wanted to call this specific one out, it's called Rsox, was pretty interesting. Primarily it targeted in the IoT realm, the OT devices, those PLC skated devices that we talked about. There was some network gear, and there was some IoT gear, just to be fair, truth in advertising. But the majority of it were industrial control systems, programmable logic controllers, these types of things that are digital equipment that operates physics, flow, pressure, voltage mixtures, robotics, arms, things of this nature. And again, these are usually real time operating systems, Vxworks and things of that nature. And there's some other more esoteric ones that are out there. And what they were doing was, it was so successful, they were saying, hey, we're going to rent this out. We're going to rent this out to people for about $30 a day. I actually think if you pay, I think, $100 a day, they give you access to the broader network, and they also provided technical support, which was, they operate these things like a real business. Right. Obviously, they're money makers. But I thought, again, what was interesting about this, the reason why I wanted to share it, was primarily based on those OT devices. So they're not safe from this either. And again, it wasn't attacking the OT device to break manufacturing of a vehicle or screw up batch manufacturing of some chemical or some pharmaceutical or blow something up on an oil pipeline. It was simply adding this device to a network of botnets to use it to attack other organizations. Now, could they have been used for what I would consider much more nefarious things like blowing up a pipeline or screwing up a medical batch that's being produced? Absolutely. They had control of these devices, but in the cases we saw, they were Iot being used for that. Another example on the ot side, and again, ot makes up one of the these pillars. Of course, for XioT, this was an attack on Siemens. And how we think of attackers on the it cyber side today is a little bit different than these because they're really, really based. So basically, if you sent packets to this specific device, and this was called the s seven plus crash, that was the name of the attack. You send packets over TCP port 102 to a remote device. It's unauthenticated. Not most, but a lot of these devices operate with no authentication. CSo, you use any kind of tool you want, Netcat, whatever it might be, to send packets port 102 to that device, it causes a DOS attack. Now, some of these devices have what are called set points. Like my temperature can go up to 75 degrees and down to 65 degrees, and it has to stay between those two. If it drops too low or goes too high, it means it passes that set point, an alarm goes off, tells my ScadA system, hey, something's happened. Take a look. Well, if you increase the temperature of something and you're dosing it IoT doesn't have any time to respond, it can't send out the packets because it's basically sucks. So maybe now the temperature is at 100 degrees instead of 75, but it can't do anything about it because it's stuck. It can't communicate that an issue has happened. The reason I want to bring this attack up, yes. It's extremely simple. Right. And it can have a big impact as well on the organization. But Siemens came out and said, hey, here are some suggestions to fix this. I like to show this because in the XioT world, XioT security today is kind of like iT security was like back in 1995, we didn't really know where all our devices are. We had kind of poor discovery, we had poor patch management and software update. Management just wasn't as broad as we would have liked or as well adopted when it came to credentials and password, rotation, complexity, things like that. It was all pretty early stage, but this is what Siemens came out and said, look, for this attack. This is what you should do. Number one, update your firmware. You should be running more recent firmware. Number two, enable access controlled. Consider using a username and password. Number three, set a password. So don't just enable it, but actually set one. And I would say set a good one and make sure it's rotated. And all the things we all know also disable unneeded protocols. A lot of these devices speak, a lot of extra protocols they don't need. Certainly on some of these devices, they probably speak TCP, IP, maybe serial over Ethernet, some more proprietary stuff like Modbus or DNP. Three, they've got wired, they've got wireless Bluetooth, Bluetooth, low energy. So some of them are actually very much what I would call hyper connected. But I just thought it was very interesting. Hey, patch your firmware, set a username and password, and get rid of services that you don't need. Again, it security kind of 1995 CSO. In our research, again, we've been doing this for a little over five years. What we found on average is there's about three, two, five XioT devices per employee in the organization. So a 10,000 person organization probably has somewhere between 30 to 50,000 XioT devices. Now, when we go into an organization, at the beginning of a proof of value, a POV, and we say, hey, take a guess, what do you think you have? Almost consistently, they're off by 40% to 60%. These say, hey, Brian, we think we have 20,000. In the back of my head, I'm thinking, okay, so they got about 40,000 devices, and that's about what it turns out to be. Now, there's a bit of a curve on this. So, for example, a law firm, they're going to have a little bit fewer. A retailer or somebody that's working with industrial control systems, they're probably going to have a lot more. So there are some differences, but on average, what we see healthcare, financial services on and on about three, two, five, devices per employee, and it's always a lot more than people think they have. So what percentage of Xiot devices operate with default passwords? So think about that. What do you think it might be? It's actually about half. About half the devices operate with default passwords. As I just said. Go Google. Default passwords, APC, UPs, it's no super secret. And it's like that for a ton of these devices. So you don't need to hack anything, you just need to log in. So whether it's Internet accessible or you're already within the internal network, it's default password. Now. Forget rotation frequency every 30, 60, 90 days, forget complexity, even when the passwords are changed, and sometimes they're changed at the point of deployment. For some newer devices, it's usually changed to something pretty weak and usually not very long. When we say uppercase, lowercase numbers, special characters, at least 20 characters, passwords, not password. All these things that we talk about in good practices, they're not happening here, right? And there's a bit of a curve to this one has, well, we say 50% on average, but upS systems, as I mentioned before, that's closer to like 100%. When we talk about audio, video equipment, we find that's actually in the high are some devices, again, where the password is changed. Two implementation. We don't see that as much actually on the enterprise side as the consumer side. We focus mostly on the enterprise side. But in those cases, when they're changed, sometimes they're just changed to what the product is, security camera or the name of the company. They're pretty basic things. It's certainly things that could be added to a brute force attack pretty easily. But default passwords half the time. So think about this one. What percentage of xiot devices operate with end of life firmware? Again, these are devices where the firmware has been dead. It's like running windows nt three, five one, or Windows Nt 4.0, which actually, I bring those up because we do see those on the critical infrastructure side, because they depreciate the servers that operate the turbine at these same rate. They depreciate the turbine, which is over decades, Iot over like five years. So the answer to this is 26, werent. 26% of the firmware is actually end of life. And the remaining 74%, the average age is about six years. Could you imagine your smartphone? I'm sure all of you watching this have some kind of smart device. If you didn't upgrade the underlying operating system for six years, or even the apps that are running on top of it. Honestly, it probably wouldn't even work. You probably couldn't even connect to the environment you wanted to connect to. Everything would just be broken. But these devices on average is six years old, which means it comes with a lot of vulnerabilities. So think about this. What percentage of IoT operates with Endeli firmware? We said that was 26%. Look what comes along with this. We're seeing that about 50% of the devices have CVSS scores. Again, one to 1010 being the greatest. CVS scores of 850 percent of these have level eight. That means they're very hackable. Can additional 18% have nine and ten? That means very hackable with almost zero skill remotely done. And to give me full administrative access to that device, that's 68%. That's almost 70% of all the devices have level eight, nine and ten. That's crazy. If I just told you your it assets had that, you jump off this video right now and go fix them. And again, I've got in a company of 10,000 people, I've got 32 50,000 of these things, half of them with default passwords, old end of life firmware, and 78% or 68% of them have level eight, nine and ten CVSS scores. What? That's insane. It's not even fair, right? This is really making things easy for the bad guys. So let's talk about the biggest offenders. Now, I dont have them all listed here. I'm not calling out a specific brand or model number or anything like that, that wouldn't be fair at all. But I did want to share some of the devices that we find time and time again that are just, they're super vulnerable and they're being exploited at a very, very high rate. So the first one, KVM switches, suck. Now I love KVM switches, stands for keyboard, video and mouse, as most of you know. And the idea is I've got one of these switches with one keyboard, one monitor and one mouse connected to 510, 50 different devices, whatever the number might be, and you find them a lot of racks and data centers and things like that, and they allow us to be extensible. The problem is, and these run Linux like most of these devices do. Linux is by far the most popular operating system we see out there for these XioT devices. These run Ubuntu. They run Ubuntu version ten quite often. We're on version 21 now, right? So it's a little dated. It's about a decade old and they're shipping and they're shipping with tons of vulnerabilities because it's a decade old version of Ubuntu, right? And you can go find out all the vulnerabilities that are on bunto version ten that come by default on these devices. Now the problem is this is a device that manages devices. So if you can get access to the KVM switch, you can cause a lot of problems on the devices. Managing network changes, power down the device, make other configuration adjustments, et cetera. Next one, lights out management. Controlled suck things is kind of like KVM on steroids, but for a point device. Now that little arrow there is pointing to what looks like an Ethernet port, but it's actually a lights out management port. What this allows you to do is access that device. By the way, that's a Linux. It's running Linux, right? It's nothing crazy. You might have heard terms like iDrac, ipmi, Ilo. It depends if it's HP, Dell or supermicro. Those are the big ones that you see in this. But it allows me to get access to the device. What's interesting about this is I can actually open up a shell. I can spawn a virtual terminal. I can actually even upload software or malware. These are very simple areas to attack. These get plugged in, nobody updates them. A lot of times there's no password, they're vulnerable. It's just giving me full access to actually do malicious things. And people don't think about it. When I say, how many devices do you think you have? Nobody's thinking about KVM switches or lights out management. And this is why. Server cabinets and racks. Here's another thing. A lot of these are smart cabinets and racks. They've got tamper detection and temperature controls and cable management power, a whole bunch of other capabilities. Again, these devices have all the same problems with the other devices we talked about. Old firmware, default passwords, extraneous protocols. These problems with this particular group of devices is, generally speaking, if you want to update the firmware on a rack, you have to do a power cycle, which means everything that's connected, two, the power supply there has two, then be cycled. So when people are scheduling downtime to update servers and databases, web servers, critical assets like that, not really taking into consideration the racks that kind of manage these devices and that they operate in, so they never get updated, which means they're highly vulnerable. You get access to a rack now, and you can do nefarious things to all the devices within that rack. Physical access controllers suck. This is a big one. We actually did a pov with a very large financial services company where weve able to, with no hacking, we were able to access and say we can open and close 6400 doors at our discretion, including things like the front door. Right, and the back door doors that are probably pretty important for them. But 6400, forget all the cybersecurity you have in place. If I can just access this system and make some changes. There's a very popular version of physical access controlled called nortech security control. I don't want to pick on them, but it's pretty well known that they were shipping with CVSS scores of 9.8 out of ten and ten out of ten on their devices. So has you unpackaged it? You took out of the box? You took it out of the wrapping paper. Oh, it looks so great. You're all excited. You plug it in, it's got level ten vulnerabilities on there, just right out of the box. So craziness. And this is allowing people to do really malicious things, because we talked before about shutting down power, stealing intellectual property, doing cryptojacking. Well, this is actually physically unlocking, or maybe during our emergency, locking doors that shouldn't be locked. So there's a lot of crazy things that could happen there with physical access controls. Another one are printers. Everyone's got them. They're super promiscuous. So printers really suck. And why do they suck so bad? When I say promiscuous, what I mean is they want to be connected. Two, so they're running wired and wireless and Bluetooth and other communication protocols. And you can connect to me via HTTP or HTTPs or SSH, or Telnet or FTP or whatever it is. They've got all these ways because these want to make it easy. And that makes sense because at the end of the day, I want to use my printer to print. The issue that we found of these guys is a lot of them are running like, they're mostly running Linux operating systems, but they've got like 60 to 80 gig hard drives, not huge, not small at today's measure, but it's a pretty big hard drive. When they're being compromised, they're being compromised because of all the problems we talked about before, default passwords, vulnerabilities, old firmware, but they're a great place to hide. And because of that big fat hard drive on there, they're extracting sensitive data. We actually saw one customer where they were pivoting from hundreds and hundreds of printers that have been compromised, attacking it assets, downloading the data compressing it, exfiltrating IoT over ICMP because no one's watching that. Some of these cases longer than a year, right? We work with hotel chains that have 20, 30, 40,000 printers, so it's not uncommon to just have an astronomical number of printers out there. And if they're all or possibly a great majority of them can be compromised and used to attack you. That's a huge attack surface, right? That no one's looking at, no one's managing, no one's monitoring. Black has, back in 2019, they announced the discovery of critical level vulnerabilities on over 10,000 different types of printer brands. Again, it's not surprising at all, but it's actually one of the most targeted. Not the most, but one of the most targeted XioT assets out there. Voiceover IP phones, video conferencing systems, I kind of put those in the same group. Those are usually Android OSS. But what's interesting about these guys is we saw some commonality between some video conferencing systems and some voiceover IP phones because of that whole white labeling and shared libraries we talked about before. So one of these phones, it's a very major vendor. If I said the brand, you'd go, oh my God, they shipped with their phone running SSH undocumented. It was never written that they even had it with default username and password. The problem with that was they didn't even have a security development lifecycle mature enough to say, hey, let's figure out what ports are running on this thing before we ship it out. And oh, lo and behold, it's running Ssh. What kind of password set on that thing? So that's pretty grievous. And I don't want to call these out and say they're doing a bad job, but when it comes to security, it's not their primary goal. They want to get to market, they want blinky lights, they want it to do cool things for the end user. And I get that. But if it's in an enterprise and opens up these vulnerabilities in the back end, of course, that's a huge risk, right? That's something that we need to concern ourselves with. I bring up the video conferencing cameras as well. Again, because of the white labeling, these shared libraries, that exact same build was existing on these video conferencing systems, which again, we talked about spying before with the cameras. These devices can be used for that as well. We've actually seen instances in production customers where those devices, in fact, were being used for that purpose. So again, these devices aren't really being built with security development lifecycle in mind. So I've covered a lot of devices here, but I haven't covered the number one. So think about what do you think is the number one biggest offender? It's security cameras. Security cameras suck. The know, security cameras are great doing their security camera thing, but we talked about some of the ones out of China that have actually been based. Certainly they're bad. They often run operating systems like busybox and other forms of Linux out there. Sometimes, again, they ship with the malware. There's exploits to take advantage of them, like we saw with the Hikvision camera and all those Internet accessible devices where you could run remote commands on these devices. But the biggest problem with security cameras is actually this. You have to understand that. Are they a dumb device? Yeah, they're a dumb device, but they have the same storage and processing capability and memory and input output capabilities as a laptop. Some of them actually even more. Some of these are really quite heavy duty. Consider them like workstation level security cameras. They're very powerful, so don't think of them like as a weak device. They're actually a very powerful device. These just happen to be an insecure, very powerful device running an operating system that allows you to do all the same things you could do on a laptop. So just think of it. That was these other thing with security cameras is most people don't have in the enterprise a handful. They have thousands. In some cases. We've seen tens and tens of thousands. We're working with a major casino that actually has over 50,000. 50,000 security cameras. Right? So the worst part of it is this, when we're talking to these organizations, two say, hey, here's the problem, we can come in here and help you. Who manages these? Oh, it's corporate security thinking. They're like, oh, no, it's network operations. Not us, it's the Iot sec. No, no, it's a third party vendor. So it's kind of like the end of Spider man. Everyone's point, who is it? Is this guy? Is this that guy? So nobody wants to take responsibility. And I get it because historically there was no way to really fix these devices at scale safely. These, you're going to have somebody, a football team of people with a paperclip to go do a physical reset to update these firmware. That would be nuts, right? You'd never do it. So if you can, what happens? It's just not done. And that's what the bad guys are counting on, that you're just not going to do anything. Now, weve mostly been talking about Enterprise Xiot a little bit on the industrial side with OT, but I want to call it just a few other areas before I go deeper. There's also Internet of battlefield things. These are tied to all the military devices out there. There are certainly things that are specific to the industrial side. We kind of got these tip of the iceberg there, but there's a lot of very specific purpose built devices for batch and discrete manufacturing. Oil and gas, power and energy, water, traffic, sanitation, agriculture, you name it. That's a really big realm of the OT devices, right? Healthcare also has a lot of specific things. Another big one is things like smart buildings, smart cities, smart ships. Difference between smart ships and smart buildings is one floats. They're all the same devices, to be quite frank. But if you look at this all collectively, it kind of gives you. Okay, wow. Beyond the enterprise, things actually has a global impact, right, on cities and counties and states and countries and regions. Everything is ultra connected. And if you think about these first official, the first unofficial IoT device was back in the 1980s. It was a coke machine. That was an arpanet. Okay. But then we fast forward to the. We had a TCP IP toaster that was connected. That was the first official IoT device. Well, Dublin, Ireland, is the first official smart city. So within 25 years, plus or minus, we went from a toaster to an entire city. So everything I've talked about here and everything I'm showing on this slide, this is only going to expand exponentially. Everything everywhere always is going to be connected with these devices, and hopefully we can get them beyond 1995 security. So I've talked a little bit about, or a lot of it, about sort of the bad side of this, right. The use cases, the things that suck the most. But let's talk about getting rid of that suck. Let's talk about making it so it's not easy for the nation state hackers, the cybercriminals, malicious insiders, so on and so forth. So what can we do? Well, the first thing is discovering, as we talked about before, three to five XIoT devices per employee is a lot. And the fact that if you don't know where even half of those devices are, that's a big problem. Now, old school discovery solutions, and I say old school because they're really based on it and it centric adaptations. You look at like a vulnerability scanner for discovery. They're going to send malform packets and kind of see what the device is going to come back as. That could be a tenable, a qualis rapid seven, great for it, not really fantastic for IoT. They can actually crash systems. You don't want to scan in an OT environment, for example, sending malformed packets to a PLC device from the 1990s, you got about a 99% chance you're going to crash that thing. In a world where availability is everything, right? So the traditional scanners don't really kind of work. Then you say, okay, well, what about packet sniffing? Well, sniffing is okay. You Iot, a lot of span ports, a lot of taps, there's a big network out there, but also a lot of the communication is encrypted, so you're not really able to glean a lot of metadata. You might be able to say, I think it's a printer, maybe even it's a HP printer, but I can't really get deep into it from sniffing to really give granular detail. So that's kind of lacking. And then there's traditional asset management solutions, again geared for it environments. A lot of them look at Mac addresses. So the OUI, the organizationally unique identifier. Oh, you're a jet direct. You must be a printer. Well, could be. Could also be a phone. Right? So those are great again for it. Not so great in this world. They have a lot of lacking. So there's a new kind of based of solution. I'm going to share some of the vendors at the end of the presentation, but it's called enterprise Xiot security platforms. And the way these usually work is they actually communicate with these devices. They interrogate these. Think of like c three po from Star wars. He could speak like a million languages and even water evaporators and things like that. Very cool. So being able to not can the device, not sniff the device, but communicate with that device in the way it was designed to be communicated with, being able to scale that for obtain million types of devices, that's really key because now that you can communicate with it, you can extract much more data, firmware model number, serial number, ports and protocols that are operating credentials. All the little isms within that product can be pulled out through that interrogation. So that gives you a very robust sort of starting point. Now I know what I've got, and it's not based on some packet scanner that was developed or based on technology in the 1990s with malformed packets or sniffing. It's actually communicating, interrogated these devices. Right. So that's step one. The other part of this is upgrading firmware and hardening. So now we found the devices, now I want to remediate it. I actually want to fix it. So back in the day, a lot of people said, well, there was really no way to fix it, so we're going to hide it behind a Vlan. Nothing against VLans. I think Vlans do add an added level of security. But, man, if you're a 10,000 person company again, 32, 50,000 xiot devices, you're going to vlan all that stuff off. And Vlans aren't the end all, be all of security. Plus, what are you evaluating? Off. You're vlanning off devices that have bad passwords, old firmware full of vulnerabilities, kind of covering your ears and your eyes and just hoping nothing's going to happen. I mean, think of it this way. Like I'm typing and, oh, I cut my hand, my left hand's bleeding and I really need to go to the doctor and get this looked at, but instead I'm going to put a sandwich bag on it and wrap it up with duct tape. Okay, well, now I'm not getting blood in my right hand or my keyboard, so that's good, I guess, but I still got a bloody hand in a bag, which is really not the best thing to do in a VlaN situation. Your seal is saying, I got all these vulnerable, broken, defaulted password devices, but they're behind a vLan, so it's okay. Not a great approach. It's great to use in addition to fixing your devices, but who wants to do that? You would never do that with your it assets. Why would you do it with Xiot? Now with enterprise XioT security platforms, part of what they do is say, okay, I know what my device is, I can communicate with it, I can also upgrade the firmware. And because of these interrogated process, there's a lot of minutiae that can be extracted. Like, hey, I'm on version seven. The latest version is version ten. Can I go right from seven to ten or do I have to go seven 8910 or let's say I'm on version five and there's a log for j vulnerability. On version five we tell the vendor, the vendor goes, look, we're not going to have a fix for this version six for like another seven months. I can't wait that long. Everybody and their brother is hacking everything with log for j right now. But version four didn't have that. So maybe I want to downgrade my devices until that new version is actually available. So when you do that interrogation, as part of the discovery phase, you actually have that type of knowledge, which is really, really cool now. So now I can actually upgrade the firmware and downgrade the firmware if I need to as well, and do pre flight checks. Two, make sure that, hey, I'm not basing this on a Mac address. I actually know this is an MRI machine, or I know this is a printer, because the last thing I want to do is I don't want to turn my MRI machine that's $2 million into a printer. That's $500 with the wrong firmware. People tend Iot to like that, right? So you have to have very high levels of accuracy. In fact, you have to be 100% accurate. There's no room for 99%, certainly 5100 percent accuracy during discovery, so you can do the upgrading of the firmware. And while you're doing that, we talked about the Siemens stuff earlier with the OT hack by sending TCP packets to port 102. Siemens said, hey, turn off some of the protocols you don't need. Well, in hardening, you might say, you know what, I only want to run wired. I don't want to run wireless on all my printers, for example. And no more bluetooth, low energy, and no more clear text protocols. In fact, I just want to run SSH and HTTPs, period. Port 22 and four, four, three. That's it. Anything else for this particular set of devices, I want to change it. I want to do that for just devices in North America or just HP printers, or just HP printers. Model ABC. One, two, three. CSO. You can be very fine grained in how you push it out, and you can push those out manually, one at a time, or say, I'm going to batch those up into a large group. Right? So again, that's another value add that you get from these enterprise XIoT security platforms. And again, this is a relatively new thing. These haven't been around. It's not like a firewall or IPS or a scanner. It's been around forever. So I found my device, I've fixed the firmware, I've hardened the device. Now I get into credentials and certs. So Pam solutions, things like Cyberark, Thicotic, Hashicorp, some of these others that you see are great. They're fantastic tools and been used forever, and they add a lot of value. They store your passwords, they have rotation policies, frequency, complexity, all that. But they don't speak to Xiot. They can't, and they don't want to have to build that. So these enterprise XioT security platforms that are again, like C three Po. They can talk to all these devices and say, hey, there's 50,000 devices on this network. I'm going to automatically enroll them in Cyberark, for example. And in doing so, I'm actually going to also tell Cyberark, just like I knew the firmware upgrade paths. I'm going to tell you for my credentials. Hey, this guy can only take a four digit pin. This guy can take ten characters, but he can't use a backslash escape sequence. Stuff for like, SQL injection, whatever. Or this guy can go up to 20 characters, but IoT can't use the number nine. We see all sorts of crazy things. I don't know who designed some of these, but there are some crazy isms that are related to that. Well, all that intelligence is put in when you create that policy, automatically enroll that device. So I've just enrolled 30,000 devices. This group falls into this category. This group that, this group that. Then you can apply your whole 30, 60, 90 day rotation, complexity, length, all those variables that you would in the IT environment. But now you're doing it with your Pam solution, which talks to an enterprise XioT security platform, which then talks to all these devices. Very clean, very simple, very scalable, and takes advantage of some of your existing controls. The same thing applies to the certs. You might find out that, especially on the network side, about a wireless access point. That's like TLS version 1.1 or 1.2, go, oh, it's too old, or it's a self signed cert, or it's an explored cert. We see this all the time, not just on network devices, but other XioT devices, too. But wireless access points are a big offender in things world. So by talking to those devices that manage the certs, just like Pam, for managing the credentials, you can go ahead and make those changes and make those updates. So I found my device, I'm managing the firmware. I'm harding, the device, I'm managing credentials, I'm managing the certs. Now you're getting your xiot devices to the level of your IT security, which is pretty incredible, because your solution before was do nothing, cover your eyes, cover your ears, and hope that nothing happens. Maybe Vlan, a couple of your devices, if you can get at it, but probably not. So we found our devices, we fixed our devices, and you'd be remiss if you said, okay, that's it. What you want to do is you want to have those devices automatically reinterrogated, probably on a daily basis. That's kind of the gold standard, right? And by reinterrogation, what I mean is let's reconfirm that years you were on version seven and you're still on version seven of the firmware, and that's the latest version. You did have a great password through Cyberark, and that password is still being managed. But what if someone walks up with a paperclip and does a hard reset? I was on seven, but now I'm on version five. I did have a great password, but now I'm back to the default password, ABC, one, two, three. Well, now you can say across my 50,000 devices, these are the five devices I have to take a look at right now because they've drifted, either because somebody did a paperclip attack, paperclip hack where they've reset the device with the little black button on the back of these XIoT devices, or maybe there was some kind of fault in the system that made it revert back to default. We all know there's reasons why these things can happen, some nefarious, some benign, but in any case, you know that something's changed. And these are the devices that now you can look at and you can manage them by exception. And this is important because it means now the enterprise IoT security platform provides scale because you can manage by exception. You can have a team manage tens of thousands or hundreds of thousands of more of devices, because now you're going to be notified when one of those devices has a material change in addition to being able to productivity push, credential changes, new hardening gold standards, if you will, firmware updates, patches, so on and so forth. So I really like the value they add in terms of monitoring for that environmental drift. To me, that's how automation equals scalability right there, plain and simple. The last thing is reporting these enterprise X Iot security platforms can actually report on what you've got. How great would it be to know that, yes, I've got 50,000 devices, and by the way, I've got 300 that have end of life firmware. I've Iot 3000 that have default passwords still. I've got 4000 that I've enrolled through my Pam, I've got these vulnerabilities, I've got 1200 that have level ten cves. These are the types of things I know, and now I can trends things over time. We were this bad last month, now we're only this bad, and hopefully next month we're going to be that bad, right? And it's this continuous process of reporting on this and having APIs that tie into your existing reporting structures, being able to talk to splunk, being able to talk to ServiceNow Demisto other tools like that is really important. And these enterprise X Iot security platforms provide that. So last slide here. I just want to touch on a few things. In general, organizations don't know what they have, so discovery is so important, so they don't know what to fix. So if you don't know what you have, forget remediation. But remediation is so key. What's the point of finding your devices if you're not going to fix them? Okay, I didn't know what I have, and now I do know what I have, and everything's broken. What do I do? Okay, what's the next step? But if you don't know what you have, you don't know what to fix, then you're not able to fix these, even if you did know what you had to fix. Are you going to be able to fix 20,000 cameras or 40,000 printers manually without some type of automated solution? And then once they stay fixed, you want to be able to monitor those devices, and people aren't able to monitor them because they don't have tools to inspect them. You can't monitor by sniffing traffic or doing a scan. You have to be able to interact with that device at scale across, again, hundreds of thousands of devices. And this isn't just putting your XioT devices at risk, your printers, your cameras, your ot devices, your network gear, but the IT and the cloud based assets as well. Like we talked about in quiet exit. Right? It's putting that infrastructure and all the sensitive data that lives in that infrastructure at risk. And this results in everything from data theft and ransomware, so on and so forth, spying, physical attacks, et cetera. Right. So the results of attacks on xit are very wide and can be extremely painful for organizations. IoT doesn't have to be that way. This new generation of solutions, these XioT security platforms, again, phosphorus, we offer one as well. But there's other vendors out there that do things. We all do things a little bit differently. Some of these companies, Armist, Nizomi, they're really focused on just discovery. Other companies like phosphorus, we look at discovery and we integrate with them as partners as well. And we also primarily focus on remediation. And some of these others do something in between. But there are solutions now. So I implore you, take a look at your IoT environment. Try to determine what it is you've got and what the problems might be. And if you think you might be in a situation where hey, what Brian just covered in this presentation makes some sense, I think I'd like to kick the tires, reach out to phosphorus or some of these companies and say hey, I'd like to do a proof of value. I'd like to find out what I've got and what problems I've got in my environment and what I can do to fix it. Because I'm telling you right now, the XioT problem is growing exponentially. It's a much larger footprint than traditional endpoint devices or cloud combined by several orders of magnitude. And again, they're all traditional servers, Linux, Android based. They're just unmanaged and insecure and access points to attack your rest of your environment. And the bad guys are hoping you're not paying attention because things is a window of opportunity for these. They know it's there, but do you know it's there? And are you going to take steps to remediate IoT? So with that, thanks everybody for listening to my presentation. Again, I'm Brian Contos, I'm the chief security officer with phosphorus. If you'd like to reach out to me, here's my email and you can hit me up on LinkedIn and Twitter. I'd love to chat more about about XioT security platforms and certainly what phosphorus and some of these other players can provide. You have a great rest of the event.