Abstract
In this talk, we’ll evaluate tools and techniques for implementing continuous security at your startup at the infrastructure level.
Quite often as DevOps engineers at startups, we’re expected to be experts in security, and that often isn’t the case. We know to keep our ports closed, and to operate on the principle of least privilege, but with infrastructure as code introducing a vulnerability is as easy as a missed line. In startup environments where things move fast, it can be easy to create an insecure cloud, especially when operating by yourself.
We’ll review the concepts of Static and Dynamic security testing, and how the both can be combined to implement into your deployment pipeline. We’ll go over open source and managed tools that can assist you in the transition to DevSecOps and continuous security, as well as give examples of how to realistically implement this at your startup, and how to explain the business value of continuous security to your leadership team.
At the end of the talk, you’ll have a clear understanding of the landscape of tools you can use today to help you secure your infrastructure, an understanding of why they can be valuable, and how to explain the business value of them to a non-technical leadership team.
Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hey, my name is Ryder and this presentation is called how to keep your
startup's cloud secure when your security team is
just you. So first and foremost youll can't
realistically one person working on it isn't enough to keep your
startup's cloud completely secure. But there are
a couple things that we can do in order to make sure that we have
the best possible security as we can do it. So we'll
do our best in this presentation. I'm just going to
do a quick about me who this is for the concept of DevOps engineers
and DevOps as a job, your startup security landscape.
So application security, infrastructure security and other as
well as some best practices that I found and then concluding it all.
So let's jump right into it. My name is Ryder Damon. I am a
developer advocate at Indeni Cloudrail and before that I was a
DevOps engineer for a number of different startups. Before that
I was a web developer, and before that I actually came from a biomedical
engineers background. So. But unusual, but a fun
career path for sure. This presentation, who it's for,
it's for me essentially. When I was first getting into DevOps,
I'm designed the presentation as the things that I wish I
knew. So if you're a DevOps engineer and you're not from a security
background and you're at a scaling startup, things is the kind
of presentation that is for you. No matter who you are.
It's important to keep a growth mindset because in DevOps we're going to be
learning new things every day. And I think that's pretty
important. So let's first talk about DevOps.
This is a list of things that I have been asked to do as a
DevOps engineer. Now, a lot of these
aren't what you would typically expect from a DevOps
professional. For instance, catching an Uber with the company server
in the trunk of the Uber. Not necessarily something you would expect of your DevOps
team. But what I'm trying to get across here is that it's a
lot. So as DevOps engineers, we're going to be asked to do a
number of different things, and some of
that involves security. Now, you might be
from a security background, but I am not.
And that's okay. I know to keep my ports closed and
I know to limit blast radius. But when it comes to application security
or infrastructure security, I am not an expert.
I'm an expert at building pipelines. I'm an expert at building
infrastructure. But the security part of it, not so much. And that's okay.
So let's talk about the security landscape at your startup.
First and foremost, I like to split it into three specific categories,
application security, infrastructure security and other.
So starting with application security, it's pretty much everything around your
application. So making sure your source code is
secure, making sure you're free of OASP top ten
vulnerabilities. Do you have any
unsanitized inputs? Are you vulnerable to SQL injection cross
site scripting? Is that a problem? Also application
composition. So what packages are you bringing into your application?
Are you using a Python package that hasn't been
updated since 2008? Things like these are
quite a problem. And this is what I would consider to be application level
security. Also, penetration testing. You've probably got
can annual penetration test at your company, and if you don't,
I highly encourage that you start one.
It can be a difficult sell at first because they'll be anywhere from
about $7000 to $50,000 on the upper end.
But really the data that it provides and the things that they find
are very valuable. So for
application security, as a DevOps engineer, you're not really going
to have the time to sit down and do your own
penetration tests of the application. You're not going
to have time to do that. Realistically, at a startup that's growing really, really fast,
you won't have the time. So what I recommend is implementing tools like
these, which are going to be a part of your deployment pipeline and are going
to help you identify issues and shift those issues,
who has to fix them over to the developer that created them?
So first, GitHub has a number of different tools to help you catch
any package security vulnerabilities, and SNCC has that too.
I think it's SNCC. It could be Snyke. I'm not really sure. It's like Nginx
all over again. So if someone knows, please let me know. But both
of those tools, what they're going to do is they're going to allow you to
identify any vulnerable packages that you're bringing into youll application
so that you can fix them right away, even before you deploy them.
OASP Z or OASP zap Z attack
proxy. What that's going to do is identify any OASP
top ten or any issues with your application.
So it's web application testing, and that's a great thing to have in your pipeline
as well. So for application security,
you're going to want to implement the tools into your deployment pipeline.
And if you're joining a startup as a DevOps engineer chances
are you're not one of the first hires. So when you get there, there's going
to be some legacy code and there's going to be a ton of technical
debt. So when you first turn on these tools, do not be surprised when
you see hundreds, sometimes thousands of different things
that are raised. That's pretty normal.
Incorporate it into your pipeline anyway. But don't fail
the pipeline just yet. Don't allow any of these tools to fail your pipeline.
Get them in there, get the visibility, start surfacing this
to other members of the team that there are problems and that we need to
fix them. Set a deadline for when you want this all
to be fixed. And on that deadline, fail the pipeline no matter what.
So how do we fix all these issues? We've turned all of things on.
How do we fix these issues? You've got to set aside time
in your sprint and I would recommend creating an ad hoc team.
So the first thing I like to do is grab the most senior member of
the engineering team for half a day,
look through all of the issues and triage them. Which ones are actually problems,
which ones are part of services that are no longer even used? Is GitHub
reporting on something that we don't even use anymore?
Figure those things out, then figure out how much
you think it's going to take in order for just you to fix it and
double that estimation. Maybe it'll take you a week to fix these problems.
Double it. Because if you've not had familiarity
with this, when you go in and you start fixing these issues,
you will fix one vulnerability and five new more will come up.
Or the package that you've just upgraded is no longer compatible
with these four packages. It's a nightmare.
Ideally you'll want to involve some other engineers. So front end engineers,
back end engineers in your ad hoc team for fixing these issues,
you don't want to do it just by yourself, but you may have a tough
time convincing management to convince these or to
allow these people to work on something that isn't building a new feature.
And that's pretty common at a startup, and that's okay. So what I would
recommend if you're facing pushback from there is don't try and pull
everyone for a full week. Don't try and get the engineering team to
focus on it for a full week. See if you can pull people day by
day and work on the vulnerabilities that
were maybe introduced by them or by someone on their team.
Something that has to do with that team's context. If it's a
day, it's a lot easier of a cell to pair program with them,
and then you can fix all the rest of it by yourself.
So let's talk about infrastructure security.
First of all, are you in a public cloud? If you're at a
startup, chances are yes. And then the next question is, are you in
one of the big three if you're in more of a developer centric
cloud? So a smaller public cloud,
it's less difficult to misconfigure
things, but as soon as you get into like GCP, AWS,
Azure, it becomes very, very easy to create an
insecure environment completely by accident. And for this
we would recommend cloud security posture management tools,
sometimes abbreviated as CSPM. Now,
cloud security posture management tool. What that's going to do is it's going
to scan your public cloud, wherever it
may be, and identify any vulnerabilities that are in it.
It's going to let you define a policy and show you whether
or not your cloud is adhering to that particular policy that
you've defined. It's going to report anything that doesn't look exactly
how it's supposed to be. So youll can go in and you can fix it.
So maybe someone accidentally deployed a machine into the wrong subnet.
It's going to report things like that. Next is implement
infrastructure as code and configuration management. So if you've never
worked with infrastructure as code before, I would highly recommend
getting involved with it today. I resisted it for
quite a bit of time because I was afraid of it.
It seemed very overwhelming to get involved with, but I promise you,
it's easier than you think, and it's a really
great thing to get involved with. The infrastructure becomes repeatable
and you're able to define things quite well. I'd recommend using a
service like terraform. Now, the benefit of this is
not just repeatable infrastructure and seeing all of your infrastructure
as code and version control, it's not that, it's that you can
actually incorporate infrastructure vulnerability scanning tools.
So whereas a CSPM tool scans your cloud environments and
reports any issues, those issues are already in production.
An infrastructure vulnerability scanning tool identifies
issues before they're going into production, so you're able to catch them
before they become actual problems, which makes it a lot easier for you.
Consider initial implementing. So how much work is this
actually going to take for you to manage? Is it worth the investment
of your time? Chances are yes, it is, but it's
not going to be the case for everyone at your particular startup.
So a couple cloud security posture management tools are orca security,
cloud rail, Us or aqua. And these are tools
that are able to scan your cloud environment and report anything that's
going wrong in youll live cloud. So any misconfigurations,
any vulnerabilities, things like that, some infrastructure as code
security tools that will catch these things before they go into the cloud?
Assuming you're using infrastructure as code, are TFSec cloud rail?
Again, this is kind of our bread and butter and checkoff by bridge crew.
So once again, use a CSPM tool,
and if you're using infrastructure as code, use an
infrastructure as code vulnerability scanning tool.
Finally, we have other security. So other security
is pretty much anything that's not application or infrastructure is
how I like to consider it at a startup. If you've ever
had the lovely pleasure of being involved with a SoC two
audit, you'll be familiar with most of these concepts. So you
might not have an IT team and youll might have employees
who are using computers, and management might
want those computers secure. So are you running an endpoint monitoring
service that actually ensures that employees are
encrypting their drives and they have firewalls running and malware
services running? Once again, not necessarily something you would expect from
youll DevOps team, but it's often something that gets handed off to us,
so it's important to keep that in mind. Other types of
security are other things that would generally be raised in a
SoC two audit. So does your company
have an office? Who has the keys for that office?
Is there security? Sorry?
Is there wifi security at the office?
Does your company keep a production server on
a desktop at the office? These are all
important security questions to consider, and for this, I would recommend
getting involved with some SoC security tools. So the first one collide is a
device endpoint management tool. That is my favorite
because it's kind of like a slack application.
You put it in slack, and then whenever someone joins the
company and sets up their device for the first time, it will continuously ping them
on slack until they get everything set up properly. So if they're missing
a firewall, it'll ping them every day until they finally do it. And that's normally
something that's my job. So I quite like that.
Vanta and Secureframe are tools that allow you to manage
your SoC two audit and your compliance for that.
And if you've ever used them, they're super helpful, not just
around audit time, but in general. So they'll connect
to your cloud providers and they'll go through and they'll scan
much like a CSPM to make sure you have no major
vulnerabilities. They're not quite going to do it to the extent as a CSPM
tool, but they will raise things like do
you not have any VPC flow logs,
which is something that you'll need in case anything happens. So you'll want to
consider those tools. They've also got agents that run on low level machines
that report package vulnerabilities. They're very, very helpful and
when it does come audit time, they allow your auditor to self
service, which is quite helpful. So these are
some recommended practices that I have found made my life easier and hopefully will
make yours as well. First is bringing infrastructure down to its
most simple version. It may be fun to use
the latest and greatest. And you know what? It's also good
for our careers to use the latest and greatest. It's good to say on your
resume that you incorporated things, brand new service so that
you can get a better job down the line. But it may not be the
best for the company. So maybe consider implementing those services
in a sandbox environment and reducing your company's
production infrastructure to the simplest version.
I would recommend doing that, but it's whatever works for you.
Next is using managed services wherever possible. So say
you're running a containerized workload in AWS.
Try consider using AWS ECS Fargate
instead of EC two backed ECS instances,
you'll have less servers to manage. It may be a tad, but more
expensive, but you won't be responsible for any of the security updates
or dealing with any outages or anything. You ship the blame and
the responsibility to AWS, and for me, that's worth a
couple extra hundred dollars a month. Next is
use infrastructure as code. As I mentioned, if you're not using it, get involved with
it. It's much easier than you think to use and it's
a great tool. Then recruit other members of your team for
feedback. So they may not know what's going
on with terraform files, or they may not understand the
cloud environment as well as you do, but they often have valuable
feedback and the more people you can get involved in the security review process,
the better. And finally shift left. And I don't mean this
in the traditional sense, I mean it as shift some of the responsibilities
of security onto the other engineers, particularly application
security. So it is a full stacks
engineer's job to ensure that the code that they're writing
is free of vulnerabilities and the package that they're bringing in has been updated
since 2008. It is your responsibility,
though, to ensure that the deployment pipelines you're building are catching these things in
case that particular developer forgets this.
So shift some of those responsibilities for them, and do that
by ensuring that you have checks in your deployment pipeline that fail the pipeline,
and don't let that particular engineer deploy until they fix their issues.
So, in conclusion, simplify everything. Use managed services
wherever possible, use infrastructure as code, implement application
security tools in your pipeline implement infrastructure security
tools in your pipeline and involve as many other people in the security process
as possible. And finally, continuously demonstrate to managed that
more people in this role is valuable. Ah,
that's it for me. If you have any questions, you can reach out to
me on any of the links below, and I thank you
for your time. See you later.