Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hey everyone, thanks for stopping by. Welcome to my talk
go from Kubernetes Yaml engineer to software engineer
with CDKs, the cloud development kit for Kubernetes.
My name is Robert and I'm a senior solutions architect
at AWS. Also, I hate
Yaml. There's simply too much Yaml in my life.
And I think you know why. The moment you're trying to
design some infrastructure, define it, deploy it,
you get in touch with Yaml. And there
were quite a few times in my professional life where I
could swear that yaml, it really
stands for yelling at my laptop. So to
get into that mood, let's engineer some yaml,
shall we? Here we have a very simple application.
It's an HTTP Echo server, and we
want to set this up as a container on our Kubernetes cluster.
And of course we're using YAml because that's like
the native thing you're doing when
you are creating Kubernetes manifests.
So let's look at that. First of all, we need a deployment,
right? And the deployment tells us what kind of
container we want to run, what image it is, what kind of ports
it has, and so on. Then we need a service.
The service is our entry door to
our small echo server. Finally,
we also need an ingress because this thing
is so great, we want to expose that to the whole world, right? So we
define an ingress object and already
we see all the problems that we have with Kubernetes
Yaml. There's a lot of repetition. You have this
app, Echo label everywhere, there's a
lot of repeated values. This port
value is everywhere. And the hardest
thing of all, we need to name things and we have
to make sure that we're using the same names at
every place where it really counts, like here, for the name of the service.
So we wrote 45 lines of
Yaml here and we're already seeing some
problems. When you're writing your Kubernetes
manifests with YAml. With yamls,
there's a lot of copy paste going on.
Now that's often because to
make a single change, actually, you often have to make multiple
changes at different locations. So everything works together.
Now, you do this either manually and hope for the best,
or you have some special tooling for that, like a templating
engine. All this setup makes it really hard to
share. And maybe you need a special package manager
to actually do that, like helm. All in all,
that creates a very high cognitive load for
engineers. I have to think about a lot of things and
some of them are not really important for me.
But is there a different way to do it? Let's look
at the CDK's approach. And the CDK's approach
is to write infrastructure as
actual code, like in general purpose
languages. What does CDK's do for
me? Well, in a nutshell, it takes some code that
we see on the left, then it has a nice CLI command
SDK synth, and then it turns that code
in all the yaml that I need to actually deploy the
application. The great thing is,
I didn't have to write that thing on the right,
but rhymed. So what are cdks for?
Kubernetes. So CDK for kubernetes, which by the
way stands for cloud development kit,
it's an open source multi language software and development
framework. I guess you figured that out.
And it helps you to model your Kubernetes resources as reusable
components. So you write code and
from that code you go to config.
And then instead of doing the
copy pastey thing that we did before, well now
instead we can apply some engineering best
practices, which is great.
Also, Sydneykates can be run anywhere,
so it's not really dependent on some special
version of some Kubernetes cluster. You can run
it locally and in the end it generates plain
vanilla Kubernetes manifests in Yaml,
and you can deploy them in any cluster, on any cloud or locally.
And you can write all of that stuff in
Python, in typescript, in Java or in go.
So you can pick your favorite language. Now,
CDKS is not the only CDK,
actually there's a family of CDKs for
you to use, and we see all of them here.
So there is the AWS CDK, that was the
first. And as the name implies, this is
about defining infrastructure on AWS.
Then we have cdks, that's what we're talking about today.
And then there's also CDKTF. And CDKTf
works the same way you write code, but the output of
that is actually a terraform file that
you can apply for all of those things.
You need libraries. So ideally
you find that other people had similar
needs when they created their architectures.
And in the best case, they created libraries
out of those architectures. And you can
find them on the construct hub, which is under
constructs dev. So the construct hub
is a place where you can find additional premade architectures
and other things for the different CDKs
case. Now, how does it feel like to actually
work with cdKs?
Let me walk you through a complete lifecycle
first of all, we need to initialize our project.
So you need to install the CDK CLI and
you can do that via NPM. The next thing is to
initialize your project. For that, CDKs has
a handy command in it, and then you need to
import all the Kubernetes API objects that you actually want
to use when you model your infrastructure. And that's what you're doing with
import. So let's have a closer look at those two
commands. The init command is to scaffold
a base project so you have everything ready to go and can
just focus on writing your infrastructure. As you can see
here, init takes an argument about what kind
of language you want to write. So you can pick between typescript, go Java
and Python, and that creates a new CDKs project.
From a template on the right side you see how it basically
looks like if you do typescript. And by the way, everything we see
today will be typescript. Please don't judge me, I'm a typescript
guy. Now the other command is import.
And import is important, ha ha,
because this command helps you to get all
the Kubernetes resources or all the Kubernetes
API objects as code.
So basically as classes. So what we can see here
is that one typical command is to say import Kates,
add a version, and that basically tells cdkates
to import a certain version of the Kubernetes
base API. And then what you will get is what
you see on the bottom right, which is a
file, typescript file or whatever you're using,
where all the API objects, the typical ones,
you know like deployments, pods,
ingress service, they are all defined here. So now
you can use them in your code. This thing has a nice trick though.
You can also give it custom resource definitions
and it will turn those resources, those API
objects into classes as well. So here's an example.
If you're using the Jenkins operator, you can have
the custom resource definition of Jenkins available.
Suddenly you can deploy Jenkins instances
with cdks, which is great. The next thing you have
to do is to actually model your Kubernetes
resources or manifest as code. So we have this bare bones
project now, but we actually need to put in some kubernetes
stuff for it to produce something.
So this is what we're going to do right now.
And the way it looks is roughly like this.
The main code file that you typically have in a
CDKs application is
looking something like this. You have a
main class, the chart class that you extend,
and the chart class essentially is representing one
kubernetes manifest. So this class essentially will turn
into one YAML file in the end. And then within
that chart you are adding your actual resources
and you're doing that with classes, or we
call them constructs in CDK speech.
And here for example, you can see that we have added
a deployment and that deployment has a configuration
about containers. And once we did that we are
almost good to go. The last thing we do have
to do is instantiate an app class that's like
the high level thing of
CDK's apps where everything lives under. And then
we instantiate the chart that we have written above and
basically add it to that application and
then we're basically done. The cool thing about that
is that as we're writing
our infrastructure in code now,
suddenly we get all the benefits from writing
general purpose languages.
We get code completion for example. So we're starting to
write objects or classes and we get
some smart proposals on what we could actually put in there.
So configuring things is much easier.
Also we get inline documentation.
So here we can see that the moment I pick the
selector as a property I want
to define it also shows me the documentation
for that. And at this point you realize that
yay, your config is typed now and that's
awesome because there's less looking up of documentation
online and more help in your ide to
get stuff done. Now let's actually write that
HTTP echo server with cdkates. How does
that look like? Well, to some extent
it still looks a lot like what we did in Yaml,
but there are some benefits. So have a look at
that. First of all, we are
using the Kube deployment class over here,
and this is essentially your vanilla deployment
definition deployments object, API object,
however you want to call it from the Kubernetes base API. And you
see that we have to define a lot of the same things as
you would with a yaml as we go along. Of course
there's also a Kube service class
and there's our ingress class. So so
far so good, not too much new,
but there are already some benefits we gain here.
We are in a programming language now, so we can
practice things like do not repeat yourself. So the port and
the labels that we might need in different places,
well now essentially we can pull them out
and we can only define them once and reference them. So that already
cleans it up a little bit.
Also about the naming is hard things.
This is easier now because we see here that this
service class we actually didn't give the service a
real name, it will just generate one for
us. It will make sure that it's a unique name so
there's no collision. And when we need the service name somewhere else,
we can just reference it by saying, hey, service name,
give me the name of that service. So naming
is easier now because some of that cdks
is doing for us. Now that we have defined our
infrastructure. We can go ahead and
create the Kubernetes manifests, the dreaded yaml out
of that. And we do that with a synth command.
So calling CDK synth will take those manifests
and create them for all those charts
in your app. And this is absolutely
plain vanilla. Kubernetes Yaml manifests.
There's nothing specific about them.
You can put them in any other tool
or process you have which works with Kubernetes
manifests. It will work because there's
essentially no trace that cdkates created that.
And that's good because now we actually want to deploy that stuff.
So it runs in kubernetes. And the way we can
do that now really depends on how we are
operating in our company. So in
the simplest way, you can just call Kubectl, apply.
The manifest will be found in the disk directory
of the CDKS project.
And you're good to go. And kubernetes will work on
that manifest and deploy it. But of course,
as I said before, you can use any tool that works with a Kubernetes
manifest. So if you are one of those fancy kids
who are already doing git ops. So you're checking in your manifests
into git. And then your Kubernetes cluster is continuously
pulling that and applying that. Well, you can do that too.
Now that you have the output, you have that manifest.
Just check it into your git. And your Gitops process
should just work. Now we did all of that
and we feel pretty good about it. So we're thinking,
hey, what we just built,
that's a great thing. Everybody should be able to use
that. So let's package that as our own
class, as our own construct and make
it available to everyone else. How are we doing that?
Well, because we are working in a normal programming
language. We can also use normal package managers
and package repositories,
artifact repositories, or however you want to call them and
put our code there to share it. And that of course, could be
privately. So we're only sharing this internally at our company with
our team. Or it could also be publicly as a software
package. And then somewhere
someone might find your cool construct.
And part of this construct of your own construct
library is this cool echo server deployment that
we just saw. We just packaged that as its own class, its own construct,
and then somebody else can use that and have an echo server and
they had to do even less work on that because you
obviously built in best practices and all that good stuff
right? Now we saw that with
a programming language we get some benefits for making
the code easier, making the configuration clearer,
removing repetition. But we
haven't yet used one
of the main benefits of programming, which is abstractions,
so abstracting things to make
it even easier and to remove undifferentiated stuff
that we don't care about really. And this is where cdks
plus comes in. And CDKs plus is
another library that builds on top of cdks.
And to explain what it really does, I have to
talk about construct levels. Now as I
explained, constructs are essentially those
classes that represent the infrastructure,
the Kubernetes API objects. Now we talk
about levels, and levels means that
these constructs can have different levels
of sophistication. So the question is
how much smarts, how much intelligence
is built into them. The lowest level,
level zero, is really just a representation of a
bare bones Kubernetes API object with like common
functionality because all API objects in Kubernetes kind of
look the same. And that's really not super interesting
for you because you don't think about
that too much. The level after that,
these are the actual Kubernetes
API objects that you use like the deployment, the service,
the pod and so on. And these are automatically generated for
you based on the version of
Kubernetes that you referenced when you did
that CDK's import command for example.
Or they are generated from that CID Yaml
file that you gave to the import command.
Now where it gets really interesting is on level two
because these are handcrafted high level APIs,
high level constructs that do a lot under the
hood for you so that it's much easier to
define your Kubernetes resources. And CDKs
is the library that is bringing a lot of those high level
constructs. So going back to
our HTTP Echo server, we can now look at
how we would define it with CDKs plus
instead of just cdks.
And this is all it takes, that's the whole thing.
CDKs plus has a special
deployment construct and this
deployments construct does a lot of things for
you under the hood. So you just have to define your
container because obviously you have it. Couldn't know that and
then if you actually want to expose
that via an ingress, you don't even have to create the ingress class.
You don't even have to instantiate the ingress class for that.
There's a handy helper method called expose
via ingress, and it does exactly what
it says. Expose this deployments
via ingress, give it a few parameters and
you're basically done. So now, before, I think we had like
40 ish lines of code, or Yaml.
Now we're down to 13. And we're
also getting some really cool benefits here, because this
part here is speaking to us, right? It says,
expose the deployment via ingress.
We understand the intent of that.
It's not just a bag of objects
that somehow relate, because there's the same port number
and the same service name or whatever.
We can see a little story here. We know what the engineer wanted
to do. So we're starting to write for humans and not for
machines. And that's an aspect of CDK
in general that I find very interesting,
because it has a lot to do with the
idea of clean code, right? That's a concept that was
made popular in the book of the same name. And the
idea of clean code is that you write code that is
easy to be understood by everyone in the team.
It can be enhanced by any developer, even by new developers,
and to make things, if things are understandable,
it makes it easy to maintain that code, improve that code and so
on. So with those speaking functions,
with those high level abstractions, we make
it easier to understand the code, we make it easier to understand
the infrastructure we are defining with that code.
To give you an example, going back to the
deployment definition we just did
with cdkates, we can
read that story out of that text, right? We said,
hey, create a deployment and then call expose via
ingress. That's the story. As a human,
that's easier to read. Now,
what happens in the background, of course,
is a lot of stuff, right?
So in the Kubernetes API
object world behind that one class,
that one construct we created, there's a deployment
definition that has been created.
That deployment somehow is
using this port because we
need to define it for the container. That deployment
needs to basically know which ports belong to it. So there's
this label that must match with the pods. We don't
see any of that here at all. This is all hidden from
us. When you have defined this once, you know what I
mean? Because this is like doing it the same way every time. You can only
mess it up. Basically, it doesn't get better.
And CDKS is hiding that relationship
completely from you. It just manages it for you so that it works,
we also need to create that ingress now. We don't need to
create it by hand, we're just calling this expose via ingress method
here. So that's done for us. And again, the ingress is
somehow needing that port.
And the ingress has a service, right? If you remember, the ingress
points to a service object and the service object also
kind of needs the port. And it's pointing to the same labels
because it needs to know which ports belong to the service object.
And now you basically know why.
Maybe you need a graph database to understand what's
happening in kubernetes, but the gist here is
that cdks is doing all of that
under the hood for you. And instead
of reading that because that's essentially
the yaml, you read that story over here
in code, and it's much easier for someone to review
that and understand what the engineer intended to do,
which is really cool. And you can take this further and
of course build your own constructs, put together whatever you
want pre configured, just expose a few values
that are really needed and everything else is happening under the hood.
You can build in your security best practices from your security
guys. So your CISO is happy you can put
in anything and then just release a very cool
high level, super smart construct that makes it easier to deploy
stuff on kubernetes. So to sum
it up, why should I code my infrastructure?
Well, the reason is that the moment you're using general purpose
languages, you can employ software engineering
best practices like clean code, do not
repeat yourself, and so on. You can also
use a familiar programming language. So instead of
working with some form of configuration language that
as time progressed became more and more like a
real programming language because it had to do so many things,
you just say, well, just use a normal programming languages that
I know, like Python or typescript. The moment
you're using a programming languages, you get great
ide in tool support, because that's what these
ides do, right? They're working best with programming languages,
even out of the box without any cool add ons.
And then you can build those powerful abstractions,
or you can use those powerful abstractions from CDks.
And as mentioned before, you can use that for your own developer
platform. You can extend the existing
abstractions or you can build your own, and then you can
share them internally or externally and make it easier for people to
use kubernetes. Now there might be a few reasons
why you do not want to use cdks and programming languages.
And one of the reasons might be that you're saying, hey, but I'm totally fine
with my existing tools and I like
my mountain of Yaml, it's a great friend for me.
One thing I can say here is that you can actually mix
and match. So with cdks
you can also import existing kubernetes manifests
in YaML and then sprinkle some new deployments,
some new API objects in with
cdks. So it's not an all or nothing thing.
If you have a new smaller project that you need to deploy in Kubernetes,
you could try out cdks now. You might also
just don't like coding, which is fine because
the coding is great until it's not right. Things break.
You have a dynamic processing
flow control flow, so things
can get complicated. Typically I don't see that
with CDK. Typically the code is pretty readable because it's
just do this resource, this resource, this resource.
So it's just reading it from top to bottom.
So that's fine. But yeah,
it can have some downsides. And if you're the author
of a helm chart, meaning you are building helm
charts, you are creating them. Then also
cdks is currently not helping you so much
because it cannot create helm charts.
You can deploy helps charts with it, that's absolutely no problem.
If you're referencing a helm chart and say I want
this deployments with these parameters, you can do that in cdks,
but it's not a helm chart creator for
that case. Not yet at least. So I
hope you got an idea on why cdks can
be really great to now
do code instead of configuration to
manage Kubernetes or manage at least
Kubernetes deployments and other objects. Here are
some resources for you on how you can get started with cdks.
So there's of course the project website CDks IO.
We already talked about the constructs hub. So there you find
community shared constructs that already
do things like building complete infrastructures,
pre defining things for you. So that's really cool.
And then you can also interact with the community.
So cdks is a CNCF sandbox
project and there's a slack under CDK
dev. And also finally there's the CDK
day, which is a completely community driven
event for the whole CDK ecosystem that's happening
once a year, but you can always get the recordings
and there all the cdks and also other cool
new tools are shown from the community. And what fancy stuff
you can do with them. So definitely check
that out. Now if you liked that presentation,
I would ask you to kindly give me some
feedback on that session and also
give us some feedback on some potential new cdks
features. So if you follow the link under the QR
code you'll find a small survey about
the session. But on the second page we're asking you
something about how you are using kubernetes and
if you would like or if you are interested in
some new cdks features that are coming up in the near future.
And to spoil it here. It has a lot to do with helm
so it would be super amazing,
especially if you are a helm user. So chart creator
or chart user or both if you would take the
survey because that will help us to figure out how we can make
cdks better for the helm world.
So that's it from me. Thanks a
lot for watching that talk. If you have any questions
you can find me on Twitter or on LinkedIn.
And if you ever try out CDK's,
which I hope you will do after this talk, I would love to
hear about it and get your feedback
on the user experience. So have
a good time and and yeah, start slaying
that Yaml dragon. See you then.