Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hello lovely people. My name is Frederic Harper.
I'm a principal developer advocate at Cube first and today
I'm going to talk to you about get shit done the
GitHub's way during the talk. When you watch the
video, please feel free to ping me on Twitter.
I'm at Fr or share screenshot or things
that I've said that youll agree or disagree with. I'm all about
discussing stuff and I don't have the only truth, so please
connect with me. I'm going to be more than happy to talk to you.
So with that said, let's jump
on the talk. We only have 30 minutes together today, so that's
going to be a high level overview. I'm going to try to put as
much content as possible with a little bit of demo so you can
see how it's working and how you can
apply this to your day to day work.
So let's start with maybe something that you already know,
something that you're comfortable with, which is DevOps.
So we are used to DevOps,
or maybe you're starting to learn about DevOps and there's that thing called Gitops
that is there since a little bit now, but still something
a concrete that is quite new for most people starting or even
people that are in their actual cloud native journey.
So when we think about Gitops,
actually, sorry, when we think about DevOps, we put
the application code in git repository most
of the time. And we're going to have that CI CD pipeline who's
going to basically validate our code
and do everything we need and deploy our code
to the cloud to on premises.
Actually that delivery pipeline will put your
application somewhere. So this is the kind of process that we are used
to when it comes to DevOps. The difference now is
that you can also do that with DevOps. But it's an important
part of Githubs is that now we think about infrastructure
as code. So I'm a big fan of infrastructure as code.
Hiac, one of the most popular out there is terraform.
That helps you to do that. And it's basically as it says,
like it's having your infrastructure as code. So youll
write down some code that will create your infrastructure
and you still put this in your git repository. And traditionally
more DevOps way of doing things. You're still going to put your configuration
sometimes in your git repository.
So it's kind of like the industry default way of doing
things. We are familiar with that. It comes easy now
because we've been doing that for a couple of years. And the thing is that
there is some flexibility of doing those because it's a principle
that is not specific just for cloud native application.
You can use that in different type of environment.
But now what we are changing when we move toward GitHubs is
that we're splitting the CI and CD pipeline.
So it's still there, you still have continuous integration,
you can still do whatever you were doing previously
in that part of the pipeline. But there's also another difference
now, instead of having the git repository that push things,
the CI actually the CD part
will grab information, will pull the information from
the git repository, and it becomes
your source of truth. And I will talk a little more
about the difference why it became the source of truth and why it may not
have been. When it comes to DevOps or using
DevOps as your way to manage your application
deployment, the things that
may be something that may not be useful for you when
it comes to GitHubs is that it's really specific for Kubernetes,
because on those pods you're going to have mechanism that
will grab the information from your git repository to always ensure
that you have the latest version on what is
in your source of truth, the git repository. So it's
Gitops in a nutshell. And Gitops really help you to manage,
as I said, cloud native systems with Kubernetes, but in a
declarative way, leveraging git. So it's a tool you
already know and you already use. You can see this as
a branch of DevOps. It's really taking its best practices.
But with the big and cases of having git,
youll source of truth and not the server. So there are some pro and
cons about that. So let me show you
a little bit about a GitHub repository structure
that we're using at Cube first. So let me switch my screen here
and hopefully right now you should see my GitHub
repository. So that was created when I
used the Kubefirst CLI, which is our product, but I'm not here to talk about
it, just to show you one example of
how your GitHubs repository could be structured.
So we have our GitHubs repository here, we have
our registry. Cube first is just because I
call my cluster like that. And you're going to
see there is those yaml file that you may recognize.
Those are basically Kubernetes declaration that
I have in my Githubs repository that's going to be read
by argo CD so we use our go CD in our ecosystem to
install update deploy application on
my cluster. So I have my ML file here
we are using the appsup apps patterns
and here you have different components for different parts
of application that are installed in my cluster.
So my GitHub repository is my source of truth.
If I go back here, I have the terraform folder which is
my infrastructure as code here. So in my case there is kind
of like three parts. So there's a part where our platform lets
you manage youll different git repository that
are associated with your cluster. So in here if I go
and repose tf I
have two git repository that were created for me when I
created my cluster and we use a terraform provider
to manage and create those. If I go back here,
I'm also managing my users directly from
the git repo. And you may say like Fred, it's starting to be a little
bit weird, but bear with me, I'm going to share a little more about the
pros and cons of using that method. So here let's
say I have a developers terraform file and I could
have the developer one user directly by
doing a commit on my git repository.
And also there's a third part for us where we manage different secrets
using Ashicorp vault that are all connected together to
the application that are pre installed on your cluster. So that's just
one way of having or defining the architecture
of your githubs repository. This is what we've done. It's a
culmination of a couple of years of experience and also based on
the needs that we have to build our platform. So everything is
there. It's the source of truth of my Kubernetes cluster
right now. So let's go back to the
slides and about
that now. Okay, I introducing you a little bit
about Githubs. Let me talk to you about the GitHubs principle.
So I show you a little bit how it looks like compared to DevOps,
but there's some principles. So Githubs, it's not a standard
per se, but there is the open Gitops group that is
working on some principle and usually
people, when we talk about Gitops we try to follow those principles.
So the first one is that it needs to be declarative.
A system that is managed by GitHubs must have its desire
state expressed declaratively. So everything should be
in the git repository. It's kid of like another way to
say that it should be version and immutable
I don't know why I struggle with that word in English. So the desired state
is basically stored in a way that enforce immutability.
So the servers are never modified after the
deploy in the sense that you don't go yourself there
and modify stuff. It's version because
you're using git and because you're using git, you retain
a complete version history. So that's kind
of like one of the benefit of GitHubs. It's pull
automatically so you don't have to push information.
It's really, there is software agents that automatically pull the desired
state declaration from the source, and there's benefits for that.
And there's some, maybe not costs, but you
may see some part of it that may not be as beneficial.
So I really want that talk, to be honest about GitHub. So I'm going to
come back just right after about the pro and costs of using the GitHubs
principle, or actually thinking about using GitHubs when
going through your cloud native journey. And the fourth principle is
continuously reconcile. So this is also one of the
pros is that the software agents, because they continuous pull
the information, they observe the actual state of your cluster versus
the desire state that is in your git repository. So it
would always attempt to apply the desired state. Hence the
source of truth is being git.
So there's the benefit about that, that I'm going to talk right
now. So the benefits, when it comes about Githubs, it's really
you're using familiar tools and processes, git, git workflows,
those are the things that you hopefully already know and use
day to day at work. There's also the benefits of infrastructure as
code. Youll have the change history because of
the file versioning, it's easy to revert.
And there is the code change revision processes.
In the best world scenario, you should always never commit
directly on the repository. You should do a pr and someone from
your team, at least one person from your team should review the
changes. So that's really good for accountability.
But also that's really good to be sure that you're not making
some mistakes, because there is someone else who will look at your change.
There's faster deployment and iteration, and it's
easily reproducible because you have the secret
source. Everything is in your git repository. There's also no
cluster drift, which means difference over time
because of server changes. So when we're doing DevOps,
like maybe you had no clusters drift because
everybody was already using your git repository as the source
of truth. But if we want to be honest it's rarely the case.
So sometimes there's an issue on the server side. There is an issue in
something in pods and what are you going to do? You're going to go in
the prod server and you're going to connect with SSH
and you're going to try to fix things and that's going to work and it's
going to be beautiful and you're going to forget about it.
So you have a change that is important for production, that is only
on the production server. But now because everything goes through youll Githubs repository,
everything is there. So there's nothing on the server.
So you're not losing changes. There is no cluster drift as I
said. From my point of view it really brings greater collaboration,
auditability and simplification of access control.
Again always coming back about the source of truth.
I see this as more secure because there is no need
for individual infrastructure access.
There's also no need for your CD pipeline to
have access to your Kubernetes clusters because it's
the other way around. We're not pushing things,
we're pulling things inside the cluster. There is more transparency because all
the git manipulation and as I said,
these agents are running inside the cluster so no need for
the CD to have access to the server. When we
think about Githubs too, there's the cost saving because
there's the automation part versus the manual execution,
deployment testing, easily reverting or rollback
some changes so it's prone to less error.
And as I said, it's also easily revert
like you can easily reproduce and revert also I would
argue that there is some cost saving there and it's
also a great approach for a simple stateless disaster recovery
model because there is no server
state. There is only like the source of truth state.
If I want to be honest there are some challenges or maybe
we could even say some cons when it cases to Gitops.
So the learning curve can be a little bit steep. Especially if you have
no knowledge of Git, if you have no real knowledge of infrastructure as
code or some of the best practices, or maybe you're not
even totally familiar with DevOps approach.
What I see as a benefit, some people could see this as like a
challenger accounts because there's more layer of approval because
everything goes through git. So in cherry, as I said before,
if my team was there they would be like no Fred, don't like when
you commit directly on the repo. Sometimes we do this
in demo and live stream and I'm always complaining and I say
that I'm going to do that in my demo right now just for the sake
of time. But like in a day to day in production, when you work
on your product, you should not do that. So you do a pr, someone will
review your changes at all level, on the infrastructure side,
on the application side, or everything you need to run your cluster.
So it means more approval, but it also
brings all the benefits that I said before. It can be a little bit harder
to customize the process. It's kubernetes centric, as I
was saying before, and things can go
wrong between the source of truth and the CD action.
So it depends on the tech you use.
And you know what, lastly, it's not magic. It doesn't
fix bad practices. So I'm sorry about that.
I know you were maybe hoping for some magic solution. When it comes to Githubs,
it's not the case. So obviously it's just
a tool. But I firmly believe that it's a tool that
really helps you to have
a better cloud native journey and ensure
that your processes in place are helping you moving
forward and be successful in your endeavor. So let
me show you a little bit more how this is exactly
working. So let me go back to my browser
here and you're going
to see here I'm in ergo CD, so it's fine if you
don't really know Argo CD. It's one
of the tools that we really like out there. There's like Flux CD.
There's other tools out there that help you to manage your githubs
and your CD pipeline when it comes to managing githubs.
But we decided to use argu. This is the tool we town, we liked
it. And if I go here and I search for an application called Flappy,
it does not exist and it's normal because it does not exist.
So what I'm going to do here, I'm going to go in my registry folder,
I'm going to go here and I'm going to say I want to add
new file. So had file create new file,
call this flappy Yaml. And I'm going to be lazy.
I have my shortcut here. As you can see, it's a
YAml declaration for an application called Flappy Bird
that's going to take a chart and install a version of
it's basically flappy bird. We rename it to Flappy Q array.
And I have my ingress route here because
I want to be able to access it with flappykara
Q first dev. So if you want to try this at home
I can share your link if you share with me but that link won't
work because I'm doing this on a local clusters on key 3d
right now. So I'm going to do the unthinkable.
I'm going to commit directly on
the main branch. So had flappy,
flappy k ray and I'm going to commit directly
here, I'm going to save that and what I'm going to go
back here so I'm going to refresh CD,
I'm going to go actually youll should have probably check in the
registry before so I'm going to be sure to sync.
So it should sync automatically. I think the default is streaming it, I never remember.
So if I synchronize that I should see at
some point here, oh registry, something is working
here. I should see my flappy key ray here.
It's out of sync so it's syncing right now and
if I give it a little bit of time. So what's happening here?
The applications would be exactly here. So what's happening here is
that flappy is being out of sync. So youll know where I was thinking about
trying to get always the latest state. So Hargo CD
checked my git repository, realized that the
application was deleted. It's not something that should happen.
So sometimes Hargo CD is
doing some weird stuff but the application
will come back right away. So if I
just fire sync again that should be back
and we should see flappy key ray.
I don't know how to. Yeah, flappy here. Something's going to happen here
in a second. It's going to be here.
There we go, flappy now let's go see. So it
should be in the process of being deploying,
going to refresh. That still happening.
It's out of sync. And what I can go here, I can check the diff
and you're going to see that there's a difference. So it's not deployed yet.
There's a lot of difference. But the job of ergo CD,
as I was starting to explain, is that it will check on my git repository,
it will look at the information that is there, it will
compare to what's deployed on my cluster and it
should update the registry. Oh,
what happened here? So it's still syncing and
we should see something. So what's going to happen now? Let me just
go back to the presentation part
and we'll come back to. Oh, actually I
think it's good now. I think he was able to deploy.
If I go here and I access Flappy Kray, it's a
page not found. So what's
happening here? I don't know what's happening here.
Let me check what's happening.
Maybe I made a mistake with my yaml file.
So it's sync. Oh, I know
what's happening. I know what's happening.
This should be good. The chart version,
flappy Kara. If I check the certificate,
should be good. That's out of sync.
There we go. Not sure what happened here.
So now it's syncing, it's installing.
Going to give it a couple of seconds and
everything should be good now. Oh my God.
So little live demo problem.
And now I have my kray application that's been
installed from my argo,
actually from my GitHub server. I shouldn't play and try to explain something because
it's obviously not working. I cannot do the same two
things at the same time. So let me go back to my
talk. So youll see,
you see, I install the application by heading
a Yaml file and argo CD.
Fetch it from the GitHub steps repository, install it,
deploy it with one small hiccup because that was a
quote unquote live demo. So some resources
if you're starting your Githubs journey, there is the
open GitHubs slash project.
This is the open GitHub project or
working group that I was talking before. So really a good place to start to
learn more about the GitHubs principle or to just to get more information about
it. A great repository from weaveworks. It's awesome
GitHub. So if you know the awesome repository that you
usually have on GitHub, there are a bunch of
awesome resources about specific topics. So this one is
a pretty good one too. And youll obviously
I need to say it if you want to replicate and start to play
with Githubs, we have an open source product that is free.
It's available@GitHub.com cube first slash Cube
first, because the name of the project is the name of the company and
youll can try it on AWS Google Cloud,
which is actually in beta right now. But like AWS and Sivo,
they are production ready and we have support for
Google Cloud. Digital vulture, they're in beta right now, but they're going
to get out to beta really soon. And you can also,
if you don't want to go on the cloud right now and pay for resources
to learn about Githubs. We have the key 3d experience that you can
also use locally. You just need to have Docker, desktop,
everything is in the documentation, so feel free to use that. So I
forgot to remove that slide. It's a recording of the talk.
It's not a live conference. But if you have any question,
don't stop there. It's not because you're watching a recording of that talk that you
need to stop there so youll can go on Qfirst IO
slack. We have a GitHubs channel and you don't even have
to use our product. It's just like, if you have questions about Githubs, if you
have question about the things that I've said during my talk, youll go there.
We're just passionate people about the Githubs principle. We're going to help you again,
even if you don't use youll product. We're just that kind of friendly
people. And also, if you don't want to go
in the Slack channel because I don't know, you're afraid that I'm going to try
to sell you how to use our product, you can always contact me,
send me an email, fred at kubefirst IO, ping me on Twitter,
I spend some time there. Or you can even connect
with me on LinkedIn. And if you prefer face to
face, unless you live in Montreal, in Canada, where I
live, where if it's the case, I would be more than happy to go
for coffee with you. But if you don't, we can have like virtual coffee
chat. So you go on Fred devcoffee,
and you're going to have access to my calendar and you can book a
free 30 minutes discussion with me. We can talk about
Githubs. We can talk about anything you want. I'm just
a social person, and if there is a way for you,
for me actually to help you, I'm going to be more than
happy to do that. So on that note, I hope it helps you
to learn a little more about Githubs. And hopefully that's going to
be, and hopefully that's going to
be a principle that you will be able to use in your
day to day at work. So on that note, have a good rest of the
conference.