Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hello, everybody, and welcome to the session. Just don't do it.
Postgres in Kubernetes, or I should
perhaps should say postgresql in Kubernetes, which is
a very hard thing to pronounce. So we'll just stick with postgres
in Kubernetes in the next 30 minutes or
so. I want to take you through some of the considerations that we
have when we look at postgres, if we look at new,
new ways of developing applications and actually incorporating
data processing in the world of postgres and into
the world of data processing.
So without much further ado, let's just jump into our
first slide. So basically
what we're going to do is look at postgres and cloud.
And there is a lot to say about this. There is
very little time because we only have like a very short slot
here. But before we dive in,
just let me quickly introduce myself. Who is this
guy sitting here in this presentation thinking that he
could tell you something about this? Well, my name is Jan Karremans.
I'm a techie in sales, which is a little bit strange, but I
started out in database technology many years ago,
first in the world of proprietary database technology,
but at some point decided there
should be more in the world. So I migrated myself from Oracle to Postgres,
currently a member of the international database
community. And just over three
years ago, I guess I got really involved in Kubernetes and really
trying to understand how this technology helps
transform the way we build applications.
And that kind of put me on a mission. I worked very closely
together with the team that did a lot
of work on how do you put postgres
in a container? How do you put the elephant in a box, if you like?
So my background again, I said I'm proprietary
databases Oracle in my case,
I am KCA trained
and I have a little bit of a postgres background.
Meanwhile, currently at this company
called Cybertech, a pure play postgres
company that helps guide organizations in
their journey in adopting postgres.
And really looking at what are some of the challenges
that you face if you move from a traditional world to this new world
and how can you get the most benefit out of it.
I believe that running data
management solutions like postgres in Kubernetes is
really the way that in the next decade or so,
looking at some of the things that we're going to talk about from an application
development perspective, from an application architecture perspective,
really are things that are going to contribute.
Good friend of mine, Mark Lindster, currently postgres
fellow described this as postgres
is the most transformative technology since Linux.
I think he's right. If you look at
the first wave of open source as I've once defined
it, bringing open source solutions
to the world really started with a little bit and
I don't mean this in a demeaning way, but with
a little bit simpler to replace technology. So you've
got tooling, you've got perhaps even something which is
a little bit more complex like an operating system, but they're
less intrinsically interwoven
into an organization as perhaps database technology.
So this first wave of open source really focused on the simpler
quote unquote technologies replacing
those from proprietary solutions to open source
solutions. I think operating systems are a very
good example of that.
In the early days we had proprietary operating systems
like VMS, like Solaris,
like Unix, et cetera. Those have all been replaced with this
open source equivalent called Linux.
Most every system is running Linux.
This change, this transformation is now also coming
to the data processing world where really
truly community governed open source solutions
like postgres is built. Also like Kubernetes is
are taking over this high end
of the IT tool spectrum. So postgres
in that sense the most transformative technology since Linux.
And this is one of my personal most favorite slides.
It's super hectic, it's super busy, but it discusses
the origins of relational database theory.
And even though relational database theory is the foundation
of postgres with all of the additions,
with all of the extras that has been built in postgres over
the years, Postgres really
is a multilingual
data processing platform. It speaks relational technology,
it speaks document database or JSON based
approaches. So that is really where
this is going today. And if you look at the chart, it started with
System R, the great debate 1974. For those of
you who are interested in history, had a bit of a discussion
around which kind of database would
win relational database theory at that point.
One. So that technology split it
up in oracle on the one side and ingress and later post ingress or
postgres project on the other hand. And that evolved
through the years to where we are today, where you can clearly see
postgres from a data processing platform is winning.
And this is why I think also in this setting,
cloud native setting, having a good understanding of
how postgres plays here is of a vital importance
from that perspective. If you then start looking, okay, so we had postgres
or we have postgres. I should say this
has been around for over 30 years. It's part
of the traditional way in which we do
monolithic application development. Then since
1015 years, the common arising of cloud, the maturing
of cloud. And then based on top of
that, you have where cloud native comes in, cloud native
really as a mindset for how do I build
applications that in this
world of cloud actually use
the power that this agile
and quick evolving technology brings?
It requires a different way of building applications.
Even though postgres is 30 years old, it feels like it
was created to go through these motions of cloud.
And cloud native, I think that's a
good thing to have in the back of your head as you start looking at
big monolithic applications, which is
nothing bad to the way that these applications are built.
But if you look at today's requirements
in organizations where new features, we need to
have the application do new things, it needs to do something different,
really is a big driver for changing
applications. If you do this with your
traditional big monolithic application, it's really hard to
make that swift shift. I mean,
traditionally you would say, I'm going to do requirement engineering,
then I'm going to do a technical design, I'm going to do an architectural
design, I'm going to build my pre code, I'm going to actually code
the application. And by the way, by the time you had
your first viable
thing that you could test, you would be two years ahead.
In those two years, so many of these requirements would have
changed that it's virtually impossible
to keep up with the
evolving of your business these days. So from
that traditional model we're shifting to, and I like to use
this equivalent with this cloud
of cards in the sky where every
bird reacts to its neighbor, right? If that swift
shifts a little bit, then all of the birds around it will shift also
a little bit and you get these gracious patterns in the
sky. I think that comparison, for me at
least, looking at how requirements influence
this, I think this is also how applications,
and if you start looking at microservices applications,
where the individual parts of the application change and
evolve, much like the position of the bird in the cloud of cards,
is basically a model that
gives you the ability to swiftly interact on
changing requirements in your business
as postgres, as a data processing engine fits
in this model, or if you would be able to do that,
that would add tremendous value.
We looked at cloud native in one
of the previous slides. This basically boils
down to how do you
adopt the cloud native mindset, which basically discusses
how developers and code reviewers, et cetera,
work on a repository, really make that into
something that evolves around a development branch,
automated testing. So there is a lot of
these things defined on how
you build an application. What are some of the KPIs around
application building an application evolvement
and development, et cetera. And how do you deliver this application
towards end users? Are your continuous
integration? How do you do continuous delivery?
When I started to look into this, one of the most profound
things that I found is that continuous
integration, I thought was the hardest part, right, where everything
you do always have to be, sorry.
Continuous delivery was the hardest part because
everything you needed to do, you need to be able to automatically
make that available to your end users, and that needs to go fluently.
But as you start diving deeper in, the continuous
integration part actually is the harder part because that defines
that your application always has to be in a state in which
you can actually deploy it. So that puts
extreme requirements on how do I implement
changes, how do I test changes, how do I continuously
ensure that the application that I have
sitting in my repository is in a state that I can
put it out there in production.
So this model
then starts defining and focusing on
the performance indicators actually required to
start doing this, this journey
for me, because originally I'm a database person looking
at this and bridging the gap from my world where
I think about data persistence and how do I
record data and et cetera, et cetera, and shifting
that mind towards, okay, so I am now also part of
this development chain. I need to start thinking about
how do I actually work through the
changes? How do I look at continuous testing, how do I look
at test data management, et cetera. So that
was really a learning experience.
And that kind of brings
us back to how do you go from a monolithic application
to agile delivery?
Cloud in that sense, changes much more than just
looking at. We all know, and again,
no disrespect, but we all know the manager that comes in, right?
And I think, Dilbert, for those of you who remember that comics,
trip also had like a phase in that manager
comes in tomorrow, we will be cloud native or
we will be agile and just like
flipping a switch. Unfortunately, it doesn't
really work like that in reality.
So going from a
traditional monolithic application, from this big steam train
that is moving forward to this flock of birds,
requires many changes in
the way you look at applications, how you approach
application development, et cetera. So this is not something
you do just on the side overnight,
right? So getting
into this, thinking through this, evolving, this is really
a next challenge. Application and
we saw that in the previous slide already.
The buzzwordbingo, I think in it we're
really good in buzzwordbingo stuff.
Everybody today is talking about AI, AI,
AI. The big question is AI
matures, which will happen in, I presume,
just a couple of years. What will it actually change
in everyday life? What will it actually change in
your day to day business? The reason for this
is because we've seen this a couple of times, right? Big data,
cloud would change everything. Yes, cloud has changed everything,
but only now, and this is 1015 years after
cloud started. We're now really starting to understand
how cloud influence day to day
it activities. So also from that perspective,
keep in mind that a lot of this stuff is buzz, it's marketing
buzz, it's sales buzz, where people try to get
you to focus on their solutions, on the things that they're bringing.
But one of the things is please step back, look at your
own requirements. And how do these buzzword
things, how does agile, how does microservices
help you deliver more value for
your company? And when it comes to cloud,
cloud native and that mindset part of it,
I think we're getting to a point where this has matured
so far that we can say, yes, we can
start out of the buzzwordbingo part and
actually look at what do these things define.
And I just put four on the slide here.
So agile development,
it focuses on deploying,
redeploying of applications, keeping your mind
in the back of your mind, the KPIs that you
want to fulfill in how fast can I deploy an
application? When we were looking at a tanker or at a
train, a change could easily take six months
to a year. Today we're looking at
bringing out an application every other hour
instead of every other year. Right? So that's
one. The microservices part really has to do
with going from this big block to the
individual nuclei in
that cloud. How can we break down an application? How can
we find the essential granules?
If you like to start working with
this flexible mechanism, this grid,
if you like that, your application is. So that's another
element which sounds very much like marketing, but actually really
has a good meaning. The DevOps
part as well. I think DevOps over the past
couple of years has really become really
settled and really well defined and ingrained
in every other organization. You don't just
install a single server, but you write an ansible playbook
or you write some script that actually does this work for
you in an omnipotent, repeatable way.
Every time you run the script, the outcome is exactly the same.
And if you want to change something, you don't fix something on the server,
but you just change the script and let the script do it for you.
As we're going to this new world, we're just going to throw away
the container and spin up a new one.
And then what I talked about just now is
CI CD. How do you continuously
guarantee that your application is in a state that
you can deploy it, but also that you can ploy it by the push of
a button? So these are really four elements
that have matured over time and really
are now in a position where they're part of the way
we look at it operations.
So that's kind of interesting.
With that, I want to briefly introduce Dora.
Dora is for me at least a report,
research and capability that Google
Drives that really focuses on the KPI
part of things. How do you start measuring
your success as a
cloud native inspired organization?
How do you perform? How quickly do you roll
out applications? What's the mean time between changes?
How fast can you restore a
specific service back to operation? If you start working
through this, what's your deployment frequency? What I said earlier,
going from an application which has a new version every year
or every six months, to an application that has a new version every
2 hours and successfully deploy
that and really have reliable numbers
around, how do I do this?
So sre site reliability
engineer really came out
of this motion where understanding
how we build and how we deploy applications really is
part of how we work in it these days.
Again, I think all very interesting background,
both from a cloud native perspective as
well as from a data management perspective where we
were looking to bring this together.
And that kind of brings us back to this a little bit.
Daddy, what are cloud made of? Well,
mostly Linux servers these days having
this understanding and really taking
a step back from, again, the buzzwordbingo and all of
the spin that's been giving, and I can't say
this often enough, just sit down and think about what
does my organization require that
I can start building that actually add value?
And I've been working with a specific organization that
has a big monolithic, old fashioned application
and what they're actually doing is looking at
deconstructing the application piece by piece,
defining very nice specific parts
of the application specific function that that application is doing.
You might even think, okay, so this is the part of the application
where a truck driver comes to a factory and they
need to go from the entrance to a specific loading bay.
How do you make this go
fluent. How do you give the truck driver an
app in which he just could say, I need to go to
loading bay three, dont 15. That is, I can
actually have a nice chart on my mobile that says go
left, go right, and you'll end up actually at that specific loading
base, specifically at the time where you need to be. So these
kind of developments is what cloud actually
enables us to do because of the interaction and because of
the open access that these
technologies and this development brings.
And that brings us
a bit back to the flock of cards, right? It's almost like
a neural network where all of these microservices,
and specifically when then you look at postgres
and the things that postgres as a platform can bring, and the
way that data can be replicated between some
of these services makes for something where not
just the individual granules, but also the link between
the granules and where every element knows
of another element, just exactly what it needs to
know can be built in such a way that it
starts working like a neural network. I think this
is a very nice future perspective for
what things that we could start doing when we bring
postgres and cloud native and the cloud native mindset even
further together. Super busy slide.
And please pause the video if you
want to read this, but this really describes
a little bit what the cloud native
mindset is about. So it's really a definition slide
that I just wanted to include for you guys and girls to
look over as we start
wrapping this up. You're probably
familiar with Kubernetes, Kubernetes as a
community governed open source project. We're going to bring
this together with the Postgres project,
one of the most powerful, one of the most widespread community
governed open source projects resulting from the work
of Michael J. Michael Stonebreaker at
the University of Berkeley, and after him, Mr.
Bruce Mumgen, the founding father of the Postgres global
Development group. So we have two very powerful
communities, and we're actually bringing them together in
this new environment where we bring these two
elements to bear for what we've just talked
about. How can we start bringing data management in this
cloud native mindset? So to that extent,
the cloud native PG project was
created, now part of CNCF.
And thank you to Gabriele Bartolini for building and driving
this project. And basically what it is, it's run postgres,
the Kubernetes way. So to Kubernetes,
Postgres is an application that has its specific
requirements, that has some specific elements to it,
running persistent data loads in Kubernetes has
often been viewed as that's challenging,
that's perhaps very awkward to do.
But at this point in time, we're at a point
where you can run postgres the Kubernetes way.
So rather than bringing Kubernetes to postgres,
what we did is take postgres and actually bring it to Kubernetes and
make it build and run the way that it
should run in both the cloud native mindset
as defined by the CNCF, what we saw in the
previous slide, but also take the goodness and benefits of
postgres and actually exploit them as best as we
can. So if you want to be part,
please scan the QR code or just look
up cloudnative PG, IO and
look at this project. Become part of the project again,
a community governed open source project.
We recently hit 3000 stars on GitHub.
So from a postgres operator perspective,
this is really the way forward. It's one of the most powerful
operators at this point.
And what it basically does is it captures a database administrator,
puts it in a box, right? So day one,
day two operations. How do you bring
all of the requirements for running a data processing
system to Kubernetes? That means high
availability, switch over, load balancing.
All of those elements are covered by
cloud native PG, in this case by the operator.
The operator basically teaches Kubernetes postgres
and for you as an end user, it's basically
just deploy and use. So from that perspective, it's a
very interesting and complete product
and process that
brings me a little bit to the last slide and I
think this picture is a very nice illustration of
that thought. We're here at
the date this presentation is given.
We're at this specific point in our journey.
This is going to go somewhere from this place.
And the beauty of this is as these
are community governed open source projects, I said it before,
you and I together define what the
rest of our journey will be like. Where will we take this
technology from here? What will be
the existing requirements? What will be some new requirements?
Or what will be things that will be asked of us
tomorrow, next week, next month, next year?
Trust in that being and adopting
Kubernetes and postgres together gives you
two very strong tools that allow you to
take your journey exactly there where you need it to go.
And I think that's fairly unique and unparalleled
in this environment. So that's
basically the message that I wanted to share with you.
What we do is we build our
applications that we build, and we try to help you,
basically, for the things that you need.
So we would love to hear, come back and get in touch with
you and hear your thoughts to what we've talked
about here today, but also what some of the other elements are.
So for that, thank you very much for being
part of this presentation and until soon.