Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hello, everyone, and welcome to our session.
Let me start this session with a bold statement.
The next big revolution in platform engineering isn't about a new tool.
It's about reimagining the entire concept of a platform.
My name is Martin Zarkowski.
I'm a senior solutions architect with AWS and have a long history in
designing and also building platforms.
And today with me is Marina Burkhardt, also solutions architect at AWS.
And Marina has a strong DevOps background and experience in
administrating and operating platforms.
And we both, as solutions architects, are very much interested in
building the platform together.
of the future.
So today we want to share our perspective on the future of platform engineering.
But first, let's look back.
Why are we doing this?
Why are we building platforms?
And what are platforms?
Would I would define it as a group of technologies, that are used in as a base.
And upon this base, upon this platform, we can develop application
processes and other technologies.
So we basically build some common components that handle
a lot of generic stuff.
And on top of that, we can build something that is special and unique.
and we know this is a concept that has been used in many industries and
they were very successful with this.
using platforms to transformize their business.
How does it work with software?
let's think of the main goals of platforms in software development.
It's about increasing the development speed and still meeting all the
requirements that we get for our software.
And in order to have this right balance between very high speed And high quality
and meeting all the requirements, we need something that is helping us.
We need abstractions.
And often those abstractions come with standardizations.
So platforms often enforce standards.
And if I talk about standards, I often hear people saying that
standards are taking away all the nice freedom and flexibility.
Because people think of standards as limitations.
But I would say, and this is a bold statement now, it's quite the opposite.
Standards actually enable innovation.
Standards help you to scale and standards remove all the friction.
I want to give you a few examples where I believe that is true.
So think of this highly standardized piece of paper.
210 by 297 millimeters.
it's not roughly 200 by 300 millimeters.
No.
It is exactly 210 by 297 millimeters.
by 297 millimeters, which is, by the way, exactly 1 16th of a square meter.
And this standard 210 by 297 millimeters, it's called DIN A4.
And I believe it's a great standard because our If we're talking about
DNA4, no one has to argue how big that piece of paper will be.
We can rely on a certain size.
And we can build something that is, supporting that.
we have fitting envelopes, we have folders, drawers, pockets.
They all, we all know the right size for them.
We can even build printers, fax machines, but we don't do that anymore.
But, like all those, things that work together with that piece of paper.
And yes, if we have that.
Standardized piece of paper, we have to adjust to that standard.
So if we write a letter, and I have this very long sentence in
mind, I have to use a line break because of 210 by 297 millimeters.
Or if I want to draw a picture, I have to accommodate that into this format.
But think about it, does that really limit you in writing texts?
Does that limit you in drawing pictures?
But we have seen great texts, we have seen a lot of art.
on a piece of paper, standardized piece of paper.
And we have the same with technology.
let me give you one example, very low level.
we are using HTTP to make sure that our browsers Can talk to web
servers, to other computers, and we are using this standard everywhere.
So actually we can't imagine a world without that standard.
yeah, there are some limitations.
There are some standardizations in, in HTP.
but this standard actually has enabled the internet and it never put
a limit to the innovations that we are seeing today with the internet.
So we have to realize a few things, locking things down
and agreeing on a few things.
can actually help us.
It can boost innovation and it can boost creativity because those
standards and abstractions, they help us to remove the cognitive
overload we would have otherwise.
So they help us with increasing the speed of development and they also help
us to still meet all the requirements.
So if we want to be successful with software development, We need
kind of abstractions and standards, so we need those platforms.
But there are many platforms and platform engineering is a term nowadays.
So what makes a platform?
A great platform.
Maria will explain some of the key characteristics for a successful platform.
there isn't a golden answer to Martin's question.
But at least I brought you here three different characteristics
that I think and consider that every single great platform should have.
in order to be successful.
So let's dive into those ones.
As we just mentioned, platforms are going to come with abstractions.
So that means abstractions and standards are going to be in place.
And you might think that those are going to limit you, but
actually totally the opposite.
A platform should be an enabler.
A platform should hide the complexity and should make it easier to use.
And in order to achieve that, you should have a high level of transparency
and make usage of shared components.
So a platform shouldn't be a black box.
A platform should have a high level of documentation so that every single
person out there in the different teams can read that documentation,
understand how the doc, understand how the components were built and which one
was the intention of the usage of them.
And also you can, make the code of the platform open source, at least
internally for your different teams.
And with a combination of two, of those two, those things, you're going to
create a shared responsibility feeling across the different teams, which at the
end is going to be beneficial for you.
Nevertheless.
you do not want to reinvent the wheel.
So that means you would like to have those components that you built shared across
the whole platform so that multiple teams can benefit of those components as well.
Second characteristics, from great platform is that you would like that
your platform is also extensible.
So that means you do not have to force it.
you should see which are the features that every single team is going to need
in the future, but at least stay flexible in such a way that you can learn which
one are the technologies that they are using, learn about the new tools
that every team might implement or use from time to time, and also stay alert
about different architectural patterns.
There are trends in the industry and in the market out there.
So that means you do not want to, force anyone to use a specific pattern,
tool, technology, integration, but make and, or build the platform in such a
way that it's attractive for multiple teams and all the teams actually to
really make usage of your platform.
When that happens.
More and more teams are going to, use or want to implement
things on top of your platform.
And therefore you would like to be able to scale that platform as well.
And that would be able just if you have a high level of automation in place.
So you can use infrastructure as code, like for example, AWS, CDK.
Thank you.
to just make usage of the possibility of having a great automation
and keep the pace of new teams onboarding in your platform.
The third characteristic is that you would like that your
platform is also evolutionary.
So that means you do not want to have a platform that is static.
You want something dynamic that keeps the pace of technology out there.
With the time, you would like that this platform actually reduces the TCO, so
the total cost of ownership, make it more efficient, and more, performant as well.
And with the time, it should be easier and easier to use as well.
How do we achieve this evolutionary characteristic?
By treating the platform as a product.
So that means having a product mindset.
Nevertheless, something that experience taught me is that you
can have the best intentions.
You could have the best, specialist knowledgeable in a specific topic.
But if you do not have mechanisms in place, there's no way that you can scale
this across your organization and have the impact that you would like to have.
to have.
Therefore, I brought you four different characteristics of those mechanisms that
I would invite you to think and consider when building and designing your platform.
Being the first one about abstractions, so we want to build a platform
in such a way that it abstracts.
It has the complexity, but it has a high level of transparency so that
everyone is going to be able to know how those components were built.
You would like to have some same defaults, so having the possibility
to set some, values per default.
That helps the teams to reduce the mistakes.
So reduce the errors, but at the same time have a high level of
flexibility because you will going to, give them a set of configurations
that are already properly set it up.
The third abstraction is about automation.
So you would like to keep the pace of technology, but increase the
speed of every development team.
and at the same time, reduce the friction of the processes.
And last but not least, those platforms should actually help you with automation.
Some control security things related.
So we want to help you to, to enforce compliance measures and
follow any regulation that you have to take in consideration, related
to the market and the industry that you are part of while having this
high level of share responsibility across the whole different teams.
And the question here would be, whom is this platform actually going to enable?
He said the platforms are very great way for centralizing expertise, but he also
said you cannot centralize innovation.
You have to leave that to the teams.
that are the closest to the customer.
Because those teams, they obviously have the best ideas and they will
build innovation on top of a platform.
Let's think about those teams that are doing the innovation.
How do those innovative teams actually work?
usually those teams are small and self sufficient.
So that means they can do most or maybe all of their work
independently from all the other teams that are also doing innovation.
And they can make most or all of their decisions on their own.
Let's think about this.
Autonomies, teams on a platform, how does that work?
Here we need to talk about the centralized versus decentralized approach.
Let me start with the maybe more traditional centralized approach.
We have a platform and that platform is designed and also
operated by a platform team.
and of course we learned platform needs to be evolutionary.
so we have to add new features to our platform all the time.
And in a perfect world, your platform team would anticipate
every single feature request.
And they would implement all the features.
before someone even asks about the feature.
But now let's look back to the reality.
So in reality, it's different.
In reality, app teams often request a new feature, and they want this
feature as soon as possible because they need it for the innovation.
And then your platform team needs to understand this new feature.
And then the platform team needs to build the new feature.
And all of this while operating a central platform that is growing a lot.
So You're basically moving the cognitive load from the app
team down to the platform team.
And if you get more and more app teams doing this, what will happen?
You will overload the platform team and your platform team
will become your bottleneck.
Because what are we doing here?
We're not only centralizing expertise.
We are also trying to centralize the platform innovation.
How does it work for a decentralized platform approach?
in that approach, the platform team is designing blueprints
as a platform component.
And all the app teams, they can use these blueprints and they can use
them and create their own versions.
of that.
So we actually put a bit more responsibility into
the hands of the developers.
So the developers, the app teams, they own the entire lifecycle of the
application, including all the platform components that they're using as a base.
And while you could think maybe they have now the cognitive overload, but
they don't get this cognitive overload because the platform components, the
blueprints, they are using abstractions.
that are provided by the platform team.
So they basically don't have to reinvent everything and build it from scratch.
They can build upon those abstractions.
So they don't get this cognitive overload to understand the full
stack completely from the beginning.
So app teams can operate their own stack, but they still meet
all the central requirements.
And if required, they can evolve the stack, maybe on their own or
together with the platform team.
So what are we doing here?
We are centralizing the expertise into those blueprints.
But then we decentralize the innovation to the app teams.
Let's think that a bit through.
How would that work in reality?
if we have this shared responsibility between app teams and platform teams.
Who's doing what?
Who is responsible for what part of this deck?
Here we need to talk about ownership patterns and Marina
will discuss a few of them.
So there are different approaches and I brought you three of the ownership
patterns that I've seen the most with our AWS customers in the way
they have treated the productive workloads in their AWS accounts.
By being the first one, Treating the whole AWS account itself as the platform.
And the way you can imagine this is like by having an AWS account vending
machine that is going to be owned by the platform engineering team, that
it's going to provide AWS accounts.
With already some, building blocks and components defined and deployed in the
AWS account to every single team out there in your organization that needs
an AWS account in order to run the resources needed for your business.
The different projects and use cases, and those components might be
related to networking, observability, security, or anything out there that
comes to your mind, where the platform engineering team is going to, for
the networking, for example, design the whole network, like a network.
how many VPCs is going to, that account is going to have, how many subnets?
What's, how does the routing table, for example, look like, and so on.
Related to observability, you might have already some, I don't know, logs
defines, metrics, some dashboard, all of the, all of this may be in CloudWatch.
And also specify and already implement some security mechanisms
in that account as well.
How can this platform engineering build such a vending machine through,
for example, the usage of AWS, organizations, AWS config, CloudWatch
and IIM, for example, as well.
The second ownership pattern is related to managed clusters, where
in the other approach, you had like a higher level of flexibility, But lower
level of abstraction that might not really be needed by every single team.
Like for example, a team that runs some containers, would not really care
about what's the VPC that it's already there in place in the AWS account,
but instead having a possibility.
to just run their containers in a cluster that it's already managed by another team.
So that means they still would like to have those standards components to
networking and observability, but tailored specifically for a managed cluster.
So for example, They would like to have some health checks, or maybe have the
possibility to monitor how many containers are running the, in the whole cluster
itself, or maybe have some possibility to, manage the users that are going to
have access to the cluster and also some input and output rules for the whole.
container itself that it's running there.
So this option is going to have a lower level of flexibility, but
a higher level of abstraction.
And the third ownership pattern is related to deployable application patterns.
So this is like providing blueprints, like providing some AWS CDK constructs.
this might.
for you not resonate as a platform and rather like blueprints and guidelines
on how to deploy things but this is yet another ownership pattern for a platform
itself and this is pretty much a modern way also of treating it than I also
seen A lot of times we're actually the teams that might benefit the most of T
of this are the high performing teams are the ones that are already used to
work with, for example, AWS accounts and need some blueprints that helps them to
deploy some applications faster with some standards that are well defined there
with some architectural patterns as well.
That might help them to reduce the development time and improve
already, the, the possibility of building something on top as well.
Having now those three ownership patterns in mind, I would like to
hand over to Martin, who is going to actually explain how by having the
possibility and the freedom of different patterns, you do not have to force.
How can I use any team in order to implement and make usage of this platform?
you don't have to decide exclusively what pattern to use.
You get the freedom of choice.
Let me show you how that could work.
Let's imagine we have this platform team and they're dealing
with, let's say, four teams.
These are decentralized dev teams.
first of all of our teams get those pre configured accounts
from the account vending machine.
And, team one is perfectly fine with that account.
So they get this pre configured account as a base level platform,
and they build everything else on top of that, completely on their own.
team two is different.
They use those managed clusters.
And on top of that, they also want to have those deployable patterns, maybe
in a cluster that could be a hand chart.
team three, deploys the application into a managed cluster, but they
don't need all those patterns on top.
They have their own way of doing things.
And team four, let's say they're the serverless game.
So they use those templated patterns for serverless applications in
their pre configured account.
now we have four different ways of doing things.
Let's reimagine the term platform.
All of this is your platform.
Some parts of your platform are more central, like the account vending
or maybe the managed cluster.
And some parts are more decentralized, like all those deployable
application patterns and so on.
And by giving all the teams the freedom of choice, you don't have to
force a team to follow one approach.
That might work for you, but that wouldn't work for me.
for them.
So the idea is to make a platform not too restrictive, but make the platform
open to innovation and to development.
And I can recommend working towards those decentralized teams with shared
patterns and services as a platform.
And you can see those blueprints that are shared on your platform as a golden path.
Maybe this concept of a golden path is very well known.
Maybe also the concept of a paved road.
And let's see, those blueprints as a golden path.
On a paved road.
While this is the golden path, there's still a bit of room
to do things differently.
On a paved road.
So that's your opinionated and supported way.
of using technology.
this is the way to make it for the teams as easy as possible to use
technology and to use your platform.
But we want to be flexible.
if someone has an alternative idea, let them go but left or right from
that golden path on that still paved road still, to a certain degree.
there are still guardrails, outside of that paved road.
Maybe.
this idea will open another road, maybe a new shortcut, maybe, this improvement
will even be your new golden path.
So basically treat your platform as a product, make it similar to your
product development, have a product mindset when building your platform.
how can you do that?
we can learn from the development of open source.
products because they follow the same principles.
So whenever you build something for your platform, make it versioned.
So a version piece of software that can be evolved by the user community,
maybe make the platform team as the main contributor, but allow other
ones to contribute and create their own version, make it decentralized.
So every component should have an independent life cycle that
makes the scaling much easier because you have smaller pieces.
And, it also reduced the blast radius and dependencies and so on and make it usable.
So if you have this product mindset that we just discussed, you will
create good accessibility like good documentation, scaffolding, a
CLI, a UI integration, and so on.
And then make it customizable.
So be open to changes.
Allow someone to parameterize your product.
your platform components, allow to choose escape hatches if possible.
Let them fork it, let them change it if they need.
Don't build a black box, make it transparent.
Make it a white box.
Don't hide the implementation.
of your platform because people rely on it and then make it self service.
so everything should be fully automated as much as possible.
So everything can be requested and deployed via APIs.
So that is an evolutionary platform guided by the principles
of open source development.
And one last thing to mention here.
Always stay ahead with your platform.
So we discussed this, a platform will always build on a certain base,
like the AWS cloud, for instance.
And we also discussed that your platform should evolve.
But here's one thing to remember, the base that you built your
platform on does also evolve.
So if you don't evolve your platform, the things that you're building
on evolve on top of that base.
And you don't adapt it to the base.
you will have a syncing platform, a platform that
has no true benefits anymore.
Maybe the platform even makes it more difficult to do things,
compared to the native approach.
So that will be a platform was a burden and no one wants to use that.
So if you always evolve your platform and you always adapt it to the base.
You will have something that we would call a floating platform, a platform
that is always ahead of the base.
And that is a platform that offers true benefits to all the users of the platform.
So a platform that enables all your teams.
So our recommendation is always build floating platforms.
What did we learn today?
So standards are not limitations.
Often I would say they are the key to innovation.
So don't build a platform for the sake of it.
of having a platform for the sake of doing platform engineering.
Build a platform that enables your teams.
A platform that is extensible and evolutionary.
And always think about decentralization of the platform with centralized blueprints.
And if you do that, make it as flexible as possible.
Build a golden path, but be flexible.
And always Stay ahead with your platform.
Thanks a lot for listening to our talk today.
we really value your feedback.
there are two links, how you can reach us.
we're happy to discuss that concept even further with you.
And if you have some feedback for us, if you liked what we told you
today, there's a feedback link.
We're happy to hear your feedback.
It only takes 15 seconds to fill out the survey.
Thank you so much for joining us today.