Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hi guys, my name is Serge and today I'm going to
speak boot platform engineering. Before I
start, let me tell you how relevant I am to this topic.
In my career I went from a developer to technical
leader and architect and managed a small team.
Then with grow into the engineering management,
director of engineering, senior director of engineering and global head
of engineering. The size of the teams which I managed and worked
with actually grew to more than 100.
And together this complexity and tasks
and dependencies and other pieces, it's not to show
that the titles in your career grow and it's
great, but to show the relevance to the topic. I'm still hands on.
That's why you will see demos in this particular session with
my teams we created different platforms,
from mobile platforms to web cloud platforms where
actually customers were able to use previously internal
development platform. Now they're using this AWS
customers running their applications. We'll speak about this
later. How to achieve this. And as you see,
I'm also a contributor to Terraform,
ECMascript and other pieces
of software and libraries and frameworks.
So let's start.
Main question you need to always to ask is why.
Always start with why? Why do we have the
problem and which problem we have and why platform
engineering will solve so problem. Here we
see kind of request to Chat GPT what
is DevOps? DevOps as you see is methodology.
Methodology, yeah, just describe the culture guidance
and yada yada. So how do we start? Go figure. Of course
I edited here, it was a bit of different answer,
but it was long and not exactly prescribed.
So I checked the
state of DevOps and we have this number of the DevOps
tools on cloud native landscape. You can open this particular URL
and you see thousands of different
tools which you can use for DevOps
infrastructure development. The cognitive cloud went
crazy as a result. Typically in
many organizations we came to the situation
when these tools and approaches use it
in very very different ways, from different hosting to security to
monitoring. It triggered their own problems, right?
That architecturally you cannot align and
provide the golden path. So technology alignment
is hard. Security problem, no way to introduce
the security scans and patches. Typically a centralized team.
Today security growing and growing and growing in the not
investment but attention side.
So there is no way to scalably run security
checks. Business problem, typical ask from
the business is how you can do with less. Now we all have this
particular questions more and more we see what happened
in an industry. So we need to solve these particular problems
and usual problems when it comes to coming to a practice.
Yeah, common everyone wants to actually have speed
to hire speed to onboarding,
right? So the time to the first commit developers
want to remove the cognitive load
and asking give me some guidance framework inside of which
I can just define my front end back end tooling. Give me the guidance.
Engineering leaders apologize. Securities went
to fix securities problems. If you remember lockflow j
or NPM chain problems when
some dependency of dependency is actually security
vulnerable. And you need to solve this immediately
and identify it. Not only solve ops want
to do some improvements and business
always asking the same can you operate with the same twice faster,
delivers the same quality or better and be more cost
efficient. Usual story the same. So you see,
thanks to experience, I believe on your side it's similar. We can
see this from different angles. Okay,
so why platform engineering? Why even speaking about this?
So if DevOps is a practice, so how
can we apply this practice, this methodology,
DevOps can be applied as platform engineering. Even Chat
GPT will answer this exactly that.
You can use platform engineering. So you can try and you
will have similar answer. And we finally
have data. So just typical question is just like don't speak
too much search, show me the data. So the data from the late state
of DevOps showing us that
the direct impact on development velocity from platform engineering
was highly, highly, just like measured and
improved, right? Similar. We have about
development speed, not velocities,
but platform team in general just removes
frictions. And to
summarize DevOps is sorry,
DevOps is the methodology and platform
engineering is a so simple examples,
right? So ECMAscript is for example a standard, right?
And Javascript in the browser or
P eight in Chrome browser as you know it is an
implementation. So how you practically can do one thing
in the way which suits you depending your browser or your
ide similar here how you apply this in organization.
Okay, so what are the objectives and goal
for platform engineering? So platform as
a product that's one of the main goals which you need
to follow. And first, when you start building platform,
make sure that main customers, it's your internal developers and your
main metrics are customers net promoter score
and adoption. So measure your adoption and how happy your customers
what they say about your platform is your metric, right? There is
billion ways to measure this. It's a big topic of different discussion.
So here is typical steps you need to do to
come to platform engineering. You can just follow them. I will not go
in exactly. You can download this presentation and
check the most important of course to make sure that you come
into building cross functional team and you implement your internal development
platform. So if you come into this question
we need to understand how,
right so typical teams topology. If I'm coming to build my
cross functional team, what I need to do first
after all this time of growth of
DevOps adoption, I believe you
defined or you will need to define before going to building
the platform some prerequisites. What is your CI
CD tooling? Is it GitLab, GitHub or other parts? What is
your cloud provider? Is it just Azure VMware
aws? You define it same your tech stack,
your target tech stack. And if you
haven't done this make sure that you're doing this before. Otherwise check
if you need platform reading at this point and
after you can check the team topologies book.
It's a great guidance in this particular era
how you can define teams and typical suggestion is
to concentrate that you have product teams access service or product
tests which result some specific goals with clear
boundaries. And also you will
typically have some kind of organization like that.
Very typical example where you
have the product teams, you have ops, security and maybe
some other aspects which a bit common teams
in practice. So when you start with
platform Jink, make sure that you find a product which did
something what is either already reusable or there is
you see some evangelists, people who are eager to help other
teams to do similar solutions.
If you read the book about
Project Phoenix, you will see that some solution which
was used by one team sometimes can save all
the components. Of course it sounds too just like
interesting, but you need to find the product team
and parts of ops and security which can define
which can bring together in the great platform
different aspects of each. What is development experience
we need in the company? What is the tech stack,
what operations processes we want to have security practices and
yada yada yada, put 510 people and start it.
If you ask when your engineering organizations have at least
roughly 50 people, then start considering this.
Of course it all depends on your needs and timing.
So at some point you will have platform engineers here.
The most important at this point will be your pilot customer.
Make sure that you have a product team which will
first of all needs your solution and wants your
solution. Just like try not never push, just make sure
that they do work doing with the pull. Make them the most happy
customer ever. Yeah, make sure that they just adopting
your solution, they doing the demos, they speaking about
you comparing when
you shouting about your platform and somebody else. Believe me, ten times ten
x better result is when somebody else is doing this,
it's more trusted by others. And so
imagine you came to first customers,
you started being known federal company as a platform.
Start working on these directors with products
to scale. Make sure that CTO you have
as your supporters and then start growing.
At some point you will make sure that everyone becomes your customer.
Security and Ops are your customer because through platform they
will deliver something to whole organizations. And at
some point you will start having sub teams inside of your platform in
infrastructure, front and back end common services authentication.
You will see this will grow. Okay, this is in short what
about teams topologies? Okay.
And now steps of evolution. So how
the hell I start with my platform, right?
Typically you have some stages of SDLC, software development,
development lifecycle. So from starting point how you do
the version control, how you do CI CDCM,
all this particular aspect. So in childhood
is very first point, make sure that you have a simple developer
portal discoverability where everyone can find and discover
what is your platform about and what does it
solve. And start using your solution and provide some
simple tools and libraries how to use
your particular solution. It might
be something simple, a library, a tool, I don't know a script in the end,
but start from adoption, start getting some attention
and then you can provide on this portal just like guidance on CI
CD testing. Some aspects you can even skip
at the beginning. So for example observability or others
from my experience for some reason just like left for a bit later
to be added into the platform. So they just continue being a bit asylum
adolescents. When you grow slowly but surely
you will have more adoption, more support
from executives and but
in from product and other architects.
So at this point you need to use something like backstage to provide a scalable
developer portal and provide CLI tools to generate
your application. CLI UI tools to
provide the guidance on your development portal. The limitation team start jumping
in. So this is adolescent and maturity. At some point
you will see that you need to update and support this developer
portal. You need to provide application templates for front
end, for back end like react, Java, Python,
you define it. You need to integrate this with databases and data
lakes, provide the test layers, security checks,
previews, environment generations,
CD providers, infrastructure as code monitoring
on top of all this. And then you will see the most funny one will
be proper gradability. Great. So this
is the stages to give you some guidance. We will return to actually this maturity
because I will give you some guidance how you can achieve some of these aspects.
Okay, so as I said, I still want
to show you how because at some point it would save me answers.
Okay, in GM portal, how the hell you do
this particular guy, you can use simple tools
like docus hours and let me show you what you can achieve.
You can say that based on your particular solution,
you created some simple static site generator, right?
And you define how
docus hours look. Docus hours is a simple static generation.
Give me just a second. Which is public free
by the way. In this particular presentation I have no affiliates, no links,
nobody paid me for this. Mostly free tools which I will show you.
It allows you to create documentation with versions. You can
say version 12345 of your platform.
You see just like it provider you different content in different
versions and start from something simple. As I said, you can download
my solution, you can download my application simple,
just bunch of files or folders. Start simple.
Provide more guidance on CI CD. This first step for
you. Okay, when you start growing, you need backstage.
For the backstage, of course you need something more scalable
and typically backstage is also free
tool which went open source
recently. And here it allows you to provide your microservices,
micro applications microfontends. We'll come to this in
more details. So this is UI application with database
integration with some other aspects. Okay,
we'll come back to backstage later. I promise you on
front end. So what the hell you do for front end because
most of your teams will deliver some front end depends
on your business. Of course some of your teams might deliver front end. Yeah,
for front end, typical playground is a
storybook. So for storybook here is an example.
It's also free tool and example of implementation. We for
example did this with my teams creating
guidewire web framework. Now it's available
for Guidewire customers and on preview for everyone.
And you start building components. You will deliver
pages using this particular components, how to use them,
how they have different look and feel, how to use
them in for example in the code. All this is easy
to be provided via storybooks.
Make sure that you deliver. As I said, the way to download
your application code and your framework
might be NPm packages. You define describe this
on development portal and then provide storybook like that.
Perfect. And of course you might provide your own templates
for react applications view next you
decided. So here are some links which you can use to define the
different type of applications for your internal customer
teams. Great.
Then we come in slowly to microservices
land which is a bit different beastie.
And here we come to an interesting solution which
is called Jhipster. I'm not sure many of you just heard
about the jhipster, so I will show you this in practice because
before backstage it was one of the greatest
available solutions. So Jhipster is
an open source tool which allow you to also define your application templates
and do a lot of integrations with CI,
CDCM and other parts.
So take a look in jhipster, I will show you how
you migrate an application. It asks me do you want to
create monolithic application? It might be microservice and
micro from end or their combination. So monolithic
name is my app one. Let's go
Springflux. I will go with more or less default
one here is by default it's JavaScript boot application. You can define
Python whatever you want. Authentication, you see it's
already bundled. SQl sure, go with SQl
MySQL. Okay, what about local cache?
You see you can do all this particular steps
as you wish. Registry. Thank you. No elasticsearch
API first here it comes to front end
react laundry UI. Sure. And then of course
it even just allows you to do different config.
Okay, I created this application and
I will actually start it.
So here in this particular application you see already generated jhipster
apps which I did before because it takes a bit of time and
it provides developer experience which is super important.
So you see when you will do precommit, it already will do the static
analysis of your code. That's where you can extend and deliver
this as a template of your application and extend this with the
best practices you want. Add security here, test execution unit,
test execution, whatever you want.
Provide the text standards, provide other
aspects as you wish. So as you see,
it's a simple maven application, sorry,
JavaScript boot application is maven.
And here we can find my
app. Nothing special, some annotations
here and here you can also find.
And here is web application. Sure, just like typical react with
error boundary. You see it's all integrated together, you can
do this separately, you can just do this together.
Okay, let's see how it actually looks and works.
As you remember, we delegated some database, right.
And it allows me to provide a user
and out of the box I have not only web UI which
I can just deliver as an application, but I can manage some
users, I have some metrics and even logs
which I can deliver to my monitoring system,
observability system and just like gather it also
second I also have out of the box API which can
be used in my developer portal. Great. Why I'm showing this?
Because it's fast. Instead of just like describing this documentation
to give me the tool. Give me the tool which making
this automatically. Right, great.
So we delivered all this we allowed imagine
just like to create this particular application but
this is not enough, right? We want to make
sure that we also can deliver these applications.
Give me a second, I will show you. You need
to integrate these applications, not with different cloud
providers, CI CD and yada yada yada.
So Jhipster allows you gives you CLI tool which you can also
deliver to your customers to integrate
with AWS,
Azure, Google Cloud platform and
many other providers. It allows you to integrate CI CD
tooling and provide different integrations
with scalp provider. Great. So you can from Jhipster
you have ability to use your templates, you have CLI,
you have ability to integrate CI CD,
great. But there is one more
thing in Jhipster and most of the items which you see here can
be greatly leveraged in backstage.
So we're going back to backstage and
in backstage I will stop my server here,
not kill my laptop. And here I created
my backstage application. Backstage is
a simple UI application.
This ability to create portfolios, UI plus database
plus monitoring.
So it was exposed by Spotify to open
source built for internal needs and now the fastest standard
for platform genius purpose to save you
months of your work.
So let it start and after that you
will see that actually backstage also
allows you to create your own application
templates. And second,
here is backstage application. And you see I took my
Myvan application and I put here as a template.
I told that you know what,
I want to allow this particular template of
my backstage which I just showed you. So I'm not bullshitting you. You see my
Java application with the same content as before with
web app, with index files.
Here they are the same files, nothing special, just literally
just copy pasted this as a content. It allows you to
create your own to have
your portal with integrated applications similar to what I showed you before.
Perfect. So here we are,
I started my own local example
of backstage and here I'm going to create my application,
based it on the template. This particular app which I showed you
before I added as a template so I can create
and integrate with my CI CD system. You got it.
I am platform developer and I want
to deliver for my wonderful teams ability to
reuse my application which I created
before with the hipster. So let's call it somehow the same,
my app name, I will use GitHub,
you can use GitHub or other integrations. So here directly you can
add this with CI CD. So here make sure that
you provide the organization name and you provide your repository.
And finally of course it will just start it will try
to create the application directly from
here based on the template which we created before from
jhipster. Great. So first it takes my
content of my application. Remember just like I said,
java and also
UI based on typescript and react. Great.
And you see it tries to deliver this to GitHub.
Okay, first just like switch the base,
it applies the name of my application,
maybe some integrations with security
aspects like security keys, and finally
it publishes to GitHub. Unfortunately sometimes it takes some
time. So I will show you that, you know what,
I just added a few things. I also added,
you know what, I want to deploy my application. I want to deploy my application
when it goes to GitHub because it will be pushed to my main branch
and I want to deploy this to AWS, I want
to provide to publish my web to three s,
three bucket and then expose with CDN.
And also maybe I want to do something else. But we
will see. First of all you have
a problem, maybe something
happened. Let me check if something what
went wrong?
We learned something, right Kush?
Why? Anyway, I will just delete
my application, so you will see that I'm not bullshitting you,
just try.
No worries.
Who knows, maybe there's time it will work, maybe not, you'll never know.
So I added the deployment to AWS
and I added terraform to my template.
Terraform is also just can be your kubernetes, can be separately,
can be for your kubernetes, some particular databases or
other. So here I also just like it because it's web application with some database.
I added the publishing to CDA to cloud from
and I want to see all this in action and I hope
this time it will push the directory
properly. Things happen in this life and
we always can learn something, right? Okay,
so once again I created
a template with jhipster.
I use this template in backstage, I added
integration with CI CD in the way which I wanted. With GitHub
you can use GitLab. I use terraform for infrastructure as
code. Perfect. So here is repository and
let's see what happens this time it just like pushed off. So you see I'm
not bullshitting you, it just directly committed to my application right
now. And because I added this wonderful GitHub
flows what happens. My CI CD pipeline
is code started to give me fruits and just
install independency and then it build and deploy to AWS. Perfect.
It will also use terraform of my choice.
Great. Of course I just did this before because it
takes a bit of time and here it is
you see this particular URL is
already published on Amazon
cloud front because it was used in my code
and directly said that where it should be deployed
on AWS. Here you provide secret keys
and you provide the s three bracket and distribution perfectly.
So from local host
to actually published application. Done.
Great. So after we got all the
steps, what's next? It's upgradability believe me
or no guys. But this is 1 million question and
I mean it. I mean it because literally
when you have small platform, your yearly cost
of the updates for your internal only customers is
measured as up to 20,000 dependent.
So when you have medium size
you already have $80,000 per year,
which might be cost of
a salary for a few people depending on
the location. So when it's large organization
starts growing for up to half a million plus dollars
per year, which is crazy amount of money. Question how
you leverage this, how you may
invest some research to decrease this cost
or eliminate this. And of course it's not only cost,
it's development frictions. Yeah, just like if you want to make this smooth and
deliver your updates for customers, you need to make sure
that this is as smooth as possible. So what the hell do
we do? That's where backstage is not exactly
the great solution. So of course
it added our portal, you see just it provided documentation path.
But when I'm publishing a new version of my application,
maybe I changed some API, maybe I changed some imports
I want to update. So how the hell I do it go
do go here. That's why I mentioned for you Jhipster.
Jhipster allows you to say you know what,
between previous version and this version there was some changes.
So create a branch of application
developer code base. So imagine product one
code base try to apply my changes updates,
I don't know imports API and provide
them the branch with this updated version so then
they can figure out what to do next. Okay, this is one
part, but sometimes changes are very specific.
It requires not only git changes but maybe changes in
the code to not follow documentation precisely.
For this you always can use code
shift. So code shift allows you to say
okay in the language of my choice, I want to do some
code transformation transformation just like on JavaScript transformation on
Java on your needs.
So they procreate some ats three from your
files and tries to apply the changes you describe here.
For example, react code mode is a JavaScript react
implementation of code
shift. How you can do this? For Javascript there is Java implementations and
others. And for dependencies
management you can use renovate it's also a free tool which
support crazy amount of languages from Python to Java
to node js and anything. This is just great. Which allows
you to say okay for different reasons, you want to update your dependency or
dependency of dependency or you don't.
Great. So all these particular pieces giving you
a strong platform. But you know what, as I said,
it's just like sneak peek pick of all the pieces
I created for you when I was preparing for this particular
presentation. A repository,
this created list of these tools and blogs
and actually different
informations and links related to platform engineering. All these pieces,
how you provide code templates, how you create microservices
front end, how you provide upgradability and apply
different aspects like I don't know, Kubernetes provider, some infrastructure
platforms, how you do all this here.
So make sure that you go in here and just find the
best tools which you want to create your platform.
If I had this at some point it would save me many months.
But now it's time for me to share my experience.
Great. And from this
great experience, sometimes that's how we call our
mistakes or learnings. Let me give
you a few advices. So make sure that pilot team
you are super involved with them and they become your
best friend and evangelist. You will get a customer and from customer
you will get evangelist at the top of your customer ladder.
Treat your platform as a product as
soon as possible. You need to deliver business value
business. I'm sorry, between us very honestly, not very
often cares about technology. They care about the product, about the
solution, about customer marketing, sales, technology.
It's your responsibility. Make sure that you solving the business
problem. So business problem with adoption is speeding up.
It's just like cost all those goodness. So for
this, treat internal teams as customers as soon as possible.
Provide them slack, provide them the
developer portal, invest in platform PM.
Just very great topics, very big pain to be honest.
Grow either internally PM or find the guy who is crazy
interested in all these pieces and know how to speak with customers and how to
build platforms or ie is eager to do this evangelize.
So you need to send regular releases, you need to be predictable. Do the
demos newsletters, make sure that others speaking about you
as in any other product, it requires the stages
of evolution and few interesting things.
So for UI adoption would make sure that you
put teams with the UX. So for example, my best partner
in crime was head of UX. Make sure that your developers with UX working
very closely that they give the design.
Then you have front end components for this same for your
ops team. Yeah. Just so if Ops teams is saying we start implementing
some monitoring solution new relic, I don't
know. Just like Datadog, make sure that you provider in the tools for this
in your platform and this goes together. They're speaking about you,
you saying that you're enabling them dumb down just like win
win and cooperate with all the architects, directors,
VPC here.
Actually you need to make sure that from
the top down you have ability to enable bottom up.
That's the thing you are looking for.
And jackpot strategy. This is my favorite. Yeah.
If you find the way how to deliver the platform
which previously was internal to your customer,
you not just cost center. And where
companies see the spend, businesses spend,
they see the profit and profit will be just like your friend.
So make sure that you find in the way how you can deliver your goodness
to customers. This way you will have
test based inside of your company great fast feedback and
you will have customers also win win situations situation
for everyone. Okay, so to summarize,
remember this maturity of our platform. I gave you some ideas
how you can provide this portal solution
for discoverability. How people can find your platform
and start with it. How you can use your application template,
how you can integrate with data and provide
test checks. Because you can include this in your CI CD
pipe as code. You can add CD
and infrastructure as code inside of your flat
templates and in backstage or jhipster just added
separate steps, template creator creation and CI
CD of your choice. Code observability also
should come there. Tracing should be integrated.
Everything should be just going to backend and front end
and infrastructure immediately and upgradability. You saw
jhipster code modes and other solutions can help you with this.
As a result you will provide technology enablement
so it's centralized way to introduce new stacks and libraries. You want
to migrate from Python to Java,
provide a new type of application and start actually working on the
feedback. New cloud adoption,
security and compliance perfect example.
Look for J or NPM
versions problem we had before with the security you
publish update in the platform in
the centralized way and immediately through this upgradability tooling.
Q. Deliver this to all your customers internal and
external. Vinvin perfect. Amazing. And also DevOps will be
happy because they can introduce the new layers together with security
common security checks across all your organization
cost one you need to measure this but it's just literally we
have state of DevOps, we have research,
it reduces cost of onboarding technology
alignment pivoting. If your company
acquired other company and want to integrate or want to change the tech
stack and upgradability cost is just a clear
win result.
Okay guys, what I can say here are the links
to the slides which I just showed you together with the resources
which I created as a GitHub repository.
The best day to start is always today. Make sure that you
start in your journey. It will really take thousand miles,
but you will get to your mountain, to the top
of to your dream, right? And make sure that you're doing this first single
step. I gave you some guidance, shared my experience.
Good luck. Now do it. Build it.
Have a nice one. Bye guys.