Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hi, I am Ricard Miranda, and I'm here to talk to you about
automation in modern project development. I'm a consultant
at closer consulting. Let's start with movement from monolith
to services. A monolith is a very
big application, and we are now observing a
movement from the monolith into splitting
the monolith into several services, so that
coordinate among themselves. What are the characteristics
of a monolith? Well, a monolith usually is a very
complex system with very complex code,
very hard to understand and usually very hard to modify.
In reality, the monoliths tend to evolve
into a big ball of mud, where there is a very
strong coupling among several parts of
the system. So it is very hard to modify one part of
the system without impacts in other starts of the system,
even when these problems do not occur.
Nevertheless, a new camera usually takes a long
time to understand the monolith, can take years to
really understand the system. The advantage of
the monolith resides mainly in the easiness
to communicate inside the monolith. A call to a function
usually is very straightforward, so you don't need to
worry about latency or the security of the
communication among functions of the same project,
or to discover where is the function.
So very easy to communicate inside
the monolith. When you go into the services world,
you go into a view where
each service is simple. It just does one thing
and does that job well, so a newcomer
will in a couple of days understand the service.
It's also easily to disposable, so you can write
a new version of your service and benchmark against
the old version, and see if this new technology or new language
or some other aspect is worthwhile to keep, or you
just should maintain the old version. The complexity emerges
at the coordination level, because you start to need to worry about
contracts for the communication among services. You need to
worry about discoverabilities. Security of the
communication and the deployment of this system
composed of services is more complex than the deployment of
the monolith. There is also another very
big difference between the monolith and the services.
A monolith responds to a variations in
the workload with scaling up and down.
So when you want to increase the workload, you install the
system in a bigger machine. And if you no longer
need to reply to such a big workload,
you decrease the machine. So can be very expensive, because if
you need a really big machine, it can have a very big economic impact,
but it may even not be possible to
purchase a bigger machine. And it's also very slow
this reply, because usually
it takes a long time to install the system in
a new machine. For instance, when you go to the services world,
you go to a very different approach to the
scalability of your system. So usually each
service scales out when there is an increase
in the demand. So it instantiates more
units of the same service. So if you have
a larger workload, you have more instance of the service.
When the workload decreases, you scale in, so you
kill instances of the service. It's much more economically,
and as you can see, you don't need to scale all the services,
just go the services that require the scaling
out or in to have a better response to the variations
of the workload. So what we need to do have
to be able to answer to the problems of this increasing
complexity of the services world. We have a
wish list. Each service should be loosely coupled.
It should be possible to develop,
maintain and test and deploy independently
each service. What we observe is that the
amount of code that you need to develop for
your application increases much more than the functional requirements
you start to need to develop code for automation
tasks in terms of CI, CD and testing
and so on. That I will speak a little bit more later in this
presentation. In terms of tooling you need,
git manages all the files that you need
in your project. Usually I take the approach of
having a git repo for each service, but some
people take another approach. I think this is the one
that has been working the best for all my teams.
The automation tool. I'm thinking about something like
Jenkins, where you automate development tasks
like building your artifacts,
testing or deploying. Artifactory is
a repository where you store versions of your
artifacts. There is another very important tool that
is the infrastructure as code. I'm thinking about
tools like terraform, where you describe the
infrastructure that you need for your application. Everything from
virtual machine to databases and tables
or vpns. Everything that you need should be described
as code. How does it work? You have continuous
integration to make new versions of
your artifacts. Each time there is a merge
of a branch to main in git.
So every time you do a pull request and pull request is approved and then
there is a merge automatically. A new version of
the artifact should be published in the artifactory.
And when you want to deploy an artifact in an
environment, you just run a deploy script where you say I want this
version in this environment and it automatically
is deployed on the target environment without
human intervention. And so how do we tackle the
complexity of everything that I've been talking about
as we have seen, we have to deal with much more
than the fractional requirements because we need to build continuous
integration layer, we need to describe everything as
code, we need to build a continuous deployment layer,
and we need to automation all kind of tests,
much more than t unit tests. You need integration test,
acceptance test, performance test, security test,
everything that you really need to be able to put your
application in production. And I want to
stress the fact that there is no space for human intervention
in terms of tasks. Humans can only be present
as gatekeepers. For instance, you may require
human authorization to deploy your
application in production, but it's just
a gatekeeper, it does not perform a real action.
And in the end we have very good news, because the
anti pattern of hands on my machine just disappears.
You no longer run anything in your machine, your machine just needs
an IDE, you don't need to run anything in your machine,
don't need to do tests in your machine, nothing. You run pipelines.
If there is a problem in the pipeline, there is a report that
you need with the problems and you need to solve those problems.
If the pipelines go green in the end of a
process, you are sure that it is ready to production.
And so automation is not an afterthought. As we have
learned from test driven development. You need
to do the tests before you write the code. So is
it an iterative process that starts with writing tests, writing the
code that solves the test and refactor. And this is a cycle,
and you go forever in this cycle. During the time that you
are improving your application, the same approach should be used
for the automation. So you need to
first have the infrastructure scode, you need to
have the automation scripts that you need for your
application at stage that you are. For instance, if you are really
in the beginning, maybe you just need automation script and
you start developing. And as your application evolves
over time, you need to add the scripts for testing
or for deployment. And same with the infrastructure as
code, you may modify it, put more infrastructure
in your project, et cetera. But it's always an iterative
process. And this approach has two big
advantages. In one end, if you build
incrementally, you are sure that you don't leave anything behind and
everything is querient. The same way that when you do TDD,
you are sure that you achieve the level of coverage that you want
and the code is testable. And it's easier
this way because if you try to do the automation
layer in the end of the project, the effort required is
much bigger than with an incremental process.
There is a second very important reason to
do this approach is that you test
the automation layer in prod so you
gain muscle in your automation layer. You are
confident when you reach production environment
that everything is working fine.
In terms of automation, your system is resilient,
is more secure, is more error prone.
And this is the reason why I say that automation is
where your project starts. Thank you.
It was a pleasure.