Transcript
This transcript was autogenerated. To make changes, submit a PR.
I will be talking about improving application and code
quality be without overengineering thank
you for joining in. What is overengineering?
Overengineering is an approach to software development
that uses more complex and advanced technologies than
are necessary to solve the tasks at hand.
Why does this phenomenon even exist?
Firstly, many developers in principle tend to complicate
code. It is often connected with the desire to
create something impressive.
Secondly, managers often bring unclear tasks.
They may either fail to describe the scope of
problem correctly. Thirdly, the information
space contains a myriad of solutions,
architectures and algorithms that capture the
interest of developers. However, the reality often dictates
that the majority of business tasks can be efficiently
addressed using straightforward solutions. With popular frameworks
and packages, these solutions may not seem
intriguing and may not enhance developers skills.
As a result, we get a complex solution that
does not always solve the task at hand.
It is very expensive for the company in terms of
both implementation and support.
As a metaphor, we could make a spacecraft to
travel between two points in the same city.
Cool, expensive, but useless.
As a specific and straightforward example,
let's consider the task of creating an
application that retrieves statistical data
upon frontend API requests.
The data sources could be other applications that
provide the required information through their
APIs. Context is important.
In reality, such an application does not require high
load. There is no requirement for response
time. There will be several dozen requests per
day. The simplest implementation for such a task
could be an application using fast API or
any other framework. This application in real
time and asynchronously makes API
requests to the specified sources.
It processes the received results
and returns JSOn in response
to the request. If needed, this implementation
can scale through a gateway by increasing the number of
workers. Such a solution would adequately
meet their clients requirements, but the context
in these task may be imprecisely described and developers
may have neglected to clarify these details. There will
always be someone who issues that. The API
for statistics should provide responses very quickly and
the API should withstand a higher level of traffic.
The process of architecture invention begins.
Suggestions emerge to separate the worker
responsible for retrieving data from sources and
the API worker. This evolves into
a system that through buses, fetches data from sources,
stores them in a local repository and then
delivers results from there.
This approach also involves modifying the data
sources. Later on, it becomes apparent
that not all source teams are willing to modify
these service and adopt the bus mechanism.
As a result, this architecture is presented
to these client. What was initially estimated to
take a few days for development turns into a project spanning
several weeks. What comes next,
during development, the team realizes that they
are falling behind schedule and begins
to cut corners with the most critical
being these reduction of code and importantly,
tests are postponed as technical
debt along with many other aspects.
As a result, the code is written and it somehow
functions. Overall, almost everyone is
satisfied except for the client.
It turned out to be very expensive.
Later, the moment arrives when modifications are needed
for these applications. Supporting this in
the application requires making changes to
several nodes of this complex system.
In a simpler scenario, supporting such changes could have been
significantly easier and faster.
What did the company end up with?
An overengineering solution that exceeded the
actual requirements, increased development
costs and time, compromised code quality due
to rusty development and postponed testing,
difficulty in accommodating changes and modifications
in the future leading to higher maintenance costs,
dissatisfaction from the client due to the high costs and
potential complications in system modifications.
In essence, the company obtained a solution that,
despite its complexity, did not align with the practical needs,
resulting in higher expenses and challenges in maintaining
and evolving these system. And this
is just a simple example.
How can we address the challenge of overengineering?
My experience indicates that implementing just
a few key changes can help avoid
unnecessary work and reduce the cost of development and maintenance.
The first key aspect is having a developers in the
team who can accurately assess the task and
the architecture. The second change is organizational.
Implementing a task refinement process is crucial.
Furthermore, a detailed examination of the architecture
is necessary if the architecture becomes intricate
and includes elements that may be useful
in the future. Consider dividing the implementation into iterations
with room for expansion. The fourth change
is the adoption of an iterative development process.
The fifth change involves refactoring and optimization.
The most crucial aspect is that of business scenarios
should be covered by tests during development.
This ensures that working interfaces of the application
remain intact during refactoring and optimization,
preventing unintended disruptions let's
delve into more detail. At the beginning,
it is necessary to thoroughly analyze the task,
considering all business requirements and clarify
all details. Architectural design involves
determining points of change and organizing the
code. Even for small changes that
can be implemented quickly and easily.
It is advisable to consider the impact of
these changes and integrate them into the current
architecture without complicating or damaging
these code. It is also important to consider
load capacity, processing time,
correctness, validation, error, handling,
the possibility of using the change or the
entire implementation as a whole.
Implementing the assigned task can lead to
a significant number of changes. If the task
can be broken down into steps increments,
it is better to do so. It is
best practice to deliver each increment to production independently
if the product allows for it.
For example, let's say we need to change a sorting
field which requires building an index.
It would be logical to first create the index
and deploy it to production.
Then, in the next stage of the code,
switch the sorting to this field.
These are already two independent increments.
As a third step, if appropriate,
the old index can be removed.
It is possible to release all of
these together. In more complex cases,
release, assembly and merge conflicts can become bottlenecks.
The new functionality can be hidden until the final
release, but it can be delivered to production in parts.
The smaller the increment, the easier it is to
work with. It is easier to conduct code reviews.
There are fewer conflicts during merging
and there is a lower probability of making a
mistake. It's better to avoid such situations.
The code should only be implemented according to
the increment plan. Stick to the chosen architectural
approach from the beginning and stay focused on the
goal. The goal, of course, should be clear to
everyone. Additionally, it's crucial at this stage
to write tests that code the increment.
These tests will not only be useful for checking current errors,
but also for future refactoring.
The main thing is not to overcomplicate
it. It's important that the business function works
correctly. After creating the increment, it is important
to deliver these code to the consumer and
gather feedback. Based on the feedback
received, it's important to update the task.
Even if these consumer didn't receive the entire feature
they requested at this stage, but only a part of it,
they may still be able to use it and
adjust the task specification accordingly.
Sometimes they may even cancel further implementation.
If the planned architectural solution remains relevant,
we simply prepare for the next step in implementing the
code accordingly to the plan or proceed
with refactoring. Please note that it's
essential to interact with the client extensively.
Refactoring is a controlled process of improving code
without adding new functionality.
The result of refactoring is clean code and
a simple design. Refactoring is an expensive
process for business.
Programmers receive salaries, but these don't add new
features. However, attempting to implement
perfect code from these start can be significantly more costly.
Refactoring is done iteratively according to the plan or
as needed. The timing of refactoring may depend
on these set of changes. Refactoring should not
involve rewriting these service from scratch.
This is reengineering for refactoring. The first
step is to identify the dirty spots.
These may include code duplication,
long methods, large classes, long parameter
lists, redundant temporary variables,
data classes, ungrouped data,
debugging information and much more then requirements
are set for what needs to be done.
One of the important criteria is the maintainability of
the code and simplification. Do not confuse refactoring
with performance optimization and issue resolution.
Refactoring does not include optimization of
queries to these database or other storage,
increasing speed, reducing memory
consumption. Performance optimization
is also an important stage in creating a quality application.
However, it often overcomplicate code comprehension
and creates a new need for refactoring.
This way the team only developers the necessary
functionality that can be maintained.
Overengineering is minimized. In such solutions.
It is essential to understand that all decisions
should be made collectively by the entire team.
Everyone should contribute to writing code and
knowledge should be shared among all team members.
Also, don't forget to document the code.
When responsibility is shared among everyone,
the quality will be good. If one person bears
their responsibility, the solution will be very subjective and often
suboptimal. I would like to emphasize separately that
it is important to provide the client with a comprehensive
plan for refactoring and optimization from
the outset. They should understand before developers
begins that the development process does not end
when the service is ready.
Most clients understand and appreciate these
if it is explained to them before
implementation begins. To write code without overengineering,
it is sufficient to plan the work before development
begins. If all team members understand the
ultimate goal and what needs to be done,
there are usually no problems. However, if decisions
are made on the fly during development, architecture is
not well thought out and the task is not fully understood.
The product will be expensive and of
very poor quality. Thank you for your attention.
Write your code thoughtfully.