Conf42 Python 2024 - Online

How to improve application and code quality without overengineering

Video size:

Abstract

Developers often tend to overcomplicate the code. Consequently, customers do not always receive what they want. This approach is costly, time-consuming, and lacks support. I will share how you can effortlessly avoid these issues and create high-quality applications that genuinely satisfy the customer.

Summary

  • Overengineering is an approach to software development that uses more complex technologies than are necessary to solve the tasks at hand. The majority of business tasks can be efficiently addressed using straightforward solutions. implementing just a few key changes can help avoid unnecessary work and reduce the cost of development and maintenance.
  • Architectural design involves determining points of change and organizing the code. It is best practice to deliver each increment to production independently. Refactoring is a controlled process of improving code without adding new functionality. Stick to the chosen architectural approach from the beginning and stay focused on the goal.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
I will be talking about improving application and code quality be without overengineering thank you for joining in. What is overengineering? Overengineering is an approach to software development that uses more complex and advanced technologies than are necessary to solve the tasks at hand. Why does this phenomenon even exist? Firstly, many developers in principle tend to complicate code. It is often connected with the desire to create something impressive. Secondly, managers often bring unclear tasks. They may either fail to describe the scope of problem correctly. Thirdly, the information space contains a myriad of solutions, architectures and algorithms that capture the interest of developers. However, the reality often dictates that the majority of business tasks can be efficiently addressed using straightforward solutions. With popular frameworks and packages, these solutions may not seem intriguing and may not enhance developers skills. As a result, we get a complex solution that does not always solve the task at hand. It is very expensive for the company in terms of both implementation and support. As a metaphor, we could make a spacecraft to travel between two points in the same city. Cool, expensive, but useless. As a specific and straightforward example, let's consider the task of creating an application that retrieves statistical data upon frontend API requests. The data sources could be other applications that provide the required information through their APIs. Context is important. In reality, such an application does not require high load. There is no requirement for response time. There will be several dozen requests per day. The simplest implementation for such a task could be an application using fast API or any other framework. This application in real time and asynchronously makes API requests to the specified sources. It processes the received results and returns JSOn in response to the request. If needed, this implementation can scale through a gateway by increasing the number of workers. Such a solution would adequately meet their clients requirements, but the context in these task may be imprecisely described and developers may have neglected to clarify these details. There will always be someone who issues that. The API for statistics should provide responses very quickly and the API should withstand a higher level of traffic. The process of architecture invention begins. Suggestions emerge to separate the worker responsible for retrieving data from sources and the API worker. This evolves into a system that through buses, fetches data from sources, stores them in a local repository and then delivers results from there. This approach also involves modifying the data sources. Later on, it becomes apparent that not all source teams are willing to modify these service and adopt the bus mechanism. As a result, this architecture is presented to these client. What was initially estimated to take a few days for development turns into a project spanning several weeks. What comes next, during development, the team realizes that they are falling behind schedule and begins to cut corners with the most critical being these reduction of code and importantly, tests are postponed as technical debt along with many other aspects. As a result, the code is written and it somehow functions. Overall, almost everyone is satisfied except for the client. It turned out to be very expensive. Later, the moment arrives when modifications are needed for these applications. Supporting this in the application requires making changes to several nodes of this complex system. In a simpler scenario, supporting such changes could have been significantly easier and faster. What did the company end up with? An overengineering solution that exceeded the actual requirements, increased development costs and time, compromised code quality due to rusty development and postponed testing, difficulty in accommodating changes and modifications in the future leading to higher maintenance costs, dissatisfaction from the client due to the high costs and potential complications in system modifications. In essence, the company obtained a solution that, despite its complexity, did not align with the practical needs, resulting in higher expenses and challenges in maintaining and evolving these system. And this is just a simple example. How can we address the challenge of overengineering? My experience indicates that implementing just a few key changes can help avoid unnecessary work and reduce the cost of development and maintenance. The first key aspect is having a developers in the team who can accurately assess the task and the architecture. The second change is organizational. Implementing a task refinement process is crucial. Furthermore, a detailed examination of the architecture is necessary if the architecture becomes intricate and includes elements that may be useful in the future. Consider dividing the implementation into iterations with room for expansion. The fourth change is the adoption of an iterative development process. The fifth change involves refactoring and optimization. The most crucial aspect is that of business scenarios should be covered by tests during development. This ensures that working interfaces of the application remain intact during refactoring and optimization, preventing unintended disruptions let's delve into more detail. At the beginning, it is necessary to thoroughly analyze the task, considering all business requirements and clarify all details. Architectural design involves determining points of change and organizing the code. Even for small changes that can be implemented quickly and easily. It is advisable to consider the impact of these changes and integrate them into the current architecture without complicating or damaging these code. It is also important to consider load capacity, processing time, correctness, validation, error, handling, the possibility of using the change or the entire implementation as a whole. Implementing the assigned task can lead to a significant number of changes. If the task can be broken down into steps increments, it is better to do so. It is best practice to deliver each increment to production independently if the product allows for it. For example, let's say we need to change a sorting field which requires building an index. It would be logical to first create the index and deploy it to production. Then, in the next stage of the code, switch the sorting to this field. These are already two independent increments. As a third step, if appropriate, the old index can be removed. It is possible to release all of these together. In more complex cases, release, assembly and merge conflicts can become bottlenecks. The new functionality can be hidden until the final release, but it can be delivered to production in parts. The smaller the increment, the easier it is to work with. It is easier to conduct code reviews. There are fewer conflicts during merging and there is a lower probability of making a mistake. It's better to avoid such situations. The code should only be implemented according to the increment plan. Stick to the chosen architectural approach from the beginning and stay focused on the goal. The goal, of course, should be clear to everyone. Additionally, it's crucial at this stage to write tests that code the increment. These tests will not only be useful for checking current errors, but also for future refactoring. The main thing is not to overcomplicate it. It's important that the business function works correctly. After creating the increment, it is important to deliver these code to the consumer and gather feedback. Based on the feedback received, it's important to update the task. Even if these consumer didn't receive the entire feature they requested at this stage, but only a part of it, they may still be able to use it and adjust the task specification accordingly. Sometimes they may even cancel further implementation. If the planned architectural solution remains relevant, we simply prepare for the next step in implementing the code accordingly to the plan or proceed with refactoring. Please note that it's essential to interact with the client extensively. Refactoring is a controlled process of improving code without adding new functionality. The result of refactoring is clean code and a simple design. Refactoring is an expensive process for business. Programmers receive salaries, but these don't add new features. However, attempting to implement perfect code from these start can be significantly more costly. Refactoring is done iteratively according to the plan or as needed. The timing of refactoring may depend on these set of changes. Refactoring should not involve rewriting these service from scratch. This is reengineering for refactoring. The first step is to identify the dirty spots. These may include code duplication, long methods, large classes, long parameter lists, redundant temporary variables, data classes, ungrouped data, debugging information and much more then requirements are set for what needs to be done. One of the important criteria is the maintainability of the code and simplification. Do not confuse refactoring with performance optimization and issue resolution. Refactoring does not include optimization of queries to these database or other storage, increasing speed, reducing memory consumption. Performance optimization is also an important stage in creating a quality application. However, it often overcomplicate code comprehension and creates a new need for refactoring. This way the team only developers the necessary functionality that can be maintained. Overengineering is minimized. In such solutions. It is essential to understand that all decisions should be made collectively by the entire team. Everyone should contribute to writing code and knowledge should be shared among all team members. Also, don't forget to document the code. When responsibility is shared among everyone, the quality will be good. If one person bears their responsibility, the solution will be very subjective and often suboptimal. I would like to emphasize separately that it is important to provide the client with a comprehensive plan for refactoring and optimization from the outset. They should understand before developers begins that the development process does not end when the service is ready. Most clients understand and appreciate these if it is explained to them before implementation begins. To write code without overengineering, it is sufficient to plan the work before development begins. If all team members understand the ultimate goal and what needs to be done, there are usually no problems. However, if decisions are made on the fly during development, architecture is not well thought out and the task is not fully understood. The product will be expensive and of very poor quality. Thank you for your attention. Write your code thoughtfully.
...

Alex Sharypov

Python Software Developer @ Playrix

Alex Sharypov's LinkedIn account



Join the community!

Learn for free, join the best tech learning community for a price of a pumpkin latte.

Annual
Monthly
Newsletter
$ 0 /mo

Event notifications, weekly newsletter

Delayed access to all content

Immediate access to Keynotes & Panels

Community
$ 8.34 /mo

Immediate access to all content

Courses, quizes & certificates

Community chats

Join the community (7 day free trial)