Conf42 DevSecOps 2024 - Online

- premiere 5PM GMT

Internal Developer Platform and Kusion: Optimize the software development process

Video size:

Abstract

In the fast-paced world of IT, integrating new technologies with cloud computing has improved software development but also increased complexity. What is the fastest way to build an Internal Development Platform and integrate it with all cloud resources to optimize the software development process?

Summary

Transcript

This transcript was autogenerated. To make changes, submit a PR.
All right, let's get started. And first of all, hello everyone. Welcome to my talk today. And this talk was brought by the Qtion stack with the open source project that I'm working on. And today I will go into the simple talk. I will address the crucial issue we encounter in software development. Understanding the challenge is essential for creating an effective solution. And driving processes in our bills. And we were going to talk about the kitchen. and how to integrate the kitchen with your developer platform. to optimize the software development project. the first quick, introduction. Allow me. Presenter. I am. I work as cloud engineer at the. are responsible for working on platform services like Kuber, engine Cloud, cloud Observability, and also, designing and building the IDP that serve a large number of developer, on the daily basis. And I'm also working on the Open source project, as is, a part of the CNCF sandbox and platform swimming landscape. And this is also the tool we're going to talk today. about the agenda today. And I will cover the background before jumping in the Qsend and IDP itself. Mostly so that we have enough context for the solution to make sense. And we will talk about the challenge thing. We're placing, specifically. the unique one that's contributed to the birth of this project. And I will explain some of the basic concepts of the Qtion, with the platform of Hadoop tool in the Qtion stack. And then we will go into the powerful capability of the Qtion, and demonstrate how integrating it in our developer platform that can enhance the Qtion. our project and driving the chat. the term DevOps, was officially introduced in 2009, The foundation of DevOps was envisioned as a cultural shift, an enhancing collaboration between the development and the operation team. However, at the concept that involved some, Organizations have veered away from it of an original intent, rather than fostering teamwork. There has been a tendency for some companies to place an excessive burden on developers that reallocate various operational responsibilities to them, which detracts from the foundational while the DevOps are fantastic in theory, but put it in the practice can be his or miss. Along with the ship to the DevOp, a few chains are making a bit charter for the developers. we are dealing with a huge variety of tune popping up everywhere. more people adopting cloud services. And some pretty complex tools become more popular like Kubernetes, Terraform, Istio, CloudFrame, or Qtion, maybe? After when developers suddenly have to, What they had around complicate, complicated tool, cloud native tool, and just do a single stop like chaining an environment variable or fixing basic database setup. this has, this has led, crazy amount of mental train. Totally messing up with the developer experience and dropping the productivity to the floor. Just take a look at how bad things have gotten over the past 20 years and honestly, it has been any size that's going to slow down or change anytime soon. We use the Chinese platform as a service set up. Almost 10 years, but we found that it's start to hold us back due to some major issue. The main problem is about, infrastructure has gotten super complex all time. People are swamped with all this infrastructure knowledge and they have to, they need to have and the platform itself has become bottleneck. When it comes to making infrastructure capability available, and they often have many teams involved, which makes the teamwork and coordination pretty tough. On the top of that, on the top of that, configuring our security parameters and the privacy is harder than ever. Thanks a lot. the platform engineering here is to make it easier for developers. Many top tech, top tech companies have set up platform teams, dedicated to build the making internal developer platform, the team approach that works like a product focusing on, simplifying and the developer journey. With, with this platform, the developer no longer have to, juggle complexing toolchain. They can focus on what, focus on what they love most. Coding, creating amazing application with a friendly approach, reduce stress and boost productivity. Allow team to work quickly and efficiently without compromise. On quality, so we are the platform team defy that we refer as the voting path for the developer. This look really do look, complex, but, I will take a check in the main point so you can understand it easily. we believe that the application team on the top left of your screen. should focus on the aspect that are not stick to the environment. Typically this is intent about that, the application team want to define what the application will have, like the workload of the application. the ascensor, the other ascensor Infrastructure like the database that application use without worrying about the specific database specification for given environment. on the bottom left of the figure, we involve that the platform team should be responsible for the certain task. They need to handle a standard workload profile. We're choosing the community deployment on the customer workload along with define the behavior line. In this point, Percy, father, more platform team, also set a standard. For the variable dependency line, that way, networking behavior. Which, that needs to be consistent. this is some, this is system time require different complication to speak to the each environment. For example, having the lack of larger database in the production and smaller one in the development. And we believe application developer should need to handle and worry about the, the whole approach. Consider two component, one being, standard capability, which we call the, QC model. and the other being the environment, specific configuration should, such as, writing database by different in different environment. The fusion platform orchestrator, which is the term we currently use and, take into take, in taking two input one from the, platform. Team one from the application team and, we refer and is efficiently merge. the two aspect allow them with the approach deployment contact, we refer as the workspace. you can think, workspace as a general concept of the, application environment. And, this come together, we call, deep facing and specification, which we hope that makes straightforward and agnostic to both environment and platform. The platform engineer will define element, elements such as model along with the environment specific configuration. And this will, match together to produce what we refer to as spec, resources facing data format. This spec will stop a uniform format, uniform format, AP cable, to both kuber the resources and cloud provider resources in infras. Ultimately, this spec acts as a single source of truth corresponding to one, one to one the environment's actual infrastructure. So we will have, going through the app configuration, in the contact of Cuon app configuration, give developer will play a crucial role to define the application component. They just focus on the spec specific? Yeah. Key elements such as the ER configuration, the monitoring system. The monitoring component and the database, cloud based database, like using a cloud and the version of database, for example, in here, and they don't need to worry about, without needing to address the infrastructure concept or the deployment. Of the required services and application itself. this stream, of course allows them to enhance their productivity, and especially in the open process and about the Cuon workspace. you can understand the workflow as the, environment, like production environment, and. Staging environment, development environment, the workspace is a concept that corresponds to a specific target environment used to deploy applications in a contemporary use. This typical refers to the Kubernetes cluster that hosts the application workload along with an optional cloud. provider, and, to provisioning necessary infrastructure resource such as database. And, in additional serving as the deployment works by, to link with, set up, platform configuration that apply to all application deployed within that, workspace. Thank you. They can, define, platform, team can define any resources on the, environment, the secret store from any provider, cloud provider, and the code that application will be deployed on. So we will go in deeper in the Qt model. This model, this job really, this, will, This is our view. We'll provide a deeper understanding of the functional objection model. We, the execution model effectively manage board C with the resources and from the, from the app com app developer and the platform publication, and translating them into the ation that show as the middle layer data format. And calculating both, resource type and the platform team. task with the managing the configuration, refer at the workspace, configuration with team, the contact execution, and typically the configuration set of the default value. For example, here, they will follow the. the platform team will define the model in the workspace that, that, the application can be used, like the database we will use, ware database with the 20 nearby upside and the instant class of the TBT three micro. And another profile is the small class with the size of 50. And the instant class is the dbt3 small. And the large class is using the dbt2 large. for more specific sector of the application required customer testing, it is possible to define a variety in the configuration. And, however, this spec will be for more advanced to use K. So it is not cover in this, top, you can learn more in the future website provided from the perspective of the application. Developer, the process begin to select the model and the subsequent created. created, creating, configuration file. This file will serve, as a declare model that will totally describe application configuration that I described in the, previous slide. Furthermore, detail is going first. We will design in the next slide. Let go into the next slide. Okay. This is the workflow. This look a little bit complex, but we will just focus on, three part of it below here is the write, generate, and application. So with the write process, first of all, platform orchestrator, platform team will define the model of the QC model, like database model. And monitoring model, storage model, anything, and it will, after defining the model, it will, the platform team will define the workspace configuration, like the environment of the application, like dev environment and product environment, that each environment, the platform engineer will provide the resources. That's what Spike can use and, Kubernetes cluster of that environment and cloud provider of that environment. And about the application developer, they will just care about the app configuration. conflict about the app workload and the database that, the database I, that application use the monitoring, that application, use the story, that application use and enter this process. The configuration are merged into what is a referral. To the, as the intent workshop at a single shot of truth, a single shot of truth, for all resources and scheduled for the point in your environment. One intent is a tab listed. The next phase is applying that during which, specific resource, at a creative or updated. Importantly, before reaching the upline step, there is a crucial step called preview. This step allows a user to review and leave different of the resources that's about to change, ensure that expectations are set for upcoming modification. Just a simple, workflow of the Qtion. And now we're going to provide some, methods to integrate the Qtion with, your IDP. And Qtion will work as the platform orchestrator in your IDP. Then the application, application developer just care about the developer control plane here. About the git repository have the app configuration file. And then the Qtion will add the CI pipeline, like building images, testing, and verifying some spot of the application workflow. the next step is the CD pipeline. It either is we'll get the application configuration, app configuration from the SoCo and the Cuon model, that platform engineering device before is we're going to, deploy the application and, Provisioning, require infrastructure for the application. This workflow will have continual delivery, everything that, application developer. Don't need to worry about the below layer, like provisioning database. how to deploy the application in each environment, the application global. Just define everything in the app configuration, and then the workflow of the pipeline. We will do it for them. The platform team will, define the Qtion model, define the, workflow of the CI pipeline or CD pipeline, define everything, set up the Qtion stack, so that, AppDeveloper just puts the code to the version control like GitHub or GitLab, Watch, watch, CI flow and CD workflow. And then after that, the application will be delivered to the something like committee engine or every infrastructure will be for business. If you have any change in the app configuration, the Qtion tech will handle that task and change like they can change the database size, they can remove the database, or add more database, for example. The Qtion currently operates as a command line interface tool that's functional as the event driven interface. Platform orchestrator is, introduced a protocol driven collaboration paradigm that allows platform owner and application developer can to execute their own independently. The Qsync then orchestrator orchestrate their contribution dynamically matching and generating the final output. Infrastructure pacing specification refer at, spec, subsequent apply by tion. However, the existing version tion with the limitation in providing the scale. Both of this, user can user have to encounter challenge related to the credential management resources. Visibility from flexibility in managing permission and the effort, required to integrate with other systems. in the response to the challenge, we are posting a Qtion server, and this will be by the version of the Qtion will operate as the long run main service. Okay. and this will expose the REST service endpoint, thereby enhance its functionality and user experience. And it can easily integrate with the internal developer portal like Backstair. We'll have more about the backstage plugin and you guys can easily integrate it in on it. about integrate on it, like fusion, presently, currently can integrate it in your CI pipeline work as the, common line. And, but in the future, using server will release and you can have your centralized platform orchestrator by every project. You can call to the platform orchestrator to handle the tasks, like deploying application provisioning infrastructure, just, wait for the fusion server. in the future in this will really show. And it's also the last slide of this talk. Thank you for attend my talk today and here is my in some information about me. You can, contact me to learn more about the Q or going through the Houston Q website here, to learn more about the project and. I'm waiting for the release of the Qt 79 server. thank you, again, for attending this talk. And, hope you guys have a nice day with the conference. Goodbye.
...

Hoang Dinh Nguyen

Cloud Engineer @ Viettel Group

Hoang Dinh Nguyen's LinkedIn account Hoang Dinh Nguyen's twitter account



Join the community!

Learn for free, join the best tech learning community for a price of a pumpkin latte.

Annual
Monthly
Newsletter
$ 0 /mo

Event notifications, weekly newsletter

Delayed access to all content

Immediate access to Keynotes & Panels

Community
$ 8.34 /mo

Immediate access to all content

Courses, quizes & certificates

Community chats

Join the community (7 day free trial)