Hello. Welcome, everybody. My name is Clara, and I want to welcome you all to this new edition off DevOps Conference 2025. In the following 30 minutes, we will review and navigate through different test plans and test strategies for the DevOps framework. So let's start. Welcome back to our session DevOps test management. strategies. Let's start by introducing myself. Who am I? My name is Clara. As I said, I'm from Argentina, but I'm currently living in Spain. I'm a passionate QA manager. I have been in the QA industry for over 14 years, and I have enjoyed each one of those years. I love our field of action. I love our quality assurance work and I'm very passionate about it. There is a say in the QA industry and we say for every development activity, there is a corresponding testing activity and I always want to honor that saying. I'm also a lifelong learner. I love to study, I love to train myself, I love to develop and acquire new skills, and I'm always trying to achieve new goals. I think, I honestly think I can learn from anybody around me. And I'm also a yoga teacher and an assiduous meditator. And I like to combine that with my work activities. I think the skills that I have accomplished during my years meditating are very useful for me during my work. Okay, but enough about me. Why are we here? Today we are going to learn a little bit about testing in a DevOps framework. So we will start in Talking about the generalities of our DevOps framework, then the QA role within it. And then we will deeply navigate into the construction of a proper test plan for DevOps. And we will review a dual testing strategy that encompass that test plan. So I hope you enjoyed this session as much as I did. And let's start. What is DevOps is a combination of cultural philosophies, practices, and tools that increase the organization's ability to deliver applications and services at a high velocity. DevOps is all about fostering collaboration, efficiency, and efficiency. quality in DevOps framework, and we have seven C's seven continuous activities that we need to accomplish and ensure those are continuous operation, continuous development, continuous integration, continuous testing, continuous deployment, or deliver continuous feedback and improvement and continuous development. Monitoring. This is, in a few words, in a nutshell, the essence and the life of the DevOps framework is not only a work framework, it's also a cultural philosophy, a mindset, and a way of we in the team to work and collaborate together. Together, evolving and improvement products at a faster pace so we are able to deliver to our clients in a velocity way within this DevOps framework. Which one is the QA role? So we have been spoken about continuous testing. So what is the role of a QA in a continuous testing framework? DevOp DevOps, it's all about fostering collaboration, efficiency, and quality. In this continuous delivery model, QA plays a crucial role by ensuring the end to end testing integrates seamlessly. Into the DevOps pipeline in DevOps, we need to construct, maintain, sustain a set off unit test cases to automate and to execute within each deliver. So continuous testing automates and validates code changes continuously, enabling rapid feedbacks and risks reduction. Enhancing, enhancing product reliability and accelerating the release cycle with a robust product ensuring quality. So to start talking about a test plan, we need to know what is a test plan and which are the parts that compose a good test plan. There are a lot of things to have in mind when you are working and constructing a test plan. I'm sure everybody here is aware of those parts. So we will quickly review the life and the foundations of a test plan to go deeply and navigate into the test plan strategies and the test plan construction. So the parts are the objective is the first thing you need to establish what is what are the objectives for this testing endeavor. Then the scope test methodologies. Approach, the assumptions you need to do, the risk assessment, the risk identification, then a mitigation plan or contingency plan. You need to establish within your test plan every role and responsibility. You need to do the proper schedule to have a proper timeframe established for your test plan. Then, we need to establish a proper defect. Tracking process tools and people involved. We need to establish the test environments that this testing this test plan is going to be conducted on our entry and exit criteria. Those must be written within the test plan. I know DevOps and agile. Are not that deeply into the documentation. We are trying to deliver faster. So documentation is relegated to a second plan to a second part, but it's not that we don't have any is minimal documentation approach. But we need to have the necessary documentation in the test plan is one of those. So you need to write into your test plan, your entry and exit criteria, then the test automation part. They want that we execute continuously and also the ones that will be added to that continuous test suite within this test project. Then our effort and estimation how long. It's going to take us and what we need, we need to do a resource allocation here for environments, tools and people. And then the test deliverables and temp. So these are the 16 steps, tasks or modules that forms. A test plan. So we will, explore the test plan itself and the test strategies. Let's just start regarding our test plan. I want to introduce you guys and girls in this opportunity to the different levels that a test plan can have in my organization, in my team. I like to create a test plan for organizational level. Project level and software development life cycle level regarding the organizational level. I always start by analyzing the organization itself, its generalities, the industry our organization is in, the legal compliance that we need to have in mind, the tools that are used organizational wise. So you need to have those in mind regarding the project level. That's regarding. That's reflecting the current project, so includes the test script, the test suites, the people itself, and about the software development life cycle level, we need to make sure we accomplish the delivery fast piece progress. And cadence. So by creating these three levels of test plans, Yay Manager, as myself, ensures that the testing activities are consistent with organizational goals, flexible to the individual project requirements, and adaptive to the specific development practices like DevOps, leading to a robust and efficient quality assurance process. We will review each one of these test plans levels. Let's start by the organizational level. When you have a test plan at organizational level, the purpose for this test plan is to establish the overreaching goals and standards for quality assurance across the entire organization. If the test plan, the organizational test plan, has never been created, This is a great opportunity for you to start this practice as this test plan will serve as foundation and baseline for every other project test plan. So in this organizational test plan, we need to point out the tools for testing that are used at organizational level. the organizational industry requirements that we need to comply and the organizational goals for quality that our complete company wants to achieve. It ensures consistency in testing practices and align the testing strategy with the organization mission and the quality objectives. What will you add into this test plan content? It includes a guideline for testing methodologies that we can use in our organization that are recommended and encourage the tools, as we said, that we can use for each project. Depending on the project, you will use, utilize more or less of those tools. If, for example, your project Is regarding the non functional aspects, you will need to use the tools for performance testing. For example, Jmeter, if your project includes user acceptance testing part only, then you will focus on manual testing, functional testing, and we need also to address in this test plan, in the organizational test plan, the resources Allocation, including the environments, servers, capability, the third party vendors, if any, and the resources within our organization, it serves as a framework for all the project specifics test that we will review now. For a project level test plan, the purpose of this, this is the most common test plan that everybody designs and develops. When you design your test plan is commonly mostly at project level because address the needs of the current project. So it will include all of the actions and steps that we review before. But our project level, it focused on the specific needs and goals for this particular project, and it includes the time map, the scope, the objectives, technologies, and all the aspects related to our project. The content will be the testing approach, the entry and exit criteria, the resources, The timeline, the estimation, of course, the specific test cases and test suite or use cases are the life of this test plan project level. Here we will include the risk analysis and the roles and responsibilities for the project. This is more detailed, is the longest of our test plans and contextualize that the organizational test plan and use the organizational standards. As a foundation, so the project level test plan is unique per project, but the organizational level test plan is the same for every push now to review the software development life cycle test plan. This will also be reutilized and can be shared within all the projects in the company if we are all user using the same software development life cycle is a guideline baseline to utilize when we are executing our project test plan. The purpose of this project, the software development life cycle test plan. It's to address the specifics of the development life cycle being used, such as Agile, Scrum, or in this case, DevOps. It ensures that testing is effectively integrated with the development process and aligned with its iterative and collaborative. Nature for this software development life cycle test plan. We will involve our project managers are developers to validate that we are planning the corresponding test activities for every each development each deploy. The content of this test plan, it's specified the testing activities tied to each phase of the software development life cycle, such as in continuous integration and continuous deployment, the automation strategies. The acceptance criteria for each iteration, the performance testing and non functional testing and monitoring practices that we are going to put in place and this helps ensures the testing is an ongoing and adaptive to changes during the development phase. process. So again, this can be reutilized in different projects is not specific to the project, but it is specific to the software development life cycle that we are using at company level. That's regarding our test plan development. We have learned how to develop a test plan in organizational level. At project level, which is the most common one and at software development life cycle level. Now that we have our test plan developed, analyzed and tailored to our needs and our project, we can decide our testing strategy. To construct a testing strategy. In this case, I, myself, will use three different testing strategies coexisting for a DevOps project. First of all, it will be our continuous testing. So this will include our automatic test run that executes every time a code is committed, allowing quick feedback and rapid interactions as DevOps baseline. But that will be general to any DevOps project, the life of a DevOps project. It's the continuous testing and it's the robust set of unit testing that is executed and run every time a code is committed. But for each project, we will need to have our own testing strategy. In this case, I would like to address a shift left testing strategy as a first, practice as a number one testing strategy within, combined within a risk based testing strategy as a secondary testing strategy. Let's review. Shift left testing as it is on the name. It stands for going to the shift, shifting, shifting to the left as much as you can. You need, you need to start your QA or testing activities as early as possible within your software, the development life cycle. It's a day identify and fix defects early in the development process by starting test activities as soon as possible. This is. Very, very important. The cheapest way to fix a defect is to find this defect within the same phase of the software development lifecycle that was introduced. That's the cheapest way to fix it. And if we can mention some industry standards, it's 15 times cheaper to fix a defect within the development phase that in production. So that number is a lot. If we reach our pre production status or stage with a lot of defects, we may consume our budget. And there are a lot of projects that never reach production because of this. because of budget running out. So shift left strategy involves QAs as early as possible. And what is the earliest possible? Static testing. So we will try to start our testing with a static testing, Practices, the static testing of the code, we can use tools for the static analysis as SonarQube or any other tools that your organization is maybe using, or you can introduce to the organization and the manual static, testing, which is the reviews, the reviews of the work products, reviews of the user stories you can conduct. And informal reviews, or if you can go more formal and conduct formal reviews or retrospectives. For informal reviews, and the most common practice that I will recommend is the peer review. So you can go doing peer review and review the requirements, the test scripts, the code, everything can be reviewed at its early stage to try to identify and fix the defects as soon and as cheap as possible. So engage testers early in the sprint or iteration, incorporate testing into the design and decoding phases. and carry out early non functional testing such as performance and security testing. Our second test strategy will be risk based testing. Risk based testing can coexist with any other Testing strategy because risk-based testing, BA is based on the execution of the test script, the execution of the use cases. So let's review a little bit for a risk-based test strategy, we prioritize the testing activities based on the risk. And the impact of different components and features by focusing on areas that present the higher risk, you can ensure critical functionalities are thoroughly tested, which is essential in DevOps environments where rapid delivery is important. So for a risk based testing strategy, we need to perform. two important actions. One is the risk analysis and the second is risk monitoring. Let's review our first activity for a risk based testing strategy. The risk analysis. The risk analysis includes two parts. The first is risk identification. You need to identify properly which are the risks within your project. How do you do that? You involve in the risk identification the correct Everybody that is a relevant stakeholder needs to be involved in a risk identification assessment. You can do it by performing interviews, retrospectives, risk workshops, which are my favorite, actually, because it's a very dynamic and collaborative way to identify the risks. Or you can use training. checklist. That can also be an offline activity and you can send those to the stakeholders by email and track the answers. But within this activity, you should end up with a proper list of risks, properly identified by every stakeholder. I cannot repeat Enough. How important is to involve the proper stakeholders in the risk identification process? Because if you don't identify the risk properly, if you, oversee a risk, or if you highlight as a risk something that is not, it will result in the complete risk based strategy. being unused. So after you conclude your risk identification process, you should end up with a list of risk. Let's say, let's say risk A, risk B and risk C. Then it's time to do your risk assessment. In this qualification of your risk, You will assign a risk level for each one of those identified risk. What is your risk level? A risk level is the combination between the risk likelihood and the risk probability. impact upon occurrence. So, for example, if risk a is very likely to occur, but the impact upon occurrence is very low, you will identify this risk in a medium level because it's very likely to happen. You want to cover that to mitigate that, but because its impact is very low, you will assign it a medium level. For risk B, for example, if the likelihood of appearance is low, but the impact upon appearance is critical, you will anyway assign a high level to this risk because you want to make sure that This will not happen in production as soon as possible and for risk C, if they have a likelihood of appearance, it's made, but the impact it's high. You will also assign it as a high risk level feature or this. Then this will be the first part of our risk based testing strategy, the risk analysis. After we have identified a risk and assigned a risk level to each one of those, we will continue to the second part, which is the risk analysis. Within the risk control, we continue perform the risk monitoring. This is a continuous activity. This never stop. Risk monitoring is part of our strategy and our testing process. This allow us to report on testing progress in terms already of residual risk level. At any point, which is very important for the stakeholders to make informed and appropriate decisions regarding going or not going to production, for example. And also the risk control part includes. The risk mitigation. Our essential mitigation action when we execute a risk based testing strategy is to execute our test scripts in a prioritized way. We prioritize According to the risk level, so the test scripts are assigned from highest to lowest to a different risk level regarding the feature of functionality, having in mind the risk assessment and identification we just did during risk analysis part. Regarding this test execution, there are two ways in the industry that we can choose to execute, a risk-based strategy to mitigate risk as soon as possible. The first execution will be a deep first approach that execute all the scripts in a strict descending order. For the level of risk from the highest to the lowest, so no matter the feature or the part of the system that we are executing the test script, we will execute all the high priority regarding the risk first and then go down into until the lowest. So that's one way to go. It's very robust. But it's time consuming and it's very structured. If we need to be more flexible or if we need to make faster decisions, we will choose a breathe first test approach. In this test approach, Each test script will be assigned to with the highest risk priority, we will assign one test script per feature or per product to the highest test priority, and we will execute those highest one per product or one per feature as soon as possible in the first round of execution, and then we will continue. In this way, we will be able to provide information regarding the sanity of our project to the stakeholders in a faster way. So we will be able to assess and evaluate the current quality if we execute breathe first approach one test script with the highest risk per feature. Normally in our projects, we start with a deep first testing approach because it is more robust, more organized, but as soon as we are starting consuming our time and our budget, we move to a brief first approach trying to provide it. Thank you. more information and to assess the quality of our project in a general way as soon as possible. So this is regarding our risk control. Let's review our key takeaways. That has been our analysis regarding the test strategies that we want to choose and a compass for our three level test plans within our DevOps framework project. Why we will choose A mixed test strategy. First, it allows you early detection of the highest important defects. It ensures efficient resource allocation. You will allocate your resources as soon as possible to the highest risk part of your project. It enhance collaboration. You will involve all the stakeholders, developers, project managers, product owners, final users. You will involve all of the stakeholders within your development of your risk strategy, and this will foster a collaboration spirit in your organization. It improves quality and speed because you will isolate the defects faster and you will have a robust test strategy with three different test plan levels and with three different test strategies, the continuous testing strategies from DevOps, a shift left test strategy and a risk based approach sustaining your other strategy. And this will enable informed decision making for your stake holders and your managers. So that's why we should always consider this mixed test strategies as a driven for our projects. So that will be all. And I just want to remember you guys, girls, team, all the, this phrase that I love from RQA environment and RQA world. The product quality is highly influenced by the quality of the process being used and applied. So if you, your process has a high quality, it will influence highly the quality of your product. So remember to invest time developing a tailored process for each of your projects. Thank you very much. And I hope you, we can stay in touch. So please add me to your LinkedIn accounts and let's continue talking about quality. I hope you have enjoyed this time. So thank you. Okay. So that was all for today's session. Thank you very much for staying with me for this 30 minutes. I hope you find this information very useful and let's keep in touch. Thank you.