Hi, my name is Gregoruk and I would like to welcome you on my talk on the Conf 42 Chaos engineering. And I would like to talk to you about the test estimation. Before we jump straight into the topic, I would like also to introduce myself a little. But so I have over twelve years of experience in the test or the quality assurance area. I did manual testing in my life, I did the test automation, I designed the frameworks for test automation, I tested the software on the real hardware, I tested it in the simulated environment, I've designed the processes, I've managed the testing teams, I connected the testing process to the other software delivery lifecycle activities and stuff like this. So if you think about the quality assurance activities, I probably did them in my life, in my career many, many times. And on top of that, I've started to work with the project management activities recently and much bigger teams. So bringing my experience together, I would like to talk to you about the test estimation. And second title to this presentation is how did I become a fortune teller? But before we start, I have to admit that I had a little bit of the dilemma. I was wondering if I should tell to you about the methods of the test estimation, or maybe I should show you how I approach this subject so give you more insights in my approach to the problem of the test estimation. And I think that you can find in Google all the test estimation methods, like how you can estimate the test. What's more important to us, to the QA people is how to approach this problem, actually. So let's start to this topic from a couple of the data which I collected. So, in Poland, the country where I came from, we have 34 million of people. Among those, we have 75,000 of fortune tellers, bionerg, therapists and seers. 56% of citizens of Poland actually uses their service. And the price per single service is in range starting from $1, ending up on even $250 per one single service. And why I'm telling all of this to you? Well, look at that. There is like almost 60% of people believing or trying to believe or wanting to believe in the activities or services provided by the fortune tellers. But when I read through all of the job descriptions on the job markets about the quality assurance engineers, I did not found any mention about the test estimation in the activities performed by the qas. That's pretty much started for me. I started to wonder why it's happening like that. And I started in this business in times when testing was perceived as some sort of negative activity, which has only focused on finding the issues and pretty much interrupt the entire process. And I thought about this like, hey, why project managers, why people who are responsible for delivery chain, they want to know the estimation date or the estimation time of the development activities, but they don't ask the same questions to the QA engineers, why is it like so, and maybe we should look at this problem from the other side of the coin. So why QA engineers are not giving the test estimation? Well, my theory is pretty simple about that. Probably nobody cared about the test estimation in the previous projects, in the previous deliveries, in the previous timeframes. So that's why the QA engineers doesn't or do not provide the test estimation anymore. And I think that we should change that, because the test estimation is one of the core activities. If you think about the testing itself, if you think about delivering the software, it's pretty much as much important has the estimation of the development activities. So let's think about the test estimation. If you think about this, what the estimation itself, what is it? So, as Master Yoda said, it's difficult to see, because always in motion is the future. And if you think about the test estimation, well, it can be hard because we're trying to predict the future. And if you think about estimation, the people who are doing the estimation, even developers itself, they are often referring to the estimation as guesstimation, because estimation is pretty much a guessing if you try to make it very simple. But for me, it's more than guessing, because there are certain rules, there are certain things which you can rely on, which makes the guessing much core, much more reliable activity, I would say. So let's think about this. Why it is worth a while to even estimate the testing effort? Well, I think that there are like four key things that make the test estimation worth a while. First of all, it helps the resource management. So for everybody who are like managing the testing process or the delivery cycle, you can manage the resources. If you know that in project a, you have still a little bit of the capacity of the testing team and you can use it to support the project b, it also can work in the other site. So if you know already that in the project a, the deadline is really tight and you do not have the resources there, you can use the other team to support these testing activities in project a. So that helps us, the estimation itself helps QA manager their resources effectively. Second of all, it gives us safety. We all know that a good business is a happy business. So if you will give the test estimation to the business or to your insight clients, po, PM, whatever, you will give them a safety, because they will know that you will not sit there and they will not scratch their heads asking themselves, our testing team, what they are actually doing and when they will finish. So they will know when they can expect you to finish the testing phase. Third of all, it is professional. So there is a lot of rumors in the QA environment that, hey, if you are estimating the test, probably you cares very professional in what you do, because only the best can do a proper test estimation. Because the test estimation, or the estimation itself, the very accurate estimation is based on the experience. So it also helps to build your own professionalism as a person who knows what they're doing. And the fourth thing, which is supporting the thesis that it is worth a while to estimate the testing effort, is that you will gain a knowledge. Because to provide the better estimation, the more accurate estimation you need to gain a knowledge, you need to understand in a better way what you're doing. What are your core activities, what are the core things which you need to check while doing the testing itself. So those four things altogether will make a test estimation worth a while. So, if you want to think about the test estimation, we should focus on three main factors. What we should estimate, when we should estimate it, and who should estimate it. Let's start from the first thing, what we should estimate. So the horse as it is, everyone can see, but what to estimate. And the thing that I was struggling the most while I was trying to estimate the test was the fact that if somebody saw my estimation of a single testing task, they thought like, okay, so you need to execute this test like 20 times. So if you need a 1 hour to execute this test, 20 times, one, it gives you 20 hours. So 20 hours done, right? Well, not really. Let's think about this. I will give you a simple example. If you will have a room where there is 100 of people, and you will need to take a photo with your phone of a one person. How much time do you need for that? Like 5 seconds. Take the phone out of your pocket, unlock it, select the camera up, point the camera, take the photo. Done. Right? Let's say 10 seconds. Okay, but if you would have to make a photo of every single person in this room. So make 100 of photos, it starts to become more complicated, right? Because even if the steps are the same, the preconditions or something, which we call the environment or the setup is different and the tear down is different, right? So you cannot say that if you would have to take a photo of hundreds of people. It will take you like thousands of seconds, right? It is much more to that. So you need to know or while trying to estimate, or while you will give the estimate to someone, you need to underline that, hey, it's not that simple that I will just do this activity a couple of times and you can just multiply the time needed for a single task. Times. How many times I need to do it? So the first thing, when you do the estimation, you need to think about what you will estimate in the sense that the sum of estimation of single testing task is not a total estimation. Also you need to include the supporting activities. So we know that actually the quality assurance activities can be divided on the quality check and quality control. So quality check is actually the moment when we execute the tests. But quality control is everything what we do around the test execution. So you need to include the time for supporting activities like writing down your test cases, connected them with the test base, with the subject of testing, with the requirements itself. So provide the two way tracking of testing and the things which you will test. Then you need to think about, okay, how much time do I need to report the bugs, to report the testing activities, to report the results, to create a testing report, to communicate this, to discuss this. So all of these activities should be included also in the test estimate. And on top of that, we should also contain or include a buffer in our test estimation. Why to do that, for the reasons which I already said. So there is not only like the testing itself, you need to prepare the environment and stuff like that, but also there are things which you cannot predict. So for example, the environment will be not available, there will be other activities. Maybe you will need to think about something. Maybe you will need to discuss whether the requirement is actually correct. Maybe you will need to get some more knowledge. So everything which cannot be predicted straight ahead, you need to think about this as a buffer, and you need to provide a buffer in your test estimation. Okay, that leads us to the second point when we should do the test estimation, because there's never enough time to do all the nothing you want. So by saying when or asking when we should estimate the testing effort, I'm not asking you about whether you should estimate this before, during or after testing, or maybe at the beginning of the project, or maybe during. But you should think about this question more about what is the place in time when I'm trying to do the test estimation, what is happening around what is like my environment, when I am estimating the test effort. And you need to think about many, many factors here. First of all, obviously the stability of their environments. So whether my testing environments are stable, maybe there's a time of a year, like in the summer when there's super hot and we need to shut down the AC. And as a part of that, our machines in the server room, they are pretty much slowed down, or they are shut down, so they are not operating at the full speed. And that obviously will impact my testing. And also how stable is our application? Maybe we recently had or observed issues with the performance of the application, or maybe it crashes while we have some certain circumstances, like exceeded the number of the users. So that should be also included in the test estimation or taken into consideration. There are other circumstances, like Covid-19 it had a very big impact on all the activities around different companies, in different industries, because when you're performing your own activities, there might be a necessity to jump to the other project and support someone because he went sick and stuff like that. So as you can see, when you're asking yourself when you should estimate the tests, you should think about all the factors which are happening around you, and that's pretty much, obviously will influence the test estimation. Jumping to the third factor we have, who should estimate the tests? Well, obviously, if it is about the testing effort, obviously the answer to this question is the QA, right? The QA engineers. But if you think about this, is it easy for everyone to estimate the testing? We already said that the test estimation is activity which is perceived as something which is done only by the experienced personnel, experienced QA engineers. Because if you have a lot of experience in the project, you can give a more accurate test estimate, right? So obviously you can bring someone who is like just starting in the QA or even the IT industry, and he will also be able to give you the estimation. But it won't be such accurate as someone who already worked in the project, work with our application, work within our environment, in our organization, in our company, and already have some background in testing, and already did the estimation in the past. So obviously the more experienced person, the better. But obviously you can discuss the estimation, you can perform the test estimation even if you are not experienced or experienced tester also, who is doing the test estimation. The people who are engaged, the people who do care about the testing activities, who do care about the quality in our project. So in other words, if you actually do care about what you do, and you perceived the quality not only as your job, like, yeah, I need to do is like starting from eight till 08:00 a.m.. Till 04:00 p.m. But you perceived this as something more, as an important activity, entire delivery chain, then you are probably the right person to estimate the tests. Sorry about that. And also while thinking about who is doing the test estimation, you should also think about the company itself because you should consider the structure in the company, the organizational culture, everything which influenced the testing. Sorry. So you should think about while doing the test estimation, how the company actually approached the problem of test estimation, or maybe approach the problem or the process of quality assurance, because it is very important sometimes, as I already said, and this is very sad to me, maybe somebody or even anybody or even no one is wondering how much time do you need for testing? Or maybe nobody is asking this question when nobody is asking. Then you should be the person who actually start this movement from the bottom of the organization and said, yeah, well, this is important for me to let you know that I need the time to actually perform my tests. And it will also give you a comfort that you will have a test estimation and people will know that you need to spend time over the testing. Okay, so what's next? If we know what to estimate, when to estimate and who should do this, what's next? Well, the biggest sin about the test estimation or about the activities which we actually perform as a QA is the fact that we are not talking about this or we not communicate this. And even if you think about the testing, you will see that if something is happening, the sooner person who is supervising our project, like PM, Po, test manager, whoever, the sooner he knows about the issues, the better. Right? So it is the same with test estimation and the estimate itself. First of all, you need to communicate this. Even if nobody cares, you need to say it up and loud like, hey, I need one week for my testing. I need like full 40 hours. That's the first thing to do with your test estimation. The second thing, it's not only about being proud from your test estimation that, hey, I actually did it, I'm experienced, I did the test estimation. Yes, I have the estimate. It's not about stopping there. You need to monitor it. So even if you think about the ISTQB certification and the foundation level knowledge, you will notice that there's a mention, but this, that it's not only to setting up the plan for testing, it's also to monitor this, whether we are going according to the plan. And obviously, if we are not going according to the plan, just update the test estimation. So if you think about the estimation and actually the safe methodology tells a lot about this. And there's one rule in safe, which directly calls this a cone of uncertainty. So let's think about the cone, which is wide at this entrance and it gets narrow at the exit. So if you think about this as a timeline and at the start of the project, you get very wide end of the cone. So if you estimate there and if you are inside the core, you can make a mistake by a lot. So your estimation have a very big margin. But if you go with time to the narrow end of the core, you will probably get more knowledge. You will probably execute some tests already. You will see how you cares progressing through your test plan and also burning down your test estimate, and you will be able to update the estimate with the bigger knowledge, with the bigger experience, with knowledge about the progress you are making through the testing phase. And probably this estimate will be more accurate and will be way better than the estimate from the very beginning. Is it bad that you gave an estimate which contained a big margin? No, because you did not have a knowledge and experience, which you have right now. So that's why I'm telling this to you, that you need to monitor it and update it. And those three things are equally important. You need to communicate the test estimate, you need to tell about this, you need to monitor, you need to update it, because think about the situation when you monitor the test estimation or the execution of the test plan and updating it. You can tell to your supervisor, like anybody who's monitoring or having an eye on the testing process, you can tell this to him, like, hey, we are not progressing according to the plan. And at this moment, I can already tell you that I will need additional two days to fulfill the testing activities. And if you say it like two weeks before deadline, it is a way better communicate that, a way better message that if you tell this one day before the release, like, hey, I need additional two days, right? That's like way worse situation. Okay, so if you have the test estimation, we know what to do with it, then we will probably meet with one of the most common issue with the test estimation, which you can find or which can be provided by our PMS Po, our test manager, our scrum masters, you name it. Can you do it faster? So they will try to do their job, they will try to manage the process, they will try to manage the deadlines, they will try to provide the best outcome in the shortest possible time. So they will ask you, can you do it faster? Aka, can you cut down your estimate? So how we should approach to this issue, to this problem, how we should protect our estimate from being but down. So as I already said, I did a very deep research of a fortune tellers before I started to prepare this presentation. And as you can already see, this might be a little bit funny that you will see this sentence that your dog is not a cat, but that's what fortune tellers do, actually. And the first advice from my side is, when we are thinking about protecting your estimate from being reduced, is to tell to the people who cares trying to do this something that they already know, like, okay, but think about this. I know that you want me to do this faster, but this is like a new project. We need the time. We need to prepare everything. We need to check the critical functionalities, we need to execute those tests. We have a test plan which you already agreed on. And that's like pretty much it takes time. I cannot make it faster. I need to do it good. I need to do it in a proper way. I cannot make it shorter. I cannot just speed the processing of the data or my inputs and stuff like that. So tell them something that they already know. Refer to the size of the project, refer to the risks of the project, refer to the impact of the project on the existing system, refer to the importance of the clients and stuff like that. There's a second thing which you can do. You can add additional buffer which can be cut down, and then it will lead to the situation when the person who is managing the process will be happy because he was able to cut down the estimate a little bit. But this can be a little bit tricky and pretty much it's not a very elegant way. The third thing which you can do when you try to protect your estimate from being reduced is ask for priorities. So if someone will come to you and say, hey, we need to make it faster, we need to finish testing faster because the deadline is there. You can ask for priorities. You can say, okay, I can make it in a shorter time, but I won't be able to test everything which I planned. So can you tell me what should be like the main focus for me to meet this deadline? And I will try to do my best and I will show you this in the testing report, that, well, based on your decision, I focused on this or that parts of the system and just do very basic checks on the other parts of the system. So that's very good practice to do. So not only accept the decision that the test should last a little bit shorter than original estimate, but also ask for priorities. What should be my main goal right now? If you are telling me so, which parts of my test plan I can just have only in the basics and which parts of my plan I should do in a normal way. And the fourth thing which you can do is use your own authority. So if you previously tested through a lot of deliveries for the project, for the product which you are working on, you can use it as your own authority and say, yeah, I know that this might seems a little bit long, but think about this, I did this many of times, I did a lot of projects for you, I did a lot of deliveries for you, I tested a lot of deliveries and I never was off with my estimate. So I know that it will just take that much time. I cannot make it shorter. It's simple as that. It's just as it is. And I know because I did it many, many times. So you can always use your own authority for that. And I will highly recommend this method because it will also build your own image as a person who knows what he's doing. So it's very important as well. So to sum up everything which we set up to this point, well, if you think about the test estimation, it has two key factors. First of all, there are activities which are focused on the estimation itself. So you need to have an answer for the question, how much time do I need for testing? How much time do I need to perform all the activities, obviously including everything which I already said, like the supporting activities, the things which you cannot predict, the organization approach, the testing, the stuff like all the things which are happening around, whether there are vacations, whether there are like c 19 or any other things which will influence the estimation itself. So that's the first thing. But the second thing, as much as important as the test estimate, is to communicate the estimate. You should answer yourself to the question who should know about it, who should be notified about this and how to monitor, how to update it, what to do with the estimation when I already have it. So if you connect those two things, I believe that this is the perfect recipe for bringing the test estimation back into one of the agenda of our activities, to make it, again, a very important and to be perceived as a very important activity in the QA field. So in other words, if you want other people to know what you are actually dont and how you cares going to do it and what is like the key activities, what you are performing and to have the better understanding of your work as a QA, you should do the test estimation because it is one of the key activities which will influence the entire process. Itself. So remember about the test estimation, remember about the communicated and joining those two together will give you a way better life in a QA area. And that's pretty much my approach on the test estimation. So I would like to thank you for listening to my talk. If you will have any additional questions, feel free to reach me out on my personal email. You can find me on LinkedIn in other social media as well. And I'm also present on the Discord channel for this conference. So if you have any additional questions, feel free to ask them. And I would like to thank you for listening to my talk and enjoy the rest of the conference. Thank you.