Hi everybody. I'm Dana Scheider, a senior software engineer at Envato, and today I'm going to talk to you about software quality management when you don't have dedicated QA staff and about what software quality can learn from manufacturing. I want to start off with a definition of quality. For the purpose of this talk. Quality is the suitability of a product for its intended purpose, aided from the perspective of the consumer. That last part is really important. It's the customer who ultimately decides the quality of your product and what quality dimensions matter to them. So the first step towards establishing software quality in an environment without QA, or with QA for that matter, is going to be identifying those quality dimensions that the consumer requires of your product. There are a few elements of software quality or quality dimensions, including functionality, the features that your product provides, how well they work, performance or speed, ux, freedom from bugs or defects, and security a lot of the time when we talk about quality, we talk about it in a really narrow way, focused on freedom from defects or bugs. But in fact, all of these elements, all of these components kind of play into the consumer's ultimate satisfaction with your product. And so all of choose should be considered dimensions of quality. So in this talk, I'm going to talk to you a little bit about traditional software QA, but I'm going to focus on total quality management and I'll go into what TQM is in a moment. Traditionally, when we're doing QA and software, we do it toward the end of the process. QA is a separate process that happens during or after the development phase. Quality analysts or qas don't influence product's definition or design, so they strictly get involved later in the process. Make sure that nothing has bugs. Make sure that edge cases have been accounted for. Sometimes they'll check up on security, things like that. The page still requires HTTPs if it's a web app, that kind of thing. But mostly they're focused on keeping things but free and aren't involved in defining those quality attributes that may matter to the customer or designing the product. Quality management in this model is focused on finding and fixing defects, and not on overall suitability of the product for its intended use. And that's where TQM really differs. Additionally, qas usually have training and experience that most developers don't. This is really key difference as well, and it's something that needs to be accounted for when you're moving to a model without qas, the knowledge that qas have is essential and needs to be imparted to your development team, if you are going to be having them take on more of that work. More to the point, that training and experience has to be imparted to everyone in your organization. But I'll get to that in a minute. So what is total quality management? Total quality management, or TQM, is a quality management philosophy widely adopted in japanese engineering. It was developed in the 1950s by management consultant William Deming, and is largely responsible for the primacy of japanese engineering in the manufacturing world. I developed a quality process that was implemented at Envato and realized subsequently that it had a lot in common with TQM, and felt that TQM had a lot to offer. So in this talk, I'm going to focus on TQM principles and how we can bring more of them to our quality management and software. TQM starts with a customer focus, and that means asking the right questions at the very beginning of the process. Are we building the right produces? Are we building it in the right way? And how well does it serve the customer's needs or wants? There's a story about the railroads in the US. The railroads didn't do so well as new shipping routes came to be developed, as trucks and planes came to be used to transport goods and people. And an analysis of the situation reveals that the railroads saw their business as running trains along tracks, and that that was a service that they were offering to their customers. But what they didn't take into account was that what they were actually doing for their customers was transporting them or their goods from point a to point b, and that prevented them from developing the right product, developing it with the kind of flexibility that they needed to continue to be successful as customer needs evolved. So that's just one compelling business case for focusing on quality in this way. Another dimension of TQM is employee empowerment. In TQM, everyone is responsible for quality, and everyone is empowered to make decisions and to make interventions on behalf of quality. Employees can speak up when something isn't right without fear of retaliation or repercussions. Employees have adequate autonomy to do their jobs. If they feel that something needs to be done differently in the interest of quality, they should be able to do that without too much red tape. Employees are not overworked or burned out. People who work too much or are grinding too hard make mistakes. And so it's really important to, from an organizational perspective, focus on employee well being and making sure that employees are working the right amount, which for knowledge workers, the data indicates, is probably under 40 hours a week. So that's really important, too. Employees get ongoing training to improve their skills and to develop new skills that become relevant as the software world evolves. So those are all components of the employee empowerment that total quality management requires, and that good software quality management also would entail. Fact based decision making is another component. This is of course much easier said than done, but it's important to foster an organisational culture that's built more around fact based decision making than around political decision making. Making sure that whoever has a good idea, that idea is adopted, even if it's the janitor, that person has had the best idea and that's going to serve the interests of the organization. People need to feel empowered to speak up against people more powerful than them. So at Envato I know that I can take my disagreement directly to the CEO, and the worst that's going to happen is he'll disagree. And that's a critical component to creating an environment where employees are able to speak up about issues pertaining to quality or security, or a number of other factors. I strongly recommend baking this into your culture as much as you can. The final key component of TQM is continuous improvement of products and processes, which fits in really well with agile, focusing on an ongoing basis on whether youre doing things in the right way. What incremental improvements can you make to your produces so that they'll serve your customers better? Are there any process changes that could be made to make this happen? An example is I'm a big fan of checklists. You need to make sure that tests and docs are being written on every Jira issue, while just add a checklist to your Jira card and make it not done until that's done. Those sorts of things are the kinds of small incremental process improvements that can make a big difference in quality. So if you're not sold yet, I want to go into the kind of advantages of TQM a little more. The big advantage with TQM is that quality management starts from the very beginning. In this approach, you're starting with product specifications, visual design, UI design. Your approach to development so agile is going to be more conducive to quality management than waterfall, for example, since it will enable you to quickly respond to changes in customer needs or changes in specifications. Code quality is a component of quality management, since again it will affect your ability to quickly add new features to keep your code up to date and secure. It's really important to think broadly about quality. Quality is not about a freedom from bugs. Anything that's done well in your organization will contribute to quality, and anything that's done poorly, I guarantee will take away from it. Quality management in TQM is focused on all elements of design and development, so you're not just fixing bugs. It also results in a produces need to release patches or make additional changes to a released product. Since by starting earlier in that process and getting more people involved, youll get it right the first time more often. Quality is always the responsibility of the entire organization. And in some ways I think that it's better to not have a dedicated QA department because that drives this point home. Sometimes when you have a dedicated QA department, it's easy to say that quality is those people's responsibility. Keep it in a silo, throw stuff over the fence to them and everybody else can wash their hands of it. Quality is always the responsibility of the entire organization, and everybody in the organization needs to be formally vested with that responsibility, needs to have the opportunity to distinguish themselves by producing quality work, and needs to be held accountable for the quality of their work and for what they contribute to the overall final product. Some companies are going to have an easier time adopting TQM than others. So what are the characteristics of organizations that can adopt TQM? It's going to be cheaper and easier for you to adopt this approach if you have some of the components built in already. As the upshot, one of the most important things is having the total buy in of management. Unfortunately, as with a lot of things that you can do to make your software development go smoother and your product quality higher, there's only so much you can do without total management buy in. And remember that total management buy in, in the case of TQM, means management accepting that they and everyone under them is collectively and individually responsible for quality. So it's really important to make sure that there's that buy in and that understanding. It's going to be easier to adopt TQM if you recognize that quality management will always require time and resources. TQM will often result in cost reduction over having traditional QA. But that comes from making higher quality work the first time and not having to go back and fix things that you've messed up. It's not about not investing in quality in the first place. Youll will always have to invest in quality. And the sooner you accept that, the sooner you can start implementing good quality practices. Yeah, if you already have some of the elements of TQM in place, like for example, one of Envato's core values is that we tell it like it is, and whether it's with each other or with customers, we always speak up when something isn't right and are always honest and straightforward about problems. And that made it a lot easier to adopt some of these components of TQM because it relies on people individually having that ability to speak up about issues. You'll have an easier time if your organization has a culture of effective and multidirectional cross functional communication. That means that at each stage of the process, there's a collaboration happening. It's not management dictates to design what product needs to happen, what product needs to be developed, design dictates to development what that product needs to look like. Development dictates to everyone past them what the overall design of the software is going to be, and so on and so forth. Because that results in a total lack of empowerment for everyone later in the process. As it happens, sometimes designers have good input on what product needs to be developed. Sometimes developers have good input on design. Well, you never know. It could be any group of people. Youll just need to have a culture where that communication goes both ways and no one is dictating to anybody else. You're going to also have an easier time adopting a TQM approach if your code bases already have good automated test covers, if you currently have a QA department, all of that stuff is going to have to be moved into your code bases and that's going to be a certain amount of work. So if you already have good automated test bases in your code base, or test coverage in your code bases, that's going to be a lot easier. You're going to have an easier time if management already produces resources and creates conditions where employees at each point in the process can do their job well. If people are having to fight to get the tools or resources that they need to do their jobs, that's going to have an impact on quality. And finally, it's important that employees not be rushed, burned out or under excessive pressure to produces quickly. That just ends up consistently being at odds with the fact that quality management requires resources and employee time. People who are rushed, burned out, or under excessive pressure tend to cut covers and quality is usually the corner that gets cut along with security. So what are some success factors then, in terms of your organizational culture? Openness and safety are critical components to a culture of quality. Even junior engineers should feel comfortable asking questions and critiquing their own and others work. Someone once pointed out to me that engineers at different levels of their careers have different strengths. Senior engineers are great at project management and high level problems. Mid level engineers are great at coding, and junior engineers are great at theory that's really stuck with me and has borne out in my experience. And what that means is that everyone in the organization has something unique that they're able to contribute. So making sure that people are able to both feel comfortable asking questions and critiquing each other's work, as well as have their critiques listened to, is critical. Additionally, employees need to feel safe admitting and discussing their own weaknesses and mistakes. Collaboration is key to quality management under TQM and stands in opposition to competition. Competitive environments don't do as well with this approach, and the reason is because when you're focused on advancing yourself and on distinguishing your work above that of the others around you, you're not focused on holistically doing a good job as a team to bring a quality product to the market. Accountability is critical to developing quality products and delivering them to happy customers, but pressure is not the way to affect that. It's important to look at people's work holistically, look at the way they contribute to the team and understand that as often as not, when someone is failing to meet their goals or perform as well as they could, it's often because they don't have the right environment or resources to do their job, or because something else such as illness is getting in the way. And true culture of accountability understands that and is focused on enabling and empowering people to do their jobs well, not on pressuring them to do so. Appropriate management involvement is critical. Management needs to be available to provide necessary resources to enhance product quality, create conditions where everyone cant succeed, give employees the autonomy to do their jobs and to make decisions about quality, and focus on a collaborative environment that's oriented towards results. I want to go into a few recommended practices for Dev teams as far as how development work plays out in an organization that has implemented a TQM like approach to software quality. Number one is that you need to communicate regularly with product and design. Like I said before, bi directional communication. Know, maybe design gives you something that at a certain viewport width is going to look really strange on your UI. Developers should be able to give that feedback and designers should take it in stride. So everyone should be willing to take feedback from everyone else. People need to feel safe, constructively criticizing their own and others work. So if a developer looks at a design and thinks hell no, if I was a user this would not work for me. They need to hopefully, tactfully give that feedback to the designer and say, I think it would be better UX if users had the ability to do this, that and the other thing again, like with QA itself, there should be ongoing communication between product and design with no siloing, no throwing things over the fence. There's not a point where design is done and development begins. Quality is the responsibility of everyone. Using agile rituals and practices greatly enhances quality and your ability to respond quickly to changes in customer needs. Kickoffs are great. They enable you as a team to decide on an appropriate approach to each piece of development work and involve people from other functions or teams as appropriate. Pairing and mobbing using structured techniques and if you're pairing remotely, using appropriate tools for remote pairing that enable remote control of a single development machine are also really valuable things to improve quality. Generally, having a couple of pairs of eyes on something is always going to be helpful to producing a better product, providing you do have a culture of collaboration over competition. Retrospectives where you candidly discuss what did or didn't go well in a particular sprint are really important and where you're focused on improvement rather than pointing fingers. Culture of blame is not useful for improving quality and that feeds into the next bullet point as well, which is blameless post mortems where if something goes wrong, you want to think about improvements to process and communication, what actually was done wrong, not who did it. Robust code review is really important, and I can't overstate this. Code review is the stage where you look for bugs. Code review should be conducted by at least one qualified reviewer who did not contribute code to the pull request. Qualified means this person is familiar with the stack. This person is familiar with the problem space. If you have back end code that you're having reviewed, you want people who know back end code reviewing it more than your front end people. Again, it's important for everyone to have potential input into the process, and everyone's input is potentially valuable. So I'm not saying to exclude reviewers that have less experience in a particular area, but make sure that code is reviewed by at least one person who does have extensive experience in that area. Code reviewers should do manual testing. They should run the code on their local machine or in staging and try to think of edge cases that will break it. During that review, they should be asking themselves how and what has been tested with this, and whether those test cases are adequate. Are there additional edge cases that haven't been considered? It's important to keep in mind in this process that the happy path is often not the most common one. In one of my previous roles, I asked our head of product how many customers went through the happy path, and he told me 5%. So sometimes the happy path is actually quite rare, and it's important to make sure that your test cases are equally robust towards the kinds of situations your customers are actually encountering more frequently. Security should also be a part of this code review code reviewers should be informed about security and a kind of rule of thumb here to take away is that a reviewer who doesnt suggest changes is usually not thorough enough. And I'm not talking but a one line change that updates a third party library or something. But if it's a larger pull request that implements a feature or fixes a more complex but then reviewers should be suggesting changes and it's important to kind of as a reviewer, take it as a red flag if you find yourself having no suggestions. Tracking all your work will also help with software quality. You want to create issues or Trello cards or however it is that you track your work, you want to create one for even small changes. Even small changes should have adequate opportunity for review and testing. And that's a big part of the reason why you should create issues for them is that it's not just a oh, I'm just going to dash off this pull request, okay, I'm going to pick up this issue and I'm going to do a good job on it. Establish standards for pull request descriptions and comments and I'm talking about technical writing here. Information that should be included in every pull request description, such as context and links to relevant information or to the Jira ticket or Trello card or issue report that this is related to a summary of changes made. A section on the considerations and reasoning what alternatives did you consider? What alternatives did you maybe try and not have them work out? Why didn't they work out? What don't you like about the approach that you took and why did you take it anyway? Those kinds of things need to go into a PR description. And finally, there should be detailed information in each PR description about testing and documentation, how the code has been tested, what test cases have been tested, and what documentation has been written or updated as a result of these changes. Testing post release is another really important practice. So using the same manual testing procedures as during code review you cant to test in production. Ideally you want to test in a pre production environment like staging as well. And when you test your final product, you want to test it in as many environments as possible where it might be run. So if this is an app that might be run on different operating systems or hardware, older and newer machines, web apps that could be run in different browsers, apps that could be run on different devices or screen sizes. You want to make sure to incorporate all of those into your post releasing testing and as much as possible into your pre releasing testing. I recommend using detailed pull request templates to make sure that pull requests include the amount of information that they need and the type of information that they need in an organized fashion. I've encountered some disagreement with this from certain people, but I stand by it. I think that having a pull request template ensures that people include the appropriate information with each pull request that the others around them need to understand the changes they made and review their code effectively. PR templates need to include context, technical changes, considerations that were made in the process of development, detailed information about testing what kinds of teams, what kinds of automated teams have been used to test this code. How can a reviewer produces a but how can a reviewer manually test the changes? And as far as writing this stuff out, I recommend actually fleshed out test cases with steps 12345. These are the steps you should go through to manually test these changes. Click this button, fill out this form with this data, that level of detail what youll go wrong sometimes, no matter how hard you try to make everything good, there's still a certain risk that once you get stuff out in production, something goes wrong. We can't always change the fact that certain configuration is different in deployed environments from development environments, or that it's different in production from staging. So again, this is kind of part of people being empowered to speak up when things can go wrong and to admit their weaknesses and mistakes. People should be comfortable talking about what could still go wrong despite all their best efforts. What are the security risks? I recommend calling this one, but specifically, you also can often benefit from screenshots or videos or gifs that illustrate the changes that you've made. Our PR templates always also include a note that a PR is the start of a conversation, not the end of one, and that's a really important thing to remember. So these are just a few of the ways that you can enhance quality in your organization in the absence of a dedicated QA department. Total quality management, which was developed for manufacturing environments, turns out to have significant advantages in software organizations as well, in that it approaches quality from an organizational standpoint and makes quality the responsibility of every person in the organization, both collectively and individually. TQM can produces defects, improve efficiency, and make the end product more suitable for its intended purpose. TQM can also reduce your QA and defect related costs, provided that you understand that you still need to invest a certain amount of resources and employee time in quality management. Adopting TQM requires allocation of resources and full management support. It. And the last takeaway that I'd like you to get from this is that dev teams can make simple changes to their process to improve quality and make it an integral part of their work. Thank you.