Hi everyone. Conf 42 excellent. Ah, what we're going to talk about today are basically microservices refactoring the patterns. This is like a very fancy name. What we are going to do is talk about how to take ugly prepared, very ugly API code and refactor it to patterns. We're going to talk about the patterns themselves also. But why we're doing that it because we want cleaner code and easy to maintain. But we'll get to that. My name is Gil Zilberfeld and I'm a trainer and a consultant on everything development, testing, product agile, whatever needs you have in order to make software better. I'm the author of two books, everyday unit testing, everyday spring testing. That's from the Java world. And you can contact me on these things, Twitter as well. The backside has all the things that you need to contact me if you have any questions. I want to start with the goal, and the goal is always code that works over and over again. Code that works. Yeah, we know that we probably want to have code that works and we want it to work over and over again, meaning that we're going to go into that code. And when we go into that code and to add features, fixed bugs, whatever, we want to continue having this experience of having code that works now and after every iteration. Now what you see is basically the feeling when you need to go into most code, not just microservice code. The reason I picked this one is because it's more focused on things that microservices do. And everybody's writing microservices these days and we may not be able to focus on the time while we're building them, on what we actually want the code to not just do, but look like. So as we write more code and add more features and so on, basically if we don't refactor, what happens is that we don't want to refactor again and again and again code gets messy and basically it becomes like a trap for the next developer that comes in. That's going to be his or hers problem. So we don't want that. We want a couple of things that care going to go the principles that we are interested in. And the first one, the code, we want it to be easy to change again. If you are not going to go into that code again, it doesn't really matter. But if we care, easy, less risky. We want the code to basically go in there, make the changes and go out. So in order to do that, there are a couple of principles that we're going to talk about and coding guide us today. One is cohesion. Cohesion is about having code that deals with the same things being in the same place, files, modules, whatever. And second is the complement of that separation of concerns. Code that deals with different things need to be separated. It makes sense to us, but also it makes sense in terms of maintainability. And that is when we're going to fix something or touch something, we'd rather focus on that thing and not all the other things that go with it. Dependencies, I don't know if you change here, need to change there, so it makes it easier to change if there are no dependencies or things are separated. Second thing, we want it easy to test. We're not going to talk about much about testing today, but it is kind of a thing that we want, because if you think about APIs, we're thinking about raising up servers and we're going to see databases and Kafka queues. The more setup that we need to have, the more dependencies that we have in our tests, that is less likely that we say, I don't want to do it, I don't want to test. It takes too much time, too risky. So the more the code doesn't have too many dependencies, fewer dependencies there are, it is easier to test. If it's easier to test, chances are we're going to test it. The second thing is about loosely coupled interfaces and events. Now, Javascript typescript, we're going to show examples today. Events, we know about this, right? Events basically decouple the sender of the event. It doesn't care about who gets the event and what it does with it. And all the event handler needs to know where it came from. That's it. So this is like a separation of concerns as well, but it's also loosely coupled. That means that we can change one without changing the other. As long as we are keeping the contract. Interfaces are the same. Basically it's a structure of the data that we're sending. It's a contract. As long as we keep that, it doesn't matter what the data is. We can change code in one place without having impact on the other. And if it's not like that, that means that we're again raising the chances of not going to test. So what are these things that I mentioned? Cohesion. I rated Wikipedia the degree to which the elements inside the module belong together. Like I said, code that deals with something needs to be at the same place. Same place could be like a package or something. Think about if you have code that does things in the UI and the API and the database layer, that's not cohesive. Separation of concerns means that each module addresses separate concerns, a set of information that affects the code. Like I said, if it deals with something, it doesn't deal with anything else. Loose coupling means that each of the component has or makes use of little or no knowledge of the definition of other separate components. It's not a definition, it is the implementation. So the interface between things becomes a contract. But what happens on either side of this contract doesn't matter. If we keep it that way, we can change one of the sides without touching the other. That, like I said, is a good thing. I'm going to show you today the demo with a set of tools. It's going to be can XJs application Mongoose for accessing the database of MongoDB and Kafka JS for doing queues. But what I'm going to show you today is not related to these tools. These are the tools that I used for this demo. But the principles apply for each tool that you're going to use. So what we're going to do, we're going to start with a mess. I prepared some mess, nice mess, going to separate things, and we're going to refactor to patterns. We're going to talk about the patterns. And we're not going to go to the testing today, but I'm going to discuss the ability to test and how we're going to do that. Our API is about scheduling a meeting. Single API, don't want to go all out. It's not a full app. The principles are going to be in there in one single API. It's like magic. The entities that we have care, basically a calendar which contains meetings. Meetings have like ids and time and room and stuff like that. And meetings have invitees. People we're going to invite, which have emails and they have invitation status. Meetings have meeting status and they have two methods, meetings schedule and invite participants. The architecture is kind of a layered architecture, right, that we all know the rest. Controller the API is going to be next js uprouter the route to s file meeting logic, which is mostly what we call the application logic, which is the most important here. It's not about the other dependencies and frameworks or databases. Datax layer. Like I said, mongos. The database which you're not going to see in live demonstration, is MongodB. This is the API. It's going to be a post API called schedule. And this is an example of the body I'm going to send like emails, which is an array of things that we're going to. The list of invitees, really the date and time, which is a string in the beginning, a topic of the meeting and the room where we want to go. It's basically it, what is the problem? You'll see in a minute. It is a mess. And the API doesn't allow testing the state by itself, meaning this is like a void method. Okay. It's a function that we're calling schedule meeting. And that means if we want to know if the meeting was scheduled, then we need either to go to the database to see it or need another API to look if the state is changed. Same for the invitees, their status and so on. And we need to test everything together. I need to raise everything together, not just the server, also the database, the patch Kafka queues to send the events out. So testing this is like an end to end test or API test as we know it, full microservice test, not easy. And if we have enough platform frameworks, enough methodology, adding a next test would not be that expensive. But if it's the first ones, it's going to be expensive. And like I said, if it's not easy to test, chances are we're going to drop it. We're not going to test it now, test it somewhere in the future. Okay, let's go to the code. Okay, so you see on the screen, this is like regular next JS application out of the box. What I did is for each web API meetings and I have schedule 123456 for our steps. You wouldn't do it like that, but for today it's going to be enough. We are starting out with three files. The first one is commons. This is basically enums and interfaces, the invitation, meeting status and the request which comes into our system and invite message which goes out into Kafka. We have the route, which is our input into our system, the API. So we have a post that gets the request, creates a calendar, gets passes the meeting requests, validates it. Talk about that. Also that the request is enough to create a meeting, create a meeting, calls the invite participants and returns some kind of either success or failure message. Our objects basically have three, we have invitee, which is kind of a data object, but for now it will be okay. Our calendar, we said that it's going to have a create meeting, only the route, yes, calls inside it. We're creating the meeting object, saving it in our array and calling schedule and the meeting schedule and meeting logic really is here where we're scheduling the meeting. And here we open the database and save the data there. This is all these things. Here we have the other method, invite participants, which is called after that and basically opens up Kafka JS creates a producer, changes the state of the invitation or the invitee, sends the information and saves the data. That's basically it. I put on purpose all my objects in one place to make it a bit more messy. But it's not too messy. But that's the starting point. Okay, so the first thing we want to do is a bit more organizations like, yes, I gave you separation between the enums and the objects. We want a bit more. The more organization that we have, it's easier to move around, move around stuff, refactor stuff. So it makes sense. Let's go to our code again. Okay, so this was like phase one in our phase two, what we're going to do, and I'm going to use domain, which comes from domain driven design. We're going to mention a couple of things from domain driven design as well. Basically, what are the domain object. Basically we kind of draw a boundary around our objects and ask what's inside and what's outside. We don't have an outside, but we do have a boundary. So inside we have our entities, the calendar and the meetings and so on. And entities in domain driven design have properties and logic like we've seen, but on the boundaries they talk with other stuff and other stuff is databases, stuff like that, Kafka and so on. It's not part of the application logic. So we're going to separate those. And the way we're going to do this created a data file and the data TS file has the mongoose stuff which was in the objects before. So the invitee schema and meeting schema care here. And also the invite message which goes into Kafka as well, because it's on the boundary what we have left. I haven't touched either the common or the domain is the calendar. The invitee didn't touch the code of the participants in the meeting as well. So all this code is here. All I did is separated the crud mongoose things and the Kafka is still in there. We'll deal with that later. We still have these things, the models that we're going to use. So we talked about coupling and cohesion. Our code is still coupled to the database. Just move stuff around. Okay. Didn't do much, but organization helps. Next thing we're going to talk about is the idea of ports and adapters, and this comes from like 40 years ago by Alistair Coburn, one of the people who created the edge of manifesto and the idea of creating application logic, and this also applies to clean code. If you're coming from these areas, is that we're going to need to separate the logic application care from all the things that it talks with. Databases, UI already we have that Kafka, why is that? Remember we talked about separation of concerns. So we want to change one thing without dealing with the impact on the other. So examples, if I want to change database from MongoDB to something else, I now need to go into the meeting code and because the meeting is coupled to the Mongoose schema, so I don't want to have that. The idea is that we're going to keep the logic clean and we're going to get there and it's going to talk through interfaces with other stuff. Now on the left hand you see UI and you think about it, API is a kind of an interface, the high is an interface. So the UI can change and the logic can change. As long as I'm keeping the contract, the post the same. So that's the idea. So we're going to push things into our boundary and create separate objects for them. The first thing I'm going to do is if you recall and show you that in a minute, the route ts file not only got information and talked to the meeting objects, it also contained logic about the validation. And another idea from domain driven design is anticorruption layer. Don't let data creates invalid entities in your logic. The reason is that if you create, let's say we created an invalid meeting, that means we need to have code for all kinds of cases of invalid meeting. What to do? If I'm past valid meeting I have to take care of all kinds of cases and that creates more complex code. We don't want that. We want application logic to be as simple as possible. And therefore one of the ways to do this is through validation in our case, or anticorruption in the words of domain driven design, don't allow to create invalid entities. So we have this kind of code, we just want to put it somewhere else. What's the validation doing? Basically a meeting is good if it has a topic, if it at least has one invitee's and in the future, in the past it's not a valid meeting. That's a good meeting, that's what the code does. So let's go back to that code. So let's look at what we have right now before we touch it. So basically this is the code here. We pass the meeting request, get the information here and this is the validation. Basically if it's valid we're going. But like I said, if I want to check this code entire server, I don't want to. So let's refactor, just move this into its own class. So we create an object called meeting validator. And I just moved things in there. I also renamed a couple of things to make it more easier, but it's the same code. So this is a meeting validator. The route now just calls it now. Note the functionality didn't change, I just moved code around. But our meeting validator now this guy is fully unit testable, doesn't have any dependencies. We can add more stuff to it and more validations. The route file doesn't need to change. We separated concerns. So that's cool and easily testable. Next thing I want to talk about is repository. So repository, we think about it as a design pattern. It's not the original design of patterns, gang of four things, but it comes from domain driven design. And the idea is that it's like a wrapper for talking to the database, or mechanism for encapsulating storage, retrieval and search behavior, which emulates a collection of objects. So basically it's an orm layer, we call it the data access layer. And of course mongoose does that and every object relational mapping tool does that. But it comes with a quirk. Every Orm tool comes with a quirk. And the idea is that because it's a general purpose tool, it doesn't talk our language or application logic, it doesn't talk about meeting and scheduling and stuff like that. It talks about schemas and stuff like that. Now you say well what's the problem with that? There's no real problem with that. But the more we have languages in our code it becomes harder to maintain. We don't want that. Cleaner code speaks one language. And remember what we wanted is our application logic to be separated from all the rest. So we're going to have some kind of adapter and a repository is a kind of adapter. Let's go back to the code. So we have our data, this is the schema and stuff like that. And like we see, we saw our domain objects connect to the database, to the mongoose server in the code, it's coupled and we have our properties here which care created and directly into mongoose. We want to do that through a repository. So let's do this through a repository. We'll create a meeting repository and basically move the code that talks to the database here. So we moved like these things here and I created on the meeting repository a method called a function called admitting which does all the things against the database that the meeting did before. But now everything sits here and it speaks the language of meetings like add meeting and update meeting. So in our domain object our meeting code is now shrunk a bit because it doesn't have all this data. All it does is have is a repository which it creates and it speaks with admitting. And now I have the flexibility of replacing mongoose with redis, I don't know, whatever. So this is a repository, the repository pattern which is kind of adapter which is what we're going to talk about next. So adapters, adapters are basically interface change like you see on the screen here. The definition is a software design pattern. This comes from an actual design pattern from the Django four book, a very boring one, a software design pattern that allows the interface of an existing class to be used as another interface. We already saw a repository, it's an interface change that behind it does something other adapters that you may know proxy has the same interface but different implementation or facade, a simpler interface around something that's very complex. So we touch the repository. They haven't touched Kafka js yet, so we're going to do that. The repository is like database adapter is like general thing for other stuff. So let's go to the code again. So look at the code that we have here. It's basically configuring the Kafka server, opening the connection, sort of creating a producer and creating the messages that we're going to send. And that's it. Here is like the repository from before. So all this needs to go somewhere else. Again it doesn't have any place in the meeting itself. So we're going to have a Kafka adapter. The bootstrap server goes here. If I want to change it somewhere else, take it from somewhere else. I need to touch the Kafka code, not the meeting code. It's here basically we have an invite interface again it's can application logic language rather than Kafka and invite message. Now all adapters repositories can be, I won't say unit tested because they don't have much code right? They just like do something can operation but they are easier to test because now I don't need for testing this the database of our database. We don't need cockpit, so the separation helps us. Apart from that, I moved the invite message here. It was before that on the boundary stuff because it now belongs to the Kafka stuff because this is how we send stuff outside. And our meeting object has also even more so shrunk. It just goes over the invitees and call the invite sender, which is not a Kafka sender by the way, calls the invite with invite message. Again, look at the code. It's application language. It doesn't have any framework language in it much. So we like that. Okay, we have one more thing to jump into, and this is mostly about this. So we have in the meeting, it currently creates the repository and the adapter and we call it coupling. Right. The meeting basically is coupled to the meeting repository and Kafk adapter because it creates them and we want to separate that. It's not possible to do it completely, of course, but it is possible to put it behind the refactor. And that means we'll have more flexibility of changing which repository we're talking with which adapter, and we can change it to Rabbitmq instead of Kafka, again without changing the meeting code. So the solution is usually a factory. Let's go to the slides. So welcome to the factory. You all know factory. Factory basically is a pattern, again from the Django folk book, a mechanism for encapsulating complex creation logic and abstracting the type of the objects for the sake of the client. Yeah, but what it really means is we're separating the creation of the object, put it somewhere else, and the usage of the object. So we think about factories and maybe even singletons. We'll see a couple of examples of that somebody else creates on their own time. The object and the user of this object just gets it out of thin air and just uses it. Singleton, if I'm already mentioning that, is a class that allows a single instance of itself to be created, to be given access to the created instance. Usually we have Singleton in order to reuse resources, save resources, not just pop things up all the times, concentrate them in one place. And we'll see a couple of examples because usually think factory comes with a singletone, it doesn't have to be there. So let's go back to the code. Okay, so I created a factories file basically, which creates factories. And I give you three examples of how to write a factory. What for each, the calendar factory is what we usually think about as a factory, right? It has a private instance that we can't access from the outside and creates the calendar. And this is like a calendar only factory, it doesn't do anything else. I can change the calendar from the outside and it only gives me a calendar, which is useful if I want to do some kind of validation before that and so on, and make sure that the creation was correct and so on. I'd like it to be in a refactor. But there's another one that you probably know about. This one, this one is the same as this thing, but this one, the meeting repository factory, this one is actually public. So what's the difference? Here it's public, here it's private. Well if it's public I can change that. If I can change that in testing scenarios I can create a mock repository, put it in or read from JSON server or JSON file rather than a full repository that goes into mongols. So having this kind of pattern, a public one, allows you to change things on the fly. Is it risky? Yes, but yeah, Javascript is risky. Anyway, the final one I have here as a factory is I move the invite message and invite sender here. And the invite sender factory validates the instance before creating one here it just creates a new Kafka adapter. So this pattern, unless I set this and it's public from the outside like in a test, it will create the default one in our case is the Kafka adapter. Now that we have this, let's look at our code. So our meetings, instead of creating it by themselves, take them from the factories. And that means that if I'm testing our meetings, I can set up the factories beforehand and then create a meeting and it will just swoop in and take what I injected in. Same goes with our route. Yes, for the calendar, which currently is just a calendar, but you can get it from somewhere else. Obviously I use patterns here. And one thing I didn't mention until now is like passing things as parameters or what we call in fancy word dependency injection, which is also a pattern that we use in order to do that. But that means somebody up there on the top of the tree needs to create everything for the little people down there. Usually it's not something that we want, a factory or some kind of a service locator or something with the get instance is better for that. Let's go back to the slides and summarize. So what did we get? We organized the code. We didn't do anything but basically create wrappers, give them nice names and move code around. But the code is now organized better. We know where to put the next feature in, because we know if it's validation, we're going to the validation object. If it's replacing the database or adding some kind of another database, we'll go to the repositories files. We have a guidance of how to access external services. With factories and adapters, we separated code. We're more flexible of either replacing or enhancing or upgrading stuff without touching. The most important thing for us, which is the logic itself, the application, the application rules, we don't want to touch that because these are the main brains and main sources of value that we write. They care more testable, because now not only we have smaller components, they can be tested separately. Now it's not replacing like an end to end test, but if we have smaller tests that give us confidence that these things actually work, we don't need to test everything around the big API test. You can have one scenario with an end to end test, not like ten scenarios of scheduling stuff. In our case, for example, cases, we have all the validation stuff that wasn't possible to test without an API beforehand. Need to create all kinds of meetings and see what happens and operate the database and so on. This separation helps us. If it helps us, that means it's easier to test. If it's easier to test, that means the bigger the chance that we're actually going to test it. Finally, maintainability, which kind of pulls everything inside it here maintainability is kind of weird word. Everything is maintainable. Everything is testable. Also given enough time, resources, motivation. But usually the motivation is external. We need to change that. It becomes easier and less risky to do so. So to summarize, know the patterns, not many of them really. There's a lot of patterns, but I mentioned like four or five. I really recommend going into domain derivative design. If you're writing microservices code, understand the concept and bounded context and boundaries and stuff of that entities, what goes in, what goes out, understand the architecture and the clean architecture thing that we're trying to achieve. And once you have these kind of guidelines and understanding, and it becomes easier to move things around. And most of the things I've done, I've done like half and half, I use vs. Code, move to another file if possible. But a lot of the times it was like manual typing, so small steps, but we need to do this in other languages. We have more stronger tools to refactor, here we have less. So the principles still apply. And like I said, the tools that you use, it doesn't matter because the patterns are the same and they are repeating. The principles are the same and eventually what you want to come to something that is more maintainable. That's what I wanted to share with you today. If you have any questions, email them to me at Gila Testingil or go on Twitter and bother me there or everywhere else. I'm doing lot of webinars and I'm doing short videos which you can go to see everything on my YouTube channel. So go there, watch the videos, subscribe and be happy. Thank you for watching this and hope to see you next time.