Conf42 JavaScript 2024 - Online

- premiere 5PM GMT

Taming Memory Leaks in Node.js: A Deep Dive into Diagnostics and Solutions

Abstract

Tired of memory leaks in your Node.js apps? Join me to master essential tools and techniques to diagnose, fix, and prevent them. Learn how to use Chrome DevTools, Node’s –inspect, and profilers to keep your apps running smoothly and boost performance in production!

Summary

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hello guys. I hope you all are well and thank you for your time. today the topic on which I will be talking, is the memory leaks in Node. js. So in this talk, we will deep dive into the, diagnostic and its solutions, to fix these leaks. First of all, I would like to thank Conf42, platform to, to give me this great opportunity to talk on this most important topic in, JavaScript. so let me start by. Introducing myself. So my name is Mohamad Ya Rafik, and I am working as a nodeJS backend developer at Find My Facility. And I am also happy to share that. I was a featured author on T-Zone with an article which was titled as JavaScript Framework, the Past, the Present, and the Future. I have around four plus years of experience in JS and web backend, development. So let's dive into our topic. what is, Memory leak. So here is a brief introduction. Like the memory leak occurs when the, program or function takes memory and keeps it if it's in no longer in. Use which results to the, the creation of the system performance. So in this topic, we will explore, like what would cause the memory to leak in this, in, not just application and their impact on the system. so why, Memory leak, memory leaks matter in Node. js. I would say the memory leak, is, is, The memory leak matters in every language, if you are using Python or C sharp or any other language memory, matters, but why most, matter in this NodeJS, we will find it out. as we all know that NodeJS. relies on an event driven architecture. the event driven architecture includes, some event loops and callbacks and functions, which executes at, at the same time without blocking other functions, ease and objects. due to this feature, it becomes more, more highly effective for input out for input, output heavy applications. So when we talk about even driven architecture, it's mean. it means that, there are so much function execution happening on the same time at the back end, so due to which, if they are not properly handled, so the memory leaks can cause, server or our application to crash the response time. I become slow and even sometime, the worst happened is application failure. when, when we talk about the long, running applications. so what are the symptoms of memory leak? Like how we identify that if the certain problem is occurring again and again. so what is cause and, Will the cause be the memory leak or not? So there are some signs, with the help of which we can identify this. So some common size, science includes, the increasing memory usage, with the passage of time, which includes the. application performance slow, and frequently garbage collection pauses. So with the help of, with the help of these signs, we can identify the leaks. And with the help of this set of symptoms, we can early, identify this and we can maintain Our application, stability on time. So I will share some like real type, real life, applicant, examples for this memory leak in production, phase. So actually we were working, actually we, as a team were working on a project where, we found some memory leaks, like persistent memory leaks, it's, in the, Production phase in an high traffic, no, just application like, like so much input, output and processing at the back end, due to which our server starts again and again. so we start, debugging the problem that what, causes the server to start again and again. So when we check. Down the issues, so it, it involves some deep, analysis of the application memory. usage patterns over the time. so at first we were very confused that from which side we should start and we should start doing debugging, whether, it's the server issue or any function or any, controller issue. So at the first it was very, messed up. after that, so it, so as I explained, It was a challenge for us to identify that what is the issue and from which point we should start, debugging, our leak or the issue, on which we are facing it. With the help of the memory snapshots and profiling tools, our team was able to pinpoint the exact issue which causes the memory leaks. So I will explain further this like snapshots and profiling tools, that how they work and how we can identify the, issues, like that. memory leak inside our Node. js application. uh, there are a few common mistakes, which causes the memory to leak in Node. js, Node. js applications. So these leaks can. Erased from, unintentional, global variables assignment, which is a very common mistake from like from the beginner, developers and, some improper, Closures are written, references in event lister or callback. I will explain like what are the, closures, the closures are the function which, retain the, reference of an variable from an outer function and it keeps with it until the function, execution complete. but sometimes these, closures are not, properly handled due to which, these, those variables are stayed. Inside the memory for a long time, due to which it like overloads the, memory and some event listeners and callbacks these, if they are not, properly handled or optimized. So these, these will cause the memory to leave. Okay. So there are some tools, with the help of which we can, Identify these leaks. first of all, I will explain the, using chrome def tools for memory profiling. so chrome def tool, is a tool like is a powerful memory profiling tool to capture, snapshots, compare heap, Okay. Location and identify memory leaks. So what are like, snapshots? So snapshots, are the timeframes where we can, check the state, of the app at any, a specific point. Like we can check the snapshot of the application before executing the function and after that as well, with the help of which we can identify the memory usage of the app before the execution and after the execution of the certain functions or objects. With the help of this, we can identify the, and analyze, if there are memory leaks and high memory usage in no J applications. Okay. there is an inbuilt tool for no J, which is called like inspect flag. like it is, like this tool, helps in the, in debugging anger, Like this is an debugging tool, which helps to identify any leak, and how to, use this built in, feature to track down memory leaks and some in efficient memory usage. So we can also track, track, memory leaks. if. If we are encountering any extensive memory usage inside our app or after the execution of certain functions, we can use this inspect tool to identify, certain issues. Okay, so heap, snapshot as as I mentioned earlier, the heap, snapshots provide a detailed view of the memory allocation in your application. with the help of. Snapshots, we can record the snaps off of our application at different intervals so we can, analyze the memory usage before the execution of certain tasks and after the execution of those tasks. So with the help of, which we can identify, like which object or function is taking more, memory and which and after the execution of which function the memory leak occurs. Okay. the memory leak also occurs when objects, that should be garbage collected, are, were, Remain in the memory. after the execution of the functions. so there might be some variables or objects which are kept inside, inside the memory and they are like no longer in use. so this also cause, the memory to leak. So it is also important to track those certain objects and the event listeners, which. We are using, so after the execution, they like, they kept alive inside the memory, like which, results the memory to overflow. Real life solutions. So actually I was working, on a project, recently and I like identify after the implementation of few functions and some event, listeners that the execution of like after the execution of those processes and functions, the, response was getting slow, time to time. then, so after that, I, analyze few functions and few event listeners. due to which I was able to identify that there is after the executions of few functions. So the, the. The solution was so, simple, the issue was the, like there was such an even listener inside my, functions and inside my apps, which kept, which used to keep the references of an unused objects, due to which the memory gets full and the execution and it takes A lot time to, to execute those functions. So the, it was hard to identify the issue at first, but once, when it was, identified, the solution was simple to make those even listener, to, optimize and to, remove those length, listeners, once they are executed or finished their execution. and ensure that the proper cleanup should be done after each execution. So now I will explain about the, Garbage collection, the garbage collection, like what's in a way, it is an V8 engine, garbage collector, within, Node. js once the execution is, like the execution of the certain, functions and objects are done. So then it, so that it. Collects those unused objects and, and even just a little bit are kept inside, inside the memory. so it, so the garbage collector frees up those memory, which no longer in, in use. uh, this helps, to avoid the memory leaks in our, application. Okay. So using, clinic. js, it is, it is an powerful, diagnostic tool, uh, specifically, designed for Node. js applications. So it helps to identify any memory leak at certain points and, see, and if there are any CPU, bottlenecks and those queries which are taking time, for execution. So it also, so it helps to, identify those issues as well. So it's better that we should know that how to use these tools efficiently. because once we know how to use these tools, it will be very easy to identify memory leaks. At the early stage, like in the, development stage before going, into your, production phase monitoring tools for, memory usage. uh, previously. I have defined some tools which we can use those manually at certain points, but there are some, monitoring tools if we use, them, to analyze our memory usage at the, like from the start to the end of our application. So these tools like help to monitor at each. Stage like, like at what stage the memory is going high and at what stage the memory is going low. So the tools like new relic and data dog, they offer some real time monitoring of, of the memory usage in production and environment. So we can use these tools, specifically if our apps are live. So we can check, the. memory usage after the execution of certain features inside our app. So with the help of these tools, we can, proactively identify memory leaks, before they, they become too big. so there are some, best practice and tips I will share with you, with the help of which we can, Prevent these leaks in our application. for instance, like a wide using, global variables as they kept in our. Memory throughout our application life cycle and the other best example, sorry, and the other best, practice or tip you can say, to ensure that the event listeners which we are using inside our app are properly. handled and removed when they are no longer needed, especially in those apps, which are like long running apps. So these tips might help you to identify and remove the any memory leak up to, uh. 90 percent for your Node. js application. Okay. The other thing is, like optimizing the event listeners and closures. So as, so as I, mentioned, mentioned it earlier that the, that due to the, event, uh, after the execution of event listener, even listeners like, and closures, have their variables and even and even listeners are there inside our memory for a long time, due to which the memory leak occurs. so these are the major sources of. Memory leaks, I believe, if they are not, probably be, properly handled. So by optimizing these, these elements, we can handle, our memory usage quite, efficiently. Okay. So there is, one method, inside our, no j which is called process dot memory. Use it. So this method is a built in, built in, method for Node. js, which helps to monitor heap and RSS memory. so the RSS memory here is known as President said size, which keeps the footprint off of the application, that how application is behaving like before the execution of the function or certain feature. And after the execution of certain feature, the execution of queries or like taking data or taking request or getting the response from the DB. We can say these are the footprints of the applications. uh, with the help of this, the execution of this method, we can check the logs and we can track like any, memory growth and identify any issue. if, if. If any function is causing the memory to leak so we can identify them before they can get escalate. Okay. I believe that it would be a challenge to, detect any memory leak in our real time systems. like I believe that it is a challenge when dealing with some. Memory leaks, with, real time systems as they have some, continuous operational going, going on all the time. we can, avoid them, by doing some well defined, infrastructure of the system, like writing a clean and nice code. And, like using the tools after, Using the, diagnostic tools, after the execution, after the execution of certain, functions and objects, uh, so this thing will lead it to, reduce any memory leak. It's, within our real time systems. So I would say the, future proofing, not just applications. we can, protect them by, by building the application in a, in an. defined infrastructure, defining, defining functions efficiently and optimizing our event listeners and our closures as well and a wide, Using global, variables. So with the help of these things, we can avoid memory leak. and also these things help in the scaling of our Node. js very easily and efficiently as well. So what is the, the. Conclusion of this talk, is that the memory leak can be a significant impact on our, performance. but with the help of the right tools and right techniques and tips, we can, Identify them and fix them efficiently and on time as well. Sorry. So this, presentation will cover some real, life, examples and, some, tools, and some best tips to help to tame this, memory links in, not just. So thank you, for your time. Thank you.
...

Muhammad Yasir Rafique

Nodejs Backend Developer @ Find My Facility

Muhammad Yasir Rafique's LinkedIn account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways