Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hello guys.
I hope you all are well and thank you for your time.
today the topic on which I will be talking, is the memory leaks in Node.
js.
So in this talk, we will deep dive into the, diagnostic and its
solutions, to fix these leaks.
First of all, I would like to thank Conf42, platform to, to give me this
great opportunity to talk on this most important topic in, JavaScript.
so let me start by.
Introducing myself.
So my name is Mohamad Ya Rafik, and I am working as a nodeJS backend
developer at Find My Facility.
And I am also happy to share that.
I was a featured author on T-Zone with an article which was titled
as JavaScript Framework, the Past, the Present, and the Future.
I have around four plus years of experience in JS
and web backend, development.
So let's dive into our topic.
what is, Memory leak.
So here is a brief introduction.
Like the memory leak occurs when the, program or function takes memory and
keeps it if it's in no longer in.
Use which results to the, the creation of the system performance.
So in this topic, we will explore, like what would cause the memory to
leak in this, in, not just application and their impact on the system.
so why, Memory leak, memory leaks matter in Node.
js.
I would say the memory leak, is, is, The memory leak matters in every
language, if you are using Python or C sharp or any other language
memory, matters, but why most, matter in this NodeJS, we will find it out.
as we all know that NodeJS.
relies on an event driven architecture.
the event driven architecture includes, some event loops and callbacks
and functions, which executes at, at the same time without blocking
other functions, ease and objects.
due to this feature, it becomes more, more highly effective for input out
for input, output heavy applications.
So when we talk about even driven architecture, it's mean.
it means that, there are so much function execution happening on the
same time at the back end, so due to which, if they are not properly
handled, so the memory leaks can cause,
server or our application to crash the response time.
I become slow and even sometime, the worst happened is application failure.
when, when we talk about the long, running applications.
so what are the symptoms of memory leak?
Like how we identify that if the certain problem is occurring again and again.
so what is cause and, Will the cause be the memory leak or not?
So there are some signs, with the help of which we can identify this.
So some common size, science includes, the increasing memory usage, with the
passage of time, which includes the.
application performance slow, and frequently garbage collection pauses.
So with the help of, with the help of these signs, we can identify the leaks.
And with the help of this set of symptoms, we can early, identify
this and we can maintain Our application, stability on time.
So I will share some like real type, real life, applicant, examples for
this memory leak in production, phase.
So actually we were working, actually we, as a team were working on a
project where, we found some memory leaks, like persistent memory leaks,
it's, in the, Production phase in an high traffic, no, just application
like, like so much input, output and processing at the back end, due to
which our server starts again and again.
so we start, debugging the problem that what, causes the
server to start again and again.
So when we check.
Down the issues, so it, it involves some deep, analysis of the application memory.
usage patterns over the time.
so at first we were very confused that from which side we should start
and we should start doing debugging, whether, it's the server issue or any
function or any, controller issue.
So at the first it was very, messed up.
after that, so it, so as I explained, It was a challenge for us to identify that
what is the issue and from which point we should start, debugging, our leak or
the issue, on which we are facing it.
With the help of the memory snapshots and profiling tools, our team was
able to pinpoint the exact issue which causes the memory leaks.
So I will explain further this like snapshots and profiling tools,
that how they work and how we can identify the, issues, like that.
memory leak inside our Node.
js application.
uh, there are a few common mistakes, which causes the memory to leak in Node.
js, Node.
js applications.
So these leaks can.
Erased from, unintentional, global variables assignment, which is a
very common mistake from like from the beginner, developers and, some
improper, Closures are written, references in event lister or callback.
I will explain like what are the, closures, the closures are the
function which, retain the, reference of an variable from an outer
function and it keeps with it until the function, execution complete.
but sometimes these, closures are not, properly handled due to which,
these, those variables are stayed.
Inside the memory for a long time, due to which it like overloads the,
memory and some event listeners and callbacks these, if they are
not, properly handled or optimized.
So these, these will cause the memory to leave.
Okay.
So there are some tools, with the help of which we can, Identify these leaks.
first of all, I will explain the, using chrome def tools for memory profiling.
so chrome def tool, is a tool like is a powerful memory profiling tool to
capture, snapshots, compare heap, Okay.
Location and identify memory leaks.
So what are like, snapshots?
So snapshots, are the timeframes where we can, check the state, of
the app at any, a specific point.
Like we can check the snapshot of the application before executing the
function and after that as well, with the help of which we can identify
the memory usage of the app before the execution and after the execution
of the certain functions or objects.
With the help of this, we can identify the, and analyze, if
there are memory leaks and high memory usage in no J applications.
Okay.
there is an inbuilt tool for no J, which is called like inspect flag.
like it is, like this tool, helps in the, in debugging anger, Like this is an
debugging tool, which helps to identify any leak, and how to, use this built
in, feature to track down memory leaks and some in efficient memory usage.
So we can also track, track, memory leaks.
if.
If we are encountering any extensive memory usage inside our app or
after the execution of certain functions, we can use this inspect
tool to identify, certain issues.
Okay, so heap, snapshot as as I mentioned earlier, the heap, snapshots
provide a detailed view of the memory allocation in your application.
with the help of.
Snapshots, we can record the snaps off of our application at different intervals
so we can, analyze the memory usage before the execution of certain tasks
and after the execution of those tasks.
So with the help of, which we can identify, like which object or
function is taking more, memory and which and after the execution of
which function the memory leak occurs.
Okay.
the memory leak also occurs when objects, that should be garbage collected,
are, were, Remain in the memory.
after the execution of the functions.
so there might be some variables or objects which are kept inside, inside the
memory and they are like no longer in use.
so this also cause, the memory to leak.
So it is also important to track those certain objects
and the event listeners, which.
We are using, so after the execution, they like, they kept
alive inside the memory, like which, results the memory to overflow.
Real life solutions.
So actually I was working, on a project,
recently and I like identify after the implementation of few functions
and some event, listeners that the execution of like after the execution
of those processes and functions, the, response was getting slow, time to time.
then, so after that, I, analyze few functions and few event listeners.
due to which I was able to identify that there is after
the executions of few functions.
So the, the.
The solution was so, simple, the issue was the, like there was such an even
listener inside my, functions and inside my apps, which kept, which used
to keep the references of an unused objects, due to which the memory gets
full and the execution and it takes A lot time to, to execute those functions.
So the, it was hard to identify the issue at first, but once, when it
was, identified, the solution was simple to make those even listener,
to, optimize and to, remove those length, listeners, once they are
executed or finished their execution.
and ensure that the proper cleanup should be done after each execution.
So now I will explain about the, Garbage collection, the garbage collection,
like what's in a way, it is an V8 engine, garbage collector, within, Node.
js once the execution is, like the execution of the certain,
functions and objects are done.
So then it, so that it.
Collects those unused objects and, and even just a little bit are
kept inside, inside the memory.
so it, so the garbage collector frees up those memory, which no longer in, in use.
uh, this helps, to avoid the memory leaks in our, application.
Okay.
So using, clinic.
js, it is, it is an powerful, diagnostic tool, uh, specifically, designed for Node.
js applications.
So it helps to identify any memory leak at certain points and, see, and if there
are any CPU, bottlenecks and those queries which are taking time, for execution.
So it also, so it helps to, identify those issues as well.
So it's better that we should know that how to use these tools efficiently.
because once we know how to use these tools, it will be very
easy to identify memory leaks.
At the early stage, like in the, development stage before going,
into your, production phase
monitoring tools for, memory usage.
uh, previously.
I have defined some tools which we can use those manually at certain
points, but there are some, monitoring tools if we use, them, to analyze our
memory usage at the, like from the start to the end of our application.
So these tools like help to monitor at each.
Stage like, like at what stage the memory is going high and at what
stage the memory is going low.
So the tools like new relic and data dog, they offer some real time
monitoring of, of the memory usage in production and environment.
So we can use these tools, specifically if our apps are live.
So we can check, the.
memory usage after the execution of certain features inside our app.
So with the help of these tools, we can, proactively identify memory leaks,
before they, they become too big.
so there are some, best practice and tips I will share with you, with
the help of which we can, Prevent these leaks in our application.
for instance, like a wide using, global variables as they kept in our.
Memory throughout our application life cycle and the other best example,
sorry, and the other best, practice or tip you can say, to ensure that
the event listeners which we are using inside our app are properly.
handled and removed when they are no longer needed, especially in those
apps, which are like long running apps.
So these tips might help you to identify and remove the any memory leak up to, uh.
90 percent for your Node.
js application.
Okay.
The other thing is, like optimizing the event listeners and closures.
So as, so as I, mentioned, mentioned it earlier that the, that due to the,
event, uh, after the execution of event listener, even listeners like,
and closures, have their variables and even and even listeners are there
inside our memory for a long time, due to which the memory leak occurs.
so these are the major sources of.
Memory leaks, I believe, if they are not, probably be, properly handled.
So by optimizing these, these elements, we can handle, our
memory usage quite, efficiently.
Okay.
So there is, one method, inside our, no j which is called process dot memory.
Use it.
So this method is a built in, built in, method for Node.
js, which helps to monitor heap and RSS memory.
so the RSS memory here is known as President said size, which keeps the
footprint off of the application, that how application is behaving
like before the execution of the function or certain feature.
And after the execution of certain feature, the execution of queries or
like taking data or taking request or getting the response from the DB.
We can say these are the footprints of the applications.
uh, with the help of this, the execution of this method, we can check
the logs and we can track like any, memory growth and identify any issue.
if, if.
If any function is causing the memory to leak so we can identify
them before they can get escalate.
Okay.
I believe that it would be a challenge to, detect any memory
leak in our real time systems.
like I believe that it is a challenge when dealing with some.
Memory leaks, with, real time systems as they have some, continuous operational
going, going on all the time.
we can, avoid them, by doing some well defined, infrastructure of the system,
like writing a clean and nice code.
And, like using the tools after, Using the, diagnostic tools, after the
execution, after the execution of certain, functions and objects, uh, so this thing
will lead it to, reduce any memory leak.
It's, within our real time systems.
So I would say the, future proofing, not just applications.
we can, protect them by, by building the application in a, in an.
defined infrastructure, defining, defining functions efficiently and optimizing our
event listeners and our closures as well and a wide, Using global, variables.
So with the help of these things, we can avoid memory leak.
and also these things help in the scaling of our Node.
js very easily and efficiently as well.
So what is the, the.
Conclusion of this talk, is that the memory leak can be a significant
impact on our, performance.
but with the help of the right tools and right techniques and tips, we
can, Identify them and fix them efficiently and on time as well.
Sorry.
So this, presentation will cover some real, life, examples and, some,
tools, and some best tips to help to tame this, memory links in, not just.
So thank you, for your time.
Thank you.