Jamaica real time feedback into the behavior of your distributed systems and observing changes exceptions errors in real time allows you to not only experiment with confidence, but respond instantly to get things working again. Those you hello, thank you for joining my session. Today we will talk about machine learning and we will share with you the different strategies on how to build machine learning mlpowered applications with JavaScript. So before I start with these talk, let me quickly introduce myself. I am Joshua Arvin Lat. People call me arVs. So arVs, arVs. So from Arvin. So the shortcut is arBs. I am the chief technology officer of Newworks Interactive Labs and I'm also an AWS machine learning hero. So there are maybe 20 to 30 something machine learning heroes all around the world, and we save the day one day at a time by sharing our knowledge in our domain expertise. I'm also the author of the books Machine Learning with Amazon Sagemaker Cookbook and machine learning Engineering on AWS. So if you are interested in learning more about machine learning experimentation and deployments in the cloud, then this book may be for you. So let's start by talking about how machine learning works and what its applications are. So machine learning is very powerful. So if we were to solve anomaly detection problems, if we were to work on product recommendation requirements and so on, then we may be able to utilize machine learning by using algorithms and data to solve our requirements. So if you are not yet familiar on how machine learning works, basically what it does is it helps us perform automated tasks which are usually performed by intelligent beings like humans. So for example, we have an image of a cat here. If this were to be processed by a human, a human would easily be able to identify if this image is a cat or not a cat. However, if this were to be processed by a machine, this may be somewhat hard to do, especially if you're going to hard code and you're not going to utilize data to process this, especially if there are an infinite number of possible variations for the cat images. For example, for this one, if you hard code the implementation of classification for the first image, how would you be able to process the following images? One of the possible solutions in order for us to properly classify if an image is a cat or not a cat is to utilize machine learning. So there are different strategies, there are different algorithms available in machine learning, but in order to solve this type of requirements, we may be able CTO use, let's say something like this, the machine learning process. The machine learning process involves utilizing data to build and train a machine learning model, which can be used to perform inference or predictions on new data. So getting back CTO the cat example, let's say that we have 100,000 images of cats. We use those 100,000 images to train a machine learning model and then that model will be evaluated and the performance would be checked. So let's say that it has 90% score. This would mean that maybe nine out of ten times it would get the answer correctly. So if you're happy with that model, then you would be able to deployed it. And then if you, let's say, have a mobile application, you took a picture of a cat. If you cases that image to that machine learning model, then that machine learning model would then give you either a one or a zero. Maybe zero would be cat and then one would be not cat. So it depends on how you implemented the inference part, but more or less that's how it works. One point that you should know is that this is a simplified version of the machine learning process. In real life. The machine learning process is circular and there are a lot of things we need to take into account, like model training. We need to make sure that we are able to refresh the model as well. Because once you have deployed the model after two weeks to one month, there might be a chance that your model has degraded already and you need to replace or refresh it. Now that we have a better idea on how machine learning works, we can now try to answer this question, why use JavaScript for machine learning? For those data scientists and machine learning practitioners listening to this talk, maybe you would also ask the same question, especially if you've used already something like Python to solve different machine learning experiments and requirements. So there are different advantages when using machine learning. But before we go straight into that one, let's answer this question first. The question that we need to answer is can we build a machine learning powered application using JavaScript? Is it a yes or is it a no? So the correct answer to this one is it's a yes. So we can build a machine learning mlpowered application using JavaScript because there are different libraries and tools available to support these types of application. So even if there are different tools and libraries available for other languages, and even if some languages, let's say Python and even r, have been customized more for machine learning, why should we consider using JavaScript for machine learning in certain use cases? So there are some scenarios where it might make sense to use JavaScript for machine learning, especially for a team which is already using JavaScript heavily. So let's say that you have a team of, let's say five developers, and they've been using JavaScript heavily for all of the projects. For a certain company, if they were to learn a new language, let's say Python, it might take them a while. And maybe the tools and the pipelines that the team has already is focusing more on the JavaScript side of things. So in order for a team to have to spend less time learning a new language and focusing on the implementation part using the same language, which is JavaScript, then types, if they were to implement and do some machine learning stuff, then they can do it with JavaScript. The second possible reason why people should use JavaScript for machine learning is that they can also perform machine learning references on the browser. So that's pretty cool because the browser, you can do some JavaScript stuff there, and instead of the inference call being made from the browser or the web app front end to the back end server, it will all happen in the browser. So we will dive deeper into that later. And here, as mentioned earlier, we can avoid the need to learn a new language, because for this one, maybe you are a front end developer and you're just starting out and you need to implement and utilize a machine learning model, then yes, you can do that using JavaScript. So you may be able to perform inference in the browser, and you may also be able to use JavaScript on the server side. And even if a lot of tools have already been prepared for, let's say, python, there are also corresponding set of tools in JavaScript which we will see later. So let's now dive a bit deeper into why and how we can perform machine learning experiments and deployed in the browser using JavaScript. On the left side we can see a simplified diagram on how server side inference work. So what do we mean by inference? When you have a machine learning model ready, you want to perform predictions, you want to use that model to perform inference, meaning you want to perform an intelligent decision or task. So in this case, let's say that the problem we're trying to solve is the identification or classification if the image is a cat or not a cat. So if you have an image in the browser, you will need to send that image to the services through a request. So that request, which includes the image, will be processed by the server and the server which has the model. The model would then be used to process the image and the output would be either one or zero. So 1 may represent, let's say not cat and these zero would represent cat. So the value, the output value would be returned back to the browser and then you can basically update the user interface using your favorite JavaScript library and you can display these cat. So if you upload an image of a dog, then the server would then respond, not cat. So that's basically how server side inference work. There's a server behind the scenes. On the right side we can see the alternative these we can perform references in the browser directly. So how would that work? When you load the page, the model would be loaded as well on the front end side and here in the second scenario, no servers are needed, at least on the processing side. So when the front end code receives some sort of event, instead of pushing the data to a request to a back end server, the inference will happen in the browser itself on the client side. So there are advantages CTO this because for example, if these are network issues, then the second scenario wouldn't be affected because it works offline. And if you don't want your users to pass their data to the server, then yes, you can do this as well because it helps with data privacy. So yes, so these are the advantages of performing machine learning references on the browser. So as mentioned, you can work offline. So let's say that after loading these page, you decided CTO disconnect your Internet connection. Your application will still be interactive and it will work just fine. So that's super helpful, especially if you have Internet connection problems or there's very limited or very slow Internet connectivity, which of course would affect latency. So here there's better latency, better performance from a network standpoint and there no servers are needed. So in some cases there are cases where you might need to have a server, but there may be cases, especially for smaller models where you can deploy it in the browser. And it also helps with data privacy because your customers or users data do not need CTO be sent to the server, to a centralized server which can be analyzed and processed by the team who built that server. So of course these are the advantages. So what may be some disadvantages to this one? So once you load that model in the browser, in terms of model security, that model can already be used or at least downloaded from these browser. So that's just the only thing you need to worry about. However, if you're not worried about model security, then this may be the solution for you. But again, these are amazing advantages. And browsers of course support javascript. Now that we know that we can do some machine learning stuff in the browser, would this mean that we would be able CTO use a machine learning library in JavaScript along with your favorite JavaScript front end, framework or library? The answer there is yes. So that's good news for us, because if you are already using libraries or frameworks like ReAPP, view and angular or other new frameworks and libraries, then it's all about doing some coding work to use your machine learning library and your model with it. So in some cases you will be able to train your machine learning model in the browser directly and in some cases as well, you'll be able to perform references in the browser also. So it really depends on your requirements. In most cases you may just need to focus on the inference part and do the training part in the server. So what are examples of libraries which can be used to perform machine learning experiments and inference deployments in the browser, the browser, the website front end. So here are some examples like Tensorflow, JS, even onx runtime for web. So those can be used. And this can easily be used with existing libraries and frameworks because they're basically just code. And these frameworks and libraries just help with processing the data and the different elements in your web application to make it more interactive and to help you manage the front end code better. So how about state management? So if you're not yet familiar with how state management works, feel free to watch my presentation from last year where I talked about how to do pragmatic state management in react, angular and ujs. So feel free to use the QR code here or just search in YouTube in order to watch this video. But you'll definitely learn a lot because state management is a topic on its own and it's better if we watch it separately. But again, if you have a data intensive application and you want to manage the machine learning stuff in the browser and do some interactions and make your applications interactive, these yes, you may need to learn about state management, especially when you're doing heavy Javascript work. All right, now that we have talked about web apps a bit, let's now talk about mobile apps. Wow. So we all thought that having machine learning models deployed in the browser is amazing. You should know that we can also deploy mobile apps, deploy machine learning models in the mobile apps itself. So how would that work? The more standard or the more common ways to perform machine learning deployments and even training is to perform all of these inside servers. So if you have a mobile application and you have data, these in order for training to happen, at least in the first scenario, we need to pass the data to the centralized system or server. These model gets trained there and the model will be used. Similar to an example earlier, where there's the client, the front end which can be web or mobile, and then there's the request and the server would be processing it because the model is in the services. However, there is a second option, an alternative where as you can see on the right side, similar to how we deploy nodejs in the browser or the client front end, we can also perform on device machine learning. This means that the data of the user which will be used during inference no longer needs to be sent to a server. So what happens here is that all the magic is happening inside the mobile app. So yes, so if the mobile app contains the model already, then yes, this is possible. And in terms of advantages, the advantages are the same. So it helps with data privacy, it works offline, no servers are needed, and it's faster. So why is it faster? It's faster because there's no more round trip from the mobile app to the server and back, especially when you need to perform inference, it can all happen in real time. So this is an example of how you can perform and do and use machine learning nodejs in your mobile app. So if you have, let's say something like react native and use Tensorflow JS, then you can do something like this. Now let's talk about machine learning in the cloud using JavaScript. So why use the cloud? So when using and utilizing the different cloud capabilities and resources, one of the advantages when you're using the cloud is that you have full control over the type and the size of resources that you're going to use to perform the needed operation. So in this case, machine learning training and deployment usually when you need to work on more complex machine learning models, there will be a big chance, especially for certain types of requirements where you will need to deal with bigger data sets and bigger models. So here we can see that if we need a stronger computer, which is significantly bigger and faster and stronger than what we have in our local machine, then types we can utilize the cloud for it. Let's say that we have a 1gb data set and it needs to be processed by a server. The training job and script running inside that server would be able to do it properly. However, if you were to process that large data set and use an algorithm which is kind of resource intensive, resource hungry, then a smaller machine would struggle with it. That's why we can utilize the cloud for these types of requirements. In addition to that, if we were to perform automated hyperparameter optimization HPO, we can utilize multiple cloud servers all at the same time to do what we need to do. So in this case we will perform machine learning experiments and we will build a model. So in most of the examples that we know and most of the basic examples that we've tried and we're aware of, we usually deal with one machine learning experiment. So one machine learning experiment may produce one model. However, in real life we would need to produce multiple models. We would do a lot of iterations in order to get the best model possible. So instead of us trying to wait for, let's say one r for one model, and if we have 100 tries, 100 different configurations for different types of training jobs, we would have to wait a really long time, let's say 100 hours, if we were to do it one at a time. However, we can do this in parallel, meaning that if we need to produce 100 machine learning models and get the best model from the results of the experiments, we can do that all at the same time. So machine learning experiment 12345, all happening at the same time, and you would be able to complete everything in maybe one to 1.5 hours. So these is amazing. And these is one of the advantages of using the cloud, because you're able to spin up and delete resources as needed. In addition to that, there are a lot of services and platforms available to help us manage cloud resources better while performing machine learning experiments and deployed. So there's something called Sagemaker, I even wrote a book about it. Right? So Sagemaker is a mature machine learning platform and it's a service of AWS. So if you've been using AWS, then this is something that you can easily use because you have an account. There are of course alternatives in other cloud providers, let's say GCP and Azure. So feel free to take a look at it, especially if your team is already using that cloud platform. But what I would like to share these is that there is of course an SDK available for JavaScript. So you have an API. So the service has an API. If you need to talk to the API, maybe perform a machine learning experiment using this algorithm, you can do that with Python. So a lot of people know that Python can be used to work with this service. However, if you do a bit more research, you will realize that different sdks have been made available in different languages, and JavaScript is one of them. So the capabilities available in Python is of course available in JavaScript, because most of the magic has been implemented already inside the service itself. All we need to do is call the right API call and we'll be able to have the much needed capability or functionality. So this is one good example. So let's say that you have a machine learning model trained and we need to deploy it instead of us trying to build our references server from scratch. Let's say something using expressjs and all the needed libraries to read the model and use it for inference. Maybe we can do that faster. Maybe we can have a real time endpoint, or maybe we can have a serverless endpoint or use other types of inference solutions by just having these right configuration. Maybe we would need just maybe five lines to six lines of code just to do all of this. And this is easily doable in maybe 30 minutes to 1 hour, especially when you have the code snippets ready to perform machine learning deployment. So again, if we were to compare this with building your own inference server, building it yourself may take three days to five days. However, if you were to use a mature platform, then with 30 minutes to 1 hour you will be able to have something which you can present to your boss already for approval. So those are some of the things that you should know, because choosing the right solution starts first by knowing what's possible. Then you can select later on when to use what. Take note that some services also allow us to utilize docker containers. And even without the container image support, we can already use JavaScript inside a lambda function. So lambda, which is a function as a services, allows us to write code and focus on the code without worrying about the server. So what we can do inside a lambda function is that using JavaScript, we can load the trained model and use it for inference. So if you use lambda with a serverless API service like API gateway, then we can deploy the model in that HTTP API without worrying about the servers. If we were to use libraries in JavaScript and we want to build a custom container image, that's also possible because lambda has container image support as well. So for simpler JavaScript projects then yes, we can use lambda with JavaScript directly. However, if we would need more complex setups and environments for our JavaScript application utilizing machine learning models, then we can utilize container image support of lambda. And lambda can be used across a variety of services and architectures. For example, you have a streaming service, let's say there's one record passing through that service because that's how the architecture was implemented. So what you can do is whenever there's new data coming out of that streaming application or service, you can have a lambda function, process one record at a time, and perform references in real time. Also, it's also possible to create graphQL powered APIs and inside that GraphQL powered API we have a lambda function, processing the request and utilizing the machine learning model to perform inference and returning the response back to the user. So this is also another powerful implementation and you can definitely use JavaScript for these types of cloud applications. So before we end these talk, let's have a quick discussion on these best practices. It's about using the right tools for the job. So right now you are aware that there are different ways to solve the same thing. However, it is crucial that we always start first with the business requirements. And again, a lot of technical professionals always end up focusing on the technical solutions because wow, that's cool, right? We can use JavaScript in the browser and do some references there in the browser directly. However, you need to be aware that this business requirements may also include security requirements as well. Because what if it was possible? However it would affect the stability of your company, especially if the model has been obtained from the browser, right? So be very aware that this is the right process. CTO follow we start first with business requirements and then that's the time we focus on the technical solution and we use a variety of building blocks to solve our requirements. At the same time, even if you know the right process, even if you have a good understanding of the business requirements and the different tools and solutions possible, take note that everything that you do will always be a function of time, meaning that you will not have an infinite amount of time to do what you need to do. So, similar to our example earlier where you have a team of five developers all able to use JavaScript, then yes, if they need to utilize JavaScript for machine learning needs, then that's possible. However, if the team is using something like Python or R for machine learning already, if you were to use JavaScript for machine learning, you would have to assess, because the current team working on the machine learning task needs to learn how to use JavaScript, especially if they have no experience using the language in the screen. We can see the different steps when performing machine learning training. So here we can define the model, we compile the model, load the data, use the data to perform machine learning training so we fit the model and then once we have the model, we evaluate it and we save it somewhere. So this process, this training step can be performed in the server and it can also be performed maybe in the browser as well. And it can be performed, I guess in the mobile apps also. So there are different ways CTO do this. So the question is where should we train the model so we can train the model? Depending on the resources needed for training, and in a majority of the cases training may be performed in the server because at least in the server side it's more manageable, especially when you're doing it in the cloud. However, there are cases where training may be performed and it's recommended sometimes to do it in the browser closer to the user because if the user's data is stuck in the browser, meaning it cannot be sent to the centralized server, then yes, you can perform training in the browser itself because it helps with data privacy. So let's say that the data which will be used contains confidential information. Instead of passing that data to the server, you can perform training closer to the user in the browser or even in the mobile app. However, in this scenario we will focus on training our machine learning model in the server and we can also deployed the model in the server and we can do this with JavaScript. So as long as we use the right set of libraries and tools and frameworks, then we can do this similar to how we're doing it in let's say Python. And this is what we talked about earlier. We can train our machine learning model in the server and then when you load the web page the model gets downloaded as well in the browser. So there are maybe one or two steps needed to convert that model to be usable by the library or framework in the browser. At the same time, if you can make the model smaller then better. Also because this is recommended generally for nodejs which are of small size because if you were to load your web application it needs to load as fast as possible. So if your web app is going to load for five minutes because the model is too big, too big, then these model is better off in the services side because the request can be performed later on. And here this is also possible. We can train the machine learning model in the server and then in the mobile app, the model can be downloaded there directly. So the advantage of this setup is that there's going to be on device references and the data of the user which will be used for inference no longer needs to be passed in the request to the server. So it helps protect the data and you can do some cool and interactive offline inference when using this type of approach. So there, so if all you have is a hammer, everything becomes a nail. So in this talk we talked about the different strategies and solutions on how to use JavaScript for machine learning. And there are different options, but feel free to to do a bit more research before trying to use a solution because once you implement this in production, it's usually hard to fix or change. So make sure to be an architect and those the right solution and tool for the job. So thank you again for listening to my talk. We talked about a lot of strategies and hope you learned something new. Bye.