Hello, my name is Pavos Kripak and today with Anna Warno we will present how to use the complex AI based forecasting methods for investments portfolio optimization. Anya is senior data scientist in seven Bullscom. She is working with the time series AI forecasting methods, also other machine learning methods and I'm a cto in the AI investments company which aims to build the complete portfolio optimization platform using the latest achievements in machine learning methods. Even more, we have built the first version of this software which is currently deployed for one of the investment funds from the Luxembourg. Today we want to tell you about the time series and what are the most advanced and available currently method. And in the second part, Anya will show the real hands on session how to work with the time series, how to use this method, what are the results, and even more, Anya will show how to use the reinforcement learning. For that purposes, of course I want some toy example, but to give you feeling and impression I will go through this method listed here very briefly and in the details. Anya will show NBIT's method and complete this NBIT's methods to the typical statistical method of the forecasting. That's the agenda. Let's start why we are talking about this topic. Recently we are observing the dynamic development of the time series forecasting method, especially machine learning based methods which has significantly higher accuracy than statistical method. Time series forecasting is applicable for many, many applications from forecasting sales stock, some make some forecasting in health and also in visual recognition and so on. But it is also applicable for financial time series and for financial time series we are able to achieve over 60% of the accuracy, which is a very significant number for the financial time series. For other time series which are easier to forecast. Of course it's not a very impressive number, but this one gives a significant edge in the portfolio construction and the results of the investments shows that. What is time series? Time series is the ordered time list of all values with the given attribute one or more, and time series forecasting is the ability to forecast the future value of that series which is not known in the given time and machine learning time series. AI forecasting methods are recently developed and achieve very good results. There are methods of the time series forecasting which are based on the machine learning models and methods. Till this time or two years ago, three years ago, there was the domination of the statistical method of the forecasting arma Rima, exponential smoothing and other methods. But recent evolution and recent breakthrough in forecasting which probably the most significant point is the M four and M five competition show that machine learning methods are simply better than statistical one by significant margin of the accuracy. M competition is probably the most prestigious and for sure scientifically baked competition related to the time series. AI forecasting methods organized by University of Nicosia and Professor Spiros Makriakis. And the fourth edition of that competition, it was over two years ago, almost three years ago, and in this fourth edition 1st 2nd place were taken by machine learning based method. One of this method will be presented very briefly today and the M five competition has been finished. Previous year was dominated by the machine learning method. What is important about M four competition is that it was on the very huge number of the time series. There was 100,000 different time series with different number of points and different frequency. So the method which won was really tested on huge data set. And this winning method is ES hybrid methods. It was designed by smell data scientist from Uber and there are some very unique elements of that method. First one is to include data preprocessing into the back propagation. So the parameters of the data processing which was exponential smoothing were learned together with learning weights of the neural networks. Second thing was very unique approach for the assembling and assigning models cto the given time series. And third thing was the neural networks. In this methods are used LSTM networks, but not typical LSTM, but residual delighted and with attention. So this residuality was one of the first application of this concept. It's very heavy applicable for the image recognition to the LSTM and two time series forecasting and it gives very good results, especially that suave use different residuality rules for different frequency of the time series. Second method which will be described and presented by Anya later in the session is the NBITS method. This is the purely machine learning method comparing to the ES hybrid which is considered hybrid and it is constructed from the block. So it is some kind of the stacked architecture with also very unique construction. There are different type of the blocks for trends, seasonality and some generic blocks. And also there are some features related to the explainability and transfer learning. And also the NBITs uses advanced model ensembling. Here we have a brief overview how the NBITs looks. Here is also the reference to the paper. More details are provided. Third method, probably the easiest one to use is Facebook profit. It is based on the general's additive models. It is available as a library and also provide quite good accuracy levels. And the important thing of that method is that we are able to calculate also the range of the uncertainty or confidential levels. Not only the one value and this method is available as a library. So it is very easy to start using this method for your own proposals. Third method or fourth method is the glue on TS. It is also complete framework of the libraries also available to download and use. It is more difficult to use than just profit, but much more powerful. There are many methods included, many way of the preprocessing data. It has a support for the cloud computing and also very strong support from the community and scientific researchers. Also here we have the paper to the TS. There are very interesting concept of different methods, different network structures, probabilistic distributions, type denveral components, it is constantly evoluted and new elements are added. And also it's available, as I said, as a library so it can be download and use. And the third method is very unique based on that. Settling machines, also called stochastic learning automata is very unique for the forecasting. But stochastic learning automata itself is known for the long time since the 60s from previous century. Here we have one of the latest paper related to that topic and the main differences of settling machine comparing to the neural networks is that it learns the probability distribution for each parameter and each of the feature and based on that it is able to forecast the given value. Yeah, that's all from my side. Now Anya will tell a little bit more, even much more than my introductionary part, how to use forecasting method on the example of the NBITs and compare NBITs with the statistical methods. And also we'll show the toy example how to use the reinforcement learning hello. Today I would like to show you what can be done with financial time series data, stockpresses and Python. We'll go through the whole process from downloading data to Python EdA. It is explanatory data analysis, time series data, preprocessing modeling for time series forecasting. And finally we will try create an automatic investment strategy. So how can we access stock price data from Python? There exists many rich API, but for just learning purposes we will use Python package y finance based on Apache software license for Yahoo Finance historical data. Thanks to this package we can easily download daily stock prices data with columns such as open, high, low, close volume, etc. We just need to pass company index. Here I have some saved example company indices and we will run one. And here we can see the output with our downloaded data frame for visualization and indicators calculation. We don't need to implement everything from scratch because Python has many libraries which might do the work for us. For example, I will use Quantstat library which has around, I think 20 implemented visualizations and functions useful for stock price analysis. Here are some of the plots drawn with this library. So after the general data analysis we will move cto the data preparation. So the step before the modeling part, because the topic of this conference is Python, not machine learning, and maybe some of you are not familiar with data science, I will not discuss with the details what is the purpose of some of the specific activities. I will just say that it will help to choose the models and hyperparameters range during the further steps. What can be done during the time series analysis and data preprocessing? Usually we do the stuff like seasonal decomposition to relate the trend or season from the data. Time series transformation might be crucial for some of the statistical models. Outlier detection helps to understand the data. Most of those tasks can be done with stats models package, which provides classes and functions for the estimation of many different statistical models. There is belief that r is much better, at least for the classical time series analysis, than Python. But this task model package, at least in my opinion well, substitutes r and even has a similar API. And here we have some example transformations which are often applied when we are dealing with stock prices. Some of them are extremely easy like lock, but some of them, like Kalman Filter, are complex and harder to implement from scratch. Fortunately in Python we have also package for that. Maybe I will show you this Kalman filter because it's interesting, it has very nice smoothing features. And why do we need data transformation? There is many reasons. Some of the models require special inputs. Sometimes it helps also with numerical issues. Okay, so we have very briefly went through examples of data preprocessing which might be applied to time series data. And we can now move to more interesting part the modeling. So our goal will be prediction of closed stock price. We will use daily and weekly frequency, usually stock prices. Series with these frequencies are very, very noisy and very hard to predict. Currently efficient models use more data than univariate series from single stock can provide. For example, recently tech data like financial news or tweets are taken also as model input for stock price prediction. But today we just want to show very simple examples. So the first step would be division of our closed time series into three sets, train valve and test. Those familiar with machine learning will know what the purpose of this division is, but for those who are not on the train set, we will train. Our model validation set is for choosing the best hyperparameters, and on the test set we can see how our model performs on unseen data. It's a good practice to firstly try with benchmark models before starting to train complex state of the art. We have three benchmarks, name prediction where we are predicting always the last seen value, and two classic statistical models, exponential smoothing and Ariba. Both of them came from stat model package. So after training the baselines, we can choose more complete model. Our choice will be NBIT deep learning neural networks which outperformed all models on famous, prestigious and four time series competition where the task was prediction on 100,000 time series from different fields including finances. There exists a few NBIS implementation, but we will use one from Pytorch forecasting as it's very well developed and based on very convenient pytorch lighting library. So after the training, choosing the best hyperparameters and evaluation of all of our models, we have time for model comparison in the data frame here we can see calculated metrics. The best methods are highlighted with green color first. Metrics such as mean absolutes error or mean squared error indicate how much values predicted by our model are close. CTo the reals as you can see, naive method achieves surprisingly good scores. However, from investor point of view, its prediction is worthless. It does not give us any information whether we can actually earn something. It's a very common mistake. During the stock price prediction, people look at metrics like mean absolute error, plot presented values and everything looks fine at the first sight. But it turns out that their model is not better than net prediction and that the predicted values are just shifted. To overcome that issue, we have to calculate how accurately model predicts prices, ups and downs. Results are placed in the last column. Okay, so all of these results presented here are only example numbers which differ a lot across stocks. Sometimes they are better, sometimes they are worse. If we decide that results are promising, we can try CTO, trust our predictions and build an investment strategy with larger and more accurate predictions horizons, our investment strategy will became more and more complete. Can we have model which will plan everything for us? Yes, we can. So we will go to the last step, creating a fully automatic investment strategy with reinforcement learning. For those who are not familiar with reinforcement learning, you probably heard about super powerful AI models which are able to became masters of go. I think that's the most popular example of usage of reinforcement learning. Or maybe you heard about models which learned how to play Atari games. Reinforcement learning is an area of machine learning concerns with how intelligent agents also take actions in environment in order to maximize the notion of cumulative reward. So it's like showing model a game show that it should maximize game score and using just on that it will learn how to win. Stock resembles game where profit is the reward. It's a very hard game with huge randomness factor, but still we can try use reinforcement learning for that. So first we need to have an environment. There is several already implemented stock simulation environments in Python like for example Finerl. Most of them is based on OpenAI gym library and we can use them, but we can also create our own environment. So I just want to emphasize that we will just make it as simple as possible, just for fun and in real scenarios to actually work well. Our solution should be more complete. Take more data, more training epochs, more agents, more complicated policies. But just for today we'll build something extremely simple. We'll use OpenAI gym library and single stock data will be used. Observation space will be just a vector with past and values. Currently owned shares plus currently owned money plus time for end plus predictions and predictions step ahead as predictions are created with models from the previous steps and our reward would be money plus owned shares multiplied by current stock price. And you have two possible actions, buying and selling shares in percentages. Each day we can sell up to 100% of owned shares and we can buy maximum free shares. Our game will always end after 20 days. In reality it's extremely short period for investment, but we'll leave it short because of the training time. We'll use PPO algorithms from stable baseline free and in stable baseline free we can add our own Pytorch models which might be good for further development. And for neural networks we will use the simplest option, multilayer perceptrons. And now I can show you the code. And the environment is longer, so it's probably less understable. The most important functions in environment class are this obligatory from gym amp like step in step function. Each day we take action, calculate the reward, take the next observation from the environment, and check whether we are not done. And here we have training and evaluation codes which are shorter. And these codes also should not be different if we use another environment. We evaluate our agent on unseen data last 365 days for one year from our time series, and we are running our model 365 times each time with different starting day. And the biggest question is whether our investment strategy work. Does it have a potential? Firstly, we can check whether we are earning anything, or maybe we are just losing money. After that we should compare our method with other strategies. There is many methods for backtesting investment strategy. There are even nice package for that in Python. But for now we will just test two very simple strategies. Buy and hold and nest strategy. Nest strategy relies on buying if the last n values are higher than the current price, and sell whether the last n values are lower than today's price, with the same limit for buying actions as in our environment. Okay, so here are the results from the four consecutive runs with four different randomly selected companies. And in real scenarios here we have profit in dollars, but the better way is to calculate it in percentage. And as we can see, our reinforcement learning agent is not the worst. Of course, these plots here, these results cannot be treated as a proof or anything because it's hard to think about fair conditions for comparisons of these strategies. But it looks like there might be potential in this method. It seems that our model actually learned something. Okay, and that's all for today. Thank you for your attention.