Hi everyone. My name is Gong. I'm a product manager at Spotify and today I'm really happy to be here to discuss product discovery and in particular a data driven approach to product discovery. So my talk will be divided into three main parts. Firstly, I will discuss, based on my experience, how we can leverage data in product discovery. Secondly, we will look into a few examples where companies like Netflix or Spotify were able to leverage the data that they collect in order to further strengthen their product mode. And finally, we will also look into factors in order for us to run data driven product discovery effectively. So without further ado, let's jump right in with the first part, how we can use data in product discovery. So product discovery is essentially a decision making process, which means it requires data as inputs in order for us to make informed decision. I think all product teams will agree with me on the importance of data in product management and more particularly in product discovery. We all want to leverage data as much as possible. So the question is how can we do that effectively? So I could argue that one of the biggest challenges for product teams when they look for answers to these questions is the abundance of data. To the extent that we are all overloaded with data in the digital world nowadays. Take established large scale products for example. We are probably looking at like millions of data attributes and billions of data points being collected every single day. So it can easily feel overwhelming for product teams not knowing which data is relevant and important in order for us to identify which problems and subsequently which solutions to work on during the product discovery process. So how can this be avoided? To answer this question, lets take a step back and look into the definition of the product discovery. So I recognize that different organizations might have slightly different understanding of product discovery. So to make it easier for us in this talk, I could define product discovery as a process of identifying key problems and their optimal solutions in order to achieve the desired outcome. So it all starts with the desired outcomes and we need to work from there in order to identify which data is relevant and important for us during the product discovery process. So lets discuss this further in the next slide. So Im quoting Tyriasautorrus, the author of Continuous Discovery Habit. Good discovery starts with a clear desired outcome. So what I would like to add here is that even though it is important to have clear desired outcomes, it is often just a starting point for the product teams. And this is because for product teams nowadays, more often than not, we are given desired business outcomes, not desired product outcomes. So whats the difference between these two? So again, im quoting Teresa here, a business outcome is a metric that moves the business forward, while the product outcomes helps us understand if the product is moving the business forward. Some examples of the business outcome include to grow the user base, to grow the revenue, to grow the profit, or to increase the retention. So business outcomes are lacking indicators. Once you are able to measure it, it is often too late to do anything about it. And this is why it is critical for product teams to be able to define desired product outcomes based on the desired business outcomes that they are given. For example, if the desired business outcomes is to grow the revenue, some potential desired product outcomes could be like existing customer will buy more, or existing customer will be willing to pay more for the same product, or new customers from new countries will start buying products or new customers from new segments will start buying products. So defining desired product outcomes based on the desired business outcomes is a major step toward effective product discovery. And this is because product outcomes are often leading indicators which enables product team to learn about the impacts and reiterate in quick cycle feedback. More importantly, product outcomes are most within the influence of the product teams. So once the desired product outcomes are defined, product teams should also make sure to translate those outcomes into appropriate product metrics. And this is to ensure that we can measure the current situation as well as validating any future impact of your work. So one tip, I have to select the right metrics to optimize for to select those metrics based on the product life cycle. And this is of course in addition to the desired business outcome that we were given. So if your product is still in its early phase, the focus is usually on the retention. This is because you want to ensure that the users can perceive the value of your product before doing anything else. Subsequently, the product metrics could be on the churn rate, on the daily active user, weekly active user ratio, or on the user engagement. So once a job business has achieved their product market fit, the focus of the business usually shifts towards scaling the business. So for product teams, metrics like sign ups, conversion or customer acquisition costs become more relevant now. And this is because it will provide valuable insights into how effective it is in terms of your customer acquisition strategy and how effective it is for you to convert them into engaged users. So for established businesses, the focus now would be on the revenue and profitability. Consequently, for product teams, you can track metrics like average revenue per user, customer lifetime value, or free to pay conversions. These metrics tell you whether the product and the business is financially sustainable and whether we can extract value for the business. So now that we have identified all the product metrics, do we think we have all the data we need in order to run the product discovery? Maybe not quite yet. So, with product metrics, we often measure quantitative data like clicks, reactions or source query, and they tell you a lot about the what of the user behavior. However, they also do not tell you about the why behind those actions like motivations, frustrations or emotions of the users. And this is where the qualitative data will come into play. So, user interviews, usability tests or customer support interactions are really good examples of qualitative data that help us understand the user perspective. They comprehend the quantitative data that are being collected automatically from our product in order for us to have a full picture of users when doing product discovery. So now that we have discussed how we can leverage data in product discovery, let's take a closer look at some examples of companies like Netflix or Spotify were able to leverage the enormous amount of data that they collect in order to further strengthen their product mode. So content is the most important product of Netflix, and they have really disrupted the entertainment industry by leveraging a vast amount of data to inform their decision on personalization and the content acquisition strategy. So Netflix invests heavily into personalization in order to improve user engagement and retention. For example, on the film recommendations, best layouts or thumbnail images were decided based on individual user behavior, and on this effort create a tremendous value. It is estimated that an uplift of $1 billion in revenue was created thanks to these improved retentions. Another way Netflix uses data is to guide their content acquisition, that is, to decide which title to acquire or to produce. So they have an enormous amount of data on user viewing habits, preference or user behavior, which then can be used for them to identify the gaps in their current offering and then make strategic investments in producing original series or acquiring existing content. So as a result, they were able to produce or acquire widely successful series that I'm sure all of us have heard of, like House of Cards, Stranger Things, and Orange is a new black. Quite similar to Netflix actually. Spotify is also famous for the ability to leverage a vast amount of data that they can collect on user listening habits, preference and user behavior to identify and build highly engaging features to make users come back for more so their widely successful features. Discovery Weekly is a prime example of personalization in action. So every Monday, users will receive a brand new playlist of songs that Spotify things that you will love based on your history. Its like you are having a dj personally DJ actually, who knows your music test better than you do. Spotifys data driven approach actually goes way beyond the personalized playlist. They use data to inform their overall product strategy and their content offerings. For example, they notice that users are manually creating different playlists for different activities, for example, like work out. So they decided to create automatic playlists catering to those needs. And Spotify does not give this data and insights for themselves. They actually use it to empower the artists on the platforms. For example, in the Spotify for artists app, those insights are being used for artists to better understand their fans and which lead to better targeted marketing and tour planning. Now that we are moving on to the final part of this presentation, let's discuss the factors required in order to effectively run a data driven product discovery. But first, let's quickly go through the common mistakes or the challenges when it comes to leveraging data in product discovery, the first mistake is to focus on the wrong metrics. We actually talk about this a lot at the very beginning of this presentation, so it is critical to prioritize metrics that actually align with the desired outcomes. The second mistake is actually being too reliant on data to the extent that data becomes a blocker instead of a leverage. This is particularly critical for product teams working on early stage products where data is usually limited. All the product teams should be comfortable making decisions with imperfect information that are required to build truly remarkable products. So one challenge I would like to highlight here is the potentially low quality of data. So when we use data, make sure that the data is accurate and of high quality because inaccurate data can lead to misthalted measurements which subsequently lead to sub optimal product decisions, keeping those are mistakes and challenges in mind. Lets discuss the factors required in order for us to effectively run a data driven product discovery. The foundational factors is to have clear goals and metrics in mind. Basically, you need a sharp vision for your product and define success metric based on that vision. This subsequently will enable you to select the appropriate data, point to measures and to check the progress. The second factor that I would like to highlight here is actually to ensure that you had the right tool sets in order to transform the raw data that you collect into valuable insights. They could be like bi platforms, could be user research tools or a B testing framework. Not only that you have the right data and valuable insights attend, you also need to ensure that you are promoting a data driven culture so that data is valued not only within your product team, but across the organization. Factor number four is about experimentation. The beauty of data is that it enables you to quantify the impact of the experimentation, validating your assumptions which later optimize product decision during the product discovery process. Last but not least is about the data hygiene. A robust data collection and validation process will ensure the accuracy and the consistency of the raw data and subsequently the high quality of your insights during the product discovery process. And this brings me to the end of my presentation today. Thank you very much for your attention and if you have any questions, please share with me over the email. Thank you and have a nice day.