Hi everyone, my name is Matteo Gabrielli. I am a solution architect at Amazon Web Services and today I want to show you how you can empower your business use cases by getting insights from text. So I will bring you through the need for automating document processing. We will have a look at the challenges and how AWS can help mitigate these challenges and then we will have a quick look at a demo that is an AWS solution called document understanding solution. So what is the current state of documents? Well, for many industry, documents are the primary business tool and we can think about documents as email, internal documentation, it could be menus, invoices, technical documentation. And all of this data is in different data stores across the business, right. And is rarely decreasing as number of documents, right. This number is usually growing and growing year over year. Garner research claimed that the organization worldwide record a 25% growth in usage of paper each year. So I don't want to focus that much on that AWS, all the challenges that this can bring, right? So how to extract information at scale and how to avoid to be prone to errors in the processes, right? So let me recap, what are the challenges in processing documents. So first of all, if you are doing this manually, probably it's really time consuming, it could be error prone and most of the time expensive. It doesn't really scale well. And most of the time if you are doing this, probably we are using some rule based system and those are not really flexible. Most of the time, if we want to extract information and we want to leverage machine learning, probably we need to have a set of skills in order to do that. And also maybe data that is labeled many times. Also we want to incorporate in this process the human in order to verify and be part of this workflow. So how AWS can help? Well, let's focus on this use case of understanding documents. So what is the first action that we have to do? Well, we are given a document and we want to extract information from there. So extract the text that is in documents. Once we have this row text, we want to make sense out of this data and we want to loop to add a human in this loop to verify that the information extracted and the sense we are making out of this data makes sense. So who is going to help in implement this workflow? It will be AWS EI services. And let's put EI services in a context. So the context in this case is the AWS machine learning stack. As you can see, there are three different layers. At the bottom you can see the machine learning frameworks and infrastructure layer. This one is intended for expert practitioner, right? You can find in there the more common framework, machine learning frameworks, Tensorflow, Pytorch, Apache, Mxnet, the deep learning EMI and the containers gpus. Then in the middle layer, those are the machine learning services that make it easy for machine learning developers and data scientists to build, train and deploy machine learning models. And this is happening. Leveraging Amazon Sagemaker and its features. What we are going to concentrate on today and is the last layer, the top one. And those are, those are these AI services allow all developers to use pretrained model to add intelligence to any application without machine learning expertise. So what we are going to have a closer look today at it will be comprehend textract Kendra and comprehend medical. Then we will also talk about Amazon augmented EI. But let's start with the first Amazon textract. So Amazon texttract is a service that extracts text form and table from document. It could be PDF or images. What are the advantages? Well that I have an intelligent OCR tool that is able to extract all of this information starting from my input and I don't have to have any machine learning experience in order to do that. I can just invoke text track through an API and this means that I can just build my workflow. So for example, given an input, use text track, extract the information and insert this information inside a database. And this eliminates the manual effort of doing this and of course can lower the document processing cost. Comprehend is the service that will help us in discovering insights and relationship in text. And as you can see the input, it could be text in general. So email chat, social phone cases that are transcribed and we want to apply comprehend on this text in order to get information as what are the entities that comprehend recognize. So are there some name of people? Are there some name of organization? So what are the key phrases? Are there some personal identifiable information? What is the sentiment of the text you are giving me? Is positive, is neutral, is negative. I can use comprehend to train and to do document classification, topic recognition, detect what is the language, syntax and also events. Where is Amazon comprehend used? Well there are several use cases but for example some of that could be intelligent document processing. That is the one we are looking at today. But there's also customer sentiment analysis contact center call analysis because maybe we want to recognize what is the sentiment during these cases or maybe what is the topics they are talking about. And also in PII detection. We will see this later in the demo. There's another version, let's call it like that of Amazon comprehend and this is Amazon comprehend medical that is based to be specialized in the medical text. So you can for example detect entities as medication, medical condition, test, treatment and procedure anatomy in the text. So body parts for example, you can extract also the relation because there will be a relation for example between a medication and a dosage and entity traits. So all of these complex way of recognizing these entities or relationship between inside the text is abstracted to you through API calls. So you can just give your input text and decide what you want to get out from there. Amazon Kendra is an intelligent search service powered by machine learning. In this case, the idea is to do some intelligent search, right? So we want to move from this lexical so using keywords to intelligent search which is searching for information by meaning this use natural language understanding and other machine learning model to take advantage of the rich context in unstructured content, the one that we are providing and get much more accurate search results. That what we can do with a lexical search, let's say later on we will also see how this compare this lexical search and intelligent search in the demo that I will show you. The last leading actor that I want you to present is Amazon augmented AI. That is it allow us to easily implement human review of machine learning prediction. How does it work? So we want to incorporate our reviews right inside our machine learning workflow. So we can do that with Amazon augmented AI. And this allow us to reduce the time in order to build all of this. Because for example, imagine if you have to do some reviews, sometimes you have to expose an interface for the user to select yes, this is correct or no, this is not correct. So Amazon augmented UI is providing you some pre built uis. You can define what kind of workflows workforce you want to dedicate in order to run these reviews. It could be a private workflow for first. Maybe you want to use a vendor that is trained on how to review the data, the prediction, or maybe you want to leverage the public workforce. This is for example the case as Amazon mechanical Turk, right? And yeah, you can integrate it with your custom machine learning model and you can use prebuilt algorithms to increase accuracy. So for example, how does it work? We can think about the client application that is sending the data and our machine learning model making predictions. Well, what can happen is that our machine learning model can return a high confidence prediction. And so we are good probably with that because we know that it's confident in that. But what if the confidence is low? Maybe we want to send it to our human review hands. In this case is where the reviewer consolidated using Amazon augmented AI and we store these results, for example in a s free bucket hands. From there we can come back to our client application. Why not? Maybe we can use the data that at step six we have in order to improve our machine learning model. Right. As an example, I wanted to bring you an integration example between texttract and Amazon augmented AI. So using textract but also then incorporate some human reviews and this could happen, for example for regulatory requirements, but also when we have to take sensitive decision and we want to decide to have a review. Or maybe it's because textract is doing its best, but probably we really want to double check because there are hard to read documents and so maybe we have to incorporate some reviews and well, of course the goal is to human to review and to provide an oversight. Let me bring you an example of what condition can we define using Amazon augmented AI? We can play with the confidence score. For example, we can say if the score is less than 90%, send it to review. But we can also say, for example, trigger a human review if you don't find a field, for example, if you are not finding the name, or if you are not finding the account number inside the document. Or we can send just a random sample of prediction to Amazon augmented Di. So review for a 10% sample of all documents. Why not? Okay, now that we have a better understanding of what are the services, the EI services that will be involved or that can be used, you can think about these EI services and the AWS services in general as Lego blocks that you are trying to put together in order to create your solution. And in this case, what I wanted to present you is the document understanding solution. And as you can see, these four components are going to work together and act. So Amazon comprehend, medical Amazon extract kendra and comprehend. So then when we get some documents in input, they are going to extract, understand information and also index this data in order to perform OCR form extraction, intelligent search table extraction, entity detection, medical entity detection and reduction controls. So let's go and see how this looks like in a demo. This is the document understanding demo solution that I have deployed inside my AWS account. Later on I will tell you more about how and where you can find this cloudformation template just to launch and to deploy it inside your AWS account. So as you can see, there's three different main thing we can do with the document understanding solution. One is the discovery. We can work with compliance so redact information and integrate this and do some workflow automation. So now we are in the list of documents. The documents that you see here are example documents, but you can of course bring your own documents in there or remove or explore inside these documents or just search by keywords. What I wanted to show you is for example going into a document and try to find if there's for example some difficult elements to extract as for example this table, right. What I can do is just preview this document hands, maybe select some part of it to extract. And as you can see is detecting the text that I'm highlighting or I can just ask to search for a keyword like health. And what I can also do is to convert all of this document into row text or maybe as we have a table in there asking to identify the table and is correctly identifying the table, one AWS, four rows and I can download this table as a CSV. Another really interesting feature. So in this case, who is doing this job is textract, doing the row text, extracting the row text or key value pairs or the table. What comprehend can do is detecting entities. So for example on this page, page five, I can say detect organization and AWS, you can see it's going to highlight what are the organization as for example this or also the American Diabetes Association. I can also, since I see a lot of numbers, ask him to detect quantities and is able to detect. Another feature that I wanted to show you is the comprehend medical one. So as you can see, when I go to medical entities this time, I'm able to also identify medical conditions, test treatment procedure, protected health information. So let's have a look, for example, a medical condition and as you can see, diabetes, prediabetes, pregnant, those are all medical condition that comprehend medical is able to identify. Another thing that I wanted to highlight is the possibility of for compliance reason to redact this information. For example, let's think if for example we want to redact all of the medical condition, I can just click here on reduct that is going to remove all of these information and then I can download my redacted document or clear the reduction. Now there are several use cases. One of these could be personally identifiable information that we want to remove easily from a document. Well, we can just redact this information automatically. Let's see how starting for example a search between documents at the moment. So if I'm here and I'm writing diabetes. Okay, let's try now to search through documents the information I'm looking at. So what about if I'm searching diabetes. As you can see there's three different tabs. The first one is the result that I get from a keyword search and those are provided leveraging elasticsearch service. Then I can get my results. The one that Kendra, the service we were looking at before is providing me. And of course in this case we are talking about a semantic search result and there's also the possibility of comparing these two results. As you can see, Kendra and I really suggest to look in the product page in the feature section at all the capabilities that scandra has. For example is highlighting the frequently asked question. So is providing me the information that I need or maybe documents and I can also provide a feedback to these results, right, if they were relevant or not to me. Okay, let's get back to the presentation for the last information that I want to share with you. Okay, so some details about the document understanding solution. So as we saw, the document understanding solution is also really nice to demonstrate the value of intelligent search. I really suggest you to do some tries also leveraging this comparison and seeing how Kendra is and works in order to provide the results the most relevant resources. Well, but maybe you are wondering how is actually all of this assembled together. Well also on the AWS solution page you can find this architectural diagram and as you can see there's several services involved. Some of that were not mentioned today, but I was talking about extract, comprehend, comprehend, medical kendra. But of course the data is also stored inside s free. We have some lambda doing some operation so executing some code. We are storing the data inside dynamodb. There is elasticsearch as well in order to provide a comparison with Kendra and the application that we are going to deploy when we deploy the document understanding solution is leveraging cognito for the authentication and cloudfront for the distribution. So what is also really nice about having and starting exploring the services from AWS solution is that for example, the document understanding follows the well architected framework. A lot of principles, hands, a lot of best practices altogether and you can use it starting from now to process hundreds of thousands of documents. The code is open source so you can find the solution and also tweak it and maybe adapt it to your specific use case. Hands is built using popular technology CDK react in order for the front end and CDK to define the infrastructure. If you are wondering how to get started on GitHub, you will find the code for the solution. There are blogs talking about that and also the page inside the AWS solution you can download or directly launch through a cloud formation one click cloud formation template deploy inside your account and if you need any help or you want to further explore, feel free to reach out to AWS these services here just a list of compliance let's say certificate programs that these services fulfill and just to have an idea that these services are used not only for proof of concept but also in productions environment to fulfill all the business use case you need. Thank you so much and if you want to explore more about machine learning on AWS, just click on your URL bar ML AWS and thank you so much for spending time with me. I am Matteo Gabrielli and please if you have any questions feel free to reach out.