Hi, everyone. I'm Ankit Virmani. I am an ethical AI and data engineering specialist. today I'll be talking about the backend AI solution and what goes into making it. the backend AI part for a very exciting solution that all three of us will be presenting, today. And this solution will focus on how we can use the marriage between artificial intelligence, software development, and human computer interaction principles to help EV1A, extraordinary ability, green card, achievers, potential achievers, and sorry. achieve their dream by learning through this chat bot over to you, Tina. Hi everyone. I'm Tina Gada. I'm a senior UX designer for Vanguard. and today I will be presenting about how a user experience and design works in a chat bot for EB1A, bot, as well as I will be talking about how the human psychic works in terms of, response and the process. Reply for any chatbot experiences. Also how development cycle works and integration of AI with human computer interaction within the mobile device. So I will be talking more details about it over to you. Namaste. Hello, everyone. I'm Namashree Chandrana. I am a mobile application development expert focusing on building applications, which, enhances user performance. And there'll be, I will be discussing about how did we integrate our client and backend to talk with LLM models to get responses, smart responses, from, The rag model. and, yeah, I'll also be focusing on just telling tools that we used for, our client side and backend side as well. Thanks so much. Tina, do you mind putting the slides? Sure. All three of us are very excited to be here and the solution that we are going to talk about today called the intelligent conversational system for personalized EB1A guidance and service discovery. Why we came through this solution was because, based on the market survey we did, it is becoming increasingly difficult to get concise, consolidated, and, accurate information about EV1A, which is also called the Extraordinary Ability Green Card, as information is scattered online. So we thought, how about we bring together Manifested in chat bot. And that's why it's called an intelligent conversational system. It's intelligent because it brings the intelligence of talking. A human, it's conversational because it's a bot and it's a system because it's a conglomeration of UI, AI, and, software development. So we'll be talking about all those three components in their capacity throughout this, presentation. Can you go to the next slide, Tina? So I think we've already done the speaker introduction, so we can skip that. We'll talk a little bit about the background, then how we had to implement the Retrieval Augmented Generation from an AI standpoint. And then Namaste, we'll talk about the mobile application with AI user experience. Tina will shed some light on how the user experience integrates with everything. We'll talk about the final product from challenges that we see while bringing all these three components together and what we are thinking about from the future aspect. So as I was talking about the background before. Ev1a expert bot was created to simplify the complications that Ev1a has been in state as of today. And some of those complications, are stemmed from the fact that there is so much information online, some of it accurate, some of it not. And most of it is the time investment that needs to be made to find a single correct source of truth So we wanted to cut down the cycles of time That are required for someone to get their queries answered and we thought what if we take All the correct data all the nice good quality data that's available All the wisdom from someone who's really knowledgeable in this space, bring that together in a bot and then have that manifest as an intelligent conversational product or even, aspirin. we recognize need of something accessible, which is something easy to use. Personalized guidance, because being extraordinary is about being different from others, being different from others, being, being unique. And with uniqueness comes the need of having a personalized solution, a personalized guidance, easy to use. We did not want users to invest time in figuring out what to do, rather than get the, get answers to the question, and a 24 7 tool available for talent worldwide. So those were the core tenets of why we designed this board and what we designed this board on. And as I was mentioning before, this bot is a unique combination of three schools of thought, Artifact and Dungeon. mobile application and, and, user experience. Let's talk about the artificial intelligence part. usually, large language models operate on a huge corpus of data, that they're trained on. And, they might or might not have all the information that while we were designing the B1A conversational system, this was manifested as hallucination. And based on the testing that we did a lot of times, then we would ask the question, we would promptly understand that the answers that we got weren't accurate. to alleviate that concern that we had during the initial development stages, we understood that we will have to ground the model using a technology called Retrieval Augmented Generation or a methodology rather to call, that's called Retrieval Augmented Generation. What it does is it retrieves the relevant information from the document we feed into the system, augments the prompt with that retrieved information, And then goes to the line language model with the prompt and the retrieved information to get the accurate answer. So the important part here is the model is not using its own intelligence, but it's using the intelligence from the retrieved information that we have given to the model using a document. Now we know, or now we can expect the kind of information we are going to get. So this is called grounding, where we are grounding the model with the information. And this meant that all three of us had to scout for a lot of good accurate documentation about various aspects of EV1A and feed that into the model as document. whatever I just said is presented on the slide in form of these four boxes. User issues a query, Knowledge Retrieval System uses that query to fetch accurate document, information from accurate document that we've already fed before. And then that fetched information augments the user input. Now you have the prompt plus that information. And that goes to the LLM model. And the model uses the augmented information to give the answer. So this was a central basis for the backend. Information that the bot would present on the front end using software, development. Over to the next slide. So this is the end to end design for the mobile application, that number three will walk us through. Which shows you the interface between user, backend, and the middleware. Namaste, over to you. Thank you so much, Ankit. so yeah, for, for this slide I will be discussing, specifically the primary feature of the application, which is getting back the response, for a user query. I will break this discussion into two parts. First, the client side and the server side. For the client side, we have currently just launched the application for iOS users. So the app is natively developed using Swift, SwiftUI, and follows a TCA architecture. The core feature of the app is the chatbot, which addresses the primary issue that we identified before. This is where the user interacts with the chatbot. It sends a request to our backend, and then our backend is responsible to fetch the Correct response for us and get it back to the client. we will be discussing more about the different screens that we have on the client side on our next slide. but I'll just move on to the server side for now. The server is built using node JS with JavaScript as a programming language. And it handles key functions like user authentication, listening to the request from the client, and also fetching the responses. So just as a user journeys upon receiving a request from the client side, the server performs prompt engineering. So this is to ensure that the response that we get from a RAG model is in the right format that we want. to make this request, we, the back end request it to the LLM model for processing, and the back end continuously monitors these requests because it takes some time to fetch the response back as LLM model takes time to, process the query, and once that is ready, we get it back on our back end, again, clean it up for the client side and then send it back to the client. And to do the right LLM model, we just experimented with different models that were available and went ahead with one of the popular ones that is available. And this was purely based on the ease of integration and the responses and the accuracy. now I'll just pass on to Tina to talk about the user experience of our app. Thank you so much, Namaste. from the user experience standpoint, I think one of the key element that I was primarily focused on is taking the user interviews, making sure that the prospect E B 1 A client or applicant who is looking for, Information online, or it could be a traditional way where they are reaching out to our mentors or professional lawyers for getting all the information. How would we, how would they want to process this information and what are the key, responses that they are looking for? So that was the first initial task where I was trying to gather as many information as possible. The, another thing which was playing an advantageous. We already have a very good tool such as chat GPT and custom, GPT available and their mobile application. So one of the very key feature which helped me was how the UI elements are working, what are the things that they are doing as a part of the competitive analysis, which could be leveraged. So that certain things, certain elements that they are doing, we are trying to leverage those elements as a part of the application design and based on the close collaboration with the development, such as that was we and understanding how the request response are working, what is the response time that we are getting and how would we, Showcase that response time in our UI pattern. That was something which was very important. but, looking at the screen, we have a very simple flow where users, if they are coming to the application for the first time, they would be seeing and they have to register to the app and use the sign up feature. But once they are signed up and they come to the app, they have to log in the. Screen, log in and use the application. And the first time experiences since user would be seeing the dashboard, where we are providing them the questions such as what are the most big question or what are the questions that we have known that users are asking based out of our research. And we have populated those questions as a part of our first user experience screen. And upon clicking on any of the given screen, given questions or entering certain question in a certain way, this responses is very accurately provided where we are giving a summarization of each and every responses, and then bulleted point of the sub-task or the action item that they might be, able to do it and follow the certain steps. Over here, there were a couple of things that we had focused, is brand guidelines since everything was from the scratch. So the brand guideline and visual hierarchy was completely managed by, me in a way where we are building the entire design system, the coding, which was done in a swift design because of which it was easier so that all the guidelines which we have followed is iOS. These and ensuring the consistency across differently and, making sure that we are focusing on accessibility and inclusivity, with respect to the colors that we have choose and making sure that each and every, contrast guidelines has been followed to make this mobile application accessible, user friendly, as well as preparing them to have a very good friendly experience at the past response time. What do you get? Thanks. Can you go to the next slide, please? Yeah, so this slide elucidates some of the challenges that we face. accuracy of the model, as I was mentioning before, large language models are trained on a corpus of data and it is expected for them, to hallucinate. So we had to use retrieval augmented generation to avoid some of the hallucinated responses that we were getting. And it was imperative for us to do so because this board endeavors to provide accurate information about each one. Namaste, do you want to take up the rest of the challenge regarding the software development? Yes, definitely. I will cover the faster response request response and the issues with integration of LLM model and the application. So I'm going to talk about this both together. One of the primary issues that we faced was the response time, and also, the requests that we sent to our model were usually airing out if you're just trying to fetch the request in one go. And this was basically because, the APIs had some token limits set. Per minute, for the current here, or, that we were on for that model, therefore, report repeated requests to the model often surpass the token limit, hence airing out our requests. Therefore, to solve this issue, we added an exponential backoff to our backend, to request to our LL model. And that actually helped us to decrease the response time to around five to seven seconds, which was. 10 to 15 seconds before. this also helped reduce the error rate because it, it solved the challenge where it, allowed us to not reach the token limit or minute, because of the exponential back off. although we are still not at a two or three second, which is ideal, for, users, to get, Get a response, but that's something for future that we want to cover. but for now we still were able to decrease it almost 10 seconds from what we were initially, over to you, Tina, for the rest of the challenges. There are a couple of things that we wanted to focus on is ensuring the consistency across the platform. So currently, we only have iOS device and in future, we would be planning to have Android as well as the web application where it's more accessible across different platform based on the user's need. So that's one thing, which, is challenging because every devices was the be and, to maintain that consistency, something that we are. Figuring out, handling the user feedback and iteration. So in the current functionality, we do not have any way where users can provide their, detail of feedback about how the interaction is happening. What are the things that they can like possibly follow? And that's something, which is limiting of another application. And that's what we want to focus on. And. Balancing the aesthetic and the functionality part of it. So there are three key challenges. That is, what we are working towards. Over to you, Namasthi. yeah, for future work, we are currently focusing on just personalized guidance. That's something we already provide with our AI model, but just more towards, any action items that we could auto generate for our users. That is something that we are looking for, that would help in their EV1A journey. obviously the implementation, that's something that Tina already talked about. Implementation of Android and web applications currently available just on iOS. having a voiceover functionality as well, this is, To reduce the typing time and also improve accessibility of the application as well. and yeah, some reminders and notifications of to do's that would be generated out of personal guidance, feature, and also designing for global audiences. Currently, the application is just released to us, audiences and it's not, unable for like global, audiences or something that is again, We're targeting for because any money, like any individual can apply for EB one, it, it could be even if they're outside of United States. So that's something also we are looking to go blow global in the future. Yeah, that's about it. I think. Thank you. Thanks. Thanks. I must be. Thanks, Tina. Thank you. I hope all of you enjoyed. The idea, the thought and the work that we have put in into the solution, we deliberately did not go into the specific, the actual NLM module, the tooling, et cetera, because we wanted to keep it very, neutral and objective and wanted to showcase our overall approach on what happens when three different individuals with three different skill sets. Come together to solve a problem that is so prominent and common in today's world using ai using ui using using software development and a common skill that all of us have which is naturally In addition to artificial intelligence to solve this problem. We do hope that you you like that presentation once again Thank you Etc. Please feel free to reach out to us You we would be more than happy to, to walk you through the specific. thanks once again, everyone, and have a good rest of your day. Thank you. Thank you.