Hello and welcome to unlocking LLM potential a journey with APIs. My name is Ricardo Covo and I'm a principal consultant on the app dev and modernization practice at Centralogic. My LinkedIn is my main forum of social media, so please feel free to reach out if you want to connect. Today we're gonna be following this agenda. I'm gonna do first an environment overview. I'm gonna show you what I'm going to be using during the presentation. There's going to be a lot of demos, we're going to come and cover the basics just so we have common terminology and understanding. We're going to follow that with a demo, then we're going to go into the meat of it, which is chat completion with tools. So tools, APIs, functions are tools that you can augment the chat with. And then there is a new feature, at least on the Azure OpenAI, which is the assistance, and I'll cover a little bit, I'll do a small overview, but a lot of the concepts will translate into that. So environment, we're going to be using OpenAI as our AI service. I have some sample web APIs that I'm going to use and then there is a visual studio project that we're going to be working with to develop this project. So let's check that out. So from, from the Azure perspective. So I am using an OpenAI service. Once you go into it, it's gonna ask you to go to the Azure AI studio to actually do your work. So I'm gonna click on that. A brand new browser is going to open. It's kind of like a different tool. So we're going to give it a little second for it to load. And one of the things that you can start looking in here is if I go into resource management keys and endpoints, this is pretty much the service, the endpoint of the service and the keys that I'm going to have to feed my client on the code to actually have access to this functionality on OpenAI Studio. Here's where you have your shad, your completions, the assistance that we're going to, so the different services that are available from OpenAI, but also important is the models, so you can deploy different type of models. So I have deployed only a GPT 35 turbo is more than enough for the demo that we're going to be doing. So this name is important because from the code I will also reference. So that's what you need to know here from the Azure part, from the web API, the sample web API that I created is in a container app. And we're going to simulate creating timesheets for a project or projects. So I've called the timesheets API and here in postman you can see how I can get the different projects, or I could add hours to a project. So getting the project is a simple get. It's taking a little bit because it's probably a cold in the container app. I have it set up that it can go up to zero instances. So it takes a little bit to come up, but you can see here that it returned an array of projects in adding hours to a project involved doing a post with a certain payload and a certain specific URL. So this is where we're going to eventually going to be using. And those are the three things that you, the two things that you need to know. Like in Azure, it's where you have your endpoint, your keys and your deployment. And from postman I've been, this is how I can test the APIs. Let's go now and show you visual studio. So I have an application running that it looks like this. It's like the simplest shot you're ever seeing. And then we're gonna be playing with some code here. It's a blazor application that is already running, but that's what we're going to be using for the demo. All right, so let's continue here. So some basics. So the service is whatever service is providing dialm, you could be, it could have been deployed locally or it could be, in this case I'm going to be using Azure OpenAI, but it could go on the chat on OpenAI itself. And then when you create, to use that service, you kind of create a client on your code and initiate it with some options. The chat is kind of the conversation that is happening. Roles are who is participating on that conversation. It could be the system is kind of like the assistant setup and then whoever is. So the end user is a user, the assistant is that reply coming from the service. And then tool and functions is another type of role that you can have and we will get into. So when you are having a conversation with the chat, you send a request message that includes the role, the content in the history, and then the service is going to send you back a response message and it has the response content and some other indicators, for example, whether you need to call functions. So this is how typically a chat conversation will start. A user will say something to the application and the first thing this application is going to say is, I'm going to set up my system and the system is context that you're giving to this chat conversation. And in this case you're setting up the system to be a helpful assistant. And then what is the actual message that the user is sending is hi there. Now the service is going to respond to you with some answer. In this case. How can I help? Now on your next step, you want to say, hey, I want ideas for a talk on Genai. Well, you have to send all the conversation back into the service because remember the service, if you don't provide them a context, then they're not really, they're not really smart. They respond based on a context that you're providing. So you have to send all the conversation if you want the conversation to evolve in a natural way. All right, so let's look at a demo. And the demo is going to look like this. So the code that we're going to look actually generates to this. So there is a space for the conversation, an input and a submit and a reset. Let's look at the code. I'm going to collapse some of these things. You can see here. The conversation is going to loop through messages and display them. And then down here is where you have your prompt and you either send a prompt or reset the chat right away. I can tell you that reset the chat is basically eliminating all the history of the messages that you have. And that basically means a new conversation. But let's see what happens when the page is started. I'm going to read the endpoint and the key from the configuration file and I'm going to initiate my client. Now. When I send the prompt, I'm going to do some state management here. I'm going to actually display the message. But here is where I say, hey, from my API client, get a chat completions async. Now chat completions is a very specific terminology. If I go back to here, you can say here completion, right? So that's kind of what you're initiating over there. So I'm going to go back and see that. To initiate that, I'm building a track completion option. This is important because this is where you actually set up your assistant for your conversation. You're going to say the deployment name is GPT 35. Remember in azure AI studio I had that deployment name and I'm setting the system message. And here I went a little funny with it and say, you are a helpful assistant that is obsessed with cookies. It's very simple setup, but I'm also here if there is any messages on that conversation, I'm going to actually set them as messages on the options. So I am sending that context back. So that is basically the base use case. So I can say hello there right now. I. Oh, there he goes. How can I assist you today? Are you in need some cookie related? So it's showing the cookie thing right on my face, even though I had no context for it. Why are you talking about cookies? Submit that. And it's just gonna, it's just gonna respond in some, in some ways. I'm just gonna reset that. It's gonna clear it. Clear the chat. All right, let's go back to the, to the presentation. So we introduce the concept of tools, and it basically gives the chat the assistance that you're creating, access to tool like coding, interpreter or knowledge retrieval. But it also is, and more importantly for the context of this presentation, is allowing me to build tools using my function calling in here. What would happen is I'm still going to send a system configuration, the message from the user, but in addition, I'm going to send it some tools and those tools are going to define the functions that I am providing to the assistant. The assistant is smart to know that you need to call some functions. So it's going to say, hey, you need to call functions and it's going to give you an id that you can use for, that you will use for reference. Now you know that you need to call functions. So you do call the functions. So the service is not going to call your APIs or your functions on its own. It's going to tell you to call it, and then you perform the actual function calling and give them a response. So when you respond to it, you're gonna say, hey, I did call this function with this id and this is the response. And then it's, and then it's gonna generate a natural language response. So in this case, you're asking, what projects do I have? So imagine if you're doing your timesheets, you first need to know, hey, what projects are available to me? And then you add hours to it. So that's the first thing that we're going to do is going to ask, what projects do I have? And then it's going to tell me, hey, you need to call a function for that. And I'm going to call the function and I'm going to tell the system what the response was and then I'm gonna, and then it's gonna send me a nice natural language response for it. Right? So let's look at the demo for that, it's very similar to the previous one that I have. You can see the same kind of chat. But in here, when I defining my completion now, I went a little more serious like standard with it. You're a helpful assistant. You will help employees fill their timesheets. But when it's time to define the tools, I'm going to say, hey, there is a gets projects function in a definition kind of looks like this. It has its own type. And then you give it the name of the project, give it a nice description. That's kind of like the prompt that does. It's kind of like the hint that we're giving the assistant of when to use this particular function. So you here will define. In this case, there are no properties that are required for this particular function. So I have configured my chat to know that there is a gets project functions. The rest gonna remain the same. I'll show you this when we call it. The other difference, when I'm processing the chat completely completion, I'm going to look for the finish recent on the response. And if it's two calls, that means that the assistants have determined that you need to make some calls. And then you actually have to go ahead and do that call yourself. So what I'm going to do is put a stop sign around here, and I'm just gonna go and say hello. You can see here the finish reason. If I go into here, hopefully let's do this here. If you go up, sorry, that didn't work very well. But if you go into here, you can say that we have a finish recent stop. That means you have nothing to do. Just continue running and yeah, how can I assist you today? And I'm just gonna ask it, can you show me my project, please? All right, so now if I do the same thing, the finished reason is, hey, you have some tools to call. So he's going to go in here, it's going to do some management of the state, and then you can have multiple tool calls. So I put them on a for each. And this function right here is going to process that call. So I think I'm going to put a stop sign right here now. And I can see here that the function that I need to call, I'm going to say, hey, it is if projects function get project functions. And it is. And here we have successfully called a function. So it's just that right now I have it hard coded just to show you the different stages. When you set up functions, it's up to you how the function gets executed. It could be executed in line like in here, or it could be a DLl call, or it could be some, a web API call, which is what we eventually want to do. So in here, I'm just saying, I'm just simulating that this is going to be my response and I have to send back that tool call id because that's how the assistant is going to reconcile. It's going to say, hey, I asked you to do something, did you do it or not? So when I go back, you say it went back again and this time it was stopped. So it was project blue, green and yellow. So it is exactly what I have here. Right? So that is basically our first iteration in calling a function call. Alright, so let's continue here. So I have changed, so I've been doing this, I paused the recording and doing so it's faster, but I have changed to my branch of API function call. And in here, what I want to show you is that the get projects, it kind of remains the same. There's no change here that is going to do anything. But I did added a second function, a second tool definition. And what this does is add hours to projects. This is the description. And again, if you can graph the description in a way that is very, very descriptive, with little words, that's going to be best because you're not going to use as many tokens as required as otherwise you would. But in here I do have to define parameters and it is an object, it has a project id, a project, a number of hours, a description in an entry date. Oh, and here I want to show you that when I set up my system, I actually gave him context on what that they would, that what the current date is, because I'm going to be saying things like please add 2 hours for projects x for today, and it needs to know what today is. If you don't set this up, the assistant, I think it gives a weird 2022 kind of date. So make sure that you set up the right context for that. And in this, so this is the required fields are hours, description and entry date. So pretty much anything, everything. Project id in truth should be one. Then the chat completion. I am now giving multiple tools for it to be able to determine what to call. And let me show you here what happens when the function name is equal to get project functions. What is going to do is going to go and I just have a very quick get projects implementation. It's just an HTTP client, it's an API wrapper HTTP client. It goes into projects and does a response, but if it is our add hours to project name, it actually goes and sends all the arguments because get projects didn't have arguments. But add hours to projects does have arguments in the function call, which is something that is coming from the chat completion. Actually tried to determine what the arguments were and is good. And I'm just going to send it to my project and what I'm going to do that because these arguments are going to come as adjacent. I'm going to, I created a small add hours model. I'm going to deserialize into that and I'm just going to make the right URL and post it. So get projects is a get and add hours to project is a post. The rest kind of remain the same. So let's see how that works. I'm going to go here and I kind of say web project are available to me. So this is kind of the same that we did before. It's going to say, hey, you need a function call and it's going to go and say yes, you probably need the get projects results. And this is going to go and hit my API. It's going to take a little bit, I guess. And if you see the project results, you can see here that I'm saying Phoenix, Griffin, Kraken is what we saw in the call over here. Phoenix, Griffin and Kraken. And I'm just gonna let it go now and see what happens the next time that it comes. The finished reason is gonna be stopped because there was a function call and I already did it, I sent it back. So now I go here and say, hey, these are the projects that are available for you. Alright, so I'm gonna say something like add 2 hours to Project Phoenix. There you go. Phoenix for today. I'm going to submit it and it's going to say, hey, you have some functions to call. So I'm going to go inside of tool called response message in here is going to say yes, you need to add hours to the project. And then here is something very cool that I want to show. So even though I said add projects to Phoenix because I'm sending all the contacts and it has all the information, it knows that the id for Phoenix is one. He knows that I'm saying 2 hours. I didn't put a description. So it just said working on task and entry date is today. So I'm just gonna close this and let it go. And it's gonna go. Okay, so let me just not stop here. I'll just go. It's gonna come back and the finish reason now is gonna be stopped. So let's see. Hey, 2 hours have been successfully added to pre phoenix for today. Let's, let's do another like add an hour to Kraken for yesterday. Description should be working on weekend. So let's see what, what that, what that does. So response is going to say, you have a tool call. That's fine. I'm just gonna stop here and let's see the arguments. If I view the arguments, project id three. I said project Kraken it is. Id 3 hours, one working on weekend. And I said yesterday. So because the assistant has the context of what day is today, it knows how to calculate yesterday. So that's great. So I'm just gonna go and do that. And it says, an hour has been successfully added to Project Kraken for yesterday. Actually gave me like a nice indication. Now, if I try to repeat this, my service has an error that doesn't allow you to add twice for the same day. So let's see, it's gonna say again, two calls, it's fine. And put this here. So the hour results was error, error that request. I'm gonna send that back to the assistant and see what the message is now. It says, I apologize, but there was an error and could not add that. Now, if I had, if my API have given a more descriptive error, then I could have sent it to the assistant to provide more input. But as it is, it just being very apologetic and putting some nice wording around it. So that's it. That's how you use functions with your LLM and allow you to call your own APIs. I hope this was very, very helpful. When I discovered you could do this. It was thanks to a colleague of mine that created this system for us. And for me, it just opened a world of opportunities. As I said in the intro, we've been building APIs for many years for other systems to integrate with it, but now we can integrate with natural language, with these APIs. And again, I feel that it opens a lot of opportunities. A few lessons learned when I was working on these demos and this talk. APIs and SDKs are changing very, very quickly. So be patient when your SDK and model versions will make a difference. So if there are things that are not working and you're wondering why, just look at that. Optimizing system prompts and function definition is crucial, and also it will save you money. And I am a C sharp guy and I think they've done a nice work with the API. But to be honest, it does seem like Python is a first class citizen on all of this. A lot of the demos sometimes are in Python, so at the very least learn to read Python and or use, or use Shaqpt or copilot to help you with that. Before I go though, I did promise to talk about assistant, so this came into preview when I was preparing for these demos, and finally it's available to me. So if you go into assistance, you can see that it's a little bit like all the setup that we did, but with a UI, I could go here and say, I'm going to open one that I created earlier. You can see that I'll give it a name and it has his own id. So when the client in the client, the new way of doing it would be that you request an instance of your assistant, aid your assistant by providing the assistant id, you create the instructions. This is kind of like the system message that you did, and then you set up the functions right here. You can see here that I have get projects and it's very similar to what I have in the code. I would create one for add hours to projects and then instead of having to send it with every call, that instance of your assistant will have it. So this is something that I think will be the future for this kind of implementation. So it is good to keep an eye on it. And with that, I hope that you have enjoyed this set of demos. And again, if you have any questions, feel free to reach out and I will make myself available, have a great conference and enjoy all the other talks.