Hello, a warm welcome at Con 42 Incident Management 2024. Today, I will be speaking about using generative AI to tackle API sprawl in enterprises. But before that, a brief background about myself. I'm Anirudh Karandikar, Principal Software Engineer at Fortune 500 Company. I'm a full stack engineer with a niche skill set in API management. As part of my work profile, I deal with the entire API lifecycle from ideation to governance. The observations, ideas expressed here are strictly my own. Now let's take a look at the agenda. First, we'll discuss about some history to build a context. Then we'll talk about the problem statement, which is API sprawl. Then we'll take a look at the proposed construct to deal this problem statement. Then we'll take a deeper look at the governance model. Then we'll explore the API discovery tool. Then we'll talk about API documentation and then some strategies to control existing API sprawl. So let's dive in. So roughly about a decade ago, most of the enterprises started their move from monolith to microservices. And for those dealing in APIs, No, what advantages or benefits microservices offer, but just to recap microservices architecture is a software design approach where application is built as a collection of small independent services. Each services focus on specific business function. So this is usually known as single responsibility model, and they offer wide range of benefits, scalability, like each individual microservices can be scaled independently irrespective of others, depending upon demand. Faster development. The modularity leads to faster development, as in different teams can work on different set of microservices. Resilience. This modular structure offers fault tolerance, and even if one service goes down, rest of the services can still serve other business functionalities. Easier maintenance. Again, this modular structure helps to make maintenance easy, and mostly there are different teams managing different set of microservices. Agility, the faster development, scalability, resilience, make microservices agile and the time to market is greatly reduced. Technological diversity. I think this is the key advantage. Unlike monolith, technology is not a limiting factor. We can very well have a set of microservices developed in Java and other set of microservices developed in other language, say Python, coexist in the same deployment architecture. And finally, cost efficiency. I think the whole microservices architecture is built or geared towards resource optimization, thus lowering the infrastructure and operation costs. If we take a look at some key milestones. So between 2011 to 2013, companies like Netflix and Amazon pioneered the adoption of microservices. 2014, 2015, this term. became widely recognized in software community. And then from 2016 to present, with the rise of cloud native technologies, containerization, mostly Docker, and container orchestration tools, read Kubernetes, microservice adoption has grown by leaps and bounds. That brings to our problem statement. With the widespread adoption of REST APIs and microservices, it has led to an explosion in the number of APIs. And I'm talking about tens of thousands of APIs. Many enterprises adopted distributed development as orgs within enterprises moved at different pace. And this setup provided team autonomy and reduced conflicts. So talking about this, like each individual team maintaining their own set of repositories, their own standards, their own guidelines, their own timelines to deploy these services into production. However, this setup created difficulties as enterprises scaled. And following were the main contributing factors. First and foremost of them is duplication. Different teams creating APIs that serve the same function. So at any given point in time, all the teams needed to solve for some common issues. example, location services or address management or user management. And to solve for these, everyone created a flavor of the same solution, thus resulting in duplication. Second problem is harder to exercise security control and major. So here the problem is twofold non standard APIs expose different attack vectors and a large number of APIs expose greater attack surface. So think of this from, from the point of view of horizontal organizations like cybersecurity or security. If they want to enforce governance, it becomes very difficult if the APIs are scattered and non standard. Third are operational inefficiencies. Again, since it's hard to enforce standards, each, each team can have different operational needs. thus creating overheads, operational overheads and leading to maintenance and performance issues and fourth, which is not directly related to the distributed architecture, but often found is inconsistent documentation. So incomplete or outdated API documentation makes it hard to find and reuse existing APIs. So even if the two internal teams were to collaborate and they are using the design, contract driven development. Then, the gap between the documentation and the actual API causes a lot of, a lot of issues. So what's the proposed construct? Like, how can we solve this particular problem? So before we look at the construct, we need to understand why centralization matters. A decentralized approach leads to scattered APIs, inconsistent standards, difficulty in tracking changes or enforcing governance. By moving to a centralized monorepo model, we can have a single repository that maintains all the API contracts, which offers not only transparency, but it also improves visibility. Teams can access, a unified source of API definition. This in turn increases collaboration and accelerates development cycles. So let's take a look at, this is like a blueprint of the proposed construct. There are different parts of the solution and we'll talk about them. So the first and foremost is centralization. And by centralization, I don't mean that, like each individual teams can still retain the implementation in their individual repositories. When centralization of the API definitions or the API contracts. This certainly needs to be centralized so that it becomes a single source of truth for all the APIs within an organization or within an enterprise. The second and the foremost step is defining business rules and standards. A few examples of this are naming nomenclature, like, how do you want to name your APIs, or how do you want to name the parts of your APIs, like base path or the actual path. Most of, what standards do you, does an enterprise want to use? Like most of the APIs today, or most of the companies today use Open API specification. beyond this. Every enterprise can define certain custom standards. For example, in OpenAPI specifications, you can extend the tags. So enterprises today can have categorization to identify their domains. There can be domains like payment, there can be domains like supply chain. So you can create separate tags to classify and further categorize the APIs. The third is programmatic enforcement, and we'll see this in the governance model. But once you define the rules and standards, there needs to be a programmatic enforcement of these rules so that all the API contracts adhere to these standards and there is no bypassing of them. The fourth point is about starter kits. So let's say we have a unified definition, like all the APIs are defined in a certain standard. Okay. Now, if we want to extend this to code, there are libraries today which can work on your API contracts and spit out a skeleton of code. What one team, what the team should do is, Go one level down. Like every API today needs some cross cutting concerns. Be it logging. it needs a standard logging format. It needs standards for observability. It needs standards for security. Your auth and auth z mechanisms. And it needs a standard way of defining errors and exceptions. These are some of the major cross cutting concerns. There can be others. In addition to this, if you're deploying using containers in a Kubernetes environment, you also need images and you can, this is a place to standardize those images. So the advantages of this are again twofold. As an enterprise, you can exercise controls or set standards on how, these crosscutting concerns can be leveraged for it. For instance, if you want to change your logging standards, this would be one place where you change that. Or if you want. To add additional policies, on security. This is the place you do so that everyone using the starter kits will automatically inherit, these standards. And the second point is from development perspective. So as a developer, I don't have to worry about this cross cutting concerns because I know that the deployment unit, generated out of the starter kit will have all these cross cutting concerns. Standardized cross cutting concerns baked into it, and all that I have, I would, I need to care is the functional code. Okay, so once, let's say we have established all of these, the next in line is how do we harness AI on top of this? So I would feed all of this rich data into an LLM. Apart from this API definition, we can also pair it with the actual deployment or lives live services data. Like these are the APIs that are actually deployed. And from production, we can get a rich set of data like on usage patterns, and other, Other observability metrics. So this combined with the source data provides very rich insights. And we will look into a few use cases, but then based on that, we can develop the API discovery to, also, like most of the enterprises today use dashboards. so there are two types of dashboards, one primarily used for troubleshooting and the other are the executive dashboards. Thank you for listening. That provide the qualitative data. So imagine, the A. I. Insights derived from this rich set. They can provide a rich context to the already existing dashboards. Those are the use cases. Now let's talk or double click a bit on the governance model, which we talked earlier again. There are many ways in which governance can be applied. Below is just a sample based on CICD paradigm. So let's say we have defined a centralized repo wherein we are maintaining our API contracts. And if I were to check in a new API contract or, if I were to check in modifications to an existing API contract, first and foremost, my pipeline would check for compliance policies, like whether my API is compliant with all the policies. That have defined. Secondly, it will run a linter on top of it and can generate maybe a score. but basically linter, it will run linter to check for standards. The third is, quality gate. once you have run for standard and compliance policies, you can define a threshold. is this API matching a certain threshold? If not, you can fail and give feedback. To the developers and only if and if it crosses that certain threshold, then you can allow that contract to move ahead in the API lifecycle. Next is the similarity check. So if you remember earlier, we talked about API sprawl and duplication being the number one reason. Contribute into this problem. So how can we prevent it right at the inception stage? So this is where we check when an API contract is being committed. We can do a similarity check using the LLM to see whether it is similar to any of the existing APIs. Now there are multiple criterias to do this similarity check. It can be done on your request response. It can be done on your descriptions. It can be done on the downstream domains that you're calling. But the idea is to check whether, to check whether you really need a net new API or what are what you are trying to achieve can be achieved by extending any of the existing APIs. So this presents a simple idea on how governance, programmatic governance can be enforced so that we have a way of enforcing standards. And also reducing duplication. Now let's take a look about if we were to build an API discovery tool, using AI, how would I do that? Before that, let's take a look at the building blocks. If I, if we were to build this tool, so first and foremost is the LLM. This would be your, chat GPT or cloud or llama or any other model. we basically need a natural language processor. Secondly, we need a vector database. This can be a Pinecone or FAS, but basically something where indexed data can reside and LLMs can do a semantic search on top of it. Third, we need an API documentation parser. There are many libraries available in the market, which can parse the OpenAPI specification or Swagger. And fourth, we would need a web interface. This is basically wherein the users would type in your natural language query and, would receive the API recommendations. So if you had this building blocks in place, These are the high level steps. So first is the data preparation. so like I said, we have our common repo containing all the API contracts and it's a rich data set. It has all the endpoint URLs. It has your request methods. It has responses. It has parameters. It has descriptions. All that is needed is to organize this in a structured format, which is very easy to feed. The most commonly used format is JSON. but there is no such restriction. Second is indexing and context building. The metadata that is prepared in step one needs to be indexed and it needs to be stored as something called as embeddings in a vector database. So in AI terms, embedding represents a real world object. So whatever So think of this if you have some nouns in the real world, those are represented as embeddings in the vector database. Secondly, we also need to build a mapping system that links API functionalities to the natural language queries. And what do I mean by that? So for example, if someone queries or someone asks, how do I get user details? Then, the LLM needs to know which API endpoints to point it to in order to answer this particular query. Transcribed Third is building the query interface. this is basically creating an interface for natural language parsing. And the other part of this is to craft prompts for LLM based on the natural language queries. And the fourth part is the response generation. this is where the relevant information is identified. LLM will generate a detailed response. Now, here, These steps, this step needs to be done in an iterative manner. Like the first response that is generated may not be detailed enough or may not be, may not have all the information, that is needed. So we need to ask for more prompts, ask for more questions, do it in an iterative fashion and enhance that particular response. The other part of it is, like whenever this response is generated, it. It can give snippets from the actual sliders or actual API definitions. And, this is known as contextual augmentation. So if you were to build, or follow all these steps, The simplified flow would look something like that. A user would go to a web interface, would type in a natural language query like, give me user details. This will be converted into a prompt, which would do a semantic search on the indexed data. That's the processing on the vector DB and LLM. And the output is the response, which we could see in the user interface. With all the necessary details and the contextual augmentation going, moving to the next topic about API documentation. So let's say we have at the setup build already. How can we use leverage the setup to do API documentation? The steps are more or less similar. So the first step is data preparation, which we have already done as we have fed our common source. Common repository data. The second is we have to define a document, documentation structure. So let's say if we want to create a document, then, what are the sections that document will have? It will have an overview. Let's say it will have endpoint details. It will have your input or output parameters. It would have some examples. and then it will have an error handling section, which will describe what exceptions or errors to expect. The third part of this is, Prompt engineering for document generation. similar to what we saw earlier, we will have to build prompts for LLM in order to provide the output needed in the needed for the documentation structure. And the fourth is response generation. The key unlock of this documentation is with automation. Again, here there are two scenarios to consider. One, it could be like a mass generation. You already have a centralized repo or an API documentation for let's say 1000, 1000 APIs. And you want to do, and you want to do a bulk. processing, and develop and generate documentation for all these thousand APIs. And the other use case is if you already have documentation, how can you augment or keep it up to date as and when you make changes to an existing API? Again, there can be several ways, but I have provided an example using CSV pipeline. So first and foremost is again, like if I were to do a check in on one of the API contracts, I would query, that repository for all these, endpoints and gather all the information, in that contract. Secondly, I will create a prompt. this is a programmatic creation of prompt and it's, it would be, pretty easy because the construct for the prompt is pretty much locked in. All I have to change, each time are the endpoint details. The third important step in the pipeline would be compiling a response. So the output that you get from LLM, it may not be formatted, like most of the developer portals that the enterprises have. Would store the documentation in, in, let's say, Markdown or HTML or PDF formats. So you need that output to be styled, let's say if someone is using Markdown. So the third step talks about that, that the needed styling to be applied on the data or on the response that is generated from the LLM. And the fourth step is to, actually integrate This generated document into a developer portal. So if the developer portal is a Git repo, the fourth step would, would raise, an MR to the developer portal. also, this is the step where developers can review the documentation request because the ones that is generated may not be. Perfect or may not have all the detailed information and that would be the stage where developers can provide even more context or they can provide some additional information like links to existing documentation or links to existing sites, so on and so forth. This approach will always ensure that your documentation stays true to the code or to your API contract that you have defined moving ahead. Strategies to address and control existing API sprawl. So above strategies are. Are good if you're starting new right when you're starting new You have all the majors that, that we talked about, you are controlling the API sprawl right at the beginning. But how about the enterprises or companies that already have tens and thousands of APIs? so I think the steps to solve the first step is still the same. Like you, you have to centralize your API definition and establish common standards. And it can be a gradual move. It need not be a big bang. Small organizations, small teams can start moving towards the centralized repo. And the more teams adopt this, the more standardization and better visibility the enterprises gain. The second use case is, or the second important step is identify unused or redundant APIs. So this is where that step, which we discussed earlier, of feeding the live deployment data into the, into the LLM would come in handy. So we can utilize the AI driven insights. to analyze API usage patterns and then identify APIs that are outdated, underutilized, or have multiple versions serving the same function. This will help streamline the API portfolio by either deprecating or phasing out unnecessary APIs. So what I mean by that is In production, if we find that certain APIs are not used at all, or certain versions of existing APIs are not used at all, then the easiest way to reduce API sprawl is to decommission unused or duplicate APIs. The third is Cluster similar APIs for consolidation. Again, earlier we saw an example wherein, as in when I tried to check in an API, it did a similarity check. That was for one API, but we can extend that same logic to the whole set of APIs and then form cohorts of similar API. Once these cohorts are formed, these can be then manually reviewed and we can generate one main API out of them and decommissioned rest others. this will effectively help to control or curb, the duplication among the existing APIs. Fourth is implement custom tagging for better management. so many a times, in the existing production, scenario, we teams or enterprises need to target one particular set of APIs. What do I mean by that? many enterprises can have public facing APIs and internal APIs. Let's say, there is a new security protocol that needs to be applied. Since the risk is greater for the public facing APIs, I want to, I want to apply this to the public facing APIs. That's where I will use the API discovery tool, write a natural language query, that hey, give me a list of all the public facing APIs. And if we already have tags for the segmentation, then you should receive this list in a matter of seconds. The other use case is deployment platforms. Let's say we have multiple deployment platforms and we want to retire one. Migrate all services from one to the other. Then again, the API discovery tool can be used to get this list. The use cases are endless. but the idea here is. Once you have an API construct, once you have an API discovery tool based on the AI construct, you can then slice and dice the API sets to suit your particular need, to suit your particular needs. So effectively to conclude or the TLDR version of this talk is generative AI can help transform the API landscape by enabling intelligent API discovery. Similarity detection and automated documentation. Secondly, developers can prevent the creation of duplicate APIs, effortlessly find existing solution and receive real time feedback or suggestion that can enhance the productivity and accelerate time to market. Third, for production or live systems, integrating AI with API gateway facilitated continuous monitoring of API usage patterns. providing data to optimize, consolidate, or retire API based on actionable insights. Not only this, but if we can go one step beyond, based on API usage patterns, we can even drive resource optimization, further lowering the operational and infrastructure costs. So to, so in essence, right use of AI becomes the catalyst That empowers enterprises to navigate API sprawl, turning complexity into a managed strategic asset. That's it. Thank you Con42 for providing the platform to present my ideas. And thank you for the viewers who have made it thus far. I sincerely hope you found a couple of ideas that might resonate with you and which you can apply in your everyday work. Enjoy the rest of the conference.