Hi everyone, welcome to my my session on how to absorb your APIs with API gateway and it is plugins. My name is Babur. I'm a developer advocate for Apache API six. You can reach out to me on these following social channels regarding the sessions and feedback. If you have any additional questions, feel free to drop me. We can jump in, call and discuss about the restings how you can API so today we're going to speak about APIs first because the APIs nowadays is one of the important core component on building software, right? And then we will discuss what is API observability. And then I will give a short intro recap how to say about API gateway and how API gateway can be the central point for observation. We're going to also have a look, this kind of section, and also you will learn what's the Apache API six, and then how to use Apache API six API gateway for you to observe your APIs. And we can also dive into some of the API observability three pillars like logging, tracing and metrics, and learn how Apache pay six can provide this kind of observability capability. And then at the end we'll have a small demo on the usage of this Apache API six API gateway plugins for observability. So with Z plan, let's get started. APIs, just three letters, right? By now we are all familiar with this term API acronym for application programming interface. We are living because in very increasingly API centric world, every service used today either uses an API or we can assume that it's API itself, right? And API is becoming very central in building and delivering your services. Also, you know, the success of your services depends on the integrity, availability and performance of your APIs. One of the tactics we usually apply is to retain all three concerns in this slide by continuously monitoring our APIs, right? And for example, we call it like API monitoring, right? Or traditionally this API monitoring, sure. I mean it does help you to stay on top of uptime, security and the performance. However, it is limited because traditional monitoring focuses on tracking noun unknowns. This means you already know what to measure, like request per second or errors per second. And traditional API monitoring also lets you monitor the system health and on the other hand, the performance, but cannot help you to identify and troubleshoot what's exactly causing the issues. On the other hand, observability involves analyzing unknowns versus to now and unknowns. When you imagine of the monitoring as monitoring was about kind of measuring things, but you know how to predict them in advance. And various observability helps you to understand how and why do things happen. Focus on what you cannot see. We call it unknowns. Right? Observability is a cool thing to find out all the unknowns in your APIs. And what is the API observability? So we know observability means identifying unknowns unknowns. But API is all about making your API more observable. Instead of relying on predetermined metrics or monitoring or waiting for the failure to happen, API lets you dive into the unknown unknowns of your APIs by observing how they work internally. And API observability also is ability to understand the system behavior and investigate the interactions between your application's components. It provides for your APIs like tracers, metrics and loggers. And nowadays API observability is already part of every API development as it addresses many problems related to the API consistency, real ability and ability to quickly iterate on new API features. When you design for full stack observability, you're getting everything you need, right? To find the issues catch breaking changes and API observability is not just used to troubleshoot engineering issues, but also it can also help troubleshoot other business and security problems. As you can see in the next slide, each teams needs API solubility for the different purposes. Let's take for engineering, it provides clear definition on how to implement the APIs for the product managers to understand adoption and consumption of your APIs, or the line of businesses need to see the business value that your API bringing in areas such as monetization, customer adoption and new changes opportunities like when you are establishing new integration or new business integrating with other APIs, or the security teams to detect and protect from any API threats. And here the question rises like what is the easiest way to make our API observable? You can ask me. So there are many solutions, right, in our modern era, but one of the simplest quick solution that came into my mind right now is by using API gateway, right? Because API observability is a sort of data collection process, API gateway, you can think of just a framework to manage your applications in the API world. Of course, API gateway is further extended with countless built in plugins to handle other cross cutting concerns. In addition to API observability like consoles like authentication, security, authorization, traffic control and many more. And most of observability platforms like promoters skywalking, open telemetry, and others, they can easily integrate with Apache API six and then you can leverage these connectors to ingest the log data from your API gateways to the further down into the services that you have, and you can drive useful metrics from there and also gain a complete visibility into the usage. Also you can manage the performance on the API gateway side and the security of your APIs in your own environment. I mean, because most of the modern API gateway providers, they are very cloud native, you can use them in every public cloud, or the private cloud, or the hybrid clouds. So what is the Apache API six? It is part of Apache Software foundation, you may already know the world's largest open source foundation, and I believe some of you are already part of this community. And nowadays it's one of the fastest growing top projects of Apache software Foundation, like you can think of Apache, Kafka or Cassandra, Hadoop, Tomcat and so on. Apache API six provides custom plugin development because we have some existing plugins and there is possibility to create a new plugins. You can write custom plugins in programming language you are very familiar with, like Java, Python, Go, Javascript, et cetera. And also there's another mechanism called plugins orchestration where you can see the list of existing plugins on the Apache PS six Ui dashboard and you can easily drag and drop the plugins into the drawing board to combine them, right? And you define some conditions. For example, as you can see on this scenario, like we can start with authentication TWT, and then if authentication successful we can run another plugin by orchestrating and by putting them in a row, and also by collecting this couple of plugins together, you can create your own plugin. Bringing our attention back to the observability. We say the core observability breaks down into three key areas, right? And like structured blocks, metrics and traces. Let's break down each pillar of this API observability and learn how Apache basics plugins can simplify these tasks. Let's have a look. Logging or locks logs are also easy to instrument and very trivial steps for API observability, right? Because each API event logged whenever some kind of API operation is in voic. And also you can collect some logs to debug your system. You can gain this analytic insights into your API activities. Also they can be used to inspect API calls in real time for debugging, auditing and record. Also you can put some time steps for each logs to identify events that happened over time. So how do these logs managed by the API gateway? Like API six, it provides, for example, some logger plugins because the API gateway sits in the middle, right, every request coming into the API gateway and then this request sent over to the back end or upstream services and in the middle means API gateway, able to collect all the logs and several logs logging plugins available as you can see in the picture like HTTP logger, basic logger. We're going to look also quickly today, or some other loggers you can see in the screen like error logger, the Google Cloud logger or kafka logger that collects kafka event logs and sends to the kafka clusters. For example STP logger plugin, as I said, capable of pushing clock data requests to HTTP or HTTPs servers and in a JSON format. Or it can also send these JSON files to the monitoring tools. We will look at these monitoring tools later on or the next. For example, the second pillar of API survey like metrics analyzing metrics like metrics, just numeric representation of data that collected over intervals of time. You can also aggregate these metrics into daily or weekly, maybe monthly frequency, and run the queries against distributed systems like elasticsearch. Or sometimes based on the metrics. You can trigger alerts to take any action later with action groups. Once API metrics are collected, you can track them with metrics tracking tools such as Prometheus. For this, Apache six provides a Prometheus plugins. It's capable of fetching all the API metrics data and you can show these metrics exported by the plugins on Grafana visually at the end in the demo section I included. You will also see how to enable this promatus HTTP logger plugin. And the last one, but not least one, is sort of scrabbit pillar is tracing, or we call it sometimes distributed tracing. It allows you to understand the life of request as it traverses your service networks, allows you to answer questions like what service has this request touched? Or how much latency was introduced by this API request. And for example Zipkin, one of the tracer plugins Apache PSX provides. There are other plugins also if you look at the Apache PSX official website, like skywalking for example, and let's say Zipkin also open source project, it also provides some distributed tracing. You can enable this Apache Zipkin plugins to collect tracing and report them to the Zipkin collector based on the Zipkin API specification. Or you can visualize it on the Zipkin Ui. So with enough suratical knowledge up to now, we can jump into the demo session about using these three plugins, HTTP Logger and Zipkin and Prometheus plugin. Together we apply this plugin for our existing ESP net the core web API with API six we're going to run matt container app with back end. I mean in the back end we have product API. Let's say it has a single endpoint that gets all the product list. You can assume we have this online market that lists products on the main page and this endpoint to get all this product information or metadata. And then we have this database just in memory database we will run also we run API six and eTCD together with Docker compose. And then we can configure this API six upstream and road for the product API. Because upstream and road very critical concepts in API gateway. Upstream means your backend service. Not only one, it can be more backend services, right? And also rod means just set of rules based on which criteria to pass these requests to the responsible upstreams. We can also create this route also in this demo session and then we will enable for these robot HTTP logger prompters and zipkin plugins. All these code samples you can find on my repo API net Docker on GitHub. You can check out the branch API observability branch and you can also see the command line examples. On my case I am using Docker desktop. I can show you here. As you can see I have a couple of containers running along with Docker. Let's say we have API sticks, Prometheus Grafana and the dashboard API six dashboard also up and running. And also I have some other like Zipkin and skywalking UI some of the containers. But it's not important to run all these containers. You can just run for example the one that is in the report. And now if you open the project API Net Docker API project, you can see some folders here, the folders, I mean like the API six config files, API six log files where you can see all the API six provided access logs or error logs. If something went wrong, you can dive into the logs and observe the next. Also I have this product API just ASP net project with a docker file also to run this API. And then I have the docker compose file that runs all these images and creates containers together. And the command line examples here for how to enable all the three the plugins. Okay, now we can start as a first step, we can start with creating upstream for our backend service. As you can see, I have prepared all this Carl command example in the code. For example, Apache API six provides admin API to interact with all these upstreams. I mean to manage all the upstreams or the roles or the services, you can always hit this admin panel and for example to create upstream or to create a rod. In my case I'm creating upstream with the product API as a backend service. And then by default we are using just around robin algorithm. It doesn't matter if you don't put this it will choose around Robin API six. In our case we have single node. That's why we don't have to care about much this algorithm case. And then if you notice also each request we're sending with API key securely to talk to the admin API. So first thing first, let's create the first upstream. I will just copy this common example and then pass to my terminal and press enter. As you can see now API six returns out a successful response. It means everything is good and everything is good and then our upstream is created. So once we have upstream created next we can create a route with HTTP logger plugin enabler in the Apache API six you can define the name for your route and also which HTTP method you would like to put as a matching criteria or the Uri which Uri pass heated and then this request send API stream like kind of rules. And also you can list what kind of plugins you would like to enable for the given URa. In my case it's saying I would like to enable HTTP logger plugin so that each time the request receive it, we log this request and send to this mockwin.org just HTTP log services. Chris Collard the logs will be sent to the log server and as you also see here, I am also showing this upstream id. It means in previous step we create upstream id. We are just now referencing bytes unique id to this upstream. So let's copy this command and pass to the terminal as well to enable HTTP logger plugin. If I bring this example kernel command example and run this comment API six return a successful response and now we can validate if this plugin is working correctly we can send request to our API products endpoint. This is our single endpoint that just simply returns us the product list. Like we have only two single products but we might have more. And now some logs should be recorded and sent to our mock server. Let's test it. If we go to this link and check our logs provided by API six sent to this server, I can open my browser and navigate to mock bin. It means this is just mock server returning us this response. But if you do logs it should show us some kind of logs. Yeah, let me do log maybe. Yes, now you can see some of the logs even a year ago sent. But if you scroll down and see a few seconds ago some other logs sent by API six. And here you can also observe what are the logs sent by API six. All these log values. You can also check on the website what kind of logs usually API six sent to the server, et cetera. Now we demonstrated like simply how HTTP logger plugins works, right? We created upstream and then we created rot. We also map given rot and upstream together and we enabled also one plugin. The next maybe step we can test also the next plugins we have, let's say promatus plugin. This is also enabling Prometheus plugin is very similar to the step with HTTP logger plugin. You need to just define inside the plugins object this an object the plugins Prometheus plugin name. We are just making this Prometheus configuration as empty. But there are some of the attributes for promoters plugin you can also define. You can check them on API six website. Now I can run this curl command to enable the Prometheus plugin. I can just copy this comment, let me take this command and also pass to my terminal if I run and press enter, all good. And now the Prometheus plugin should be enabled. And then if you return back to this example and run, let's say one more time hit endpoint. I mean our API products endpoint. Let me hit one more time. As you can see yeah it's responding. But what about metrics? We should be able to see metrics right after we requested this endpoint. Let's see what kind of metrics API six collecting to get these metrics you can navigate to API six promatusmetrics, but you can change this pass under Prometheus config file. Sorry, this is here below. For example, you can see the metrics pass. You can define whatever metrics pass you would like to with some targets. In my case I have one target in config the API six target. And if I run these metrics to get some metric comments, you can see lots of metrics returned by API six like API six bandwidth, some ETCD modifying indexes and some latency information. And importantly we have this API six HTTP status metric available. That's actually our call when we did this API products call and as you can see it's collecting some HTTP code information and also route number and code IP address like from which code this request sent. Also we can visually see these metrics because by going to this localhost 1990 targets on your service target because we are running also a promoters container. If you go to these targets you can see I have API six up and running this promoted metrics and just local host metrics and you can see by filtering API six HTTP status only metrics. Some of the graphs available with these matching uras how you see from on the promises ui. But you can also see these metrics on Grafana in this easiest way. Most of the developments teams now use the Grafana as a dashboard. Now if I navigate to another local host port where Grafana dashboard is running, I think it's in 3000. Here you can see that the Apache API six the dashboard, whereas some data will be sent from promoters to the Grafana dashboard. But here is empty data because we need to configure by configure some settings on the Apache six configuration file. So next, the last one. Let's try to enable the Zipkin plugins as a tracer and see how Apache Ps six zipkin plugin can collect the trace information. So I can just get this similar configuration plugin configuration for the zipkin. We can define the endpoint for this zipkin API. As you can see I'm giving directly the IP address of my container. In real case you should have the IP address of your server for this metric. The tracers and the same we are using referencing to upstream id. I will copy this carol command and let's pass to the terminal and press enter. Now the Zipkin plugins is enabled and we can test it by maybe sending some kind of API request once again. And then we can go to for example on the browser, open the Zipkin server and see if we have some kind of tracers available by the API six server name. Yeah, if you press run a query in 2 seconds, the query should show the result. And as you can see, there's one API request that we did against API products, endpoint and some here the details, the tracing details. Of course here we have single service, but if you have a couple of services you can see how tracing works properly. So it was the last plugin and for today I would like to show you, but you can now ask the questions like why you are enabling single plugin at a given time instead of using all the plugins together. What if I would like to use three plugins together? Is it possible? Yes, of course it's possible. You can, for example this API six enable multiple plugins together. I have this example also. Let's say you can run multiple plugins for the given role just by defining inside the plugins object all the plugins you would like to enable. For example, if I just copy this command and again this pass to the terminal, we just simply enabled three plugins together like HTTP, logger and tracer. And also the metrics three plugins and also if you navigate to Apache API six official website official website let me bring here on the browser. Here's API six apache.org. You can find some documentation under the plugin session, or you can take a look how to get started. There are some azure observability plugins as you can see divided into by tracer and metrics loggers. Let's say you can find all the logger plugins here and all the metrics plugins like a data doc or the tracers like oftentelemetry, also widely used as a tracer tool. So with that we are close to finish. Let me jump into my slides at the end. As I mentioned earlier, on the top of observability plugins you can use some of the plugins API six provides, like for the traffic management. For example, you can use limit count plugin and feel free to check out other available plugins on the Apache API six official website. Again, thank you a lot to join me today and I hope you get some kind of knowledge about observability and API observability and also how to make API observable by using API gateways like open source Apache API six. So now if you have any questions, feel free to ask. You can leave in the chat.