Conf42 Chaos Engineering 2022 - Online

How to Win Friends and Influence People with API First

Video size:

Abstract

Ever got into a (virtual) fist fight with your development team? Unclear requirements, repeated rework, and other miscommunications can really tax your working relationships. Learn how an API-first development approach can salvage your working relationships and increase the velocity of your development process. API-first design and development is an industry standard, and in this presentation, we will review tactics for using mock servers in designing, integrating, and sharing your applications.

Summary

  • Senior developer advocate at Postman to talk about winning friends. Postman is an API collaboration platform used by 17 million people. The talk will go all the way from thinking about your API to implementing it. The aim is to show the benefits of different ways to do it.
  • The benefits of API first, practical tools and workflows. We also want to reduce the number of issues that can come up later on in your production app using API first. You might be wondering what this has to do with chaos engineering.
  • What does API first mean to you? It can mean different things. There's defining and designing APIs and schemas before beginning development. Some companies have transitioned from that to API first. It's a workflow that may well be very familiar with you.
  • There's a few companies that are known for the API first approach. Twitter is very famous at the moment for handling everything using apifirst. Chaos engineering is a disciplined approach to identifying failures before they become outages. Different tools and workflows can help you achieve API first within your company.
  • There's three things that I'm going to talk about here, API specifications, mock savers and consumer driven contract testing. The last one is probably going to be one of the most relevant here because it involves testing things against production.
  • Mock servers are basically generating some mock endpoints for all your API endpoints, but they're live, anyone can call them. A good practice is to have your front end team use the mock server while you develop the backend in parallel. Once the back end is ready, you can swap the URL of the mock servers with the one from production.
  • Consumer driven contract testing is what stops your app from breaking at the last minute. Benefits of it include earlier validation, decoupling dependencies, faster growth and freedom from constraints. Is it apifirst for you or is API first for everyone?

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Jamaica make up real time feedback into the behavior of your distributed systems and observing changes exceptions. Errors in real time allows you to not only experiment with confidence, but respond instantly to get things working again. You good morning, good afternoon, or good evening based on where you are. My name is Alemi. I'm a senior developer advocate at Postman. I'm french, but I actually live in London and I'm here today to talk about winning friends. It might not be something that you are looking for, but we kind of see how you can do that with API first within your company. This is not a title to hide marketing stuff. I do work at Postman and I'm going to chat about it on this slide and then just a little bit at the start. But it's going to be like a high view of what is API first, but quick talk there. Postman is an API collaboration platform used by 17 million people, and the good thing is me working there means that I get to talk with a lot of people that are using APIs in different setups, in different sizes, teams, et cetera, et cetera. So that talk is mostly going to be me sharing what I've learned from all of these people that I've talked to. Going back to the talk, the name of the talk is how, how, how, how to win friends and influence first. So the same way we have a lot of users, we do have a lot of people using APIs in general. So we'll go all the way from when you start off thinking about your API, all the way to implementing it, et cetera. And the aim of that talk is to show the benefits of different ways to do it. To do that, I'll go through these five points. First, what does API first mean? I think it's very important to define. So we're all talking about the same thing, the benefits of API first, practical tools and workflows. And that's where I'm just going to jump back to Postman, but also other tools, some industry examples and some key points. You might be wondering what this has to do with chaos engineering. Hopefully this is going to be cleared out very soon when we keep going through the talk. But basically we also want to reduce the number of issues that can come up later on in your production app using API first. And you see that in some follow up slides. So let's start with what does API first mean to you? Well, it can mean different things and usually if we were in the live setup, I would be yelling at you and be like, oh, you, what does it mean? But I'm here in front of my computer talking to my camera. So the good thing is we actually have a survey that we do with Postman and we run it every year. If you're familiar with the state of cows engineering that Kremlin does, we kind of do the same for the state of the API and we do it. I think we have like 28,000 people from the API industry that answer that survey. So it's a good enough representation. And that's what we asked them. We asked them, what do you think API first is? 11% of them were not sure, and that's actually 2% down from the previous year. So that is 2021. The previous year was 2020. So more and more people are aware of it. But even then, in the 89% left, there's still some different ways to see what is API first. So there's defining and designing APIs and schemas before beginning development, developing APIs before developing the applications that are going to integrate with it, or very much like going all the way to the start and defining the business requirements before even going into defining and designing your API. I think to define what is API first, we can go the complete opposite, and the complete opposite of API first, you would think it's API last, but we actually call that code first. It's a workflow that may well be very familiar with you. Start with the core functionality, so that could be your back end, you're a booking service, how do you handle that? And then on top of that, once you have your booking service that's working, put an API, slap an API on top, and then you'll have a web app that's going to come in and plug itself to it. Start coding the API, use your core functionality or booking application, et cetera. But then someone else comes in. Maybe you want an admin app to handle your bookings, or there's a mobile app that comes in and you need to change your core functionality, you need to upgrade your API, add new endpoints, et cetera. Or maybe you have a completely different type of service that is going to plug in, maybe a third party booking, et cetera. And you need an entirely new API. And as you can see, there's many use cases here and it's just going to keep on changing and you'll have to evolve all of the parts of your app, all of your layers, whether it's the core functionality, the API, et cetera, to manage that. So some companies have transitioned from that to API first, and one of them is Etsy. What happened with Etsy is that they had only a website and then they realized they wanted to create mobile apps on top, but they had to rebuild everything. So that's Stephanie Schumer from Etsy. And what she said is all of the code that was built for the website then had to be rebuilt in our API to be used by our iOS and Android apps. So basically recreate different APIs for each of these platforms. And one of the, I saw that tweet a while back now, but one of the way that I think really represents that when you have them disconnected with the code first approach, when you have your back end, your front end, and then you need to connect them together, this is what happens. Two parts that are individually, they are perfect, right? They work fine. I can walk on that pavement from one side to the other, but then when I want to switch from one to the other, there's a disconnect. That's kind of the, if you go with cut Apifirst, what might happen? So now let's have a look at the workflow for API first. In the API, first case, you start with your API. Surprise, the interface. So you think about who's going to use it, your products, your customers, typically your application engineers, they will be the first one to consume it. So again, like from your web app, admin app, mobile API, et cetera. But that paves the way for more platforms and services to connect to it. And because you start with the API, whenever you start thinking about implementing a new functionality on top, well, you'll add it to the API first, and then both parties, whether it's whoever consumes or ever produces the backend, they'll evolve at the same time. So give back cut ApIfirst, you code, and that's a lower level type of implementation of codefirst that we're looking at now. You start with the code, you commit your code to a source repository, people are going to start integrated with it, then you deploy it. It doesn't have deploying to production, it can be deploying to a stage environment, et cetera. And then people can eventually test it, debug it, document it, and there's a few problems with that. And this is how you do not make friends. And this is the problems we've kind of identified with the code. ApIfirst approach bottlenecks, delay contributors and consumers. And the bottleneck is very much because you have this one entity at the start, the code, if you don't add anything to it, all of the other functionalities are blocked and it's a funnel rather than a lifecycle. Yeah, I'm not quite sure what's the right word here, but it's jacko. There's a lot of rework that happens late in the development cycle. You might be building stuff that no one wants, you're writing code and then eventually it goes all the way to deploying it and it was got what people wanted, so maybe it's not going to be used. The end product might not be well documented, and the API is constrained by what you have in the backend. And if someone wants like a new endpoint, you'll have to rework your backend, et cetera. In the API first case, you start with your API and we will see that later. But from just defining your API using an API definition and different types of specification, you can already generate a lot of stuff. So you can generate documentation, you can generate mocks for your front end developers to start heating and kind of see how they could build the front end. And you can generate tests as well. You may as well be able to generate server side code. There's a few tools that allow you to do that out there. That said, my background is in software engineering. I do like to write my own code sometimes, and I do like to know what I'm writing. So some people might not be confident giving the tasks to some code generation thing, but it's there. But once you have that, you generate code or you write your code, and now you have your id, you have the environment where your code lives and you can start integrating with, and then you have your tests that run between whatever you design at first and whatever is in production. And that's a one to one, because you've first thought about how your API was going to be used by consumers and therefore you know what you want to build in the backend. So you might already see a few key differences, and by then maybe you're already API first, probably have tons of friends already, because how do you make friends? By doing earlier validation. So because you start from the beginning of defining what your API is going to do, you get feedback much earlier in the development cycle and your team can adapt to new inputs. If we go all the way to the cost of it, you can know how much resources you're going to have to put against it, et cetera. You're decoupling dependencies. And the point here being there's not that bottleneck anymore because you have your API defined somewhere and you can generate all these other items from it, and then your teams can actually work in parallel developers testers writer just basing their work on the definition. They can do their own thing. Obviously this creates faster growth. The lifecycle is much faster from when you need a new endpoint, for example, to deploying it. Freedom from constraints. If you focus on the API instead of the code, kind of freeze the design that's already baked in there. There's a few companies that are known for the API first approach, but before jumping into that, I'm going to make the connection with chaos engineering. You might know, know they do some chaos engineering stuff and I'm pretty sure they'll be part of that conference. But from a shorty story of chaos engineering on their website, this is what they say. Chaos engineering is preventive medicine. So chaos engineering is a disciplined approach to identifying failures before they become outages. And this would probably more apply to when you're in production. But the principle of API first is the same because you're kind of starting to already think about how it's going to be implemented, who is going to be using it, et cetera. What's the saying in English? Better safe than sorry. That's very much like what's behind API first. And looking at companies that do that, there's a few very well known ones. Twitter is very famous at the moment for handling everything using apifirst. So their new spaces, API using that, et cetera. Another one that is also pretty well known in the chaos engineering realm is Netflix. Actually. I forgot the name of the tool that they created, but the main chaos engineering one, but basically what they've done as well and what allowed them to have that much flexibility, like in production, et cetera, was moving to API first. And this is from 2013, which is nine years from now. I mean, nine years before now, which is a long time ago. But what they said, or what Ben Christensen said, a single team should not become a bottleneck, nor need to have expertise on every client application to create optimized endpoints. Rapid innovations through fast decoupled development cycles across a wide variety of device types, and distributed ownership and expertise across teams should be enabled. And you probably see a lot of similarities between that and the benefits of apifirst. So you have the rapid innovation cycles, you have the wide variety of diva slides that are supported, and then you have the different teams working together. So yeah, that's what happens when you go API first. But let's go back at the different ways we define it. We actually surveyed something else in that survey. We looked how people that are actually thinking of themselves as being API first were defining it. So what came out of that is that most people were defining API first or most of the people that were calling themselves as full on API first because you can be at different stages of that. They were defining it as defining and designing API and schemas before development. And that's the definition we're going to go with now. So let's kind of dig deeper and now look at the different tools and workflows that can help you achieve API first within your company. Again, I've said it before, I work at Postman. I'm going to talk a bit about Postman here. There's other tools out there. There's actually website called OpenAPi tools which lists it's meant to be or we want it to be an exhaustive list. And I say we don't support it. We're not the ones maintaining it. But it's out there. If you're interested, there's tons of tools there that you can go and check out. But main question, you have a monolith somewhere and you're wondering how do you transform a clouded API development workflow? There's three things that I'm going to talk about here, API specifications, and I've mentioned it a bit before, mock savers and consumer driven contract testing. The last one is probably going to be one of the most relevant here because it involves testing things against production. But let's start with API specifications. API specifications have can API specification is the specification you're going to use to define your API. Once you have defined your API, you have an API definition. So there's different ways, there's different description format, there's open API B, two, e, three, there's Raml, graphql, async API. If you're doing async stuff, there's ton of them out there. So then you have the document. So that's the API definition. So that's the description of your specific API. So if you go back to Twitter, for example, a description of the Twitter spaces API, and then you have the documentation. So again, how does an API consumer come in and use your API? Well, you give them an API reference and it's kind of a guide of how do you use that API, what you can do with it, what you would get back from it, et cetera. Again, if you go through that, that already saves you a lot of time later on because from the API definition you can generate tons of elements, tests, documentation, mock server, et cetera, et cetera. So we kind of surveyed people as well to see what type of API specification they were using. And this is a bit of a heavy one. So on the left you can see the type of definition of specification. So swagger, open API, graphql, et cetera. And then by color you have orange, never heard of it. Yellow is aware but don't use it. Can blue is use it and then the dark blue is use it and love it. So JSon schema is at the top there is the most used, is the most loved. It is a specification for JSOn. It's not used to describe the API, but it's still up there and you're probably using JSon if you're doing anything with APIs. And then you can see in order it's open API two, also known as swagger two, open API three, graphql and then async APIs and other type of async or lesser used API specification. This is self reported. So that's when we ask people, and because it's postman surveying people, it might be BSD a bit, but also because we are postman, we have other type of data that we can get just from people that are doing things in the app. This is our finding. So it's kind of similar to what you can see in the previous one, but you can see still a wide use of open API two, then open API three, and then in this small thing, in this small yellow one, this wager one, waddle and raml, which are very minimal usage. So this is a quick video of what you can do with open API. So you have your API definition here, you can add a different version and you have automatic validation. So the same way you would have like a code linter or something like that that warns you of any errors. You have the same thing for your API definition, and you can obviously collaborate on this type of stuff. So if you're within a team, you can tag people and you can tell them oh, come and check, this is the right way to do it. And that's the whole earlier validation that we talked about before. Just as you're defining your API, you can already ask the other teams what they would be wanting to see. Now let's move on to the next tool that you could use. Mock servers. If you're not familiar with mock servers, it's basically generating some mock endpoints for all your API endpoints, but they're live, anyone can call them. And basically instead of hitting your production servers or even a stage server, it's just going to reply with the default response from the API. And you can define different ways based on the parameters that are sent with the query, et cetera. But that's a good way to have your front end team. A good practice is to have your front end team use the mock server, heat it, et cetera, while you develop the backend in parallel. And then once the back end is ready, you can just kind of swap the URL of the mock servers with the one from production or sharing. And it should work because both of them are defined from the same API definition. So let's have a look how you can generate them. So we mentioned you have your API definition, just from your API definition. You can go ahead, say that you want to generate it from the API definition, give it a name and then generate it. So give it another name and then create a mock server. So now this should give you a URL. And as I mentioned, that's how it works in postman. All the tools do that gives you a URL. You can go and start creating that one. So I already had a collection. So my documentation generated from that API definition can go ahead, swap the URL that is hitting my local host with the one of the mock server that I just generated, save it. And I can hit a live endpoint. So I can just send that request and I can see that I have a live endpoint that's sending data back to me. And now I can send that URL to anyone else in my team and they'll be able to use that same endpoint, start creating whatever items they need to create using that. So we use that open API spec to generate the mock server. The testers can start writing tests against it, the writers can dig in, documenting against it, et cetera. The last workflow we'll talk about, and this one might be the most relevant to the chaos engineering folks out there. Consumer driven contract testing and I'm going to quote here the principles of Cos engineering, which some of you may be familiar with. But one of the first paragraph in there is even when all of the individual services in a distributed system are functioning properly, the interactions between the services can cause unpredictable outcomes. Unpredictable outcomes completed by rare but disruptive real world events that affect production environments make this distributed system inherently chaotic. And if this is too long, et cetera. I always have a meme to explain things another way. This is a bit outdated now, but basically consumer driven contract testing is what stops your app from breaking at the last minute because you have these tests that are running against your client, et cetera. Let's jump into it so you can actually understand what I'm talking about what is consumer driven contract testing. So the API provider, in that case, I can be any company, I can be Netflix, it can be Spotify or whatever, I'll give my consumers a contract. And that contract is in our example, the API definition, the consumer. So any of my users or partner, they write tests against it. So you can see on the right here, I tell them this is what my API is, and they tell me using test, this is how I'm about to use your API. And then the last step is this creates the contract and we can then test against it. So me as the provider, whenever I'm going to make an update to that API definition, I'm going to run tests from my consumers in my pipeline. And when I say consumers, this API consumer can be internal, it can be like a team within my company. It can be an external team, it can be any external users. But the point being, if I see that something's break, well, I can stop the deployment and avoid anything breaking in production. And that goes back to the preventive medicine thing that we talked earlier on. So the way we do it at Postman, each squad, so squad is in charge of different parts of the app or wider services, has a blueprint collection based on the API definition. From that we generate mock servers to have parallel development from different teams. And then we'll also do CDC testing between the different squads. So let's say we have an IM squad, which is in charge of everything, logging in, et cetera, or permissions. They'll have all these other squads relying on what they do, and they'll get their contract, run it as part of their pipelines to make sure they do not break anything. Now let's have a look at our last point. What are the key points for becoming API first? If I've convinced you that you should be apifirst, if you want to give it a try, or if you're already API first, but you just want to make sure that you tick all the boxes. Let's have a look quickly at how we defined it. There's different meaning. API first development and API first design. And the way we used during that talk was the API first design. So it's the designing and defining my API before. When I think about creating any resources, benefits of it, again, earlier validation, decoupling dependencies, faster growth and freedom from constraints. But one of the things that you may wonder is, is apifirst for you or is API first for everyone? Well, tons of services out there. We love to support multiple platforms and services and devices as well. So having this API first or this API in the middle, defining what all the consumers are going to do is a great way to do it. And there's a ton more cloud usage out there as well. And when you're using different cloud providers, what do you look at when you're going to use the services? You're probably looking at the API. None of these friends are slowing down. I don't think anyone is going to tell me wrong unless everything changed between the recording of that video and when it's being published. But I don't think these friends are slowing down. That said, API first, let's say you're just creating an app for your personal usage, et cetera. It might not be worth the trouble, but as soon as you have a few different users, devices, et cetera, then you probably want to go for API first. I'll leave you with two more quotes. Developers are used to code first, but there are definitely benefits to moving to that. And that's a quote from Andrew, whose last name I do not know, who said API first in organization means that we can think of what is possible rather than what is not possible. When it comes to providing the best features and functionality for our product and services to our members, we no longer have the difficulties of customizations or integrations with systems and applications. So that's one. And the other one, which is a much simpler one coming from Kinlane, the chief evangelist, the postman. It's going to be hard if you're planning to turn to API first. That's a big journey, but it's a good one. The benefits are incredible at the end, but especially if you've been code for us this whole time, it is going to be hard, but there's tons of tools out there to help you. There's tons of companies. I'm here if you have any questions around Apifirst in general. But as we see every time we do this survey, we see the usage of Apifirst growing, we see the knowledge of Apifirst growing. So I don't think it's going anywhere. The sooner you start, the better place you'll be and the more friends you'll make. With that said, if you want to learn more about API first, we have a website called API first world that you can go to and check it. It's a quick novel that kind of goes through very high level of what is API first or what is an Apifirst world. It's very light. Do not expect any very technical details. It's very much a comic, which actually we can find it here. Just going to try to show you, literally just a comic with someone that talks to you. But yeah, with that said, thank you for watching. I'm available on most networks. If you find me on LinkedIn on twitter. So twitter.com slash alemi, LinkedIn, alemi, etc. Find me. Ask me any questions if you have any around API first or just API in general and have a good.
...

Arlemi Turpault

Senior Developer Advocate @ Postman

Arlemi Turpault's LinkedIn account Arlemi Turpault's twitter account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways