Hello and thank you for joining me today at Conf 42, Cloud native. Today I'm going to give a talk on something that I feel very passionately about and I think is absolutely one of the most exciting new trends in cloud native, even though it has been around for longer than, longer than might meet the eye. Initially, my talk is on ephemeral environments, and it's called redefining preproduction, because with these environments, we're are seeing pre production environments catching up to the advancements that have been made in production application deployment. My name is Natalie Lunbeck and I work at Shipyard, and I'm also the primary author of Ephemeralinvironments IO, which is an open source community resource guide. You can go there right now. You can also check out our GitHub open a PR if you have anything you'd like to contribute. Always looking for new stuff there. All right, so let's get started. So I wanted to give a quick overview of what we're talking about today. I wanted to start off with how teams are using ephemeral environments, but most of all, how teams are using static environments, because many of them are stuck there not necessarily by choice, but by habit. I wanted to introduce the concept of ephemeral environments, and then I wanted to kind of go into how ephemeral environments exactly simplify the SDLC software development lifecycle. All right, so this slide, depending on how well you know it, this might be something that gives you a lot of stress and anxiety. And hopefully it doesn't. Hopefully, your environment workflow does not look like this. But if it does, you can very much see why we do not like this. Well, first off, this is not reduced or abstracted. You probably might only have four static environments total for your whole team, which is absolutely crazy. When you think about the volume of PRs that you have opened every day, how are you possibly supposed to test those one by one in this, which you really can't? That's where bottlenecks come in. And then one of the things about this image that gives me the most anxiety is you can see how these environments are actually just coupled together, and they share a lot of dependencies. And if you know how easy it is to break staging, you also know how staging can then in turn take down all your other environments. And you never want to be that person on your team. So fingers crossed, this isn't what you're doing. But if you are doing that, you've probably thought about a more modern solution, because this has been done for a very long time. And I think modern application workflows, like, we've had so many advancements in production, but why is pre productionlike stuck in the past the way that it is? And that's something that I want to talk about, because if you do have a static environments workflow set up, you've probably thought about how you can change that, but you know that it isn't exactly anything trivial. So I wanted to define what exactly is an ephemeral environment. And this is courtesy of ephemeralinvironments IO. It is a short lived, encapsulated deployment of an application, automatically leveraged on every code change through Gitops. Sounds pretty great, right? Let's go a little further into that. They are robust on demand copies of your application for running tests, previewing features, and collaborating async across teams. So this is honestly instantly an improvement, because before, if you had queues of PRs going into staging and maybe you had another testing environment, but you really didn't get to do the type of comprehensive and frequent testing that you would desire. Ephemeral environments would mean that in theory you really could just open a PR, which would trigger a workflow to spin up an ephemeral environment, and it would automatically trigger the rest of your CI workflow that might not even be run till production. So it would be running your end to end tests, your integration tests, and of course unit tests, even though those are a little easier to do. Although end to end tests and integration tests obviously need to interact with your database, so you are going to need some full stack application there. So this is already a game changer, but I think that you know that. And hopefully, again, hopefully this is not your SDLC, but this is what it tends to look like with long lived environments, environments that never turn off. You're developing to the point where you're ready to open a pull request and then queue in one of your colleagues for code review, merge it to main, deploy it to staging, along with everything else that might get deployed to staging in a day, and against all like 15 of those PRS branch that are merged into staging, your end to end tests are going to run. And without doubt, I think it's pretty fair to say that this might break something. This might take down your staging environment, which is not necessarily bad. I think it's always great to uncover bugs. It's always a learning experience, but that's not great, because maybe your staging environment is coupled to your other static environments, so everything's taken down and oh well, nobody else's features have been able to be tested. So that's a pretty major concern. So then you're going to go back, you're going to hope that you're fixing the correct bug, but I guess you won't know for a few days because you'll open that PR, you'll loop in someone again for a code review, merge it to main, deploy it to staging, and really hope that you fixed your bug there. But at that point it's been days and days in between, and you might keep repeating this depending on how complex your fix is. And you really want to see those results which you can't necessarily see on your local environment because of all the dependencies that you might not have access to until later on in the pipeline. And then of course at a certain point it passes and along with a bunch of other things, it's deployed to production, which is great, but that's a pretty slow release process and I don't think that that's at all ideal and it's something that you could be doing a lot faster with. Of course, now that environments have caught up to application infrastructure. So I wanted to just show you a visual of what this looks like when you're using ephemeral environments. And this is really nice because now we can visualize multiple things at once, multiple branch being worked on. This is when you're developing a feature to these point where it's ready to be opened as a pull request. And then you just click open pull request and immediately your CI workflow, probably the same one that you're using much later in your pipeline, gets triggered and it takes that URL from your ephemeral environment, points it to your end to end tests, points your end to end tests to it, and they run against the environment and then that way they'll uncover bugs within 15 minutes or however long they take to run after you've opened that PR. So instead of context switching, you can just kind of sit back, relax, wait for your test to fail and then go right back. And you can take a lot more risks here, get a lot done faster just because you're aware that you have this opportunity to correct them kind of on the spot instead of several days later. So this is really nice, so everyone can merge their PR with confidence, merge it to main, deploy it to staging, and of course your end to end tests will run again on your staging environment. But this time it's really nice because they're running against branches that have already been tested. So anything that they're looking for are just conflicts between the branches being interacting together instead of bugs, and the standalone branches themselves. And then you can deploy, and we tend to see people deploying around 50% faster, 50% more features, which is absolutely great. Why wouldn't you do it this way? So I wanted to introduce to you a project. I mentioned this at the beginning, but this is my open source community site, maintained with a few other excellent contributors. It's called ephemeralinvironments IO. And back in 2021, when I was starting working with shipyard, I was trying to learn about ephemeral environments. This was a new concept to me, and I really realized pretty quickly that there are no online resources on ephemeral environments. So I talked to my team and I talked to some other professionals in the cloud native community, and they gave me some great best practices that ephemeral environments should have in an ideal world. And I wanted to talk to you about them, because if you're building or looking for an ephemeral environment solution, it really should check all these boxes in order for you to get these most value out of such a solution. Wanted to go into the six pillars of things that we deemed most important to ephemeral environments, and the first one of those is the dev workflow. So really in a perfect world, ephemeral environments should integrate into your dev workflow pretty seamlessly with no overhead. I mean, you usually would configure them with your git provider, and then you should just be able to develop normally, press all the buttons you normally do, use the same exact git workflow and they will just pop up. And generally what this looks like is when you open a PR that triggers your CI workflow and your ephemeral environment management platform is listening for changes, and it will spin up ephemeral environments, it will spin up one based on your PR and it will give you a link which you can access it, and then every commit you make thereafter on that PR will also be reflected in the same environment. And this is honestly really great for development, because as you're working on a branch, you can keep making changes and then seeing them live, interacting with them in ways that they would behave in production, which honestly, you don't normally get that until you're actually in production. And this is really great for QA. It simplifies it, it allows QA to happen much earlier on in the pipeline, and then also on an individual per branch basis instead of multiple features being tested in tandem. And then it should be to the point where you have the confidence, where once you merge a branch, you feel like it's tested, it passes all these cases, even your end to end tests are able to be run against it. So you can kind of almost skip that staging step and just deploy it with confidence, which feels really good to do. And then another feature is that ephemeral environments should promote collaboration, especially across teams that don't normally get to work together this easily. So for example, developers working with developers, a nice thing about this is code review can happen much earlier, and it can happen in addition to traditional code review. They can interact with a production like environment and see those changes, and they can contribute to the branch and see their changes spun up in that same environment. So it's honestly an excellent place for collaboration. Also wonderful place for product to work with developers, because product can give developers a plan for a feature, developers can make that happen, share the link and product can interact with it instead of just seeing a screen share or screenshots or just any type of non thorough interaction that wouldn't really give them the idea of how a feature should be becoming. So it's really great because they're able to interact with it as much as they might need in an as realistic environment as possible. And then of course, as we talked about a little earlier, this is much easier for QA as well to collaborate, just because they can check out a branch much earlier on in the process and both user test it and run automated tests, which does bring us to our next point, which is testing. So ephemeral environments, one of their biggest use cases is for testing, and that's both user and automated testing because they are very solid environments for both. The type of testing that you don't normally get to do sometimes until production even. And what makes this so easy is they use your CI CD pipeline and generally what you can do, since they're so easily shareable and accessible, you can just point your CI CD pipeline at the ephemeral environments and it can use that link and that bypass to run the tests against it. And this, I'm going to say this word a lot, but it promotes more user authenticated testing and QA, more testing more often. You're able to do it as soon as a feature is ready and more and more times. And because you're doing it at the feature level and then also on every subsequent merge you are testing, sometimes we see 40 times more often, which sounds fake, but no people have done it, which is really absolutely crazy, but it can never be a bad thing. More and more and more testing is absolutely what you want. It's never a bad thing to catch bugs earlier. And your team is often very thankful because nobody likes to delay a release. And one of the virtues of ephemeral environments is they're on demand. So you should absolutely go all in with this, and you should make them Gitops first. They should really spin up when a branch is ready. I know I keep saying this, but it's absolutely essential to them. There really shouldn't be any manual intervention that prevents people from using them. So if they are Gitops enabled, they should be there on significant code changes, and then they should go away on their own to not incur additional cloud costs. And because of this, there's no time limit, really. You can have async collab, let's say you're working in the morning and you want to collaborate with someone who has later hours than you. You can just spin it down, wait a few hours, and then your coworker can spin it up and work on it then. And nobody's getting blocked because everyone's on their own schedule and you're not worried about incurring additional costs because this is so much cheaper than your current static solution. And it's really nice because you will have more of these. So everyone can have as many environments as they want. You could really even have one per branch and have them just whenever you need them, which is huge virtue. And this is a very important section. It's something that you absolutely can't overlook. But I would like to say, I mean, I'd like to say that because they're isolated, ephemeral environments tend to be more secure. They should be entirely decoupled from any dependencies or other environments. And this isn't just for security reasons, but also just so in case something breaks, they don't take down your entire environment pipeline. But you still should be sure to take a few measures into account. One of those is sanitizing your data. You should remove any PIi just because you shouldn't be testing with it anyway. So not exclusive to ephemeral environments, of course, but still very important. And then you should enable SSO. You really shouldn't be doing any of the authentication logic on your own. That's just one more thing to worry about, one more vulnerability. And when you use an SSO provider, maybe you're using GitHub, maybe Google, then you don't really need to worry about any breaches because that's on them. But you are also able to have introduced granularity and make sure that you're only allowing people from your organization or people who might be stakeholders. And then the last, but certainly not the least, probably one of the most important considerations for ephemeral environments is cost control. And this is really nice because since they are ephemeral by nature, they're short lived, they are really cost efficient by nature, which is not just great for your organization's wallet, but also great for the environment, because you're not using electricity that you don't need to be using. If you're using the cloud in an elastic way, you're really only using the resources that you need. And this takes us into another great benefit, is you don't really tend to encounter the same resource constraints that your team might have had previously. And a big part of this is because when you have just a few static environments going 24/7 that's just a lot of time that you're not using them. So of course you can only afford a few of them to really be cost effective. But if you have environments that only last for a few hours, if even that each, you can really have as many environments as you want at a given time and still come out ahead with your savings. I wanted to thank you very much for joining me today as I talked about ephemeral environments. I certainly hope you found something useful from this presentation and if you have any more questions or would like to talk more or find out how you can contribute to ephemeralinvironments IO, you can email me. Natalie at Shipyard build always happy to talk about this and I thoroughly enjoy giving this presentation. I love talking about ephemeral environments and I think that they're just such a fun and interesting topic, and I'm very glad that they've picked up the pace in recent years and I'm looking forward to see seeing what the environment landscape looks like in a few years. Well, thank you again and I hope you enjoy the rest of comp 42 cloud native. There are some great talks coming up and I know you won't be disappointed. All right, have a wonderful rest of your day. Thank you.