Conf42 Kube Native 2024 - Online

- premiere 5PM GMT

A deep dive into Perses, the GitOps Native dashboard visualization tool

Abstract

Perses is to Prometheus as what Grafana was for graphite. And even though Grafana has become so much more, Perses is an extremely interesting new entry into the dashboad/visualisation space. This talk will have an up to date demo and some of the unique advantages Perses offers over alternatives.

Summary

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hello, everyone. Today I'll be talking about Perseus. Not just today. You can watch me whenever you want. The GitOps native dashboard visualization tool. Now I think about it. When you're watching it, it's always today. First of all, I want to say congratulations to Perseus. They've been accepted into the CNCF. At the sandbox maturity level, which basically means that it's not just a hobby project from some developers anymore. It's now officially a CNCF project. Then about me this is me. And there I'm Erwin de Keizer. I'm a CNCF landscaper, as I like to call myself, or a developer engineer. And I'm with Fullstack, one of the first KCSPs of the Netherlands. And a big thank you for them for supporting me in my Strange, odd obsession with talking about cloud native projects in a word purple onesie. Also, just to be clear, I have no affiliation with the Perseus project whatsoever. I'm just someone that is enthusiastic about it and likes about it. Likes to talk about these kinds of things. Also, sorry, this is the only time I'll be using this template for the slides, because it takes forever. And, so these are Percy's goals. . Let's summarize them a bit clearer. Also from the get go, I need to say these are goals. They are not achieved milestones. We'll get to that later. So they want to be the standard dashboard visualization tool for Prometheus. They want to be GitOps compatible, which is Perfect, because we all love GitOps, right? And they want to provide different NPM packages, which on the surface sounds a bit odd. Is it a goal to provide NPM packages? Apparently it is, because they want their advances in visualizations to also benefit the entire ecosystem. Even Prometheus, the Prometheus UI could use their visualizations in the future. It's also aiming to be Kubernetes native, which means that they adopt the same data model as Kubernetes. And it should provide native CRDs for, to create visualizations. How awesome would it be if you just deploy your service monitor together with your Percy's dashboard and it gets automatically provisioned in Percy's. And they also want to support a static validation of the dashboard format which is great especially if you combine it with GitOps and CICD. So you know that when you're provisioning your dashboards and you have a pipeline that the dashboard will actually work or at least not break. And they also want to support plugins, which, okay. And now I will show you a live demo. First of all, we have a Docker compose file. that I would like to show you. So we do tfc. As you can see, we're just starting at Prometheus. And at Perseus, we'll do the latest without a version, because YOLO. We do docker compose up. And I'll open Perseus. As you can see, some data persisted. I'll throw it all away to show you how everything is done. First of all, if we go into the admin area, we have global variables. We have global data sources and global secrets, which are basically these things that can be used by all projects. And when you add a project, let's call it ClickOps. Good suggestion. You'll see that we also have variables, data sources, and secrets. So everything can be defined on a global level and on a project level. We can also see the config that it was started with. And we have the devils button right here. We'll Just ignore that exists. So in the kit, in the ClickOps project, we're going to add a Toyota source, because we started Prometheus. Let's call it Prometheus. Make the default, no proxy access, because we can reach Prometheus on the Prometheus hostname from within Perseus, but not from the browser. We'll save it. And we'll try to create a dashboard. I'll just quickly go through things go back later. And here you can see which panel group. So you can have multiple panel groups. You add a name. We have a time series chart, which is basically the default chart everyone uses. And check go routines. Instantly get a visualization. If you go into settings you can see what you have to change. You can add the legends. It's only for meters in this case. There's multiple options here. Can bars line with. I think the defaults are quite decent. It actually looks good by default. So if we add this, we can also make it bigger and then save it. And as you can see, it immediately made the count of coroutines go up. And there's also this button that can instantly make one panel full screen. And if you want, you can also see the JSON that was created for this dashboard. And that's about it for the interface. We'll get back to it later. But there's also a sea light tool. It's like Cube, CTL, cube Kettle Cube Control. I don't know. But then it's for Percys, personally, I think it should have been called Per Kettle 'cause it just sounds fun. But they went with per CLI, I'm not sure if it's per CLI or Berkeley for the sake of this. Presentation, I will call it Berkeley because that's just a lot funnier to pronounce. And here we'll go through these files. So we have some files here. We have a data source. We have a project. First of all, we should create the project. Because everything else depends on the project. And that's why, this one is a bit more clear, it's numbered like this, which makes sure that everything gets applied in the correct order. So when we now do Berkeley apply MD, you see that project, data source dashboard have been created. And if you go back to the homepage, we now also have a Persis 2024 project. And then we can go into the black box monitor. And here we see some extra things that are possible with for the dashboards. So these variables, they are created by reading labels from Prometheus itself. And so we have the gauge charge, we have a single stat which nicely translates seconds into a human readable format. Because if you take a look at this raw data, if I make this a decimal, you would see that it's fine. 5. 96 million seconds, which is not something that I can instantly translate in my mind into a normal duration. But if you make it seconds, it makes a lot more sense. I'll also, I show you the variable. So we created a variable ourselves, which is a list it's created from Prometheus label values. So it uses the default Prometheus. The label name is instance. And we select the series by using the job black box, because I have a black box monitor attached somewhere. You can also allow multiple values and all options, but for now we didn't do that. And then you can see that. The toggle works and automatically gets updated with new values as well. Here you also have the things that you normally see when you use Grafana as well. So you can have the date interval. If you're interested in another interval, you can also select that. You can set that auto refresh. And again, you can see the dashboard itself. We also have other dashboards, which basically amount to the same thing. So are not that interesting. Something I could show you is if I edit this panel group, it's not very descriptive. I call this usage charts and This one can be a bit more like this. And we'll make this a bit bigger. This is where the magic happens. And then we save the chart. And if we now go back to the command line, we can directly get dash. I think we call it per 2024. We see these dashboards that are available and if we do Berkeley, describe dashboard. We updated Wirecard per, sorry, 2024 and we do it in the Wirecard dashboard. We can also see that a div was created. If we check out the div you'll see that the white space has changed quite a lot. We'll just save this one. And for the sake of the demo, we'll do it again. because it's a lot easier to see the changes that were made. If the whitespace doesn't change, we'll save it again. We'll do the same this and then we do a kitch. And now you see that the div is quite small because we actually only changed the width and the alignment and you also correctly see the revisions. So if we commit this, we now have these changes in code. Big question, of course, is how does it stack up against Grafana? I think this is a question of personal preference because customization wise, it's a lot more limited. Grafana right now. It's, there are just many features. I think the most essential features are there. But the thing I really like about Grafana is the explore tab. There's nothing like that right now. Style. I think the default dashboards of Precise look better than the default dashboards of Grafana. That's a matter of personal preference, of course. The ease of use I think it's about the same. I've used Grafana a lot. I think for the past 10 years, that's been the go to dashboarding tool for me. Thank you. So I know my way around it, but I've only recently started using Percy's and it's actually quite easy to learn and understand. The GitObsability is very nice, as you just saw with the Berkeley CLI tool. And the future I don't know. It's bright, I think. It's been accepted into the CNCF. There's a lot going on with the project. There's a lot of potential, but it's also still a bit murky, if you ask me. I don't know what level it will reach in the coming time. I think we also need to talk about dashboards as code. The Percy's project itself really pushes the dashboards as code concept or DAC. And right now they have two solutions for it. Excuse me. You can use Q or the Go SDK. And what is Q anyway? The, it stands for configure, unify, execute. So it allows you to validate, define and use dynamic and text based data. So it's basically a better toml. I don't know. It's like YAML and toml and Python and Go had a four way and this is the very odd baby it is possible to generate Q templates or schemas from Go types. It's what the Percy's project does. So there the Q definitions are always aligned with the actual code. And Q also allows you to have if statements, for loops. So it makes it easier to use some kind of programming language which is Maybe you shouldn't call Q a programming language, but it does have a lot of similarities, more than TML or YAML, and allows you to generate dashboards from there. So if you want a uniform style across all your dashboards, it's quite easy to achieve, especially if you have multiple environments that you're monitoring, like development, acceptance, and production. If the only change is the data source, then you can just use the same dashboards across all those environments. If there's some naming conventions or labels that are different across. environments, you can use Q with a for loop and some kind of if statement to make sure that the same dashboard gets deployed to all environments with the changes that are needed. I also have an example of what the Q format looks like. So we go to Q. We have a dashboard. Oh, yep. For people that are familiar with Go, this looks very familiar at least the package and the imports. So it uses imports from GitHub as well, the queue schemas to know the schemas of the things we're using. And. This is what a dashboard will look like. It looks very similar to the YAML, but then with more pound signs. And if we do Q export out YAML, we actually get the same YAML out that was generated by the Q file that can be understood by Perseus. And of course, this was the queue. There's also the Go SDK. I also have a demo for that. If you go to Go in this project then we have a Go file, which looks very similar as that what we just saw. So this is the Go to create a new dashboard. And if we do Berkeley Duck Build Oh, sorry, Berkeley Duck You will see it will output it here And if we do bot build main output We again get a YAML you can also just run the project The main go file. It will also give you a YAML output that you can use with the Percy's CLI. Okay, now for the gotchas. Dashboards and panels don't save unless you click save. This is something that will annoy you especially because it's not that clear that you're in edit mode. Now you're in edit mode. You can easily think, okay, let's go here. But if we go back to the wire guard, that this chart that we just deleted is back again. If you refresh, it's instantly back. This can be very annoying, especially if you were busy adding a lot of panels and then just decide to refresh, you will lose work that way. The names don't match the names in the CLI. If you create something in the user interface with spaces or special characters. They get replaced with underscores in a way that could surprise you. A lot of units are missing and not customizable. I think this is the biggest one. It's very cloud native. oriented. So you can have bytes, you can have time, you can have rates, but stuff that's missing is like currencies or even SI units. So I also have a dashboard that monitors my power usage. There's no option to use the kilowatt or kilowatt hours as your unit. The entire Kubernetes integration is still missing. There is a Kubernetes operator GitHub repository, which does have some activity, but there's the Kubernetes integration doesn't exist yet. They also have a concept that's called ephemeral dashboard, which is basically a dashboard that has a TTL, which sounds like a great idea for CICD workflows. But if you try to use it it doesn't work with Berkeley yet. The documentation still is limited. And by default, the Berkeley doc setup to set up dashboards as code fails on macOS because of some permission issues. They are fixable. It's not the end of the world, but there's still some way to go. Also, because we're all SREs, and DevOps engineers, and software engineers, this is also an invitation for all of you to get involved. I think this project has a lot of future. And if you want to improve this project and want to use it, please contribute. I will thank you a lot personally. Send me an email if you've contributed and I will send you a personal thank you note as well. For the rest, this is it. Thank you very much for taking the time to watch all this and hopefully see you next time.
...

Erwin de Keijzer

DevOps Engineer @ TrueFullstaq

Erwin de Keijzer's LinkedIn account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways