Hi, everyone. My name is Joe, and welcome to my CON42 JavaScript 2024 session. Today, I'm going to talk about the TypeScript magic, like how to achieve end to end type safety across the full stack. First off, please allow me to introduce myself a little bit. As I said, my name is Joe and I'm currently a developer advocate at Dragonfly. And I'm a huge fan of K pop as well. And sometimes I just feel that I attended too many K pop concerts. Before Dragonfly, I mostly do backend development. So I've been doing that with different languages, JavaScript, TypeScript. Go Rust, just to name some of my favorites. And from there, I also developed a very strong interest in databases. So yeah, today I'm going to talk about, backend and, like the type safety. And I'm very passionate about what I do and I like memes as well. I'm going to be using memes in this talk and I hope you enjoy them as well. Before that, please allow me to introduce a little bit about the project I'm working on right now. It's called Dragonfly. So Dragonfly is an in memory data store. it is compatible to Redis. So it can be used as a dropping replacement, but the architecture is built from scratch. So it's just happened to be compatible. It's not a fork of Redis, right? Because of the new modern multi threaded architecture, Dragonfly is able to handle 25 times more throughput on a single multi core instance. So because of that, because Dragonfly can fully utilize your multi core hardware. If you take one If you take the binary of Dragonfly, you take the binary and you put it on a multi core, powerful multi core server and that one Dragonfly instance is going to be able to fully utilize your hardware. Whereas in comparison for Redis, because Redis is single threaded mostly, Redis can only utilize, like a single core. That's why if you put Redis on a multiple machine, even on that one machine, some sort of clustering solution is needed for Redis. So awesome. it's a, to me, it's like a great, it's one of my favorite data store right now. And, it is like the next chain in memory, the story, if you have, if you're interested, yeah, feel free to check us out and, we have a cloud offering as well. And, yeah, I'm going to go, I'm going to be showing like TypeScript, JavaScript code right now. just, One more note for the compatibility of Dragonfly, just like how compatible we're talking about, right? So this is a snippet. You can use this IO Redis library to create a client and that client will be able to talk to a Redis instance and you can send commands from there. So if you ever want to switch to Dragonfly, The only code you need to change is the URL. So you pointed to a different, dragonfly URL. In this case, I'm using a dragonfly cloud instance. And that's it from there. You can start to send the familiar commands that you use for Redis. Cool. And, let's talk about programming languages, right? So I do believe in the saying of using the right tools for the right things. And, yeah, for instance, Python has like a huge ecosystem of, data analysis, AI, everything. So if you're in that. domain or sector, it's natural just to use Python. But for me, as a backend developer, I think I personally have a very strong preference of strong and statically typed languages because when I receive, let's say we use JSON, right? When I receive a JSON payload from the client, it's so crucial for me to Make sure that the payload conforms the request, the shape of the request that we want. And we also validate each of the fields in there, right? So this strong and static typing and all the validations we do. that just makes me feel like sleep much better at nights. So let's dive in, TypeScript. yeah, as you can tell, TypeScript is, is, strongly typed. And if you ask me what's the, what like, what's one of the most important, project within the JavaScript ecosystem, I would argue that it is TypeScript because, it gives like, So many guarantees, right now than before, right? TypeScript provides something called the utility type. what does it mean? Let's take a look at the example. Imagine we are building like a bookstore, a website, maybe like the next Amazon, right? this is a type we can define and we can define so that we, we can use in our code. We want all our book record to look like this, right? By saying book record, because we are trying to save this object in the database as well. So it needs an ID. But what if I'm just receiving the request from the client, like adding a new book to the store, right? We don't want the client to specify the ID, right? Because, we want to generate the ID in the backend. It's more secure, maybe using a serial, maybe using like a UID or something, right? So when that happens, what we can do is that we can use this utility type from TypeScript. One example is called the omit utility type. So as you can see, we're using angle. brackets here, right? So the mindset would be if we're using round brackets, that's calling a function or calling a method. So we're operating on the object or the value. But when we see angle brackets, we're operating on the types. like on a higher level, right? So in this case, we're using the omit utility type to remove the ID field from book record and it returns a new type we can assign to, a new type called book request. By doing that, this book request type has everything in book record except for ID, right? So yeah, if you want to add a new book, to the store, we need all this information, but we're going to generate the ID on the backend, right? So similarly, we can, we have other utility types in TypeScript as well. Another example would be the peak utility type, it's doing the reverse. In this case, we want to have a new type called book preview, right? So what we do is that we use the peak utility type. And we take only one field, the title field from book record, and the end result will look like this, right? So we're, operating, playing with the types, we're using composition to generate new types. And, Yeah. And they can be used like whenever necessary and appropriate in your code. And the final example would be the read only utility type, where if you use that all the fields within the original book record type became read only and immutable. So yeah. And, that's just like three of them. We have like many other utility types in there as well. Cool. So now we have types, but, We defined the shape of the JSON objects that we want, right? But we haven't talked about validation. For instance, ISBN, that's a standard for, a serial for the books. And, it is a string, but it needs to meet. certain criteria, right? So how do we do that? Let me introduce you to one of my favorite libraries in TypeScript that is called Zod, right? So this is general Zod. And this is actually the logo of the library. So what Zod does is that let's take a look at this code snippet using Zod, right? So we're, yeah, as you can see, it's quite intuitive API, right? It looks. like similar to our type definition for book record before, but, it's actually different. First off, obviously we import the library, we import the Zed object. I'm saying Zed, you can tell that I live in Canada. We're importing the Zed object from the library, right? And from there, we're using , like z the string, Z, the date and everything, and z the object, right? So we're calling methods, and as you can see here, it's a account. So what we get here, the book record is actually an object, not a type, right? Because it's a value, right? And as you can see here, we are specifying the I-S-B-N-S string as well. So all this API. The that object, the that string and that thing which we're just telling, using Z that for my book record, I want it to be an object. And within this object, I want to have these fields, right? And for each of the fields, I want it to be a string, and not only string for id, I want it to be UU ID as well. Similarly for other id, right? For ISPN, we're doing a string right now. We can actually do better by adding a regex validation, right? I took this from, LLM, I don't remember which one, it's probably not the most strict, regex, but, it's, it's a good enough regex to know that, your, the stream being passed in is a valid ISBN, right? we're not only doing type. Here, we're on, we're also doing validation, like how we want the string to look like when I get the payload. And again, remember that it is constant. And this is a value, right? It is an object, not type yet. Cool. So we have, once we have this object called book record, what we can do is that we can use that to validate, other objects, right? In this case, I'm using the book record object. object and I call the parse function on it. And then I'm giving it a like arbitrary object, right? This object, obviously it won't pass because first off it's missing a few fields. And secondly, the other ID is supposed to be a new ID, not a one, two, three, another random string, right? So if I call book record up parse, and I pass in this object, the validation will fail so that we know that this particular, single field object with other ID is not what we want. You cannot send this to my backend server, right? Cool. So we have this book record. Again, it is an object and we're using all the magic from the Zot library. And I'm about to show you like more magic over here. So what we can do is that book record, with the smaller case letter B, that is the validator object we can use to validate. other objects, right? But we can also use the z. infer method from the library as well, and I pass in type of, this object. It will return me a type. I'm talking about type Right now, not object anymore, right? So Zot can infer a type from the validator object that we defined earlier. And the type, as you can guess, it looks like this. It has all the field and type requirements, right? So, where to use that? obviously, we use this book record validator object to validate. other objects, right? Whereas this book record with type capital B, we use this type, we can use that to specify, in our methods, in our handlers, where we're never appropriate. So that we know that we're expecting such a type in TypeScript, right? So Zot gives you both the power of doing validation as well as inferring, types from those validator objects. Okay. Thanks. Cool. So another example, similar to before, what if I'm creating a new book in the store, right? So it's very similar to utility types, but this time around we're doing again, the object first. So we already have the book record validator object, right? So it has all the fields, including ID. Right now I can create another object called book request. And I'm calling the omit method from the, the book record object, and I'm just kicking the ID field out of the thing, right? Then this book request, it is also a validator object that can be used to validate in this case request, because in the request, We don't want ID. We don't want the client to specify an ID, right? And then again, we can use the infer to infer what, infer a type for the book request validator object. And that gives us the book request type, as expected. so super powerful. And, that's just scratching the surface. You, if you go through the Zot documentation, you see a lot of like magical combinations, helpers, different methods that you can use to, construct your validator objects and infer type from the objects. That's awesome. so far, so good. What we have right now, we, by writing the code, we. We have the request validator object. We have one for book, you can define like many more. So we have the validator object as well as, the inferred type definitions of the book record the book request. So yeah, we're about to, get there. We have things to validate for, things like request, And I'm going to show you how to use it. So this is a, an API that I'm going to show you. It's called drizzle. And it's actually this drizzle. it is, one of the hottest, ORM right now. So let's see what drizzle can do. So drizzle is an ORM. So it is supposed to be used to work with databases, right? And as you can see, the API is similar. We're creating like a book table, called books. And what should be contained in this, book table? We have similar things, the ID, the title, the other ID, right? And this API, it's very intuitive because, it matches what we would see, in the database. So we specify the primary key, all the fields are non null, I like non null. And, We have like different mode for, for the types as well. because this could be a stream, but we're specifying like, treat this as a date, right? Yeah. So that's cool. but it is the kind of the table we define in the database, right? So yes, database can have like many constraints, like primary key, like not no, but It will be a little bit hard to do the validation, like what we did with ISBN on the database. level, right? Yeah. You can, in, in Postgres, you may be able to, define like a custom type to, to do that. But, yeah, another way to do this, to do the things that we add a validation layer for our table as well. So again, this is, code snippet using drizzle and we're defining the table format. Once we have the table format over here, right? That's the table format we defined in the previous slide. We can also use the integration, Drizzle and Zod, the two libraries, like they were very considerate. Drizzle made this, integration with Zod as well, right? So we have the table layer, right? But we can also define an insert schema that adds more validation before the insertion, right? And it will be based on the table schema that we define. So again, we're using, we're manipulating this object. We're using composition and we're making them, more powerful. And in this case, we're building from the ground up because we started at the database layer, right? So we define the table first. Once we have the table here, we're adding validation before insertion for this table, right? So what we do here. is that, we specified if the ID is not provided, the in source schema will do a default of UUIDV7, right? We also want the book title to be to have like at least one characters, right? And maybe this for some reason, the online bookstore, we care about books only after 2000, right? So it's random, but as you can see, we can add. More constraint in code as the validation layer before we do an insertion. Similarly, we're doing like a regex for ISBN as well. And, yeah, obviously if you want to add a, new book category in the store, like a new book title, we want the, we want to have zero copy or more, right? Not negative. So stuff like this, some of them can be done, in the database layer as well. But, just to show an example, we're doing the validation, here in code. And again, this is the insert schema object, right? And we're adding validation, but based on the table definition that we have. Cool. yeah, this is just showing that, something to remember that the book insert schema will use a default, UIDV7 and, we won't use, whatever been passed. And yeah, here are the validations that I just talk about. we can add over here as well. Cool. So now we have the books, the table schema definition, right? From the previous slide. And this insert schema is what we have, from the previous slide. So what, so again, book insert, we have the ID, using default UIDV7 already, but for the book request, for a request that is creating a new book title in our system, we still don't have And the ID, right? So that's one thing. Another thing is that in the insert schema, we're already specifying that the publication date needs to be a date, right? But when the JSON comes in, from the request as the payload, maybe it is a string. So what we can do is that we, again, we operate on this object again, right? So based on the book insert schema, because insert that's the last step already, before the insertion, right? So the request is what's, what we get, from the, from HTTP, from the client as the payload, right? So what we do is that we kick the ID field again, We also keep the publication date, right? Because in book insert schema, this field is off type daytime, it's not string anymore. And then here we reset the key. This is dumb, but just show like what you can do with, with Zod and drizzle, right? So we set the key back again and we parse it. From the original string from the request. And if the parse successful, then it turns to a date, but if it failed, then, we reject that, right? So now we have three objects. not types yet, right? So we have this books, the table definition, we have an insert schema, that is what we need, what we can, do validation before insertion. And we also derived like the request schema from insertion by, kicking some of the fields and doing some like transformation in here, right? as you can guess, now we can infer the type. that from the objects from the validation objects that we defined before right for book insert for book request we can infer the type as book insert and book request and as you can tell as you can guess this is equivalent to this if you write it by hand it has everything it has the title but it doesn't have id right yeah, that's amazing. Let's recap again, what we have, right? So by doing the, by doing Zod and by doing Drizzle with Zod, what we achieve is that we have two layers of validator. objects and their corresponding type definitions, right? So one layer is for request is like what we get from the client. And one layer is what we do before insertion. one may argue that do you need those like many layers of validation? Yeah, it will consume like some of the computation for sure. But, depending on like how paranoid you are or like you're the system requirement, right? So I do validation everywhere when necessary. And this is just showing that from the request layer down to the. Database layer, we all have validators and their corresponding type definitions and the type definitions. We didn't write by hand, right? They're all inferred from those validator objects. Cool. Next up, let's put this, these things that we have into a more, into a server, right? So the server, library I'm going to use is called Hono. And, it is, getting traction now as well, because this framework uses only standard web APIs, right? So it runs on all the JavaScript, TypeScript runtimes, like node, like DNO, like BOM, right? Like whole node is like right ones run anywhere in the JavaScript, TypeScript runtime world. Awesome. So here is a simple, whole new app and, it's quite intuitive. We create an app and we define the route and the handlers for this app, right? So we have because we've been talking about the bookstore, right? We have one endpoint that allows, maybe our internal user to create a new book title in our system. So we accept it as the post method, and we're using another yet another integration called the, the whole nose odd validator integration over here. So what we're doing here right now is that we use the, we're buying the validator. So when the request comes in, It will validate the JSON payload using the book request schema, right? So if it doesn't pass, if it doesn't pass, the, it doesn't have like enough fields, and maybe the ISBN doesn't match the regex, it will get rejected before moving forward, right? So that's like the guard here. Now we feel so confident that whatever we get out from the C request valid, Method is a valid book request, right? So again, here I'm using the validator object. And I pass it to the Zot validator for Hono. So it does the validation automatically. And here, this is the book request that book request type that we inferred from the book request schema validator object, right? This is quite amazing because we do the validation. This is just an object, right? It's just like a validator object, but we can know that after the validation is done, we should get a valid object called, validate the request and it has to be of this inferred type, right? Cool. So that's the request validation layer. And moving along, we can, this is automatic, right? But before insertion, if you're paranoid, you can use the book insert schema validator object to parse this request again. Remember what this does? This actually adds the UUID. for the ID field, right? Because in the book request, we don't have that, right? But for the book insert type, we have the ID, right? So yeah, you can say that it is a parse or a validate, but it's not. Yeah, it does the, yeah, it does the validation we defined for the insert schema. And one of the things that it does is to add the UID field there as well. Cool. So here I'm not actually doing the insertion, but you have to grasp, right? Cool. So this is the one endpoint server app that we built and it's folded, right? So what else we can do? I talk about the validation and the request layer and the database layer, but we're talking about full stack, right? So that's for this code here as well. What we can do is that we can actually export a type and we just directly call the type of function on the route that we defined for our application. In this case, we have only one endpoint and it needs to, it accept a request that conforms with the book request schema, right? So we can export this type and that's the server side. On the client side, Hono also provides this tool where, this is the server type we import, from the export over here. And then Hono also provides you a client, right? And with those two things, what we can do is that we can use this Hono client. And we specify by using the angle brackets, the server type, we just export it. And then we give it an endpoint. In this case, the server was running here, right? So 3000. So we specify the endpoint here as well. What we get this client is a strong type strongly typed client, right? It knows that it has endpoints like dark books. And you can call post on that. And, this endpoint only accept a JSON like this. So if I make any mistakes, say if I mistyped, I'm using the S I'm using a get, or I'm, using a different field name or whatever. From a type perspective, it will immediately know that I made a mistake, right? Because again, this type is inferring, like from the server, like what I can do, it is that powerful. So if you code this, from your server side, switching to client side, all the validations we just built, you feel so confident building applications, right? And again, let's recap. So before we have this, we have the request validator object, we have the record, the before insert validator object and their corresponding types, right? And it talks to say your primary database Postgres in this case. Now we also have a front end client object, this guy, which is also strongly typed. And he knows like which methods, which endpoints, It is allowed to call and what kind of objects you need to pass in. So yeah, it is that powerful. Cool. So since I'm working with dragonfly, I wonder if, there's such a thing for cash as well, because we have to request and, we do the validation and before insertion, we do the validation and it goes to the primary database. What about, what if like we need to cash some of the data and does it have some sort of things that's similar as well? it turns out that we have library, called. Redis OM, and it is an object mapper for, in memory data stores like, Redis, but because Dragonfly is compatible with Redis, you can use this library with Dragonfly as well. But unfortunately, yeah, we can define schema very similarly, but unfortunately, we don't have integrations with libraries. We like Zod at the moment, yet. So with that said, we can still. Define a schema and you can do it stores the data as the Jason data type, in ready. readies uses modules to support things like Jason search time series and those kinds of stuff. Whereas in dragonfly, Jason and search are native building. So you don't need to install additional modules on dragonfly. back here, RedisOM is, can be used to manipulate the JSON type, not the string type, in, in Dragonfly or in Redis, right? yeah, so we don't have the validation integration yet, but I would argue that, this is also good. and also sometimes you don't really need to. So you need to use like the JSON type in Redis or in Dragonfly when you need to manipulate those individual fields. But if you're doing like a cache, if you don't care about modifying individual fields, then the string data type of Dragonfly might be the way to go as well. I would argue that So yeah, depending on your use case, I feel that, the strongly typed and validated thing, like across the client down to the primary database is enough. The caching layer, we don't have such a thing yet, it's, it's a good to have maybe in the future. And, you can also cache your. your data as a blob, in a string data type of Dragonfly as well. So with that, you have a quite strong and safe and typed and validated, safety from your client side to the server side and then to your data layer like Dragonfly or Postgres, right? Cool. So yeah, if I work in code base like this, at the beginning, it's a little bit harder because you're always thinking about the validator objects. And then, you use the infer function to get the type definitions as well. But later on, you feel so confident because you know that your code wouldn't. Like they're always validated, right? And you can't have a mistake by passing an object that has a missing value that will be caught at the type level, right? if you're doing such a thing, cool. yeah, those are the tools that I want to share with you and introduce. to you today, and, just to, maybe you've heard like other solutions, as well, things like GraphQL and Swagger, so what's the difference between the things, right? to me, GraphQL, Swagger, they're awesome tools as well. And if you're using like multiple languages, they help. But given the fact that it uses a DSL, so they have like their own domain specific languages, right? So you need a way to work with them. Work with the GraphQL, schema, work with your like Swagger schema and maybe do some code generation, right? Whereas in our case, we didn't use a DSL. So that's one less thing to worry about. Another alternative is maybe you've also heard of is called TRPC. it can do the client server type safety as well. but compared to the tools that I introduced, one drawback of TRPC is that it doesn't, expose a standard HTTP API, right? So what we use like whole node backend, it still uses the standard HTTP API. So if you're the only consumer of your API, then TRPC might be a better solution, but, in the future, if you anticipate other people to consume your API as well, then you know, our way could be better because, you as the consumer, you still have the full type safety, whereas other consumers need to look at your, your API specification and code accordingly, right? Cool. With that said, yeah, I think that's a small, simple example, but we did indeed, achieved full stack type safety today. And, yeah, please try those awesome tools, you JSTS community. I love them. And then finally, if you're interested in doing in memory data, like caching, like real time stats, give Dragonfly a try as well. yeah, it's a great, awesome project, very strong, very modern, multi threaded, Redis alternative. And here's my QR code of my LinkedIn. If you're want to get connected with me, feel free as well. Thank you so much.