Transcript
This transcript was autogenerated. To make changes, submit a PR.
Welcome, everyone.
I'm Mohamed Moussaoui.
And today we are diving into NATS, the cloud native challenger
to Kafka's messaging throne.
Get ready to explore how NATS is shaking up the world of distributed messaging.
And to kick things off, let's dive into a real world scenario that illustrates
why efficient messaging systems are crucial in more in modern applications.
So imagine a smart city where various systems communicate seamlessly.
We have a weather station that reports high temperatures, triggering
notifications to citizens, and meanwhile smart thermostats
adjust to manage power loads.
This interconnected scenario is made possible by efficient messaging systems.
So in today's world, several messaging systems are widely used to enable such
complex and real time environments.
But before talking about that, let's quickly define what is a messaging system.
So a messaging system is the backbone of distributed applications,
enabling different Parts off a system to communicate efficiently.
It's what allows our smart city example to function smoothly in the real world.
For example, we have Apache Kafka that is widely used and it's
popular for its high throughout distributed event streaming platform.
We have also rapid MQ that is known for its reliability and support
for multiple messaging protocols.
And it's widely used for IoT systems.
We have also Apache Pulsar that is another contender offering
multi tenancy and geo replication.
These systems have been the backbone of many base of many applications,
powering everything from financial transactions to social media feeds.
For example, Apache Kafka is developed by LinkedIn for its its feeds.
So these systems have proven their worth in handling large volumes of data.
And enabling real time communication between different parts of a system.
However, as we move further into the cloud native era, new challenges are emerging.
And this is where NAT comes into play.
Current streaming technologies face several significant challenges.
First of all, we have the infrastructure complexity that
can quickly become overwhelming.
Especially as systems scale, our setting up and maintaining streaming
technologies requires a deep understanding of these systems and
significant or personal effort.
Scalability also is another constant challenge as the volume
of data and requests grows.
Handling extreme scale requires Resource optimizations can lead to
substantial resource consumption, and maintaining data consistency
and reliability across distributed systems also presents complications.
Ensuring message delivery in distributed environments can be difficult,
particularly in scenarios involving network failures or node outages.
These are the core issues in the messaging systems.
And here enter NAT, the messaging system designed to tackle these issues head on.
NAT is a simple, secure, and high performance open source messaging system.
It's designed for cloud native applications, designed for IoT messaging,
and microservice architectures.
And also it's designed to go to the edge, from the cloud to the edge.
And NAT stands out with its ease of deployment.
Built in security features and ability to handle millions of messages per second.
It's built to tackle the challenges we just discussed.
NAT sounds promising, doesn't it?
But you might be wondering, what specific features does it offer to
address the problems we've discussed?
So let's dive deeper into NAT's capabilities.
NAT's architecture is divided into two main components.
We have NAT Core and NAT Jetstream.
The foundation of every functionality in NAT is published.
Publish subscribe model, and it's often referred to as pops up.
So let's start with the.
At its heart, as I said, we have the PubSub model, where publishers
send messages to specific subjects and subscribers receive messages
from one or more subjects.
These subjects are organized hierarchically like a tree
structure, as we see here.
The hierarchy is built using dots, and while publishers can only send
a message to one specific subject.
Subscribers have more flexibility.
They can subscribe to one or multiple subjects using wildcards or will cards
We have two types of full cards the first one the asterisk which matches
one token in the Hierarchy and the second one is the greater than symbol
which can match one or more tokens
so about Building upon the Pub Sub foundation, NAT also supports a
request reply model, REC REP, enabling more complex communication patterns.
And for load balancing, NAT implements built in queue groups, efficiently
distributing messages among subscribers.
And we'll see that in the demo.
This feature is crucial for our horizontal scaling of microservices.
And additionally, NAT's clustering provides high availability, utilizing a
gossip protocol for server discovery and a raft based system for leader election.
Now when we enable this stream, that is just a test.
So easy to enable just streamer in that we unlock a whole new realm of possibilities.
We have just stream that provides a built in processes engine, allowing
messages to be stored and replayed later.
And this capability enables fault tolerant and high availability configurations.
Also with just streams streaming functionality, we gain fine green control
or control over messages handling.
We can configure.
replay policies, for example, specifying that a consumer should receive stored
messages at their original publication rate rather than as fast as possible.
We can also set retention policies based on time, message count, or size.
And further, furthermore, this stream can be configured as a shared queue,
shared work queue, facilitating desk Task distribution among multiple workers.
Also, the stream offers a key value store built on top of the
streams, which can effectively replace Redis in many scenarios.
This key fetch value store allows us to perform operations like storing,
storing and retrieving key value pairs, setting expiration times
for data and even executing atomic operations such as Compile and swap.
Its flexibility makes it either for configuration management,
distributing, distributed locking, or as simple as a cache.
Another powerful feature in the object store is the object
store in the ingest stream.
This is particularly useful for handling larger data sets of
files within your NAT system.
You can store and retrieve large objects, manage their metadata,
and implement versioning.
And it's especially valuable in scenarios requiring large data sets.
file distribution across your system or when maintaining a history of
object changes in is necessary and Last thing is security.
So security is the priority in NATS.
We have features including TLS for encryption We have in key for
authentication and GWT for authorizations So now support multi tenancy through
separate accounts, each with its own subject space and security policies.
To address scalability and deployment flexibility, Gstream is multi
tenant, horizontally scalable and supports various deployment models.
A particularly interesting aspect also of Jetstream is its decoupled flow control.
This means producers and consumers can operate independently.
If you have a slow consumer, it doesn't impact your producers or other consumers.
This feature is available.
Invaluable for building resilient systems, allowing scenarios where a
consumer can catch up on historical data before transition that transition
into real time processing all without affecting the rest of the system.
So in conclusion the combination of NETSCORE and GESTURING creates a
powerful and flexible system for building distributed applications, whether you are
implementing a simple messaging, A complex data streaming or need persistent storage.
NATs provide a comparative solution to meet diverse architectural needs.
Let's pass to the demo.
Install NATs, it's very easy.
You can go to the NATs.
io website and install it with directly with the Exe, or you can
also install it with Choco, Or any package manager in your system.
For me, I already have NATS.
I will show you how to start the server using the NATS CLI command.
It's very easy.
We have the server built in the NATS CLI.
You can easily say, NATS server run.
And it will start.
A local development net server.
We can also specify that we want to activate the stream by passing
the re stream flag, or we can configure our server using confi.
Um,
In the file, we can activate the G stream, we can add accounts, we
can activate web socket to connect our web apps directly to that.
We can activate the the endpoint of the web socket and we can, as I
said, we can act like configure the clustering the separate cluster.
We can activate the account.
But let's keep it simple, and we just start NAT without gesturing.
Today I will show you just the pub, sub, and the request reply models.
But you can check the website for the connotation and for seeing
what we can do with gesturing.
Let's say we, we stay on the on the smart city example.
And we have, uh, for example, our our with the application that is connecting
to subscribing to temperature,
temperature, um, of a city and for a zone, for example.
We have now a subscriber waiting for messages in the subject
temperature Casablanca zone one.
And we can easily send a message using.
Not spub.
Gotcha.
Got it.
Okay.
Zone one, and we can specify the message.
We can send any type of message, we can send GOB on.
We can just send portal or just a simple string 26.
And here the sorry, the wrong the message is received and we
specify in which subject receive the message and also the text.
Also what we can do, we can say, for example, we have another application
that is, for example, the thermostat is waiting for temperature.
In all these zones of Caza Blanc.
We can specify the world cut assess, and it'll connect to
every zone of the the city.
So if we send the same message, it'll be received.
By these two subscribers, but imagine we are sending a message to zone two,
for example, it will be received Only by the second, Subscriber because it's
saying that I want all the Temperatures of the city Kazanbakh and also we can,
for example we can we can subscribe to all the stages and all the zones.
For example, we say temperature and we pass the greater than symbol.
And so now it's subscribing to all the cities and all the zones.
For example, if I send the same to a bat, for example, it will
send it will be received by this subscriber, not by this one.
And if we send the same thing to Casa Blanca, as we test, we tested Casa
It will be received by the two because this one is waiting for
all the tokens that comes after the first token temperature.
Let's say we have multiple servers that are on multiple nodes that are waiting
for the data from the from the um, the data of the temperature of all the cities.
And we want to say we want to load balance the messages.
For example, we have
a subscriber that's Is waiting for all the temperatures on all the cities, and we can
say, for example it's part of the queue
thermostat, for example.
And we have another node waiting for the same thing,
in the same queue,
and a third one.
And let's say we are sending
And let's say we are sending a message every every second.
In the Casablanca and you have the count
and
so we are sending a message every one second to the, this this
subject with the same message.
And as we see here, the The message, the messages are evenly received
by the three the three subscribers.
And imagine we have the third one that is down,
and we have the two that are staying up, and they will receive, of
course, the rest of the messages.
The last thing that I want to show you is the request reply model.
For example Let's say that we have a server that is waiting
for requests on the on the
reply.
It's replying on the subject, slogans, Palestine, and is always responding
with from the river to the sea.
And we have another server that is waiting or asking for this message.
Thank you very much.
So we can send request, slogans, Palestine, and we can say, free Palestine.
We will receive the the message from the replier.
And as I said, everything is built on top of the pub sub model.
So if we say, let's listen to every.
Every subject, a sub to every subject.
And if we send a request here, the same request, we will see that the
request, even if it's a request, it's in the same time pub we are publishing.
To to subject, and we receive the response in the inbox.
This one is, inbox is the inbox of this requestor, and it have all, it have the
same, it's the same ID of the inbox.
So that's it for today.
You can check the website of nas io to have more more examples of the of
the net score and the net nas ing.