hello and welcome back.
In this video, we are going to see properties of pub sub and
how it can effect your choices while designing a system.
So let's get started.
So the first point here is about topics.
So we have seen the topics act as intermediaries between the publishers
and subscriber, but there is one more important point that the topics
also act like a database solution.
So how they act like a database solution.
So whenever you're going to publish messages to your topics, then the messages
are going to be stored in a persistent storage, so you can use them later.
So with that, there's one more property attached, that is, you
are guaranteed at least once delivery of messages, focus on that.
At least once delivery of message.
It means that if a subscriber subscribes through a topic, it is
guaranteed that subscriber is going to receive your message at least once.
Now, if I talk about subscribers, there is one property attached to them.
Whenever they are going to receive a message, which would be pushed by a topic.
Then those subscriber are likely going to send an acknowledgement.
Or you can say an ACK to the topic why they do it?
because as we have just discussed that a message is guaranteed
to be deliver at least once.
So we need to tell the topic, okay, we have got this
message once, so you can stop.
So that's how it works.
Now with that, there is one more problem.
What is the problem?
So when I say at least once delivery being guaranteed, but sometime your
message might be actually be sent more than once, because notice I
didn't said Exactly once delivery.
But what I said, at least once delivery, both are two different terms.
So if I would have said exactly once delivery, it means that your
messages are going to be delivered exactly one time, no matter what.
But pub sub system doesn't work like that.
It works on at least once delivery, where it is guaranteed.
You're going to receive your message at least once, but your
message can be pushed out to your subscriber more than once also.
So now let's see a situation where it can happen.
So let's say you have a message M1 which gets pushed out to all of the subscribers.
And let's say there is a subscriber S1, which subscribes to that message.
And then what happens that subscriber S1 loses a connection, maybe due
to a network called or some kind of issue going on in your network.
So with that, what happens?
That subscriber S1, get the message, but that subscriber is not able to send
an acknowledgement to the topic back.
With that what can happen?
Your topic doesn't know that your S1 subscriber has received a message or not,
because your S1 subscriber has not sent any acknowledgement back to your topic.
So what can happen?
So when your S1 eventually reconnects, the topic may realize, okay, the m1 message
hasn't been sent to S1, or maybe that subscriber S1 hasn't received it yet.
So what your topic is going to do, the topic might send
that message a second time.
So in this case, you can see clearly how that subscriber S1 received the
message multiple times just because the subscriber S1 was not able to send an ACK
or acknowledgement back to your topic.
So that's why it is a very important concept to know about, because here
we are talking about at least once delivery where we are sure your
subscribers are going to get messages at least once, but we cannot guarantee.
After one time, how many times they can get the message more.
So that is something that we have to keep in mind.
And it's a very important concept.
And we have to be aware of while dealing with the pub-up system.
Now with that, we also get to a topic which is known as item potent operation.
So what is an item potent operation, maybe, you know,
this already, or maybe not.
So basically the item potent operation is a kind of operation that has
same ultimate outcome, regardless of how many times it is performed.
It means that if we perform an operation and the result is always
same, then that's an item, potent operation you can think of an scenario.
So let's say we have message m1 and that contains some form of
operation and what we are going to do?
We are going to perform that operation multiple times.
So after performing that operation multiple times, if it has the
same outcome, no matter what, how many times you perform it, then
that's an item potent operation.
so let's take an example of that.
Think about, you're trying to set the status of something to complete.
Now that's an item, potent operation.
Why?
Because if you set someone's status to complete, even if you perform
that operation five times, your status is still going to be complete.
It's not going to change.
So nothing else is going to happen.
So that's an item potent operation,
but now let's discuss about non item potent operation.
So if you have an operation that increases some kind of counter.
For an example, a like on YouTube video.
So that's not an item potent operation because here what's happening because of
that operation that you are performing, your value is getting changed.
So for an example, if you perform liking of video operation five times,
instead of once, then the counter will always going to gets incremented
by five times instead of one.
So that is a non item potent operation.
So again, we have to keep in mind this thing when we are dealing
with pub sub system, that we are guaranteed at least once.
Delivery of message.
So if we deliver a message multiple times, Or it contains some kind
of non item potent operation.
That could be bad for our system depending on the operation perform.
So for an example, if you are doing some kind of operation and you just want to do
that operation only once, then the pub sub system might not going to work for you.
And in those cases, the up system will have drawbacks and
your system can fall apart.
So you have to keep in mind when using pub sub system.
If you want your operation to be performed only once, then pub sub
system might not be a great choice.
Now let's get back to the properties of pub sub system.
We will talk about topics.
So topics also give us ordering of messages.
And that is very cool thing about it.
So if you're going to send or publish messages to the topics,
they're going to be pushed to the subscriber in the same order.
So you can think of that.
If you're publishing five messages to a topic, then subscriber will
receive those five messages in exact same order as they were published.
So that's a very cool thing about a pub subsystem.
So you can think a pub subsystem like a queue where the first piece of
data that goes into it , is the first one that comes out of it as well.
So let's take an example for that.
So suppose you are dealing with a stock execution order system
and you have a pub sub system for that to execute those orders.
So if you perform those operations sequentially, you can use a pub sub
system because a pub sub system is going to give you that guarantee of
operation being executed in order.
Now the next example, maybe you are sending some stock prices to the
client and maybe you want to send those stock prices to appear in
the sequential way in your order,
So that's where it can be useful and another really good use case of pub
subsystem is, if you want to replay your messages in a pub sub topic,
if you want to, you can do that.
So for an example, let's say a subscriber request for a particular
message again to do it's processing.
So you can do it with a pub sub system very easily.
Now let's discuss about another property of a pub sub system.
So your pub sub system can help you to achieve clear separation of
concerns and separation of duties by publishing to a different topics.
What does that mean?
So if you have a big system, you will surely have a lot of work to do
in it, and if you want to separate the task between multiple servers,
you can easily do it with pub-up.
So what you can do, you can have a topic for each kind of task.
And what can happen.
You will have a subscriber which effectively is going to be a server,
which is going to consume that topic and can perform the work.
For an example here, if you're dealing with a stock based system.
So what you can do, you can publish the stock prices, maybe to the topic T1
and news alert, maybe to the topic T2.
So you can see that clearly we are defining the boundaries for
our news alerts and stock prices.
They're not going to be mixed together.
And our subscribers will always know where to look for the specific kind of data.
So with that pub sub system can help you to achieve clear separation of concerns
and separation of duties in your system and it is really helpful, especially if
you're dealing with a distributed system where you want your task to be assigned to
a specific servers so they can work on it.
Now, if you're thinking about what are the popular pub sub systems out there.
So one of the very popular one is Apache Kafka,
and another one may be Google cloud pub-up and AWS as well.
So these solutions that are listed out here give you a lot of things out of box.
So when I say out of box, I will take an example of one thing.
So maybe they give you solutions, like end to end encryption.
What does that mean?
It means that whenever you're going to send messages over to the network,
they are going to be encrypted.
So it means that your subscriber will only know how to read them.
No one else can read it.
So this has been given by the Google cloud platform or AWS, and there are
a lot of other functionality as well that comes with your pub sub system
out of the box and free as well.
So one more example, like Google cloud gives you auto
scaling options for your topic.
It means that Google cloud system says to you, don't worry about your topics, how
to scale them, just publish the messages
and take care of what kind of data goes to in which topic, and they say that we
are going to take care of how to scale it.
You don't have to worry about that.
So a lot of things can be given to you by your cloud provider out of the box.
You just have to look out to the different options that are available.
So with that, I will say that publish and subscribe pattern is a very
powerful pattern that you should use in your system design interview.
So if you're excited to know about how to use this pub sub system in a system design
interview, then we are just very close to our system design interview questions.
Be sure you also check them out so you can know how all the things are tied
together, that we are learning until now.
So thanks for watching this video.
I hope that you enjoyed it.
Goodbye.