Event-Driven Architecture: Async Communication in Microservices
What is Event-Driven Architecture (EDA)?
EDA is a system design style where services communicate indirectly by emitting and reacting to events, rather than calling each other directly.
- Event: A fact that something happened in the past (e.g., "Order Created"). Events are immutable and self-contained.
- Command: An instruction to do something (e.g., "Place Order").
- Query: A request for data (e.g., "Get Order Details").
Core Components
- Producer: Service that publishes an event.
- Broker (Event Router): Accepts events and routes them to interested consumers (e.g., Kafka, RabbitMQ). For a deeper understanding of message brokers, see the RabbitMQ Introduction: Message Broker Basics for Microservices.
- Consumer: Service that subscribes to and processes events.
Synchronous REST vs. EDA: An Example
Consider an order flow involving User, Order, Payment, Inventory, and Notification services.
REST (Synchronous) Problems
- Availability: All services must be up; a single failure breaks the flow.
- Latency Accumulation: Total latency = sum of each service's latency (t1 + t2 + t3 + ...).
- Cascading Failure: A slow service can overwhelm others, causing a chain of failures.
- Tight Coupling: Order service must know how to interact with Payment, Inventory, & Notification.
- Scaling Issue: Cannot scale one service independently without breaking the system.
EDA Solution
- User places an order.
- Order service checks inventory in real-time, saves the order as "pending," publishes an "Order Created" event, and immediately responds to the user ("Order Accepted").
- The Event Router pushes the event to interested services (Payment, Inventory) for parallel, async processing.
- Payment processes payment, Inventory reserves stock; each publishes its own success event.
- The router pushes these events to Notification (sends email) and Order service (updates status to "completed").
Benefits: Improved user response time, loose coupling, independent scaling, and resilience to temporary failures.
Event Movement Models
- Push Model: Broker immediately pushes events to consumers. Risk of overwhelming slow consumers.
- Pull Model: Broker stores events; consumers request them at their own pace (e.g., Kafka).
EDA Models: Pub/Sub vs. Streaming
- Pub/Sub (e.g., RabbitMQ): Events delivered to active consumers, then forgotten. No replay capability for new subscribers.
- Streaming (e.g., Kafka): Events are stored in logs (retained for a configurable period). Consumers can replay events or read from any point.
Key Challenges of EDA
- Eventual Consistency: Data may be stale briefly before becoming up-to-date.
- Duplicate Events: Brokers guarantee at-least-once delivery; consumers must be idempotent.
- Ordering Problems: Events may arrive out of order, requiring careful handling (e.g., sequence IDs).
- Schema Evolution: Changing the event schema can crash consumers; versioning is essential.
- Debugging Complexity: Async, parallel processing makes distributed tracing harder.
- Poison Messages: A single malformed event can block processing.
- Operational Overhead: Requires monitoring consumer lag, queue depth, throughput, and partitions.
When to Use EDA (Use Cases)
- One event needs to trigger multiple consumers (e.g., order placed -> payment, inventory, analytics).
- Long-running business workflows where failure is possible (e.g., order -> shipment -> delivery).
- Eventual consistency is acceptable for parts of the flow.
- Real-time analytics on a continuous data stream.
Key Takeaway: Use synchronous (critical path) for real-time operations, and EDA for async, non-critical processing that benefits from loose coupling and resilience. To learn more about when to adopt microservices and their benefits, check out Microservices Explained: Benefits, Challenges, When to Use. For a broader perspective on challenges in distributed systems, see Message Queues in System Design: Deep Dive with a Former Meta Engineer.
Welcome back and today in distributed microservices we are going to start the asynchronous communication. So till now
whatever we have done in distributed microservices where we have more than one microservices and they're talking to
each other it's mostly happen in a synchronous manner. But when we are dealing with microservices the very
important aspect is your eventdriven architecture the asynchronous communication. Okay. So in this
topic I will first cover the introduction of event- driven architecture. I will not go deep into
okay let's go into this specific event-driven uh you can say that event router either Kafka Rabbit MQ or
something. So in this topic my agenda would be to have a introduction introduction of EDA. Okay. So yes in
depth explanation of a specific event router we might go into the next topic itself. So let's start without wasting
any further time. What is EDA? What is eventdriven architecture. So considered it as a system design
style where services do not call each other directly instead service emit an event and other
service react to those event. Okay. So you have one service you have multiple services you can say that they never
call directly to each other. They just publish an event. Hey, now you might say what is an event? When
we are saying hey they emit an event you might say what is an event then event I would say in a simple way that something
a fact that something happened in the past. Okay. So see there are one major difference between event
order and not order say command. Event order command and query. Event means something happened in the
past like order created right? Command is like you are giving the command. Hey place an order. Place an order is a
command but order created is an event which happened. Okay. Query is means that you are fetching something like hey
fetch me the details of the order which are placed. So these three are different things. So when we are saying that
service emit event so event are immutable. They cannot be changed once created.
Okay. So it because they are already created in the past. You can't change it. What h exactly happened and it's a
past tense like order created. Okay, we are not saying place order. Place order means you are giving a command but we
are not giving a command. We just telling the fact what has happened in the past. Okay, so past tense is
describe what happened not what to do and self-contain. So every event has all the information needed to understand it
better. So once so now service emits event. So it emits an event and this event is listened by some
other services. Okay. And this event contain all the information
okay and based on that these services react on top of it. Don't worry about it. uh but only thing I uh I want you to
understand at this point of time is that no two services are communicate directly with each other. They are now
interacting based on events. One emit an event others consume those event. Okay. And this event is something
which is immutable like you can't change it. Order has been created. It's created now. You can't undo the order creation
part. It's order is created. Now you listen to that event. Okay, order created. Now you decide what you want to
do. Okay, but you can't change it. So in rest distributed microservices, the same example I would take for an
order creation. Let's say you have a user service, you have an order service, you have payment service, inventory
service, notification service. So I have taken an example of four five services. Now in the rest distributed
microservices what used to happen is user place an order. Okay. Now in the order first it will do it will check the
inventory real time whether inventory is there or not. Okay. Then it will try to process the payment. Once the payment is
done it will reserve the inventory and after the inventory is done it will send the notification. So these are all
different services and this order services is now talking with each of the different services to complete the flow
and after that it returns that order has been placed successful. Now in this part if you see here there
are many disadvantages we can easily see. So problems the first problem is availability.
All service must be available at the same time any service down request will fail. So in this part let's say
to complete this flow this should be available this should be available this should be available this
should be available. Let's say even if inventory goes down this flow cannot be success. Okay. So the first thing is
availability. Second thing is latency accumulation. Your total latency is that sum of all
the service latency like how much time this place order request takes then this plus this plus this plus this. You got
it right? So your latency is now accumulated t1 + ts2 + t3 + t4 like this
cascading failure. One slow service can bring down the other service and entire flow itself.
If for example your order service is fast, your payment service is fast but your inventory service is too slow then
there are is a casket failure because let's say you are sending it 10 requests but it only able to handle five requests
for five requests it is denying. So those five failure will cascade to the payment service and cascade service will
uh you can say that uh the payment service will cascade it to the order service. So there is a cascading
failure. Okay. The tight coupling. The tight coupling in that in the sense if you see the
order is talking with payment. Order is talking with inventory. Order is talking with notification. So order has the
knowledge of all three services. How to interact? What response I will get and many other things. So there is a very
tight coupling in this rest distributed system. And another thing is scaling issue. We
cannot scale just one service independent independently. You can't just scale payment service and your
order service, inventory service, notification service. Uh let's say it's not scale that much. So what I mean to
say you scale based on load. So you just can't scale payment service that okay earlier it was taking 100 requests per
minute. Now you increase the scalability and it now it accept accepting 1,000 requests per
minute. Even though it is accepting thousand requests but what about inventory it is not scaled right? So it
will only have the capability to handle 100 requests rest 900 will fail. So cascading failure impact will come on
this only. So scaling issue is there. So now here if you see that there are so this is just you can say the top five I
have listed down when you have a flow like this when I say that flow like this means long
running flow. So consider this as a wrong long running flow order placed then you check the inventory then you
process the payment then you reserve the inventory then you send the notification. So this in the long
running flow means multiple services are included. Okay. So in that the rest distributed microservices has various
challenges and these are just top five. So generally in those scenario eventdriven architecture is highly used
in the industry. Okay. And again this is just an example I am showing you whatever this order service inventory
service. It's not like this is the exact flow for an order. So consider this just an example for differentiating what
between rest and the EDA event driven architecture. So now in the event-driven architecture
if you have to just design the same thing into this part. So we still have a user order service. Now here we have an
event router. So see at this since this is the first part of this event-driven architecture
either it could be either Kafka, Rabbit MQ or any different router you can say that but here currently you can say that
it's a generic term I have I have used event router okay and then you have a let's say
payment service, inventory service, notification service. Now we have everything same but apart from that we
have another uh you can say that kind of uh platform like a event router. Now what happened the first is user
place an order. Now here if you see that I am doing a realtime processing still check the realtime inventory. Yes, if
inventory is present save order status equals to pending and it will publish an event to this
event router that order created. So one event is published. So this is an event event publish order created and it send
the response to the user order is accepted. Okay. So your first part is completed. So now here if you see user
got the response very early and till now what we have done is we have just stored the order in the
pending state and we have just publish an event that order created event uh to this event router. Okay. And the
response has been gone to the user that order accepted. Now the second lag which is a sync now it's doesn't have to be in
real time. Okay. So now as sync and parallel processing here in this box. So now the event has been published
right to this event router what order created. So what it will do this event router it will push the order created
event to all those who are interested. Okay. So now let's say that payment service is interested in this event and
inventory service is also interested in this event order created event. Okay. So now here if you see that it just push
the event it don't tell anything what you have to do it just tell that hey order is created now whatever you have
to do do it now payment service what it will do is it will try to process the payment
inventory service try to reserve the inventory and once done they will also publish their event hey my payment
success is done okay and reserve inventory. Hey, I am able to reserve the inventory properly. It just publish an
event. So these are also an event. Now event router in the third line
whoever is interested into payment success event it will push the event to that. Now let's say notification service
is interested in payment success event. So it just push the event. So now here if you see what event router is doing is
services publish an event and what event router is doing is pass the event to the other
services which are interested. So it act as a mediator or a broker. Okay.
So now let's say in this one uh when payment service publish an event that hey payment has been success this broker
this event router what it does is it just pass this event to whichever other service are interested in. So
notification service is interested in this event and it now got the payment success event. So it send the mail okay
and it also send this payment success event to order service and it will update its status to completed.
This is just an example to show that in case of event-driven architecture different microservices are talking with
each other based on events. Okay. So the same flow we are dividing or you can say that modifying into this
EDA and this is one perfect use case where EDA can be used. Okay.
So what are the advantages of eventdriven architecture from the above flow we can see
very simple like whatever the disadvantages of synchronous chain we have seen generally are the advantages
of event-driven architecture. One thing is loose coupling no direct dependency on one another.
Scalability service can scale independently like now you can scale payment service as much as you want. Uh
it can process payment like depending upon the event it can accept so many events. It can process the events. So
now it can scale independently. Okay. Better resilience temporary issue do not break the whole system. Now let's say
that reserve inventory is failing but the whole system is not down. order is still getting accepted. Okay,
maybe for temporary purpose it is failing but once it is get up all the pending events it can go and start
working on it. So your system is not completely failed. Okay, replay we can reprocess the old event
that capability is there. Improvement in latency. Yes, for sure. Uh because you are doing ing processing, parallel
processing and even the response time to the user is improved a lot. Now user don't have to
wait till this whole let's say 5 second. Now user has to wait only let's say 1 second. Here it got the response. But
now because of using parallel processing and all uh everything will get complete sooner.
Okay. So latency get improved. So from the above understanding can we take out the key components? The
key component is we have a producer the service which publish an event. The service which publish an event.
Then we have a broker the router. So it accept the event and what it will do is it will pass the event to consumer. So
what is consumer service which act on the events you can say that which
act on the events or which consume the events. Okay. So producer publish an event and event broker or router pass
the event to a specific services which are interested in it. So these these are the major components
of event-driven architecture very high level. Now the next thing uh introductory part
I would just say that how do events move through this. Okay. So how the events move through event-driven architecture.
Now let's say events are published. Now from here how it got passed to this. Okay. So one is push model. In the push
model what happen is producer publish an event to the broker and the broker push the message immediately to the consumer
which are interested in these events. Now consumer must handle whatever rate broker send and sometime it is possible
that consumer can get overwhelmed with it. Okay. Another is pull model. In the pull model producer publishes an event
to a broker. Now broker store the messages. Consumer can ask from the broker that
hey give me this many messages. Broker will send that. consumer process at its own pace and then ask more messages
again and continue the process. So this is known as pull model. So there are many uh you can say that
broker which works on pull model, push model or both. We will see that. But this is just a very uh high level
understanding that we can have here push and pull model also. So that when we so this is consider that
that we are not going in depth how push model works that is depending upon like which framework we are using let's say
if you go with Kafka then how the pushpull works in Kafka then we have to go independently into that framework
flow okay if you're going with rabbit mq how push pull model we have to enable how it works we have to go into that but
here today the intention is that We know the term that's all. Okay. Then there is something called EDM
model. So there are two types of EDM model. One is pubsub and another is streaming. So whenever somebody say that
pub uh pops up Google pops up you you might have heard of right uh publisher subscriber model generally
events are published to active consumers and then forgotten if new consumer joined let's say
tomorrow it won't get yesterday messages because messages are not stored and whatever Whatever the active
consumers are there only they will get the events. Okay. So they the pops sub model do not have capability to replay
the events or to store the events or maintain the history. You can say that like rabbit MQ exchange.
Then another model is streaming model. Events are appended in logs forever or retain based. So you can decide whether
you want to retain for 7 days, 3 days, 4 days or you don't want to delete at all. So events are stored in logs and we can
replay right. So that is possible with the streaming. So now
let's say that if new consumer joined say tomorrow, new consumer joined tomorrow and it can read it can decide
that it can read from the latest offset or read from the day one. This is possible because we are retaining the
data like from the day you have written you can start from that day also. Okay. Example is our Kafka. So a Kafka
is a streaming model there logs are stored. Rabbit MQ is your popsup where uh you can say that events are not
restored. [snorts] But what are the challenges of
event-driven architecture? Currently everything looks good, right? We have so many advantages that there are so many
challenges and these are critical like you can say that whenever you are working in the highle design part of u
any any uh question where you have to use eventdriven architecture mostly the questions or challenges comes into this
part. The first challenge is its eventual consistence. Eventual consistency means let's say
it is processing something in a sing manner. Now when you hit it let's say get call you might get a stale data a
stale or old data but eventually when you hit another after some time you might get latest updated data. So it's
an eventual consistence. You might get a stale data if you try quickly but eventually if you try after some time
you might get latest up-to-date data duplicate events. See the event routers guarantee at least once delivery. So it
can deliver events more than once. So duplicates are normal. So we have to handle it properly.
Ordering problems. Now let's say that your publisher has published two events but the consumer got first
event two and then event one. This is very very important problem. Okay. So this need to be handled very well.
Okay. So we'll see that we will share real world problems how we will pick one topic and we'll try to build out all
these things. Okay. But today we just highlighting the challenges and each challenge we have to solve. Okay. So
when we will go with let's say Kafka and when we will build Kafka then after that we have to go one by one all the
challenges there would be more. So here I have just listed around six seven but there are so many so we have to take one
challenges and see how it has to be industry standard to solve those challenges. But I wanted to just mention
here that there are so many challenges. It's not like EDA is perfect. Another is schema evolution. If you break the
schema, all consumer will crash. Right? Because I told you event has a specific schema which has all the information and
consumers consume this events. So if you change something in this schema, you remove something or you updated
something. So consumers might need to be updated also otherwise they will crash. So you need to maintain the versioning
properly debugging complexity. Distributed tracing across a syncing hops are little
difficult because now everything is happening inc manner. So see this is happening in a singing manner. This is
happening in a singing manner. This is happening in parallel. So when parallel and comes into the picture debugging is
not that easy. Okay. So yes, debugging got a little bit complex or here poison message generally one bad message can
block the event router itself. So I don't I won't go into this deep what is poison message here now because it
will complicate the things and to understand the poison message like I need to tell the framework first like
whichever we are discussing but understand that there are because events are published to this
event router let's say if one event is bad okay and nobody is able to consume it and what if that event is is still
kept into the event router. Okay, so very high level you can say that there are possibilities that one bad message
also can block the event router too that generally known as poison message. So that challenge is also present.
Then again another challenge we have is operational overhead. You have to monitor the consumer lag. Okay. Hey, you
are publishing an event five per minute but consumer is able to consume or process only one per minute. So how much
delay [snorts] is lagging? Monitor the que is Q is able to maintain the events which are getting published or
not. Okay. What is the throughput partitions and all? So you need to maintain everything size and all. So
there is an operational overhead also. So it is not like you just use EDA and everything works out of the box. It is
little you can say that uh proper understanding proper knowledge is required otherwise
these challenges are so critical that instead of getting advantage we might uh get into more trouble. Okay, our DB
could get easily corrupted very easily. Okay, this order problem itself if not handled properly it could easily corrupt
the DB itself. Now what are the use cases of event-driven architecture? Like sometime
what happen is we need to have a mindset like okay when I should go for rest when I should go for event- driven
architecture. So there are see many scenarios but high level I will give you four. One is that one event and there
are multiple consumers for that event. Then it's a very clear indication that we have to use event-driven
architecture. Let's say for example order is placed then inventory has to do something payment has to do something
email notification service has to do something the analytics team has to do something. So you can't order can't talk
with all of them. Okay. So it can publish an event and you can do whatever you want to do. Okay. So it's a very
good use case for the event-driven architecture and that's why I have used that for an example
longunning business workflow. That's the same thing that here if you see when transaction is started and when your
transaction got end you can say that. Okay. So order, payment, shipment, delivery like everything
should be up successful and then only your order placement is done. So it's a very
long running business flow. Okay. And it's a failure prone. Okay. It can be you can say that something can be failed
in the midway. Okay. And this flow takes a lot of time. So if the flow is not in the critical
path then we can say that it can be broken into multi-step which is a failureprone.
Okay. So you can always think critical part non-critical
part. So critical part should be always real time. Okay. Like here I am doing that.
This is my critical part. Check the real time inventory. I don't want to do it in a singing. When I get
the place order request, I check real time stored into the DB status request to pending publish an event. This is all
my critical. Okay, that's why I'm doing in a real time not in a sing. So once this is done then my
non-critical part starts which I can go for eventdriven architecture that okay even if it takes some time even let's
say it can take 2 seconds more or let's say 4 seconds more it's not like something is very urgently waiting so
consider that as a non-critical part okay so we can think of that it's a good use case for EDA
Another is eventual consistency is acceptable. If you are have a use case where eventual consistency is okay, then
it's a very good hint that EDA might be used. realtime analytics like data is
continuously coming and you have to work on that data then definitely it's an EDA use case like multiple events are
continuously coming coming and you have a consumer which continuously consuming those events and doing some realtime
analytics okay data need to be continuously processed then is also a use case of EDA so this is very high
level four use cases of EDA but we can have more and this is Just one introductory video of what is EDA before
we jump into a specific event router and most probably I will pick Kafka for it. Okay. Okay guys, see you soon. Bye.
In synchronous REST, services communicate by directly calling each other, which couples them together and means all services must be available for a request to succeed. In EDA, services communicate indirectly by publishing and consuming events via a broker, which decouples them and allows for asynchronous, parallel processing, improving resilience and user response time.
With EDA, the order service can immediately respond to the user saying 'Order Accepted' after saving the order as 'pending' and publishing an event. The rest of the processing (payment, inventory, notification) happens asynchronously in the background, so the user is not blocked waiting for all downstream services to complete.
The two main models are Push and Pull. In the Push model, the broker immediately sends events to consumers, which can overwhelm slow consumers. In the Pull model, the broker stores events and consumers request them at their own pace, as seen in Kafka, providing better control and resilience.
In Pub/Sub (e.g., RabbitMQ), events are delivered to active consumers and then forgotten, so new consumers cannot replay past events. In Streaming (e.g., Kafka), events are stored in logs for a configurable retention period, allowing consumers to replay events or read from any point in the log.
Key challenges include managing eventual consistency (data may be briefly stale), handling duplicate events (consumers must be idempotent), preserving event order, evolving event schemas safely, debugging complex async flows, and the operational overhead of monitoring consumer lag, queue depth, and partitions.
Use EDA when one event needs to trigger multiple consumers (e.g., order placed triggers payment, inventory, and analytics), for long-running workflows where failures are possible, when eventual consistency is acceptable, or for real-time analytics on continuous data streams.
Keep this summary
Save it to LunaNotes and it becomes a real note in your library — editable, searchable, and ready to turn into flashcards or a diagram. Free to start.
Save to LunaNotesOr summarise for another video.
32K+ people have turned 65K+ videos into notes
“Very useful tool. I'm using it for the last couple weeks and I'm loving it so far. Makes learning on YouTube a great experience.”
This summary and transcript were automatically generated using AI with the Free YouTube transcript summary tool by LunaNotes.
Related summaries
Microservices Explained: Benefits, Challenges, When to Use
This video provides a clear, beginner-friendly introduction to microservices architecture. It contrasts microservices with monolithic applications, explaining core problems like scalability, fault isolation, and deployment speed, and details when a monolithic approach might still be the better choice.
RabbitMQ Introduction: Message Broker Basics for Microservices
This video launches a new series on RabbitMQ, an open-source message broker written in Erlang. It explains how RabbitMQ facilitates asynchronous communication between microservices using message queues, enabling scalability, load distribution, and fault tolerance. The overview covers core concepts like producers, consumers, FIFO queues, supported protocols (AMQP, MQTT, HTTP), and differences from Kafka.
Microservices Explained: What They Are, Problems They Solve & When Not To Use
This comprehensive guide explains the core concepts of microservices architecture, contrasting it with traditional monolithic applications. You'll learn what problems microservices solve, such as scalability and fault isolation, and also discover key scenarios where a monolithic approach is still the better choice.
Comprehensive Overview of Online Event Management System Project
This video provides a detailed guide on creating an Online Event Management System, covering project introduction, required tools, features, and practical output presentation. It also discusses the software development life cycle (SDLC) and testing methodologies relevant to the project.
Complete System Design Course: Scalable Architectures & Key Concepts
This comprehensive system design tutorial covers everything from basic components and SQL/NoSQL databases to advanced topics like load balancing, caching, partitioning, replication, and the CAP theorem. Learn how to build scalable applications capable of serving millions of users, with a practical video streaming design example.
Most viewed summaries
A Comprehensive Guide to Using Stable Diffusion Forge UI
Explore the Stable Diffusion Forge UI, customizable settings, models, and more to enhance your image generation experience.
Kolonyalismo at Imperyalismo: Ang Kasaysayan ng Pagsakop sa Pilipinas
Tuklasin ang kasaysayan ng kolonyalismo at imperyalismo sa Pilipinas sa pamamagitan ni Ferdinand Magellan.
Mastering Inpainting with Stable Diffusion: Fix Mistakes and Enhance Your Images
Learn to fix mistakes and enhance images with Stable Diffusion's inpainting features effectively.
Pamamaraan at Patakarang Kolonyal ng mga Espanyol sa Pilipinas
Tuklasin ang mga pamamaraan at patakaran ng mga Espanyol sa Pilipinas, at ang epekto nito sa mga Pilipino.
How to Install and Configure Forge: A New Stable Diffusion Web UI
Learn to install and configure the new Forge web UI for Stable Diffusion, with tips on models and settings.
Found this summary useful?
Take it with you. One click puts it in your own LunaNotes library.
Save to LunaNotes