Streaming Agents: agents that run inside the data stream

The usual way to build an agent is as a service: it receives a request, queries what it needs, reasons and answers. It is the model we inherited from web applications and it works well when somebody is asking a question.

There is a second way, less well known, that changes where the agent lives: instead of being a service you query, the agent is a process that runs inside the event stream. Nobody invokes it. It activates when the event it is responsible for occurs.

The difference is not about infrastructure

At first glance this looks like a deployment decision. In practice it changes three fundamental things.

The agent stops asking for context. A service-style agent that needs a customer’s history issues a query and waits. An agent running inside the stream already has that state at hand, because the processing engine has been maintaining it with every event that went by. The difference between querying and having is typically hundreds of milliseconds per decision, which at production volume is not a detail.

The agent reacts instead of waiting. A service does nothing until somebody calls it. That means the process only begins when a human or a system decides to start it. When the agent lives in the stream, the trigger is the fact itself. Network degradation does not wait for the customer to call before someone looks at it.

The agent scales the way the stream scales. The engine already knows how to distribute work by partition and guarantee that events for the same entity are handled by the same task. That is exactly the distribution a stateful agent needs, and it comes solved.

What you gain in traceability

There is a less obvious benefit that in regulated sectors is usually the deciding one.

When the agent runs inside the stream, both the data it saw and the decision it made are events in the same log. That enables something very hard to do with a service-style agent: replaying the case. You can go back to the exact position in the stream, run the version of the agent that was live that day and see why it decided what it decided.

With a service-style agent, that reconstruction requires having separately logged the request, the response from every system it queried and the model version, and trusting that the three records line up. They almost never do.

When it is the wrong fit

This pattern does not replace the conversational agent, and forcing it where it does not belong is an expensive mistake.

It is the wrong fit when a person initiates the interaction. If the use case is somebody typing in a chat, the service-style agent is the right form.

It is the wrong fit when the reasoning is long and multi-step. An event stream is optimized to process a lot at low latency. An agent that is going to make fifteen tool calls and take forty seconds does not fit that model, and putting it there creates backpressure upstream.

It is the wrong fit when volume is low. At a hundred events a day, the operational complexity is not worth it.

The practical rule: if the trigger is a business fact, volume is high and the decision has to come out fast, the agent belongs in the stream. If the trigger is a person and the reasoning is open-ended, it belongs as a service.

The two running side by side

In most real implementations both end up running, and that is the healthy architecture.

The agent in the stream does the continuous work: it evaluates every transaction, keeps context up to date, detects what falls outside the norm and triggers action when it should. The conversational agent handles the person, and when it needs to know the state of the business it queries the context the first one has been maintaining.

Put another way: one builds the truth about the present, the other explains it. That is a considerably cleaner division of labor than asking a single agent to do both.

Where to start

If you already have an event platform running, the first case is usually the most boring one on purpose: take a rule that runs in batch today and move it to the stream, with no agents involved. That validates the state model, the sizing and the operational side.

With that working, adding reasoning on top of the same stream is an incremental step rather than a leap. The most common mistake we see is the reverse: starting with the agent and finding out later that the data platform it needed was not ready.