Streaming architecture

Streaming Architecture

Concept of the skill

What it is: streaming-architecture is the discipline for designing one logical result as an ordered sequence of values over time, with explicit flow control, framing, error, resume, and termination semantics.

Mental model: Every streaming design has a producer, stream, consumer, backpressure path, and termination signal. Transports encode those primitives differently; they do not replace the design work.

Why it exists: Some results are too large, too slow, or too useful early to wait for a complete batch response. Streaming gives earlier value and bounded memory only when the contract is explicit.

What it is NOT: It is not browser freshness UX, one-shot API payload design, durable worker execution, model/tool transcript protocol design, event-contract design, or the page-level rendering taxonomy.

Adjacent concepts: real-time-updates owns browser freshness and live-dashboard UX; api-design owns one bounded request/response surface; background-jobs owns durable queued work; tool-call-flow owns model/tool message-history protocol; rendering-models owns CSR/SSR/SSG/RSC taxonomy; event-contract-design owns event payload and topic contracts.

One-line analogy: A streaming architecture is a conveyor belt: boxes move one at a time, the dock signals when it is full, and a final marker says the shipment is complete.

Common misconception: The transport is not the concept. The concept is the contract for ordered incremental delivery, backpressure, and termination.

Coverage

The discipline of designing systems that emit and consume sequences of values over time with explicit flow control. Covers the five primitives (producer, stream, consumer, backpressure, termination), the transport mechanisms (HTTP chunked transfer, SSE, WebSocket, HTTP/2 streams, gRPC streaming, WHATWG Streams, Node streams), the directionality and backpressure-model taxonomies, in-stream error semantics, delivery guarantees, the design contract between producer and consumer, and the failure modes that streaming systems exhibit at scale (slow consumer, abandoned consumer, mid-stream disconnect, head-of-line blocking, partial-result correctness).

Philosophy of the skill

Streaming is the response to a category of problem that batch request/response cannot solve: results that are too big to materialize, too slow to wait for, or too useful at the front to delay until the back arrives. The cost is a more demanding contract between producer and consumer — error semantics get harder, backpressure must be explicit, connections must be managed — but for the problem class it serves, batch is not an inferior streaming; batch is wrong.

The deeper philosophy is that streaming is a contract about time. The five primitives — producer, stream, consumer, backpressure, termination — are the same whether the transport is an SSE event source, a gRPC bidirectional RPC, a WHATWG ReadableStream piping into a TransformStream, an RSC chunked response, or an LLM emitting tokens. A practitioner who learns the contract once can move between transports at the cost of an encoding translation; a practitioner who learns only one transport's API conflates the contract with its encoding and treats every new streaming surface as a new concept.

The discipline of streaming architecture is to know when streaming is the right shape, to design the contract explicitly when it is, and to make backpressure and termination first-class in that contract rather than emergent properties of the transport.

The Five Primitives

Primitive	What it is	What it owns	Failure mode if absent
Producer	The source of values	Emission rate, ordering, framing	Stream cannot exist
Stream	The ordered emission channel	Carrying values in order; no random access	Values arrive out of order or lost in transit
Consumer	The processing sink	Processing rate, ack of received values	Producer has no purpose; values discarded
Backpressure	Flow-control signal upstream	Matching producer rate to consumer rate	Memory exhaustion, dropped values, crash
Termination	Explicit end-of-stream signal	Distinguishing "done" from "quiet"	Consumer waits forever; resource leak

Any streaming system can be analyzed as: who is the producer, what is the stream's framing, who is the consumer, how does backpressure travel upstream, and how is termination signaled. A streaming system that has no answer for any of these is incomplete and will fail under load.

Transport Comparison

Transport	Directionality	Framing	Backpressure	Reconnect	Typical use
HTTP chunked transfer (RFC 9112)	Server→client	Length-prefixed chunks	TCP-level only	None	Large response body of unknown length
Server-Sent Events (HTML LS, EventSource)	Server→client	Newline-delimited event:/data: lines	TCP-level only	Built-in via Last-Event-ID	Live feeds, progress, LLM token streams
WebSocket (RFC 6455)	Bidirectional	Length-prefixed frames	Application-level	None (manual)	Chat, real-time games, collaborative editing
HTTP/2 streams (RFC 9113)	Bidirectional per stream	Per-stream framing with WINDOW_UPDATE flow control	Built-in via WINDOW_UPDATE	None (manual)	gRPC transport, multiplexed APIs
gRPC streaming	Server/client/bidi	Protobuf-framed values	Built-in via HTTP/2 flow control	Manual	Typed RPC, microservice streams
WHATWG ReadableStream	In-process	Reader queues	Built-in via pull model	N/A	Browser-side stream composition
Node.js Readable	In-process	Object or buffer chunks	Built-in via highWaterMark	N/A	Server-side file/network plumbing

Selection rule: pick directionality first (one-way → SSE or HTTP chunked; two-way → WebSocket or gRPC bidi), then framing needs (binary structured → gRPC or WebSocket; text events → SSE; opaque bytes → HTTP chunked), then infrastructure compatibility (HTTP/1.1 proxies often break WebSocket and SSE; HTTP/2 proxies are friendlier).

Server-Sent Events — The Streaming Default For Server→Client

SSE is the lowest-ceremony, highest-compatibility transport for server→client streaming. The HTML Living Standard EventSource API ships in every modern browser.

GET /stream HTTP/1.1
Accept: text/event-stream

HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-store

event: token
data: {"id":1,"text":"Hello"}

event: token
data: {"id":2,"text":" world"}

event: done
data: {}

Properties:

• One-way (server→client). Client can only initiate the connection; once established, only the server sends.
• UTF-8 text events with event:, data:, and id: fields.
• Built-in reconnect via Last-Event-ID header on the resume request.
• Works through HTTP/1.1 proxies; no upgrade handshake.
• No backpressure beyond TCP-level flow control — application-level pacing must be designed in if the producer can outrun the consumer.

For LLM token streaming, progress bars, status feeds, dashboards, and any one-way live update channel, SSE is the right starting point. Move to WebSocket only if bidirectionality is required.

WebSocket — When Bidirectionality Is Required

WebSocket (RFC 6455) is a bidirectional, framed, binary-or-text protocol upgraded from HTTP. Both ends can send at any time.

GET /ws HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: ...
Sec-WebSocket-Version: 13

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: ...

Properties:

• Full-duplex framed message protocol.
• Application-level backpressure required — neither end's send rate is automatically matched to the other end's receive rate.
• No built-in reconnect; the application must handle close codes and resume manually.
• Sensitive to HTTP/1.1 intermediaries that buffer or close idle connections.

For collaborative editing, multiplayer games, chat, and any interaction where both sides need to send unsolicited messages, WebSocket is the right transport. For one-way server-driven updates, it is more machinery than necessary and SSE is usually a better fit.

Backpressure In Detail

The slow-consumer failure mode is the most consequential streaming failure. A producer that emits at 1000 events/sec and a consumer that processes at 100/sec produces 900 events/sec of accumulation. After one minute, the buffer holds 54,000 events; after one hour, 3.24 million. Without backpressure, this exhausts memory.

Backpressure strategy	How it works	Trade-off
Pull (consumer asks)	Producer emits only when consumer calls read()	Implicit; correct by construction; requires pull-capable producer
Credit-based push	Consumer signals "I can accept N more"; producer emits up to N then waits	Explicit; works over network; adds round-trip latency
Buffered push with drop	Producer emits freely; buffer drops oldest/newest on overflow	Bounded memory; lossy; only acceptable when loss is OK
Buffered push with block	Producer emits freely; producer blocks when buffer full	Bounded memory; propagates slowness upstream; only works in-process
Sampling	Consumer samples N values/sec, discarding the rest	Lossy by design; correct for telemetry; wrong for correctness streams

For each new streaming endpoint, the answer to "what happens when the consumer is slower than the producer?" must be one of these strategies — explicitly, not by accident.

Termination And Resume

Termination is a distinct message, not the absence of new values. A consumer that interprets a 10-second silence as "the stream ended" will be wrong on any production network. Explicit termination signals:

Transport	Termination signal
HTTP chunked	Zero-length chunk
SSE	Server closes the connection; client may auto-reconnect
WebSocket	Close frame with status code
gRPC	Status message on the trailer
WHATWG ReadableStream	Reader's read() returns {done: true}

Resume after disconnect is a separate concern. SSE has it built in (Last-Event-ID); WebSocket requires application-level resume tokens; gRPC offers reconnect but not exactly-once across reconnects. A streaming consumer must be designed for "the connection dropped at value 4,732; reconnect and continue at 4,733" if that semantic matters.

In-Stream Errors

Strategy	When to use	Cost
Fail-fast (terminate stream on error)	The error invalidates everything after	Loses partial-results value of streaming
In-band error value	Errors are part of the value type (e.g., a tool-call result with an error payload)	Forces consumers to handle two value shapes
Out-of-band signal (HTTP trailer, WebSocket close code)	The stream is a sequence of successful values; errors are exceptional	Consumer must watch two channels

The choice depends on whether the consumer can usefully proceed past an error. For LLM token streams, an error mid-generation is usually fatal — fail-fast. For a search-results stream, one row's permission error need not stop the others — in-band errors. For a long-lived telemetry stream, errors are out-of-band by convention.

Streaming In Modern Web Frameworks

Framework feature	Underlying mechanism
React Server Components streaming	HTTP chunked transfer; framework-specific chunk format
Next.js Suspense streaming	Streaming SSR over HTTP chunked transfer
Remix loader streaming with defer()	Promise serialization over the response stream
fetch() response body	WHATWG ReadableStream wrapping the network response
Node res.write() / res.end()	Node Readable on the response object
LLM token streaming SDKs	HTTP response streams, often SSE-like event frames; SDK parses frames into iterable values

Each of these is the same five-primitive contract dressed in a framework's API. The framework adds typing, suspense integration, error boundary handling, and ergonomic composition — but the underlying contract is the streaming-architecture primitive.

Verification

After applying this skill, verify:

• Every streaming endpoint has a named answer for: who is the producer, who is the consumer, how is the stream framed, how does backpressure travel upstream, how is termination signaled.
• No long-lived connection assumes silence means "done" — termination is always a distinct signal.
• No streaming consumer materializes the full stream into a collection unless the stream is known-bounded and small.
• Backpressure strategy is explicit, not emergent: pull, credit-based push, drop-on-overflow, or block-on-overflow are named choices.
• Mid-stream errors have a defined encoding (fail-fast, in-band, or out-of-band) — they are not left to "whatever the transport does."
• If reconnect/resume matters for correctness, the protocol carries enough state (last-event-id, resume token) for the consumer to resume without gaps or duplicates.
• SSE is used for one-way; WebSocket for bidirectional; gRPC streaming for typed inter-service streams — choices are justified by directionality and framing, not by familiarity.
• The streaming endpoint's behavior under a deliberately slow consumer has been tested, not assumed.

Do NOT Use When

Instead of this skill	Use	Why
Designing the message-history protocol between a model and a tool runtime	tool-call-flow	tool-call-flow is a specialization of streaming for the model↔runtime cycle; this skill is the underlying primitive
Designing event payload contracts or domain-event topic semantics	event-contract-design	event-contract-design owns named occurrence payloads and topic contracts; streaming owns ordered-emission channels
Designing the JSON shape of a single response payload	api-design	api-design owns request/response surfaces; streaming-architecture owns multi-value-over-time surfaces
Keeping a browser dashboard, notification list, or progress view fresh	real-time-updates	real-time-updates owns user-visible freshness UX, reconnect catch-up, and transport selection for browser views
Moving long-running work into a durable queue with retry policy	background-jobs	background-jobs owns durable execution and retries; streaming-architecture owns incremental delivery while a producer is emitting
Designing the page-level rendering taxonomy	rendering-models	rendering-models owns CSR/SSR/SSG/RSC; this skill owns the streaming primitive those depend on

Key Sources

• IETF. RFC 9112 — HTTP/1.1, § 7.1 Chunked Transfer Coding. The base mechanism for streaming an HTTP body of unknown length.
• IETF. RFC 9113 — HTTP/2, § 5 Streams and Multiplexing. HTTP/2's stream primitive and per-stream flow control (WINDOW_UPDATE frames).
• IETF. RFC 6455 — The WebSocket Protocol. The canonical specification for the bidirectional framed-message protocol.
• WHATWG. HTML Living Standard — Server-Sent Events. The EventSource API and the text/event-stream protocol.
• WHATWG. Streams Living Standard. ReadableStream, WritableStream, TransformStream — the in-browser streaming primitives.
• Node.js. Stream API documentation. Readable, Writable, Duplex, Transform; backpressure via highWaterMark and pipe().
• gRPC Authors. gRPC Concepts — RPC Lifecycle. Server-streaming, client-streaming, and bidirectional-streaming RPC modes.
• Reactive Streams. Reactive Streams Specification. The cross-language specification for asynchronous stream processing with non-blocking backpressure — the basis of Akka Streams, RxJava, Project Reactor.
• React. renderToPipeableStream. The server-rendering API that pipes React HTML to a Node.js stream and supports aborting unfinished rendering.
• Mozilla Developer Network. Using readable streams. Practical reference for browser-side stream consumption.

Skill Graph context

<!-- skill-graph-context:start (generated — do not edit by hand) -->

Classification

• Subject: backend-engineering
• Public: true
• Domain: engineering/realtime
• Scope: Teaching the portable architecture discipline for incremental value delivery over time: producer, stream, consumer, backpressure, framing, termination, reconnect/resume, in-stream error semantics, and transport trade-offs across HTTP chunked transfer, Server-Sent Events, WebSocket, HTTP/2/gRPC, WHATWG Streams, Node streams, and React server rendering streams. Applies when one logical result is delivered as many ordered chunks or messages and the system must reason about slow consumers, abandoned consumers, partial-result correctness, and resource bounds. Excludes browser freshness UX and live-dashboard transport selection (real-time-updates), single request/response payload design (api-design), durable worker execution and retries (background-jobs), model/tool transcript protocol design (tool-call-flow), event payload/domain-event contracts (event-contract-design), and page-level rendering-model taxonomy (rendering-models).

When to use

• model a producer, stream, consumer, backpressure, and termination contract for an SSE progress stream
• choose a pull, credit-based push, drop, block, or sampling backpressure strategy for a stream whose producer outruns its consumer
• compare HTTP chunked transfer, SSE, WebSocket, gRPC streaming, WHATWG Streams, and Node streams by directionality, framing, backpressure, and termination
• Triggers: streaming-architecture, how should this endpoint stream, producer stream consumer backpressure termination, what's the backpressure story, partial result delivery

Not for

• design the JSON shape and status codes for a single request-response API payload
• choose polling, SSE, or WebSocket for browser dashboard freshness UX
• move a slow CSV export into a background job and define retry policy

Related skills

• Verify with: client-server-boundary, performance-budgets, api-design, real-time-updates
• Related: event-contract-design, rendering-models, real-time-updates, api-design, background-jobs, client-server-boundary, performance-budgets, tool-call-flow

Concept

• Mental model: |
• Purpose: |
• Boundary: |
• Analogy: A streaming architecture is to data delivery what a conveyor belt is to a factory's order fulfillment — you do not wait for an entire shipment to be assembled before any piece leaves the warehouse; the belt moves boxes one at a time, the loading dock signals when it's full (backpressure), a final marker indicates the shipment is complete (termination), and the receiving truck can start unloading the first box while the last one is still being assembled. A conveyor with no full-dock signal flings boxes onto the floor; a conveyor with no end-marker keeps the truck driver waiting forever.
• Common misconception: |

Grounding

• Mode: universal
• Truth sources: https://www.rfc-editor.org/rfc/rfc9112#name-chunked-transfer-coding, https://www.rfc-editor.org/rfc/rfc9113#name-streams-and-multiplexing, https://www.rfc-editor.org/rfc/rfc6455, https://html.spec.whatwg.org/multipage/server-sent-events.html, https://streams.spec.whatwg.org/, https://nodejs.org/api/stream.html, https://grpc.io/docs/what-is-grpc/core-concepts/, https://www.reactive-streams.org/, https://react.dev/reference/react-dom/server/renderToPipeableStream, https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Using_readable_streams

Keywords

• streaming, stream, backpressure, SSE, server-sent events, chunked transfer, HTTP/2, WebSocket, WHATWG Streams, ReadableStream

<!-- skill-graph-context:end -->