Case Study: Messaging Systems
Fundamental Style for Messaging Systems
Elements
Filter -> Each filter exposes a very simple interface: it receives messages on the inbound pipe, processes the message, and publishes the results to the outbound pipe.
Pipe -> The pipe connects one filter to the next, sending output messages from one filter to the next.
Pipes
In msessaging systems, the obvious implementation for such a pipe is the Message Channel.
The pipe allows one component to send a message into the pipe so that it can be consumed later by another process that is unknown to the component.
Message Channel provide language, platform and location independence between the filters.
Heuristic
It is useful to design the components so that they communicate with an abstract pipe interface. The implementation of that interface can then be swapped out to use a real channel or an alternative implementation such as an in-memory queue (for purposes such as testing, for example).
Trade-offs
Composability, Flexibility, Extensibility
Structure can be composed into different solutions by connecting filters to different pipes. We can add new filters, omit existing ones or rearrange them into a new sequence -- all without having to change the filters themselves.
Performance
One of the potential downsides of a Pipes and Filters architecture is the larger number of required channels.
Publishing a message to a channel involves a certain amount of overhead because the data has to be translated from the application-internal format into the messaging infrastructure's own format. If we are using a long chain of filters, we are paying for the gain in flexibility with potentially lower performance due to repeated message data conversion.
Testability
Using Pipes and Filters also improves testability, an often overlooked benefit. We can test each individual processing steps by passing a Test Message to the component and comparing the results to the expected outcome.
High-Throughput
We can deploy multiple parallel instances of that process to improve throughput. Competing Consumers is needed to guarantee that each message on the channel is consumed by exactly one of N available processors. This allows us to speed up the most time-intensive process and improve overall throughput.
Heuristic
Parallelizing filters works best if each filter is stateless.
References
Hohpe, G., & Woolf, B. (2003). Enterprise integration patterns: Designing, building, and deploying messaging solutions. http://www.gbv.de/dms/tib-ub-hannover/666294666.pdf
Last updated